This study circle was designed for Rhode Island practitioners who had been working on the development of standards for the state. The circle had several goals:
A bibliography of readings studied appears at the end of this document. A guide to the development of this study circle, including complete copies of readings, discussion questions and activities guidelines will soon be available. This guide will serve as a model of how practitioners might develop study circles to link research to their own community's needs.
In all, nine practitioners (three administrators, three administrator/teachers and three teachers) participated. Regular attendance at all four meetings proved to be a major difficulty. Three participants were forced to drop out; only one was able to attend all four meetings. Timing was a major issue. Meetings were scheduled during the latter months of the academic year; many participants' scheduled were restricted by grant-writing and other program commitments. Regular between-meeting email contact served to keep all participants connected to the ongoing process.
Four sessions (each averaged 90 minutes to two hours) were scheduled. Participants agreed in the end that there hadn't been enough time to cover all the ground we would have liked. The possibility of a fifth meeting was discussed, but summer schedules made arranging one too difficult.
Our two group products, also attached, are letters: one to EFF and another Rhode Island practitioners and other interested parties. The first letter has been posted to the EFF listserv. The second has been distributed to practitioners at a state standards conference, posted to the Literacy resources Rhode Island (LRRI) web site, and posted in LRRI's bimonthly bulletin to Rhode island practitioners.
Following is a summary of the meetings.
In our first meeting, we set goals for the circle and discussed what we felt we needed to know to improve our standards development process. We shared specific concerns about EFF and about the strengths and weaknesses in RI's approach to standards. These needs and concerns emerge in the notes that follow.
A. Study Circle Goals
1. To learn more about EFF, especially addressing:
2. To clarify and develop the RI standards
B. Pressing Questions/Concerns about Standards
Notes from discussion on questions and concerns:
A major concern raised in the discussion was our state's need to address OTL standards. Interestingly, this discussion did not resurface in meetings two, three or four. That may attributed in part to a study circle design that did not anticipate such a discussion. Our agenda took the conversation in other directions.
The point was made that in the face of limited funding, OTL standards would emphasize quality over quantity. One participant argued that while the dialogue (on OTL) is legitimate, attempts to educate all stakeholders regarding this concept haven't succeeded.
It was then pointed out that Massachusetts has been successful in establishing OTL standards, including standards for class size, hours, and practice. Also pointed out was the fact that TESOL is working on program standards, addressing key concerns such as who's teaching.
The point was then made that in Rhode Island, nothing is being done to address OTL. Another participant commented that Massachusetts required an intensive, decade- long advocacy effort to pave the way for its standards. It was added that with the adult ed. environment in Rhode Island being what it is, these issues cannot or will not be addressed any time soon. It was suggested part of OTL standards has to be standards of public support and that the state's current funding levels per student suggested that public support was very low.
C. Strengths in Our State's Standards Development Process (brainstorm)
D. Weaknesses in Our State's Standards Development Process (brainstorm)
Key comments from discussion on strengths/weaknesses:
The discussion shifted from OTL to our group's need to define performance standards (a chunk of our work which was bumped to the second session in the second session focused on defining content/performance standards - more to come on this).
Our first meeting had set group goals and purposes, and discussed the groups own views of what was/wasn't working in our standards development process. Our second meeting's purpose was to examine the idea of content vs. performance standards, and discuss the validity of standards in general. (As with the first meeting, this session dealt with EFF in a relatively small way; EFF becomes pretty much the exclusive focus of the third and fourth meetings.)
The first activity of the meeting aimed at clarifying an issue which had been raised by participants earlier. Participants had agreed that they were unclear on the distinction between the idea of content standards and performance standards. The standards drafted by participants and other RI practitioners had been called content standards by the Department of Education's Adult Ed. specialist, who pointed out that what he actually needed to develop the state plan were performance standards. Some practitioners who'd volunteered to work on developing the standards admitted that they were unclear on the distinction.
In our first activity, definitions of both content and performance standards were drawn from all of the assigned readings and from additional sources and pasted together on a worksheet. Participants were asked to discuss the distinctions between the different definitions and then develop a pair of definitions of their own. Once the definitions were developed and posted, practitioners discussed the two types of standards, their functions, their imprecise nature, and the field's need for them. Following is summary of the major points agreed upon through discussion.
Our Rhode Island Standards are not performance standards.
They are closer to content standards in that they describe what we want learners to know how to do. They have overlaps - similar or same standards written into different levels. To make performance standards, we would need to describe the degree of proficiency or "competence" with which the skills are performed. (the term "mastery" was originally used, but an argument arose suggesting that terms such as "mastery" and "expectations" were K-12 terms, in which the end goal of the process is more uniform) These standards would ideally be as "fluid" a measurement of content benchmarks as possible. Ultimately, though, such measurements would need to be subjective at least to some degree, such as the A-F grading system is subjective to some degree.
Pressure from funders has deeply impacted the standards movement.
Much in our move to develop standards has come with the sense of "develop them yourself, or have something else imposed upon you" attached to it.
Different funders have impacted our move to develop standards in different ways. The RI Foundation, for example, has offered workshops, evaluations, etc. The Dept. of Ed has offered verbal support for the development process. The United Way has applied great pressure to "develop funds or else". Recipients of United Way funding who were once able to use internal testing processes to measure progress now must use standardized tests. Such pressure places a different burden on practitioners and affects what happens in the classroom.v
Tom Sticht's arguments against the idea that literacy development can be "measured" using standardized tests has merit, but is not a rationale for discarding all efforts to measure performance. Rather, it was interpreted as a call for balance in approaching the issue. A crucial point agreed to by all participants was that while some things can not be measured or even implemented with perfect uniformity, if those things were going to be connected to money, which is a uniform system of support, they would need to be measured in a uniform way. Thus, the best we could draw from Sticht was a certain degree of balance - a caution that while we try to be as precise as possible we should know that absolute precision is not possible. It should be openly acknowledged that the measurement construct is imprecise but with a valid purpose and place. The idea of using other, previously developed standards from other states was discussed.
At the close of the meeting, practitioners were asked to read the opening chapters to the EFF content standards book and examine the standards. It was explained that meetings 3&4 would focus on EFF content standards, and on the progression through the performance framework to performance standards.
In the meeting, the focus was principally on initial participant response to learning about EFF through the assigned readings outlined in summaries of meetings 1 and 2. At first, the discussion focused on the EFF Content Standards book (Certain sections were assigned; practitioners were encouraged to read others as possible. Most studied all of the book). Following is a summary of discussions.
Most participants did not respond favorably to the book.
Calling it "Wheels upon Wheels", they criticized the book for spending too much time describing how EFF arrived at its standards and not enough time discussing where practitioners could go with them. In general, participants suggested that they would be better served by a publication that attached the standards to "ways in which teachers are using them" in a more accessible way.There is much to admire in the EFF development process.
It was pointed out by one that the publication made the process of developing EFF clear, and admired the process for its aim at inclusiveness and at developing tools to use to give practice defined ways beyond traditional literacy standards and assessments. Another attractive feature mentioned was EFF's attention to "learning to learn", interpersonal and decision-making skills as well as communications skills.Some felt EFF was impractical.
Most agreed that EFF was in large part an admirable piece of work, Several, however, questioned EFF's practicality. One participant wondered how readily such "soft" skills might be measured and called the standards "too broad." Another agreed that it didn't seem practical to look at "sixteen standards, each with four parts" for each student in a class twice a year. It was also pointed out that many of the standards seemed difficult to apply to basic literacy classes.In summary, the discussions focused largely on what one participant called the failure of researchers to bridge the gap between research and practice. In essence, practitioners argued that the current EFF product was not as useful as it might be because it was difficult to bring into the classroom.
A short exercise followed in which participants were asked to write several brief statements beginning with "EFF standards are..." on sticky paper. They were then asked to place their comments into columns under sticky papers labeled +, O, and - (symbolizing positive, neutral and negative statements). The lists follow:
EFF standards are:
+ - plus
Both cognitive and action based
Designed to target learning-to-learn
Related to student goals and what students want from returning to school Related to roles adults play O similar to SCANS - negative Difficult for practitioners to apply and use in class (Some parts are) extremely difficult to measure A turnoff for many teachers Trees vs. forest (can see EFF but not students) Limited in scope Not specific but generalized Needlessly complicated (The manual is) wordy (The manual is) visually confusing Overly complex Much ado aboutŠ Looking quickly at these ideas, it might be suggested that participants seemed to favor the purpose behind EFF and admired what EFF aims to do. The problem, it seems, lies largely with the inaccessibility of the new Content Standards Publication and with the amount of work that would be required to bring the standards into the classroom (participants were reminded that the work of developing performance measures and assessment processes lies ahead for EFF).Meeting four focused primarily on developing study circle product. There was some initial discussion on points not previously raised concerning EFF. An exercise was presented which aimed to introduce practitioners to the idea of using the EFF performance framework's guiding questions. After some discussion on reasons why participants favored/ did not favor doing this exercise (more below), the conversation moving toward the product development. By the time the group returned to the subject of the exercise, several practitioners needed to leave the meeting. The idea of meeting for a fifth time was discussed and rejected, and it was agreed that participants would do the exercise at home and compare results in the future. Following is a summary of discussions on EFF, the frameworks exercise, and the circle's products.
Further thoughts on EFF
A news issue which was raised during the group's final discussion of EFF concerned the pressures of accountability. On page 100 of the Content Standards book, the point was made that an important EFF feature is that because it aims to be accountable primarily to the learner, it should be easy to be accountable to funders and others. One participant suggested that EFF's message is that practice can make a stronger case to funders if it can demonstrate that it is a sustainable system with a developed system of accountability.
The participant then warned that this perspective on sustaining and professionalizing the field was misguided. She contended that it ultimately amounted to "a lot of hoop-jumping" without an consensus on how the implementation of such a system might be funded. The points were made that practice has long been asked to prove itself worthy of funding, that efforts to do so (such as EFF) often represented a narrowing of the field's focus, and that the field would always struggle under such conditions because efforts to prove themselves accountable were not sufficiently funded. It was suggested that the field might be better served by framing its struggle toward professionalization as a civil rights issue- one in which age discrimination and the failures of the K-12 system were highlighted as arguments to support increased government funding of adult education as a means of securing equal educational opportunity for adult learners.
Another critical point raised during this discussion concerned the idea that the EFF movement would inevitably be narrowed down to a standardized test. To some this development, alluded to on pages 63 and 64 of the standards book, represented a disappointing turning away from EFF's effort to be learner-centered and to measure learning in critical areas not usually addressed effectively by standardized tests. Other participants noted that teachers (GED teachers in particular) were already teaching to tests, and that the importance of covering the range of skills the EFF standards addressed would not be diminished. The challenge, it was said, would be to develop lessons that increased holistic development in the skills areas while preparing students for testing.
Using the EFF Performance Framework to connect EFF and Rhode Island Content standards to performance measures
An exercise was presented in which practitioners were asked to use the performance framework developed by EFF to see if participants could create performance measures to correspond with selected content standards. The intent of this exercise was twofold. The first purpose was to determine whether or not the framework might be used as a tool by Rhode Island practitioners in their effort to develop performance standards for the state. The second was to see, through working with them, if the language and aim of the EFF content standards might inform the Rhode Island process.
At that point, the meeting was nearly finished, so it was agreed that participants would try to do the exercise at home and return the work to the facilitator through the mail. As of the writing of this report, no exercise forms have been returned. When the data is in from practitioners, an addendum to this report will be produced and made available to Rhode Island practitioners. It is our hope that this addendum will offer guidance for the development of performance standards to correspond with the content standards currently being developed.
Products
Following are the two products produced by the members of the study circle. The substance of the letters was drawn from notes on a specific brainstorming discussion aimed at directly addressing each audience and notes from other circle discussions. As there was no time at the fourth meeting to revise drafts, revisions were made via email in the months that followed the final meeting.
These products have been published as noted in the introduction to this report. Publications of these products in other locations, including NCSALL publications, has not presently been arranged.
In the spring and summer of 2000, several practitioners who were instrumental in the development of standards drafts in Rhode Island participated in a study circle sponsored by the National Center for the Study of Adult Learning and Literacy (NCSALL). The circle aimed to provide us with a closer look at standards reform in general and at both the Equipped for the Future (EFF) standards and Rhode Island's own standards development processes in particular. As part of this work, we have developed the following points aimed at encouraging dialogue and action among those involved in Rhode Island's standards development process. It is our hope that practitioners who attend the Department of Education's August 23 workshop and others who will take the state's standards work forward will consider the following points:
The standards development process in Rhode Island is only beginning. The drafted Rhode Island Content Standards for ESOL and Citizenship, developed through the hard work of a dedicated group of practitioners, must now be brought to a larger audience for review and revision. A consensus on what Rhode Island adult learners need to know must be reached. Once the content standards are agreed to, performance standards will need to be developed. Again, a broad outreach will be necessary to ensure that consensus is reached on what constitutes effective performance. Once performance standards are developed, the field will need to develop agreed upon assessment procedures to measure performance. At all phases of development, if they are to be at once clear, direct, measurable and comprehensive, the standards will need to be field tested and validated by practice.
Such a complex process will require buy-in and support from all potential stakeholders, including learners, practitioners, and funders. It is our hope that an outcome of the DOE workshop will be a raised level of awareness of the need for increased participation in the process among practitioners. It is our hope also that our process be informed by Equipped for the Future's effort to build standards which are based upon learner input into what needs to be known and taught. If we are to develop standards that are truly responsive to learner needs, then learners should be encouraged to participate in the process whenever and however possible. Finally, it is our hope that adequate funding be made available to support a standards development process that is grounded in the needs of all stakeholders and in sound educational practice.
To this point in our standards development process, practitioner participation (including the professional development efforts) has been entirely voluntary and largely practitioner-driven. The enabling of a practitioner-driven system is highly commendable, as it ensures that standards will be developed which reflect and respond to what happens in our classrooms. It is our opinion, however, that a lack of adequate funding has slowed the process significantly and threatens to undercut it. Without adequate support, this effort will not be able to extend its reach or sustain its momentum.
Participants in the development process will require ongoing professional development activities such as the study circle and the state standards workshop to work effectively. Practitioners who field test and ultimately use the standards will require ongoing professional development to learn how to incorporate content standards into lesson planning and performance standards into measuring progress. To engage practitioners at the level of participation necessary to the process of professional growth, and to ensure the development of standards which raise the quality of service our field provides, practitioners will need to be supported in a manner which befits professional work.
We encourage practitioners and other stakeholders to move this discussion forward through posting responses to this letter on the LRRI listserv, contacting NCSALL PDRN leader David Hayes atPDRNRI@aol.com, and especially through participation in the ongoing standards development process.
Respectfully,
NCSALL Study Circle Participants