Accountability Requirements and Professional Development in the US Adult Basic and Literacy Education System

Even before the 2001 enactment of the No Child Left Behind legislation, the education bill that holds schools in the US accountable for student achievement, ‘adult education [had] become part and parcel of the new federal trend to encourage the setting of national education goals and standards and holding programs accountable for demonstrating achievements’ (Sticht 1998). Now, almost ten years after enacting the Workforce Investment Act (1998), the legislation that required states to report how adult students were making progress towards educational and work goals, the field is just beginning to take stock of whether accountability has helped or hurt our adult education system. In the US school system (kindergarten to 12th grade for children five to 18), several researchers have investigated the effect of stronger accountability requirements on professional development systems. Berry et al. (2003), in a study of 250 teachers and principals in schools across six Southeastern US states found that results were mixed: Although high-stakes accountability systems help focus professional development efforts on the curricular needs of students, little evidence exists to support the claim that such systems help teachers change their practice to enhance student learning...A tendency exists...to narrow the focus of professional development activities to tested subjects or provide general support that is disconnected from curricular needs. (Association for Supervision and Curriculum Development 2004:3)


Introduction
The purpose of this article is to provide initial information, through the perceptions of practitioners in the adult basic and literacy education (ABLE) field, about how the implementation of the National Reporting System (NRS) (the accountability system in the US for ABLE), has affected professional development systems, activities, and opportunities.One hypothesis would be that the effects are just as mixed in adult basic education programs as they have been in schools for children.

Characteristics of Professional Development in the US Adult Basic and Literacy Education System
The professional development system for adult basic education, literacy and language teachers in the US.like the service delivery system itself, is administered primarily through state-wide and private agencies rather than federal agencies.The system has evolved considerably over the past 20 years.In 1991, with the passing of the National Literacy Act, states received support to establish State Literacy Resource Centers supported by the newly-instituted National Institute for Literacy.They were tasked with providing comprehensive professional development-both pre-service and in-servicefor all adult basic education teachers in their states.States were to spend a minimum of 15% of their adult basic education funding on professional development and research activities, two-thirds of which was required to be spent on teacher training (Belzer et al 2001).However, Congress cut funding for the State Literacy Resource Centers in 1995 (Burt & Keenan 1998).With the reauthorisation and reconfiguring of the National Literacy Act into the Workforce Investment Act in 1998, states were allowed to spend a maximum of 12.5% on 'leadership' for a 'wide variety of support and coordination efforts' (Belzer et al 2001), with no specific minimum for teacher training or professional development.Thus, unless a state's legislature also provided funding to augment the federal allocation, money for professional development decreased, particularly in states with limited adult basic education dollars (Tolbert 2001).
The types of professional development offered to adult basic education and literacy teachers have continued to be predominantly in-service workshops and occasional conferences, rather than longer-term opportunities for development.The most recent national evaluation (RMC Research Corporation 1996), which surveyed all states' use of federal monies for adult education professional development, found that single-session workshops accounted for 38% of all professional development activities, followed by institutes or courses (24%), and statewide or regional conferences (11%).Although more states now offer some type of new teacher orientation, either online, face-to-face, or text-based (National Adult Education Professional Development Consortium 2007), access to professional developmentparticularly in-service training-is additionally limited by adult literacy teachers' part-time status, lack of paid professional development release time, and difficulties attending professional development activities: Currently, the conditions of the ABE occupation are such that those in the field will never be able to participate systematically in the very activities they see as necessary to doing their jobs well.Educators claim the desire for professional development is present; readily accessible opportunities to fulfill that desire are most notably not.(Wilson & Corbett 2001:26) The circumstances in which adult basic and literacy education practitioners work, is an important background for understanding the effects of accountability on the professional development they are offered.

A short history of accountability: the National Reporting System
Adult basic education in the US is primarily funded by the federal government and by state legislatures, unlike the school system for children (which is primarily funded and managed by local school districts), and the higher education system (a combination of state-funded public institutions and privately funded colleges and universities).As the budget deficit grows, the federal government is increasingly interested in the return on investment for the almost $500 million allocated to ABLE annually.Over the years, demands for information about the return on this investment have varied.The National Literacy Act of 1991 asked states to voluntarily report on whether ABLE programs had high-quality curricula, instruction and professional development; however, the Workforce Investment Act of 1998 required a mandatory, quantitative reporting designed to inform Congress about how many individual adult students meet specific societal and employment goals (American Institutes for Research 2005).
The 1998 Workforce Investment Act, Title II, the Adult Education and Family Literacy Act, came down firmly on the side of outcomes and impacts that reflect, like the title, an emphasis on individual readiness as a worker and a parent, mandating that 'adult education services are authorised in order to assist adults to become employed, be involved in their children's education, and gain a secondary credential (GED or high school diploma)' (Bingman 2000).Under this law, states were required to set performance targets for three core adult basic education outcomes, indicated by (1) improvements in literacy, language, mathematics and other basic skills; (2) placement, retention or career advancement in or completion of postsecondary education or employment; and (3) receipt of a secondary school diploma or equivalent (Workforce Investment Act 1998).States have been required to show progress each year towards these performance targets, and there are funding sanctions for not doing so.In order to establish a common system for states to report to the federal government, the Division of Adult Education and Literacy of the U.S. Department of Education funded the development of the NRS, which 'established measures for the core indicators required by the Workforce Investment Act' (Bingman 2000:3).The NRS required each state to develop a system for providing data on all adult students who enter programs that receive federal funding.The state in turn requires individual ABLE programs to follow each student with standardised tests to determine achievement and progress towards goals.Program staff must at least attempt to test every student before instruction (pre-test) and then again after a certain number of hours of instruction (post-test).Programs must record each student's goal (whether related to improving skills, education level, or employment) as she or he enrols in the program, and report to the state against progress towards that goal each year.
With stronger monitoring by the federal government of states' progress towards adult education achievement and goals, and with stronger monitoring by the states of local programs' performance, a shift towards a much more intensive, standardised system of documenting each adult student's progress began in 1999.Teachers and program staff were now required to provide quantitative information about each student's skills when they arrived at the program and periodically thereafter.In a six-state review of the impacts of federal policies (including the Workforce Investment Act), on ABLE programs and practitioners, Belzer (2003) found that: Programs also placed greater emphasis on testing and documentation, as well as on goal setting.Rather than replacing classroom processes, these changes often added an extra burden to already over-taxed instructors (iv).
Belzer noted that in four out of the six states the state agency used the existing professional development system or other state-wide supports to help programs respond to the NRS requirements, usually with limited funding.Although Belzer concluded that using the pre-existing professional development system helped programs through the change brought about by greater accountability demands, program administrators in Belzer's study reported 'devoting more work time to documentation and less to educational leadership and professional development for their staff' (39).If this is so, then it appears that the NRS must certainly have had an impact on the professional development offered to practitioners, at least during the time period when the NRS was being introduced.However, we still have little information about the longer-term affects of accountability requirements on professional development activities and systems.

Methodology
Due to the lack of research in this area, this study required an exploratory design to generate initial perceptions of those working in the field.These opinions and perceptions can then be translated into hypotheses for further research.The key research questions driving this study included: -Have the requirements for practitioners to participate in professional development related to accountability and assessment changed since the NRS accountability system was instituted, and if so, how? -Has the total proportion of professional development in programs or states related to accountability and assessment increased since NRS, and by how much?-To what extent and in what ways has the NRS affected professional development and the type of training ABLE teachers receive?Data collection for this exploratory study began by identifying and contacting approximately 20 experts in the field of professional development in adult basic and literacy education.In order to find respondents who had worked in the field long enough to know what it was like before and after the NRS came into existence (approximately 1999-2001, depending on the state), only experts who had worked in the field since before 2000 were contacted.Eventually, in-depth phone interviews with ten of these experts were conducted in the Summer and Autumn of 2007 using a semi-structured protocol.
Interviewees were asked about years of experience and depth of knowledge regarding adult basic and literacy education professional development, and the interviews continued with questions about changes in professional development related to accountability and assessment over time in their state or area and about their perceptions of the extent and type of effects the NRS has had on professional development.Each interview lasted approximately 40-90 minutes, transcribed verbatim by the researcher.The ten experts interviewed included three state Department of Education project monitors (one each from a western, a southeastern and a northeastern US state); a national expert in English for Speakers of Other Languages (ESOL) instruction and professional development; a national expert in adult basic education and literacy; three directors of statewide professional development systems (one from a western state, and two from northeastern states); and two state-level providers of professional development (one each from a southeastern and a northeastern state).These professional development experts had worked in the field from eight to over 30 years.When asked on a scale of one to five how knowledgeable they felt they were about professional development in their state or nationally, all replied that they were very or extremely knowledgeable (nine out of the ten responded with a five).
This study also employed a ten-question, on-line convenience survey of subscribers to the Association of Adult Literacy Professional Developers listserv in December 2007-January 2008.The survey questions mirrored the interview questions, asking whether, to what extent, and how the NRS had affected professional development in the respondents' program, state or area.Sixty-nine practitioners responded to the survey.There was a wide range of experience among the respondents, with roughly one third of the sample each having less than five, between five and ten, and more than ten years professional development experience.Of the 69, almost half normally worked at the program level, while 44% worked at the state level, 6% worked at the national level, and the rest worked at 'other' (unspecified) levels.The comments, always eloquent, from both survey respondents and interviewees indicate that it is difficult for many practitioners in the ABLE field to separate the effects of the NRS system in general from its effects on professional development.In reporting findings, I have purposefully selected only those comments that relate specifically to professional development systems, activities and opportunities.

Findings Changes in Professional Development Requirements
The requirement for practitioners to participate in accountabilityrelated professional development appears to have risen considerably since the NRS began.Thirty-nine percent of respondents indicated that teachers in their state were required to learn about student assessment or accountability requirements before the NRS took effect, compared to 74% reporting that teachers now are required to attend such trainings.In large part, this seems to be due to the increased expectation in many states that teachers take on the burden of administering tests to each adult student: The primary staff (and sometimes the only staff) in ABLE programs are the teachers, since many smaller programs do not have specific staff to conduct intake assessments.It appears that more teachers nowadays are required to attend professional development on test administration and accountability policies.In some states, this represents a net gain in the amount of professional development teachers can attend.Yet respondents see this as a mixed benefit: The training and development of teachers are now taken into account as a real agent of change in the process of educating adults.By having a focus to move toward, program administrators have begun to understand how important the development and professional support to teachers really is.(Survey respondent) A positive effect is when programs learn how to use the data to improve services or target classroom instruction, so long as it doesn't become professional development around teaching to the test.However, in professional development, an important negative effect is that the PD dollars are increasingly being spent towards learning how to be good 'bean-counters'...While data is ultimately a good thing, one could argue that the resources to support this training should not become a burden on the PD system and instead should be funded separately.(Surveyrespondent) Even though professional development around assessment is tedious and 'yucky', it has brought teachers together in a way that rarely happens.(Survey respondent)

Changes in professional development offerings
With increased requirements for practitioners to attend professional development (PD) focused on accountability and assessment, the proportion of such professional development also increased since the NRS.Only 7.2 % of respondents reported that the proportion of accountability-focused professional development totalled more than a third of all PD offered in their program, state or area.By contrast, 36% of respondents reported that accountability-focused PD equalled more than a third of all PD in the first few years after the NRS started (during 'the transition' to accountability).Now, approximately seven years after NRS started, 50% of the survey respondents indicate that such professional development accounts for more than a third of the total professional development offered, and one-quarter of the respondents indicate that it represents more than half of the PD offered in their program.As one survey respondent explained, 'Other trainings have been tabled in order to train teachers in test administration' which has led to an imbalance of PD offerings.
The NRS has skewed professional development to a very limited range of topics and performances, which do not reflect the desired work and performance of learners in programs.
(Survey respondent) Some recognise the importance of finding a balance in the focus of the professional development provided: We need to make sure that our training is focused on providing support and materials to our teachers and tutors AND complying with NRS requirements . ..; we cannot privilege NRS requirements above providing excellent staff development in all aspects of our work.(Survey respondent) The effect of the National Reporting System accountability system on professional development The majority of respondents to the survey (78%) and all of the professional development experts interviewed indicated that the NRS had had either a moderate or strong effect on the PD system in their program, state or area.Only 4% of the survey respondents reported that it had no effect on their PD system, but few believe that the NRS has mainly had a negative effect.Instead, the majority of experts and 57% of the survey respondents believe that the NRS has had a 'mixed' (both positive and negative) effect on professional development.In addition almost a third of the survey respondents believed the NRS has had a largely positive effect on professional development in their program, state or area.However, what counts as positive or negative varies; the same focus-such as training on administering tests-was seen as positive by one respondent and negative by another.
Negative Effects: 'Negative' effects on professional development included changes in the content/topic of professional development, with a greater focus on 'teaching to the test': More trainings are focused on using data, more for teachers on using software, less focused on instructional content.NRS has caused the field to be more accountable across the board--improve quality, sense of improvement, and growth.It's a better looking system.But I don't think the structure to really capitalise on the opportunities is in place well enough.It's still hit or miss . . .(National ABLE expert) Another negative effect of accountability on professional development was the perception of greater focus on showing gains and meeting goals: Training around [student] goal setting is really critical, but I'm seeing that when learners come in to the program and are asked to set goals, staff has had training not in effectively setting goals, but training in how do you document the setting of goals for students so that they start in a low place, a small goal, and achieve that goal easily…learners will say, 'I want to get my GED', and there's a shock.'Oh no, we can't say that, so let's break that down into the smallest of goals that can be achieved in the year, like 'read to a child three times a week.'Not meaningful and not significant.Training is geared to increasing performance on NRS on paper.However, having said that, the training I've sat in for instructors is outstanding, really excellent.
(National ABLE expert) Capacity-building programs are not happening with the [federally-funded] system, but in programs not part of NRS . . .[they are looking at] career pathways, health literacy, financial literacy . . . the more creative things that I think encourage retention are the things that we're not seeing so much in NRS, not responding to the needs of the learners.Outside NRS, programs are still looking for innovation to serve those who haven't been served.(National ABLE expert) Money that could have been spent on action research and practitioner inquiry is now used to support training on things like the CASAS and NRS, neither of which support teacher change in the actual classroom.It's beginning to feel like a fool's errand… (Survey respondent) Positive Effects: Others do see the effects of the accountability requirements on professional development as positive, including a greater focus on reporting student gains and helping students reach broader goals: The major impact has been to make the administration, people who are operating these programs, focus more.Because of the requirements the state gives us, if we meet our NRS goals, then we get incentives.

(State Department of Education monitor)
There is an increasing focus on English language learners getting into transitions to credit-bearing education or a vocational ladder.ESOL has to 'count' for something.This is not just a result of the NRS but the NRS contributes to it, because ESOL programs need to report against hard goals of getting a job, getting into post-secondary, etc. . ..This is supported by recent research, particularly the community college studies, showing a 'tipping point' for ESL instruction: when English language students get instruction, a credential and college credit, this opens the doors to higher wages. . . .it's not just about learning English in order to survive, but learning English in order to contribute to the workforce and secure a living wage job, and teachers and programs need professional development to help them adapt.(National ESOL expert) At first, the required training was seen by many as unnecessary and too much.Now, people have come to realise that understanding the accountability system has its benefits.
Programs have been able to target gaps in service, and some have used this information to plan professional development for instructors and other staff.(Survey respondent) Respondents also talked about the improved quality of professional development because it has to 'count' more: [There] has been a gradual moving towards PD with better presenters and follow-up…[There is] more understanding that you don't just pick and choose the workshops you want to go to, based on a whim.Now, PD has to count for more…[There is] an understanding of how professional development needs to feed into better student achievement, not just teachers being more well-rounded.(National ESOL expert) The quality of the training has increased because there's additional competition between the states.Reports saying, 'look at these six states', and then look at how our state is doing better.Not appropriate competition, but it is showing legislators that adult education is improving dramatically and asking for more resources.(National ABLE expert) A third positive effect was a stronger focus on professional development that is 'focused' and addresses 'the basics': [NRS] made us think more about the types of PD opportunities we offer.We have foundations in teaching math, foundations in family literacy, in teaching reading and writing, which all gear towards improving the quality of instruction that is given to students.Part of that is in order to maintain funding, because we have to show that we are meeting and moving students along the NRS continuum.For professional development sake, it's had a positive impact . . .In the past, we would let instructors choose [PD activities] based on whatever they wanted to take...Now, we are more guided.Everything we try to offer will impact assessment, analysing that assessment, quality of instruction, tips and teaching strategies to meet these things.

Conclusion
The effect of accountability requirements on professional development appear to be as mixed in ABLE programs as it is in schools for children.On the one hand, some practitioners perceive positive effects from the way that accountability has sharpened the focus of professional development and improved its quality.If so, this is a heartening but expensive and timeintensive process that may have positive long-term effects on the services that adult students receive.However, the negative potential is dismaying.According to the reports of respondents, in more than a few cases, professional development has narrowed to a focus on teaching to the test.
Even more distressing is the report that, in some states, ABLE teachers are offered professional development in how to gear their instruction towards helping adult students perform better on the Test of Adult Basic Education (TABE).This is a standardised test measuring grade-level equivalent reading comprehension that programs primarily used to place adult students in the appropriate-level class but which (for lack of a better option) programs now use as a post-test to gauge literacy achievement.This surely defeats the purpose of helping adult students to reach their goals, since no one could ever imagine that an adult student would choose to enrol in adult basic education with the specific goal of doing better on the TABE.
What is also worrisome is the increase in the percentage of accountability-focused professional development activities.Given the small amount of time ABLE teachers received for professional development, and given the turnover of primarily part-time teachers in this field, one wonders how much instruction-related professional development new teachers receive, compared to the training they receive on how to administer assessments.
Further and longer-term research studies are needed to confirm and expand on the hypotheses emerging from this exploratory study.Without funding for a random selection of interviewees and respondents, they should not be generalised to the field as a whole.However, this exploratory study raises some questions for future research.For example, what is the actual change in quantity of accountability-focused vs. instruction-focused professional development that practitioners actually receive, and is there a differential effect for part-time and full-time practitioners when there are increased requirements for accountability-focused professional development?Is there a transition over time, after an accountability system becomes institutionalised, in the type of effect it has on professional development systems, such that initial negative effects give way to more positive ones that focus more on documented strategies to improve instruction, based on the data about student achievement?What are the characteristics of professional development systems in states where professional development is perceived to have improved since the inception of the NRS accountability system, and can those be duplicated in other states?
However, the main implication of this exploratory study is that states must work harder to ensure a balance between good accountability and good professional development.This will probably require more funding from the federal and state governments to increase the overall amount of professional development, and it will require states and programs to pay more attention to the specific needs of both teachers and adult students, beyond scoring well on tests.
New teachers are required to take training on how to give the TABE[Test of Adult Basic Education].Teachers are required to take the TABE themselves, know the skills that are needed for students.Training on how to administer it, analyze and make it useful for them instructionally, when it's appropriate to readminister, and how to follow all the guidelines of the test administration manual.Throughout the state, it's mostly the teachers who administer the tests.(State Department of Education monitor) Those providing PD in ESOL have heard from states.While they seem interested in learning about student needs, they are much more 'up front' about needing students to get higher scores on [standardised tests].That is the end goal for some states.Some teachers in some states are attending training on how to help their students get better scores on the TABE or CASAS.Teachers perceive this is what PD is aimed at--getting higher scores on a test.
(National ESOL expert)My biggest concern is that you're squeezing out time for training on curriculum and instruction.We need to 'stop measuring the pig and start feeding it'.. .But if you think about part-time teachers, there were real limits to how much time they could put in.So if they did [training on standardised test] and [the annual] conference, that was about all they had . . .There's also training on how to work with students on setting goals, collecting information, and entering information on outcomes.That occupies a lot of energy… (Director of state professional development system) What it's done has helped all organisations, all states to increase at the local level the process for accountability . ..[that's an] excellent thing, but doesn't encourage going beyond the structure.Dumbs it down at the same time that it's building it up.NRS has brought everyone up to a certain level, but not a lot of creativity beyond that.(NationalABLE expert) Finally, respondents indicated that the use of data as an instigator and source of information for solving problems has been a positive effect of accountability, and professional development oriented toward that goal:Having something that is an instructional strategy that is student focused causes [practitioners] to question . . .they examine preconceived notions of what they're doing, and then they have the data skills to find the answers to those questions.