Our Mission

The National Data Evaluation Center supports early literacy achievement using science-based methods in research and evaluation. We use technology to provide educators with the data they need to evaluate their Reading Recovery^® implementations and make timely, well-informed decisions.

Reading Recovery and Scientifically Based Research

The Guidance for the U.S. Department of Education Reading First Program requires the following:

When reviewing research findings to determine whether they meet the criteria for scientifically based reading research, State educational agencies, local educational agencies, and schools should consider the extent to which the research meets each of the criteria. Questions for consideration about each criterion include:

Use of rigorous, systematic and empirical methods - Does the work have a solid theoretical or research foundation? Was it carefully designed to avoid biased findings and unwarranted claims of effectiveness? Does the research clearly delineate how it was conducted, by whom it was conducted, and on whom it was conducted?

The structure and design of Reading Recovery are consistent with a large body of substantial research on how children learn to read and write. Marie Clay's basic research on reading and writing behaviors of young children began in the 1960s. She began to research reading difficulties through a series of studies in the 1970s that led to the development of Reading Recovery (including field trials, follow-up studies, replication studies, monitoring studies, and subgroup studies).

Change in Reading Recovery is a deliberate, careful, ongoing process based on continuous research. Refinements in practice are based on current research in language and literacy learning and teaching, as well as research and evaluation directly related to the program.

In addition, numerous studies have examined the effectiveness of Reading Recovery for children with reading difficulties. A notable example is the rigorously controlled experimental study of randomly assigned groups in 40 elementary schools (Pinnell, Lyons, DeFord, Bryk, & Seltzer, 1994). Reading Recovery subjects performed significantly better than other treatment and comparison groups on all measures. Essential differences were related to one-to-one instruction, the lesson framework, and teacher training.

Research on Reading Recovery and Descubriendo la Lectura (Reading Recovery in Spanish) uses systematic, empirical methods to collect data annually on all children receiving the service. Data collection, analysis, and assessment are standardized nationally. Data are also collected on a random sample of grade level peers to provide a comparison group. Methods and materials are detailed in Site reports as well as the annual national report.

Adequacy of the data analyses to test the stated hypotheses and justify the general conclusions drawn - Was the research designed to minimize alternative explanations for observed effects? Are the observed effects consistent with the overall conclusions and claims of effectiveness? Does the research present convincing documentation that the observed results were the result of the intervention? Does the research make clear what populations were studied (i.e., does it describe the participants' ages, as well as their demographic, cognitive, academic and behavioral characteristics) and does it describe to whom the findings can be generalized? Does the study provide a full description of the outcome measures?

The goal of Reading Recovery and Descubriendo la Lectura is to dramatically reduce the number of learners who have extreme difficulty with literacy learning and the cost of these learners to educational systems. This goal calls for Reading Recovery children to make faster than average progress so they can work within an average group setting in the regular classroom.
Hypotheses that are tested annually include:

• Reading Recovery children will increase their skills in the following areas necessary for reading: letter identification, reading vocabulary, concepts about print, writing vocabulary, hearing and recording sounds (phonemic awareness and letter-sound relationships), and text reading.

• Children who successfully complete Reading Recovery will perform on literacy measures within an average band of their classmates who did not need the intervention. This performance gives them greater access to classroom learning.

• Children who successfully complete Reading Recovery will continue to make gains in text reading and writing vocabulary after leaving the intervention and continue to perform competitively with peers who were not initially at risk.

Annual data analyses test these hypotheses and reveal that a large majority of children with Complete Interventions do make accelerated progress and work within the average of their classrooms; they also continue to make progress after leaving the intervention. Because these children were the lowest literacy achievers in the first grade in their school, we can verify the reduction of the number of children with extreme literacy difficulties.

Rigorous data analysis procedures are summarized below:

• The National Data Evaluation Center (NDEC) collects data on all children served and provides annual reports of quantitative data. Nine research questions guide data analysis, all of which test intervention effectiveness and efficiency. Conclusions about intervention outcomes are based on the answers to these research questions, which yield information about the children served, the outcome status of each child, progress of all children on multiple literacy measures, comparison of Reading Recovery children with a random sample of their peers, classroom teacher perspectives of children's literacy performance, information about retention and special education, analysis of time in intervention, etc. An annual report is published.

• Data from NDEC are provided for every site, district, and school involved in Reading Recovery and Descubriendo la Lectura.

• Longitudinal studies answer questions about subsequent performance. Findings from two states provide examples:
  Texas: Two longitudinal studies (Askew et al., in press) followed children through Grade 4 and found that 80–85% of the children (originally the lowest-achieving first graders) who were successful in Reading Recovery in Grade 1 passed the fourth grade Texas Assessment of Academic Skills (TAAS) reading test. Preliminary findings in a third study show that more than 80% of Reading Recovery children for whom TAAS data were available (whether successfully discontinued or not) passed TAAS reading tests in Grades 4 and 5.
  Ohio: A follow-up study (Hovest & Day, 1997) found that of 2,714 former Reading Recovery students, 71% passed the reading portion and 75% passed the writing portion of the Ohio Fourth Grade Proficiency Test.

Again, because these children were the lowest literacy achievers in their first grade classrooms, these studies demonstrate that Reading Recovery plays a role in dramatically reducing the number of children with extreme literacy difficulties.

Reliance on measurements or observational methods that provided valid data across evaluators and observers and across multiple measurements and observations - Are the data based on a single-investigator, single-classroom study, or were similar data collected by multiple investigators in numerous locations? What procedures were in place to minimize researcher biases? Do observed results "hold up" over time? Are the study interventions described in sufficient detail to allow for replicability? Does the research explain how instructional fidelity was ensured and assessed?

An Observation Survey of Early Literacy Achievement (Clay, 2002) is used as a pre-test for selection purposes, a measure for making exit decisions, and an end-of-year test to examine change over time. Stanines are developed for three points in time, and each task has established reliability and validity measures. The Observation Survey is widely used by researchers and by practitioners in schools.
The Observation Survey is an empirical observation instrument that yields five scores on measures of essential factors leading to effective literacy. Extensive training is provided in the systematic, objective procedures specified for the administration, scoring, and analysis of this instrument.

Observational data are also collected daily using systematic and controlled procedures. Teachers use this information to make teaching decisions for each child.
Teachers are specially trained for a full academic year (six semester hours of graduate credit) in the use of observation and in research-based procedures for working with individual children. Teaching is based on systematically collected data.

Strong professional development continues after the initial training year. Research supports the importance of ongoing development.

Descubriendo la Lectura (Reading Recovery in Spanish) meets the same criteria for systematic and controlled empirical methods that draw on observation. There are very few instruments available to assess Spanish-speaking learners; the Observation Survey is available in Spanish.

Acceptance by a peer-reviewed journal or approved by a panel of independent experts through a comparably rigorous, objective and scientific review – Has the research been carefully reviewed by unbiased individuals who were not part of the research study? Have the findings been subjected to external scrutiny and verification?

The following peer-reviewed research articles or research reviews offer support for various aspects of Reading Recovery. Please consult the Reading Recovery Review (Askew, Fountas, Lyons, Pinnell, & Schmitt, 1998) available from the Reading Recovery Council of North America for a more complete list.

Clay, M. M. (1967). "The reading behaviour of five-year-old children: A research report." New Zealand Journal of Educational Studies, 2, 11–31.

Clay, M. M. (1968). "A syntactic analysis of reading errors." Journal of Verbal Learning and Verbal Behaviour, 7, 434–438.

Clay, M. M. (1969). "Reading errors and self-correction behavior." British Journal of Educational Psychology, 39, 47–56.

Clay, M. M. (1970). "An increasing effect of disorientation on the discrimination of print: A developmental study." Journal of Experimental Child Psychology, 9, 297–306.

Clay, M. M., & Imlach, R. H. (1971). "Juncture, pitch, and stress as reading behaviour variables." Journal of Verbal Learning and Verbal Behaviour, 10, 133–139.

Additionally, the following study used a randomized trial methodology, the most rigorous methodology recommended by the U.S. Department of Education. This study demonstrated the effectiveness of the Reading Recovery intervention.

Pinnell, G. S., Lyons C. A., Deford, D. E., Bryk, A., & Seltzer, M. (1994). "Comparing instructional models for the literacy education of high-risk first graders." Reading Research Quarterly, 29, 8–39.

Additional References Cited:

Askew, B. J., E. Kaye, et al. (2002). "Making a case for prevention in education." Literacy Teaching and Learning: An International Journal of Early Reading and Writing, 6(2): 43-73.

Clay, M. M. (2002). An Observation Survey of Early Literacy Achievement, Second Edition. Portsmouth, NH, Heinemann.

Hovest, C., & Day, J. (1997, February). "Sustaining gains: Ohio's Reading Recovery students in fourth grade." Paper presented at the 12th Annual Reading Recovery Conference and National Institute, Columbus, OH.