Recognition & Response - Research Synthesis on RTI
   
 
 
    Home  |  About this Site  |  Contact Us  |  Press  |  RTI Action Network   
   
  Home  
   
 
Research Synthesis on RTI
 
 
 
Print E-mail

From Recognition and Response: An Early Intervening System for Young Children At-Risk for Learning Disabilities; Research Synthesis and Recommendations, pages 19-28)

Method

Search strategy

Search terms. The search of the literature was conducted to identify both conceptual and empirical articles related to RTI; however, only the empirical studies investigating the efficacy of RTI were included in the research synthesis. The conceptual articles were used to provide background information and describe the theoretical and conceptual underpinnings of RTI. Articles were identified by conducting searches through several online databases. The following keywords were used during these searches: early identification, learning disabilities/difficulties, responsiveness/response-to-instruction/intervention/treatment, and
early intervention.

Sources. Searches were conducted in the following databases: ERIC, EducationFullText, and PsychLit. Journal issues focusing on RTI were also searched for relevant articles (see Learning Disabilities Research and Practice, 2003, Vol. 18, No. 3; Learning Disabilities Quarterly, 2005, Vol. 25, No. 1; and Journal of Learning Disabilities, 2005, Vol. 28, No. 6). In conjunction with a national validation of the research synthesis, reviewers were asked to identify any additional articles for inclusion in the paper. Two additional studies were identified for inclusion through this process.

Selection criteria. The three criteria used to determine the inclusion of studies were:

1. Age of subjects. The study included children ages 4-8 years. The upper limit for inclusion in the synthesis was third grade.

2. Nature of disability. The study included children characterized as nonresponders, at risk for learning disabilities/difficulties, or as having math and/or reading disabilities.

3. Intervention. The study included at least one of the three components of RTI: (1) the use of multiple tiers of intervention, with increasingly intense interventions that guide its implementation; (2) a problem-solv-ing or standard treatment protocol approach used to identify and provide supplemental instruction to children who were not responsive to general classroom instruction, or to identify the characteristics of these children; and (3) an integrated data collection/assessment system to guide decision-making for identification purposes and/or instructional supports.

Articles Selected for review. A total of 14 empirical studies met the three criteria for inclusion in the review and served as the current research base regarding the efficacy of RTI. A detailed summary of the empirical studies can be found in Appendix A.

Development of Abstracts. Abstracts for the 14 empirical studies were developed to provide (1) a summary of the existing efficacy research on RTI and (2) a framework from which the empirical studies could be analyzed
for overall quality. The structure for the development of the abstracts was guided by recommendations
outlined by the National Research Council’s (2005) Advancing Scientific Research in Education (see Table 1 for recommended components of abstracts of empirical studies). Table 2 describes the process of developing the abstracts of studies included in the review.

Image

Analysis procedures. The first step involved the creation of summary tables for use in analyzing information from the empirical studies (see Appendix A). The next step involved the development of a quality indicators rating scale to evaluate the quality of experimental and quasi-experimental research designs (see Appendix C for rating scale). The third step involved the evaluation of the quality of the research design in each study.

Research design. Appendix A provides a summary of the research designs across studies. Most studies employedeither experimental or quasi-experimental designs; however, three studies were classified as causal-comparative, one employed a regression discontinuity research design, and one used a combined experimental/causal-comparative research design. Table 3 provides a list of the study designs.

Image

ACausal comparative research designs refer to designs in which participants are assigned to groups according to predetermined criteria following the implementation of an intervention. B Regression discontinuity research designs refer to designs in which participants are assigned to groups according to predetermined criteria (e.g., using pretest measures) prior to the implementation of an intervention.

Development of quality indicators rating scale. A rating scale was developed to measure the quality of the research conducted on RTI. Criteria outlined in Exceptional Children’s special issue (Volume 71, Number 3, Winter 2005) on evidence-based practice in special education served as the basis for the development of the rating scale (see Gersten, et al., 2005). In this article, the authors describe both essential and desirable indicators; all of which were included in the quality rating scale developed for this synthesis.

Specifically, a rating scale was developed for studies that employed experimental/quasi-experimental research designs, which represented the primary research design identified through the formulation of the abstracts. Although other research designs were identified for a small number of studies during the development of the abstracts (see Table 3), the research team determined that the rating scale developed for experimental/quasi-experimental designs could be used to evaluate the quality of the research in these cases.

The criteria outlined in the Exceptional Children issue were adapted by applying a 4-point Likert scale, which enabled the research team to rate specific items as adequate (4), partial (3), inadequate (2), unknown (1), or not applicable (N/A). Through the use of this scale, the research team could assess the extent to which each quality indicator had been addressed by a particular research study rather than simply evaluating whether an indicator was present or not.

Evaluation of the empirical studies. To evaluate the research base regarding RTI, two members of the research team used the rating scale to assess each empirical article. Both researchers evaluated all of the studies (n = 14). The scores assigned by the researchers for each study were then compared for reliability purposes. Consensus was achieved for all studies, even in cases where reliability was 80% or greater. Overall interrater reliability ranged from 62% to 100%, with a mean of 87%.

Quality ratings for empirical studies. The overall score for the 14 empirical studies (M = 3.15, SD = 0.66) demonstrated that the research articles were of good quality. Items that were rated as adequate were most often related to providing information about whether participants demonstrated the difficulties/disabilities presented (essential) (M = 3.93, SD = 0.28), Recognition & Response | Coleman, Buysse, & Neitzel, 2006
measurement of outcomes (essential) (M = 3.93, SD = 0.28), data analysis techniques used (essential)
(M = 4.00, SD = 0.00), and presentation of results (desirable) (M = 4.00, SD = 0.00).

However, the studies were particularly weak in providing comparable attrition rates across samples (desirable) (M = 2.30, SD = 1.37), describing audio-or videotape techniques to capture the implementation of the interventions (desirable) (M = 2.50, SD = 1.57), collection of maintenance data beyond an immediate posttest (desirable) (M = 1.86, SD = 1.41), and providing criterion and construct validity of the measures used (desirable) (M = 1.67, SD = 1.07). The mean scores across studies for the individual items on the quality indicators rating scale can be found in the Appendix D.

Search Results

Characteristics of study participants

Table 4 summarizes the background information regarding the participants, including the sample size, ages of the participants, race/ethnicity, and criteria for inclusion. Sample sizes ranged from 36 to 273 across studies, with a mean of 116 children. Across the studies, 1,627 children in kindergarten through 3rd grade participated. The majority of the children were enrolled in first, second, and third grades.

Image

Three of the studies included children who were performing below expectation in one or more areas (Burns & Senesac, 2005; Torgesen & Davis, 1996; Torgesen, et al., 1999). Nine Recognition & Response | Coleman, Buysse, & Neitzel, 2006 additional studies focused on children at risk for academic failure in one or more areas (e.g., math, reading) (Case, Speece, & Molloy, 2003; Coyne, et al., 2004; Fuchs, Fuchs, & Prentice, 2004; Fuchs, et al., 2005; O’Connor, 2000; O’Connor, Harty, & Fulmer, 2005; O’Connor, Harty, Fulmer, & Bell, 2005; Speece & Case, 2001; Vaughn, Linan-Thompson, & Hickman, 2003). Students in the remaining studies were characterized as being poor readers (Vellutino, et al., 1996), or being nonresponsive to otherwise effective instruction (McMaster, et al., 2005).

Description of multi-tier approach. Appendix E contains a description of the implementation of the multi-tier approach to identification and intervention used in each of the studies. Because a number of studies did not explicitly state which tiers were implemented, it was necessary for the research team to make inferences about the specific tiers that were used in these cases. Each of the studies in this research synthesis was described based upon the three-tier approach discussed earlier in this document.

Two of the studies focused primarily on the use of Tier 1, with an emphasis on issues related to identification. In these studies, the researchers examined (1) the validity of the dual–dis-crepancy approach (i.e., the use of various identification criteria, dual discrepancy identification versus IQ reading achievement discrepancy; (Burns & Senesac, 2005; Speece & Case, 2001).

Eight of the studies used Tier 1 (identification) in combination with Tier 2 (e.g., differentiated instruction, curriculum modifications) with respect to the implementation of RTI (Case, Speece, & Molloy, 2003; Coyne, et al., 2004; Fuchs, Fuchs, & Prentice, 2004; Fuchs, et al., 2005; McMaster, et al., 2005; O’Connor, et al., 2005; Torgesen & Davis, 1996; Vaughn, Linan-Thompson, & Hickman, 2003). In these studies, researchers implemented various aspects of RTI (e.g., screening, differentiated instruction); however, none of them implemented all three tiers. For example, the majority of the studies used some type of screening process (e.g., curriculum-based measures, standardized assessment), which led to the identification of children who were characterized as nonresponders, learning disabled, or at risk for academic failure in one or more content areas (see Appendix E for complete information about the multi-tier approach across studies).

Upon completion of this identification process, students then were provided with some type of specialized intervention (e.g., curriculum modifications, differentiated instruction); however, the efficacy of the multi-tier approach to identification and intervention was not the primary focus of the study. For example, one study analyzed predictors of responsiveness to intervention (Torgesen & Davis, 1996). Other studies addressed the effects of specific interventions on the development of children who were reported to be non-responsive to general classroom instruction. The interventions in these studies ranged from mathematical problem-solving to phonological instruction (Fuchs, Fuchs, & Prentice, 2004; Fuchs et al., 2005; Vaughn, Linan-Thompson, & Hickman, 2003; O’Connor et al., 2005). The remaining studies focused on topics such as (1) determining factors that mediate beginning reading instruction and (2) examining the validity of the dual discrepancy approach to identification commonly used in RTI (Case, Speece, & Molloy, 2003; Coyne, et al., 2004).

Further examination of the studies revealed that several did not follow the tiers of RTI sequentially as a hierarchy of intervention (i.e., Tier 1, Tier 2, Tier 3). For instance, two studies went directly from Tier 1 (identification) to Tier 3 (individualized interventions) (Torgesen, et al., 1999; Vellutino et al., 1996). Another study identified children through a Tier 1 screening process and then randomly assigned children to either a Tier 2 (peer-mediated small groups) or Tier 3 intervention (individualized intervention) (McMaster, et al., 2005), whereas only Tiers 2 and 3 were examined in another study (O’Connor, Harty, & Fulmer, 2005).

An additional study implemented the multi-tiered approach by moving students from Tier 1 to Tier 3 and then back to Tier 2 (O’Connor, 2000). This study looked at the effects of a layered approach to intervention; however, the tiers were not implemented sequentially. None of the studies directly assessed the effects of a complete intervention hierarchy implemented in sequential order.

Other characteristics of interventions

Standard treatment protocol versus problem-Solving approach. The studies were analyzed to determine whether a standard treatment protocol or problem-solving approach was used as the primary means for making decisions about the interventions. In the majority of the studies, a standard treatment protocol was used in which an empirically validated treatment was implemented for all targeted children (see Appendix E). One study (Case, Speece, & Molloy, 2003) used a problem-solving approach in which a collaborative consultation model was used to determine interventions for individual children; however, the focus of this study was on identification rather than the efficacy of the problem-solving approach used. Another study employed a combination of a standard treatment protocol and problem-solving approach in which a standard treatment protocol was tailored to individual.

Content of intervention. The studies were then analyzed to determine the content of the interventions used across studies (see Appendix A for a complete description of the characteristics of the interventions across studies). In nine of the studies, language and literacy-based skills were the primary focus of the interventions (Coyne, et al., 2004; McMaster, et al., 2005; O’Connor, 2000; O’Connor, Harty, & Fulmer, 2005; O’Connor, et al., 2005; Torgesen & Davis, 1996; Torgesen et al., 1999; Vellutino et al., 1996; Vaughn, Linan-Thompson, & Hickman, 2003). The content of the interventions included classroom reading instruction, phonological awareness instruction, peer-mediated strategies, one-to-one blending, explicit instruction in the alphabetic principle, and reading with fluency. Two additional studies focused on math interventions delivered either in a whole-class format or through small group tutoring (Fuchs, Fuchs, & Prentice, 2004; Fuchs, et al., 2005). The remaining studies did not deliver an intervention; instead, they focused explicitly on identification in Tier 1.
children’s needs (Vellutino, et al., 1996).

Delivery and duration of interventions. The individuals delivering the interventions across all studies included the lead researchers, other members of the research team, and teachers; however, in most of the studies, researchers were responsible for collecting assessment data and determining which children were nonresponsive to regular classroom instruction. The duration of the interventions in the studies ranged from twelve weeks to four years (in the case of O’Connor et al., 2005).

Assessing student progress and outcomes

Information about assessment methods and outcome variables is summarized to address the following issues: (1) when the outcome and progress monitoring measures were administered (e.g., pretest/posttest, multiple data points), (2) what types of measures were administered (e.g., standardized measures, observation instruments), and (3) what academic or behavioral outcomes were assessed (e.g., phonological awareness, decoding). A summary of the student measures used across studies can be found in Appendix A.

Timing of assessment for outcomes. With respect to when the outcome measures were administered, four of the studies employed a pretest/posttest methodology (Burns & Senesac, 2005; Coyne, et al., 2004; Fuchs, Fuchs, & Prentice, 2004; Torgesen & Davis, 1996), whereas the remaining studies used a combination of pretest/posttest and progress monitoring at regular intervals (ranging from biweekly to three times per year) (Case, Speece, & Molloy, 2003; McMaster et al., 2005; O’Connor, 2000; O’Connor, Harty, & Fulmer, 2005; O’Connor et al., 2005; Speece & Case, 2001; Torgesen et al., 1999; Vaughn, Linan-Thompson, & Hickman, 2003; Vellutino et al., 1996).

Academic and behavioral outcomes assessed. Improvements or changes in read-ing/language skills, including rapid naming, phonological processing, and vocabulary were assessed in the majority of the studies (Burns & Senesac, 2005; Case, Speece, & Molloy, 2003; Coyne et al., 2004; McMaster et al., 2005; O’Connor, 2000; O’Connor, Harty, & Fulmer, 2005; O’Connor, et al., 2005; Speece & Case, 2001; Torgesen & Davis, 1996; Torgesen, et al., 1999; Vaughn, Linan-Thompson, & Hickman, 2003; Vellutino et al., 1996). Four of the studies included measures of cognitive functioning or intelligence (Case, Speece, & Molloy, 2003; Fuchs, et al., 2005; Speece & Case, 2001; Vellutino et al., 1996). Only four of the studies included mathematical and/or neuropsychological measures (Fuchs, Fuchs, & Prentice, 2004; Fuchs, et al., 2005; Torgesen et al., 1999; Vellutino, et al., 1996). Other studies included classroom observation measures of behavior (e.g., behavior problems, attention, social skills), parent questionnaires, results from state assessments, and teacher interviews/rating scales (Case, Speece, & Molloy, 2003; Fuchs, Fuchs, & Prentice, 2004; Fuchs et al., 2005; Speece & Case, 2001; Torgesen et al., 1999; Vaughn, Linan-Thompson, & Hickman, 2003).

Methods of progress monitoring. With respect to the types of measures, the majority of the measures used for pretest/posttest purposes consisted of standardized assessments. In five of the studies, curriculum-based measurement (CBM) was the primary method of monitoring student progress. Both rate and level of growth were monitored as a means of determining a student’s response to instruction (i.e., the dual-discrepancy model) (Case, Speece, & Molloy, 2003; Fuchs et al., 2005; McMaster et al., 2005; Speece & Case, 2001; Vaughn, Linan-Thompson, & Hickman, 2003). In five other studies, however, progress was monitored through the use of standardized assessments or with measures developed for the current or previous studies (O’Connor, 2000; O’Connor, Harty, & Fulmer, 2005; O’Connor, et al., 2005; Torgesen et al., 1999; Vellutino et al., 1996). Only one study conducted periodic classroom observations to evaluate the quality of general instruction being implemented in the classrooms (Case, Speece, & Molloy, 2003).

Effect sizes reported. Ten of the studies reported effect size calculations in addition to inferential statistics in relation to the effectiveness of the interventions. The effect sizes ranged from -0.81 to 6.06 (see Table 5 for the range of effect sizes reported for each study.)

Image

Implementation fidelity. Twelve of the studies focused on the use of an intervention for the purposes of remediation and prevention. All of these studies provided detailed descriptions of the components of the intervention, and nine studies included implementation fidelity measures (Case, Speece, & Molloy, 2003; Coyne et al., 2004; Fuchs, Fuchs, & Prentice, 2004; Fuchs, et al., 2005; McMaster et al., 2005; O’Connor, et al., 2005; Torgesen, et al., 1999; Vaughn, Linan-Thompson, & Hickman, 2003; Vellutino et al., 1996). In one of the studies, however, fidelity of implementation was not relevant because the causal-compar-ative design did not involve delivering an intervention (Case, Speece, & Molloy, 2003). The absence of implementation fidelity in the other three studies makes it difficult to determine the validity of the intervention models because little information was provided about whether or not the interventions were implemented as intended.

Synthesis Findings and Conclusions

The findings from this research synthesis indicate that there is an emerging body of empirical evidence to support claims that RTI is an effective method for identifying children at risk for learning difficulties and for providing specialized interventions either to ameliorate or to prevent the occurrence of learning disabilities. There was considerable variability, however, across studies in how RTI was defined, implemented, and evaluated, which limits the findings. Although there was general agreement across studies about the conceptualization of RTI in terms of its key components and tiered implementation, there was very little agreement about the specific assessment or data monitoring procedures, the nature and focus of specialized intervention strategies, who delivered the interventions, the duration and intensity of the interventions, and benchmarks used for determining when a new phase should be initiated for individual children.

As an intervention, and across all studies, RTI varied on a number of dimensions that included (1) the functions of RTI (e.g., assessing factors that mediate reading achievement, comparing a dual-discrepancy model with IQ-reading achievement, assessing the tiered approach to assessment and intervention); (2) the individuals delivering the interventions (e.g., members of the research team, teachers); (3) the length of time children participated in the intervention (i.e., from twelve weeks to four years) and likely the duration of the sessions (although this information was not reported in the majority of studies); (4) what type of intervention approach was used (standard treatment protocol versus problem-solving); and (5) outcome measures used. It is noteworthy that only one study (Case, Speece, & Molloy, 2003) included an assessment of the quality of the general education curriculum and instruction, a defining feature of RTI used to determine whether the majority of students are achieving benchmarks in learning and behavior in Tier 1 prior to implementing differentiated instruction.

The studies also differed in their implementation of the multi-tier approach. Although the majority of studies used multiple tiers of intervention, the focus of these studies remained on the effects of a specific intervention or interventions within the context of a tiered model of intervention. The vast majority of studies assessed the effects of a particular intervention, focused on improving student outcomes, that was independent of the tiered RTI approach. Two studies (McMaster et al., 2005; O’Connor, 2000) implemented an RTI model using all three tiers; however, the tiers were not implemented sequentially. Furthermore, the focus of one of the studies (McMaster, 2005) was on the effects of the interventions rather than the multi-tier approach. None of the studies directly assessed the effectiveness of implementing a three-tier approach to assessment and intervention (another defining feature of RTI). Additional research is needed to understand which features contribute to the efficacy of RTI in practice.

The research synthesis also found that the majority of studies included school-age children as study participants (primarily in grades 1 to 3), which leaves unanswered questions about how an RTI model could be implemented effectively in early education settings that enroll 3- and 4-year-olds. Some research (Coyne et al., 2004; Torgesen & Davis, 1996) provides preliminary evidence that kindergarteners who are at risk for learning difficulties can catch up by first grade, if provided the appropriate supports in kindergarten. Moreover, the findings from these studies indicate that gains made by these children were maintained through the first part of first grade. Other research (O’Connor, 2000; O’Connor, Harty, & Fulmer, 2005; O’Connor, et al., 2005) supports the use of a multi-tier approach prior to first grade as well. However, these latter studies were implemented over a four–year period beginning in kindergarten; thus, the findings do not reflect the direct benefits of using RTI prior to first grade. Despite this, the findings from these studies indicate that the later incidence of placement in special education decreased as a result of taking part in an RTI model starting in kindergarten. The findings from these studies suggest that intervening in kindergarten, and possibly earlier, is a promising practice that could produce positive outcomes for young children who are at risk for learning difficulties in primary school.

Finally, the primary focus for intervention in a majority of the studies was language and literacy, with a particular emphasis on phonological awareness. Far less is known about the applicability of RTI for children who experience difficulties in other domains, such as math, social-emotional development, behavior, and for other precursors of learning disabilities that have been identified in the literature for younger children, including language delays, attention, and self-regulation difficulties (Lowenthal, 1998; McCardle, Scarborough, & Catts, 2001).

In conclusion, the research synthesis findings suggest that RTI is a promising approach, particularly because of its focus on sound instructional principles, such as effectively teaching all children, intervening early, using research-based interventions and instruction, monitoring student progress, and using assessment data to inform instructional decision-making (NASDSE, 2005). Further research is needed to understand the unique contributions of each of these elements of RTI as well as how these elements constitute an intervention package.

Next: Evidence-based practices

 
 
 
RecognitionandResponse.org was developed and is managed by the National Center for Learning Disabilities, which is solely responsible for its content.
Funding was made possible by grants from the Emily Hall Tremaine Foundation and the Cisco Systems Foundation.

Copyright © 2010 National Center for Learning Disabilities, Inc. All Rights Reserved.