Previous Research Study
Pull-Out or In-Class:
Choosing the Most Effective Model of Instructional Delivery for At-Risk Students
Philip Goldfeder, Ph. D.
EDUC-6620-020 Collaborative Action Research
Finding the most effective instructional delivery model for at-risk students has been a topic of hot debate since the inception of the Elementary and Secondary Education Act in 1965, in which federal legislation was first passed to allocate funds for compensatory education (Passow, 1989). Since that time, the overall effectiveness of Title 1 programs has been challenged, charging that the various models of instruction are ineffective in decreasing the gap between economically-disadvantaged students and their middle-class counterparts. Despite this, funds have steadily increased for Title 1 programs, most of which subscribe to the pull-out model of instruction.
Historically, most Title 1 funds have been focused on preschool and elementary school students, with an emphasis on the remediation of reading and mathematics skills in a small group setting (Passow, 1989). The individual school district, and in some cases the individual schools, make the final decision about the content of services and about which model of instruction is most effective for their student population. As a result, it is the responsibility of the district to evaluate its Title 1 program and to choose the most effective instructional delivery model. Therefore, this research for action project set to determine the best model for the at-risk students in the
Currently, the reading and mathematics basic skills program for the fourth and fifth grade at-risk students at the
I suspect that an effective model has not yet been identified for our students because teachers lack the resources and time to evaluate the efficacy of the current model. I also suspect that teachers are not aware of other available models and the efficacy of these models as implemented in other districts.
Because I have the power to make recommendations to my school administrators regarding the best method of instructional delivery for the mathematics basic skills program, I felt a responsibility to investigate the efficacy of the pull-out model of instruction for at-risk students. Thus, I have researched this topic by conducting an investigation of research that studied the various models. I also used surveys to determine teachers’ current perceptions of efficacy to prepare for future comparisons to student achievement. The goal of my research was to identify the most effective model, to make plans for implementing it in my school, and to ultimately examine the resulting student performance in the classroom and on standardized tests.
1) Is the pull-out model for basic skills math instruction more effective than the in-class model in strengthening the skills struggling students need to be successful in the regular classroom?
2) Is the pull-out model for basic skills math instruction more effective than the in-class model in raising struggling students’ standardized test scores?
3) What other models of instruction have been proven effective for basic skills math instruction?
Review of the Literature
In my review of the literature, I found that the research on the different Title 1 instructional models was conflicting. While some researchers determined that the pull-out model was advantageous to some students in certain subjects, other studies found that there was no significant difference between achievements in different settings. Still others asserted that all instruction should be delivered in the regular classroom setting. In fact, Mullins and Summers noted that “the evaluation literature is so vast and its results so varied that virtually any hypothesis can be supported by a number of studies” (as cited by Passow, 1987, p. 7). However, the studies were unified by one common idea: researchers agreed that educators need to do more to close the achievement gap.
Description of Major Instructional Delivery Models
According to Passow (1989), the delivery of Title 1 services can have an impact on the effectiveness of the program. He identified four major models: 1) pull-out, 2) add-on, 3) in-class, and 4) replacement (Passow, 1989). Pull-out models take place in a location other than the regular classroom (Passow, 1989). Add-on programs are those that provide services in addition to the regular school day (Passow, 1989). In-class models are those in which a specialized teacher provides services to at-risk students in the regular classroom (Passow, 1989). Finally, replacement programs provide all of the instruction an at-risk student will receive in a given subject, always in a separate classroom (Passow, 1989).
The pull-out model can be further divided into three categories, as described by Madden and Slavin (1987) and McLaughlin and Vacha (1992). These include diagnostic-prescriptive programs, where students are identified through testing programs; tutoring programs, in which teachers, paraprofessionals, volunteers, or older students work one-on-one with at-risk students; and computer-assisted instruction, where students work on computers during their basic skills time (McLaughlin & Vacha, 1992). Of the three types of pull-out instruction, diagnostic-prescriptive programs represented the most common model for at-risk children (McLaughlin & Vacha, 1992). Therefore, research on this type of pull-out instruction is most available and will be discussed further later in this paper.
Madden and Slavin (1987)
Research conducted by Madden and Slavin (1987) found that the achievement of at-risk students can be increased through the implementation of intensive pull-out programs. In addition, the researchers concluded that a combination of extensive modifications to the regular instructional program and intensive pull-out support may be the most effective model of instruction for at-risk students (Madden and Slavin, 1987). The most effective programs, according to Madden and Slavin, were those that addressed students’ unique needs and adapted direct instruction so that it matched students’ readiness levels (1987).
The focus of their study was to evaluate and determine the most effective and replicable practices for pull-out instruction for elementary students, and thus its scope did not include in-class practices. It followed the best evidence synthesis method, which, according to Madden and Slavin, “combines the features of meta-analytic and narrative reviews” (1987, p. 3). The study evaluated published and unpublished articles, school district reports, government documents, and reports submitted to the Joint Dissemination Review Panel (JDRP), a U.S. Department of Education panel which reviews the evaluations of programs supported by federal funds (Madden & Slavin, 1987). Programs discussed in the study were required to assess the effects of the pull-out programs on standardized scales during at least one semester, and experimental groups of at-risk students had to be compared to control groups (Madden & Slavin, 1987). According to Madden and Slavin, all pull-out programs discussed were effective for at-risk students (1987).
One such study presented by Madden and Slavin (1987) evaluated the diagnostic-prescriptive pull-out model program in
McLaughlin and Vacha (1992)
McLaughlin and Vacha (1992) found similar results in their study of programs for at-risk students. In their analysis, research was compiled using computer searches of several databases, including the Educational Resources Information Center (ERIC), the Current Index to Journals in Education (CIJE), Resources in Education (RIE), and Psychological Abstracts (Psych Lit) (McLaughlin & Vacha, 1992). The programs shared were evaluated using Slavin's best-evidence hypothesis (as cited by McLaughlin & Vacha, 1992) and some programs were certified by the JDRP (McLaughlin & Vacha, 1992).
In their review of the research, McLaughlin and Vacha discussed and evaluated several different models of instruction for at-risk students, including pull-out and in-class (1992). They found that schools using diagnostic-prescriptive programs showed at-risk student gains in math, but not in reading, and concluded that this type of pull-out program was effective, although gains were small (McLaughlin & Vacha, 1992).
Tutoring programs, however, had a larger impact on at-risk student achievement (McLaughlin & Vacha, 1992). In fact, McLaughlin and Vacha (1992) commented that many of the Title 1 programs approved by the JDRP included a tutoring component. For example, programs developed by the Dade County Florida – Training for Turnabout Volunteers (TTV) and the School Volunteer Development Project (SVDP) demonstrated success in increasing at-risk student achievement in both math and reading, although more significant gains were made in math (McLaughlin & Vacha, 1992).
Another form of pull-out instruction evaluated by McLaughlin and Vacha that helped at-risk students make positive gains was computer-assisted instruction (1992). In a study conducted by Ragosta (as cited by McLaughlin & Vacha, 1992), students who received a combination of computer-assisted instruction and small-group instruction with the teacher made small but significant gains.
Jakubowski and Ogletree (1993)
Jakubowski and Ogletree, however, found different results in their 1993 study of intervention methods for at-risk students. Like the research conducted by Madden and Slavin (1987) and McLaughlin and Vacha (1992), Jakubowski and Ogletree reviewed the findings of other researchers to evaluate the effectiveness of the pull-out model for both math and reading instruction (1993). In addition, Jakubowski and Ogletree conducted their own study on the effectiveness of the pull-out model in reading instruction (1993). Unlike Madden and Slavin (1987) and McLaughlin and Vacha (1992), Jakubowski and Ogletree (1993) concluded that there were no significant achievement gains for at-risk students who received pull-out interventions when compared to students who did not receive pull-out services.
In the experiment conducted by Jakubowski and Ogletree (1987), two groups of 15 fifth and sixth grade students were compared to determine the effectiveness of a pull-out reading intervention program. For one year, one group was provided pull-out services while the other group had in-class instruction (Jakubowski & Ogletree, 1987). Student achievement was assessed using pretest and posttest scores from the Iowa Test of Basic Skills (Jakubowski & Ogletree, 1987). The study found that there was no significant difference between scores for at-risk students receiving pull-out services (Jakubowski & Ogletree, 1987).
Although the study conducted by Jakubowski and Ogletree (1987) focused on reading achievement, the researchers also evaluated findings about both math and reading instruction from the Instructional Dimensions Study. Conducted by the National Institute of Education, the study was part of a mandate to evaluate compensatory education provided through federal funds (Jakubowski & Ogletree, 1987). One focus of the study was to compare the effects of pull-out and in-class instruction on students receiving compensatory services under Title 1 (Jakubowski & Ogletree, 1987). The study collected data from 400 classrooms in more than 100 schools in 14 districts (Jakubowski & Ogletree, 1987). Findings were conflicting. According to Jakubowski and Ogletree (1987), the National Institute of Education found that students made the same gains when taught individually or in groups, and that pull-out programs were significantly more effective for some groups but not for others. For example, first graders made significant gains in in-class programs, while third graders gained more in pull-out programs in math but showed no difference in reading (Jakubowski & Ogletree, 1987).
A second national study evaluated by Jakubowski and Ogletree also emphasized the method of instructional delivery in its research (1987). This study, conducted by the Far West Laboratory for Educational Research and Development, was the result of criticisms of the pull-out model (Glass & Smith; Kimbrough & Hill; Allington; Johnson, Allington, & Afflerbach; as cited by Jakubowski & Ogletree, 1987). According to the results of this study, no method of instruction was more effective than the other (Jakubowski & Ogletree, 1987). The study did, however, contradict findings of other studies that stated that pull-out programs take valuable learning time from the regular class instruction; rather, most schools redistributed instructional time across programs (Jakubowski & Ogletree, 1987).
Passow (1989), however, concluded in his research of compensatory education that all instruction should be mainstreamed. In his evaluation of Title 1/Chapter 1 instruction, he stated that educators
need to capitalize on research and experience, and design and implement educational engagements and opportunities which will ‘close the gap’ by dealing with the disadvantaged in an integrated, articulated fashion, drawing on all of the personnel and material resources of the school, the family, and the community. To do this does not mean that Chapter 1 should be eliminated, but rather that its personnel and material resources should join the mainstream of education for the optimum cognitive and affective development of those youngsters who enter school at a disadvantage for whatever reason. Only then will we be able to attain the twin goals of equity and excellence. (Passow, 1989, p. 44)
In his comprehensive analysis, Passow shared the results and opinions of other researchers in the field (1989). For example, Passow cited Doyle as stating that compensatory education focuses on small steps towards the mastery of content, rather than the creation of coherent mental representations necessary to make connections, resulting in higher scores on standardized tests but lower achievement in regular school programs (1989). He also demonstrated a concern that teachers of pull-out programs and regular classroom teachers infrequently coordinate instruction, which leads at-risk students to think that skills learned in each setting are unrelated (Passow, 1989).
In math, Passow stated that at-risk students receive drill-and-practice instruction rather than practice using higher-order thinking skills, which widens rather than reduces the achievement gap (1989). He quoted Roberg, who stated “if one views mathematics as things human beings do…there is nothing in the programs…that would give low-income students an opportunity to do any important mathematics” (1989, p. 23).
According to Passow, Title 1 programs should not consist only of pull-out programs (1989). Through his evaluation of the research, he determined that at-risk students should be provided the same balanced curriculum as high-achieving students, which should not be aimed at achieving success in tests of minimum skills (Passow, 1989). He concluded by stating that although test scores may rise, the gap in at-risk students’ total education will widen
Noteworthy within the research was the study conducted by VanScoy (1997). VanScoy evaluated classroom teachers’ preferences for the pull-out and in-class model for compensatory education using Title 1 funds (1997). Although VanScoy did not assess the effectiveness of the two programs, and in fact recommended the future assessment by comparing standardized test scores, she did explore the connection between teachers’ preferences and their current practices. VanScoy (1997) explained that because teachers are a major factor in the success of a program, it is important to determine their preferences when considering reform.
VanScoy constructed a survey to collect data from 214 teachers and she analyzed her data using the Likert scale, relative frequency tables, bar graphs, and Chi square to test her hypotheses (1997). From her data, she concluded that teachers who used the in-class model preferred the in-class model, and teachers who used the pull-out model preferred the pull-out model (VanScoy, 1997). Of teachers who were using both the in-class and pull-out model, 42% preferred in-class, 33% preferred pull-out, and 25% preferred a mix of both models (VanScoy, 1997).
While this study did not provide data about the effectiveness of pull-out versus in-class programs for at-risk students, it suggested that teachers are most comfortable using their current model and thus are more likely to be successful in delivering instruction in this way
Discussion of the Literature
As stated above, the literature reviewed presented conflicting data and opinions about the efficacy of pull-out and in-class models of instruction for at-risk students. Although research compiled by Madden and Slavin (1987) was thorough and well-funded, it lacked a comparison to studies conducted about the effectiveness of the in-class model. The results of McLaughlin and Vacha’s (1992) study were also well-researched, and they also offered data about in-class models of instruction. This allowed for the comparison between the two models as well as information to be used for combining methods (McLaughlin & Vacha, 1992).
The research provided by Jakubowski and Ogletree (1987) was valuable because it provided studies from other researchers and also included a direct comparison between achievement levels of at-risk students receiving instruction through the pull-out and in-class models. However, the results showed that there were no significant differences in achievement. Instead, the research suggested that the quality of the instruction had a greater effect on student achievement than the model followed. Therefore, one is unable to conclude which model is more effective from this study; rather, the next step would be to evaluate the content and quality of the Title 1 services.
Passow (1987) provided several studies and expert opinions that favored the in-class model of instruction for at-risk students. His research was helpful for my investigation because it included references to studies that were specific to math instructional models for at-risk students (Passow, 1987). Furthermore, Passow’s research served as an excellent resource for my collaborative action research plan because it was both detailed and thorough (1987).
VanScoy’s (1997) study was also valuable for the purposes of my collaborative action research plan because its focus was most closely aligned to my investigation. Therefore, research discussed within her paper was quite helpful for preparing and comparing the results of my study. Furthermore, her research was the most current of the pieces of literature I found and thus was the most applicable to today’s educational climate. Although her study did not evaluate the efficacy of the pull-out and push-in model of instruction for at-risk students, it provided suggestions for next steps to help make a final determination.
As a result of this research, it became apparent that there are several instructional delivery models that have found to be effective for raising the achievement of at-risk students. This realization led to the inclusion of my third research question, which focused on finding other effective models of basic skills instruction. The research also caused me to consider the content and quality of instruction when determining the efficacy of the instructional models in my study. This was the direct result of the research compiled by Madden and Slavin (1987) and Passow (1989), who suggested that the setting for Title 1 services is not what matters; rather it is the program itself that has the greatest impact on student achievement.
While the research discussed in these five pieces of literature presented conflicting views on the efficacy of the pull-out versus the in-class model of compensatory education for at-risk students, all agreed that the overall achievement gap is not closing at a satisfactory rate. The research showed that the gains made by at-risk students, if any, need to be greater. As stated by Passow (1989), it is our responsibility as educators to continue to research, evaluate, design, and implement instructional practices to provide the best instruction possible to all of our students.
Data Collection Plan and Methodology
Because I conducted a research for action study, the majority of my data came from existing research reports and journal articles. I found several sources that reported on topics similar to mine that discussed programs that had proven most effective in other districts. I also surveyed classroom teachers and other basic skills teachers to determine which models were used and to analyze the teachers’ thoughts and opinions of the efficacy of those models. A list of the sources used to determine answers to each of the three research questions discussed can be found in the data collection matrix in Appendix A.
To answer my first research question about the efficacy of the pull-out model in strengthening the skills struggling students need to be successful in the regular classroom, I determined that the best sources of information were classroom teacher surveys (see Appendix
B) and existing research reports. The multiple-choice classroom teacher survey was both valid and reliable because it was carefully designed to elicit responses specific to the efficacy of the instructional delivery models and because I had no reasons to be suspicious of the accuracy of the answers provided to me by the classroom teachers (Sagor, 2000).
The first research report, Curriculum and Instruction in Chapter 1 Programs: A Look Back and Ahead (Passow, 1989), was a comprehensive analysis of the results and opinions of other researchers in the field regarding the various instructional delivery models for at-risk students. This source was valid because it was a non-biased report on published studies, and there were no intervening variables that caused me to distrust the data shared (Sagor, 2000). It was reliable because Passow himself had triangulated the data by providing multiple independent sources to support his results (Sagor, 2000).
The second research report, The 2-5 Collaborative In-Class Model; A Restructuring of the Title I Program (Hagopian, 1996), was a study conducted in
The fourth data source for my first research question was also a research report. Titled The Effectiveness of Chapter I Pull-Out Programs on Reading Achievement (Jakubowski, & Ogletree, 1993), the study included a report on published research. In addition, the authors conducted their own research to determine the effectiveness of a pull-out program (Jakubowski & Ogletree, 1987). In their report of published studies, Jakubowski and Ogletree evaluated findings about both math and reading instruction from the Instructional Dimensions Study (1987). Conducted by the National Institute of Education, this study was part of a mandate to evaluate the compensatory education provided through federal funds and it compared the effects of pull-out and in-class instruction on students receiving compensatory services under Title 1 (Jakubowski & Ogletree, 1987). This data source was valid because the instruments used in the study measured the effects of the two different types of instruction. It was reliable because it was a federal mandate and thus I was confident that it was monitored closely for accuracy.
Like my first research question, I investigated the answers to my second research question by using teacher surveys and research reports. This research question related to the efficacy of the pull-out model in raising struggling students’ standardized test scores. Ideas related to this question were included on the classroom teacher survey discussed above (see Appendix B). Furthermore, I distributed multiple-choice surveys to basic skills teachers (see Appendix C). By surveying basic skills teachers, I was able to get a picture of which models were currently used, liked, and found to be effective from the point of view of the basic skills teacher. As a basic skills teacher myself, I know that one of my major concerns is my students’ achievement on standardized tests. Therefore, a survey of basic skills teachers gave me a different view of the efficacy of the programs, because their concerns were different than those of the classroom teachers. As discussed above, the surveys were both valid and reliable because
I carefully designed them to be specific to the efficacy of the instructional delivery models and because the teachers were qualified by their experiences to answer the questions accurately.
The research report titled Effective Pull-Out Programs For Students At Risk (Madden, & Slavin, 1987) is the second source used to determine the efficacy of the pull-out model for increasing student achievement on standardized tests. The focus of this study was to evaluate and determine the most effective and replicable practices for pull-out instruction for elementary students. The study evaluated published and unpublished articles, school district reports, government documents, and reports submitted to the JDRP, a U.S. Department of Education panel which reviews the evaluations of programs supported by federal funds (Madden & Slavin, 1987). Programs discussed in the study were required to assess the effects of the pull-out programs on standardized scales during at least one semester, and experimental groups of at-risk students had to be compared to control groups (Madden & Slavin, 1987). This source demonstrated validity because it claimed to report on student achievement on standardized tests, which it did. Also, it was reliable because it must be accurate to be approved by the U.S. Department of Education.
I referred to a previously used data collection tool as my third source for my second research question. This source was the research report titled The Effectiveness of Chapter I Pull-Out Programs on Reading Achievement (Jakubowski, & Ogletree,1993), and it was also used as my fourth collection tool for my first question. This source included information about standardized testing as well as classroom achievement, and thus it was appropriate for finding answers for both research questions. Its validity and reliability is discussed above.
My third and final research question was designed to determine which other models of instruction have proven effective for basic skills math instruction. Similar to my choices for data collection tools for my first two questions, I also used teacher surveys as one of my data sources. As with my second question, I surveyed both classroom teachers and basic skills teachers to get a more accurate picture of what models were being followed and what had proven effective. A discussion of the validity and reliability can be found above.
My second data source is the research report titled School Programs for At-Risk Children and Youth: A Review (McLaughlin & Vacha 1992). In their analysis, research was compiled using computer searches of several databases to evaluate several different models of instruction for at-risk students, including pull-out and in-class (McLaughlin & Vacha 1992). It was both a valid and reliable source because, as in the studies discussed above, there were no reasons to distrust the data and some of the studies discussed were also approved by JDRP (McLaughlin & Vacha 1992).
My third and fourth data sources were research reports I also used to answer my first two research questions. These two sources were Curriculum and Instruction in Chapter 1 Programs: A Look Back and Ahead (Passow, 1989) and Effective Pull-Out Programs For Students At Risk (Madden & Slavin, 1987). Each source included a discussion of the various instructional models and their overall efficacy. Their validity and reliability is discussed above.
To complete my triangulated data collection plan, I distributed my surveys to classroom teachers and to basic skills teachers. Classroom teachers were surveyed through
While much of the data collected during this study was conflicting, there were some patterns that emerged during the scientific coding process as described by Sagor (2000). First, the data showed that Title 1 programs can have a small but positive effect on the achievement of at-risk students. The data also showed that no one program is consistently effective; rather, programs must be tailored to meet the needs of the specific group of students. While gains were most substantial in earlier years, gains were not sustained when students discontinued services. Gains were also lost during the summer months. Several pieces of data pointed to the quality of the program, rather than the delivery model, as having the greatest impact on
Effects on Classroom Achievement
The classroom teacher survey indicated that neither pull-out nor in-class instruction resulted in an increase in classroom achievement for at risk students. However, 38% of participants reported an increase when a combination of both methods was used. This was supported by findings shared by Madden and Slavin that stated that the most effective programs are a combination of extensive modifications of the regular instructional program to assist in-class Title 1 students and intensive interventions as a pull-out program (1987).
The survey also indicated that the most common reason for a lack of classroom achievement for students receiving pull-out services was that students missed too much of the regular classroom instruction. However, all of these teachers also reported that they taught new material while their basic skills students were receiving pull-out services. This suggested that classroom achievement for at-risk students can be positively affected by restricting the classroom teachers from presenting new material while the basic skills students are out of the classroom.
The data presented several reasons why the pull-out model of instructional delivery has not resulted in significant gains in past practices. Passow (1989), Hagopian (1996), and Jakubowski and Ogletree (1993) all reported that pull-out instruction could result in decreased instructional time due to moving to a new location, the fragmentation of mathematical concepts, stigma resulting in lower expectations and simpler assignments in the regular classroom, and a lack of collaboration between the basic skills instructor and the regular classroom teacher. Hagopian (1996) furthered this by connecting the lack of coordination between teachers with the reduction in instructional time. In his study of fourth and fifth graders in a large city school, he found that there is a greater congruence in an in-class program (Hagopian, 1996). Additionally, the surveyed classroom teachers agreed that a lack of collaboration contributed to the low classroom achievement of basic skills students. For pull-out instruction to be effective in strengthening the skills struggling students need to be successful in the regular classroom, program designers must find ways to reduce or eliminate the impact of the factors discussed above.
Carter, however, reported several positive results of the pull-out instructional delivery model (as cited by Passow, 1989). He explained that pull-out programs can produce a more positive learning environment, smaller instructional groups, higher staff-to-student ratios, more student on-task behavior, less behavioral management issues, a more harmonious classroom atmosphere, fewer negative comments by teachers, a higher quality of cognitive monitoring, and a stronger organization of activities. Again, the implications of these findings are that when designing an effective pull-out program, one must capitalize on the strengths and decrease or eliminate the impact of the weaknesses.
The data showed that the amount of time, the size of the group, and the match between the curriculum and the assessment measures were more related to student achievement than the delivery model. According to the survey of basic skills teachers, of those that reported student gains more than half reported working with groups of between four and seven students. Furthermore, successful basic skills instructors met with their students at least three times per week, and each session was usually at least one period long. Data collected from Madden and Slavin (1987) indicated that programs that closely matched instruction to the pre and post assessments showed a greater increase in student achievement. Additionally, Passow (1989) reported that the success of a program depended on its goals.
Finally, some of the data indicated that certain groups profited more from pull-out programs than others. According to Jakubowski and Ogletree, first graders benefited from in-class instruction while third graders showed greater increases when participating in a pull-out math program. This contradicted findings reported by Ascher, who found that pull-out students had lower achievement gains than those who remained in the classroom (as cited by Hagopian). In the survey of the classroom teachers, not one teacher reported that students made significant gains when using either the pull-out or in-class model. Rather, gains were only made when students received both types of services. This supports the theory that choosing the most effective delivery model will depend on the students serviced.
Effects on Standardized Test Scores
In the survey of classroom teachers, it was found that the greatest increase in standardized test scores occurred when both the pull-out and in-class model of instructional delivery was used. This is not supported by the survey of basic skills teachers, who reported that the pull-out model resulted in the greatest increase in standardized tests scores. Of the basic skills teachers who reported test score increases, all included test preparation activities in their instruction for at least one month, and 25% of respondents reported that their entire program was test preparation. All of the classroom teachers and basic skills teachers who reported an increase in standardized tests scores also believed that the basic skills program played a role, regardless of the instructional delivery model.
There was no data to support that pull-out programs alone resulted in significant gains in standardized tests scores for at-risk students. Madden and Slavin reported that very few models presented convincing evidence of effectiveness; rather, most showed little or no growth in successive years of fall-to-spring test scores (1987). Jakubowski and Ogletree reported that when using the Iowa Test of Basic Skills as a measure, there were not significant gains when comparing pull-out and in-class classes after one year of instruction (1993). According to data from Madden and Slavin (1987), most students experienced a fall-to-spring gain of 8-12 points, and a decline over the summer of the same magnitude. However, the researchers cautioned that these findings may have been the result of a loss of high-scoring students from the program.
Data collected from Jakubowski and Ogletree, however, reported that although there were no significant gains, pull-out programs did have small but positive effects (1993). They indicated that gains made by students in pull-out programs were slightly larger than those made by students who did not participate; comparison scores of students receiving services and students qualifying for but not receiving services showed that those who did not receive services did not improve as much as those that did, and their percentile ranks declined (Jakubowski & Ogletree, 1993).
Only two pull-out programs were shown to be effective in raising the standardized tests scores for at-risk students, and each showed flaws in measuring these gains. The first study, as reported by Madden and Slavin (1987), showed a gain of 3.0 normal curve equivalent (NCE) points in math. However, these results varied wildly when measured in alternate ways, such as by using fall-to-spring scores as compared to spring-to-spring scores. As a result, it is difficult to trust the findings of this study. The other study reported a 10.4 point increase (Madden and Slavin, 1987). Researchers suspected, however, that the instruction provided was focused on
The data suggested that even when at-risk students received pull-out services that increased their standardized tests scores, they did not move substantially toward their more advantaged counterparts. In fact, the more similar the comparison group was to the students receiving services, the greater the achievement benefits associated with program participation. Therefore, Jakubowski and Ogletree report that the Title 1 instruction was not successful in closing the achievement gap (1993).
Other Effective Models
Surveys returned by classroom teachers and basic skills instructors did not indicate the use of instructional delivery models other than pull-out and in-class. Therefore, this source was unable to be used for data collection purposes, as previously proposed in the data collection plan. Instead, data has been collected from previous studies.
Data collected from Mullins and Summers reported that no one program will work across the country and for all years of schooling; they found that it is likely that only programs developed for specific categories of students with specific socioeconomic backgrounds will be effective (as cited by Passow, 1989). Passow also reported that in addition to pull-out and in-class models, there are two other major models of instructional delivery that have proven effective for the different categories of students: add-on programs and replacement programs (1989). McLaughlin and Vacha (1992) and Madden and Slavin (1987) also reported that diagnostic prescriptive programs, tutoring programs, computer assisted instruction, class-wide peer tutoring, applied behavior analysis, direct instruction of basic skills, and precision teaching were found to be effective in some situations. According to data compiled by McLaughlin and Vacha, effective programs had three common components: early intervention, skill-based tutoring, and some type of tracking system to monitor student progress (1992).
The data collected from Passow (1989) showed that add-on programs are most successful when used as an early intervention; effective programs focused on in-class achievement, the development of a positive attitude towards school for both students and parents, and the enhancement of feelings of self-worth. The data revealed that add-on programs are a rare delivery model, and thus there is no other data that discussed add-on programs.
The most commonly used model was the diagnostic- prescriptive program; in an analysis of performance by fifth grade students in
Similar to the diagnostic-prescriptive program in that it provides direct instruction based on student need, tutoring programs were found to be effective for some students. Data collected from McLaughlin and Vacha found that when tutors were trained continuously, students made gains of 1.1 points in math (1992). Furthermore, the researchers reported that many of the programs approved by the JDRP included a tutoring component (McLaughlin and Vacha, 1992). Madden and Slavin, however, indicated that there is little data to prove that tutoring has long-term effects on student achievement (1987). Another type of tutoring, class-wide in-class tutoring, was shown to be effective by several controlled studies (McLaughlin & Vacha, 1993). In this type of tutoring, one teacher manages a class-wide daily tutoring program for 30 minutes.
Computer Assisted Instruction
Another delivery model that has resulted in some positive gains, provides direct instruction based on student needs, and keeps records of student progress is computer assisted instruction (CAI). McLaughlin and Vacha (1992) reported that studies in
Applied Behavior Analysis
McLaughlin and Vacha reported that the JDRP granted approval of applied behavior analysis for Title 1 programs, in which teachers learn the basic principles of learning, are trained to implement effective and systemic procedures, are required to identify precise goals, and collect data in an ongoing manner to determine student progress (1992). Data indicated that the strength of the program is that it takes place in-class and does not suffer from the weaknesses inherent to pull-out programs (McLaughlin and Vacha, 1992).
Direct Instruction of Basic Skills
Data gathered from McLaughlin and Vacha indicated that the direct instruction of basic skills affected student gains for six years following instruction, and that direct instruction in kindergarten generated later student gains in total math achievement (1992). Data also showed that this type of scripted program, in which teachers learn to sequence skill introduction, present new concepts directly, provide modeling and relevant examples, guide repeated practice, and test for mastery, resulted in more students finishing high school and applying to and graduating from college (McLaughlin & Vacha, 1992).
Several studies, as well as evaluations performed by the JDRP, have shown that precision teaching can have a positive effect for at-risk students (McLaughlin & Vacha, 1992). The data showed that when used to pinpoint behavioral strategies, design instructional programs, evaluate learning continuously, analyze data, and make instructional decisions based on child progress, teachers have been from two to five times more successful in developing strategies to assist at-risk students (McLaughlin & Vacha, 1992).
The data discussed above provided several findings on the efficacy of the pull-out instructional delivery model for at-risk students for classroom achievement and standardized tests scores. Some of these findings supported each other while others were conflicting. As a result, it is not possible to make a final determination regarding which model would be most effective for the students in my district. As reported by Passow (1989), Cooley (as cited by Jakubowski and Ogletree, 1993), and Mullins and Summers (as cited by Passow, 1989), no one program was consistently effective for all groups; rather programs must be tailored. Therefore, the flaw in this research is that it does not study the effects of the various instructional delivery models on the students in my district. To answer my research questions in regard to my specific students, I will need design a plan to investigate the efficacy of the pull-out model in my
I have served as a fifth grade basic skills teacher in the
This year, I will be implementing the in-class model of instructional delivery. This will give me the opportunity to compare at-risk student achievement in the classroom and on the NJ ASK when participating in an in-class program to the past achievement of pull-out students. A timeline for the implementation of the action plan can be found in Appendix D.
According to Jakubowski and Ogletree (1993), it is critical to ensure that the control and experimental groups are comparable at the beginning of a study. Therefore, I will need to carefully evaluate the achievement levels of my groups of students at the start of my investigation. I will do this by comparing the class grades of each group to see if they are at a similar level. I will also compare the fourth grade spring NJ ASK scores to see if student group performance was similar in the different years. If the groups are comparable, I can proceed with my study. If the groups are substantially different, I will need to take this into consideration when analyzing the results at the end of my investigation.
After analyzing students’ fourth grade scores to determine comparison levels, I will implement the in-class model of instructional delivery for my fifth graders. Based on my research, I have determined that the optimum number of class sessions is between three and five times per week. I have also determined that sessions should last one period long. Therefore, I will create a schedule that allows me to provide in-class basic skills instruction to my at-risk students at least three times per week for sessions of 42 minutes each. My research also revealed that Title 1 programs were most successful when classroom teachers and basic skills instructors collaborated frequently. Therefore, I will design a schedule that allows me to meet with the classroom teachers at least once per month to discuss student progress and to plan for
Madden and Slavin (1987) cautioned that in some studies, the choice of measurement tool greatly affected the results of the investigation. They suggested that the most valid measurement tool is spring-to-spring assessments (Madden & Slavin, 1987). In fact, in their own investigations, Madden and Slavin required that studies had to be at least a year’s duration; fall-to-spring NCE gains had been found to have serious methodological problems and were not considered adequate for proving efficacy (1987). Therefore, for each group I will compare gains made from the fourth grade NJ ASK scores to the fifth grade NJ ASK scores. I will also compare final fourth grade class achievement to final fifth grade class achievement for each year. Finally, I will redistribute the classroom teacher survey to determine teachers’ perceptions for the experimental year.
It is our duty as educators to provide the most effective instruction possible for all of our students. As a basic skills teacher, it is my responsibility to research, evaluate, and implement the most effective instructional delivery model for my at-risk students. To accept the current model without scrutiny opens the possibility of denying success to the students most in need. This is irresponsible, unethical and inexcusable. For this reason, it was both surprising and frightening to learn that while the pull-out and in-class models have received criticism from researchers, districts have done little to evaluate their effectiveness in their schools in order to
While the data I have collected indicated that Title I programs have small but positive effects on student achievement, it also revealed that no program model is consistently effective for all groups. Programs must be carefully tailored to the population served to close the achievement gap between at-risk students and their more advantaged counterparts. Therefore, it is necessary to investigate the efficacy of the pull-out model as compared to the in-class model in the
Hagopian, G. (1996). The 2-5 Collaborative In-Class Model: A Restructuring of the Title I Program.
Jakubowski, D., & Ogletree, E. J. (1993). The Effectiveness of Chapter I Pull-Out Programs on Reading Achievement.
Madden, N. A., & Slavin, R. E. (1987). Effective pull-out programs for students at risk. [
McLaughlin, T. F., & Vacha, E. F. (1992). School Programs for At-Risk Children and Youth: A Review. Education and Treatment of Children. 15 (3), 255-67.
Passow, A. H. (1989). Curriculum and Instruction in Chapter 1 Programs: A Look Back and A Look Ahead. ERIC Clearinghouse on Urban Education, Teachers College,
Sagor, R. (2000). Guiding school improvement with action research.
VanScoy, L. J. (1997). A Study To Determine the Effects of In-Class and Pull-Out Instruction for Title 1 Students in