Data-Based Decision Making

Of course, the building principal is ultimately responsible. However, there should be a data management team (DMT)who handles the data. The DMT should be two or three people (usually two) who know how to use data. That is usually the school psychologist and a teacher. There job is to make sure the data are collected in a standardized manner, to present the data in a consumable format, and to participate in the grade level team meeting at which the data are discussed.

Yes. Ted Christ at the University of Minnesota (Christ, 2006) found that eight (8) weeks of data are needed, assuming data are collected twice a week using appropriate standardization, in order for the rates of growth to be reliable enough for decisions. If you are collecting data less frequently, then more weeks are needed. Certainly within eight (8) weeks one could conclude that the intervention is working or that a different intervention should be tried within Tier II, but I would not make a resource decision (e.g., try a Tier III intervention) until enough data are collected for a reliable decision.

The grade level team should look at the data at least once each month.

I suggest general education. I also suggest that students with IEPs participate in general education benchmark assessments.

Yes. AIMSweb doesn't do everything I would like it to do, but the probes are well constructed. I also think the data management system is good. My only recommendation for assessments of a child who is LD is to monitor progress in the skill being taught and don't just rely on ORF, unless reading fluency is the target of the intervention.

Here are some suggestions: standardized administration, use of reliable data, identifying class wide problems by examining class medians, identifying students who need a Tier II intervention, determining rate of growth, evaluating rate of growth, and using a data-decision making framework.

I'm not sure what the question is. Yes, math data can be collected as a group and in fact is advantageous over oral reading fluency in that respect. We also need to collect both general outcome measures and subskill measures.

You answered your own question. It is very tough to beat ORF for 3rd and 4th graders. ORF correlates highly with just about any indicator of good reading (comprehension, prosody, etc.). The difference is that an ORF assessment takes just a few minutes whereas direct assessments of the other constructs take much longer and don't really tell you much that you didn't already know.

As for identifying kids who need interventions, I suggest looking at class wide problems first, but then using a resource allocation model in which the lowest 20 percent receive a tier 2 intervention. There is no research base to support a triage approach in which the lowest five (5) percent go right to tier 3. Thus, I suggest starting with tier 2 for just about every kid. Certainly some exceptions can be made, but I do not recommend a systematic triage.

With data! Get the data to them as quickly as possible and show them the high correlation with what they see in the classroom and state test scores. I also talk a great deal about efficiency and false positive/negatives. By efficiency I mean that I can get more useful information in a 3 minute assessment as I can with many much longer assessments or even from directly working with the kid for a period of time.

Your question is probably contextualized within a larger one that addresses buy in etc. I suggest checking out Ervin et al. (2006) article in School Psychology Review. We also included chapters from major implementation sites in our Handbook of Response to Intervention, which might be helpful too.

That is a policy decision. Much like transfers with children with IEPs, we have to look at the data that were collected in the previous schools and see if those data address our criteria, needs, etc. My hope is that those data could be converted to a rate of growth index that can continue to be monitored so as not to start all over again with data collection. However, the school will need to examine the data and see if they can be used.

The mainstreaming question for children with learning disabilities is a topic for a differt debate. I'll address the first part of the question, which is classroom teachers implementing interventions. I do think it is both practical and realistic. Tier 2 interventions have to be set up systematically so that they are part of the daily routine and somewhat easily selected and implemented. In other words, Tier 2 has to run like a well-oiled machine. Tier 3 is more individualized and requires teachers to implement interventions more unsystematically. That can be problematic but can be addressed in a high performing problem-solving team.

A good data management system (e.g., AIMSweb). Also, some schools record the data as they go (e.g., laptops at the data-collection stations) or some write the scores on an alphabetic printout and have clerical support staff enter them later.

Great question! You can do it normatively. Compute slopes for an entire grade then compute the mean and standard deviation. Your target would then be within one standard deviation of the grade-level mean. Alternatively, you could develop criterion-referenced goals by computing the benchmark assessment criteria based on the relationship between assessment data and state test score. Then determine the rate of growth necessary to maintain that level. For example, let's assume that a student who scores a 50 in the fall, 70 in the winter, and 85 in the spring (those numbers are made up) would likely pass the state (80% accurately predicting that he would pass). A student would have to grow at a rate of 35 words/minute across 32 weeks to maintain those levels. That is a rate of growth of 1.09 words/minute per week. That could serve as the criterion.

This is a great question with an easy answer that would be difficult to answer here. First, I suggest assessing students who are identified as struggling learners in the National Reading Panel areas. There are some good systems to do so. For younger students you could use nonsense word fluency to assess phonics, and something like phoneme segmentation fluency for phonemic awareness. There is also Renaissance Learning's Star Early Literacy assessment that assesses all areas of the NRP. Finally, other measures such as the Comprehensive Test of Phonological Processing looks at phonics, phonemic awareness etc. After identifying the root of the problem (i.e., comprehension, fluency, phonics, or phonemic awareness), the intervention can directly address the deficit.

A direct one! The federal regulations for LD identification explicitly mention that our primary variable of interest is wether or not a student will "make progress sufficient enough to meet state-approved results." That is a euphimism for passing the state test. Also, if we contextualize RtI as the systematic use of assessment data to most effeciently allocate resources in order to improve student learning (Burns & VanDerHeyden, 2006), then RtI is primarily a tool to enhance student learning for ALL students; which is the spirit of AYP as well.

No! You need a person or two to be the data management team who will then present the information to your grade-level team. A school psychologists can do this, but so could other people on staff who have training, expertise, and interest in data (e.g., know the difference between median and mean).

Oral reading fluency for grades 2 through 8. MAZE is probably best for grades 9 through 12, and the early skill indicators (e.g., letter naming fluency etc.) for younger students. However, I also suggest using group comprehension measures such as the Measures of Academic Progress and I know Ed Shapiro has found the 4Sight measures useful in Pennsylvania.

I mentioned AIMSweb because they currently have cornered the RtI market. AIMSweb ORF probes are well constructed and the data management system is easy to use. However, there are others. I encourage people to look at mClass by Wireless Generation, and Star Early Literacy by Rennaissance Learning. Edcheckup and Easy CBM also have good systems. I would look for a system that can a) give you data for each student, a class median, and a grade-level average, b) provides a graph of student progress, c) provides a slope of growth for that rate of progress, and d) very important - allows you to use other sources of data within the same management system.

1. Do we have a classwide problem?
2. What students need a tier 2 intervention (after the classwide problem is gone or didn't exist to begin with)?
3. Are the students making sufficient progress in tier 2?
4. What students need a tier 3 intervention or are there students that we should refer to the problem-solving team (as part of tier 3)?
5. Are there student that should be evaluated for special education eligibility?

That is not an easy one to answer. A comprehensive evaluation for special education eligibility is required by law, is a student's right, and is best practice. However, what makes up that evaluation is determined by the multidisciplinary evaluation team. The team might very well determine that they need (of course) parental information, developmental history, etc., but that those data with student response data are all that are needed to determine eligibility. However, there may be other sources of data that are needed and those should be collected. It is not unusual to use a standardized assessment of reading (e.g., Kaufman Test of Educational Achievement) or an adaptive behavior scale. Vision screenings, etc., would be used if there are questions about those areas.

The bigger answer is do we need to do a discrepancy model, and the short answer is no - you do not. However, until a district is READY to use RtI data for eligibility decision, I suggest using the traditional approach. A district is ready when they have DATA to support that what they are doing is RtI, that they are implementing it correctly, and that the interventions are occurring. Only then can the data be used for LD identification. I would so much rather a district use discrepancy approaches than do a poor job of using RtI data.

Remember, the RtI framework is an approach to resource allocation. CBM, district-wide, and state tests should be plenty of data to inform those decisions. Classroom teachers certainly would need different data, but those would likely not inform the RtI framework.

Once a student is identified as a struggling reader with these data, I then suggest sampling phonics, phonemic awareness, oral reading fluency and comprehension (the latter two are likely already collected). Those data will help determine WHAT to do. I also suggest looking at the accuracy of the skill. A child who works quickly but inaccurately, one who is slow an inaccurate, and one who is accurate and slow would likely require different interventions for the same skill (e.g., phonics).

Yes. Please check out the National Center on Student Progress Monitoring website for more information. You can access a chart that details and reviews progress monitoring tools at Review of Progress Monitoring Tools on the National Center on Student Progress Monitoring website.

Yes! Especially for screening. I also suggest monitoring progress with a general outcome measure (e.g., oral reading fluency) AND a measure of the skill in which the intervention is occurring (e.g., monitoring progress with nonsense word fluency for a phonics intervention). However, only use reliable data. Using psychometrically inferior tools results weakens the assessment systems. Many schools are using informal reading inventories like the Developmental Reading Assessment to assess reading for all kids. Those data might be very helpful to a classroom teacher, but they are not reliable enough to inform a resource-allocation decision-making framework. Moreover, assessment tools should also meet basic psychometric requirements as well (i.e., result in reliable data and valid decisions).

I am referring to identifying class-wide problems, identifying students for Tier 2 interventions, deciding if students are making sufficient progress (and the intervention is working), and should a Tier 3 intervention be attempted. Does that answer your question?