Building Education Programs that Work for All Students

Forcing teachers to use innovations almost never works. Implementation experts call these "conscripted staff" and avoid them like the plague.

Buy in is a big part of the Selection Process. In essence, you present a lot of information about the innovation (who does what to achieve student benefits, what kinds of support will be there for teachers, etc). Then you engage the teachers in a conversation about the innovation until they have had their questions/ concerns answered. Then you ask for those who are thinking about using the innovation to attend a meeting where you get into the details of the innovation. Then, you ask for volunteers to be in the first group of teachers to learn how to use the innovation. Then, you interview each teacher individually using some of the methods on the website noted above to see if the teacher has the background knowledge, experience, and skills that are important for using the innovation. We think of this as a mutual selection process: the teacher is choosing to use the innovation and you are choosing the teacher to try it. Both of you need to make a fully informed choice.

This takes longer but helps assure a good chance at success. As we say, pay now by taking time to carry out the selection process or pay later trying to deal with disgruntled folks who do all they can to sabotage the effort.

See the answer to the question above. The Exploration Stage of implementation and selection of staff are crical (see the article "Exploration adnd Adoption"). How you start has a big influence on how you end up.

Support for teachers and others involved in using RtI to benefit students starts early and never ends. Once teachers are selected (see previous answers), they need training so they know the basic philosophy, values, knowledge, and skills they will need.

The key to success is to follow training with skilled coaching. This sometimes is a master teacher, administrator, or staff person already skilled in RtI. The coach knows the technical aspects of RtI and the craft knowledge (how does this fit into all the other things teachers/ others do). A coach also has the requisite coaching skills (how to teach adults in a respectful manner, how to offer technical and emotional support, etc).

Another important support in practice is regular performance assessments of teachers using RtI. It is too bad that most people view evaluation negatively and shy away from it. In practice, performance evaluations can be used as sensitive feedback for teachers, coaches, and administrators who all "own the results" as their own and make positive and constructive decisions based on those data. The hardest thing in the world is to get sensitive, honest feedback from a trusted source -- skilled coaches fulfill that role when using innovations such as RtI.

Teachers need to feel that training, coaching, and administration are there for them and are being helpful at every turn. Good schools have 360-degree feedback where twice a year teachers rate those around them re: helpfulness, communication, concern, etc. Teachers are the "producers" in education systems - the rest of us are their assistants. How helpful are we? Let's ask the teachers.

Don't you people ever ask an easy question!! I can tell we have a bunch of smart, experienced people on the other side of this computer screen.

The solution is embedded in your question -- you set expectations when introducing RtI. During the Exploration Stage you meet with teachers and staff (see previous questions) and with key stakeholders like parents and partners in other systems that you will need for tier 2 and 3 services for kids. Like we discussed for teacher selection, you spend whatever time is needed to describe RtI, answer questions, address concerns, and get folks on board.

Any effective change takes a couple of years to really take hold. You can describe to the stakeholders the implementation process (see the "Stages of Implementation Defined") and the care you are taking to make sure the changes you are about to make (i.e. using RtI) will actually be effective for students who need extra help. The last thing you or they want is to make changes that don't work or maybe make the situation worse than it is.

I sometimes resort to begging for their patience while giving them signposts to mark progress and hold me accountable for doing the job well. You may want to establish an Advisory Group so parents and stakeholders can help during the process.

Quick and dirty attempts to use innovations rarely work. The patient approach works 80% of the time. You have to choose which route to take up front. Parents, teachers, and administrators all want the best things for their children/ students. You need to get buy in on the process for making effective changes that actually have a good chance for student benefits.

You are not alone on this one. The issue of knowing whether we are doing what we say we will do can be found in most attempted implementation efforts.

Implementation scientists talk about "fidelity" -- we prefer to talk about performance assessments (see the "Staff Evaluation" article). Part of using innovations such as RtI is to get teachers used to having someone in the classroom or in meetings observing the process and offering helpful advice (i.e. coaching) or rating based on a standard protocol (i.e. performance assessment). Teacher comfort with this process is heavily dependent on the style and helpfulness of these observations. Remember, kids need a 4:1 ratio of positive to negative feedback; adults need a 10:1 ratio! Use lots of descriptive praise! Critical feedback can wait for the relationship to be established.

There are lots of reasons teachers do not use an innovation and we often are quick to blame the teacher. In implementation work, we are looking to fix the problem, not the blame. It may be the teacher does not feel prepared or comfortable -- these are selection, training, and coaching issues and, ultimately, administrative issues. It may be the teacher does not feel he/she is "doing it right" -- this person needs the information from a performance evaluation. Maybe the teacher feels like no one cares -- this is an issue with the tone/ climate the administration has created in the school. In our experience the teacher is rarely to blame, we have to hold ourselves accountable for creating the conditions under which we know day to day how each teacher is doing in every classroom.

Wow! This is the whole thing!

The National Research Implementation Network (NIRN) Website, and the monograph, Implementation Research: A Synthesis of the Literature, will give you a lot of information. A new book for educators also is available:

Wallace, F., Blase, K., Fixsen, D., & Naoom, S. (2008). Implementing the findings of research: Bridging the gap between knowledge and practice. Washington, DC: Education Research Service.

Now, to your question of effective implementation in education. With the advent of the "evidence-based movement" everyone feels compelled to "have one" at their school or in their state. This has led people to jump in with both feet but with their eyes closed. This is not unique to education, this is happening in all of human services so you will run into this same problem as you try to find partner services for tiers 2 and 3. Stakeholders and partners may say they are using an EBP, but are they? A litmus test is to ask to see their fidelity assessment data for the past year.

Effective implementation means that the innovation is being used as intended (as seen in the results of performance assessments) with demonstrated benefits to students (as seen in data re: the 5 Rs: Reading, wRiting, aRithmetic, Responsibility, and Relationships).

Effective implementation almost always requires changes in schools, districts, and state/ federal policies, procedures, structures, roles, and functions. We should plan for this and not be surprised in Years 2 and 3 when our hard work starts to go down the drain for lack of support from the organizations and systems in which we work. We are all in this together and share a common purpose; we need to coordinate our efforts to achieve improved student outcomes.

You probably can't -- it IS more work at first. Like learning to play the piano, or (god help you) golf, learning to do things differently takes time, effort, and the willingness to be uncertain/ uncomfortable for awhile while you practice. The payoff comes later when you can (with practiced ease) be a more effective teacher for more students more of the time (VERY satisfying!).

RtI and other innovations are, by definition, new ways of doing our work in education. During the teacher selection process you need to be honest with teachers and tell them about the extra effort that will be required and the extra supports they will get from you/ the re-configured school to help them through the awkward stage of implementation.

For implementation, professional development consists of two parts:

1. Sharing information about recent developments in the field, learning about innovations (such as RtI), and working on two-way communication. These sessions serve as preparation for change.
2. Building competence and continual improvement of education knowledge, skills, and abilities (KSA) though selection, training, coaching, and performance assessments in the context of a facilitative administration within the school, district, and state. These sessions are about accomplishing change.

A big problem right now is the lack of implementation capacity -- there are too few knowledgable implementation practitioners in education (or other fields). In education, we have to make a concerted federal-state-local effort to build this capacity in the next 5 years.

I am with your teachers -- I don't like that approach either.

This is such a good question and it relates to all human services as well as education. NCLB has been helpful in many ways but the accountability system leaves much to be desired. One unintended outcome is that educators across the board now have little respect for or appreciation of how a good data system could help.

We agree with Michael Fullan and other system thinkers who say that accountability without capacity development is wasted effort. OK, so now we know that reading scores have not changed for 25 years (accountability). What are we going to do about it - the same old thing, or a well-implemented change that uses reading scores as the criterion for success (capacity development). In capacity development, the accountability for success shifts to the Implementation Team -- the people who are planning the change, organizing resources for the changes, supervising the changes, and rapidly modifying strategies/ tactics to achieve the desired results.

For RtI and most innovations in education, there are no well-established performance assessments to assure the core intervention components are in place and being used competently. Have your teachers go to work on developing one and trying it out in each other's classrooms. That should help with their buy in and help them get over the fear of being accountable at the same time. Once teachers see how helpful the information can be, they will insist of getting the information regularly.

Implementation is a process that has one purpose -- sustainable benefits to students. The problem is that implementation efforts take time and school wide benefits to students are maybe a couple of years away. Thus, you need intermediate measures including:

1. pre-post tests of teacher training -- did the teachers learn the KSA we intended to teach to prepare them for their new RtI functions? Good post test scores should be highly correlated with later teacher performance assessments. 2. coaching feedback -- regular assessments of coaching helpfulness, communication, concern, etc of each coach by the teachers. Good scores for a coach should be highly correlated with later teacher performance assessments. 3. performance assessment information -- a good performance assessment measure is HIGHLY CORRELATED WITH STUDENT OUTCOMES -- if effect, the performance assessment is your reliable predictor of later student outcomes. Instead of waiting 2 years and being disappointed in student outcomes, you can do teacher performance assessments today and provide more coaching and support for those teachers who are not meeting criteria. In essence, this allows you to bring the future into the present and take corrective action.

Having data at all these levels (and more) are part of the Decision Support Data System (one component of the NIRN frameworks) that allows a school/ system to constantly improve education for students by constantly improving the supports for teachers.

This is a good question and an important part of any use of an innovation like RtI. As noted in a previous answer, involving the staff in developing a measure of "RtI competence" is one good way to reduce anxiety. Mostly, the idea is to have teachers more comfortable with having someone in their classroom observing what they do and say. Having frequent visits by a coach helps with this. If not a coach, then other staff can drop in and stay for a class.

The key to comfort is making sure the teacher gets a LOT of positive feedback from the visitor (the 10:1 ratio of descriptive praise to constructive feedback referred to earlier). Who doesn't like compliments? After a while, teachers can't wait for the next visitor to show up!

The best staff performance evaluations are specific to the topic/intervention being used. What content should be presented in what way to help students learn? We talk about context, content, and competence as dimensions of performance evaluation (See "Staff Evaluation"). Above all, the information gleaned from an evaluation must be useful to the teacher, the coach, and the administration of the school to help them improve education practices.

Who should conduct the evaluations? Someone who is a skilled observer, knows how to give conceptual feedback to adults, and knows the innovation (formal and craft knowledge). Like selection interviewers, trainers, coaches, and administrators, evaluators need to know what to do and how to do it competently.

When first beginning to use an innovation, everyone needs feedback more often. The coach is working with the teacher several times a week. Performance evaluations can occur monthly at first (to provide more formal feedback to the teacher and the coach) then taper off to quarterly during the first year. Maybe in Year 2 you only need to do a performance evaluation in October and March for teachers who regularly meet all performance criteria and regularly achieve terrific outcomes for students. The evaluations can be done more frequently with struggling teachers as needed.

A rule of thumb in implementation work is to develop the capacity to do one thing well, then use the capacity you developed to do more things well. For example, struggling schools recover more quickly when they choose one new intervention and implement it successfully in the first year or two, then move on to implementing other innovations. Trying to do too much at once is a losing strategy for most organizations.

So, I would start with the most willing and able teachers in one or two early grades and implement RtI fully by developing the protocols for staff selection interviews, methods for training RtI skills to criterion, methods for coaching teachers in the classroom, good performance assessments, and measures of student outcomes. Then, I would use this implementation capacity to expand to the next grades, then the whole school, then to all the other schools in the district. But, you will never get to the end point without developing the implementation capacity with those first few teachers and students.

Note that the effort to develop implementation capacity (selection, training, coaching, performance assessments, organization changes, system changes) far outweighs the effort to "do RtI" in a few classrooms. However, unless the implementation capacity is developed, RtI will not be expadable or sustainable in the education system. Pay now to see the benefits for all students in the state later on.

A few of the other answers give some ideas about the extent of change required to make effective use of RtI with students. A fundamental question you are asking is, "Is RtI worth doing in our school?" If your core curriculum is working well for 90% of the students, maybe you don't need RtI. Or, maybe you only have a few students in your school and it is only the occasional student that needs extra help so overhauling the curriculum may not be worth it (you have other ways to individualize supports for struggling students).

From an implementation point of view, success begins with a very clear specification of the need for an intervention. There is a big problem we all recognize and we agree that this innovation will help us solve this problem. If you don't have a big problem, all the work of implementation will not be worth it.

If you do decide to proceed, innovations like RtI build on current strengths of individuals and organizations. However, RtI almost always requires changes in how teachers and administrators and other staff do their work and this takes a big commitment (that is why the need for RtI has to be clear up front).

One school official told me that "we have been using RtI for years -- randomly tried interventions." This is just a caution, don't just rename what you have always done and call it RtI. RtI really is a different approach to education and deserves to be done effectively in every school.

Oregon and Virginia have taken about two years to introduce RtI to educators in their states and get buy in to proceed. Other states also have taken a year or more to introduce innovations in other human services, so these time frames may be about right.

As we say, pay now or pay later. You can take the time up front or spend a lot of time later on trying to solve problems with people who feel left out, disrespected, etc., etc.

At a district level or school level the time frames may be shorter given fewer people who need to be contacted and brought into the thinking process.

Parents have a huge stake in the education of their children. The problems faced by parents often get in the way of involvement in school activities/ homework/ etc. There are many families with both parents working more than 40 hours a week (it takes almost 80 hours a week at minimum wage for a family to REACH the poverty line) and there are many social and family obligations that compete with school time.

Most effective methods involve face time with parents so are time consuming for educators. Working through the PTA and parent/family advocacy groups may help since they already are organized and interested. Too often, parents get involved by expressing their dissatisfaction/ concern with changes that already have been made. This is OK -- you can then help them get involved in learning more about the changes you have made and have them join with others in working toward a better way.

With RtI, this will be a critical issue faced by every school. Kids who need tier 2 or 3 services will require more individual attention and more parent involvement. As usual, respect, communication, and sincerity go a long way toward creating functional partnerships with parents and others.

This is a tough one since we are fundamentally opposed to "dabbling" in the use of innovations. If we are going to do it, let's do it so the innovation will be effective and we will have the capacity to sustain the effective use of the innovation to benefit students over the next few decades.

What you can do is to start "creating readiness" for full use of RtI. As stated earlier, clear evidence of need is essential to the Exploration Stage of implementation. You can collect data/ examples to show why RtI is needed. Another key factor is comfort with classroom visits and performance evaluations so these could be initiated even in a rudimentary way.

Finally, the plan-do-study-act (PDSA) cycle (described in the article "Initiate and Manage Change") can be used to begin to establish a problem solving climate in the school. The nice thing about the PDSA cycle is that it can be applied to any problem, big or small, and it incorporates some of the adaptive qualities/ methods inherent in RtI.

Sure, ask the easy question!!

According to Chao (2007), various definitions often are used for change and include:

1. Planned change (a process and a technology aimed at improving the health and performance of an organization),
2. Organizational development (the continuous application of a single or several techniques focused on improvement),
3. Organizational transition (planned change from current state to future state where the future state is reasonably well defined), and,
4. System transformation (planned change from current state to future state where the future state is emerging and is not clearly defined). (see Brief Overview of the Literature Pertaining to Scale Up)

Let's start with the premise that current systems are designed, intentionally or not, to accomplish exactly the outcomes currently achieved. If we want to implement RtI or other innovations to achieve better outcomes, we have to expect and plan for changes in schools, districts, and state/federal education systems. Otherwise, current systems will wear us down with their old rules and regulations designed for something else but not designed for RtI.

System transformation is fundamental change in education systems to more perfectly support the new ways of work in RtI classrooms, schools, and districts. How much change will be required in which parts of the education system? We will not know until we get there.

In our opinion, transformation has no past tense. Once we begin to improve the system we will never be done so we have to build in continual improvement as part of the new system design (e.g. Barber, M., & Fullan, M. (2005). Tri-level development: Putting systems thinking into action. Education Weekly, 24(25), 34-35.)

My work with the State Implementation of Scaling-up Evidence-based Practices Center (SISEP)  is built on these premises.  Visits ScalingUp.org for more information.

Resources always are at issue in education and other human services. In our experience, there is an initial extra cost to develop the implementation capacity to use an innovation effectively (see previous answers). This is mostly due to the fact that you still have to carry out the old methods while you are developing the new methods. When the new methods become the standard, costs usually go down, often with an overall savings.

This is why we need system transformation -- systems need to invest in helping schools and districts develop implementation capacity. From a system perspective, the Year 2 or 3 savings in fully implemented schools can pay for the extra Year 1 costs in schools just starting down the implementation path. The net of this equation is zero.

I wish we had some other word to use -- "training" has come to be so over used it has lost its meaning.

Training as part of implementation is all about function (see the Preservice Training article on the National Implementation Research Network (NIRN)).

• Good training is done by trainers who know the innovation very well (ideally, ex-practitioners who have used the innovation themselves) and have good adult teaching skills (conceptual teaching methods, conceptual feedback skills, lots of great examples, a plethora of rationales)
• Good training includes a clear description of the intervention, its methods, the philosophy and values that underlie it, and the data that support it.
• Good training has demonstrations of the intervention (live or on tape) to exemplify the intervention.
• Good training has behavior rehearsals with feedback until competence is reached on fundamental skill components.
• Good training always has pre-post tests of knowledge and skills to make sure the trainees have actually learned what we hope to teach.
• Good training is revised frequently based on feedback from coaches and data from staff performance assessments.

We often say, if you do not have pre-post tests of skills, you do not have training. All you have is attendance.

This is a little harsh but we have to get out of the rut somehow.

I like the way you think!

Ideally, the state/district would have Regional Implementation Teams to help you (see the concept papaer I co-authored: Developing the Capacity for Scaling Up the Effective Use of Evidence-Based Programs in State Departments of Education). These would be people skilled at implementation (stages, drivers), organization change, system transformation, and continual improvement.

Short of that, you will need staff time to form/ develop your own mini-version of an Implementation Team (at least 2 full time people for at least 18 months) to establish the implementation capacity to plan, arrange, and do teacher/staff selection, training, coaching, performance assessments, and school changes. These folks will have to be quick on their feet and learn implementation skills on the fly (a lot of tolerance for ambiguity and failure).

You also will need the full support of the district/state leadership and their promise to make changes in their domains to support your work in the school. I would ask them to commit to at least one meeting per month at your school to hear about your progress and to hear about how your experiences at the practice level can inform changes in their policies, procedures, structures, roles, and functions within the district/ state. Often, state folks want to just send you a check; when it comes to implementation you need their hearts and minds more than their money.

See the answer to Claudith Thompson's question. It is more work in the beginning but it is also energizing to make changes and see the positive effects.

There is no easy way to implement research in practice settings. This is true in education, human services, business, manufacturing, medicine, architecture, and engineering. Implementation is the "to" in science to service or research to practice. Implementation is the science and practice of using science in practice.

In education, the problem is not with the research (well, maybe a little). The problem is with our lack of capacity to implement the research in classrooms and schools with real benefits to students. Over the years we have developed system capacity to do financial accounting, student transportation, and information technology (to name a few). Soon, we will develop the capacity to implement scientifically based innovations reliably and effectively across the nation. In the meantime, we will struggle making effective use of the products of science.

The responses to DG and others describes the Exploration Stage of implementation, how to get buy in for the idea, and how to select the first group of teachers.

Successful interventions have data to support them or have an excellent foundation in the experience of teachers or staff. We are not hung up on the exact definition of an "evidence-based intervention."

From an implementation point of view, a good intervention is one that is:

1. Clearly described (what it is/ what it is not)
2. Operationalized (what you do and say when doing the intervention and is teachable to others)
3. Contextualized (what does the host organization need to do to support the effective use of the intervention) and has a well established:
4. Performance assessment (a practical measure of competence of those purported to be using the intervention)

A lot of well-researched interventions do not have these qualities so they are very difficult to implement. Many practice-based good ideas do have these qualities and can be implemented more readily. Successful implementation does not depend upon randomized group designs or meta-analyses; it does depend on clarity re: the intervention and how to assess its uses. (See the article "Core Implementation Components") for a discussion of "core components."

I am glad you asked this question since I have wondered why we have three tiers in which to place an infinite variety of students.

RtI is an example of an "adaptive technology" -- there are other examples in other fields (Google it and you will find some). It is a way to individualize interventions within the scope of a standard way of deciding who needs what and when they need it. Adaptive technologies (RtI) hold the practitioner (teacher) accountable for assuring the success of each consumer (student). If the student is not successful using the current curriculum/ teaching methods it is up to the teacher to choose another and measure the effects for the student. This process continues until success is achieved.

So, if a student is struggling, what is a teacher to do? What are the available options? This is a huge issue in any school system and involves our colleagues in mental health, child welfare, substance abuse, health, etc etc. Will we accept whatever is available or will we help assure high quality services within the school and within the community?

Our colleagues who operate short term, homebased interventions for children and families run into this same problem. They are most successful in well run service environments where they can "hand off" families to effective options.

Good question! RtI is all about being more flexible and changing things as needed to better educate all students. I can only imagine that the first thing a teacher might do is re-organize the classroom itself to allow more small-group instruction for those students who need it.

I think this is where measurement comes in to play in a variety of ways. Data help teachers make a decision to do something differently and data help the teacher see that the different thing is working or not working. Most teachers really like seeing positive results from their changes so they don't mind doing the small groups. Yes, it is one more thing to do, but it is done with a clear purpose in mind and with clear benefits to students. That turns out to be more satisfying and less of a hassle.

Teachers are in the business of helping kids learn -- this is a way to demonstrate good outcomes and feed their passion for teaching.

I am not all that familiar with the blueprint documents but I have heard from RtI folks that they are very useful. Some of the information in the Get Started section of the RTI Action Network Website also provides guidance for getting started. The Concerns Based Adoption Model also provides good questions to consider during the start up phase.

From an implementation point of view, the key things we look for are a clear specification of an important problem/ need that needs to be resolved, fully informed buy in by all stakeholders, and a clear plan of action for how to proceed when you actually begin the implementation process. You can look at the National Implementation Research Network (NIRN) Website for more information on the "Exploration Stage" article as well.

This is a good question -- we often talk about the "first right steps" that help produce later successes. When groups come together for the first time (especially when you already know each other in other contexts), it is important to develop the terms of reference for the group -- what is the purpose, how will you conduct your business, how will discussions occur, how will you resolve conflicts, etc etc (Google "terms of reference" and you will find good info -- this is common practice in Canada).

Getting off to a good start is especially important for RtI where you will be reviewing and interpreting data and making recommendations for changes in teacher behavior and school supports for teachers. As always, you want to build on the strengths of your current group and current relationships among members but orient the group to the new purpose and methods. Good leadership at the school level and at these early meetings is essential to arriving at concensus and being productive.

The scenario you have described is all too typical of efforts to implement innovations of any kind. Lots of struggles as folks try to learn about implementation.

Our strong suggestion is to bring district personnel into the mix at an early stage and have them become acutely aware of the struggles you are having at the practice level. Policy makers and district/ state leaders get very little systematic feedback on the results of their policy making/ decision making so this would be a way of informing their decisions in the future (see Scalingup.org for a more detailed discussion of the PEP-PIP cycle).

This is a great issue. We have found that we go to a lot of trouble to generate good data only to find that many people do not know how to understand or interpret it. The School-Wide PBS folks (http://www.pbis.org/main.htm) are now building special training to help administrators and others know what to do with data.

RtI implementers will need to pay attention to this issue and make sure they are there to help model how to use data to make good decisions about kids and to see the impact of student learning.

This is what developing implementation capacity is all about. Implementation Teams have the knowledge and skills to fit implementation activities to the building and staff, building on unique combinations of strengths and finding ways to fill the gaps. In the midst of this, the core intervention components must be preserved to achieve the desired student benefits. Go to ScalingUp.org for more information on how to increase capacity and achieve useful scale for any innovation.

You are most welcome. I have enjoyed this whole process!
Dean