PBI READY FOR THE BIG TIME

Recent articles on the Catalpa web site make a strong case that the PBI school visit is ready to take on a national role in the reform of American schools.

Visits generate legitimate findings about what actually goes on in the daily practice of teaching and learning in a school. The accuracy and richness of these findings supplement test score information so that states and districts are able to generate and support school-based plans that have a real chance of making a positivie difference.

Go here for more on why PBI is ready for the big time.

Go here for the basics of PBI.

The question for you:

Would PBI strengthen Obama's plans for reforming schools?

UP-DATE ON SALT

Go here for the new Beacon story.

Go here for SALT background.

Go here for the Taste of SALT, the recent story on SALT's future.

Question for you:

What should Rhode Island's new commissioner, Deborah Gist, learn about SALT? Should she continue the Department's strong support for SALT and work to strengthen it even further?

THE TASTE OF SALT

The SALT visit has been a key part of my work for the last 12 years. This is the Rhode Island Department of Education’s (RIDE) bold project to develop better and more effective school accountability. Establishing rigorous SALT school visits as part of the state’s comprehensive school accountability system has been one important crucible for defining and testing the principles of Practice-Based Inquiry.®

Rhode Island is “taking stock” of the value of the SALT visit. This is the time for you to say what you think. Your participation in this conversation could very well influence what is decided about whether the SALT visit survives the Rhode Island state budget crisis and what it will become in the future.

For those who have had no experience with SALT, the SALT visit provides the best example of Practice-Based Inquiry® at work. The SALT visit raises questions about what would work best in your situation.

I thought these conversations would better enhance our thinking if I conducted two conversations at once —one with Rhode Islanders who know the SALT visit and another with folks who are interested in the PBI® school visit but have had little experience with SALT.

You may want to first read about the SALT school visit on the Catalpa web-site. Specific links are:

For a description of SALT
For a summary of the survey results
For the 2005 complete report of the survey
For the news about SALT’s future

To stimulate these conversation, I am providing two different sets of questions, each one followed by comment space. Respond freely to the lead questions or to what someone else has written. I will join in now and then, as well as make sure the conversation remains civil.

Tom Wilson


What do you think about using the school visit as a means of assessing your school(s)?
(Note: These questions are for those who are not familiar with Practice-Based Inquiry or the SALT visit.)

1. What do you think is missing from how the performance of your schools is judged?

2. Does it overlook the value of using the rigorous judgments of teachers? Should that change? What difference would that make?

3. What do you find most appealing about the school visit? Do you think it would work in your situation?

4. What more would like to know about SALT and/or Practice-Based Inquiry®?

What has been the value of SALT visits for Rhode Island?
(Note: These questions are clearly aimed at Rhode Islanders, but no one is excluded.)

1. If you have served on a SALT visit team, what did you learn from that experience that made a difference to how well you teach or administer a school?

2. If there has been a SALT visit or a Practice-Based Inquiry® visit at the school where you work, what difference did the visit report make for your school?

3. If there was a SALT visit this year (2008-09) in the school where you work, did you notice any differences in either the visit or the report? How did the visit and the visit report compare to the last visit and report?

4. What do you think a SALT chair should tell a team about how to write a finding about a school that the team thinks the school will not like?

5. How much do you agree or disagree with the principal who said, “SALT is one of the best things RIDE has done to support good teaching and learning in RI schools?” Why?

HOW PRACTICE-BASED INQUIRY® SOLVES THE PROBLEM OF HAVING ONLY A HAMMER

If a hammer is your only tool, every problem starts looking like a nail.

--Abraham Maslow


A TIME TO THINK AND ACT ANEW

As 2008 comes to a close, we know that things must change if we are to thrive or perhaps even if we are to survive. The presidential election gave us and much of the world an exhilarating dose of hope that we will turn our frustration about the systemic dysfunctions of so many of our public institutions into creative, effective and democratic change. The possibility that we can do that and the promise that holds for the future of the world is exhilarating.

Those of us who work for and with American schools have an advantage over those who work to improve other institutions. During the last 30 years, we have made a deliberate, national and historic effort to bring change to public schools. We know that our efforts have not resulted in enough better learning for students. Our advantage now is that instead of asking, “How do we change schools?” we have the basis to ask, “Why has successful school change been so hard to achieve?”

How we answer this question will shape our course. We could use the history of the last 30 years as evidence that nothing new will make much difference. We could blame people and institutions. Then we would watch from the sidelines as another period of innovation takes place. We could say, “Whatever” - and just go along with whatever new program is put forward. Or we could decide that our assumptions and hypotheses about school change really do make a difference to the results we achieve and that we may need to change them to be effective. This essay probes this possibility.

One of the tenets of the recent reform effort is that change in education must be driven by “data.” This has brought about more harm than good in how we think about what we need to know and do to be effective in improving student learning in schools.

The problem is not that we hold competing or confused views of what we want our schools to provide. Most of us agree about what we want. We want our schools to be places where our children actually learn. We want that learning to be deep, broad, skilled and useful. We want our schools to be safe and good places for our children “to come to.” We want our kids and our modern democracy to thrive.

The problem is not simply with the data we use. We know a lot more about our schools than we used to know. The increase in data about school performance is exponential. We have improved the accuracy of how we collect data, including reaching greater agreement about definitions of terms - such as “the drop-out rate” - and creating more sophisticated statistical tools to make comparisons and to draw out causes.

The problem with saying “data must drive change” is that we do not gather data about what must change – what actually happens in schools that causes students to learn or not to learn. We define data as a number we can precisely measure. This definition excludes the evidence we could collect to learn what actually happens in schools when learning does or does not happen. We excuse this critical oversight by saying, “Well, that is too complex.” It is too complex for our assumptions about the nature of evidence. If we had legitimate evidence about school practice, we would have knowledge to anchor the test data in the reality of the action that causes learning. That would allow us to broaden our working definition of learning so it would be closer to what we want it to be. That would provide a more powerful basis for designing or changing action that would improve learning.

THE PROBLEM: “WHAT DID YOU LEARN IN SCHOOL TODAY?”

When we ask our children, “What did you learn in school today?” they may not answer directly, but at least this simple question is on course. We seldom ask that question when we assess school performance. When we think about how America can improve the learning its schools offers, we don’t know or talk about what actually happens daily in the classrooms. If we knew that, we would find a way to talk about that so it would make public policy more productive and intervention strategies more direct, local and effective.

What we measure is not connected to the reality of either what schools are or what they ought to be. What we know about how learning happens in schools is too often under-theorized, leaving us in a vacuum of generalized concepts on which to base new action. Because the data is disconnected from the reality of learning, it limits our ability to determine what programs will lead to the changes we want. We fill the vacuum with “data driven” plans. We substitute the operational variables that are designed to gather data for the ideas we want to test. We create words to describe what teachers should do. This new jargon is not built on concrete knowledge but on abstract variables.

Generalizations and jargon about teaching fill the vacuum rather than provide good ideas. The made-up words take on a life of their own and become more real than real. Since this language about schools is invented, it does not evolve the way real language evolves; we must teach it to teachers in professional development workshops. Professional development consultants say that teachers, who attend their sessions will “internalize” the new words and the results of the data, and then teach in a more scientific way and be better rewarded than those who did not attend. In reality, while the language they use to talk about teaching may change, their teaching does not change much.

Teaching is not about thoughtfully improving one’s practice but about "talking the talk" and “delivering the curriculum.” Conclusions about how to improve teaching become a muddle of jargon-filled generalizations or simple operational variables that are associated in this way or that with other operational variables. Recommendations for action too often yield pretentious program names and more jargon, which struggles to be more precise than it is and which covers up, rather than explains, competing meanings. Much of it is unreal, pretentious and even silly. Meanwhile the realities of teaching, the nature of practice and the importance of each teacher’s practice to learning outcomes just sit there. And as a result, the hoped for improvement in learning in schools doesn’t happen.

Managers and policy makers commonly call for change by convening conferences to “take stock,” seek a new beginning, generate a new definition of education in the 21st Century, or build new strategic plans at one level or another of America’s education enterprise. Advocates of this or that idea declare that the “door of opportunity” is ajar, and then push old ideas decked out in new clothes onto the public runway. Those of us who have been through it before know that there will be a new set of words, a new shuffling of the deck chairs. We also know that the problem of having only one tool, such as a hammer, is likely again to exclude the consideration of the reality of the practice of school learning. Another scenario for improvement will be pushed that is unlikely to have any real effect on the reality of practice, since we are unlikely to affect something we ignore.

BRINGING PROFESSIONAL PRACTICE INTO FOCUS

We lack explanations or descriptions of what the daily practice of learning in schools actually is. Yet, most of us would agree that thousands of pieces of daily human behavior, words and actions determine whether a student learns or not during a particular day. Since we don’t believe we can measure that, we find it is better to ignore it. Daily school practice is not often in the scope of what we think educators should know about, and educators would agree.

Since we ignore the most important professional contribution an individual teacher does or does not make, it is easy to think that what matters most is what policy makers, researchers, and supervisors do. Teachers do not need to think very hard about what they do, but to “deliver the curriculum.” They are the object rather than the subject of most of our research and the target rather than the author of much of our policy.

If we think anew, we will acknowledge teachers not only for being good, devoted adults, but as professional practitioners whose work, knowledge and judgment play the most important role in how well schools perform in providing solid learning for their students. It is important we acknowledge that it is through the hard work of continually sharpening their skills so students will learn that they show real devotion, rather than simply asserting that they are devoted. We can then see why a teacher’s professional skills are as important as the skills of lawyers and doctors. What are those skills?

The good practicing professional knows how to learn from her experience about how to do her profession well. This knowledge about practice is learned only from the actual experience of a practitioner, in this case, a teacher teaching. It is not learned in a class on “differentiating instruction.” It is the knowledge the practitioner uses to connect the formal knowledge she gains from professional development to her decisions about what action to take at a given time with given students. Many of these decisions are small, and the teacher makes them “on her feet” doing her practice. Practice-based knowledge is what distinguishes a practitioner from a researcher or a knowledge worker. The quality of practice-based knowledge helps distinguish a good professional practitioner from a mediocre one.

A practitioner builds this knowledge from making thousands of specific judgments about what to do to match the particularities of the specific person she is teaching and the particularities of what she is teaching. She judges the rightness of her decisions in large part by the immediate and delayed response of the student. From these judgments, she builds a body of knowledge about what works for her when and what the consequences are. Researchers do not model the way to gain this knowledge of the specific and particular. Researchers work to measure a variable in order to test a hypothesis and be able to describe the probability of different possible outcomes when one variable is influencing another.

To teach mindfully - not relying on habit - a practitioner must use her professional judgment. Professional judgment is not a mushy, idiosyncratic feeling; it is a set of cognitive skills that inform a practitioner’s decisions about what is the best action to take at a particular moment. While informed by a moral sense and the teacher’s own set of actual standards for student performance, it is not a set of rules. It is the teachers’ tool to deal with an unpredictable set of possibilities in the student, the social context and the substance of the lesson. It can consider both the immediate context the teacher faces and the short and long-term goals she hopes she and the student will reach. It allows her to have a basis to act, even when the knowledge she has does not exactly cover the immediate situation. It is never perfect. It is the best she has, and the experience she has today will improve the action she takes tomorrow.

Effective practitioners know how to learn from particular incidents and evidence. They know how to adjust their thinking and their action to match what they have learned. They know that the right judgment about the best action to take with one person is not necessarily the right action to take with another. The right action today is not necessarily the right action tomorrow.
While we have emphasized teachers, the construct of “a school’s learning practice” includes students as well. (That is another essay.)

Another important implication is the design of effective school accountability schemes. The low effectiveness of most accountability systems perpetuated over the last 30 years can be explained by their systemic disregard for the importance of professional practice in school learning. Teachers are much more likely to resist the value of accountability, when it holds them accountable for something they do not or cannot control. But, if they are held accountable and supported for improving their actual practice, accountability becomes, fair, relevant and a useful tool for change.

Teachers listening to a SALT school visit report will listen, study and “take on board” the team’s conclusions about the school’s actual learning practice. If they sense “meanness” in the tone of the conclusions, they may become defensive. Further, since the writing is directly about the school’s practice, teachers do not need complex planning tools designed to make inappropriate information more appropriate through scaffolding and other conceptual schemes to fix the problem.

When practicing teachers shape the evidence, as they do on a PBI visit team, the likelihood of a productive school response is increased, even if the message is that the school is facing major difficulties in how well it teaches.

Daily practice is the work of teachers. We pay them to make judgments and perform actions every day. They know that.

The Catalpa website has more on the meaning of practice, including examples from teaching and other professions.

The reasons we avoid practice-based knowledge are:

• What we know about practice doesn’t match the conventional, limited thinking about research methods in education. Educational research is unable to deal with the immediacy, complexity and specificity of the nature of practice.

• What we know about practice is far behind similar thinking in other professions, including medicine.

• In what seems to be an intellectual throwback to the 1920s, the prevailing assumption among high level supervisors and data professionals in school offices and legislatures is that, since it can’t be measured by our conventional means of measurement, it shouldn’t be measured. It also means, particularly in this era of “data driven” decision making, that we should not even discuss it when we discuss what needs to happen to improve school learning.

This leads to three outcomes. One, we substitute artifacts that we can measure - student test scores - for actual phenomena. Two, we ignore or renounce the phenomena of actual practice because we think it is mushy, soft or imaginary. Three, we separate what policy people learn about what make schools work from the reality of the nature of the work that teachers do.

It is time to think anew. This is not an argument that we stop working to find more precise definitions of operational variables and more sophisticated statistical analyses of their relationships to improve our current data gathering operations. That change will add quality to data and make the analysis of that data more precise, but that is not thinking anew about research and how the shape of our success in improving schools is partly a function of how we gather, analyze and use data about schools. True change involves fixing the flaw in research that limits how we think about schools and school change.

We must become more scientific about how we study schools and more practical about what knowledge is important. It is no longer smart to claim that conventional research methodologies are all that we have for generating legitimate conclusions about issues we care about. It is no longer good science to force the phenomena we want to study into a box that fits what the methodology can study. The false precision we create, when we create a world we can measure with limited tools, is not in the service of inquiry, truth or good education. It is no longer smart to argue that this limitation is fine, since all that we need to know is what school inputs are associated with what learning outputs.

We should begin with the phenomena we want to know more about - learning practice in schools. We should turn to figure out how to measure that as well as we can, not by using ‘only a hammer” but by using all of the tools that are available. Like a good scientist, we must begin with what we want to know about in the real world and then find a way to learn more about how it actually works and what causes it to work well. As our knowledge develops, our methodologies for understanding it will unfold and our understanding of the phenomenon will become ever more precise. And this time, our knowledge will be about something that is real about how schools work, rather than something about a conceptual artifact that serves a research methodology and that requires made-up words and concepts.

Beginning of Digression

While it would be right to argue that introducing Practice-Based Inquiry® as a methodology at this point would be self-serving, I want to ask indulgence on that. After 12 years of fully absorbing and often lonely work on Practice-Based Inquiry, I am convinced that this is a solid and sensible tool to use to promote real reform in school learning. That is up against 25 years I have spent in leadership and research positions in several national programs to reform schools. It is true that I need to promote Practice-Based Inquiry to earn my living. But it is also true that I am convinced that this tool promotes a way to think about reform that is productive because it is based on the reality of actual school practice, which is the phenomenum that determines whether students learn. It uses a different methodology to understand the complexity and particularity of good professional practice.

It is true that many misunderstand how Practice-Based Inquiry works as a rigorous research methodology. They think its emphasis is misplaced and superfluous. This misunderstanding leads to conducting visits (or audit procedures) that do not realize its full potential as an instrument of school improvement. Again and again, many different visit processes do not consider in a disciplined way how in their social interactions a team of practitioners can construct accurate and legitimate conclusions. I want this essay to minimize these misunderstandings.

The last section of this essay on Practice-Based Inquiry doesn’t fully explain Practice-Based Inquiry. That is the work of the web site. Hopefully, it will give you the background to follow the links and an argument that is strong enough to motivate you.

Tom Wilson

End of Digression

PRACTICE-BASED INQUIRY®

A research strategy for generating findings that are certain about the quality of professional practice presents enormous methodological problems. Yet, its potential value is so great that for more than a 100 years different agencies around the world have worked to create such a strategy. Agencies as diverse as school systems, American universities, Her Majesty’s Inspectorate in England, and accrediting associations for schools, medicine and other professions in the United States have been part of this effort.

The professional peer visit is one important solution. A team of professionals visits an institution, examines the practice of its peers and writes a report that sums up its judgment about how well practice is delivered at the institution in question. This is consistent with the common-sense dictum, “If you want to know how well live human action is really going, go see it.” That applies to the live action resulting from the actual practice of professionals.

If you want to know if a school is doing all it can do to help your child learn, visit it, see what is happening, talk to the teachers, and decide if it is good enough. What is going on is so complex, so particular and so important that the only way to learn about it is by going there. What happens every day is what most directly determines whether and if a student is learning and what the school is doing to make learning happen. If you want to be certain that what you conclude is right, go with a team and conduct a rigorous inquiry. If you want to learn whether students are learning well, teachers are teaching well and the school is doing all it can to support good teaching and learning, make those the central questions of your inquiry. Use direct, active language in talking and writing about it. Do that rigorously, and maybe we will learn enough so that one day we will move to a more precise level of knowledge that requires a more precise methodology.

The use of a professional visit to learn about how well practice is taking place has a long history and tradition. While some historians trace the tradition of professional visits to the Middle Ages (and some to even the beginning of the Christian era), the visit has been used extensively during the last 150 years by more modern institutions such as public schools and hospitals.

Practice-Based Inquiry® is based in this historic tradition. It is an inquiry technology for judging professional practice. It consists of a set of principles, concepts and procedures that transform the traditional professional peer visit into a research approach that sets new standards for rigor. It gives the traditional professional visit the rigor it needs to meet modern standards for legitimacy.

It provides the design and conduct of professional peer visits with the necessary conceptual underpinning so that the findings of visiting teams are accurate, useful and legitimate. It is a powerful tool for improving professional practice in schools and other institutions that rely on the services of professional practitioners.

It generates valid conclusions with specialized skill. And, in addition, the assumptions it makes about professional judgment, evidence, consensus and research legitimacy provoke productive change in how we think we can make our schools more effective learning institutions.

Its roots in the centrality of school practice requires us to think anew, and its use will help ensure that school change comes to mean improvement in student learning.

YOUR TURN

You are welcome to make any reasonably civil comments about PBI and this essay. Here are some pertinent questions:

• Do you think that the experience educators have had with reform during the last 30 years gives us an advantage? Why or why not?

• How important do you think the problem is having only a hammer? Does PBI provide a good solution?

• What are your comments about PBI® and its value in facing the current challenges of American education?

• How much chance do you think the Obama administration has to improve actual student learning in our public schools?

If a new comment form is not showing, select comments.


Practice-Based Inquiry^® is a registered trademark of Catalpa Ltd. Catalpa™ & In-School Visit™ are trademarks of Catalpa Ltd. All content © copyright 2008-9, Catalpa Ltd.