Early in Sir Arthur Conan Doyle’s 1891 short story A Scandal in Bohemia, Sherlock Holmes gives Watson a lecture on the difference between seeing and observing. To test how well Watson understood, Holmes hands him an unaddressed, undated, anonymous letter that had just arrived on the doormat. It announced that an unnamed visitor would shortly arrive to consult Holmes, perhaps wearing a mask. Watson read it, declared it to be mysterious, and asked Holmes what it meant. The great detective replied: I have no data yet. It is a capital mistake to theorize before one has data. Insensibly, one begins to twist facts to suit theories instead of theories to suit facts.
Of course all mediators apply this wisdom when they start and progress a case. Questioning, listening, noting, delving, querying, testing, clarifying, re-questioning, never assuming. Data provides the power that enables a mediator to assist the parties toward a mutually acceptable outcome. But does it stop there? When it comes to skills and techniques, could the data that is currently available be greatly improved?
The current body of mediation research data that underpins mediators’ skills and techniques is highly fragmented. Most has been conducted on a small national or local scale by individual provider and academic institutions. The data is often based on statistics, surveys and class exercises with available participants, such as trainees and students. These controlled laboratory studies by scholars and service providers are typically cost-efficient and fast, enabling relatively easy analysis. They have been crucial in the early development of mediation. However, as commentators have noted, the classroom behavior of students and trainees, not to mention surveys, can differ markedly from action in the real world[1]See the meta-analysis of 490 articles on business negotiation between 1995 and 2015 in Two Decades of Business Negotiation Research: an overview and suggestions for future studies by Prof. Henrik … Continue reading, challenging the robustness of the data.
The more accurate, but also more difficult, way to conduct research is out in the field, in live action. Field research is largely based on actual observations of the live action by skilled researchers. The quality of the results, once analyzed, is more credible than from lab studies. Unlike lab work with students and trainees, researchers in the field cannot establish, control and manipulate the factors and variables that come into play. This means that the results need to be assessed differently to establish a true correlation between certain mediation techniques and specific outcomes. There has been little real field research in mediation because it is costly, complicated and parties need to allow the presence of a researcher.
But as we start a New Year, soon a new decade, let’s dare to be a bit adventurous.
What if a vast range of mediation skills and techniques could be radically improved by new data derived from large-scale national and international field research?
What if mediation field research could happen in a coordinated manner, maintaining a critical level of universal definitions that allows comparison of findings, thus avoiding duplication of effort and cost, accelerating the generation of a truly credible global body of “mediation science”, overcoming the management and cost considerations that currently spook this prospect?
What if data from synchronized field research really could be made freely available online to assist mediators select the processes and tools that would be most likely to benefit a wide range of circumstances – instead of having to rely only on a haphazard combination of training, instinct, experience, experimentation and habit?
What if, to give one example of potentially hundreds, field research in a particular situation could determine the statistical probability of a certain style of mediation, or a particular hybrid process, being more or less likely to meet the needs of the parties? Or, to give another example, what if field research could establish the most likely successful approach, process or questioning sequence where the parties have different cultural backgrounds?
What if field research could establish the optimal time to mediate in certain situations?
What if the financial value of a conventional mediation, or a hybrid process like Arb-Med-Arb, or other things that are known or assumed to be significant but still remain largely unproven, could be credibly established by fieldwork data?
What if the costs of large-scale coordinated field research could be funded by grant-giving institutions, governments and the prime beneficiaries of such research who would be motivated to allow the presence of researchers to observe mediations?
These may be rhetorical questions, but we should not dismiss them as fanciful. If more targeted field research really would generate great new value for parties and mediators, and therefore absolutely needs to be done, why not just get on and do it?
Should a conversation now start about the value and do-ability of these propositions?
In the 1970s and 80s, researcher Neil Rackham devised and led the largest ever field-based study into successful techniques in the art of selling. His work focused on questioning techniques. It took 12 years and cost the current equivalent of $30 million. Rackham’s books derived from his field study data, such as SPIN Selling, are among the most sold sales books ever. In terms of more and better sales of products and services worldwide, the value generated by the field studies of Rackham and his associates is inestimable. His work is also applicable to negotiators and mediators[2]See: What Negotiators Can Learn from Modern Sales Theory by Ava J. Abramowitz (practicing mediator), Volume 1, Chapter 31, The Negotiator’s Desk Reference (2017) ed. Chris Honeyman and Andrea … Continue reading.
Has the mediation world spent too long developing lab-based facts to suit its theories?
Might it start to hone new theories of mediation excellence from field-based facts?
Please take a moment to provide your thoughts by way of comment on this post. If you prefer to comment in private, my email address is on my website. If, as I hope, this subject stirs a reaction, I will try to summarize the range of opinions with a further post.
________________________
To make sure you do not miss out on regular updates from the Kluwer Mediation Blog, please subscribe here.
References
Hi Michael
Your article raises a number of important issues that has bedevilled the modern mediation profession for some time. Principally, what makes a mediator.
If I understand you correctly you hope that real-world research will find the magic bullet/ the key that unlocks the door to “Data” that ??? Not sure what then follows. What does this data do? Is it a magic formula that can be applied or is it a lightning rod of inspiration to be pased and analysed.
A number of Australian academics have proposed similar views in trying to equate measurement with understanding. It is a very Newtonian form of thinking.
I have written a critique on this Newtonian approach pointing out that in the complexity of human relationships, within which mediators operate, data is never repeatable. This has been highlighted by the replication movement which has thrown into doubt the validity of the much of the so called evidence-based research and observational case studies carried out in the social science field. This is a great problem for theorists and academics and for your argument. See https://www.mediate.com/articles/rooney-mediation-mediator.cfm
In my view there is a better answer to what makes a mediator and it comes from complexity theory. I have drawn a lot of inspiration and understanding from Prof Dave Snowden’s work on his Cynefin approach see https://www.liv.asn.au/Staying-Informed/LIJ/LIJ/December-2017/A-new-approach
Channelling Sir Arthur Conan Doyle practising mediators especially those using the facilitative approach operate best by allowing data to emerge through the process of probing first and then responding :-
“They will probe first and then sense and respond to the reaction and they will try and suspend any attachment to their memories, desires and the need to understand what is happening and will try and not be deterred by blockages and impasses. They will allow their intuition to guide them through the session rather than letting the mechanical side of their brain be the master”. (From the above mediate.com article).
Thank you Michael for starting this conversation. It is an important one at this time as we slowly see the emergence of mediation as a fully recognised mediation profession.
Greg – thank you. The comment by Paul Lurie expresses my view better than I can myself. Real-world data is likely to be superior in quality and more useful in practice, and, in my view, more in the complexity than the linear sphere (my thinking has never been described as Newtonian before!). Data generation from expert observation of a large number of live mediations involving vastly different circumstances, issues, parties, mediators, cultures, etc inevitably provides data that may validate some established practices but, more interestingly, generate new techniques, styles and approaches that typically work better in one situation or another. That was the idea I was trying to advance, and if the brevity of the post failed to capture that, do forgive me! By the way, there are few magic bullets – you describe an improved data bank well, as “a lightening rod of inspiration” to which I would add – credibility. Michael
Totally agree. My own field observations in Egypt (from talking with the handful of practicing mediators) are very interesting and would merit further and more robust research as mediation develops. Although most trained mediators were strictly trained in the facilitative style, and were genuinely convinced by its merits, practice revealed that parties insist on an evaluative style to which most mediators did oblige. Gathering more information and testing some approaches (like taking more time in explaining different techniques rather than quickly satisfy the parties in evaluating) or designing new training (for example, including more evaluative modules in future training to cater for market needs). Also, new training should be informed by the types of questions mediators ask for example — from discussions with peers in Lebanon, some have taken the initiative of providing different types of questions that work better within the local context.
In the context of an Arab initiative between different arbitration/mediation centers, collecting empirical evidence is actually one of our action plan items to ensure that while we try to follow international developments, we provide a product that is recognizable in the region.
To conclude, I find the possibilities endless with the type of information that can be gleaned from such exercise and totally support it.
I agree. The best techniques of mediation can be identified from reliable data and become the basis for techniques for mediators to use. A good example is in the medical industry. Doctors use surgical techniques based on successful outcomes reducing illness based on data published in peer review journals. Six years ago we created the Guided Choice Mediation project for exactly this reason. Our working hypothesis is that users consider the best mediators can cure the disease of an unresolved dispute earlier and with less expense. Mediation research that does exist tends to focus on just settlements and not when they occur. The Global Pound Conference supplied data indicating the correctness of our thesis. We are now gathering and teaching techniques that provide potential cures that can be verified by data. Further research will confirm that the optimal time to hire a mediator is as soon as possible, even if the parties are not ready to start a traditional mediation involving settlment negotiations. During the pre-negotiation phase, the mediator acts as a process faciliator helping to confidentially diagnose the causes of impasse and suggesting the most efficient settlement techniques.
Paul – thank you for this and the medical analogy. Please see my comment to Greg, above. Guided Choice is a great contribution to the mediation field, and thanks for mentioning the GPC data – which, in my interpretation, indirectly support the need for field studies. Michael
Michael! I could have written this posting myself—except that I wouldn’t have done so eloquent and thorough a job. You read my mind! I am about to take over as editor of an online ADR journal and one of the first topics I’m hoping to explore is the paucity of research in the field. Why are mediators so convinced that what they do is effective or appropriate? How can they be so confident about their work when they rarely ask the parties or keep track of their responses? Is it beyond the capabilities of those who work in ADR to devise experiments that test pet theories, or to muster funding? Or are we too complacent?
I’ve been in the field since 1989 and long ago realized that mediation is essentially folk wisdom; individual mediators treat their personal philosophies, habits, and experiences as universal truths. Even if they were inclined to take research into account, they don’t know what exists or where to find it.
Last year I helped a colleague teach settlement skills. A student asked, “What does the research show about the impact of what we wear?” My colleague responded with generalities of how parties respond, based on her own experience. The more correct answer was “There is no such research.”
Recently, I had the rare opportunity to be a party in a human rights mediation where I was the complainant. I asked a colleague who had done many human rights mediations to come with me as a support person. My colleague was appalled that the mediator did not put me and the respondent in the same room and make us talk face-to-face. Me? I was absolutely at ease with the model that the mediator used. Moreover, I was delighted with the outcome. The settlement gave me two private-interest remedies (including money) and three or four public-interest remedies. But my colleague deemed the process to have been wrong and, worse, unethical. Which one of us is correct? Why?
A conversation about the value and do-ability of your research propositions is long overdue.
Genevieve – Your support for the idea is important and greatly appreciated and the examples are pertinent. It’s not just that more research is needed, but it also needs to be the most relevant, the highest quality and the most “buyable” by mediators and parties. Thank you! Michael
As ever, an interesting challenge Michael! The key question though is anticipated by you: “There has been little real field research in mediation because it is costly, complicated and parties need to allow the presence of a researcher”. The question is How? What are your specific proposals for conducting this research? Then “What ifs” might become easier to respond to! And what if, unlike selling, mediation is not amenable to this kind of scientific inquiry? Mind you, I suspect Paul is suggesting that they are.
John – thank you. All the “What iffs” are things that, in my humble view, are achievable given (1) recognition of the need (and therefore unlock the funding from those that recognise the need – eg companies and governments) and (2) the skills and determination to go out there and generate the data in the right way using the right resources. I mentioned Neil Rackham in my post for a reason – this extraordinary individual has done both of these things, and, as a result, has immeasurably improved the efficacy of the art of selling on a global scale by creating the scientific data to underpin 21st Century sales theory. If Kluwer invite me to offer a follow-up post to report on this conversation, I’ll also suggest ways in which it can be done – including ways to get parties (especially) and mediators to accept the presence of skilled researchers. Appreciate your input, John. Michael
Thanks very much for your thought-provoking blog. Intuitively it seems right that the application of data, perhaps even ‘big data’, to mediation will improve outcomes. I am so painfully aware of the lack of empirical data in my own jurisdiction (Scotland) that I’ve even taken the drastic step of doing a PhD. I should say this is ‘small data’: I’ve conducted in-depth interviews with 24 mediation consumers, all referred under court rules. It’s too early to talk about findings, but their rich, complex perspectives convince me of how much neglected information we allow to depart at the end of each case.
However, your blog also raises technical and philosophical issues. On a technical note I think we need to be cautious about applying medical models to a social process like mediation. Clearly surgeons have been able to improve outcomes and reduce death rates by reference to data. Pilots even more so. But can we really compare death rates and settlement rates? Some cases probably shouldn’t settle, and in every mediation we have at least two, sometime many more, autonomous decision-makers to deal with, each with their own unique reaction to our techniques + mediation techniques differ from surgical ones in the imprecision of their definition and application + in the social world, everyone you are observing is also observing you, meaning that the whole conflict system adapts to and changes with each participant. Complexity theory may be best suited.
On a more profound note, our field has long struggled with the ‘is/ought’ divide. The ‘pioneers’ (mediation has been around for millennia) began by describing what they did (‘what is’); as soon as these descriptions were published they tended to become prescriptions (‘what ought to be’). I fear the same will happen to whatever data we can harvest. To put it simplistically, if I learn that most commercial mediations in the USA take place entirely in private session (caucus) and that the success rate is 75%, should I apply that approach? What if the success rate is 82%? Or 62%?
I could go on, but I fear I’m writing a whole blog! To conclude, I’m all for more research and more data, but a bit wary of how it’s obtained, who it features, when it’s done, what it claims, why it’s conducted and who interprets it.
The problem with data analysis of mediation is that there is no uniform definition of what is a mediation process. We should searach for techniques for dispute resolution that give users satisfaction with a dispute resolution process. One such technique is using a mediator to get earlier and less expensive resolutions than the alternatives of litigation or arbitration. What techniques are used to get this result? This includes early hire of a mediator who knows how to diagnose the needs of decision-makers to overcome impasse. How do you define “early.” It means before significant expense has been incurred. The Guided Choice project supports the conclusion that mediators hired before the significant legal expense is incurred, and help settle disputes, produce the best results for users. As everyone seems to agree this is just common sense. Perhaps the results should be measured by “sales.” :). Perhaps the largest commercial case administrators could ask their users to identify satisfaction based on “early.”
The next question is what is the best way to train mediators who are early hires. We call such mediators Guiding Mediators. We train them to separate impasse diagnosis and process facilitation from participation in the settlement negotiation phase. This also seems to be common sense.
Paul Lurie
Something like twenty five to thirty years ago I was part of the creation of CAADRS (now Resolution Systems Institute) which just wanted to answer a “simple” question: when is the best time to send (or encourage) what kind of cases to mediation as an aid to the judiciary. We still have the same question. As aluded to, as much as some in the “social sciences” emphasis the second word in the phrase, it is not like medicine–and even there, the science seems to be looking more toward individual differences, rather than just a generic “body” for future improved results of treatment. Since there seems to be a limited “political” interest in more “efficient” dispute resolution–and a distinct interest group or two who really aren’t (read judges and lawyers), hard to see how we are going to do much more than stumble ahead as we have in the past.
Michael,
I’m late to the discussion, but have a somewhat different reason from the other comments (and perhaps your own) for endorsing this idea.
One of the things that struck me during the Global Pound Conference (a previous Leathes’ idea!) was the contrast between the hunger in the dispute resolution community for new ideas and new tools for resolving conflict, on the one hand, and our capacity/incapacity as lawyers who are not scientists to digest and properly incorporate science into our practices, on the other. I don’t mean the latter to be disrespectful to those, but rather a reflection of the risk that many of us (myself included) tend to play fast and loose with what we can glean from science. There’s a joke that goes, “the first rule of the Dunning-Krueger Club is that you do not know that you’re a member of the Dunning-Krueger club.”
For example, in teaching negotiation in my company, I used to mention the study in Israel about criminal judges and their likelihood of granting a parole request based on whether they reviewed the application early in the morning/closer to breakfast (likely) or closer to their lunch break (unlikely). Sitting in the Florence court of appeals one morning waiting for our case to be called, and watching the court breeze through the easiest cases early in their schedule, it occurred to me that there could easily be other explanations for the statistical result found in Israel (such as judges hearing the easier applications first so they get through a bigger chunk of their docket).
It necessary and inevitable that we will import scientific learnings into our practices, but as lawyers most of us lack the tools to fully appreciate all the cautionary aspects that scientists typically use. We may not even be aware, for example, of the problems or risks of “p-hacking” and the validity of purportedly empirical studies (the debunked “power pose” study, etc), even before asking whether it is appropriate to import these studies to the dynamics of a dispute negotiation.
Are we responsibly importing valid science or are, instead, applying leaches to wounded parties and drilling holes into litigants’ heads to release their demons, calling it state of the art science? The answer is… I don’t know.
Like your proposal for international standards in mediation (which led to the creation of the International Mediation Institute), the proposal for investment in mediation science is not far-fetched. A lot of work is already being done on negotiation and decision-making. It just is needs to be united under an umbrella of mediation and dispute negotiation.
Mike, thank you, especially for emphasising the dangers of “p-hacking” or the manipulation or “dredging” of data to generate spurious, pseudoscientific results. This is easily, and often innocently, done in the mediation field simply because there is so little dependable coalface research.
I am glad you, as a corporate user, do not consider it far-fetched to strive for the creation of a strong and credible scientific underpinning for mediation. Among other benefits, once such data is made readily accessible and smartly presented, it would enable mediating to be seen by users as a genuine art form (by which I mean the application of scientific method into practical use).
Perhaps, more than any other influencing factor, it is the credibility of, and obvious link between, the science of mediation and the art of mediating that would entice disputants and deal makers to become instinctively inclined to opt for mediation rather than be pushed. It is to be hoped that the great thought leaders in the field will press for such better quality research, and make it happen.
To endorse your last point, this needs to be done on a unified, or at least coordinated, and hopefully worldwide, basis.
Michael, your post is a timely reminder that despite its long history, mediation is still more art than science. To begin with, I would like to endorse Mike McIlwrath’s observation that it is important to bring scientific learnings into professional practice. Speaking from the perspective of someone who teaches at a business school, I am acutely aware of the need to ‘turn knowledge into action’ but also ‘action into knowledge’. Apart from a few surveys and the anecdotal ‘evidence’ and opinions of mediators themselves, we know very little about the impact of factors such as the characteristics of the parties, the process, the mediator and of the dispute itself not only on settlement rates but also on the quality of settlements. We know even less about the impact of culture (of the parties and/or the mediator) on mediation approaches and models. As several commentators have pointed out here, field research in this context indeed poses difficulties of access and confidentiality. Yet this has been done before (see. E.g., D. M. Kolb, The Mediators (1983) – 14 collective bargaining mediations observed; R. Peeples, C. Harris & T. Metzloff. (2007) Following the Script: An Empirical Analysis of Court Ordered Mediation of Medical Malpractice Cases J. Disp. Resol. – medical malpractice cases observed. Available here: https://scholarship.law.missouri.edu/cgi/viewcontent.cgi?referer=&httpsredir=1&article=1522&context=jdr).
As far as replicating or generalizability of the results of field research is concerned, this can be overcome through appropriate research design.
To get even richer and more accurate data, field research can be supplemented with (1) an analysis of mediators’ contemporary notes (see, e.g., D. Klerman & L. Klerman. (2015). Inside the Caucus: An Empirical Analysis of Mediation from Within. Journal of Empirical Legal Studies Volume 12, Issue 4, 686–715) and (2) pre- and post-process surveys of, or interviews with mediators and parties to a dispute (see e.g. D. Henderson (1996). Mediation Success: An Empirical Analysis, 11 Ohio State J. Disp. Resol. 105. Available here: https://kb.osu.edu/bitstream/handle/1811/79731/OSJDR_V11N1_105.pdf?sequence=1https://kb.osu.edu/bitstream/handle/1811/79731/OSJDR_V11N1_105.pdf?sequence=1 )
By combining field research and other forms of data collection, we might just be able to move mediation from its reputation as a kind of dark art to something that is, at least in part, based on hard evidence. At the very least, it provides an opportunity to put the anecdotal ‘evidence’ of mediation practitioners to the test!
Barney Jordaan, Professor of Management Practice: Negotiation, Conflict Management, Mediation at Vlerick Business School, Belgium