It seems obvious that important decisions should be made in a consistent manner. In a court of law one expects that the same verdict, and the same sentence, would be handed out whoever the judge and jury were. In education we expect the same exam outcome independent of the examiner. In business contracts should be awarded, following a tendering process, in a consistent manner. This post explores different types of consistency, and whether in some situations achieving quality decision-making may require accepting some inconsistency in outcome.
Inconsistent decision-making leads to a perception of unfairness. It can lead to the waste of resources and a loss of confidence in the system. This can encourage people to subvert the system by game playing to get the outcome they want.
In a recent paper, Friesen and colleagues look at the consistency of decision-making in a particular context; the work of research ethics committees. These committee decide whether a research project involving patients is ethical and should be allowed to happen. They have an important role in protecting research subjects and in promoting good research. There can be a perception by some researchers that decision-making of these committees is not consistent.
In this paper the authors look at decision-making in relation to research ethics committees. However, a lot of what they say applies to businesses, such as tendering for contracts and HR decisions.
Procedure and outcome
There are two types of consistency that important for decision making. Procedural consistency concerns the processes that are used to come to the decision. Content consistency is about whether the decision is the same regardless of who and when it is made.
It is possible to have one without the other. For example, if a contract was decided by a coin toss, there would be procedural consistency. However, there would be no content consistency. This might seem fair and acceptable in some settings such as deciding which cricket team should bat first. But it probably not be acceptable for the awarding of a multi-million pound contract.
For full procedural consistency it would be important for all the factors used in making a decision to be described, as well as how they are to be used. This protects against information that should be irrelevant to the decision-making creeping into the process (such as the gender or ethnicity of an applicant).
That is all well and good for simple decisions. If you are tendering for the supply of paperclips, then the main factors will be the price, quality and time of delivery. But it will be different for complex decisions where there are many factors to be considered. It may be difficult to be objective about some factors, and they can interact in complex ways. If you are tendering for a new building factors such as aesthetics, environment, fair employment, cost and quality will often be in tension with each other.
Friesen and colleagues argue that it is important that there is procedural consistency, and that there is transparency on how decisions are made. This will normally increase content consistency (unless the procedure involves tossing coins!). It also increases confidence in the process.
Following an algorithm to make decisions
However, there is a dilemma for decision makers. It is possible to characterise two extreme forms of decision-making. The first is algorithmic decision-making in which the process and information used to make the decision is completely defined in advance. This should result in both procedural and content consistency. However, the decisions may be unfair because some factors are relevant, but not specified, will be left out. Many who has been involved in tendering for complex project will be aware that a pure algorithmic process can on occasion lead to perverse outcomes.
Discretion in decision making
The alternative is discretionary decision-making. As its name suggests, this allows the person or body making a decision to use any information that they consider relevant. A court of law operates a largely discretionary decision-making process.
This approach has an inherent risk of inconsistency. The same decision might not be reached if the process were run again. However, in many cases it may be more realistic, and lead to better decision-making as it allows all relevant factors, and how they interact, to be considered.
It must be recognised that often there is no perfect decision. When considering aesthetics, or balancing up to companies with contrasting track records on gender equality and environmental protection, you may need to accept that the decision will depend on the values and beliefs of the decision-makers.
How do we prevent this being abused?
However, there are safeguards that need to be in place. There is a need for procedural consistency; the decision-makers have to be competent and where appropriate trained. They could should consider key bits of information and not consider (or be shown) information that is totally irrelevant. The grounds of the decision should be transparent; decision-makers should articulate the reasons for their decision. This both increases trust in the process, but also forces the decision-makers to assess and check the basis on which they made their judgement. Finally, when possible, there should be rigorous analysis of decision making over time to see if there are any bias in the decision making (for example, analysis of pay has revealed an unacceptable gender pay gap in many organisations, the result of bias in decisions on pay and promotion).
While the study looked at research ethics committees, it has a lot to say about decision-making in other fields, including education and business. While there is considerable merit in using algorithmic-based decision-making in many cases, there will be times when it is not the best way. In these circumstances we should be explicit about the fact that we are using discretionary approaches, make sure that the safeguards are in place and be aware of the dangers of this approach.
Reference
Friesen, P., Nadia Mohd Yusof, A., & Sheehan, M. (2019). Should the Decisions of Institutional Review Boards Be Consistent? Ethics & Human Research. 41. 2-14. 10.1002/eahr.500022.
Declaration of interests
Andrew George is a non-executive director of the Health Research Authority that is responsible for the research ethics service in United Kingdom. He was involved in discussions on earlier versions of this paper. He writes in a personal capacity.
Andrew George
An executive coach and consultant. Andrew has been held senior roles in universities. His interests include medical research and innovation, education, leadership and research ethics.
Web page: ajtg.co.uk
Blog page: andrewgeorgeblog.com
Twitter: @ProfAGeorge
LinkedIn: linkedin.com/in/ajtgltd
© 2019, Andrew George, all rights reserved
Published 19 August 2019
Comments