Machine learning is a rapidly growing field centered on powerful algorithms for solving problems quickly with the ability to improve themselves over time. However, despite the ‘intelligence’ demonstrated by machine learning algorithms, blindly trusting them can lead to dangerous decision-making: in extreme circumstances, for example, facial recognition software based on machine learning has led to wrongful arrests by the police.
How does the 1000minds PAPRIKA method relate to machine learning, and how can we avoid such pitfalls in decision-making?
PAPRIKA vs machine learning
Although the PAPRIKA method is based on a computer algorithm, it is not machine learning. Machine learning, and data mining in general, works by identifying patterns in datasets. Sophisticated computational and statistical techniques are used to quantify relationships between the outcomes of interest and observed variables (e.g. ‘criteria’) in the data set, resulting in the creation of an algorithm.
In contrast, PAPRIKA can be thought of as a method for decision-makers (and potentially other stakeholders) to train an algorithm based on their expert knowledge and preferences. PAPRIKA elicits this information from decision-makers by asking them simple questions involving trade-offs between the criteria for the decision at hand. As revealed by their answers, decision-makers’ preferences are codified in an algorithm to simulate (or to predict) their choices. Such algorithms are usually more accurate (valid and reliable) than the original experts! Algorithms don't get tired or distracted, or overly focused on a single criterion, or forget to take some criteria into consideration – e.g. see ‘Noisy’ Expert Judgements.
Because data mining works by looking for patterns in datasets, it is only as valid and reliable as the data used. When there are no data available for mining then PAPRIKA is superior (PAPRIKA, in effect, extracts the required preference data out of decision-makers themselves). And if the data used for data mining are in some way biased (e.g. incomplete or prejudiced), this bias might be reproduced, and potentially amplified, in the algorithms created – leading to flawed decisions.
Furthermore, whereas the algorithms produced by machine learning are often incomprehensible (even to their creators themselves!), PAPRIKA is fully transparent, auditable and revisable. This is because decision-makers’ answers to PAPRIKA’s trade-off questions, upon which the resulting algorithm is based, are recorded in 1000minds. This way, decision-makers can always review and revise their answers at any time they like – a transparency that is typically not present in machine learning programs.
However, with a solid set of reliable data, machine learning can provide the best evidence base for decision-making, and if there is a feedback loop, the machine learning algorithm may be able to improve itself. On the other hand, machine learning often still fails to take into account more human criteria, such as strategic, political or ethical considerations.
A recent real-world example
During the Covid-19 pandemic, both machine learning and 1000minds were used to create tools for prioritizing patients for intensive treatment. In the first few months, there was scant data with which to train an algorithm, whereas tools based on 1000minds were quickly created as the pandemic unfolded based on expert knowledge and experience. Later, machine learning was able to predict patient outcomes from treatment, but tools like 1000minds were still required alongside to consider ethical considerations such as equity of access for at-risk groups, and reciprocity for health workers who endangered themselves by treating others.
Making better decisions
When making decisions, transparency and reliability of the decision-making process are vital. 1000minds offers transparency at every step of the decision-making process, and includes checks to maximize the accuracy of your preference data. Our numerous awards and the hundreds of academic publications that reference 1000minds corroborate the strength and reliability of the PAPRIKA method, so you (and others) can have confidence in your decisions.
Curious about how 1000minds can help you? Sign up for our free trial or book a demo with our friendly team today.
Share this post on:
