How a scientific approach can lead to better decision-making

groundbreaking book

There is probably no more important skill in life than good decision-making. So much of what happens to us in life, both large and small, is at least partly determined by our past decisions. Though external factors are always at play, and we rely heavily on good fortune and also the circumstances in which we find ourselves, everybody knows that making the right choice is often the key to success in all walks of life. Of course, this is why we so often agonize over whether we are doing the right thing in a given situation.

Often, the most important decisions are the classic ‘life choices’, such as what you should study, what career to choose, whether to move to another city or move abroad and even who to marry. We also make millions of more minor decisions throughout our lives, from what bar to visit on a Friday night to what salary to request in a new job. Though these decisions are often inconsequential, we also know that sometimes even the smallest detail can make a huge difference.

So how and why do we make decisions? How can we make better choices, and how can we evaluate the ones we make?

Am I making the right choice?

When you make a decision, it is almost impossible to know at the time if it is a good one or not. Decisions are inherently related to the future, and the future by its very nature is uncertain. If you are lucky, you have prior information, such as the weather forecast before deciding what to wear for the day for instance or the past behavior or performance of an employee you are considering for a promotion. However, in most cases, the information you have available is incomplete at best. We also know that information can be unreliable, and that past performance is no guarantee of future success.

Even once you have made your choice, by what timeline do you judge its success? One classic example is the stock market. Maybe you decide to invest your first $100 and pick out a promising clean energy company. If the share price rises by 10% or 20% in the first week — or month, or year — you will surely feel like you have made the right choice and might even want to buy more. You also know, however, that at some point in the future, a market crash could turn your profit into damaging losses in a matter of hours and make a mockery of your former confidence.

You might also wonder whether it even makes sense to judge a decision by its outcome. Perhaps it was due to chance or circumstances beyond your control. Additionally, often the outcome is mixed, with both positive and negative aspects. Indeed, how many of us have experienced a pang of regret at a decision we made many years ago when faced with a fork in the road, even if the choice we made was, in many ways, a good one?

Should you trust your gut or experience?

Faced with so much uncertainty, one reaction is to rely on your gut feeling — to make a decision that just feels right. Most of the time, there is often nothing wrong with relying on your intuition. After all, they are the sum of not only all your past experiences, but also the wisdom of the people around you and millions of years of evolution. Often, if your senses are telling you to run from danger or stand and fight, there is a very good reason to trust them.

In certain cases, however, our instincts can be misleading or even obstructive. As psychologist and economist Daniel Kahneman argues in his groundbreaking book on decision-making Thinking, Fast and Slow, ‘fast-thinking’, where we rely on our intuition, dominates our day-to-day lives and is often highly effective. However, when it comes to more complicated situations, fast thinking can cause us to vastly over-simplify the situation at hand, often resulting in costly errors. Indeed, numerous studies have identified many instances where an over-reliance on intuition leads to illogical and inefficient decision-making.

Experience is a virtue and a vice

One way to guard against overly instinctive or hasty decision-making is through experience. From a child who only needs to touch the hotplate once to realize that they should never do it again, to a driver who learns how to make safer choices on the road, the world is full of clear examples of how experience can help us improve our decision-making skills.

At the same time, there is the danger that experience can also be a liability in a constantly changing environment. Sometimes, expecting the future to look and behave like the present or the past is a weakness. This is particularly true with regards to technology or other groundbreaking inventions, where experience and preconceived ideas can sometimes make it harder to understand the importance of what we are looking at. The famous quote from renowned economist Paul Krugman concerning the internet in 1998 stated: “The internet will fade away because most people have nothing to say to each other. By 2005 it will be clear that the internet’s impact on the global economy has been no greater than the fax machine.”

So, does this mean that it’s impossible to make good decisions and it’s better to simply toss a coin? Not exactly. The key is to combine your intuition and experience with other tools to guard against potential errors and make a truly informed choice. In recent years, experts have been working hard to explore key questions that help us attain a better understanding of decision-making. 

A scientific approach

Modern decision-making theory tells us that if you can combine your intuitive instincts and experience with a more objective, data-based scientific approach, you are far more likely to succeed in the long-run. We are most familiar with the scientific method in terms of traditional scientific fields such as physics, biology and chemistry. Most of us recall our high school science lessons, where we were taught to confirm that water really boils at 212 degrees Fahrenheit using a thermometer or investigate the real-life effects of the process of photosynthesis through a structured observation of plants. This idea has now been extended to other fields, with a significant degree of success.

When we talk about the scientific approach to decision-making, it refers to the idea of looking for objective truths and useful data to make informed decisions. A simple example would be if you work for a company that is interested in investing in a new area of business, and you are tasked with performing an analysis to help determine if it is worth the money. While some leaders might decide based on instinct or a hunch, a more considered approach would be to examine the situation carefully and to present a strong, structured case based on empirical qualitative and/or quantitative data either in favor or against the move.

The magic formula

Over time, we have established a successful formula to help us produce exactly this kind of empirical evidence. Though there are some variations to the scientific approach, we can summarize the general formula as follows:

Question

Most research starts with a question, usually related to a requirement or a problem related to the organization, such as: ‘How can we improve the test scores of the children at our school?’ or ‘How can we generate more income for our company?’

Research

The next step is to gather any relevant information or data that might give you an indication of what direction to go in. In our school example, you could gather the test data of students from the past five years, ideally with detailed information on age, aptitude and other variables. You might also be tempted to conduct some qualitative research, such as interviewing students and teachers about what they think might help. If you are looking to increase revenues for your company, you might look at sales figures in terms of growth, industry sector and overall profit, or try to identify new target markets to exploit.

Hypothesize

You can then establish a working hypothesis that can be properly tested and investigated. This means it should not be an open-ended, generalized statement, such as: ‘Our products would be popular in Italy’. Instead, it should be a closed, testable position, such as: ‘We could sell 5-10,000 items of XYZ in Northern Italy among the 35-49 age group annually’, or ‘Special after-school study sessions in Grade 7 will increase their end-of-year test scores by 10-15%’.

Test

Next, you should put the hypothesis into action through targeted testing. At this stage, because you are not yet sure if your hypothesis is correct, it makes sense to conduct smaller, targeted testing in a concentrated area. With the school, you might want to offer special study sessions to one group of students, with a second group of similar students in the same year acting as a control group, and then compare the results. With the company, it may be a good idea to trial sales in a single town or store, and gauge reactions.

Experiment

Once you have established the initial validity or otherwise of your hypothesis, it is important to introduce different variables. First results can often be misleading, and the greater understanding you have, the better your chances of reaching the right decision. With school testing, the next step might be to experiment with different age groups or teachers, or to increase or decrease the number of study sessions, to see what, if any, difference it makes. With the company, you may want to try different products, prices or locations, to see what impact they have on sales.

Analyze

Once you have collected a sufficient amount of information, you will need to analyze and evaluate the results in depth. Is the increase in test scores sufficient to warrant the extra classes? At what age groups are additional study sessions more or less beneficial? Is the expected profit to be made from selling your products in a new market or territory enough to justify the added expenditure to establish yourself in the area?

Conclude

When all is said and done, there is no use in collecting data or information for its own sake. Sooner or later, you or your organization will have to make a decision — extra classes or not, whether to target a new market or continue to build on your successes in other areas. Here, the key is to reach a clear decision based on empirical evidence, while also being prepared to revisit your decision at a later date.

A thirst for expertise

Today, leaders in all walks of life are becoming more open to the importance of data and useful, objective information. Some call it the Moneyball effect, named after the book by Michael Lewis that described how the Oakland A’s, a poorly financed baseball team, were able to outperform their richer peers by mining baseball statistics for anomalies that could help them identify talented but overlooked players. The popularity of this kind of approach now means that many organizations actively seek out people who have a high degree of knowledge in complex decision-making. 

Fortunately, as interest in this kind of systemic thinking and high-level analysis grows, many higher education institutes are looking to provide leadership courses that can help you learn how to make better, evidence-based decisions in a wide number of areas. The Spalding University Doctorate in Education online, for example, will equip you with the skills needed to become an expert decision-maker in the field of education. In this course, students have the chance to learn about the use of the scientific approach in research and to improve their decision-making with the help of empirical evidence. 

By the completion of the two-year degree, conducted entirely online through this prestigious university, students can expect to have gained the knowledge, skills and abilities to conduct quantitative, qualitative and mixed research in a wide range of different disciplines. In addition, they will also have the chance to examine leadership methods and the systemic thinking that is crucial to surviving and thriving in the modern global economy, learn about the importance of ethical leadership and the best practices and key theories behind it, and become familiar with effective and creative ways of implementing transformative change.

Expert decision-makers always in demand 

Partly because our globalized, digitalized world is so complex, graduates of this kind of course, who have clearly demonstrated an ability to conduct in-depth research in a variety of fields and apply it to make more informed decisions, are greatly in demand in a wide range of different fields. In the education sector, graduates from Spalding University’s course, for example, have gone on to work as principals, department heads or deans. Others go on to work in business administration, where data-driven strategy is now the norm, or in healthcare, where leaders and other decision-makers typically have access to a huge wealth of information and statistical data, but are under pressure to make impactful decisions in a high-pressure environment.

Naturally, making the right choices on a consistent basis and driving change is about more than collecting data. It is also essential to develop excellent communication and interpersonal skills so that you are able to get your point across, while naturally, an in-depth knowledge of your chosen industry or field is also vital. Finally, if you want to not only arrive at the right decision but also implement it effectively, you will be reliant on the people around you to help make it happen.

Become a tool for change

Overall, we can say that while some people do manage to get through life successfully on the basis of their intuition and experience alone, there are also countless examples of where a more scientific approach to decision-making could have yielded far more beneficial results.

As we have said above, intuition is not to be discounted, but it does make sense to be able to challenge them or, if necessary, to back them up. The scientific method offers a potential framework for conducting empirical research that can help you build a more efficient and effective organization. It is particularly important and useful if you are concerned with an innovative or groundbreaking measure or approach, where resistance is usually high and uncertainty can be pervasive. Here, data and empirical evidence can be used to present a powerful case that the strategy you are advocating for is the right one.

Whatever your career path or chosen profession, if you want to become an effective decision maker, it is essential to learn and apply the best theories, practices and techniques available. If you are able to combine that with experience and a persuasive method of communication, you will be on the right path to producing meaningful change.