Today’s episode is something of a double-header, as we address two different but related topics: the scientific method, and how Andy and Jeff’s efforts fit into it. In the first half of the show, Andy and Jeff talk about their personal histories, and how their experiences prepared them for tackling the mystery of MH370.
In the second half, Jeff describes how working on this mystery has shaped his understanding of the scientific method, and in particular how scientist deal with uncertainties in data and in their knowledge of initial conditions. Understanding so-called Bayesian methods is crucial, because it’s the approach that search officials used in defining the the search area on the seabed of the southern Indian Ocean.
Bayes theorem has crept into the public discourse in recent years; you’ll often hear about people “updating their priors” and the like, meaning that they’ve changed their mind based on new information. In this episode Jeff refers to it as a kind of reverse probability estimate, in the sense that rather than using one’s knowledge of a current system state to calculate a future state, you use knowledge of the current state to assess what previous conditions might have given rise to it.
There’s a neat example of this idea in The Book of Why by Judea Pearl and Dana Mackenzie. Imagine that you have a 10-foot-long billiards table. If you fling a cue ball so that it vigorously bounces off multiple sides and effectively winds up in a random location, what is the probability that it will wind up within 1 foot of the far-left hand edge? It’s easy to see that the probability is 10 percent.
Now imagine that you have a billiards table of unknown length and the cue ball is one foot from the left-hand edge. How long is the table? The answer you give depends on your knowledge of possible prior states. For instance, it may be the case that all billiard tables are either 10 or 14 feet long. Or maybe they can be arbitrarily long.
The searchers for MH370 found themselves in an analagous situation when it came to defining a search area using the Inmarsat data. They had to ask themselves, “What is the universe of possible flights that could have given rise to this data, and what is the relative probability of each?” This would then allow them to calculate a probability heat map on the seabed of the southern ocean of where the plane’s wreckage might be located. We know a lot about how they carried out this work because the Australian scientists who carried it out wrote a very detailed and lucid account which they published in 2015 as Bayesian Methods in the Search for MH370.
To watch Deep Dive: MH370 on YouTube, click the image above. To listen on Apple Music, click here. It’s also now available on most other podcast platforms as well, including Spotify.