Spending couple of weeks down the south coast of New South Wales, spotting dolphins and echidnas, has slowed down my blogging. Fortunately, regular contributor James Glover has once more come to the rescue with a guest post. This time his topic is wheelchairs and air-travel.

Perhaps you’ve heard of a recent court case in which a wheelchair user, Sheila King, took Jetstar to court (and lost) on the basis of the Disabilities Discrimination Act? If you are a wheelchair user and you book a flight on one of our airline carriers then a fairly obvious thing won’t happen. Unlike say a bus you won’t be able to board the aircraft in your chair and be strapped in for the journey. What actually happens is that when making the booking you tick a box (or tell the booker on the phone) that you are in a wheelchair. If there are seats available for wheelies when you get to the airport you will give up your chair and be made to use a specially designed “wheelchair” (its a chair, it has wheels) that is designed to be fit the narrow corridor of most planes which I am sure you are aware of – their narrowness, for you, only apparent when the person ahead of you is blocking the aisle loading 3 pieces of carry on luggage into the overhead lockers while chatting to their new friends in the seat they are meant to occupy. We all suffer this situation. These “wheelchairs” are not designed to be used without help, they are more like children’s toy carts and cannot be operated by the user as the wheels are very small and low down. For a wheelchair user to be taken out of their wheelchair in a public place can be quite discombobulating. Many wheelchair users develop a personal relationship with their chair – it is after all a place you spend many of your waking hours.

Digression. The very first time I was in a wheelchair outside the confines of a hospital ward (it was a hospital wheelchair but is the exact same model I now own, like I said it is personal) I was being pushed by none other than the proprietor of this very website! Without going into the details let’s just say it was a pretty dramatic event and we both learned a valuable lesson in wheelchair use and the wheelchair repair workshop at the hospital was kept busy. But I digress.

So here is the thing. According to Google about 1% of the population uses wheelchairs. And a Jetstar plane has about 200 seats so they expect to get about 2 wheelchair users on average per flight. So what is the problem with only allowing this same number on each flight, as some airlines do? Well the problem is that statistically wheelchair users don’t travel in pairs and sometimes there will be less than 2 users and sometimes there will be more. Just as if you toss 10 coins sometimes there will be fewer than 5 heads (the average or expected number) and sometimes there will be more. Only on average will there be 5. In fact it is a simple problem to work out the probability of there being, say, n wheelchair users, given the average of 1% on a 200 seat plane. This is called the Binomial Distribution. If you have access to Excel then the function Binomdist(n,200,1%) will tell you this probability. Before I give you some numbers I admit that the overall population average may not be the same as the average flying on planes. It may be less than 1% due to wheelchair users being put off flying. But maybe on some routes it is higher: but I am guessing the annual “snowbird” migration of retired people from the northern United States to Florida at the start of Winter would track above the 1% rate.

So here are the Binomial probability figures.

For example, assuming a 1% chance of any given passenger a 200 hundred seat plane being in a wheelchair, the probability that there will be exactly 4 wheelchair passengers wanting seats is 9%. To work out the probability of a passenger being denied a seat on their preferred flight, we will assume that we’re dealing with an airline where more than two wheelchair passenges book on a flight, then at least on passenger will have to change their travel plans. From the table above, the chance of the flight only having 0, 1 or 2 wheelchair passengers totals 68%, so there’s a 32% chance that there will be at least one wheelchair passenger who cannot fly. For any one wheelchair passenger, there is a (n-1)/(n+1) chance of being bumped if n other wheelchair passengers book on the flight. Weighting that by the probably that there are n passengers and adding it up for all n>1 gives a probability of 27%. As a frequent flyer in a wheelchair, you can expect to miss out on a seat quite regularly! [Note: these calculations have been updated: the editor's "corrections" were undone. Ed.]

I am quite fortunate now that I no longer need to travel in my wheelchair. But as I still use a walking stick I wait for everyone else to get off the plane. You sit there, looking behind you to see if everyone else has left. But there are always these strange people who seem to sit there at the back of the plane and wait for 10 minutes or more, after everyone has disembarked, before even moving. You wonder why the airline staff don’t just hurry them off? I assume they aren’t disabled because they are sitting at the back of the plane. If airlines really had a problem with the extra time that getting wheelies off the plane then they could make this up by just moving these people along.

When I first read about this case my initial response was that being disabled and traveling is a bit of challenge anyway and you just get on with it. But the more I thought about it I wondered if the airlines just took it for granted that wheelchair users would change their plans to fit in with the rules. I am glad Sheila King took the issue up!

Everyone knows hang-gliding is risky. How could throwing yourself off a mountain not be? But then again, driving across town is risky too. In both cases, the risks are in fact very low and assessing and comparing small risks is tricky.

Ronald A. Howard, the pioneer of the field of decision analysis (not the Happy Days star turned director) put it this way:

A problem we continually face in describing risks is how to discuss small probabilities. It appears that many people consider probabilities less than 1 in 100 to be “too small to worry about.” Yet many of life’s serious risks, and medical risks in particular, often fall into this range.

R. A. Howard (1989)

Howard’s solution was to come up with a better scale than percentages to measure small risks. Shopping for coffee you would not ask for 0.00025 tons  (unless you were naturally irritating), you would ask for 250 grams. In the same way, talking about a 1/125,000 or 0.000008 risk of death associated with a hang-gliding flight is rather awkward. With that in mind. Howard coined the term “microprobability” (μp) to refer to an event with a chance of 1 in 1 million and a 1 in 1 million chance of death he calls a “micromort” (μmt). We can now describe the risk of hang-gliding as 8 micromorts and you would have to drive around 3,000km in a car before accumulating a risk of 8μmt, which helps compare these two remote risks.

Before going too far with micromorts, it is worth getting a sense of just how small the probabilities involved really are. Howard observes that the chance of flipping a coin 20 times and getting 20 heads in a row is around 1μp and the chance of being dealt a royal flush in poker is about 1.5μp. In a post about visualising risk I wrote about “risk characterisation theatres” or, for more remote risks, a “risk characterisation stadium”. The lonely little spot in this stadium of 10,000 seats represents a risk of 100μp.

One enthusiastic user of the micromort for comparing remote risks is Professor David Spiegelhalter, a British statistician who holds the professorship of the “Public Understanding of Risk” at the University of Cambridge. He recently gave a public lecture on quantifying uncertainty at the London School of Economics*. The chart below provides a micromort comparison adapted from some of the mortality statistics appearing in Spiegelhalter’s lecture. They are UK figures and some would certainly vary from country to country.

Based on these figures, a car trip across town comes in at a mere 0.003μmt (or perhaps 3 “nanomorts”) and so is much less risk, if less fun, than a hang-gliding flight.

It is worth noting that assessing the risk of different modes of travel can be controversial. It is important to be very clear whether comparisons are being made based on risk per annum, risk per unit distance or risk per trip. These different approaches will result in very different figures. For example, for most people plane trips are relatively infrequent (which will make annual risks look better), but the distances travelled are much greater (so the per unit distance risk will look much better than the per trip risk).

Here are two final statistics to round out the context for the micromort unit of measurement: the average risk of premature death (i.e. dying of non-natural causes) in a single day for someone living in a developed nation is about 1μmt and the risk for a British soldier serving in Afghanistan for one day is about 33μmt.

*Thanks to Stephen from the SURF group for bringing this lecture to my attention.

In this follow-up guest post on The Stubborn Mule, Mark Lauer takes a closer look at the relationship between national development and fertility rates.

STOP PRESS: Switzerland’s population would be decimated in just two generations if it weren’t for advances in their development.

At least, that’s what the modelling in a recent Nature paper projects.  The paper, widely reported in The New York Times, The Washington Post and The Economist, amongst others, was the subject of my recent Stubborn Mule guest post.  In that post, I shared an animated chart and some statistical arguments that cast doubt on the paper’s conclusion.  In this post, I’ll take a firmer stance: the conclusion is plain wrong.  But to understand why, we’ll have to delve a little deeper into their model.  Still, I’ll try to keep things as non-technical as possible.

First, let’s recap the evidence presented in the paper.  It comprised three parts: a snapshot chart (republished in most of the reportage), a trajectory chart, and the results of an econometric model.  As argued in my earlier post, the snapshot is misleading for several reasons, not least the distorted scales.  And the trajectory chart suffers from a serious statistical bias, also explained in my earlier post.  I’ll reproduce here my chart showing the same information without the bias.

That leaves the econometric model.  From reading the paper, where details of the model are sketchy, I had wrongly inferred that the model suffered the same statistical bias as the trajectory chart.  I have since looked at the supplementary information for the paper, and at the SAS code used to run the model.  From these, it is clear that a fixed HDI threshold of 0.86 is used to define when a country’s fertility should begin to increase.  So there’s no statistical bias.  However, I discovered far more serious problems.

Possibly the best demonstration of these problems is to plot the so-called “null hypothesis” trajectories in the model.  These show what the model expects would have happened if fertility changes were wholly insensitive to development after development passed the threshold.  The model determines the sensitivity of fertility to development by measuring the true trajectories relative to these null trajectories.  Here they are.

Given these null trajectories, it is perhaps not surprising that the model finds a statistically significant positive effect for development.  But let’s consider just how negative the null trajectories are.  According to these trajectories, the TFR in Switzerland in 2005 would have been just 0.66.  After only one generation at this TFR, a country’s native population would shrink by a factor of 3.  Two generations would collapse the population to barely over a tenth of the original.  This is Switzerland we are talking about here.

Likewise, according to these null trajectories, the TFR in Germany, the Netherlands and the United States would have fallen to 0.75, 0.72 and 0.83 respectively in 2005.  In fact, 15 of the 37 countries in the model have null trajectories which drive TFR below 1.0 in 2005.  Remember a TFR below 1.0 means the native population will more than halve every generation.  Given this null hypothesis, it would be extremely surprising if it weren’t rejected by the data.

The obvious question is how can the model possibly make such dire projections?  The answer lies in a combination of two things: the set of countries chosen for inclusion in the model, and the use of so-called “time fixed-effects”.  In the words of the paper:

This specification controls for … time trends, and thus allows us to test whether the reversal … persists after controlling for potentially confounding factors such as … common time trends.

To do this, the model contains so-called “year dummies”, factors that represent the impact of a particular calendar year.  For example, being in 1990 is modelled as having the same fixed impact on the fertility of all countries.  Being in 1991 is modelled as having another impact, but the same one across all countries.  And so on.  These fertility impacts are estimated from the data as part of the econometric analysis.  Here is a chart showing the values of the year dummies estimated in the model.

For example, in 1982, fertility in all countries is estimated to fall by 0.066 simply due to the fact of it being 1982.  Taken together, these year dummies impose a steeply declining path of fertility.  Now notice the resemblance between this path and the null trajectories above. The time trend estimates dominate the modelling.

But we still haven’t explained why they are so negative.  The answer is that estimates for the year dummies are made using the entire data set, including fertility changes in countries whose HDI is below the threshold.  For example, in 1982 twelve of the countries have HDI scores below 0.86, some of them substantially below.  Kuwait had an HDI of 0.75 in 1982 (a year when its TFR was 4.87).  In the same year, South Korea’s HDI was only 0.76 and Chile’s just 0.74.  Those are the development levels of Iran, Armenia and Ecuador today.  But the model uses the fertility changes of Kuwait, South Korea and Chile in 1982 to establish a baseline for developed countries like Germany, Switzerland and the United States.

By including a dozen or so developing countries, whose fertility rates start out higher and fall quickly, the model estimates a steeply declining time trend.  But those same data points are excluded from the HDI impact estimation because the corresponding countries are below the threshold in those years.  As a result, HDI alone is forced to explain the absence of the steeply declining time trend in developed countries.

The paper’s criteria for inclusion of a country in the model is that its HDI must reach 0.85 by 2005.  This allows countries like Argentina and the Slovak Republic to just scrape in, countries that provide no information whatsoever about fertility changes beyond an HDI of 0.86 because they never reach that threshold.  Yet these countries contribute to the time trend estimation, and they exhibit large falls in fertility because they have higher fertility to start with.

To see this graphically, let’s consider what happens when we change the criteria for inclusion.  Here is a chart showing the modelled “reversal” in fertility found by applying the paper’s model while varying the criterion for inclusion of countries.

The x-axis shows the minimum level of HDI in 2005 which allows a country to be included.  The labelled points indicate the numbers of countries that are included as a result.  The blue point is the criterion chosen in the paper.  The point labelled 27 represents Australia, Austria, Belgium, Canada, Cyprus, Denmark, Finland, France, Germany, Greece, Iceland, Ireland, Israel, Italy, Japan, South Korea, Luxembourg, the Netherlands, New Zealand, Norway, Portugal, Slovenia, Spain, Sweden, Switzerland, the United Kingdom, and the United States.  These are all developed countries, though Cyprus and Portugal could be considered borderline cases.

The ten additional countries used in the paper are Argentina, Chile, the Czech Republic, Estonia, Hungary, Kuwait, Malta, Poland, the Slovak Republic and the United Arab Emirates — all developing countries, or only recently developed.  Including these countries drives down the time trend estimates (that is, the year dummies) and drives up the measured reversal.  If we include still more developing countries, this reversal goes higher.  If we use all 143 countries in the data set, the measured reversal is that HDI increases fertility in developed countries at a rate of almost 11 (that is, a 0.05 increase in HDI yields a 0.55 increase in TFR).  So even if we believe the author’s model, their reversal rate of around 4 is still wrong, because by including more data we converge on a rate over 2.5 times their estimate.

More sensibly, we should exclude the developing countries.  And, as the chart above shows, the reversal then disappears: fertility marginally decreases with increasing development.  Better still, we should modify the model to only use data points from years where the corresponding country is above the threshold.  Under this model, the rate of decline in fertility with development is even more negative (the Mathematica source code for all the charts and analysis in this post has been uploaded to GitHub).

When I first read about this paper, I was impressed.  But after looking at the data I had serious doubts.  Now I am convinced that its conclusion is unsupportable.  At best, we can say that fertility stabilises or falls more slowly once development reaches the threshold.  But that would be a reasonably predictable result — there are lower bounds to fertility after all.  And it wouldn’t generate headlines in the New York Times, the Washington Post and the Economist.

In this first ever guest post on The Stubborn Mule, Mark Lauer takes a careful look at the relationship between national development and fertility rates.

Recently The Economist and the Washington Post reported a research paper in Nature on the relationship between development and fertility across a large number of countries.  The main conclusion of the paper is that, once countries get beyond a certain level of development, their fertility rates cease to fall and begin to rise again dramatically.  In this post I’ll show an animated view of the data that casts serious doubt on this conclusion, and explain where I believe the researchers went wrong.

But first, let’s review the data.  The World Bank publishes the World Development Indicators Online, which includes time series by country of the Total Fertility Rate (TFR).  This statistic is an estimate of the number of children each woman would be expected to have if she bore them according to current national age-specific fertility rates throughout her lifetime.  In 2005, Australia’s TFR was 1.77, while Niger’s was 7.67 and the Ukraine’s only 1.2.

The Human Development Index (HDI) is defined by the UN as a measure of development, and combines life expectancy, literacy, school enrolments and GDP.  Using these statistics, again from the World Bank database, the paper’s authors construct annual time series of HDI by country from 1975 until 2005.  For example, in 2005, Australia’s HDI was 0.966, the highest amongst all 143 countries in the data set.  Ukraine’s HDI was 0.786, while poor old Niger’s was just 0.3.

A figure from the paper was reproduced by The Economist; it shows two snapshots of the relationship between HDI and TFR, one from 1975 and one from 2005.  Both show the well-known fact that as development increases, fertility generally falls.  However, the 2005 picture appears to show that countries with an HDI above a certain threshold become more fertile again as they develop further.  A fitted curve on the chart suggests that TFR rises from 1.5 to 2.0 as HDI goes from 0.92 to 0.98.

Of course, this is only a snapshot.  If there really is a consistent positive influence of advanced development on fertility, then we ought to see it in the trajectories through time for individual countries. So to explore this, I’ve used a Mathematica notebook to generate an animated bubble chart.  The full source code is on GitHub, including a PDF version for anyone without Mathematica but still curious.  After downloading the data directly from Nature’s website, the program plots one bubble per country, with area proportional to the country’s current population.

Unlike with the figure in The Economist, here it is difficult to see any turn upwards in fertility rates at high development levels.  In fact, the entire shape of the figure looks different.  This is because the figure in The Economist uses axes that over-emphasise changes in the lower right corner.  It uses a logarithmic scale for TFR and a reflected logarithmic scale for HDI (actually the negative of the logarithm of 1.0 minus the HDI).  These rather strange choices aren’t mentioned in the paper, so you’ll have to look closely at their tick labels to notice this.

To help focus on the critical region, I’ve also zoomed in on the bottom right hand corner in the following version of the bubble chart.

One interesting feature of these charts is that one large Asian country, namely Russia, and a collection of smaller European countries, dart leftwards during the period 1989 to 1997.  The smaller countries are all eastern European ones, like Romania, Bulgaria and the Ukraine (within Mathematica you can hover over the bubbles to find this out, and even pause, forward or rewind the animation).  In the former Soviet Union and its satellites, the transition from communism to capitalism brought a crushing combination of higher mortality and lower fertility.  In Russia, this continues today.  One side effect of this is to create a cluster of low fertility countries near the threshold HDI of 0.86 in the 2005 snapshot.  This enhances the impression in the snapshot that fertility switches direction beyond this development level.

But the paper’s conclusion isn’t just based on these snapshots.  The authors fit a sophisticated econometric model to the time series of all 37 countries that reached an HDI of 0.85, a model that is even supposed to account for time fixed-effects (changes in TFR due only to the passage of time).  They find that the threshold at which fertility reverses is 0.86, and that beyond this

an HDI increase of 0.05 results in an increase of the TFR by 0.204.

This means that countries which develop from an HDI of 0.92 to 0.98 should see an increase in TFR of about 0.25.  This is only about half as steep as the curve in their snapshot figure, but is still a significant rate of increase.

However, even this rate is rather surprising.  Amongst all 37 countries, only two exhibit such a steep rise in fertility relative to development between the year they first reach an HDI of 0.86 and 2005, and one of these only barely.  The latter country is the United States, which manages to raise TFR by 0.211 per 0.05 increase in HDI.  The other is the Czech Republic, which only reaches an HDI of 0.86 in 2001, and so only covers four years.  Here is a plot of the trajectories of all countries that reached an HDI of 0.86, beginning in the first year they did this.  Most of them actually show decreases in TFR.

So how do the authors of the paper manage a statistically significant result (at the 0.1% level) that is so widely different from the data?  The answer could well lie in their choice of the reference year, the year in which they consider each country to have passed the threshold.  Rather than using a fixed threshold as I’ve done above, they express TFR

relative to the lowest TFR that was observed while a country’s HDI was within the window of 0.85–0.9.  The reference year is the first year in which this lowest TFR is observed.

In other words, their definition of when a country reaches the threshold depends on its path of TFR values.  In particular, they choose the year when TFR is at its lowest.

Does this choice statistically bias the subsequent trajectories of TFR upwards?  I leave this question as a simple statistical exercise for the reader, but I will mention that the window of 0.85–0.9 is wider than it looks.  Amongst countries that reached an HDI of 0.9, the average time taken to pass through that window is almost 15 years, while the entire data set only covers 30 years.

Finally I’d like to thank Sean for offering this space for my meandering thoughts.  I hope you enjoy the charts.  And remember, don’t believe everything you see in The Economist.

UPDATE:

To show that the statistical bias identified above is substantial, I’ve programmed a quick simulation to measure it.  The simulation makes some assumptions about distributions, and estimates parameters from the original data.  As such it gives only a rough indication of the size of the bias – there are many alternative possibilities, which would lead to larger or smaller biases, especially within a more complex econometric estimation.

In the simulation, each of the advanced countries begins exactly where it was in the year that it first reached an HDI of 0.85.  Thereafter, a trajectory is randomly generated for each country, with zero mean for changes in fertility.  That is, in the simulation, fertility does not increase on average at all¹.  As in the paper, a threshold is found for each country based on the year with lowest TFR within the HDI window.  All shifts in TFR thereafter are used to measure the impact of HDI on TFR (which is actually non-existent).

Here is a sample of the trajectories so generated, along with the fitted response from the paper.

The resulting simulations find, on average, that a 0.06 increase in HDI leads to an increase of about 0.075 in TFR, despite that fact that there is no connection whatsoever.  The range of results is quite broad, with an increase of 0.12 in TFR also being a likely outcome.  This is half of the value found in the paper; in other words, simulations of a simplified case where HDI does not influence TFR at all, can easily generate half of the paper’s result.

Of course, if the result is not due to statistical bias, then the authors can easily prove this.  They need only rerun their analysis using a fixed HDI threshold, rather than one that depends on the path of TFR.  Until they do, their conclusion will remain dubious.

¹ For the technically minded, the HDI follows a random walk with drift and volatility matching those of advanced countries, and the TFR follows an uncorrelated random walk with volatility matching the advanced countries, but with zero drift.  The full source code and results have been uploaded to the Github repository.

FURTHER UPDATE:

More details can be found in the follow-up post to this one, Fertility Declines Don’t Reverse with Development.

I have now uploaded the swine flu data to a Swivel data set. I will update this data set periodically and so the rankings in the chart below should stay reasonably up to date.

Data sources: Guardian Data Blog, CIA World Fact Book.

UPDATE: A number of people have told me that in a number of places, including Victoria and much of the US, testing for swine flu has ceased. This means that the “lab confirmed” swine flu count will become increasingly meaningless over time, so I have decided to stop updating this data.

The Guardian have been publishing swine flu data on their Data Blog. They are sourcing their data from by the World Health Organisation, the US Centers for Disease Control, country health agencies and press reports, which makes their data the most up to date I have found. One thing missing from their data is a sense of scale for each country. Of course populous countries like the US have had a high number of infections, but this also means that  hidden in the smaller numbers are some fairly significant infection rates in countries with smaller national populations.

So, in a return to a theme started in my Olympic medal tally posts, here are the top 20 swine-flu afflicted countries ranked by lab-confirmed infections per million population (as at 15 June 2009).

Mexico was the source of the swine flu outbreak, but is now only ranked 5th in infections per capita, while Australia has rocketed to 4th place, largely due to the extensive outbreak in Victoria. As The Guardian continue to update their data, I will monitor the per capita rankings.

UPDATE: The data has been updated and Panama has now fallen to 4th place, which puts Canada in the lead and Australia in 3rd place. I have now uploaded the data to Swivel.

Photo Credit: johnmuk (Creative Commons)