People have been having fun with the ‘Get a brain morans!’ meme for a few years.

But I think it was all a misunderstanding based on poor punctuation; he’s simply telling us all to be more mindful of spatial autocorrelation!

In a recent report, the Fraser Institute came up with some numbers estimating the proportion of Canadians that travel abroad for procedures.  In their words:

While there is no readily available data on the number of Canadians travelling abroad for health care, it is possible to produce an estimate of these numbers from data gathered through the Fraser Institute’s Waiting Your Turn survey and from the Canadian Institute.

Here’s the problem

It’s also possible to estimate these numbers by consulting an astrologist or a Ouija board. The important question for me is whether or not the estimates that the Fraser Institute provides are accurate enough to be useful.  Fraser Institute reports on health care have been criticized in the past for not matching provincial data on wait times, and for other issues related to the collection of the data (see this and this as examples from a few years ago).  Furthermore, companies that are in the business of facilitating medical tourism in Canada have not done well, suggesting that there may not be much demand for their services.

Anyone who is interested in understanding or even describing Canadian medical tourism faces a real challenge; there are little data and/or research on who or why people travel for health care.  However, my perspective is that bad data cannot be a place-holder for no data; in other words, we should not resort to answering important questions with bad data just because nothing better is available.

The Fraser Institute data come from self reported survey of physician specialists across Canada.  The response rate is around 20%, but varies across provinces and specialties–the response rates are considerably lower in Ontario and Quebec, for example.  The physicians are asked to estimate the percentage of their patients who have travelled outside of Canada for medical care.  There are several potential sources of error here.  Three of note include:

1. The sample of specialists may not reflect the population of specialists because of a participation bias and the relatively low response rate
2. The physicians may incorrectly recall information–for example, they may be inclined to remember the anomalies (people who travel outside of Canada for care) and over-estimate the percentage as a result
3. While the overall sample size is reasonable, the sample is small across the categories, making inferences difficult

Bad numbers?

In addition, the authors seemed to have made a mistake in the calculation of their statistics.  The title of this table says ‘average percentage of patients receiving treatment outside of Canada, 2014’.

It is inappropriate to calculate an average of per cent values in this way unless all physicians in their sample saw the exact same number of patients.  Instead they should be reporting a weighted average.  Weighting the responses of physicians is important because the purpose here is to understand the percentage overall or in groups, not the specific percentages for each physician.  A physician with 100 patients, 2 of which are medical tourists is contributing less information to the national percentage than a physician with 1000 patients, 20 of which are medical tourists. Assuming that the title is what it says it is (an average percentage and not a weighted average percentage) then the % values in this table are all wrong by some amount, and possibly by a large amount.

My conclusion

When the (apparent) miscalculation in the table above is combined with the systematic and random errors associated with these data, I would not put much faith in the findings of this report.  Medical tourism does happen, but it seems to occur infrequently enough that it shouldn’t play much role in any serious discussion of health policy.  The fact that this report was widely discussed by the media (and some politicians) shouldn’t be used as evidence medical tourism is common or increasing in frequency.