COVID-19 mortality: the delicate exercise of international comparison

As early as April 2020, INED opened an internet platform for collecting and analyzing international data on the demography of COVID-19 deaths. International comparison of those data is crucial for studying the dynamic of the pandemic and the effect of countries’ different health policies. Each country set up its own counting system, which in turn evolved as the epidemic wore on, making it difficult to compare the mortality differences observed over time and space. It is nonetheless possible to compare groups of countries on the basis of a study using INED data from fifteen European countries, the United States, and the Republic of Korea. Analysis of data from the first wave of the pandemic brings to light considerable differences depending on the criteria countries used; comparing groups of countries allows for a more nuanced assessment of disparities in COVID-19 mortality levels.

Most obstacles to international comparison arise from data heterogeneity

To track the effects of the epidemic, each country had to design new data collection systems, or at least speedily adapt its existing systems. This means that the data collected differ not only between countries but also within them over time—a process that induced artificial changes in pandemic trends and generated imperfect real-time demographic data. It is hard to assess what proportion of mortality is attributable to epidemiological variations. For example, early on in the pandemic in France, in-hospital deaths were the only ones taken into account in the daily death count; that count rose considerably after nursing home deaths were added. Specifically, on April 21, 2020, the total number of hospital deaths, which had already grown to 12,900, jumped to 20,796 when nursing home deaths were taken into account; the new figure put France at the same mortality level as Spain and Italy. All analyses of COVID-19 statistics must take account three key points: (a) data definitions; e.g. cause of death, testing strategies, case-confirmation mechanism, and consideration of “probable cases”; 2) data collection, e.g., system type, coverage by place of death, verification, and reporting time lag; and 3) data publication, e.g., reference date and frequency.

Considerable disparities by country criteria 

At the start of the pandemic, countries reported deaths as being due to COVID-19 only if they had been confirmed by laboratory test. As the pandemic continued, some countries began counting “probable cases” and adding them to their mortality statistics. For example, Belgium initially reported only deaths whose cause was confirmed by a PCR test, whereas later on their count came to include presumed or probable cases. Of the 9,765 deaths attributed to COVID-19 in Belgium as of July 2, 2020, only 60% (5,828) had been confirmed by PCR testing. If the Netherlands had applied the same method of analysis as Belgium (i.e., counting both laboratory-confirmed and suspected cases), it would have reported about 30% more deaths than it did. This example illustrates the need to take account of the particularities of each country’s data and remain vigilant with regard to analyses of COVID-19 mortality.

Classifying countries into groups for relevant international comparisons

Countries can be ranked by how complete their first-wave counts were. National vital statistics systems furnish the most comprehensive information: exhaustive, standardized, verified mortality data on the population as a whole; countries using this type of statistics constitute the first group. (The other two groups are countries whose counts were based on conservative data sources and countries whose counts were based on restricted data sources; see Figure 1). For example, comparative mortality indices—which compare the number of COVID-19 deaths found in a given group with the number that would be obtained using the norm of age- and sex-specific mortality rates (established on the basis of a reference population)—differ in France by data source: rates obtained using Santé Publique France (SpF) hospital data are one-third lower than those using civil registry statistics from the Centre d’Épidemiologie sur les Causes Médicales de Décès (CépiDC) (Figure 1).

Disparities in mortality levels

It has often been said that older persons account for most of the excess mortality from COVID-19. Indeed, the wider the data coverage, the greater the excess mortality observed in the 80-89 age bracket. The study also confirms excess male adult mortality due to COVID-19, with the highest shares found in France, England, and Wales and the lowest in Belgium and Scotland. This excess mortality is highly comparable to that found for mortality from all causes. The differences highlight the utility of calculating COVID-19 mortality rates by age and sex to obtain more relevant international comparison indicators.

The worldwide COVID pandemic has revealed the importance of national vital statistics systems and the information they contain, as well as the importance of making that information available quickly.

Figure 1 - COVID-19 comparative mortality indices by country and data source type as of July 15, 2020