Jenny Garcia, Catalina Torres, Magali Barbieri, Carlo Giovanni Camarda, Emmanuelle Cambois, Arianna Caporali, France Meslé, Svitlana Poniakina and Jean-Marie Robine

tell us about the article "Differences in COVID-19 Mortality: Implications of Imperfect and Diverse Data Collection Systems".

(Interview conducted in June 2021)

The COVID-19 pandemic demanded that a new data collection system be developed or at least that existing systems be expeditiously updated. Collected data vary not only by country but within countries as collection gradually improves and coverage becomes more extensive. This process has induced artifactual changes in pandemic patterns, thus making the available real-time demographic data imperfect. To avoid introducing biases in international comparisons, any analysis of COVID-19 statistics must factor in variations in data coverage and representativeness. The obstacles to international comparisons have most to do with official data heterogeneity on three points: (a) data definitions; e.g. cause of death, testing strategies, case-confirmation mechanism, and consideration of “probable cases,” 2) data collection, e.g., system type, coverage by place of death, verification, and reporting time lag, and 3) data publication, e.g., reference date and frequency.

For example, early on in the pandemic, Belgium’s national public health institute, Sciensano, only reported PCR-confirmed deaths counts (deaths whose cause had been confirmed by laboratory testing). As the pandemic developed, death counts began to include presumed or probable cases, following symptoms and/or documented contact with a positive case. Of the 9,765 deaths attributed to COVID-19 in Belgium as of July 2, 2020, only 60% (5,828) had been confirmed by PCR testing. In the Netherlands, the National Institute for Public Health and the Environment (Rijksinstituut voor Volksgezondheid en Milieu [RIVM]) published official COVID-19 death counts that include only laboratory-confirmed cases. And from the beginning of the pandemic to June 30, 2020, 6,113 deaths were attributed to COVID-19. However, over the same period, the vital statistics system of Statistic Netherlands (Centraal Bureau voor de Statistiek [CBC]), reported 7,797 deaths confirmed by a positive laboratory test and an additional 2,270 suspected COVID-19 deaths. Had the Netherlands applied the Belgian definition that counts both laboratory-confirmed and suspected cases, the country would therefore have reported about 30% more deaths than it did. 

Given the cross-country variations in data collection and reporting, it is essential to consider the particularities of COVID-19 mortality data so as to better interpret results from any statistical or demographic analyses. 

Despite the numerous data limitations we identified in conjunction with international comparison, we have also shown that meaningful information from country comparisons can be extracted when using imperfect statistics. International comparisons can be made for countries with similar data. We defined three data groups by characteristics observed during the first COVID-19 wave, when testing availability and testing strategies varied greatly by country and few countries were publishing civil registration data. These groups differ by the completeness of their death counts, as follows: comprehensive, conservative, or restricted counts. Using this classification, we were able to rank 16 countries in these groups using available data up to September 2020. 

National vital statistics systems are the source of the most comprehensive data: exhaustive, standardized and verified mortality information for the population as a whole. Official COVID-19 death counts based on those statistics are not highly dependent on a country’s testing capacity. Although surveillance systems and health agencies allow for the more rapid production of estimates of COVID-19 death counts by facilitating daily monitoring of pandemic trends and pace, their counts are relatively conservative, e.g., counting only laboratory-confirmed deaths due to COVID-19, or restricted, e.g., counting only hospital inpatient deaths. The underreporting inherent in these types of data collection systems is determined by the share of deaths that occur within and are registered by a given health system; similarly, the share of in-hospital deaths is directly related to the extent of the given system’s resources (number of hospitals, hospital beds, and in intensive care beds, etc.).

Excess male mortality from COVID-19 is found in all the countries studied. It is highly comparable to what is found for death from all causes; similarly, the slightly higher excess mortality found for men in France compared with other countries is of the same magnitude as that observed for all deaths. This can be explained primarily by the fact that men are more vulnerable to death than women due to their more frequent risk behaviors while women are more attentive to their health than men. It is too early to tell whether behavioral differences between men and women in conjunction with COVID-19—e.g., social distancing and vaccination—together with inequalities in risk exposure had the effect of reinforcing or, on the contrary, reducing the excess male mortality observed in non-crisis conditions.