Methodology

Our Protocol

The curves presented on the site are derived from modelling scenarios that have impacted public policies implemented in France. They were mainly produced by the Pasteur Institute and Inserm. As the data are not available in free access, we have extracted them from the report curves using WebPlotDigitilizer.

To allow for a proper evaluation of the modelling scenarios, only those scenarios with verified assumptions were retained, including the hypotheses regarding the restrictive measures implemented. In order not to introduce bias (e.g., taking only those scenarios that overestimated reality, or the opposite), we studied all the reports. We analysed:

all the models from the Pasteur Institute reports, published on their web page
the models quoted in the Scientific Council reports, which led us to identify Inserm scenarios for January-February 2021
the Imperial College modelling for the 1st lockdown, cited by the Scientific Council which had a major impact on French public policies
a retrospective google search with delimited dates for each month and the keywords “INSERM”, “Pasteur”, “modeling”, “scenario”. This allowed to identify scenarios not officially published but nevertheless cited by the press: Pasteur Institute for the 2nd lockdownof November 2020, and Pasteur Institute for the lockdown exit of the Ile-de-France in May 2020.

Finally, we focus on hospital indicators, and exclude the number of cases from our analysis. We believe that there are two reasons for this exclusion. The first is that the number of cases is a less objective indicator than the number of occupied beds, because it depends in particular on the propensity of the French to be tested, which varies over time. Secondly, the number of cases does not say much about the danger of infections. From January 2022 onwards, the number of cases is 10 times higher than the pre-Omicron figures, without a proportional impact on the hospital.

Comprehensive list of reports

The complete list of modelling scenarios, and whether or not they were included in the site analysis (with the justification) is available below, along with links to the reports in question.

Institut Pasteur Modelling (click to unfold)

April 29, 2020 : included (not officially published but transcribed by the press)
September 25, 2020 : excluded (restricitve measures were taken in September, so the scenarios are no longer comparable with reality)
Octobre 30, 2020 : included (these are the models of the 2nd lockdown, not published on the Institut Pasteur website but transcribed by the press)
February 8, 2021 : included
February 23, 2021 : included
March 29, 2021 : no scenarios in the report
April 6, 2021 : excluded (the curves presented are a counterfactual of the epidemic evolution in the absence of vaccination)
April 26, 2021 : included
May 21, 2021 : included. We did not need to analyze ourselves, the comparison being already carried out in an informal way on twitter by Simon Cauchemez in charge of the modeling team of the Institut Pasteur
July 9, 2021 : excluded (the implementation of the sanitary pass was not included in the modeling assumptions, so the scenarios are no longer comparable with reality)
July 26, 2021 : included (the scenarios take into account the implementation of the sanitary pass)
August 5, 2021 : included (the scenarios take into account the implementation of the sanitary pass)
September 6, 2021 : no scenarios in the report
October 4, 2021 : included
November 29, 2021 : excluded (impact of the Omicron variant 2 weeks later, which was not included in the models). We were still able to compare short-term projections during the short 2-week window before the Omicron variant distorted the results.
December 2, 2021 : excluded (impact of the Omicron variant 2 weeks later, which was not included in the models). We were still able to compare short-term projections during the short 2-week window before the Omicron variant distorted the results.
Decemeber 27, 2021 : included through the report of January 7. Indeed, it is specified that “given the uncertainties […], it is not possible to quantify precisely the impact that the Omicron wave will have”. About a hundred scenarios were then produced. The following report of January 7, which we have analyzed, uses this report as a basis, after excluding the scenarios that do not fit the properties of Omicron.
January 7, 2022 : included
February 15, 2022 : this report does not produce a scenario but a retrospective on the previous report of January 7. We discuss this in our analysis of the latter report.
February 21, 2022 : excluded. Focuses only on the number of cases, which is an exclusionary criterion in our methodology as explained above.
March 10, 2022 : excluded. Focuses only on the number of cases, which is an exclusionary criterion in our methodology as explained above.

INSERM Modelling (click to unfold)

January-February 2021 models: included because they appear in the report of the Scientific Council of January 29, 2021 suggesting the establishment of a strict confinement. They have also been published in the press. These models appeared in the Inserm reports of January 16, 2021, February 2, 2021 and February 14, 2021.

Imperial College Modelling (click to unfold)

modelling of report 12, and in particular its appendix: included as a major argument in favor of lockdown in the report of the Scientific Council of March 12, 2020. It has also been cited by many media as having been a decisive argument in favor of lockdown, e.g. in Le Monde or in Libération