Aggregating computational models and human judgment for more accurate forecasts of the US COVID-19 outbreak


COVID-19 was discovered in Wuhan, China in Dec. 2019 and has caused over 13M confirmed infections and 573K deaths world-wide. Government officials at the state and federal level have asked for forecasts to support public health decisions and mitigate the impact of COVID-19. Computational models excel when structured datasets are abundant. But it is difficult to include in these models subjective and unstructured data that is important during an evolving outbreak. We can gain access to this broader set of unstructured data by adding, to an ensemble of computational models, predictions from expert and trained forecasters. Past work has shown human judgment can produce accurate forecasts of infectious disease agents such as: Chikungunya, Seasonal Influenza, and most recently the early COVID-19 outbreak. A hybrid ensemble of computational models and consensus of subject matter experts and trained forecasters (a meta-forecast) is a novel framework that we believe can provide more accurate, more actionable forecasts of the COVID-19 outbreak.

The long-term goal for this work is to build a meta-forecasting toolkit: open source code to combine computational models and human judgment predictions. The overall objective of the proposed research is to combine computational models and predictions from experts and trained forecasters into a single meta- forecast of 1-4 week ahead number of new confirmed cases and number of new deaths of COVID-19 in the US. We hypothesize algorithms to combine computational models and human judgment can (i) improve predictive accuracy of the US COVID-19 outbreak, (ii) offer access to additional data and information computational models alone cannot provide, and (iii) inspire future work and more widespread use if these methods are packaged and standardized. To meet this overall objective and test the above hypotheses, we will pursue the following aims:

  1. Build an ensemble of forecasts from computational models and predictions from expert/trained forecasters.

  2. Provide summary forecasts and reports of the ongoing US COVID-19 outbreak for members of the MIDAS community, members of the CDC and CSTE, and the general public. Summaries will include commentary solicited from subject matter experts and trained forecasters computational models alone cannot offer.

  3. Develop software to produce meta-forecasts and make this code open source and available for the scientific community to build upon our work combining expert judgment and computational models.