In 2014/15 an Ebola outbreak of unprecedented dimensions afflicted the West African countries of Liberia, Guinea, and Sierra Leone. We performed a systematic review of manuscripts that forecasted the outbreak while it was occurring, and derive implications on the ways results could be interpreted by policy-makers. We reviewed 26 manuscripts, published between 2014 and April 2015, that presented forecasts of the West African Ebola outbreak. Forecasted case counts varied widely. An important determinant of forecast accuracy for case counts was how far into the future predictions were made. Generally, those that made forecasts less than 2 months into the future tended to be more accurate than those that made forecasts more than 10 weeks into the future. The exceptions were parsimonious statistical models in which the decay of the rate of spread of the pathogen among susceptible individuals was dealt with explicitly. Regarding future outbreaks, the most important lessons for policy makers when using similar modeling results are: i) uncertainty of forecasts will be higher in the beginning of the outbreak, ii) when data are limited, forecasts produced by models designed to inform specific decisions should be used in complimentary fashion for robust decision making - for this outbreak, two statistical models produced the most reliable case counts forecasts, but did not allow to understand the impact of interventions, while several compartmental models could estimate the impact of interventions but required data that was not available; iii) timely collection of essential data is necessary for optimal model use.