- Tags:: 🗞️Articles , Forecasting, Prophet
- Author:: Jules Bertrand, Hugo Vasselin (DS @ Artefact, a DS consulting company)
- Link:: Is Facebook Prophet suited for doing good predictions in a real-world project? | by Hugo Vasselin | Artefact Engineering and Data Science | Mar, 2022 | Medium
- Source date:: 2022-03-17
- Finished date:: 2022-03-24
Another interesting Prophet article that does a deep dive on using it for real world predictions. Prophet gives you something okeish quickly, but requires careful monitoring and intervention.
The dangerous trend
The trend explains the majority of the prediction and, as pointed out in Is Facebooks Prophet the Time-Series Messiah, or Just a Very Naughty Boy, it is dangerous. And also, only explainable by external factors. Small values of changepoint_prior_scale lead to better generalization.
Prophet by default does not introduce changepoints in the last 20% of the series, precisely to avoid dangerous inferences of the trend (by modeling the slope of the last piece of the trend with a small number of points at the end), but in this case, it worked better for them to push this frontier further.
Weak weekly seasonality
They claim that weekly seasonality is limited. They didn’t have good results modeling weekly seasonality, particularly to model rapid drops on Sundays. Not sure about their explanation: they claim that increasing the Fourier order of the weekly seasonality is not useful because fast variations of the signal won’t be captured if the signal is only sampled 7 times. But you are not wanting to capture many rapid variations, only a sharp drop on Saturday, so the sampling shouldn’t be a problem there.
Would love to know more about their solution:
stacking two models, including our Prophet model and a statistical model computing predictions based on the average weight of week days over the last 6 weeks.
Holidays
A well known problem pointed out by them is:
it is often difficult to know whether a recurring event is already included in prophet’s seasonality or whether it needs to be flagged manually to help the algorithm.
And not sure either about their solution:
We observed similar issues with various public holidays that are moving around every year, and we sometimes had to stack Prophet with a statistical model to better fit holidays’ effects. However, we do not have any consistent solution to offer.
You will need to feature cross yourself:
Example: let’s say that you have an impact on your time-series at Business Day + 1 after a holiday (HBD+1). Then providing Prophet with an event “HBD+1” will allow it to better fit this effect. But here, the impact depends largely on whether the day HBD+1 is on a Monday, Saturday etc. And It is not linear nor multiplicative. Hence you’ll have to add all the features “HBD+1_monday”, “HBD+1_tuesday”,…
And as the author says, other methods don’t really require that (e.g., tree-based inference), but everything is a tradeoff: you then might have very high run times cart - Pre-computing feature crosses when using XGBoost? - Cross Validated (stackexchange.com)
History length
Another well known problem of Prophet is the inability to put weight on observations:
Interestingly, sometimes adding more historical data can be counterproductive and lead to a drop in forecast accuracy. Prophet does not allow to put more weight on the most recent observations, so more data can lead to a decrease of forecast accuracy.
Hyperparams and instability
Regarding hyperparameters:
Re-training the model every month had better results. In conclusion, despite the temporal cross-validation the hyperparameters were not stable across time.
I wonder how many months the author include (I suspect that if the CV does not have a whole year, this is more likely. Also, COVID as the author points out).
Alternatives
If you want to provide some explainability and build a decent model easily and quickly, Prophet is an option to consider. However if you care about stability and forecast accuracy, consider using another kind of algorithm, such as tree-based models, Orbit or DeepAR.