![\"Image\"](\"https:\/\/www.artefact.com\/\/wp-content\/uploads\/2022\/03\/hugo-vasselin.jpeg\")
![\"Image\"](\"https:\/\/www.artefact.com\/\/wp-content\/uploads\/2022\/03\/jules-bertrand.jpeg\")
![](\"https:\/\/www.artefact.com\/\/wp-content\/uploads\/2021\/04\/Medium-Blog.png\" "\\"Medium")
<\/span><\/div>We tested Prophet in a real-world project, on 3 main aspects: feature engineering, interpretability, and use in production & maintenance and here are our conclusions:<\/p>\n<\/div>
Feature engineering and modelling:<\/h3><\/div>- <\/i><\/span>A native handling of trend and seasonality features, that makes Prophet a good baseline model if the time series follows business cycles.<\/div><\/li>
- <\/i><\/span>However if your signal is noisy, fine-tuning the model\u2019s performance can be a hassle.<\/div><\/li><\/ul>
Interpretability:<\/h3><\/div>- <\/i><\/span>Thanks to its native breakdown of the forecast into time components, the model\u2019s decisions are easy to interpret<\/div><\/li>
- <\/i><\/span>Nevertheless it\u2019s not precise enough to be used to measure the impact of an external event<\/div><\/li><\/ul>
Stability of the results \/ Maintenance:<\/h3><\/div>- <\/i><\/span>Prophet provides an interpretable model with good performance in a very short time<\/div><\/li>
- <\/i><\/span>However if you care about stability and forecast accuracy, consider using another kind of algorithm, such as tree-based models.<\/div><\/li><\/ul>
Introduction<\/h2><\/div>Recently,\u00a0an article<\/a>\u00a0pointed out that\u00a0Facebook Prophet<\/a>\u00a0was performing poorly on some edge cases. Indeed, this package is so popular that it tends to be considered as THE go-to tool for every forecasting use case. This launched a debate within the community and\u00a0even its creator reacted<\/a>.
\nThere is a lot of literature on how to make a single forecast with Prophet. But there is a lack of empirical feedback on how it behaves in production, when you need to provide forecasts on a daily basis. We have tested and proven this tool for 6 months on a real business project. Here are some takeaways.<\/p>\n<\/div>We used Prophet to forecast call arrivals in call centers<\/a> for one of the biggest Telecom companies in Europe, to optimise the quality of customer service.<\/p>\nWe applied the\u00a0GLADS framework<\/a>\u00a0to draft our data approach:<\/p>\n<\/div>- <\/i><\/span>Target<\/strong>: number of incoming calls<\/div><\/li>
- <\/i><\/span>Granularity<\/strong>: day \/ geographic zone \/ agent skills level (eg. cell phone technical assistance).<\/div><\/li>
- <\/i><\/span>Horizon<\/strong>:\u00a05 months (M+5)<\/div><\/li>
- <\/i><\/span>Frequency:<\/strong> daily refresh<\/div><\/li>
- <\/i><\/span>\n
SKU:<\/strong>\u00a0agent skills<\/p>\n<\/div><\/li><\/ul>An important constraint was the need for\u00a0interpretability<\/strong>\u00a0of results. Indeed, our predictions are consumed by human planners whose goal is to maintain a SLA (X% of calls must be answered in a given period) while minimising costs. Their work consists of 3 missions:<\/p>\n<\/div>- <\/i><\/span>Analysing<\/strong>\u00a0and understanding the main call drivers and their evolution<\/div><\/li>
- <\/i><\/span>\n
Forecasting <\/strong>call volumes using their business knowledge<\/p>\n<\/div><\/li>- <\/i><\/span>\n
Allocating<\/strong> \u2014\u00a0<\/strong>Adjusting the supply of call centre agents according to demand prediction.<\/p>\n<\/div><\/li><\/ul>Main\u00a0call drivers<\/strong> differ depending on SKUs.\u00a0For example, Commercial assistance drivers include:<\/p>\n<\/div>- <\/i><\/span>Number of clients<\/div><\/li>
- <\/i><\/span>Marketing events and campaign pruning<\/div><\/li>
- <\/i><\/span>Level of quality of service (snowball effect)<\/div><\/li>
- <\/i><\/span>\n
Holidays<\/p>\n<\/div><\/li><\/ul>
Our time series have strong seasonalities, and follow economic cycles. They are not stationary. In this case, Prophet is an appropriate choice. We chose to test it, as well as other ML algorithms\u00a0frequently used for this task<\/a>.<\/p>\n<\/div>![\"Image\"](\"data:image\/svg+xml,%3Csvg%20xmlns%3D%27http%3A%2F%2Fwww.w3.org%2F2000%2Fsvg%27%20width%3D%27587%27%20height%3D%270%27%20viewBox%3D%270%200%20587%200%27%3E%3Crect%20width%3D%27587%27%20height%3D%270%27%20fill-opacity%3D%220%22%2F%3E%3C%2Fsvg%3E\")
Feature engineering and modelling<\/h2><\/div>Prophet has many undeniable advantages that are especially useful for a business-oriented forecasting project, and which fully justify its popularity.<\/p>\n
One of them is its\u00a0ease of use. <\/strong>Only a few lines of code and\u00a0almost no feature engineering<\/strong>\u00a0are needed to have a good baseline. To learn more about how it works and how to use it, read\u00a0this Twitter thread<\/a>\u00a0from its creator, as well as\u00a0this excellent blog post<\/a>\u00a0(with\u00a0code<\/a>).<\/p>\n<\/div>Time features, <\/strong>such as trend and seasonality\u00a0<\/strong>are created natively \u2014 Exit the rolling means, lags, and other tricky features required by machine learning (ML) tree-based models.<\/p>\nHowever it can be quite a hassle to fine-tune if multiple events disturb the signal.<\/p>\n<\/div>
Trend<\/h2><\/div>A great advantage compared to autoregressive models (eg. ARIMA) is that Prophet\u00a0doesn\u2019t require stationary time series<\/strong>: a trend component is generated natively.<\/p>\nFor these reasons, Prophet\u00a0works quite well for middle-term predictions<\/strong>\u00a0(we tried it at M+5 horizon), even though the confidence interval is quite large.<\/p>\n<\/div>Trend can be correctly estimated\u00a0without any external data<\/strong>\u00a0(such as the number of customers). This is quite powerful, because when you predict call arrivals, you don\u2019t know how many customers you will have in 5 months, so you cannot use this feature.
\nFor instance, this plot shows the number of Fiber-Optic Internet (FIO) customers over time (a growing market for our telecom company), compared to the trend component learned by Prophet on call volumes. There is a strong correlation (Pearson coeff.: 0.988) between the volume of calls and the number of clients. So the model\u00a0learns the trend well<\/strong>, and only from the time-series itself.<\/p>\n<\/div>![\"Image\"](\"data:image\/svg+xml,%3Csvg%20xmlns%3D%27http%3A%2F%2Fwww.w3.org%2F2000%2Fsvg%27%20width%3D%27520%27%20height%3D%270%27%20viewBox%3D%270%200%20520%200%27%3E%3Crect%20width%3D%27520%27%20height%3D%270%27%20fill-opacity%3D%220%22%2F%3E%3C%2Fsvg%3E\")
We identified some tips and tricks to make the trend a real benefit in your Prophet based forecasting.<\/p>\n<\/div>
Trend: Watch your changepoints<\/em><\/h3><\/div>One of the strengths of Prophet is also a weakness for the stability of the results and the performance: The\u00a0trend component tends to explain the vast majority of the prediction<\/strong> \u2014\u00a0<\/strong>around 90% for our case study. Thus, if the trend is not well estimated, performance will dramatically drop and you can lose over 20 points of MAPE by over or under predicting over time. You have multiple parameters to help Prophet adjust the trend, including the `changepoint_prior_scale` and the `changepoint_range`<\/p>\n<\/div>Finding the right changepoint_prior_scale value is essential<\/strong>, because this parameter sets the flexibility of the trend. The higher it is, the more flexible the trend. Our strategy is to grid search this parameter with these values (in a logarithmic scale): [0.001, 0.01, 0.1, 0.5]. Having relatively small values will generally lead to better generalisation.<\/p>\n<\/div>Trend: Make the most out of the most recent data<\/em><\/h3><\/div>![\"Image\"](\"data:image\/svg+xml,%3Csvg%20xmlns%3D%27http%3A%2F%2Fwww.w3.org%2F2000%2Fsvg%27%20width%3D%27636%27%20height%3D%270%27%20viewBox%3D%270%200%20636%200%27%3E%3Crect%20width%3D%27636%27%20height%3D%270%27%20fill-opacity%3D%220%22%2F%3E%3C%2Fsvg%3E\")
Next point of attention: by default Prophet takes into accountonly the first 80% of the history <\/strong>to\u00a0approximate the trend<\/a>\u00a0by specifying 25 potential changepoints. We found that increasing this value of 80% by changing the changepoint_range parameter was improving performance. Indeed this will also take into account the latest data, which are more important to explain the level of calls in our case. For instance in this graph you can see that 2021 data is important because it has a decreasing trend. We chose to take into account the whole training set except the last 7 days instead of 80%.<\/p>\n<\/div>Trend: Clean your outliers<\/em><\/h3><\/div>Another thing to consider is the sensitivity to outliers<\/strong>. The official documentation\u00a0recommends removing them<\/a>: \u201cProphet is able to handle the outliers in the history, but only by fitting them with trend changes\u201d.<\/p>\n<\/div>Seasonalities<\/h2><\/div>Seasonality components,<\/strong>\u00a0such as daily and yearly, are estimated with Fourier series. Because our time series have strong seasonal business cycles, we found that Prophet works quite well.<\/p>\nBut when we deep dive a bit, we found that\u00a0weekly seasonality modelling\u00a0is limited<\/strong>. Our time-series had very low Saturdays compared to the rest of the week (and no values on Sundays). We spent a lot of time and had a lot of trouble trying to improve our predictions on this particular day of the week.<\/p>\n<\/div>We Finally had to use a workaround:\u00a0stacking two models, including our Prophet model and a statistical model<\/strong>\u00a0computing predictions based on the average weight of week days over the last 6 weeks. And this despite having tried to change all parameters in Prophet\u2019s weekly seasonality.<\/p>\nOur understanding: On a 7-days-period seasonality (6 in our case without Sundays), Increasing the Fourier order is not that useful, because the interval between 0 and 2pi is only divided in a few segments (6 or 7). And a low Fourier order means that it cannot fit extreme values and rapid changes<\/strong>\u00a0in seasonality, like Saturdays in the example above.<\/p>\nSome other tricks to fine-tune seasonalities:<\/p>\n<\/div>
- <\/i><\/span>Fits better with a multiplicative mode<\/div><\/li>
- <\/i><\/span>Add custom seasonalities<\/div><\/li>
- <\/i><\/span>Change Fourrier\u2019s order<\/div><\/li><\/ul>
Events and extra regressors<\/h2><\/div>For\u00a0events\u00a0and extra-regressors\u00a0<\/strong>as well Prophet can be handy.<\/p>\n
Interpretability:<\/h3><\/div>- <\/i><\/span>Thanks to its native breakdown of the forecast into time components, the model\u2019s decisions are easy to interpret<\/div><\/li>
- <\/i><\/span>Nevertheless it\u2019s not precise enough to be used to measure the impact of an external event<\/div><\/li><\/ul>
Stability of the results \/ Maintenance:<\/h3><\/div>- <\/i><\/span>Prophet provides an interpretable model with good performance in a very short time<\/div><\/li>
- <\/i><\/span>However if you care about stability and forecast accuracy, consider using another kind of algorithm, such as tree-based models.<\/div><\/li><\/ul>
Introduction<\/h2><\/div>Recently,\u00a0an article<\/a>\u00a0pointed out that\u00a0Facebook Prophet<\/a>\u00a0was performing poorly on some edge cases. Indeed, this package is so popular that it tends to be considered as THE go-to tool for every forecasting use case. This launched a debate within the community and\u00a0even its creator reacted<\/a>.
\nThere is a lot of literature on how to make a single forecast with Prophet. But there is a lack of empirical feedback on how it behaves in production, when you need to provide forecasts on a daily basis. We have tested and proven this tool for 6 months on a real business project. Here are some takeaways.<\/p>\n<\/div>We used Prophet to forecast call arrivals in call centers<\/a> for one of the biggest Telecom companies in Europe, to optimise the quality of customer service.<\/p>\nWe applied the\u00a0GLADS framework<\/a>\u00a0to draft our data approach:<\/p>\n<\/div>- <\/i><\/span>Target<\/strong>: number of incoming calls<\/div><\/li>
- <\/i><\/span>Granularity<\/strong>: day \/ geographic zone \/ agent skills level (eg. cell phone technical assistance).<\/div><\/li>
- <\/i><\/span>Horizon<\/strong>:\u00a05 months (M+5)<\/div><\/li>
- <\/i><\/span>Frequency:<\/strong> daily refresh<\/div><\/li>
- <\/i><\/span>\n
SKU:<\/strong>\u00a0agent skills<\/p>\n<\/div><\/li><\/ul>An important constraint was the need for\u00a0interpretability<\/strong>\u00a0of results. Indeed, our predictions are consumed by human planners whose goal is to maintain a SLA (X% of calls must be answered in a given period) while minimising costs. Their work consists of 3 missions:<\/p>\n<\/div>- <\/i><\/span>Analysing<\/strong>\u00a0and understanding the main call drivers and their evolution<\/div><\/li>
- <\/i><\/span>\n
Forecasting <\/strong>call volumes using their business knowledge<\/p>\n<\/div><\/li>- <\/i><\/span>\n
Allocating<\/strong> \u2014\u00a0<\/strong>Adjusting the supply of call centre agents according to demand prediction.<\/p>\n<\/div><\/li><\/ul>Main\u00a0call drivers<\/strong> differ depending on SKUs.\u00a0For example, Commercial assistance drivers include:<\/p>\n<\/div>- <\/i><\/span>Number of clients<\/div><\/li>
- <\/i><\/span>Marketing events and campaign pruning<\/div><\/li>
- <\/i><\/span>Level of quality of service (snowball effect)<\/div><\/li>
- <\/i><\/span>\n
Holidays<\/p>\n<\/div><\/li><\/ul>
Our time series have strong seasonalities, and follow economic cycles. They are not stationary. In this case, Prophet is an appropriate choice. We chose to test it, as well as other ML algorithms\u00a0frequently used for this task<\/a>.<\/p>\n<\/div>Feature engineering and modelling<\/h2><\/div>Prophet has many undeniable advantages that are especially useful for a business-oriented forecasting project, and which fully justify its popularity.<\/p>\n
One of them is its\u00a0ease of use. <\/strong>Only a few lines of code and\u00a0almost no feature engineering<\/strong>\u00a0are needed to have a good baseline. To learn more about how it works and how to use it, read\u00a0this Twitter thread<\/a>\u00a0from its creator, as well as\u00a0this excellent blog post<\/a>\u00a0(with\u00a0code<\/a>).<\/p>\n<\/div>Time features, <\/strong>such as trend and seasonality\u00a0<\/strong>are created natively \u2014 Exit the rolling means, lags, and other tricky features required by machine learning (ML) tree-based models.<\/p>\nHowever it can be quite a hassle to fine-tune if multiple events disturb the signal.<\/p>\n<\/div>
Trend<\/h2><\/div>A great advantage compared to autoregressive models (eg. ARIMA) is that Prophet\u00a0doesn\u2019t require stationary time series<\/strong>: a trend component is generated natively.<\/p>\nFor these reasons, Prophet\u00a0works quite well for middle-term predictions<\/strong>\u00a0(we tried it at M+5 horizon), even though the confidence interval is quite large.<\/p>\n<\/div>Trend can be correctly estimated\u00a0without any external data<\/strong>\u00a0(such as the number of customers). This is quite powerful, because when you predict call arrivals, you don\u2019t know how many customers you will have in 5 months, so you cannot use this feature.
\nFor instance, this plot shows the number of Fiber-Optic Internet (FIO) customers over time (a growing market for our telecom company), compared to the trend component learned by Prophet on call volumes. There is a strong correlation (Pearson coeff.: 0.988) between the volume of calls and the number of clients. So the model\u00a0learns the trend well<\/strong>, and only from the time-series itself.<\/p>\n<\/div>We identified some tips and tricks to make the trend a real benefit in your Prophet based forecasting.<\/p>\n<\/div>
Trend: Watch your changepoints<\/em><\/h3><\/div>One of the strengths of Prophet is also a weakness for the stability of the results and the performance: The\u00a0trend component tends to explain the vast majority of the prediction<\/strong> \u2014\u00a0<\/strong>around 90% for our case study. Thus, if the trend is not well estimated, performance will dramatically drop and you can lose over 20 points of MAPE by over or under predicting over time. You have multiple parameters to help Prophet adjust the trend, including the `changepoint_prior_scale` and the `changepoint_range`<\/p>\n<\/div>Finding the right changepoint_prior_scale value is essential<\/strong>, because this parameter sets the flexibility of the trend. The higher it is, the more flexible the trend. Our strategy is to grid search this parameter with these values (in a logarithmic scale): [0.001, 0.01, 0.1, 0.5]. Having relatively small values will generally lead to better generalisation.<\/p>\n<\/div>Trend: Make the most out of the most recent data<\/em><\/h3><\/div>Next point of attention: by default Prophet takes into accountonly the first 80% of the history <\/strong>to\u00a0approximate the trend<\/a>\u00a0by specifying 25 potential changepoints. We found that increasing this value of 80% by changing the changepoint_range parameter was improving performance. Indeed this will also take into account the latest data, which are more important to explain the level of calls in our case. For instance in this graph you can see that 2021 data is important because it has a decreasing trend. We chose to take into account the whole training set except the last 7 days instead of 80%.<\/p>\n<\/div>Trend: Clean your outliers<\/em><\/h3><\/div>Another thing to consider is the sensitivity to outliers<\/strong>. The official documentation\u00a0recommends removing them<\/a>: \u201cProphet is able to handle the outliers in the history, but only by fitting them with trend changes\u201d.<\/p>\n<\/div>Seasonalities<\/h2><\/div>Seasonality components,<\/strong>\u00a0such as daily and yearly, are estimated with Fourier series. Because our time series have strong seasonal business cycles, we found that Prophet works quite well.<\/p>\nBut when we deep dive a bit, we found that\u00a0weekly seasonality modelling\u00a0is limited<\/strong>. Our time-series had very low Saturdays compared to the rest of the week (and no values on Sundays). We spent a lot of time and had a lot of trouble trying to improve our predictions on this particular day of the week.<\/p>\n<\/div>We Finally had to use a workaround:\u00a0stacking two models, including our Prophet model and a statistical model<\/strong>\u00a0computing predictions based on the average weight of week days over the last 6 weeks. And this despite having tried to change all parameters in Prophet\u2019s weekly seasonality.<\/p>\nOur understanding: On a 7-days-period seasonality (6 in our case without Sundays), Increasing the Fourier order is not that useful, because the interval between 0 and 2pi is only divided in a few segments (6 or 7). And a low Fourier order means that it cannot fit extreme values and rapid changes<\/strong>\u00a0in seasonality, like Saturdays in the example above.<\/p>\nSome other tricks to fine-tune seasonalities:<\/p>\n<\/div>
- <\/i><\/span>Fits better with a multiplicative mode<\/div><\/li>
- <\/i><\/span>Add custom seasonalities<\/div><\/li>
- <\/i><\/span>Change Fourrier\u2019s order<\/div><\/li><\/ul>
Events and extra regressors<\/h2><\/div>For\u00a0events\u00a0and extra-regressors\u00a0<\/strong>as well Prophet can be handy.<\/p>\n
Stability of the results \/ Maintenance:<\/h3><\/div>- <\/i><\/span>Prophet provides an interpretable model with good performance in a very short time<\/div><\/li>
- <\/i><\/span>However if you care about stability and forecast accuracy, consider using another kind of algorithm, such as tree-based models.<\/div><\/li><\/ul>
Introduction<\/h2><\/div>Recently,\u00a0an article<\/a>\u00a0pointed out that\u00a0Facebook Prophet<\/a>\u00a0was performing poorly on some edge cases. Indeed, this package is so popular that it tends to be considered as THE go-to tool for every forecasting use case. This launched a debate within the community and\u00a0even its creator reacted<\/a>.
\nThere is a lot of literature on how to make a single forecast with Prophet. But there is a lack of empirical feedback on how it behaves in production, when you need to provide forecasts on a daily basis. We have tested and proven this tool for 6 months on a real business project. Here are some takeaways.<\/p>\n<\/div>We used Prophet to forecast call arrivals in call centers<\/a> for one of the biggest Telecom companies in Europe, to optimise the quality of customer service.<\/p>\nWe applied the\u00a0GLADS framework<\/a>\u00a0to draft our data approach:<\/p>\n<\/div>- <\/i><\/span>Target<\/strong>: number of incoming calls<\/div><\/li>
- <\/i><\/span>Granularity<\/strong>: day \/ geographic zone \/ agent skills level (eg. cell phone technical assistance).<\/div><\/li>
- <\/i><\/span>Horizon<\/strong>:\u00a05 months (M+5)<\/div><\/li>
- <\/i><\/span>Frequency:<\/strong> daily refresh<\/div><\/li>
- <\/i><\/span>\n
SKU:<\/strong>\u00a0agent skills<\/p>\n<\/div><\/li><\/ul>An important constraint was the need for\u00a0interpretability<\/strong>\u00a0of results. Indeed, our predictions are consumed by human planners whose goal is to maintain a SLA (X% of calls must be answered in a given period) while minimising costs. Their work consists of 3 missions:<\/p>\n<\/div>- <\/i><\/span>Analysing<\/strong>\u00a0and understanding the main call drivers and their evolution<\/div><\/li>
- <\/i><\/span>\n
Forecasting <\/strong>call volumes using their business knowledge<\/p>\n<\/div><\/li>- <\/i><\/span>\n
Allocating<\/strong> \u2014\u00a0<\/strong>Adjusting the supply of call centre agents according to demand prediction.<\/p>\n<\/div><\/li><\/ul>Main\u00a0call drivers<\/strong> differ depending on SKUs.\u00a0For example, Commercial assistance drivers include:<\/p>\n<\/div>- <\/i><\/span>Number of clients<\/div><\/li>
- <\/i><\/span>Marketing events and campaign pruning<\/div><\/li>
- <\/i><\/span>Level of quality of service (snowball effect)<\/div><\/li>
- <\/i><\/span>\n
Holidays<\/p>\n<\/div><\/li><\/ul>
Our time series have strong seasonalities, and follow economic cycles. They are not stationary. In this case, Prophet is an appropriate choice. We chose to test it, as well as other ML algorithms\u00a0frequently used for this task<\/a>.<\/p>\n<\/div>Feature engineering and modelling<\/h2><\/div>Prophet has many undeniable advantages that are especially useful for a business-oriented forecasting project, and which fully justify its popularity.<\/p>\n
One of them is its\u00a0ease of use. <\/strong>Only a few lines of code and\u00a0almost no feature engineering<\/strong>\u00a0are needed to have a good baseline. To learn more about how it works and how to use it, read\u00a0this Twitter thread<\/a>\u00a0from its creator, as well as\u00a0this excellent blog post<\/a>\u00a0(with\u00a0code<\/a>).<\/p>\n<\/div>Time features, <\/strong>such as trend and seasonality\u00a0<\/strong>are created natively \u2014 Exit the rolling means, lags, and other tricky features required by machine learning (ML) tree-based models.<\/p>\nHowever it can be quite a hassle to fine-tune if multiple events disturb the signal.<\/p>\n<\/div>
Trend<\/h2><\/div>A great advantage compared to autoregressive models (eg. ARIMA) is that Prophet\u00a0doesn\u2019t require stationary time series<\/strong>: a trend component is generated natively.<\/p>\nFor these reasons, Prophet\u00a0works quite well for middle-term predictions<\/strong>\u00a0(we tried it at M+5 horizon), even though the confidence interval is quite large.<\/p>\n<\/div>Trend can be correctly estimated\u00a0without any external data<\/strong>\u00a0(such as the number of customers). This is quite powerful, because when you predict call arrivals, you don\u2019t know how many customers you will have in 5 months, so you cannot use this feature.
\nFor instance, this plot shows the number of Fiber-Optic Internet (FIO) customers over time (a growing market for our telecom company), compared to the trend component learned by Prophet on call volumes. There is a strong correlation (Pearson coeff.: 0.988) between the volume of calls and the number of clients. So the model\u00a0learns the trend well<\/strong>, and only from the time-series itself.<\/p>\n<\/div>We identified some tips and tricks to make the trend a real benefit in your Prophet based forecasting.<\/p>\n<\/div>
Trend: Watch your changepoints<\/em><\/h3><\/div>One of the strengths of Prophet is also a weakness for the stability of the results and the performance: The\u00a0trend component tends to explain the vast majority of the prediction<\/strong> \u2014\u00a0<\/strong>around 90% for our case study. Thus, if the trend is not well estimated, performance will dramatically drop and you can lose over 20 points of MAPE by over or under predicting over time. You have multiple parameters to help Prophet adjust the trend, including the `changepoint_prior_scale` and the `changepoint_range`<\/p>\n<\/div>Finding the right changepoint_prior_scale value is essential<\/strong>, because this parameter sets the flexibility of the trend. The higher it is, the more flexible the trend. Our strategy is to grid search this parameter with these values (in a logarithmic scale): [0.001, 0.01, 0.1, 0.5]. Having relatively small values will generally lead to better generalisation.<\/p>\n<\/div>Trend: Make the most out of the most recent data<\/em><\/h3><\/div>Next point of attention: by default Prophet takes into accountonly the first 80% of the history <\/strong>to\u00a0approximate the trend<\/a>\u00a0by specifying 25 potential changepoints. We found that increasing this value of 80% by changing the changepoint_range parameter was improving performance. Indeed this will also take into account the latest data, which are more important to explain the level of calls in our case. For instance in this graph you can see that 2021 data is important because it has a decreasing trend. We chose to take into account the whole training set except the last 7 days instead of 80%.<\/p>\n<\/div>Trend: Clean your outliers<\/em><\/h3><\/div>Another thing to consider is the sensitivity to outliers<\/strong>. The official documentation\u00a0recommends removing them<\/a>: \u201cProphet is able to handle the outliers in the history, but only by fitting them with trend changes\u201d.<\/p>\n<\/div>Seasonalities<\/h2><\/div>Seasonality components,<\/strong>\u00a0such as daily and yearly, are estimated with Fourier series. Because our time series have strong seasonal business cycles, we found that Prophet works quite well.<\/p>\nBut when we deep dive a bit, we found that\u00a0weekly seasonality modelling\u00a0is limited<\/strong>. Our time-series had very low Saturdays compared to the rest of the week (and no values on Sundays). We spent a lot of time and had a lot of trouble trying to improve our predictions on this particular day of the week.<\/p>\n<\/div>We Finally had to use a workaround:\u00a0stacking two models, including our Prophet model and a statistical model<\/strong>\u00a0computing predictions based on the average weight of week days over the last 6 weeks. And this despite having tried to change all parameters in Prophet\u2019s weekly seasonality.<\/p>\nOur understanding: On a 7-days-period seasonality (6 in our case without Sundays), Increasing the Fourier order is not that useful, because the interval between 0 and 2pi is only divided in a few segments (6 or 7). And a low Fourier order means that it cannot fit extreme values and rapid changes<\/strong>\u00a0in seasonality, like Saturdays in the example above.<\/p>\nSome other tricks to fine-tune seasonalities:<\/p>\n<\/div>
- <\/i><\/span>Fits better with a multiplicative mode<\/div><\/li>
- <\/i><\/span>Add custom seasonalities<\/div><\/li>
- <\/i><\/span>Change Fourrier\u2019s order<\/div><\/li><\/ul>
Events and extra regressors<\/h2><\/div>For\u00a0events\u00a0and extra-regressors\u00a0<\/strong>as well Prophet can be handy.<\/p>\n
Introduction<\/h2><\/div>Recently,\u00a0an article<\/a>\u00a0pointed out that\u00a0Facebook Prophet<\/a>\u00a0was performing poorly on some edge cases. Indeed, this package is so popular that it tends to be considered as THE go-to tool for every forecasting use case. This launched a debate within the community and\u00a0even its creator reacted<\/a>.
\nThere is a lot of literature on how to make a single forecast with Prophet. But there is a lack of empirical feedback on how it behaves in production, when you need to provide forecasts on a daily basis. We have tested and proven this tool for 6 months on a real business project. Here are some takeaways.<\/p>\n<\/div>We used Prophet to forecast call arrivals in call centers<\/a> for one of the biggest Telecom companies in Europe, to optimise the quality of customer service.<\/p>\nWe applied the\u00a0GLADS framework<\/a>\u00a0to draft our data approach:<\/p>\n<\/div>- <\/i><\/span>Target<\/strong>: number of incoming calls<\/div><\/li>
- <\/i><\/span>Granularity<\/strong>: day \/ geographic zone \/ agent skills level (eg. cell phone technical assistance).<\/div><\/li>
- <\/i><\/span>Horizon<\/strong>:\u00a05 months (M+5)<\/div><\/li>
- <\/i><\/span>Frequency:<\/strong> daily refresh<\/div><\/li>
- <\/i><\/span>\n
SKU:<\/strong>\u00a0agent skills<\/p>\n<\/div><\/li><\/ul>An important constraint was the need for\u00a0interpretability<\/strong>\u00a0of results. Indeed, our predictions are consumed by human planners whose goal is to maintain a SLA (X% of calls must be answered in a given period) while minimising costs. Their work consists of 3 missions:<\/p>\n<\/div>- <\/i><\/span>Analysing<\/strong>\u00a0and understanding the main call drivers and their evolution<\/div><\/li>
- <\/i><\/span>\n
Forecasting <\/strong>call volumes using their business knowledge<\/p>\n<\/div><\/li>- <\/i><\/span>\n
Allocating<\/strong> \u2014\u00a0<\/strong>Adjusting the supply of call centre agents according to demand prediction.<\/p>\n<\/div><\/li><\/ul>Main\u00a0call drivers<\/strong> differ depending on SKUs.\u00a0For example, Commercial assistance drivers include:<\/p>\n<\/div>- <\/i><\/span>Number of clients<\/div><\/li>
- <\/i><\/span>Marketing events and campaign pruning<\/div><\/li>
- <\/i><\/span>Level of quality of service (snowball effect)<\/div><\/li>
- <\/i><\/span>\n
Holidays<\/p>\n<\/div><\/li><\/ul>
Our time series have strong seasonalities, and follow economic cycles. They are not stationary. In this case, Prophet is an appropriate choice. We chose to test it, as well as other ML algorithms\u00a0frequently used for this task<\/a>.<\/p>\n<\/div>Feature engineering and modelling<\/h2><\/div>Prophet has many undeniable advantages that are especially useful for a business-oriented forecasting project, and which fully justify its popularity.<\/p>\n
One of them is its\u00a0ease of use. <\/strong>Only a few lines of code and\u00a0almost no feature engineering<\/strong>\u00a0are needed to have a good baseline. To learn more about how it works and how to use it, read\u00a0this Twitter thread<\/a>\u00a0from its creator, as well as\u00a0this excellent blog post<\/a>\u00a0(with\u00a0code<\/a>).<\/p>\n<\/div>Time features, <\/strong>such as trend and seasonality\u00a0<\/strong>are created natively \u2014 Exit the rolling means, lags, and other tricky features required by machine learning (ML) tree-based models.<\/p>\nHowever it can be quite a hassle to fine-tune if multiple events disturb the signal.<\/p>\n<\/div>
Trend<\/h2><\/div>A great advantage compared to autoregressive models (eg. ARIMA) is that Prophet\u00a0doesn\u2019t require stationary time series<\/strong>: a trend component is generated natively.<\/p>\nFor these reasons, Prophet\u00a0works quite well for middle-term predictions<\/strong>\u00a0(we tried it at M+5 horizon), even though the confidence interval is quite large.<\/p>\n<\/div>Trend can be correctly estimated\u00a0without any external data<\/strong>\u00a0(such as the number of customers). This is quite powerful, because when you predict call arrivals, you don\u2019t know how many customers you will have in 5 months, so you cannot use this feature.
\nFor instance, this plot shows the number of Fiber-Optic Internet (FIO) customers over time (a growing market for our telecom company), compared to the trend component learned by Prophet on call volumes. There is a strong correlation (Pearson coeff.: 0.988) between the volume of calls and the number of clients. So the model\u00a0learns the trend well<\/strong>, and only from the time-series itself.<\/p>\n<\/div>We identified some tips and tricks to make the trend a real benefit in your Prophet based forecasting.<\/p>\n<\/div>
Trend: Watch your changepoints<\/em><\/h3><\/div>One of the strengths of Prophet is also a weakness for the stability of the results and the performance: The\u00a0trend component tends to explain the vast majority of the prediction<\/strong> \u2014\u00a0<\/strong>around 90% for our case study. Thus, if the trend is not well estimated, performance will dramatically drop and you can lose over 20 points of MAPE by over or under predicting over time. You have multiple parameters to help Prophet adjust the trend, including the `changepoint_prior_scale` and the `changepoint_range`<\/p>\n<\/div>Finding the right changepoint_prior_scale value is essential<\/strong>, because this parameter sets the flexibility of the trend. The higher it is, the more flexible the trend. Our strategy is to grid search this parameter with these values (in a logarithmic scale): [0.001, 0.01, 0.1, 0.5]. Having relatively small values will generally lead to better generalisation.<\/p>\n<\/div>Trend: Make the most out of the most recent data<\/em><\/h3><\/div>Next point of attention: by default Prophet takes into accountonly the first 80% of the history <\/strong>to\u00a0approximate the trend<\/a>\u00a0by specifying 25 potential changepoints. We found that increasing this value of 80% by changing the changepoint_range parameter was improving performance. Indeed this will also take into account the latest data, which are more important to explain the level of calls in our case. For instance in this graph you can see that 2021 data is important because it has a decreasing trend. We chose to take into account the whole training set except the last 7 days instead of 80%.<\/p>\n<\/div>Trend: Clean your outliers<\/em><\/h3><\/div>Another thing to consider is the sensitivity to outliers<\/strong>. The official documentation\u00a0recommends removing them<\/a>: \u201cProphet is able to handle the outliers in the history, but only by fitting them with trend changes\u201d.<\/p>\n<\/div>Seasonalities<\/h2><\/div>Seasonality components,<\/strong>\u00a0such as daily and yearly, are estimated with Fourier series. Because our time series have strong seasonal business cycles, we found that Prophet works quite well.<\/p>\nBut when we deep dive a bit, we found that\u00a0weekly seasonality modelling\u00a0is limited<\/strong>. Our time-series had very low Saturdays compared to the rest of the week (and no values on Sundays). We spent a lot of time and had a lot of trouble trying to improve our predictions on this particular day of the week.<\/p>\n<\/div>We Finally had to use a workaround:\u00a0stacking two models, including our Prophet model and a statistical model<\/strong>\u00a0computing predictions based on the average weight of week days over the last 6 weeks. And this despite having tried to change all parameters in Prophet\u2019s weekly seasonality.<\/p>\nOur understanding: On a 7-days-period seasonality (6 in our case without Sundays), Increasing the Fourier order is not that useful, because the interval between 0 and 2pi is only divided in a few segments (6 or 7). And a low Fourier order means that it cannot fit extreme values and rapid changes<\/strong>\u00a0in seasonality, like Saturdays in the example above.<\/p>\nSome other tricks to fine-tune seasonalities:<\/p>\n<\/div>
- <\/i><\/span>Fits better with a multiplicative mode<\/div><\/li>
- <\/i><\/span>Add custom seasonalities<\/div><\/li>
- <\/i><\/span>Change Fourrier\u2019s order<\/div><\/li><\/ul>
Events and extra regressors<\/h2><\/div>For\u00a0events\u00a0and extra-regressors\u00a0<\/strong>as well Prophet can be handy.<\/p>\n
Recently,\u00a0an article<\/a>\u00a0pointed out that\u00a0Facebook Prophet<\/a>\u00a0was performing poorly on some edge cases. Indeed, this package is so popular that it tends to be considered as THE go-to tool for every forecasting use case. This launched a debate within the community and\u00a0even its creator reacted<\/a>. We used Prophet to forecast call arrivals in call centers<\/a> for one of the biggest Telecom companies in Europe, to optimise the quality of customer service.<\/p>\n We applied the\u00a0GLADS framework<\/a>\u00a0to draft our data approach:<\/p>\n<\/div> SKU:<\/strong>\u00a0agent skills<\/p>\n<\/div><\/li><\/ul> An important constraint was the need for\u00a0interpretability<\/strong>\u00a0of results. Indeed, our predictions are consumed by human planners whose goal is to maintain a SLA (X% of calls must be answered in a given period) while minimising costs. Their work consists of 3 missions:<\/p>\n<\/div> Forecasting <\/strong>call volumes using their business knowledge<\/p>\n<\/div><\/li> Allocating<\/strong> \u2014\u00a0<\/strong>Adjusting the supply of call centre agents according to demand prediction.<\/p>\n<\/div><\/li><\/ul> Main\u00a0call drivers<\/strong> differ depending on SKUs.\u00a0For example, Commercial assistance drivers include:<\/p>\n<\/div> Holidays<\/p>\n<\/div><\/li><\/ul> Our time series have strong seasonalities, and follow economic cycles. They are not stationary. In this case, Prophet is an appropriate choice. We chose to test it, as well as other ML algorithms\u00a0frequently used for this task<\/a>.<\/p>\n<\/div> Prophet has many undeniable advantages that are especially useful for a business-oriented forecasting project, and which fully justify its popularity.<\/p>\n One of them is its\u00a0ease of use. <\/strong>Only a few lines of code and\u00a0almost no feature engineering<\/strong>\u00a0are needed to have a good baseline. To learn more about how it works and how to use it, read\u00a0this Twitter thread<\/a>\u00a0from its creator, as well as\u00a0this excellent blog post<\/a>\u00a0(with\u00a0code<\/a>).<\/p>\n<\/div> Time features, <\/strong>such as trend and seasonality\u00a0<\/strong>are created natively \u2014 Exit the rolling means, lags, and other tricky features required by machine learning (ML) tree-based models.<\/p>\n However it can be quite a hassle to fine-tune if multiple events disturb the signal.<\/p>\n<\/div> A great advantage compared to autoregressive models (eg. ARIMA) is that Prophet\u00a0doesn\u2019t require stationary time series<\/strong>: a trend component is generated natively.<\/p>\n For these reasons, Prophet\u00a0works quite well for middle-term predictions<\/strong>\u00a0(we tried it at M+5 horizon), even though the confidence interval is quite large.<\/p>\n<\/div> Trend can be correctly estimated\u00a0without any external data<\/strong>\u00a0(such as the number of customers). This is quite powerful, because when you predict call arrivals, you don\u2019t know how many customers you will have in 5 months, so you cannot use this feature. We identified some tips and tricks to make the trend a real benefit in your Prophet based forecasting.<\/p>\n<\/div> One of the strengths of Prophet is also a weakness for the stability of the results and the performance: The\u00a0trend component tends to explain the vast majority of the prediction<\/strong> \u2014\u00a0<\/strong>around 90% for our case study. Thus, if the trend is not well estimated, performance will dramatically drop and you can lose over 20 points of MAPE by over or under predicting over time. You have multiple parameters to help Prophet adjust the trend, including the `changepoint_prior_scale` and the `changepoint_range`<\/p>\n<\/div> Finding the right changepoint_prior_scale value is essential<\/strong>, because this parameter sets the flexibility of the trend. The higher it is, the more flexible the trend. Our strategy is to grid search this parameter with these values (in a logarithmic scale): [0.001, 0.01, 0.1, 0.5]. Having relatively small values will generally lead to better generalisation.<\/p>\n<\/div> Next point of attention: by default Prophet takes into accountonly the first 80% of the history <\/strong>to\u00a0approximate the trend<\/a>\u00a0by specifying 25 potential changepoints. We found that increasing this value of 80% by changing the changepoint_range parameter was improving performance. Indeed this will also take into account the latest data, which are more important to explain the level of calls in our case. For instance in this graph you can see that 2021 data is important because it has a decreasing trend. We chose to take into account the whole training set except the last 7 days instead of 80%.<\/p>\n<\/div> Another thing to consider is the sensitivity to outliers<\/strong>. The official documentation\u00a0recommends removing them<\/a>: \u201cProphet is able to handle the outliers in the history, but only by fitting them with trend changes\u201d.<\/p>\n<\/div> Seasonality components,<\/strong>\u00a0such as daily and yearly, are estimated with Fourier series. Because our time series have strong seasonal business cycles, we found that Prophet works quite well.<\/p>\n But when we deep dive a bit, we found that\u00a0weekly seasonality modelling\u00a0is limited<\/strong>. Our time-series had very low Saturdays compared to the rest of the week (and no values on Sundays). We spent a lot of time and had a lot of trouble trying to improve our predictions on this particular day of the week.<\/p>\n<\/div> We Finally had to use a workaround:\u00a0stacking two models, including our Prophet model and a statistical model<\/strong>\u00a0computing predictions based on the average weight of week days over the last 6 weeks. And this despite having tried to change all parameters in Prophet\u2019s weekly seasonality.<\/p>\n Our understanding: On a 7-days-period seasonality (6 in our case without Sundays), Increasing the Fourier order is not that useful, because the interval between 0 and 2pi is only divided in a few segments (6 or 7). And a low Fourier order means that it cannot fit extreme values and rapid changes<\/strong>\u00a0in seasonality, like Saturdays in the example above.<\/p>\n Some other tricks to fine-tune seasonalities:<\/p>\n<\/div> For\u00a0events\u00a0and extra-regressors\u00a0<\/strong>as well Prophet can be handy.<\/p>\n
\nThere is a lot of literature on how to make a single forecast with Prophet. But there is a lack of empirical feedback on how it behaves in production, when you need to provide forecasts on a daily basis. We have tested and proven this tool for 6 months on a real business project. Here are some takeaways.<\/p>\n<\/div>Feature engineering and modelling<\/h2><\/div>
Trend<\/h2><\/div>
\nFor instance, this plot shows the number of Fiber-Optic Internet (FIO) customers over time (a growing market for our telecom company), compared to the trend component learned by Prophet on call volumes. There is a strong correlation (Pearson coeff.: 0.988) between the volume of calls and the number of clients. So the model\u00a0learns the trend well<\/strong>, and only from the time-series itself.<\/p>\n<\/div>Trend: Watch your changepoints<\/em><\/h3><\/div>
Trend: Make the most out of the most recent data<\/em><\/h3><\/div>
Trend: Clean your outliers<\/em><\/h3><\/div>
Seasonalities<\/h2><\/div>
Events and extra regressors<\/h2><\/div>
![\"Image\"](\"https:\/\/www.artefact.com\/\/wp-content\/uploads\/2022\/03\/article-hugo1.png\")
![\"Image\"](\"https:\/\/www.artefact.com\/\/wp-content\/uploads\/2022\/03\/article-hugo2.png\")
![\"Image\"](\"https:\/\/www.artefact.com\/\/wp-content\/uploads\/2022\/03\/article-hugo3.png\")
![\"Image\"](\"https:\/\/www.artefact.com\/\/wp-content\/uploads\/2022\/03\/article-hugo4.png\")
![\"Image\"](\"https:\/\/www.artefact.com\/\/wp-content\/uploads\/2022\/03\/article-hugo5.png\")