Time series forecasting has been a cornerstone of enterprise resource planning for decades. Predictions about future demand are essential for making critical decisions, such as determining inventory levels, hiring labor, capital investments in production and fulfillment infrastructure, and pricing goods and services. However, these forecasts are rarely perfect, and during the 2010s, many organizations reported forecast accuracies of only 50-60% due to computational limitations and limited access to advanced forecasting capabilities.
With the widespread adoption of cloud computing and improved accessibility to external data sources such as weather and event data, organizations began to see improvements in forecast accuracy. Now, as we enter the era of generative AI, a new class of models known as time series transformers is poised to take these improvements even further. These models, similar to large language models like ChatGPT, predict the next value in a numerical sequence by identifying subtle patterns in data.
Understanding Time Series Transformers
Generative AI models are a form of deep neural network, a complex machine learning architecture that combines numerous inputs to arrive at a predicted value. The specialized architecture of these models, known as transformers, is particularly effective at recognizing complex relationships between values in long sequences.
To train a time series transformer, a deep neural network is exposed to large volumes of time series data, allowing it to learn complex patterns of relationships. Once trained, the model can predict new values in previously unseen sequences, using the knowledge gained during training.
Addressing Common Time Series Challenges
While the concept of time series transformers is promising, forecasting practitioners may face three common challenges. First, time series may operate on different scales, which is addressed through scaling techniques that enable the model to recognize patterns regardless of the original scale. Second, daily, weekly, and annual seasonal patterns must be considered, achieved through a process called self-attention that allows the model to identify the influence of prior values on a future value. Finally, many time series are influenced by external factors, and some transformer models incorporate these variables into the forecasting process.
A Look at Four Popular Time Series Transformer Models
-
Chronos: Developed by Amazon, this open-source model takes a simplistic approach by interpreting time series as a specialized language with its own relational patterns. Despite its basic approach, it has demonstrated impressive results as a general-purpose forecasting solution.
-
TimesFM: Created by Google Research, this model offers fine-grained control over how inputs and outputs are organized, affecting the detection of seasonal patterns and computation times. It is a powerful and flexible tool for time series forecasting.
-
Moirai: Developed by Salesforce AI Research, this universal model supports missing values and external variables. It allows tuning the model to the seasonal patterns in datasets by adjusting the size of data patches, showing strong performance against other models.
-
TimeGPT: A proprietary model that supports external variables but not missing values. It focuses on ease of use, allowing organizations to generate forecasts with just a single line of code via a public API.
Getting Started with Transformer Forecasting on Databricks
With so many model options available, the key question is how to start evaluating them using proprietary data. Databricks, known for its scalability and efficient use of cloud resources, has been a popular platform for forecasting. The introduction of this new class of models simply expands the options for performing forecasts in this environment.
To help organizations understand how to use these models in Databricks, a series of notebooks has been created to demonstrate how to generate forecasts with each of the four models mentioned above. These notebooks can be downloaded and used in the Databricks environment, providing a foundation for organizations to adapt the code to other similar models and leverage generative AI in their resource planning processes.
Get started with forecasting modeling on Databricks today with this series of notebooks and discover how generative AI can improve the accuracy of your forecasts.
