Identifying a faulty turbine in a wind farm, which can require analyzing hundreds of signals and millions of data points, is like looking for a needle in a haystack.
Engineers often optimize this complex problem using deep learning models that can detect anomalies in the measurements taken repeatedly by each turbine over time (called time series data).
However, with hundreds of wind turbines recording dozens of signals every hour, training a deep learning model to analyze time series data is costly and laborious. To make matters worse, the model may need to be retrained after deployment and wind farm operators may lack the necessary machine learning expertise.
In a new study, MIT researchers have found that large language models (LLMs) have the potential to be more efficient anomaly detectors for time series data. Importantly, these pre-trained models are ready to use right out of the box.
The researchers developed a framework called SigLLM, which includes a component that transforms time series data into text-based inputs that an LLM can process. A user can feed this prepared data into the model and tell it to start detecting anomalies. The LLM can also be used to predict future time series data points as part of an anomaly detection pipeline.
While LLMs couldn’t beat state-of-the-art deep learning models at anomaly detection, they performed just as well as some other AI approaches. If researchers can improve the performance of LLMs, this framework could help engineers detect potential problems in equipment like heavy machinery or satellites before they occur, without having to train an expensive deep learning model.
“Since this is only the first iteration, we didn’t expect to get it right on the first try, but these results show that there is an opportunity to use LLMs for complex anomaly detection tasks,” says Sarah Alnegheimish, a doctoral student in electrical engineering and computer science (EECS) and lead author of a paper on SigLLM.
Her co-authors include Linh Nguyen, an EECS student, Laure Berti-Equille, a research director at the French National Research Institute for Sustainable Development, and lead author Kalyan Veeramachaneni, a senior scientist in the Information and Decision Systems Laboratory. The research will be presented at the IEEE Conference on Data Science and Advanced Analytics.
A standard solution
Large language models are autoregressive, meaning they can understand that the latest values in sequential data depend on previous values. For example, models like GPT-4 can predict the next word in a sentence based on the previous words.
Because time series data is sequential, the researchers thought that the autoregressive nature of LLMs might make them well suited to detecting anomalies in this type of data.
However, they wanted to develop a technique that does not require fine-tuning, a process in which engineers retrain a general LLM on a small amount of task-specific data to make it an expert at a particular task. Instead, the researchers use a ready-made LLM, without any additional training steps.
But before they could use it, they had to convert time series data into text-based inputs that the language model could process.
This was achieved through a series of transformations that capture the most important parts of the time series while representing data using the fewest number of tokens. Tokens are the basic inputs to an LLM, and more tokens require more computations.
“If you are not very careful with these steps, you may lose an important part of your data and lose that information,” says Alnegheimish.
After the researchers figured out how to transform time series data, they developed two approaches to anomaly detection.
Approaches to anomaly detection
In the first method, which they call “prompter,” they feed the prepared data into the model and ask it to locate anomalous values.
“We had to do several iterations to figure out the right prompts for a given time series. It’s not easy to understand how these LLMs ingest and process the data,” adds Alnegheimish.
In the second approach, called Detector, they use the LLM as a forecasting tool to predict the next value from a time series. The researchers compare the predicted value with the actual value. A large discrepancy indicates that the actual value is likely an anomaly.
With Detector, the LLM would be part of an anomaly detection pipeline, while Prompter would do the job itself. In practice, Detector performed better than Prompter, which generated many false positives.
“I think with the prompter approach, we have imposed too many hurdles on LLM students. We have given them a more difficult problem,” says Veeramachaneni.
When they compared both approaches to current techniques, Detector outperformed the transformer-based AI models on seven of the eleven datasets evaluated, even though the LLM required neither training nor fine-tuning.
In the future, an LLM could also provide plain language explanations for its predictions so that an operator could better understand why an LLM identified a particular data point as anomalous.
However, modern deep learning models performed significantly better than LLMs, showing that there is still a lot of work to be done before an LLM can be used for anomaly detection.
“What needs to happen to make it perform as well as these state-of-the-art models? That’s the million-dollar question we’re currently asking. An LLM-based anomaly detector needs to be a game-changer for us to make this effort worthwhile,” says Veeramachaneni.
In the future, researchers hope to see if performance can be improved through fine-tuning, but this would require additional time and money, as well as expertise for training.
Their LLM approaches take between 30 minutes and two hours to produce results, so increasing speed is a key area of future work. The researchers also want to study LLMs to understand how they detect anomalies, in the hope of finding a way to improve their performance.
“When it comes to complex tasks like detecting anomalies in time series, LLMs are really a good choice. Maybe LLMs can also be used to tackle other complex tasks?” says Alnegheimish.
This research was supported by SES SA, Iberdrola and ScottishPower Renewables and Hyundai Motor Company.
