The primary objective of this blog is to demonstrate techniques used to customize open-source large language models (LLMs) for analyzing time series data and evaluate the effectiveness of Generative AI (GenAI) based pipelines in enhancing data analytics platforms such as Splunk, Grafana, and InfluxDB. In this blog, we aim to provide a comprehensive exploration of using small language models (SLMs) in predictive maintenance and data analytics tasks, offering actionable insights into their integration strategies, data processing capabilities, and performance across different hardware platforms. We will delve into practical use-cases that demonstrate how LLMs can enhance data-driven decision-making and drive innovation across industries. We will also dive into the critical process of synthetic data preparation and its role in fine-tuning open-source foundational language models like Mistral and Llama. The discussion will include a detailed analysis of embedding LLMs into existing 3rd party analytics ecosystems, addressing both operational challenges and technical implementation steps, and will culminate in a comparative performance assessment of inference pipelines running on RISC-V based Esperanto Chip (ET-SoC-1) energy efficient accelerators and traditional GPU architectures, evaluating key metrics such as execution time, power consumption, memory usage, throughput, and accuracy. 

The current landscape of industrial error management and predictive maintenance is fraught with significant challenges that impede operational efficiency and technological advancement. Existing systems suffer from inefficient error categorization processes that heavily rely on manual user intervention, leading to time-consuming and error-prone debugging scenarios. Traditional predictive models often provide superficial machine failure predictions without delving into the underlying root causes of equipment malfunctions. Moreover, data analysis remains complex and inaccessible, requiring specialized expertise to navigate and interpret intricate data visualizations, which creates substantial barriers to actionable insights.

In response to these challenges, our proposed solution leverages customized language models to revolutionize operational workflows and predictive maintenance strategies. By automating error categorization and providing sophisticated debugging solutions, we aim to dramatically reduce the resources and time traditionally spent on manual troubleshooting. Our innovative pipeline transcends conventional prediction models by offering deep, contextual insights into potential failure mechanisms, thereby facilitating more accurate forecasting and preemptive maintenance actions that can significantly reduce operational downtime and associated costs. Central to this approach is an AI-driven interactive question-answer system integrated directly into the dashboard, which empowers users to extract immediate, context-aware insights without requiring advanced technical interpretation skills.

We use a Generative AI based pipeline to develop a comprehensive preventive maintenance system which monitors and detects the health of the Esperanto Server Platforms. By identifying critical monitoring parameters such as system temperature, power consumption, computational throughput, and hardware component performance metrics, we develop a sophisticated approach to breaking down complex operational insights into granular, actionable components. The pipeline involves collecting real-world and synthetic training data and employing data augmentation techniques to create robust, representative datasets that can effectively train SLMs. Our methodology focuses on developing a holistic framework for assessing server health and efficiency, incorporating uni-modal data analysis techniques that leverage open-source large language models to transform raw performance metrics into intelligible, predictive insights. The validation process rigorously evaluates model performance through comprehensive benchmarking, examining key metrics like inference accuracy, computational efficiency, and the model’s ability to generalize across diverse operational scenarios.

We propose a GenAI based inference pipeline as outlined in Figure 1. The operational data is collected from respective Esperanto servers and then preprocessed. This preprocessed data is fed into a fine tuned “Cause Analyzer LLM” Mistral based LLM model, which identifies errors, categorizes them, and suggests debugging solutions. The enriched output is then inputted into another Mistral fine tuned LLM called the “Time Series Predictor LLM”, which forecasts future failure probabilities and pinpoints potential causes. Both LLMs run on high-performance accelerators, such as GPUs or Esperanto cards, ensuring efficient processing.

Figure 1: Proposed GenAI Based Time Series Data Analytics Pipeline

The predictions generated by the Time Series Predictor and pre-processed data are stored in InfluxDB, a specialized time-series database. For real-time analysis and operational monitoring, this data is visualized on a Splunk dashboard, where it is accessible through interactive graphs and charts. Additionally, a sophisticated Question and Answer system, hosted on an accelerator—either GPU or Esperanto card—facilitates dynamic interaction with the dashboard, providing users with immediate, context-aware responses to inquiries, enhancing decision-making processes.

The rationale for choosing Large Language Models (LLMs) in predictive maintenance is rooted in their unparalleled ability to process and comprehend complex textual data. Unlike traditional machine learning models, LLMs excel at extracting and synthesizing nuanced information from diverse sources such as error logs, maintenance records, and operational notes.

When compared to traditional machine learning models, LLMs demonstrate significant advantages in handling diverse data types. While conventional models typically excel in numerical data analysis, LLMs can understand and generate human-like text, providing a holistic approach to data interpretation. Their transfer learning capabilities allow them to leverage pre-trained models across vast datasets, enabling more efficient and adaptable fine-tuning for specific tasks.

The selection of InfluxDB and Splunk as complementary tools in this inference pipeline was strategic and purposeful. InfluxDB was chosen for its exceptional performance in handling time-series data, offering rapid write and retrieval capabilities that are crucial for real-time performance monitoring. Splunk complements this by providing powerful data aggregation, analysis, and visualization tools that can process large volumes of log data and generate insightful, real-time monitoring dashboards. The integration of these tools can be achieved through direct API connections, middleware solutions, and custom scripting, each offering unique advantages in facilitating seamless data communication and processing.

This section describes how our fine tuned LLMs interact with and leverage data from InfluxDB and how the processed insights are subsequently visualized and analyzed using Splunk. The integration ensures a seamless flow of data and enables real-time analytics capabilities.

A client is instantiated with parameters such as URL, token, and organization details for InfluxDB, allowing secure and customized access to the database. Data is queried from InfluxDB using a Flux query. This step involves retrieving the required datasets for analysis, structured to include essential metrics and identifiers. Before sending data to Splunk, it is essential to ensure it adheres to the format Splunk expects. This might include converting timestamps to ISO 8601 format and handling null values. Proper authentication headers are set up to ensure secure data transfer to Splunk, utilizing an authorization token. The LLM is hosted on a dedicated server, ensuring reliable performance and availability. An API endpoint is created to facilitate communication between the LLM and the Splunk dashboard, allowing the LLM to receive and process queries. A custom app is developed within Splunk, which acts as the intermediary, handling requests and responses between the Splunk user interface and the LLM. The script is configured in the ‘commands.conf' file within Splunk. This configuration ensures that any user query entered on the dashboard triggers the LLM API to fetch the response. Responses generated by the LLM are stored in a designated Splunk index. This approach ensures that all interactions and outputs are logged. Responses are retrieved using Splunk Processing Language (SPL) and displayed on the dashboard. This setup allows users to see real-time results from their queries.

And finally, we develop a prototype dashboard which provides a comprehensive and intuitive insight to server performance we are monitoring, featuring a time-series graph with dynamic color coding that instantly communicates server efficiency levels. This visualization is complemented by an interactive component impact analysis feature to drill down into specific server elements, providing granular insights into individual component contributions to overall system performance. To further enhance user experience and data accessibility, the dashboard integrates an advanced LLM-powered question-answering. Underpinning these features is a robust, customizable alert system designed to proactively notify users of significant efficiency fluctuations, thus transforming server monitoring from a reactive to a predictive and strategic operational approach.

Figure 2: Prototype Dashboard Showing the Implementation of the Analytics Pipeline Under the Hood

Based on the inference pipeline above for implementing an Esperanto server based preventive maintenance system, we would need to train 2 separate LLMs: a first model which will identifies errors, categorizes them, and suggests debugging solutions and a second model which forecasts future failure probabilities and pinpoints potential causes.

Figure 3: Overall Training Pipeline

For the first “causal model”, we collected log data from Esperanto servers. The data below is Esperanto server logs which has a manually created debugging cheatsheet containing error and cause of different error symptoms.

Figure 4: Sample Labeled Error Symptom / Cause

For cases where the labels were not available, we utilized GPT-4 to label the data with appropriate error classifications. This process involved feeding GPT-4 contextual information specific to our server operations and trying different prompt to ensure that the labeling was both accurate and relevant.

GPT-4 was also used to generate synthetic logs that resemble our server logs. Additionally, we employed back-translation technique to increase the number of logs from 85 to 170. This method is particularly effective in diversifying the linguistic expressions without altering the underlying meaning.

The dataset was split into a 90:10 ratio for training and validation. The table below shows an example of the generated dataset

Figure 5: Sample Generated Dataset

The fine-tuning was conducted using the Mistral 7B and Llama-3 8B models, chosen for their effectiveness in handling the complexity of log data. The primary objective during fine-tuning was to enhance the model’s precision in categorizing errors and diagnosing causes, tailored to the nuances of data from Esperanto servers. Sample LLM prompt used: For the given log_error from my server data, Categorize the log into errors, Identify the Cause and Provide Debugging Solution

Figure 6: Model’s Learning Progress

The plots shown provide a clear visual representation of the model’s learning progress over five training epochs. The right graph, depicting training loss, shows a significant reduction in loss from initially near 1 to less than 0.4, indicating that the model is learning from the training data. The left graph, showing evaluation loss, also depicts a steady decrease, dropping from about 0.6 to around 0.3, which suggests that the model is generalizing without overfitting.

We then employed cosine similarity to evaluate the model’s accuracy, focusing on how closely the model’s output for predicted causes aligns with the manually labeled causes.

Figure 17: ET-SoC-1 Against GPU (A100) Runtime Metric

The landscape of data analytics is being transformed by large language models through several key innovations. Researchers are focusing on streamlining the integration of LLM pipelines into existing third-party data analytics frameworks, which enables more seamless technological adoption. Simultaneously, advanced functionalities are being developed to implement interactive question-and-answer systems within these tools, allowing for deeper and more nuanced insights. A critical approach to enhancing LLM effectiveness involves utilizing synthetic data for specialized training. Moreover, ongoing performance comparisons are revealing scenarios where LLMs demonstrate superior capabilities compared to traditional time series analysis algorithms, suggesting a significant potential for disruption in data analytics methodologies.

As part of this effort, we exhibited how small language models can be integrated into existing data analytics platforms. We developed integration techniques that incorporate LLM pipelines into existing third-party frameworks, enabling a more fluid and dynamic analytical ecosystem. By implementing a question-and-answer system within these tools, we’ve transformed traditional data analytics into an interactive and insightful experience. Our approach includes training methodologies utilizing synthetic data, which allows us to tailor LLMs for specific vertical use-cases to enhance their applicability and precision.

1. Jin, M., Wang, S., Ma, L., Chu, Z., Zhang, J. Y., Shi, X., Chen, P.-Y., Liang, Y., Li, Y.-F., Pan, S., et al.Time-llm: Time series forecasting by reprogramming large language models.In International Conference on Machine Learning, 2024.

2. Lopez-Lira, A. and Tang, Y.Can chatgpt forecast stock price movements? return predictability and large language models.arXiv preprint arXiv:2304.07619, 2023.

3. Liu, X., Hu, J., Li, Y., Diao, S., Liang, Y., Hooi, B., and Zimmermann, R.Unitime: A language-empowered unified model for cross-domain time series http://forecasting.In The Web Conference 2024 (WWW), 2024b.

• Fine Tuned Cause Analyzer Model

• Fine Tuned Time Series Predictor Model

• Time-LLM: Repogram an LLM for Time Series Forecasting

• Leveraging the Power of Large Language Models (LLMs) to Drive Progress in the Manufacturing Sector

• How LangChain Agents Improve Time Series Analysis?

• Predictive Maintenance and Generative AI | Waylay Blog

• LLM Applications are Revolutionizing the Manufacturing Industry

• https://docs.splunk.com/Documentation/Splunk/9.2.2/SearchTutorial/Createnewdashboard