Master Thesis: Streamable Multivariate Time Series Anomaly Detection

Location

Bretten | Germany

Job description

Topic

Streamable Multivariate Time Series Anomaly Detection for Cloud Service Infrastructures

Motivation and Goals 

Automatic anomaly detection is an important tool for monitoring complex cloud service infrastructures for B2B communications. Multivariate anomalies here arise simultaneously from a variety of metrics and the context of individual services. A changing workload may be related to the number of successful processes, the elimination of processing errors, and declining orders from a discount retailer.

In operation, previously unknown or rare errors occur, comparatively few anomalies can be labeled by experts, and data for training ML models are insufficiently cleaned of anomalies. The goal of this work is to develop a stream-oriented, multivariate anomaly detector and an alert communication system, as well as to evaluate the system on the example of a cloud service infrastructure with the provided data.

Tasks

• Investigation and evaluation of different approaches for anomaly detection with a focus on Deep Neural Networks.
• Pre-processing, filtering, cleaning, as well as enrichment of monitoring data, message tracking data, and the cloud structure data for the anomaly detector. Here, message tracking captures metadata as documents are processed with the various cloud services. Historical data is available for several years in a data lake. Further time series are to be generated from the metadata
• Development and implementation of the AI anomaly detector as well as a framework for the regular training of the ML models and the stream-oriented detection of anomalies
• Development and implementation of a dynamic alert system suitable for different users such as system operators or customers, as well as analysis and evaluation of the anomalies
• Development of criteria for the evaluation of the system

Contact Recruiting:

Daniel Iwtschenko

Job tags

Salary