How to Teach your Anomaly Detection System to Correlate Abnormal Behavior

Abnormal data trends rarely occur on their own. Influencing or related metrics are usually involved. For example, let’s say that a remote data center goes offline and doesn’t come back up. The anomaly in this case isn’t just a power failure, it’s a power failure plus a failure in a backup generator. 

Some systems might show you one of these anomalies, leaving you to search for other affected metrics which can take hours, days or even weeks. Correlation, on the other hand, instantly lists related anomalies so you can quickly and painlessly understand what the leading dimension is and which metrics are impacted.

If you don’t use anomaly detection, you won’t understand the cause of your outage until a support crew reaches the site. With anomaly detection, however, you can quickly discover both related anomalies, making it that much easier for you to get back online. 

Finding related metrics and anomalies

Behavioral topology learning provides a method for data scientists to understand relationships between millions of metrics at scale. This lets them combine related anomalies into stories, mitigate errors and examine their root causes. When implemented correctly, this system can filter out unrelated metrics from the results for greater accuracy.

There are several methods within ...


Read More on Datafloq