How to Detect Infrastructure Change with Baseline Deviation Alerts


Steve Mahoney, SevOne's Product Line Manager, discusses standard deviation as it relates to performance monitoring metrics. Learn how to detect infrastructure changes by setting baseline deviation alerts.


Hi, I am Steve Mahoney and I manage SevOne’s newest product line or performance log appliance. Today, I’d like to talk a little bit about standard deviation. Lot of folks ask me, what standard deviation is? How it applies to SevOne and what value it can give to us when it comes to performance statistics? First, I want to talk a little bit about what standard deviation is and I don’t want to dive into the deep mathematics about it, instead I’ll just use a simple example and hopefully illustrate it. Standard deviation is a statistical measurement about the deviation from an average value within a dataset.

If we look at this bell curve, this bell curve illustrates a normal distribution and we know it’s normal because within one standard deviation, the majority of our values occur, outliers are rare and that’s out beyond 3, 4 standard deviations. This is a good way for us to understand what standard deviation is, what the kinds of numbers might represent without having to understand the calculation. Now that we understand standard deviation, let’s look at an example of how network performance might apply to this. For every statistical metric that SevOne collects, we actually establish what we call a baseline. This dotted line represents the baseline for network utilization throughout the day

The hump here in the evening illustrates a backup that runs nightly. Now, if I look at actual measurements for a particular day, I would expect it to actually follow this baseline pretty closely and in the evening, you might see that backup. If I was just measuring static thresholds where I am looking at high, low values without this baseline, the static threshold would trip every single evening when these backups go off. That false positive is something we want to avoid. Instead by looking at the closeness of this performance data to the baseline that we’ve established, we can see that it falls well within a standard deviation away from that average value. No false positives, we don’t have to worry about filling up our inboxes with alerts.

We can actually do the opposite. Look at an example where the backup occurs at the wrong time of the day, say right in the middle of the day, the backup occurs not in the evening. Well, our highly, low threshold would have caught that, but now we can guarantee that with the same threshold, we can understand that this is an anomaly that we don’t want to see. It’s many standard deviations away. This is an outlier from our dataset. Hopefully, you can see that having a standard deviation setup and being able to understand what’s normal in terms of my dataset, can really help me eliminate false positives and find anomalies before there are real problems. If you want to learn more about all this, you can go to today. Thanks for watching.