Removing Visibility Gaps from Service Provider Networks: Session 2


Join SevOne's Dave Hegenbarth,SE of Global Strategic Alliances, as he discusses the best practices for cutting mean time to repair in half during this live demonstration. Downtime degrades the user experience and jeopardizes revenue, thereby causing widespread business effects and possible strategic failures. Learn specifically how SevOne will not only reduce downtime, but will additionally cut repair costs.



Okay, again, welcome everybody to our Demo with Dave this morning, the topic being removing visibility gaps from service provider networks. Obviously, all of your phone lines have been muted so if you have any questions along the way, certainly we'll have a period at the end where we can do a Q and A, but if you have a question that you think of while Dave is doing the presentation today, please feel free to use the Q and A chat window and we'll try to address as many of those questions as we can online. My name, again, is Scott Frymire with SevOne and I'm happy to introduce today the original, the authentic, the one of a kind Dave Hegenbarth. I know has Dave has been absent from a couple Demo with Dave's recently, but he is again with us today. Sometimes we like to give him a little bit of a vacation from doing these biweekly demos, but Dave, glad to have you back and I'm going to turn things over to you today.

Thank you, Scott. Yes, it's great to back, and welcome everybody, whether you're morning or afternoon or evening, welcome to Demo with Dave on a Friday right before Christmas. I thought I'd throw up a little Christmas cheer in my opening slide. We are going to talk about covering visibility gap in service provider networks. As Scott said, please let us know if you have questions through the chat window and we'll try to answer those.

Hopefully everyone can see my slide here. Talking a little bit about where SevOne is different in the marketplace. We're talking about scalability and flexibility here and what we've done is we've built a performance monitoring product that is low cost appliances or allowed to collect a whole lot of data and then we use our SevOne clustering technology to bring that all back together in a single pane of glass. You get very real time reports for very large data sets and this is something different in the marketplace, right? We've all been used to waiting maybe 5 minutes, 10 minutes, even an hour for complex reports to run.

With the SevOne Cluster Technology, we have the ability to split up that load across multiple sets of memory, multiple sets of CPUs, etc., to produce very quick reports on large amounts of data.

What do we grab? We grab performance statistics from a very large range of different types of devices from the traditional SNMP and ICMP of most network monitoring systems all the way through to what we're going to talk about today when we say third party or xStats data. This is pretty much any sort of structured time stamped data that comes into the box. We have metrics for HTTP, DNS. We've got the ability to collect and report on NetFlow. We have a myriad of technologies that all bring large amounts of data into SevOne appliance. We knew we'd never be the only device in the data center, right? We had to be able to be very flexible to talk with lots of different types of devices on the outbound side too. Via our open API, we have the ability to send what we've recorded back out to other management platforms, if you will, so maybe we're taking in config management from a provisioning systems or maybe the config management system wants to understand the types of devices or interfaces we've discovered.

The API is very much bi-directional so we can take information in and we can push information back out. We do have a mobile app so we can see event notifications as they happen in real time on our phone or our iPad and as always, our interface is HTML. We are just using a browser so you can do that from your laptop, your iPad, your phone, wherever, and get very quick reports.

The challenge today, helping service providers with their performance visibility gaps really stems around this. In a service provider environment, we typically have many different types of equipment. It's not a homogeneous. We buy all one type of gear, nor is it all one type of gear, right? We have anything from the RAN to the eNodes to the CPGs or the Converged Packet Gateways and the LTE 4G to all different types of devices in the network. Typically, when we buy into a strategy, whether it's CPG or whatever it is from a particular vendor, it comes along with its own EMS, Element Management System and these element mangement systems are not there to provide performance statistics. They're actually there to provide config management updates to the devices.

Everything that they control that manufacture might have Cisco's platform. You might have Ericsson's platform. You might have the Acatel Lucent stuff. You have a lot of these different pieces of gear that have been put together and they all have config and some performance things. They're all structured differetly by different manufacturers, but we need to understand the performance of the whole IP infrastructure as it's performing today.

What did we do? We came along. We realized there's lots of devices, sometimes probes, all kinds of different things. They each have their own console and EMS. Then what we did was we just went to those EMSs and we said, "Hey. What is the structure of your performance data? Is it available CSV? Is it available as VX?" However it's available, we've written a set of adapters that goes out and grabs specific performance metrics and pulls them in to SevOne. We normalize that data so now it's all SevOne data and now I can put the performance of a particular Triple A authentication server right next to a CPG, right next a EXFO test of how my network is performing. All of those diplayed on the very same screen and do all the great things we do, like hold the performance data for up to a year, be able to baseline that so we understand what normal is. It will show you all this in the dashboard and then I'm able to alert on it. Out of the whole infrastructure. That's really the problem that we were out to solve.

Just a view of our customers by industry column if you will or vertical. In the Telco space, you can see those, financial space and the techs and others. The point of this slide is really to say that one thing is that all of these companies have IT infrastructure that's very key to running their business without their network, without the devices that push traffic across the network, without the applications that authenticate people or allow them to understand. They would not have a business. All of these guys have invested largely in SevOne to show them across the board from end to end their IT performance statistics.

I'm going to flip over to my dashboard and what we see right here is a very big map. I've blown it up and stretched it out to show you an example of what we monitor and some of you have probably seen this if you've seen this demo before. An example of the different parts of pieces in an LTE environment. Now this does not have be LTE. It could be 2G gear. It could be 3G gear. We could be even in ISP or in MSO looking at different types of gear. The idea is that each of these parts and pieces of an IT infrastructure is going to have very different key performance indicators, KPIs, and they're probably going to be very different manufacturers. We're going to have Cisco and ALU and Huawei and Ericsson and MSN doing different functions within the network, yet we want to see an end-to-end view of how it's performing. We also want to be made clear the health of the network devices or set a device or region that's underperforming. You can see here on this particular status map. I've placed the health of each of these functions.

We can see here we have one that's not functioning so well. I could either click on this and get some metrics as they pop up there. Basically what I have is a list of how my performance is by functional area. I have my access layer. I have my core. I have my gateways. I have the IMS applications, impossibly the radio edge. Again, all of these are performance thresholds so set different thresholds for different things in the network and I have them rolled up and broken out by functional group here. Maybe I want to drill in and look at things that are in the packet core of my network because I see 19 and 73 of my thresholds have a particular violation.

Rolling up, looking at some of the supporting data for this, we first see I have here a BRICS test that's showing me one way and two way and I'm actually seeing mostly 2 way data right now, but from end to end, what is the performance of my network? We can see here measured in milliseconds, I have a spike that goes up to just about one milliseconds. It's very important that I understand backhaul where this is used so from that cell tower and what's the response time of the Ethernet backhaul I'm getting across the network?

We can also look at in here, this is a little further up the stack. In layer 3, this happens to be. Maybe I have a mobile app and this an HTTP connect time from my mobile app across the network. I want to understand in milliseconds how long does it take to connect and how long does it take to possibly download a test page. Again, these are being provided by Brics Expo Test data that's coming in so the probes are out there getting data into their element management system and we are collecting the data from that element management system into SevOne so we can take a look at it.

All of our dashboards are very interactive so I could come in here and say, "Wow, something went on in this particular thing" and I can zoom in to see how that happened over that period of time or I can zoom out. I wonder if this happens all the time. I can look at past 48 hours and just as rapidly, I can come in and say, "Hey. Let's look at the last 4 weeks." I have very granular for 4 weeks and you saw how fast that redraws. Some of the theme as you look at this is diverse data and then speed, scale, and simplicity. I'll show you a little later how easy it is to build a particular dashboard.

From there, we're looking at access gateways. We're breaking this up into different parts of the networks, but these are UV attach rates by a particular region so we can see how many attached per region. We can also take a look at that as a real time graph so this is a stack ranking as an hourly average, but here we can actually see it as it happened over the past 48 hours. We can also look at the MMEs for authentication errors and rates as well. We see a number of different MME stats out of this network over some period of time. Again, I can zoom in, zoom out, et cetera.

I can also bring it in to my IP infrastructure. This is nothing more than SNMP data so I'm actually pulling this course which, using SNMP, I'm stack ranking my most utilized ports. Again, just looking for something that might be out of the ordinary or overutilized or just want to know which port on this particular core IP switch is using the most data.

We can take a look at load balancers in the network. In terms of connection per second, this is also SNMP data coming from our F5 box and we just want to understand the rate at which people are getting load balanced for different applications. Maybe Triple A authentication is one case. If we want to understand the performance of our load balancer with our switch in the IP network and then we're actually looking at firewall rates and we're looking at current connections per second, things like that.

All of this is a lot of different information from a lot of different sources that talks about or talks to or proves out how an infrastructure such as this mock up of MLG 4G infrastructure. All of the data is going to be different types. We always get the question, "Which data points can you grab?" Which data points to grab really are dependent on which data points the EMS systems that that manufacturer have deployed and metrics, we can use. I'm going to jump back to a slide for just a second to illustrate this. As long as we understand who's providing this data, so this vendor EMS data, whether it's Cisco or Huawei, what have you, we can take any sort of sort of data, we've even done binary and we can parse that to produce these types of graphs and charts. You can end up with looking at your real time stat for whatever it might be.

Very flexible engine and we have a number of xStats adapters built so that adapter takes the CSV or the ASN.1 data and then formulates it into a consumable form that we can then import in. We have various ones and we always have. It's a very open platform so you can always write more or different ones as we encounter new and different types of gear.

One of the other things was they talk about the ease of building these sorts of dashboards and I want to go back through that again. I know some of you have probably seen it. It's very easy to create the next new really cool dashboard. If I start with "create a report" here, what we're going to see is just choices of what sort of report I would want to build and I can say, "You know what? I really am interested in metrics and I want to understand maybe my CPG metrics" and then I can say, "Look, I have all kinds of different regions to choose from, but if I look at the Bismark CPG, what are the KPIs that I might be interested in?" I'll take a look and from there, what are the KPIs from that control traffic? Maybe I'm interested in bearers established, establishing channels and I might want to do that for a couple sites. We saw that that was the Bismark CPG, but if we look, there were others. I keep typing "CPU" when I really meant "CPG".

Maybe we'll look at Bismark and Chicago. Doesn't matter, I'm probably going to be interested in the same control traffic from both and I want to understand bearers established so now I have those 2 KPIs that I really want to graph. I can say, "Do I want to see it as a percentage possibly? Do I want to aggregate the data or just keep the as polled data? Do I want to do analysis?" Maybe I want to see my baseline so SevOne has the ability to generate a baseline. What do I mean by that? The baselines are built every 15 minutes of a 7 day week so I have Sunday from 0 hundred hours to 0 15, 0 15 to 0 30. I have Monday 8 am to 8:15 is one data point, 8:15 to 8:30 is another data point, 8:30 to 8:45 et cetera. That's on a rolling basis so every Monday 8-8:15 I roll them up and that's my point of normal.

Maybe I want to take a look at normal for this particular set of data. I can set the time span and the time zone that that's in. I can choose any number of graph types so line to bar to pie, stack whatever I might want. Maybe I want a stack here instead of a line because I'm going to say, "I want to understand the composite of Chicago and Bismark at the same time. If I get an overview of what I've picked, you can do aggregation here, the size, the data, et cetera. If I click "finish", I get a stack graph of my bears established in there. Let's see bearers number established.

With that, I have a nice little graph. If I want to add more to this, I certainly can come right back in to the same sort of metrics. I can look at another set of groups. I can actually zoom in or zoom out and when I get the report built that I want, I have a number of things that I can export as a PDF. I come in here and I say, "Yeah, I would like this as a PDF" and all of the sudden is a "what you see, what you get" version of this PDF that I could ship off to someone to say, "Hey. You need to go something in Chicago or Bismark or what have you. All of our KPIs work this way so I have a roll up of a lot of different data types. Really what the data is, the KPI that's most important to you is the one we want to get in here and as long as those key performance indicators or statistic, we can use xStats or we can pull using SNMP or some other method to bring them in. Once they're in, we can do all kinds of good things.

One of the things I did was I didn't show you my baseline because I actually picked a graph type that doesn't visualize that very well, so maybe I'll come back here real quick and edit my graph. I can go to my visualization so I can say, "I wanted a line graph, not that one." If I click "finish", all of a sudden you can see the baseline in here so you can see the data and then you can see the baseline average so I can tell where am I against my baseline. I'm looking pretty good.

I can also set how many standard deviations away from baseline. If I take a look at my settings and my data analysis, I could say, "I'd like to see what it would look like if I was 3 standard deviations above or below my baseline normal. Then I turn that on and now I can visualize if I was to set that, a threshold of deviation over baseline, what would 3 standard deviations at that particular time on a Friday look like? I've shown you the standard deviations, the baseline or understanding normal and the data that traverses or in this case, the number of established per second over a period of time. As I showed you before as well, I can change the time span. We can take this to a PDF. Lots of great things we can do, but the idea behind this is collect as much data as you can so that complete end to end visbility of the performance of your network, the IP infrastructure, all the parts and pieces that make it up and do that in very short real time views, but also, as you see here, I span almost a month in this particular graph.

Bring together multiple different reports very quickly into a single pane of glass, all web-based so you can get to it from your phone, your tablet, your PC, and you can understand the performance of your infrastructure. You can schedule these reports to be mailed out every hour, every day, every week, and that's a whole lot about how we are different in the marketplace.

With that, I think I'm going to ask Scott if there are any questions that have come in over the period and attempt to answer anything that anybody would like to hear.

Thanks, Dave, for the demo and I would encourage if anybody has a question, they can go ahead and submit it via the Q and A panel and we'll get to those questions.

While we're waiting for any questions to come in, I do want to maybe call out one specific example or use case how SevOne would use to help remove visibility gaps from a service provider network. In this case, I'm going to talk about a top tier wireless provider who actually was looking to monitor the simcard activation process during the iPhone 5 rollout. This was actually occurring over Black Friday which, as you know, is probably the busiest shopping day of the year so they were expecting a lot of traffic. Prior to SevOne, they had no way of really monitoring that activation process and, of course when we're talking about this wireless provider, they're not making their money based off the sales of the phones. They're making it based off of the activation and subscription services.

From a business health perspective, it's critical that they make sure those activations were going through okay and they had to quality reports to show how many activations were being processed, how many activations were rejected, and making sure, of course, at the end of the day that the end user had a positive experience and was able to activate their iPhone 5. What they did with SevOne, they actually built a dashboard that represented the entire data path of the sim OTA process from the user equipment all the way back to the activation host and what in the data path, what are all the devices and systems in that data path that they could represent in this dashboard and then monitor with synthetic SLA tests and such. In this case, whether it was looking at the routers or the servers themselves and the components of the servers, I know they had some other systems that they needed to have performance visbility of.

Even going back to what Dave said just a little while ago, even if traditional SNMP polling was not available in some of these systems that were in the data path, SevOne was actually able to provide one of those xStats adapters to pull that performance data into the performance dashboard and then they could monitor the success of those activations. When we talk about performance visibility, we're not just talking about the different metrics and data points that we can poll from SevOne, but sometimes, SevOne is working for other EMSs and other systems to bring that performance data in alongside the polled metrics from the SevOne application as well. With that, they were able to ensure successful sim OTA activation process.

I thought that was a pretty interesting example of how we're able to help service provider increase visibility of their network.

Dave, I did get one question here. I'll just say, I mentioned xStats adapters, you talked about it earlier. Matthew was wondering if for the data that's pulled in via these x stats adapters, can that data also be baselined and alerted off of as well just like you would with normal SevOne polled data.

Absolutely. It's a great thing. Once we get the data in, whether its from xStats, WMI, Windows world, what have you. Once it's in, it's all just a data point. It's a KPI and so we get all the good benefits of storing it for year. We can do capacity planning on it. We can do alerts from baseline visualization, everything like that. Doesn't matter the data type, once we've absorbed it, we can do all the cool SevOne things that we do.

Okay. Great. And if there's any other questions and you'd like to go ahead and submit them via the Q and A text panel, please do so at this time.

In the meantime, Dave pulled up a slide here so I will remind everybody that come January 10th, we will have our next Demo with Dave topic which will be taking advantage of advanced NetFlow and Dave will be with us again for that demonstration so we look forward, if you've got anything out of today's presentation, we look forward to talking to you again and learning a little bit more about advanced net flow capabilities.

Okay, Dave, I don't see any other questions coming in via Q and A right now. Thank you, Dave, again, for your demonstration, your time today and thank you to everybody who joined the call and for your time. I certainly wish you guys all a very happy holiday and look forward to chatting with you again soon.

Thanks, Scott. Thanks, everybody. I am Dave from Demo with Dave and you can reach me at If a question comes to you later today or some other time, again that's Thank you, everybody, for your time. Really appreciate it.