This is a good writeup on how important proper monitoring tools are, and on solarwinds.
Well done Steve. On Thu, Nov 10, 2016 at 10:31 PM, That One Guy /sarcasm <[email protected]> wrote: > We are running a demo of this. It started out as an eyeballing a netflow > collector and analyzer I dont have to poke all the time. we started > scrutinizer, liked it, but found out the price scale killed any chance of > getting it approved > > the pricing for this wasnt as bad, and the sales guy has some incentives, > but the whole package was alot, and I didnt intend on even looking at the > monitoring side because port based pricing models can quickly get out of > hand > > as part of the initial configuration i seeded the auto discovery just to get > through the setup. in the mean time, some other stuff came up and i i got > busy, this was friday or thursday > > we have been having some intermittent issues with periodic slowness to some > customers, the symptoms were that of a bottleneck. We had to throw some > static routes into our OSPF network defeating dynamics to force traffic out > one connection, thinking maybe it was a saturated lower quality upstream, no > noteable relief. so we thought maybe we were saturating a backhaul that was > getting to high percentage utilization, we added a redundancy and further > split traffic up with static routes. no joy. it was at a point where the > next step was just going site by site auditing every device...fun since the > issue was intermittent, that means multiple times > > the sales guy wanted me to commit to getting this thing up and running by > this weekend so next week we could list out what we want from it and how we > achieve it, or if we cant do it. > > so yesterday i go to turn on the flows and send them to the server, the > weird slowness is going on so its irritating me. > > i decided to clear out the alarms from installation and low and behold > theres an alarm on a named interface of one of the routers i tossed in on > discovery saying 90 percent or more usage. this is a 366mb licensed link on > a gigabit interface, so im quite curious. I drill into the detail, the port > is running at 100mb and saturating, i flap the port and its back to gigabit. > > we only monitor with powercode currently, we have snmpc but its old and shut > off. Ive toyed with a whole bunch of other opensource and low cost systems > but never had enough time to actually drill down and learn them, i did just > get a book on nagios because it was cheap on ebay. > > powercode is worthless for any amount of invasive alerting or monitoring at > any detail, if i want ports identified other than by port number it requires > an individual probe. pita. its good for long term static monitoring and some > real time tools, but its not an NMS. > > the point here, is the solarwinds tool is sweet, and for the 100 interface > package with a promotion the cost is doable if one takes into account the > time investment of the other opensource platforms, installation, learning > curve, back end configuration, and plethora of gotchas. > > this particular issue could have cost us a good deal in man hours tracing > it, refunds to customers for service impacts, and potential long term loss > of customers. > > now, once i knew where the issue was, i knew exactly where to look in our > existing data to verify it. 20/20 hindsight doesnt mean those are the > toolsets that would have been picked out first. if this tool had been in > production use, we would have known the first time the link negotiated down, > and addressed it before there was any noteable service impact. > > > If you are very frugal in your interface selection, this can be a good > choice for an nms (i havent played with the atlas map other than dropping > some stuff on it) if you dont want to dick around with a diy solution. its > cheaper if you dont add the netflow analyzer package. Its solar winds so its > pretty, and user friendly. the flow analyzer does route monitoring too, i > havent looked at that, but the salesguy says he thinks we can visualize our > ospf with the network atlas component, if thats the case the boss will > likely drop cash. licensing is perpetual with 20% yearly for maintenance if > you want it > > > > http://www.solarwinds.com/network-bandwidth-analyzer-pack > > > -- > If you only see yourself as part of the team but you don't see your team as > part of yourself you have already failed as part of the team.
