When Active Active Isn’t
One of the things I look for when doing network assessments is the availability of syslog data. I’m always amazed at how few sites seem to be using this valuable source of information! Over the years, Terry Slattery and I have written a lot about syslog and syslog-NG, both in private reports and in our blog posts. It feels like it’s again time to blog about it and encourage the readers to not be “syslog slackers!” I’ll be brief — and then point you to some references.
I hope you’re not turning up your nose at standard Linux syslog (old, basic, whatever). It’s simple; it works. Yes, finding the golden needles in the haystack takes tools. Doable!
Our basic recommendations:
Terry Slattery put me onto the summarize-nmslog script by Darin Davis. I’ve been hacking up a version of that for years. It’s not really in a state where I want to expose it in public. Darin’s posted version seems to have gone off Google’s radar now. So let’s say, if you email me, I’ll be glad to send you the script “as is” — some PERL/regex skills needed.
In part, every time I visit a new site, I seem to have to spend one to two hours hacking the regular expressions to match the local date/time format. The script still produces pretty useful results: frequency counts by overall Cisco message type, then by type by router. At one large site I was at, the Splunk expert was able to produce something similar in about a day.
A great tweak to this, thanks to someone at a customer site (Nikolay!): Take the bottom part of the output, the per-message per-router counts, add column headers, and import into Excel, then set up as a pivot table. Makes it all easier to read.
Why you should care: At one site, I saw a huge number of OSPF adjacency changes over one week. Time to go look at the router(s) involved. Duplex mismatches, ditto. CDP VLANs allowed on trunk mismatches, ditto. STP instability. All stuff that might show up as performance or connectivity dropouts. Many things that do not show up with SNMP polling. With syslog, a chance to get proactive!
Highly recommended; there are nuances the above does not address. (I did say “brief.”)
Comments are welcome, both in agreement or constructive disagreement about the above. I enjoy hearing from readers and carrying on deeper discussion via comments. Thanks in advance!
When Active Active Isn’t
Tips for Considering New Technology
SD-WAN plus Equinix equals Global WAN