Click here to request your free 14-day trial of Cisco Umbrella through NetCraftsmen today!

I just finished posting a neat syslog summary script that was passed to me by Phil Koontz.  He uses it every day to keep track of syslog events.

The summary page produced by this script is shown below.  I’ve found that it scales well on large networks that generate a lot of events.  Phil has used it on a network of over 2500 routers and switches.  I’m currently using it on a network of nearly 600 routers and switches.  At those scales, the report is seldom longer than two or three pages.

Summary of Cisco syslog Messages on    Sun Oct 11 23:59:01 2009

Cisco Messages:

18  LINEPROTO-5-UPDOWN
9  OSPF-5-ADJCHG
7  SNMP-3-AUTHFAIL
2  BGP-5-ADJCHANGE
2  LINK-3-UPDOWN
1  BGP-3-NOTIFICATION

Messages sorted by frequency and source device:

8       d04-3550-03      d04-3550-03       LINEPROTO-5-UPDOWN FastEthernet0/13
4       d19-3400-01      d19-3400-01       LINEPROTO-5-UPDOWN FastEthernet0/19
2       d02-2811-01      d02-2811-01       SNMP-3-AUTHFAIL
2       d03-2811-01      d03-2811-01       SNMP-3-AUTHFAIL
2       d45-3560-01      d45-3560-01       LINEPROTO-5-UPDOWN GigabitEthernet0/17
2       d19-3400-01      d19-3400-01       LINK-3-UPDOWN FastEthernet0/19
2       d48-7604-01      d48-7604-01       OSPF-5-ADJCHG
2       d16-7604-01      d16-7604-01       BGP-5-ADJCHANGE
2       d16-7604-01      d16-7604-01       SNMP-3-AUTHFAIL
2       d64-3550-05      d64-3550-05       LINEPROTO-5-UPDOWN FastEthernet0/2
2       d22-7604-01      d22-7604-01       OSPF-5-ADJCHG
1       d14-6504-01      d14-6504-01       OSPF-5-ADJCHG
1       d38-7604-01      d38-7604-01       OSPF-5-ADJCHG
1       d38-7604-01      d38-7604-01       SNMP-3-AUTHFAIL
1       d89-3560-01      d89-3560-01       LINEPROTO-5-UPDOWN Vlan3264
1       d89-3560-01      d89-3560-01       OSPF-5-ADJCHG
1       d16-7604-01      d16-7604-01       BGP-3-NOTIFICATION
1       d27-3560-01      d27-3560-01       LINEPROTO-5-UPDOWN Vlan3265
1       d27-3560-01      d27-3560-01       OSPF-5-ADJCHG
1       d92-3560-01      d92-3560-01       OSPF-5-ADJCHG

Included in the download zip file are scripts for web page links to the summary reports and for automatically running it from cron.  It is intended to be installed on a Linux-type system.  I’ve used syslog-ng as the underlying syslog server because it is free, scales to large syslog volume, automatically does log file rotation, and has great filtering and forwarding capabilities.

I use the report to look for rare, but critical events — often the ones that appear only once per month.  I also look for unstable interfaces or devices  — they have higher counts.  Reading the summary is certainly better than scanning a long syslog log file from the prior day. I find that a quick scan of the summary is a good first step in monitoring what’s happening on the network.  It is easy to implement and use.  I like to think of it as a quick event analysis system and the cost is certainly right – just the time and effort required to add to an existing syslog server.

If you improve the scripts, and don’t mind sharing your improvements, please send them to me and I’ll post them, with credit to you.  I’d like to see the second column be the device IP address instead of another copy of the name.  And I’d like to have it look up the interface descriptions, possibly taken from a config file repository, and add that after the interface name for interface events.  The zip file includes the original script from Phil as well as a modified version that we’ve been using in customer implementations.

I’d like to know what you use to track events.  In upcoming posts, I’ll share what I think are the fundamental requirements for handling events.

-Terry

_____________________________________________________________________________________________

Re-posted with Permission 

NetCraftsmen would like to acknowledge Infoblox for their permission to re-post this article which originally appeared in the Applied Infrastructure blog under http://www.infoblox.com/en/communities/blogs.html

infoblox-logo

Terry Slattery

Terry Slattery

Principal Architect

Terry Slattery is a Principal Architect at NetCraftsmen, an advanced network consulting firm that specializes in high-profile and challenging network consulting jobs. Terry is currently working on network management, SDN, business strategy consulting, and interesting legal cases. He is the founder of Netcordia, inventor of NetMRI, has been a successful technology innovator in networking during the past 20 years, and is co-inventor on two patents. He has a long history of network consulting and design work, including some of the first Cisco consulting and training. As a consultant to Cisco, he led the development of the current Cisco IOS command line interface. Prior to Netcordia, Terry founded Chesapeake Computer Consultants, which became a Cisco premier training and consulting partner. At Chesapeake, he co-invented and patented the v-LAB system to provide hands-on access to real hardware for the hands-on component of internetwork training classes. Terry co-authored the successful McGraw-Hill text "Advanced IP Routing in Cisco Networks," is the second CCIE (1026) awarded, and is a regular speaker at Enterprise Connect and Interop. He currently blogs at TechTarget, No Jitter and our very own NetCraftsmen.

View more Posts

 

Nick Kelly

Cybersecurity Engineer, Cisco

Nick has over 20 years of experience in Security Operations and Security Sales. He is an avid student of cybersecurity and regularly engages with the Infosec community at events like BSides, RVASec, Derbycon and more. The son of an FBI forensics director, Nick holds a B.S. in Criminal Justice and is one of Cisco’s Fire Jumper Elite members. When he’s not working, he writes cyberpunk and punches aliens on his Playstation.

 

Virgilio “BONG” dela Cruz Jr.

CCDP, CCNA V, CCNP, Cisco IPS Express Security for AM/EE
Field Solutions Architect, Tech Data

Virgilio “Bong” has sixteen years of professional experience in IT industry from academe, technical and customer support, pre-sales, post sales, project management, training and enablement. He has worked in Cisco Technical Assistance Center (TAC) as a member of the WAN and LAN Switching team. Bong now works for Tech Data as the Field Solutions Architect with a focus on Cisco Security and holds a few Cisco certifications including Fire Jumper Elite.

 

John Cavanaugh

CCIE #1066, CCDE #20070002, CCAr
Chief Technology Officer, Practice Lead Security Services, NetCraftsmen

John is our CTO and the practice lead for a talented team of consultants focused on designing and delivering scalable and secure infrastructure solutions to customers across multiple industry verticals and technologies. Previously he has held several positions including Executive Director/Chief Architect for Global Network Services at JPMorgan Chase. In that capacity, he led a team managing network architecture and services.  Prior to his role at JPMorgan Chase, John was a Distinguished Engineer at Cisco working across a number of verticals including Higher Education, Finance, Retail, Government, and Health Care.

He is an expert in working with groups to identify business needs, and align technology strategies to enable business strategies, building in agility and scalability to allow for future changes. John is experienced in the architecture and design of highly available, secure, network infrastructure and data centers, and has worked on projects worldwide. He has worked in both the business and regulatory environments for the design and deployment of complex IT infrastructures.