Audit Logs on Cisco Unified Communications Manager (CUCM)

Background

I will keep the background discussion to a minimum, because I am sure most readers of this blog understand the need for an auditing solution.  To summarize, admins need a way to look at an audit trail for administrative changes that occur in their CUCM cluster.  Basically, in large organizations there is a need to look at “what has changed” so that if there is an issue reported that is related to human error, the error can be quickly identified and fixed.  There is also an accountability aspect as it is likely that senior support folks want to identify education gaps in their lower tier support teams.

Software Revisions

The ability to enable auditing for administrative tasks was included in CUCM version 7.1(2) and later.  In my lab, I tested this feature using a CUCM 7.1(3b)SU2 cluster.

Enabling Audit Logs

By default, audit logs for administrative level tasks are enabled.  To access this feature, you will need to browse the audit log parameters from Cisco Unified Serviceability.  The following procedures can be used to accomplish this task:

1. Connect to Cisco Unified Serviceability on your CUCM cluster: https://mypublisher/ccmservice/

2. Go to Tools>Audit Log Configuration

3. Depending on your particular operational model, you may have people with different levels of authorization.  Authorization levels are implemented via the Groups and Roles configuration objects.  If you have a need to customize roles or Group/Role assignments you will want to look at the “Standard Audit Log Administration” role to understand the level of access required for users/groups that need to look at audit logs.

Accessing Audit Logs

At this point, you have enabled audit logs for administrative tasks.  As with other logs and trace files on the system, you can access these logs in several ways.  For this example, we will walk through using RTMT to view audit logs.

1. Load RTMT on your admin workstation

2. Connect to your CUCM cluster using a user ID that has the “Standard Audit Log Administration” role (or equivalent) assigned.

3. Go to System>Tools>Trace>Trace and Logs Central

4. Under “Trace and Logs Central”, you will see an “Audit Logs” folder.  Double click the folder to start the log collection wizard.

One of the benefits in using RTMT is that you can collect and download AuditLogs either as a one time request or you can schedule the download.  For those that like to run pro-active analysis (via scripts and the like) you may want to look at the “Schedule Download” option.  As with other traces/logs, once you specify the search criteria RTMT will provide a folder tree structure that can be used to navigate to the audit logs stored on the server.

 

Deciphering the Audit Log

The log file entries are lengthy but parsing them is not at all difficult.  Let’s look at an example.  I have a dummy phone on my CUCM cluster.  The phone was already created prior to testing the audit log feature.  What I did was add a phone number to the device.  In the AuditLog this action is captured as follows:

04/06/2010 11:03:15.295 |LogMessage
 UserID :ccmadministrator
 ClientAddress :192.168.1.4
 Severity :5
 EventType :GeneralConfigurationUpdate
 ResourceAccessed:CUCMAdmin

 EventStatus :Success
 AuditDetails : record in table numplan with key field dnorpattern = 2025552222 added
 ComponentID :Cisco CUCM Administration
 App ID:Cisco Tomcat
 Cluster ID: Node ID:iecucm01 

Now, the log entry itself will be a single line in the trace file but I have broken it out in something a tad more legible.  You will see that the log identifies the user who performed the change (UserID), the IP address of the machine the user was using (ClientAddress) along with other pertinent information.  Of particular interest is the AuditDetails section of the logged event.  In this example, we can see that I added a directory number.   Before dissecting this section, we should address an obvious question: which device was this performed on?

This is where things are tricky.  If you look at previous log entries, you will find when the user (ccmadministrator in our example) connected to the CCMadmin web portal.  However, the AuditLog will not track this user’s movements through the various web pages.  So, if I clicked on a device, and then clicked on “Add new line”.  Those events are not recorded.  Only when I actually “save” the change will it be recorded in the AuditLog.

Why?  Well, it boils down to the fact that the Audit Log is tracking changes to the database.  It’s focus is on what has changed in the database, not what has changed in the admin interface.  This is actually a necessary distinction, since changes to the database can come from a variety of sources:  BAT, CCMAdmin, AXL/SOAP (e.g. UCCX creates a JTAPI trigger).  If you look at our example above, you will see that the AuditDetails says:

record in table numplan with key field dnorpattern=2025552222 added

Numplan is a database table and if we wanted to find out what device this number was added to, we would need to take a few extra steps.  One step could be to run an ad hoc query from the command line to see what gives:

admin:run sql select d.name as device,n.dnorpattern,dmap.numplanindex
 from numplan as n inner join devicenumplanmap as dmap on dmap.fknumplan=n.pkid
 inner join device as d on dmap.fkdevice=d.pkid
 where n.dnorpattern='2025552222'
device dnorpattern numplanindex
=============== =========== ============
SEP00258418EE10 2025552222 3

From this output, we have identified the device and the line appearance the new number is assigned.  Yeah, I know the query is ugly.  I can’t resist you see, I do most of my admin tasks via custom queries.  That’s just me.  You could certainly figure this out via the CCMAdmin web pages by going to Call Routing>Route Plan Report.  Then search for the directory number “2025552222”.

So, you still have some work to do when you want to analyze the audit log, but the data now available is a great improvement over what you would have to do in the past.  I won’t even waste your cycles on that discussion.

Other Things That Are Logged

As you play around with the Audit Log, you will find that it captures other interesting things as well.  Such as commands entered by users on the CLI.  This is handy if you want to find out who restarted the TomCat service or who reloaded the server.  Small things like that.  Of course, if you want that data to have any meaning, you may want to investigate the CLI command: set account name.

The Audit Log will also capture events like a user uploading a custom banner page to the CUCM system.  The custom banner page is another new security feature added to CUCM 7.1(2).

Where is the Audit Log

For those folks who don’t dig on using RTMT for certain tasks, and would just rather download the file or view it directly on the console I have two pieces of information:

1. The files are stored here:  activelog audit/AuditApp/*

2. If you go this route, I recommend downloading the files to your workstations and then viewing/parsing/filtering.  Especially during production hours.

Conclusion

Well, the AuditLog feature isn’t 100% perfect as you still have to tear yourself away from CNN or ESPN to actually do some work.  But, it is getting there and the information that is recorded in the AuditLog is very handy when you need to determine the “how”, “when”, and “why”.  Hmmm, maybe it won’t tell you the “why” but it sure will point you to the person that can!