Some Best Operational Process Practices
[Guest article from my friend Dominic Basta. Thanks again, Dominic!]
I thought I’d share my experiences working with switch profiles on Cisco Nexus5000 switches. Cisco suggest using switch profiles for the following reasons:
I decided to test setting Switch Profiles up to provide a single point of configuration change for our bowtie setup Nexus 5548s and 2232 FEX. The switch profiles would protect us from vPC misconfigurations and insure configuration redundancy across the parent switches.
If you have a symmetric network, using port profiles to configure port-channels and the physical member ports for vPCs can be a very good idea.
Here is the basic design topology of the data center row:
There are a couple of basic concepts to know before you start configuring the switch profile.
The first step is to enable CFSoIP communication over the Mgmt0 interfaces:
Test-5k-sw1# configure terminal Test-5k-sw1(config)# cfs ipv4 distribute Test-5k-sw1(config)# exit Test-5k-sw1#
NOTE: The cfs ipv4 distribute command enables CFSoIP communication over the Mgmt0 interfaces. During my first setup I didn’t issue the command, and while I was able to ping across the Mgmt0 interfaces, the switches will not sync.
The next step is to configure a switch profile with a name and a peer destination IP address. Creating the switch profile is quite easy, instead of using the config t command you use the config sync command.
Test-5k-sw1# config sync Test-5k-sw1(config-sync)# switch-profile test-sync Test-5k-sw1(config-sync-sp)# sync-peers destination 10.0.0.2
NOTE: You must create the switch profile with the same name on each switch, and the switches must configure each other as a peer across the mgmt0 interface. When connectivity is established between switches with the same active switch profile, the switch profiles are synchronized.
After both peers are configured and synchronized, you enter the shared commands for both switches on one switch in the switch profile. It works similar to configuring the commands from the the regular configure terminal mode but none of the commands you enter are placed or “committed” into the running configuration until you enter the commit command. For example:
Test-5k-sw1# config sync Test-5k-sw1(config-sync)# switch-profile test-sync Test-5k-sw1(config-sync-sp)# interface port-channel20 Test-5k-sw1(config-sync-sp)# speed 1000 Test-5k-sw1(config-sync-sp)# interface Ethernet1/1 Test-5k-sw1(config-sync-sp)# speed 1000 Test-5k-sw1(config-sync-sp)# channel-group 20
One thing to recognize is that as soon as a commit is successful, the switch saves the configuration buffer and drops you out of the switch profile configuration mode. So if you needed to add more configurations, you will need to re-enter the switch profile. I’ve found that when working with switch profiles I like to use the verify and the show switch-profile name buffer commands fairly often:
Test-5k-sw1(config-sync-sp)# verify Verification Successful Test-5k-sw1(config-sync-sp)#
NOTE: The verify command verifies the commands in the switch profile buffer can be added to both the local and peer switches. The verify command does not drop you out of the switch profile, so you can basically use it after every command to make sure you will not later fail the Mutual Exclusion Checks.
Test-5k-sw1(config-sync-sp)# show switch-profile test-sync buffer switch-profile : test-sync ---------------------------------------------------------- Seq-no Command ---------------------------------------------------------- 1 interface port-channel20 1.1 speed 1000 2 interface Ethernet1/1 2.1 speed 1000 2.2 channel-group 20
After you finish configuring and verifying all the commands you want to synch between peers, you need to commit the commands:
Test-5k-sw1(config-sync-sp)# verify Verification Successful Test-5k-sw1(config-sync-sp)# commit Commit Successful Test-5k-sw1(config-sync)# exit Test-5k-sw1#
I found using the switch profile is quite easy but a bit quirky at times. We have since moved the switch profile into production with a few issues, but for the most part it does ease the pain of configuring both N5Ks.
I believe that configuring a switch profile on an existing terminal-configured N5K isn’t something to taken lightly. I have found that using the import command to import an existing interface configuration appears to fail more times than work for reasons that are often hard to diagnose.
However, in our secondary data center the 5548s and 2232s were configured from the start with the switch profiles and switch profiles work VERY well there. I am hopeful that in a few releases to come the rest of the quirks in switch profiles will be worked out.
Here is a good document to use as a reference if you want to start your own lab switch:
Configuring Switch Profiles
Some Best Operational Process Practices
Networking by the 95th Percentile
The Business of Diagnosing Slow Applications