Thank you for attending our healthcare security webinar! View our recording on our event page. Webinar

12/19
2017
Carole Warner Reece

Troubleshooting Trunks in a NX-OS Environment (To Trunk or Not to Trunk)

One of my customers has been converting his core 6500 switches over to Nexus 7700 switches. In the first project phase, the plan was to try to just move what was currently configured on the old cores over to the new core devices. Since this was mainly a switched environment, this seemed like a reasonable plan. He was planning to move one site at a time.

On the second morning of one of the maintenance windows to the new devices, we ran into an issue. When we reconnected one of the last access switches planned for the morning, the link came up, the port-channel came up, but we lost connectivity with the devices on the access switch and the access switch itself.

We checked the configs on the new core switch; the configuration looked fine:

This closely matched the old 6500 configuration:


The differences make sense. The NX-OS default for the channel-group 3 interface command is mode on. The default trunk encapsulation for NX-OS is 802.1q.

We noticed that the two links were up, we were forming neighbors, and were up in a port-channel:

We reconnected the link back to the 6500, and regained connectivity.

Since we had other devices to migrate, we noted this issue, and moved on to the other connections planned for that maintenance window.

Finding the Issue

Later that morning, we looked at the remote switch after it was reconnected to the 6500 switch.

We noticed that Gig 1/1/1 had no configured trunk commands, but was still up in trunk mode. The customer mentioned that since the 3850s and newer switches seemed to just come up as trunks, they had not always configured a switchport mode trunk on the uplinks.

After a bit of review, we determined that the default switchport mode for newer Cisco switch interfaces is dynamic auto, which matches the mode auto we saw in the show interface trunk command. The configuration guide states “The interface becomes a trunk interface if the neighboring interface is set to trunk or desirable mode.”

However, the 3850 interfaces also did not become trunks when connected to the Nexus switch that was configured with the switchport mode trunk command.

Reviewing other documentation, we noticed:

  • On a 6500: switchport mode trunk — Puts the LAN port into permanent trunking mode and negotiates to convert the link into a trunk link. The LAN port becomes a trunk port even if the neighboring port does not agree to the change.
  • On a 3850: switchport mode trunk — Puts the interface into permanent trunking mode and negotiates to convert the neighboring link into a trunk link. The interface becomes a trunk interface even if the neighboring interface is not a trunk interface.

The 3850s run dynamic trunk protocol (DTP) by default. The 3850 configuration guide assumes that the remote switch is also running DTP, which 6500s do by default.

  • On a N7K: switchport mode trunk — Specifies the trunking VLAN interface in Layer 2. A trunk port can carry traffic in one or more VLANs (based on the trunk allowed VLAN list configuration) on the same physical link.

OK, this identifies the root issue. The NX-OS documentation does not mention negotiating with the remote site to do anything. This makes sense when you know/remember that DTP is NOT supported on NX-OS. The 3850’s port channel came up as an access port in the default VLAN 1.

This access port was not a match for the trunk configuration on the NX-OS switch. No traffic was configured to use the default VLAN 1, so A-Core1-7K lost connectivity to the other VLANs on B-Acc-103. We needed to have a matching trunk on the remote access switch side.

Solution

We added the switchport mode trunk command to B-Acc-103 while it was connected the 6500; it stayed up as a trunk. We switched the link back to the N7K, and the trunk stayed up. The problem was resolved.

The default behavior of switches sometimes changes. This can be an issue when you are relying on defaults.

Related Posts:

Carole Warner Reece

Architect

A senior network consultant with more than fifteen years of industry experience, Carole is one of our most highly experienced network professionals. Her current focus is on the data center and on network infrastructure.

View more Posts

 

Virgilio “BONG” dela Cruz Jr.

CCDP, CCNA V, CCNP, Cisco IPS Express Security for AM/EE
Field Solutions Architect, Tech Data

Virgilio “Bong” has sixteen years of professional experience in IT industry from academe, technical and customer support, pre-sales, post sales, project management, training and enablement. He has worked in Cisco Technical Assistance Center (TAC) as a member of the WAN and LAN Switching team. Bong now works for Tech Data as the Field Solutions Architect with a focus on Cisco Security and holds a few Cisco certifications including Fire Jumper Elite.

 

John Cavanaugh

CCIE #1066, CCDE #20070002, CCAr
Chief Technology Officer, Practice Lead Security Services, NetCraftsmen

John is our CTO and the practice lead for a talented team of consultants focused on designing and delivering scalable and secure infrastructure solutions to customers across multiple industry verticals and technologies. Previously he has held several positions including Executive Director/Chief Architect for Global Network Services at JPMorgan Chase. In that capacity, he led a team managing network architecture and services.  Prior to his role at JPMorgan Chase, John was a Distinguished Engineer at Cisco working across a number of verticals including Higher Education, Finance, Retail, Government, and Health Care.

He is an expert in working with groups to identify business needs, and align technology strategies to enable business strategies, building in agility and scalability to allow for future changes. John is experienced in the architecture and design of highly available, secure, network infrastructure and data centers, and has worked on projects worldwide. He has worked in both the business and regulatory environments for the design and deployment of complex IT infrastructures.