Recently I was talking to one of my co-workers about connecting back-to-back virtual Portchannels (vPCs) from one pair of Nexus 7000s to another pair of Nexus 7000s (or 5000s). For example, you might have a data center with lots of VLANs, and decide that while you are waiting for TRILL and before you implement FabricPath, you will minimize STP with vPCs. The back-to-back vPCs will give you a loopless design.
Cisco mentions back-to-back vPCs in the Cisco Data Center Interconnect Design and Implementation Guide
http://www.cisco.com/en/US/solutions/collateral/ns340/ns517/ns224/ns949/ns304/ns975/data_center_interconnect_design_guide.pdf
under the “2 Sites vPC-vPC Case Study”, and also calls them multi-layer vPCs in their recent configuration guides.
I decided put together a set of configurations on paper to see what they would look like on N7Ks. Here is the layout:
In my example, 7K-A & 7K-B are at the core layer, 7K-C and 7K-D are at the distribution or access layer. I used different PortChannel numbers mostly to show you could, and I thought it might help illustrate the different devices. I chose different interface numbers for the same reason.
Here are the slightly commented config fragments I put together:
. . .
hostname 7K-A
!
feature lacp
feature vpc
! enable the VLANs needed
vlan 1-50
!
! enable ports that will be used for peer link w/ 7K-B
interface ethernet 7/1-2
switchport
switchport mode trunk
switchport trunk allowed vlan 1-50
switchport trunk native vlan 6
channel-group 20 mode active
exit
!
! enable peer keepalive (in this case, use separate VRF)
vrf context pkal
interface ethernet 8/1
vrf member pkal
ip address 172.23.145.17/30
no shut
!
! configure VPC switch domain
vpc domain 1
peer-keepalive destination 172.23.145.18 source 172.23.145.17 vrf pkal
!
! configure the peer link
interface port-channel 20
vpc peer-link
exit
!
! Configure the interface for the port channel to 7K-C off of the vPC
interface ethernet 7/9
switchport
switchport mode trunk
switchport trunk allowed vlan 1-50
switchport trunk native vlan 6
channel-group 50 mode active
exit
interface port-channel 50
vpc 50
exit
!
. . .
hostname 7K-B
!
feature lacp
feature vpc
! enable the VLANs needed
vlan 1-50
!
! enable ports that will be used for peer link to 7K-A
interface ethernet 8/1-2
switchport
switchport mode trunk
switchport trunk allowed vlan 1-50
switchport trunk native vlan 6
channel-group 20 mode active
exit
!
! enable peer keepalive (in this case, use separate VRF)
vrf context pkal
interface ethernet 9/1
vrf member pkal
ip address 172.23.145.18/30
no shut
!
! configure VPC switch domain
vpc domain 1
peer-keepalive destination 172.23.145.17 source 172.23.145.18 vrf pkal
!
! configure the peer link
interface port-channel 20
vpc peer-link
exit
!
! Configure the interface for the port channel 7K-D off of the vPC.
interface ethernet 8/9
switchport
switchport mode trunk
switchport trunk allowed vlan 1-50
switchport trunk native vlan 6
channel-group 50 mode active
exit
interface port-channel 50
vpc 50
exit
. . .
!
. . .
hostname 7K-C
!
feature lacp
feature vpc
! enable the VLANs needed
vlan 1-50
!
! enable ports that will be used for peer link
interface ethernet 3/1-2
switchport
switchport mode trunk
switchport trunk allowed vlan 1-50
switchport trunk native vlan 6
channel-group 10 mode active
exit
!
! enable peer keepalive (in this case, use separate VRF)
vrf context pkal2
interface ethernet 4/1
vrf member pkal2
ip address 172.23.146.33/30
no shut
!
! configure VPC switch domain
! note the 7K-C / 7K-D vPC topology has a different Domain ID than the 7K-A / 7K-B vPC topology
vpc domain 3
peer-keepalive destination 172.23.146.34 source 172.23.146.33 vrf pkal2
!
! configure the peer link
interface port-channel 10
vpc peer-link
exit
!
! Configure the interface for the port channel to 7K-A off of the vPC.
interface ethernet 3/10
switchport
switchport mode trunk
switchport trunk allowed vlan 1-50
switchport trunk native vlan 6
channel-group 40 mode active
exit
interface port-channel 40
vpc 40
exit
. . .
!
. . .
hostname 7K-D
!
feature lacp
feature vpc
! enable the VLANs needed
vlan 1-50
!
! enable ports that will be used for peer link
interface ethernet 8/1-2
switchport
switchport mode trunk
switchport trunk allowed vlan 1-50
switchport trunk native vlan 6
channel-group 10 mode active
exit
!
! enable peer keepalive (in this case, use separate VRF)
vrf context pkal
interface ethernet 9/1
vrf member pkal2
ip address 172.23.146.34/30
no shut
!
! configure VPC switch domain
! note the 7K-C / 7K-D vPC topology has a different Domain ID than the 7K-A / 7K-B vPC topology
vpc domain 3
peer-keepalive destination 172.23.146.33 source 172.23.146.34 vrf pkal2
!
! configure the peer link
interface port-channel 10
vpc peer-link
exit
!
! Configure the interface for the port channel to the remote data center off of the vPC.
interface ethernet 8/9
switchport
switchport mode trunk
switchport trunk allowed vlan 1-50
switchport trunk native vlan 6
channel-group 40 mode active
exit
interface port-channel 40
vpc 40
exit
. . .
!
— cwr
_____________________________________________________________________________________________
If you would like some additional details, the following references should be helpful:
Configuring vPCs from Cisco Nexus 7000 Series NX-OS Interfaces Configuration Guide, Release 5.x
Virtual PortChannels: Building Networks without Spanning Tree Protocol
very nice….I think you may want to add "Domain 2" to the middle of the diagram….
thanks for sharing!
Mike
Hi Mike –
I don’t think "Domain 2" is needed in the diagram. A vPC domain is just an arbitrary number assigned to a pair of switches that share vPC information. I was trying to show two domains (1 and 3), and that they do not need to be sequential.
Carole
I notice that you write ‘or 5000s’, well it doesn’t work todo the same configuration in Nexus 5000 since it doesn’t support creating more VRF’s for the peer keepalive (at least not in the 4.2(1)N2(1) that I have been working with. You should update the text either by removing the ‘or 5000s’ or mention which NX-OS version that is needed to enable this in a Nexus 5000 (if possible at all) or multiple configurations depending on hardware platform.
Going back and looking at this again, I may have been referencing it shows VPC Domain 3 in the drawing, but shows VPC Domain 2 in the configurations…..I think that is what I meant to correct….
Thanks, Mike
Hi,
Thanks carole. Very nice and clean representation. Well at that level we all know that Nexus is very keen and choosy when it comes to version and hardware specifications. I think all we need to know is the actual concept, configs can be taken anytime from cisco design guide lines as noone on earth can remember the encyclopaedia for the whole set of commands related to NS-OS. Thanks
Very instructive.
I think you need to update the diagram to reflect the configuration. 7K-C and 7K-D are shown in the diagram to be in VPC Domain 3, but the configuration files for them show VPC Domain 2 being configured.
Hi Ron –
Thanks, I missed the mismatch on the diagram and the configs. I updated the 7K-C & the 7K-D configs to show domain 3, which was my original intent…
Carole
Mike – I may have finally figured out your comment – I just updated the configs so the 7K-C and 7K-D configs showed domain 3…
Carole
Hi Carole..
Have a question here.. how do you combine 4 nexus 5000s and hook up with 2 fex 2000s
Hello All,
I’ve been attempting this configuration using sets of Nexus 7k’s and the port-channel does not come up. My scenarios is connecting 2 data centers via 2-10gig dark fiber links.
After opening a TAC request, I’m told that a total of 4 links are required with 2 separate vPC’s connecting the 4 N7K switches in the traditional sense of using vPC.
Have you tested this configuration to work? My only difference is the vPC number is identical on both sides.
-Brian
Hi Carole,
This is a great writeup and will come into use in some of my deployments. I’m assuming that a key point here is that you do indeed have to use different VPC domains, correct?
Thanks for sharing!
what happens, if traffic is received from another vPC member link say a VM via UCS via FI-A under 7K-C, now 7K-C’s link to 7K-A 3/10 breaks, so we’ve extended the vlan across the vpc peer-link, frame is sent across this link, here my understanding would be VSL bit is set to prevent a loop and the upstream B2B vPC member (40) link is no longer valid to exit the traffic flow?
Ignore previous answered in the N7K vPC best practices guide, seems the data-plane gets re-programmed in this failure scenario to allow this flow:
“The only exception to this rule occurs when vPC member port goes down. vPC peer devices exchange memberport
states and reprogram in hardware the vPC loop avoidance logic for that particular vPC. The peer-link is then
used as backup path for optimal resiliency. Traffic need not ingress a vPC member port for this rule to be
applicable.”
I am implementing the same topology in 2 Pair of 3548 Nexus switch and getting blocking port channel which is configured for back to back vPC at one end .
My topology is
4500 Series CORE-SW-01 (As Primary Root ) —–NEXUS SW01 & Nexus SW02–in single port channel at Core and vPC port at both Nexus switch
4500 Series CORE-SW-02 (As Secondary Root ) —–NEXUS SW03 & Nexus SW04–in single port channel at Core and vPC port at both Nexus switch
and after deploying the same topology as shown in above diagram
and getting block at NEXUS SW03 & Nexus SW04 –port channel 40
PVST is running in both Core Switch while RSTP is running all four Nexus switch
Kindly suggest
Ganesh