This is the second article on my series on QoS on the Nexus 5000 (N5K) and Nexus 2000 (N2K) for medical grade networks. The first article was QoS on the Nexus 5000/2000 – Part 1. (I apologize it took me so long to get back to this series!)
In the previous article, I discussed some MQC background as well as key aspects of QoS on Nexus 5000/2000 switches. I also discussed queuing and queue structure on Nexus 5000 and 2000 switches. (I don’t mind if you want to go review that article before going on…)
I also provided a brief outline of the six steps to configure QoS on a Nexus 5000/2000 switch:
For this article I want to build on the outlined steps and provide some samples for configuring medical grade QoS on the Nexus 5000/2000. In my scenario, I want to classify traffic based on DSCP when possible, and will use COS when I must.
I will be using the following QoS Classifications for Medical Grade Networks:
The example environment includes Nexus 5548s as well as Nexus 2232s. The Nexus 5000/2000 switches will be the trust boundary for edge devices and will follow the same ACL configuration guidelines defined elsewhere in the enterprise.
Configuring Global Traffic Classification Based on COS
At this time the Nexus 2148, Nexus 2232, and Nexus 2248 modules can only support CoS based traffic classification. Therefore, I use a global classification policy based on COS that is applied globally all N5K/N2K ports using a type qos policy and type QOS class-maps as follows:
! To support Nexus 2000s, classify traffic by COS. ! This classification will be for global COS to qos-group mapping. class-map type qos match-all COS1-GLOBAL match cos 1 class-map type qos match-all COS2-GLOBAL match cos 2 class-map type qos match-all COS3-GLOBAL match cos 3 class-map type qos match-all COS4-GLOBAL match cos 4 class-map type qos match-all COS5-GLOBAL match cos 5 class-map type qos match-all COS6-GLOBAL match cos 6 ! The default class already matches COS 0. ! ! Assign each global COS class to a qos-group in a qos policy-map policy-map type qos CLASSIFY-GLOBAL class COS6-GLOBAL set qos-group 3 class COS5-GLOBAL set qos-group 5 class COS4-GLOBAL set qos-group 4 class COS3-GLOBAL set qos-group 3 class COS2-GLOBAL set qos-group 3 class COS1-GLOBAL set qos-group 2 ! The default class is assigned to qos-group 0 by default.
Note: Although the N5548 supports user configured qos-group 1, I deliberately do not use qos-group 1 because the Nexus 5010 and 5020 switches hard code qos-group 1 for FCoE traffic.
As needed, I also hard code COS values for traffic received from the host ports on the N2K:
! interface “interface n2k/mod/port[-port] untagged cos 0 ! set the CoS value to 0 for traffic received on the host interfaces !interface “interface n2k/mod/port,n2k/mod/port ! untagged cos 5 ! selectively apply a CoS value for voice/UC servers as needed end
ip access-list QOS-NETWORK-CONTROL remark Network protocol traffic permit pim any any permit tcp any any eq bgp permit tcp any eq bgp any ! ip access-list QOS-SIGNALING remark Deny voice, video, and audio streams expressly NOT related to remark QoS call signaling functions. remark SCCP and Secure SCP permit tcp any any eq 2000 permit tcp any any eq 2443 ... remark H323 Q931 and H225 Call Signaling permit udp any any eq 1719 permit tcp any any eq 1720 ! ip access-list QOS-VOICE remark Phone media traffic between valid endpoints (IP phones, IPTgateways, remark & IPT servers) permit udp 10.16.0.0 0.15.255.255 range 16384 32768 10.16.0.0 0.15.255.255 range 16384 32768 ! ip access-list QOS-CLINICAL-LIFE-CRITICAL remark Clinical Life Critical traffic between associated clinical subnets. remark This ACL deliberately left blank - Reserved for Future Use ! ip access-list QOS-MULTIMEDIA-CONFERENCING remark Multimedia Conferencing traffic between sanctioned applications and remark associated TCP/UDP ports. ! remark Traversal Calls permit udp any gt 1023 10.96.0.0 0.15.255.63 gt 1023 permit udp 10.96.0.0 0.15.255.63 gt 1023 any gt 1023 ! ip access-list QOS-REAL-TIME-INTERACTIVE remark Real-Time Interactive traffic (i.e. Immersive TelePresence) permit udp 10.96.0.64 0.15.255.63 gt 1023 10.96.0.64 0.15.255.63 gt 1023 ! ip access-list QOS-MULTIMEDIA-STREAMING remark Multimedia Streaming traffic from sanctioned streaming audio/video endpoints remark at the access layer. permit udp 10.96.0.128 0.15.255.63 gt 1023 10.96.0.128 0.15.255.63 gt 1023 ! ip access-list QOS-LOW-LATENCY-DATA remark Latency sensitive Data application traffic remark This ACL deliberately left blank - Reserved for Future Use ! ip access-list QOS-NETWORK-MANAGEMENT ! Might want to list out net-management subnets ! if high-volume telnet application traffic is observed. remark Utilities for access to network devices. permit tcp any any eq telnet permit tcp any eq telnet any permit tcp any any eq 22 permit tcp any eq 22 any permit udp any any eq syslog permit udp any eq syslog any permit tcp any any eq tacacs permit tcp any eq tacacs any ... ! ip access-list QOS-HIGH-THROUGHPUT-DATA remark High-throughput data traffic (FTP, etc.) remark This ACL deliberately left blank - Reserved for Future Use ! ip access-list QOS-LOW-PRIORITY-DATA remark Low priority (i.e. Scavenger) traffic from non-business oriented applications ! Add additional internal networks as necessary remark --- External Networks --- ! Add external networks as necessary remark ------ permit ip any any !
Class-maps will use these ACLs:
! class-map type qos match-any IN-NETWORK-CONTROL description Network Control match access-group name QOS-NETWORK-CONTROL ! class-map type qos match-any IN-VOICE description Voice/VoIP/IPT match access-group name QOS-VOICE ! class-map type qos match-any IN-CLINICAL-LIFE-CRITICAL description Clinical Life Critical match access-group name QOS-CLINICAL-LIFE-CRITICAL ! class-map type qos match-any IN-MULTIMEDIA-CONFERENCING description Multimedia Conferencing match access-group name QOS-MULTIMEDIA-CONFERENCING ! class-map type qos match-any IN-REAL-TIME-INTERACTIVE description Real-Time Interactive (i.e. Immersive TelePresence) match access-group name QOS-REAL-TIME-INTERACTIVE ! class-map type qos match-any IN-MULTIMEDIA-STREAMING description Multimedia Streaming match access-group name QOS-MULTIMEDIA-STREAMING ! class-map type qos match-any IN-SIGNALING description Call Signaling match access-group name QOS-SIGNALING ! class-map type qos match-any IN-LOW-LATENCY-DATA description Low-Latency Data (i.e. Scavenger Class) match access-group name QOS-LOW-LATENCY-DATA ! class-map type qos match-any IN-NETWORK-MANAGEMENT description Network Management match access-group name QOS-NETWORK-MANAGEMENT ! class-map type qos match-any IN-HIGH-THROUGHPUT-DATA description High-throughput data traffic (FTP, etc.) match access-group name QOS-HIGH-THROUGHPUT-DATA ! class-map type qos match-any IN-LOW-PRIORITY-DATA description Low priority data applications (i.e. non-business apps) match access-group name QOS-LOW-PRIORITY-DATA !
These classes are assigned to a qos-group and marked with a DSCP value using a policy-map. The policy-map will later be applied to the N5K interfaces. Note that multiple traffic classes are assigned to the same qos-group:
policy-map type qos IN-MARKING-5K description Inbound classification/marking policy for trust boundaries. class IN-VOICE set dscp ef set qos-group 5 class IN-CLINICAL-LIFE-CRITICAL set dscp cs5 set qos-group 5 ! class IN-MULTIMEDIA-CONFERENCING set dscp af41 set qos-group 4 class IN-REAL-TIME-INTERACTIVE set dscp cs4 set qos-group 4 ! class IN-NETWORK-CONTROL set dscp CS6 set qos-group 3 ! class IN-MULTIMEDIA-STREAMING set dscp af31 set qos-group 3 class IN-SIGNALING set dscp cs3 set qos-group 3 class IN-LOW-LATENCY-DATA set dscp af21 set qos-group 3 class IN-NETWORK-MANAGEMENT set dscp cs2 set qos-group 3 ! class IN-HIGH-THROUGHPUT-DATA set dscp af11 set qos-group 2 class IN-LOW-PRIORITY-DATA set dscp cs1 set qos-group 2 ! class CLASS-DEFAULT set dscp default end
This service-policy will be applied all the physical interfaces and port-channel interfaces to classify traffic based on DSCP except the FEX host interface range Ethernet 100/1/1- 199/x/x. The more specific IN-MARKING-5K interface policy will take precedence over the global CLASSIFY-GLOBAL policy.
Note: For a portchannel member, the service-policy needs to be configured under the interface port-channel, not to individual channel ports.
interface “ethernet mod/port[-port]” or “port-channel #” ! Use a range of interfaces if possible. ! Use a range of interfaces if possible. service-policy input IN-MARKING-5K ! end
Marking COS Based on QOS-Groups for Traffic Sent to N2K Ports
Since the N2K only supports COS based QOS, a policy is needed to establish the COS value of traffic from the N5K ports. This traffic has been marked with a DSCP value, and placed in a qos-group. The GLOBALNETWORKPOLICY is a global policy that marks COS values based on existing qos-groups, allowing traffic from the N5K to be queued and scheduled on the N2K.
The following type network-qos configuration commands support the global policy that marks COS values based on existing qos-groups.
! ! Set CoS based on qos-groups for traffic sent from the N5K to the N2K class-map type network-qos COS5 match qos-group 5 class-map type network-qos COS4 match qos-group 4 class-map type network-qos COS6-3-2 match qos-group 3 class-map type network-qos COS1 match qos-group 2 ! policy-map type network-qos GLOBALNETWORKPOLICY class type network-qos COS5 set cos 5 mtu 9216 class type network-qos COS4 set cos 4 mtu 9216 class type network-qos COS6-3-2 set cos 3 mtu 9216 class type network-qos COS1 set cos 1 mtu 9216 !
Note: the “mtu 9216” command is used to set the jumbo mtu size globally. To verify this setting on a N5K, you need to use the “show queuing interface ethX/Y” command because the standard “show interface ethX/Y” will fib and incorrectly show a MTU of 1500.
Configuring Egress Scheduling and Queuing Based on QOS-Groups
The egress queue structures for the Nexus 5000/Nexus 2232/Nexus 2248 consists of 6 hardware queues for data plane traffic. All traffic in one of the six qos-groups is sent to a queue. If you add a system class, a queue is assigned to the class. Bandwidth is not automatically dedicated to user-defined system classes so must be manually configured. For this environment, we will be using 5 of the hardware queues.
Note: There are only 2 hardware queues for data plane traffic on the Nexus 2148.
The following type queuing configuration commands support the transmit queue assignments and bandwidth allocations:
class-map type queuing QUEUING-COS5 match qos-group 5 class-map type queuing QUEUING-COS4 match qos-group 4 class-map type queuing QUEUING-COS6-3-2 match qos-group 3 class-map type queuing QUEUING-COS1 match qos-group 2 ! a class-default is predefined which matches qos-group 0 ! ! policy-map type queuing QUEUING-GLOBAL class type queuing class-default bandwidth percent 25 class type queuing class-fcoe bandwidth percent 0 ! ! Before you can successfully allocate bandwidth to a class you must ! first reduce the default bandwidth configuration on class-default ! and class-fcoe ! class type queuing QUEUING-COS5 priority bandwidth percent 20 ! one class in each policy map can have strict priority set on it class type queuing QUEUING-COS4 bandwidth percent 10 class type queuing QUEUING-COS6-3-2 bandwidth percent 40 class type queuing QUEUING-COS1 bandwidth percent 5 !
! system qos service-policy type qos input CLASSIFY-GLOBAL !
The input qos policy allows the CoS based classification to be applied globally to support the N2K ports.
Follow these guidelines when globally establishing the system qos configuration on a Nexus 5000 switch.
! system qos service-policy type queuing output QUEUING-GLOBAL ! The output queueing policy determines the bandwidth allocation for both ! N5k interfaces and N2K host interfaces. service-policy type queuing input QUEUING-GLOBAL ! The input queuing policy determines how bandwidth is shared for the N2K ! uplink in the direction from N2K to N5k. service-policy type network-qos GLOBALNETWORKPOLICY ! The network-qos policy allows queuing and scheduling of traffic ! from the N5K to the N2K. !
The global system qos configuration on a Nexus 5000/2000 switch is used unless service policies are configured at the interface level.
Note: Since they are more specific, an interface-level policy always takes precedence over system class configuration or the default configuration.
Applying a traffic classification policy based on DSCP to support the non- N2K ports
The last step is to apply the interface level QoS configuration based on DSCP.
! interface “ethernet mod/port[-port]” or “port-channel #” ! Use a range of interfaces if possible. service-policy input IN-MARKING-5K ! end
The more specific service-policy on the physical interface is used to classify traffic based on DSCP on each non-FEX interface.
Note: For a portchannel member, the service-policy needs to be configured under the interface port-channel, not to individual channel ports.
I will discuss a sample configuration for Trust Boundaries in my Trust Boundaries and QoS on the Nexus 5000/2000 – Part 3 article in this series.
More on Medical Grade Networks
For more information on medical grade networks, you may want to review the following articles:
Some QoS Gotchas
DNS Security: The First Layer in the Cybersecurity Fight
Nick has over 20 years of experience in Security Operations and Security Sales. He is an avid student of cybersecurity and regularly engages with the Infosec community at events like BSides, RVASec, Derbycon and more. The son of an FBI forensics director, Nick holds a B.S. in Criminal Justice and is one of Cisco’s Fire Jumper Elite members. When he’s not working, he writes cyberpunk and punches aliens on his Playstation.
Virgilio “Bong” has sixteen years of professional experience in IT industry from academe, technical and customer support, pre-sales, post sales, project management, training and enablement. He has worked in Cisco Technical Assistance Center (TAC) as a member of the WAN and LAN Switching team. Bong now works for Tech Data as the Field Solutions Architect with a focus on Cisco Security and holds a few Cisco certifications including Fire Jumper Elite.
John is our CTO and the practice lead for a talented team of consultants focused on designing and delivering scalable and secure infrastructure solutions to customers across multiple industry verticals and technologies. Previously he has held several positions including Executive Director/Chief Architect for Global Network Services at JPMorgan Chase. In that capacity, he led a team managing network architecture and services. Prior to his role at JPMorgan Chase, John was a Distinguished Engineer at Cisco working across a number of verticals including Higher Education, Finance, Retail, Government, and Health Care.
He is an expert in working with groups to identify business needs, and align technology strategies to enable business strategies, building in agility and scalability to allow for future changes. John is experienced in the architecture and design of highly available, secure, network infrastructure and data centers, and has worked on projects worldwide. He has worked in both the business and regulatory environments for the design and deployment of complex IT infrastructures.