Click here to request your free 14-day trial of Cisco Umbrella through NetCraftsmen today!

4/24
2012
Carole Warner Reece

Troubleshooting Nexus 5500s and 1 GE SFPs

The newer versions of NX-OS support 1G or 10G on the N5K, but the N5K is not auto-speed sensing. A mini case study follows.

One of my customers just put some some GLC-SX-MM optics and a fiber patch between a 3750 and a N5K. He happened to look at status on Network Assistant on the 3750, and this error appeared:

Description: Gi1/1/1: This port has been disabled because Non Compliant  Gigabit Interface Converter (GBIC) connector detected.

Recommendation: Replace connector with cisco compliant Gigabit Interface Converter (GBIC) connector. Refer switch technical documentation to determine cisco compliant connector. Enable the port again.

This seemed odd, since the GLC-SX-MM is supported on both devices. I tried shut/no shut on both ends of the port, but that did not help. I then wondered – does he have a bad SFP? I poked around a bit.

The CLI show log info on the 3750 was also unhappy:

*Apr  16 01:48:41.190: %PHY-4-SFP_NOT_SUPPORTED: The SFP in Gi1/1/1 is not supported
*Apr  16 01:48:41.190: %PM-4-ERR_DISABLE: gbic-invalid error detected on Gi1/1/1, putting Gi1/1/1 in err-disable state

Strange, the GLC-SX-MM is supported on the 3750. The interface capabilities looked fine:

3750_sw1#sh int gig 1/1/1 capa
GigabitEthernet1/1/1
  Model:                 WS-C3750X-48P
  Type:                  1000BaseSX SFP
  Speed:                 1000
  Duplex:                full
  Trunk encap. type:     802.1Q,ISL
  Trunk mode:            on,off,desirable,nonegotiate
  Channel:               yes
  Broadcast suppression: percentage(0-100)
  Flowcontrol:           rx-(off,on,desired),tx-(none)
  Fast Start:            yes
  QoS scheduling:        rx-(not configurable on per port basis),
                         tx-(4q3t) (3t: Two configurable values and one fixed.)
  CoS rewrite:           yes
  ToS rewrite:           yes
  UDLD:                  yes
  Inline power:          no
  SPAN:                  source/destination
  PortSecure:            yes

  Dot1x:                 yes
3750_sw1#

The status on the 3750 was not connected:

3750_sw1#sh int gig 1/1/1 status

Port      Name               Status       Vlan       Duplex  Speed Type
Gi1/1/1                      notconnect   1            auto   auto 1000BaseSX SFP
3750_sw1#

I then went to look at the N5K. It claimed it has an invalid SFP of an unkown type:

N5K1# sh int e1/5 status

--------------------------------------------------------------------------------
Port          Name               Status    Vlan      Duplex  Speed   Type
--------------------------------------------------------------------------------
Eth1/5        --                 sfpInvali 1         full    10G     SFP-1000BAS
N5K1#
N5K1# sh int e1/5 capa
Ethernet1/5
  Model:                 N5K-C5548UP-SUP
  Type (SFP capable):    10Gbase-(unknown)
  Speed:                 1000,10000
  Duplex:                full
  Trunk encap. type:     802.1Q
  Channel:               yes
  Broadcast suppression: no
  Flowcontrol:           rx-(off/on),tx-(off/on)
  Rate mode:             none
  QOS scheduling:        rx-(6q1t),tx-(1p6q0t)
  CoS rewrite:           no
  ToS rewrite:           no
  SPAN:                  yes
  UDLD:                  yes
  Link Debounce:         yes
  Link Debounce Time:    yes
  MDIX:                  no
  Pvlan Trunk capable:   yes
  TDR capable:           no
  FabricPath capable:    yes
  Port mode:             Switched
  FEX Fabric:            yes

N5K1#

The N5K also claimed there was no transceiver:

N5K1#sh log | inc 1/5
...
2012 Apr 6 18:10:26 N5K1 %ETHPORT-5-IF_DOWN_NONE: Interface Ethernet1/5 is down (None)
...
2012 Apr 6 19:06:10 N5K1 %ETHPORT-5-IF_HARDWARE: Interface Ethernet1/5, hardware type changed to No-Transceiver

I scratched my head a bit, and then came up with the underlying issue – when installing 1GE optics in the N5K, you need to manually set the speed. If you look back, you will see the previous ‘sh int e1/5 status‘ on the N5K shows a speed of 10G. So after the appropriate “speed 1000” command to the e1/5 interface on the N5K, the link came up on both ends.

N5K1# sh int e1/5 stat

--------------------------------------------------------------------------------
Port          Name               Status    Vlan      Duplex  Speed   Type
--------------------------------------------------------------------------------
Eth1/5        --                 connected 1         full    1000     SFP-1000BAS
N5K1#

Summary: The newer versions of NX-OS support 1G or 10G on the N5K, but the N5K is not auto-speed sensing. If you run into something similar, even error messages on the remote device, check the port speed on the N5K.

— cwr

Carole Warner Reece

Architect

A senior network consultant with more than fifteen years of industry experience, Carole is one of our most highly experienced network professionals. Her current focus is on the data center and on network infrastructure.

View more Posts

 

Virgilio “BONG” dela Cruz Jr.

CCDP, CCNA V, CCNP, Cisco IPS Express Security for AM/EE
Field Solutions Architect, Tech Data

Virgilio “Bong” has sixteen years of professional experience in IT industry from academe, technical and customer support, pre-sales, post sales, project management, training and enablement. He has worked in Cisco Technical Assistance Center (TAC) as a member of the WAN and LAN Switching team. Bong now works for Tech Data as the Field Solutions Architect with a focus on Cisco Security and holds a few Cisco certifications including Fire Jumper Elite.

 

John Cavanaugh

CCIE #1066, CCDE #20070002, CCAr
Chief Technology Officer, Practice Lead Security Services, NetCraftsmen

John is our CTO and the practice lead for a talented team of consultants focused on designing and delivering scalable and secure infrastructure solutions to customers across multiple industry verticals and technologies. Previously he has held several positions including Executive Director/Chief Architect for Global Network Services at JPMorgan Chase. In that capacity, he led a team managing network architecture and services.  Prior to his role at JPMorgan Chase, John was a Distinguished Engineer at Cisco working across a number of verticals including Higher Education, Finance, Retail, Government, and Health Care.

He is an expert in working with groups to identify business needs, and align technology strategies to enable business strategies, building in agility and scalability to allow for future changes. John is experienced in the architecture and design of highly available, secure, network infrastructure and data centers, and has worked on projects worldwide. He has worked in both the business and regulatory environments for the design and deployment of complex IT infrastructures.