Last week when I was working on my LAB I found a pretty interesting status message on my VCM. By the by I am using a C7000 enclosure and mixed of total 10 blades which combines half height and full height.
So now when I look at the Status message of my enclosure domain I found there is no communication from VC (Virtual Connect).
I started having a hard time finding what happened to our VCM (Virtual Connect Ethernet Module). All of a sudden we lost communications to our servers and while investigating I can see that both our VC is showing it is gone. Below are the screenshots. While I went back to my enclosure it is showing me the LAN connections are blinking. So now am completely confused as to what direction I need to go for.
Further troubleshooting revealed that apparently there is some problem with the DNS and it is a known issue. So digging to it we got the external CA (Customer Advisory) out which talked about this. So now there could be two ways to get out of it.
Quick fix is to disable DNS on all the Virtual Connect Modules. Permanent fix is to upgrade to VC 3.17 firmware. I immediately started working on the first option which would not have create any issues at all. So I went ahead to the Enclosure Settings > Enclosure Bay IP Addressing and then looked under the “Interconnect Bays” tab and removed DNS server IP address entries from the bays that include VC Ethernet Modules and clicked Apply.
Within 5 minutes, the DNS settings for the modules get updated and normal module communication has been restored. If that does not happen then from the VCM GUI, select “Tools” => “Reset Virtual Connect Manager.” This will force resynchronization of the modules if not synchronized already.