Hello Everyone,
I'm working on deploying a tried and tested xcat database that was successful
on Cisco Catalyst switches. The MN is now hooked up to a Cisco Nexus 5672U
running Nexus 6000 IOS and manages a ToR fex switch attached to IMM on the
nodes and a partnered 5624Q connected to the mellanox 10gb ports on the nodes.
RACK 1
xCAT MN -->5672U (mstrswitch) --> 5624Q (switch 1 )
--> 2248TP-E
(FEX 101)
I've tried lots of configurations and have attached the configurations I have
right now to this e-mail. I'm getting extremely slow discoveries. One node is
discovered every 10-15 minutes. The nodes sometimes do not discover at all and
sit in a limbo where they are in genesis kernel with the correct hostname but
xCAT does not see them. Similar happens with nodes that show the blue discover
beacon, they have their hostname and correct static IP but xCAT does not see
the device as discovered and rpower, rbeacon and rinv time out. I can ping and
do a name resolution on the node but it does not respond to rpower commands.
These are some of the messages worth noting that I receive on journal
Apr 30 14:37:30 serverx.cluster xcat[1813]: xcatd: Processing discovery request
from 172.16.1.4
Apr 30 14:37:31 serverx.cluster xcat[1813]: Discover info: configure static BMC
ip:r1c1n1p1i1-bmc for host_node:r1c1n1p1i1.
Apr 30 14:37:31 serverx.cluster xcat[8003]: xCAT: Allowing rspconfig to
r1c1n1p1i1 ip=r1c1n1p1i1-bmc for root from localhost
Apr 30 14:37:45 serverx.cluster xcatd[1813]: Discovery worker: switch instance:
nodediscover instance: Failed to notify 172.16.1.4 that it's actually r1c1n1p1i1
Apr 30 14:37:46 serverx.cluster xcat[1813]: Discover info: configure static BMC
ip:r1c1n1p1i1-bmc for host_node:r1c1n1p1i1.
Apr 30 14:37:46 serverx.cluster xcat[8029]: xCAT: Allowing rspconfig to
r1c1n1p1i1 ip=r1c1n1p1i1-bmc for root from localhost
Apr 30 14:38:01 serverx.cluster xcatd[1813]: Discovery worker: fsp instance:
nodediscover instance: Failed to notify 172.16.1.4 that it's actually
r1c1n1p1i1.
I've narrowed down a lot of this to the nexus switch and things work entirely
different on these switches. Can someone chime in on what I could be missing
from my switch configuration?
Thank you,
Edward Nunez
Lead Integration Network Engineer| Certified : CCNA R&S
SHI International Corp.| [email protected]<mailto:[email protected]> | Tel:
1(732)564-8188
hostname mstrswitch
feature telnet
cfs eth distribute
feature interface-vlan
feature lacp
feature dhcp
feature vpc
feature lldp
feature vtp
feature fex
username admin password 5 XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX role
network-admin
username admin sshkey ssh-rsa
AAAAB3NzaC1yc2EAAAADAQABAAAAgQC5rwIBd70ycykhsGPEOIFmj65N//VCpClfvwXTcVY6AVnbVg5AcpwHp1r2AP
0sl/N/C038AxU1OesVwTsgr1wZ3jIxNyZHiJ3QtLXbPUS9LJFN5YeT+9AtaylNOduczf1r3LxfO50z2xEhHoiXbZ4pc7f/TQsqVUU9L8pSd2n8ew==
username xcat password 5 XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX role
network-operator
banner motd #Nexus 6000 Switch
#
ip domain-lookup
logging event link-status default
service unsupported-transceiver
ip access-list acl-01
10 permit ip 172.16.0.255/32 any
fex 100
pinning max-links 1
description "FEX0100"
fex 101
pinning max-links 1
description "FEX0101"
fex 102
pinning max-links 1
description "fex102"
snmp-server user xcat network-operator auth sha
XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX localizedkey
snmp-server user admin network-admin auth sha
XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX localizedkey
snmp-server enable traps callhome event-notify
snmp-server enable traps callhome smtp-send-fail
snmp-server enable traps cfs state-change-notif
snmp-server enable traps lldp lldpRemTablesChange
snmp-server enable traps cfs merge-failure
snmp-server enable traps aaa server-state-change
snmp-server enable traps upgrade UpgradeOpNotifyOnCompletion
snmp-server enable traps upgrade UpgradeJobStatusNotify
snmp-server enable traps feature-control FeatureOpStatusChange
snmp-server enable traps sysmgr cseFailSwCoreNotifyExtended
snmp-server enable traps config ccmCLIRunningConfigChanged
snmp-server enable traps snmp authentication
snmp-server enable traps link cisco-xcvr-mon-status-chg
snmp-server enable traps vtp notifs
snmp-server enable traps vtp vlancreate
snmp-server enable traps vtp vlandelete
snmp-server enable traps bridge newroot
snmp-server enable traps bridge topologychange
snmp-server enable traps stpx inconsistency
snmp-server enable traps stpx root-inconsistency
snmp-server enable traps stpx loop-inconsistency
snmp-server enable traps poe portonoff
snmp-server enable traps poe pwrusageon
snmp-server enable traps poe pwrusageoff
snmp-server enable traps poe police
snmp-server community public group network-operator
snmp-server community private group network-admin
snmp-server community public use-acl acl-01
snmp-server community private use-acl acl-01
spanning-tree port type edge default
service dhcp
ip dhcp relay
vrf context management
vpc domain 50
peer-keepalive destination 1.1.1.1 source 192.168.70.248
interface Vlan1
no shutdown
ip address 172.16.0.254/12
interface Ethernet1/1
interface Ethernet1/2
interface Ethernet1/3
interface Ethernet1/4
interface Ethernet1/5
switchport mode trunk
switchport trunk allowed vlan 1
interface Ethernet1/6
switchport mode trunk
switchport trunk allowed vlan 1
interface Ethernet1/7
switchport mode trunk
switchport trunk allowed vlan 1
interface Ethernet1/8
switchport mode trunk
switchport trunk allowed vlan 1
interface Ethernet1/9
interface Ethernet1/10
interface Ethernet1/11
interface Ethernet1/12
interface Ethernet1/13
interface Ethernet1/14
interface Ethernet1/15
interface Ethernet1/16
interface Ethernet1/17
interface Ethernet1/18
interface Ethernet1/19
interface Ethernet1/20
interface Ethernet1/21
interface Ethernet1/22
interface Ethernet1/23
interface Ethernet1/24
interface Ethernet1/25
interface Ethernet1/26
interface Ethernet1/27
interface Ethernet1/28
interface Ethernet1/29
interface Ethernet1/30
interface Ethernet1/31
interface Ethernet1/32
interface Ethernet1/33
interface Ethernet1/34
interface Ethernet1/35
interface Ethernet1/36
interface Ethernet1/37
interface Ethernet1/38
interface Ethernet1/39
interface Ethernet1/40
interface Ethernet1/41
interface Ethernet1/42
interface Ethernet1/43
interface Ethernet1/44
interface Ethernet1/45
interface Ethernet1/46
interface Ethernet1/47
interface Ethernet1/48
speed 1000
interface Ethernet2/1/1
switchport mode fex-fabric
fex associate 101
interface Ethernet2/1/2
switchport mode fex-fabric
fex associate 102
interface Ethernet2/1/3
interface Ethernet2/1/4
interface Ethernet2/2/1
interface Ethernet2/2/2
interface Ethernet2/2/3
interface Ethernet2/2/4
interface Ethernet2/3/1
interface Ethernet2/3/2
interface Ethernet2/3/3
interface Ethernet2/3/4
interface Ethernet2/4/1
interface Ethernet2/4/2
interface Ethernet2/4/3
interface Ethernet2/4/4
interface Ethernet2/5/1
interface Ethernet2/5/2
interface Ethernet2/5/3
interface Ethernet2/5/4
interface Ethernet2/6/1
interface Ethernet2/6/2
interface Ethernet2/6/3
interface Ethernet2/6/4
interface mgmt0
vrf member management
interface Ethernet101/1/1
interface Ethernet101/1/2
interface Ethernet101/1/3
interface Ethernet101/1/4
interface Ethernet101/1/5
interface Ethernet101/1/6
interface Ethernet101/1/7
interface Ethernet101/1/8
interface Ethernet101/1/9
interface Ethernet101/1/10
interface Ethernet101/1/11
interface Ethernet101/1/12
interface Ethernet101/1/13
interface Ethernet101/1/14
interface Ethernet101/1/15
interface Ethernet101/1/16
interface Ethernet101/1/17
interface Ethernet101/1/18
interface Ethernet101/1/19
interface Ethernet101/1/20
interface Ethernet101/1/21
interface Ethernet101/1/22
interface Ethernet101/1/23
interface Ethernet101/1/24
interface Ethernet101/1/25
interface Ethernet101/1/26
interface Ethernet101/1/27
interface Ethernet101/1/28
interface Ethernet101/1/29
interface Ethernet101/1/30
interface Ethernet101/1/31
interface Ethernet101/1/32
interface Ethernet101/1/33
interface Ethernet101/1/34
interface Ethernet101/1/35
interface Ethernet101/1/36
interface Ethernet101/1/37
interface Ethernet101/1/38
interface Ethernet101/1/39
interface Ethernet101/1/40
interface Ethernet101/1/41
interface Ethernet101/1/42
interface Ethernet101/1/43
interface Ethernet101/1/44
interface Ethernet101/1/45
interface Ethernet101/1/46
interface Ethernet101/1/47
interface Ethernet101/1/48
line console
line vty
boot kickstart bootflash:/n6000-uk9-kickstart.7.0.8.N1.1.bin
boot system bootflash:/n6000-uk9.7.0.8.N1.1.bin
poap transit
version 7.1(0)N1(1b)
interface breakout slot 1 port 1-12 map 10g-4x
interface breakout slot 2 port 1-12 map 10g-4x
hostname switch1
no feature telnet
feature interface-vlan
feature lacp
feature dhcp
feature lldp
feature vtp
username admin password 5 XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX role
network-admin
username admin sshkey ssh-rsa
AAAAB3NzaC1yc2EAAAADAQABAAAAgQDuHMrsMcUaG9Gr+FUrxS9Cb/eMC/TSaXCcI+uvnRhMidUyWd2enzd2fV+WvU
kHgBw8yawNQGAeD7Va8UuhRjf+bcLpnz2OH87f5ZVqorsPxGzYbmCSj0LbAf9O8RxZNelB74NEMZOA8OZuT0xixF4uw4/oU4LhCpD4OnDZR60EPw==
username xcat password 5 XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX role
network-operator
ip domain-lookup
logging event link-status default
service unsupported-transceiver
ip access-list acl-01
10 permit ip 172.16.0.255/32 any
snmp-server user xcat network-operator auth sha
XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX localizedkey
snmp-server user admin network-admin auth sha
XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX localizedkey
snmp-server enable traps callhome event-notify
snmp-server enable traps callhome smtp-send-fail
snmp-server enable traps cfs state-change-notif
snmp-server enable traps lldp lldpRemTablesChange
snmp-server enable traps cfs merge-failure
snmp-server enable traps aaa server-state-change
snmp-server enable traps feature-control FeatureOpStatusChange
snmp-server enable traps sysmgr cseFailSwCoreNotifyExtended
snmp-server enable traps config ccmCLIRunningConfigChanged
snmp-server enable traps snmp authentication
snmp-server enable traps link cisco-xcvr-mon-status-chg
snmp-server enable traps vtp notifs
snmp-server enable traps vtp vlancreate
snmp-server enable traps vtp vlandelete
snmp-server enable traps bridge newroot
snmp-server enable traps bridge topologychange
snmp-server enable traps stpx inconsistency
snmp-server enable traps stpx root-inconsistency
snmp-server enable traps stpx loop-inconsistency
snmp-server enable traps system Clock-change-notification
snmp-server enable traps feature-control ciscoFeatOpStatusChange
snmp-server enable traps syslog message-generated
snmp-server community private group network-admin
snmp-server community public group network-operator
snmp-server community private use-acl acl-01
snmp-server community public use-acl acl-01
spanning-tree port type edge default
service dhcp
ip dhcp relay
vrf context management
interface Vlan1
no shutdown
ip address 172.16.0.1/12
interface Ethernet1/1/1
interface Ethernet1/1/2
interface Ethernet1/1/3
interface Ethernet1/1/4
interface Ethernet1/2/1
interface Ethernet1/2/2
interface Ethernet1/2/3
interface Ethernet1/2/4
interface Ethernet1/3/1
interface Ethernet1/3/2
interface Ethernet1/3/3
interface Ethernet1/3/4
interface Ethernet1/4/1
interface Ethernet1/4/2
interface Ethernet1/4/3
interface Ethernet1/4/4
interface Ethernet1/5/1
interface Ethernet1/5/2
interface Ethernet1/5/3
interface Ethernet1/5/4
interface Ethernet1/6/1
interface Ethernet1/6/2
interface Ethernet1/6/3
interface Ethernet1/6/4
interface Ethernet1/7/1
interface Ethernet1/7/2
interface Ethernet1/7/3
interface Ethernet1/7/4
interface Ethernet1/8/1
interface Ethernet1/8/2
interface Ethernet1/8/3
interface Ethernet1/8/4
interface Ethernet1/9/1
interface Ethernet1/9/2
interface Ethernet1/9/3
interface Ethernet1/9/4
interface Ethernet1/10/1
interface Ethernet1/10/2
interface Ethernet1/10/3
interface Ethernet1/10/4
interface Ethernet1/11/1
switchport mode trunk
switchport trunk allowed vlan 1
interface Ethernet1/11/2
switchport mode trunk
switchport trunk allowed vlan 1
interface Ethernet1/11/3
switchport mode trunk
switchport trunk allowed vlan 1
interface Ethernet1/11/4
switchport mode trunk
switchport trunk allowed vlan 1
interface Ethernet1/12/1
interface Ethernet1/12/2
interface Ethernet1/12/3
interface Ethernet1/12/4
interface Ethernet2/1/1
interface Ethernet2/1/2
interface Ethernet2/1/3
interface Ethernet2/1/4
interface Ethernet2/2/1
interface Ethernet2/2/2
interface Ethernet2/2/3
interface Ethernet2/2/4
interface Ethernet2/3/1
interface Ethernet2/3/2
interface Ethernet2/3/3
interface Ethernet2/3/4
interface Ethernet2/4/1
interface Ethernet2/4/2
interface Ethernet2/4/3
interface Ethernet2/4/4
interface Ethernet2/5/1
interface Ethernet2/5/2
interface Ethernet2/5/3
interface Ethernet2/5/4
interface Ethernet2/6/1
interface Ethernet2/6/2
interface Ethernet2/6/3
interface Ethernet2/6/4
interface Ethernet2/7/1
interface Ethernet2/7/2
interface Ethernet2/7/3
interface Ethernet2/7/4
interface Ethernet2/8/1
interface Ethernet2/8/2
interface Ethernet2/8/3
interface Ethernet2/8/4
interface Ethernet2/9/1
interface Ethernet2/9/2
interface Ethernet2/9/3
interface Ethernet2/9/4
interface Ethernet2/10/1
interface Ethernet2/10/2
interface Ethernet2/10/3
interface Ethernet2/10/4
interface Ethernet2/11/1
interface Ethernet2/11/2
interface Ethernet2/11/3
interface Ethernet2/11/4
interface Ethernet2/12/1
interface Ethernet2/12/2
interface Ethernet2/12/3
interface Ethernet2/12/4
interface mgmt0
vrf member management
line console
line vty
boot kickstart bootflash:/n6000-uk9-kickstart.7.1.0.N1.1b.bin
boot system bootflash:/n6000-uk9.7.1.0.N1.1b.bin
poap transit
------------------------------------------------------------------------------
Find and fix application performance issues faster with Applications Manager
Applications Manager provides deep performance insights into multiple tiers of
your business applications. It resolves application problems quickly and
reduces your MTTR. Get your free trial!
https://ad.doubleclick.net/ddm/clk/302982198;130105516;z
_______________________________________________
xCAT-user mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/xcat-user
