Re: [gpfsug-discuss] [EXTERNAL] Re: CES: IP address won't assign: "handleNetworkProblem with lock held"

2016-10-17 Thread Olaf Weiser 
ah .. I see. sorry, should have checked
that , so to stay with this example, the  IP
address 10.30.22.176is set by CES.. as a floating service IP .. something is insane.. are the smb/NFS
services running (systemctl ...)    and can you access the exports
from outside ?From:      
 "Oesterlin, Robert"
To:      
 gpfsug main discussion
list Date:      
 10/17/2016 12:02 PMSubject:    
   Re: [gpfsug-discuss]
[EXTERNAL] Re: CES: IP address won't assign: "handleNetworkProblem
with lock held"Sent by:    
   gpfsug-discuss-boun...@spectrumscale.orgNo - the :0 and :1 address are floating
addresses *assigned by CES* - it created those interfaces. The issue seems
to be that these are assigned and CES doesn't know it.  Bob OesterlinSr Storage Engineer, Nuance HPC Grid  From: 
on behalf of Olaf Weiser Reply-To: gpfsug main discussion list Date: Monday, October 17, 2016 at 1:53 PMTo: gpfsug main discussion list Subject: [EXTERNAL] Re: [gpfsug-discuss] CES: IP address won't assign:
"handleNetworkProblem with lock held" ah .. I see.. seems, that you already
has IP aliases around .. GPFS don't like it... eg. your node tct-gw01.infra.us.grid.nuance.com:
     inet 10.30.22.160/24   has already an alias -  10.30.22.176
... if I understand you answers correctly...from the doc'... [...] you need to provide a static IP adress, that is
not already[...] as an alias [...]  ___gpfsug-discuss mailing listgpfsug-discuss at spectrumscale.orghttp://gpfsug.org/mailman/listinfo/gpfsug-discuss

___
gpfsug-discuss mailing list
gpfsug-discuss at spectrumscale.org
http://gpfsug.org/mailman/listinfo/gpfsug-discuss

Re: [gpfsug-discuss] [EXTERNAL] Re: CES: IP address won't assign: "handleNetworkProblem with lock held"

2016-10-17 Thread Oesterlin, Robert 
No - the :0 and :1 address are floating addresses *assigned by CES* - it 
created those interfaces. The issue seems to be that these are assigned and CES 
doesn't know it.


Bob Oesterlin
Sr Storage Engineer, Nuance HPC Grid



From:  on behalf of Olaf Weiser 

Reply-To: gpfsug main discussion list 
Date: Monday, October 17, 2016 at 1:53 PM
To: gpfsug main discussion list 
Subject: [EXTERNAL] Re: [gpfsug-discuss] CES: IP address won't assign: 
"handleNetworkProblem with lock held"

ah .. I see.. seems, that you already has IP aliases around .. GPFS don't like 
it...
eg. your node tct-gw01.infra.us.grid.nuance.com:  inet 10.30.22.160/24
has already an alias -  10.30.22.176 ... if I understand you answers 
correctly...

from the doc'... [...] you need to provide a static IP adress, that is not 
already[...] as an alias [...]

___
gpfsug-discuss mailing list
gpfsug-discuss at spectrumscale.org
http://gpfsug.org/mailman/listinfo/gpfsug-discuss

Re: [gpfsug-discuss] CES: IP address won't assign: "handleNetworkProblem with lock held"

2016-10-17 Thread Olaf Weiser 
ah .. I see.. seems, that you already has
IP aliases around .. GPFS don't like it... eg. your node tct-gw01.infra.us.grid.nuance.com:
     inet 10.30.22.160/24    has already an alias -  10.30.22.176 ... if I understand you answers correctly...from the doc'... [...] you need to provide
a static IP adress, that is not already[...] as an alias
[...]  Mit freundlichen Grüßen / Kind regards Olaf Weiser EMEA Storage Competence Center Mainz, German / IBM Systems, Storage Platform,---IBM DeutschlandIBM Allee 171139 EhningenPhone: +49-170-579-44-66E-Mail: olaf.wei...@de.ibm.com---IBM Deutschland GmbH / Vorsitzender des Aufsichtsrats: Martin JetterGeschäftsführung: Martina Koederitz (Vorsitzende), Susanne Peter, Norbert
Janzen, Dr. Christian Keller, Ivo Koerner, Markus KoernerSitz der Gesellschaft: Ehningen / Registergericht: Amtsgericht Stuttgart,
HRB 14562 / WEEE-Reg.-Nr. DE 99369940 From:      
 "Oesterlin, Robert"
To:      
 gpfsug main discussion
list Date:      
 10/17/2016 11:43 AMSubject:    
   [gpfsug-discuss]
CES: IP address won't assign:        "handleNetworkProblem
with lock held"Sent by:    
   gpfsug-discuss-boun...@spectrumscale.orgYes - so interesting - it looks like
the nodes have the addresses assigned but CES doesn’t know that. [root@tct-gw01 ~]# mmlscluster --ces GPFS cluster information  GPFS cluster name:    
    nrg1-tct.nrg1.us.grid.nuance.com  GPFS cluster id:    
      17869514639699411874 Cluster Export Services global parameters-  Shared root directory:  
             /gpfs/fs1/ces  Enabled Services:    
                NFS  Log level:      
                     0  Address distribution policy:
         even-coverage Node  Daemon node name  
         IP address       CES IP
address list---   4   tct-gw01.infra.us.grid.nuance.com
10.30.22.160     None   5   tct-gw02.infra.us.grid.nuance.com
10.30.22.161     None   6   tct-gw03.infra.us.grid.nuance.com
10.30.22.162     None   7   tct-gw04.infra.us.grid.nuance.com
10.30.22.163     None [root@tct-gw01 ~]# mmdsh -N cesnodes
"ip a | grep 10.30.22." | sort -k 1tct-gw01.infra.us.grid.nuance.com:
     inet 10.30.22.160/24 brd 10.30.22.255 scope global
bond0tct-gw01.infra.us.grid.nuance.com:
     inet 10.30.22.176/24 brd 10.30.22.255 scope global
secondary bond0:0 tct-gw02.infra.us.grid.nuance.com:
     inet 10.30.22.161/24 brd 10.30.22.255 scope global
bond0tct-gw02.infra.us.grid.nuance.com:
     inet 10.30.22.177/24 brd 10.30.22.255 scope global
secondary bond0:0tct-gw02.infra.us.grid.nuance.com:
     inet 10.30.22.178/24 brd 10.30.22.255 scope global
secondary bond0:1 tct-gw03.infra.us.grid.nuance.com:
     inet 10.30.22.162/24 brd 10.30.22.255 scope global
bond0tct-gw03.infra.us.grid.nuance.com:
     inet 10.30.22.178/24 brd 10.30.22.255 scope global
secondary bond0:0 tct-gw04.infra.us.grid.nuance.com:
     inet 10.30.22.163/24 brd 10.30.22.255 scope global
bond0tct-gw04.infra.us.grid.nuance.com:
     inet 10.30.22.179/24 brd 10.30.22.255 scope global
secondary bond0:0  Bob OesterlinSr Storage Engineer, Nuance HPC Grid  From: 
on behalf of Olaf Weiser Reply-To: gpfsug main discussion list Date: Monday, October 17, 2016 at 1:27 PMTo: gpfsug main discussion list Subject: [EXTERNAL] Re: [gpfsug-discuss] CES: IP address won't assign:
"handleNetworkProblem with lock held" ip a | grep 10.30.22.___gpfsug-discuss mailing listgpfsug-discuss at spectrumscale.orghttp://gpfsug.org/mailman/listinfo/gpfsug-discuss___
gpfsug-discuss mailing list
gpfsug-discuss at spectrumscale.org
http://gpfsug.org/mailman/listinfo/gpfsug-discuss

Re: [gpfsug-discuss] CES: IP address won't assign: "handleNetworkProblem with lock held"

2016-10-17 Thread Olaf Weiser 
simple question  -sorry for that -
your Nodes.. do they have an IP address in the same subnet as your IP address
listed here ?and if, is this network up n running
so that GPFS can find/detect it ?what tells mmlscluster --ces ?from each node - assuming class C /24
network , do a ip a | grep 10.30.22.cheersFrom:      
 "Oesterlin, Robert"
To:      
 gpfsug main discussion
list Date:      
 10/17/2016 08:00 AMSubject:    
   [gpfsug-discuss]
CES: IP address won't assign:        "handleNetworkProblem
with lock held"Sent by:    
   gpfsug-discuss-boun...@spectrumscale.orgCan anyone help me pinpoint the issue
here? These message repeat and the IP addresses never get assigned. [root@tct-gw01 ~]# tail /var/mmfs/gen/mmfslogMon Oct 17 10:57:55 EDT 2016: mmcesnetworkmonitor:
Found unassigned address 10.30.22.178Mon Oct 17 10:57:55 EDT 2016: mmcesnetworkmonitor:
Found unassigned address 10.30.22.179Mon Oct 17 10:57:55 EDT 2016: mmcesnetworkmonitor:
Found unassigned address 10.30.22.177Mon Oct 17 10:57:55 EDT 2016: mmcesnetworkmonitor:
Found unassigned address 10.30.22.176Mon Oct 17 10:57:55 EDT 2016: mmcesnetworkmonitor:
handleNetworkProblem with lock held: assignIP 10.30.22.178_0-_+,10.30.22.179_0-_+,10.30.22.177_0-_+,10.30.22.176_0-_+
1Mon Oct 17 10:57:55 EDT 2016: mmcesnetworkmonitor:
Assigning addresses: 10.30.22.178_0-_+,10.30.22.179_0-_+,10.30.22.177_0-_+,10.30.22.176_0-_+Mon Oct 17 10:57:55 EDT 2016: mmcesnetworkmonitor:
moveCesIPs: 10.30.22.178_0-_+,10.30.22.179_0-_+,10.30.22.177_0-_+,10.30.22.176_0-_+  [root@tct-gw01 ~]# mmces state show
-aNODE          
                     
    AUTH          AUTH_OBJ    
 NETWORK       NFS          
OBJ           SMB        
  CEStct-gw01.infra.us.grid.nuance.com  
     DISABLED      DISABLED    
 HEALTHY       HEALTHY       DISABLED
     DISABLED      HEALTHYtct-gw02.infra.us.grid.nuance.com  
     DISABLED      DISABLED    
 HEALTHY       HEALTHY       DISABLED
     DISABLED      HEALTHYtct-gw03.infra.us.grid.nuance.com  
     DISABLED      DISABLED    
 HEALTHY       HEALTHY       DISABLED
     DISABLED      HEALTHYtct-gw04.infra.us.grid.nuance.com  
     DISABLED      DISABLED    
 HEALTHY       HEALTHY       DISABLED
     DISABLED      HEALTHY[root@tct-gw01 ~]# mmces address list Address        
Node                    
           Group      Attribute-10.30.22.178    unassigned
                     
   none       none10.30.22.179    unassigned
                     
   none       none10.30.22.177    unassigned
                     
   none       none10.30.22.176    unassigned
                     
   none       none  Bob OesterlinSr Storage Engineer, Nuance HPC Grid507-269-0413 ___gpfsug-discuss mailing listgpfsug-discuss at spectrumscale.orghttp://gpfsug.org/mailman/listinfo/gpfsug-discuss

___
gpfsug-discuss mailing list
gpfsug-discuss at spectrumscale.org
http://gpfsug.org/mailman/listinfo/gpfsug-discuss