Re: [gpfsug-discuss] [Newsletter] Re: Problem with mmlscluster and callback scripts

Andrew Beattie Mon, 10 Sep 2018 05:20:02 -0700

Matthias,

If you are only using 2 nodes you need to use tie breaker disks,

other wise as per Aaron's comment to avoid split brain scenarios you need to have a minimum of 3 nodes to validate

(n/2+1) quorum nodes

Can you create a 3rd node as a virtual server and apply the role of quorum, to it, it also becomes useful as a gui server / management server?

regards,

Andrew Beattie

Software Defined Storage - IT Specialist

Phone: 614-2133-7927

E-mail: [email protected]

----- Original message -----
From: Matthias Knigge <[email protected]>
Sent by: [email protected]
To: gpfsug main discussion list <[email protected]>
Cc:
Subject: Re: [gpfsug-discuss] [Newsletter] Re: Problem with mmlscluster and callback scripts
Date: Mon, Sep 10, 2018 9:21 PM

Hi Fred,

I have the same problem with the version 5.0.1.0.

Thanks,

Matthias

Best Regards

Matthias Knigge
R&D File Based Media Solutions

Rohde & Schwarz
GmbH & Co. KG
Hanomaghof 1
30449 Hannover
Telefon +49 511 67 80 7 213
Fax +49 511 37 19 74
Internet: [email protected]
------------------------------------------------------------
Geschäftsführung / Executive Board: Christian Leicher (Vorsitzender / Chairman), Peter Riedel, Sitz der Gesellschaft / Company's Place of Business: München, Registereintrag / Commercial Register No.: HRA 16 270, Persönlich haftender Gesellschafter / Personally Liable Partner: RUSEG Verwaltungs-GmbH, Sitz der Gesellschaft / Company's Place of Business: München, Registereintrag / Commercial Register No.: HRB 7 534, Umsatzsteuer-Identifikationsnummer (USt-IdNr.) / VAT Identification No.: DE 130 256 683, Elektro-Altgeräte Register (EAR) / WEEE Register No.: DE 240 437 86

From: [email protected] <[email protected]> On Behalf Of Frederick Stock
Sent: Friday, September 07, 2018 3:20 PM
To: gpfsug main discussion list <[email protected]>
Subject: *EXT* [Newsletter] Re: [gpfsug-discuss] Problem with mmlscluster and callback scripts

Are you really running version 5.0.2? If so then I presume you have a beta version since it has not yet been released. For beta problems there is a specific feedback mechanism that should be used to report problems.

Fred
__________________________________________________
Fred Stock | IBM Pittsburgh Lab | 720-430-8821
[email protected]

From: Matthias Knigge <[email protected]>
To: "[email protected]" <[email protected]>
Date: 09/07/2018 08:08 AM
Subject: [gpfsug-discuss] Problem with mmlscluster and callback scripts
Sent by: [email protected]

Hello together,

I am using the version 5.0.2.0 of GPFS and have problems with the command mmlscluster and callback-scripts. It is a small cluster of two nodes only. If I shutdown one of the nodes sometimes mmlscluster reports the following output:
[root@gpfs-tier1 gpfs5.2]# mmgetstate

Node number Node name GPFS state
-------------------------------------------
1 gpfs-tier1 arbitrating
[root@gpfs-tier1 gpfs5.2]# mmlscluster
ssh: connect to host gpfs-tier2 port 22: No route to host
mmlscluster: Unable to retrieve GPFS cluster files from node gpfs-tier2
mmlscluster: Command failed. Examine previous error messages to determine cause.

Normally the output is like this:

[root@gpfs-tier1 gpfs5.2]# mmlscluster

GPFS cluster information
========================
GPFS cluster name: TIERCLUSTER.gpfs-tier1
GPFS cluster id: 12458173498278694815
GPFS UID domain: TIERCLUSTER.gpfs-tier1
Remote shell command: /usr/bin/ssh
Remote file copy command: /usr/bin/scp
Repository type: server-based

GPFS cluster configuration servers:
-----------------------------------
Primary server: gpfs-tier2
Secondary server: gpfs-tier1

Node Daemon node name IP address Admin node name Designation
----------------------------------------------------------------------
1 gpfs-tier1 192.168.178.10 gpfs-tier1 quorum-manager
2 gpfs-tier2 192.168.178.11 gpfs-tier2 quorum-manager

[root@gpfs-tier1 gpfs5.2]# mmlscallback
NodeDownCallback
command = /var/mmfs/rs/nodedown.ksh
priority = 1
event = quorumNodeLeave
parms = %eventNode %quorumNodes

NodeUpCallback
command = /var/mmfs/rs/nodeup.ksh
priority = 1
event = quorumNodeJoin
parms = %eventNode %quorumNodes

If I shutdown the filesystem via mmshutdown the callback script works but if I shutdown the whole node the scripts does not run.
The latest log-entry in mmfs.log.latest shows only this information:

2018-09-07_13:12:36.724+0200: [I] Cluster Manager connection broke. Probing cluster TIERCLUSTER.gpfs-tier1
2018-09-07_13:12:37.226+0200: [E] Unable to contact enough other quorum nodes during cluster probe.
2018-09-07_13:12:37.226+0200: [E] Lost membership in cluster TIERCLUSTER.gpfs-tier1. Unmounting file systems.
2018-09-07_13:12:38.448+0200: [N] Connecting to 192.168.178.11 gpfs-tier2 <c0p1>

Could anybody help me in this case? I want to try to start a script if one node goes down or up to change the roles for starting the filesystem. The callback event NodeLeave and NodeJoin do not run too.
Any more information required? If yes, please let me know!

Many thanks in advance and a nice weekend!
Matthias

Best Regards

Matthias Knigge
R&D File Based Media Solutions

Rohde & Schwarz
GmbH & Co. KG
Hanomaghof 1
30449 Hannover
Telefon +49 511 67 80 7 213
Fax +49 511 37 19 74
Internet: [email protected]
------------------------------------------------------------
Geschäftsführung / Executive Board: Christian Leicher (Vorsitzender / Chairman), Peter Riedel, Sitz der Gesellschaft / Company's Place of Business: München, Registereintrag / Commercial Register No.: HRA 16 270, Persönlich haftender Gesellschafter / Personally Liable Partner: RUSEG Verwaltungs-GmbH, Sitz der Gesellschaft / Company's Place of Business: München, Registereintrag / Commercial Register No.: HRB 7 534, Umsatzsteuer-Identifikationsnummer (USt-IdNr.) / VAT Identification No.: DE 130 256 683, Elektro-Altgeräte Register (EAR) / WEEE Register No.: DE 240 437 86
_______________________________________________
gpfsug-discuss mailing list
gpfsug-discuss at spectrumscale.org
http://gpfsug.org/mailman/listinfo/gpfsug-discuss

_______________________________________________
gpfsug-discuss mailing list
gpfsug-discuss at spectrumscale.org
http://gpfsug.org/mailman/listinfo/gpfsug-discuss

_______________________________________________
gpfsug-discuss mailing list
gpfsug-discuss at spectrumscale.org
http://gpfsug.org/mailman/listinfo/gpfsug-discuss

Re: [gpfsug-discuss] [Newsletter] Re: Problem with mmlscluster and callback scripts

Reply via email to