Re: [gpfsug-discuss] How to join GNR nodes to a non-GNR cluster
For: - A NetApp system with hardware RAID- SpectrumScale 4.2.3-13 running on top of the NetApp < --- Are these NSD servers in their GPFS roles (where Scale "runs on top"?- A GL2 system with ESS 5.3.2.1 (Spectrum Scale 5.0.2-1)What I need to do is to merge the GL2 in the other GPFS cluster (running on the NetApp) without loosing, of course, the RecoveryGroup configuration, etc.I'd like to ask the experts1. whether it is feasible, considering the difference in the GPFS versions, architectures differences (x86_64 vs. power)2. if yes, whether anyone already did something like this and what is the best strategy suggested3. finally: is there any documentation dedicated to that, or at least inspiring the correct procedure ? .. Some observations: 1) Why do you want to MERGE the GL2 into a single cluster with the rest cluster, rather than simply allowing remote mount of the ESS servers by the other GPFS (NSD client) nodes? 2) Interop of the mix of 4.2 and 5.0 levels should be allowed by our coexistence rules. 3) Mixing x86 and Power, especially as NSD servers, should pose no issues. Having them as separate file systems (NetApp vs. ESS) means no concerns regarding varying architectures within the same fs serving or failover scheme. Mixing such as compute nodes would mean some performance differences across the different clients, but you haven't described your compute (NSD client) details. Lyle - Original message -From: "Tomer Perry" Sent by: gpfsug-discuss-boun...@spectrumscale.orgTo: gpfsug main discussion list Cc:Subject: [EXTERNAL] Re: [gpfsug-discuss] How to join GNR nodes to a non-GNR clusterDate: Tue, Dec 3, 2019 10:03 AM Hi,Actually, I believe that GNR is not a limiting factor here. mmexportfs and mmimportfs ( man mm??portfs) will export/import GNR configuration as well:"If the specified file system device is a IBM Spectrum Scale RAID-based file system, then all affected IBM Spectrum Scale RAID objects will be exported as well. This includes recovery groups, declustered arrays, vdisks, and any other file systems that are based on these objects. For more information about IBM Spectrum Scale RAID, see IBM Spectrum Scale RAID: Administration. "OTOH, I suspect that due to the version mismatch, it wouldn't work - since I would assume that the cluster config version is to high for the NetApp based cluster.I would also suspect that the filesystem version on the ESS will be different.Regards,Tomer PerryScalable I/O Development (Spectrum Scale)email: t...@il.ibm.com1 Azrieli Center, Tel Aviv 67021, IsraelGlobal Tel: +1 720 3422758Israel Tel: +972 3 9188625Mobile: +972 52 2554625From: "Olaf Weiser" To: gpfsug main discussion list Date: 03/12/2019 16:54Subject: [EXTERNAL] Re: [gpfsug-discuss] How to join GNR nodes to a non-GNR clusterSent by: gpfsug-discuss-boun...@spectrumscale.org Hallo"merging" 2 different GPFS cluster into one .. is not possible ..for sure you can do "nested" mounts .. .but that's most likely not, what you want to do .. if you want to add a GL2 (or any other ESS) ..to an existing (other) cluster... - you can't preserve ESS's RG definitions...you need to create the RGs after adding the IO-nodes to the existing cluster... so if you got a new ESS.. (no data on it) .. simply unconfigure cluster .. .. add the nodes to your existing cluster.. and then start configuring the RGsFrom: "Dorigo Alvise (PSI)" To: "gpfsug-discuss@spectrumscale.org" Date: 12/03/2019 09:35 AMSubject: [EXTERNAL] [gpfsug-discuss] How to join GNR nodes to a non-GNR clusterSent by: gpfsug-discuss-boun...@spectrumscale.org Hello everyone,I have:- A NetApp system with hardware RAID- SpectrumScale 4.2.3-13 running on top of the NetApp- A GL2 system with ESS 5.3.2.1 (Spectrum Scale 5.0.2-1)What I need to do is to merge the GL2 in the other GPFS cluster (running on the NetApp) without loosing, of course, the RecoveryGroup configuration, etc.I'd like to ask the experts1. whether it is feasible, considering the difference in the GPFS versions, architectures differences (x86_64 vs. power)2. if yes, whether anyone already did something like this and what is the best strategy suggested3. finally: is there any documentation dedicated to that, or at least inspiring the correct procedure ?Thank you very much, Alvise Dorigo___gpfsug-discuss mailing listgpfsug-discuss at spectrumscale.orghttp://gpfsug.org/mailman/listinfo/gpfsug-discuss___gpfsug-discuss mailing listgpfsug-discuss at spectrumscale.orghttp://gpfsug.org/mailman/listinfo/gpfsug-discuss ___gpfsug-discuss mailing listgpfsug-discuss at spectrumscale.orghttp://gpfsug.org/mailman/listinfo/gpfsug-discuss ___ gpfsug-discuss mailing list gpfsug-discuss at
Re: [gpfsug-discuss] How to join GNR nodes to a non-GNR cluster
It's also the case that the recommended ESS config is to be its own cluster, remote mounted by any compute nodes which access the storage. Lyle - Original message -From: "Marc A Kaplan" Sent by: gpfsug-discuss-boun...@spectrumscale.orgTo: gpfsug main discussion list Cc:Subject: [EXTERNAL] Re: [gpfsug-discuss] How to join GNR nodes to a non-GNR clusterDate: Tue, Dec 3, 2019 2:15 PM IF you have everything properly licensed and then you reconfigure... It may work okay... But then you may come up short if you ask for IBM support or service...So depending how much support you need or desire... Or take the easier and supported path... And probably accomplish most of what you need -- let each cluster be and remote mount onto clients which could be on any connected cluster.Jonathan Buzzard ---12/03/2019 10:04:46 AM---On 03/12/2019 14:54, Olaf Weiser wrote: > HalloFrom: Jonathan Buzzard To: "gpfsug-discuss@spectrumscale.org" Date: 12/03/2019 10:04 AMSubject: [EXTERNAL] Re: [gpfsug-discuss] How to join GNR nodes to a non-GNR clusterSent by: gpfsug-discuss-boun...@spectrumscale.org On 03/12/2019 14:54, Olaf Weiser wrote:> Hallo> "merging" 2 different GPFS cluster into one .. is not possible ..> for sure you can do "nested" mounts .. .but that's most likely not, what> you want to do ..>> if you want to add a GL2 (or any other ESS) ..to an existing (other)> cluster... - you can't preserve ESS's RG definitions...> you need to create the RGs after adding the IO-nodes to the existing> cluster...>> so if you got a new ESS.. (no data on it) .. simply unconfigure cluster> .. .. add the nodes to your existing cluster.. and then start> configuring the RGs>I was under the impression (from post by IBM employees on this list)that you are not allowed to mix GNR, ESS, DSS, classical GPFS, DDN GPFSetc. in the same cluster. Not a technical limitation but a licensing one.JAB.--Jonathan A. Buzzard Tel: +44141-5483420HPC System Administrator, ARCHIE-WeSt.University of Strathclyde, John Anderson Building, Glasgow. G4 0NG___gpfsug-discuss mailing listgpfsug-discuss at spectrumscale.orghttp://gpfsug.org/mailman/listinfo/gpfsug-discuss ___gpfsug-discuss mailing listgpfsug-discuss at spectrumscale.orghttp://gpfsug.org/mailman/listinfo/gpfsug-discuss ___ gpfsug-discuss mailing list gpfsug-discuss at spectrumscale.org http://gpfsug.org/mailman/listinfo/gpfsug-discuss
Re: [gpfsug-discuss] How to join GNR nodes to a non-GNR cluster
IF you have everything properly licensed and then you reconfigure... It may work okay... But then you may come up short if you ask for IBM support or service... So depending how much support you need or desire... Or take the easier and supported path... And probably accomplish most of what you need -- let each cluster be and remote mount onto clients which could be on any connected cluster. From: Jonathan Buzzard To: "gpfsug-discuss@spectrumscale.org" Date: 12/03/2019 10:04 AM Subject:[EXTERNAL] Re: [gpfsug-discuss] How to join GNR nodes to a non-GNR cluster Sent by:gpfsug-discuss-boun...@spectrumscale.org On 03/12/2019 14:54, Olaf Weiser wrote: > Hallo > "merging" 2 different GPFS cluster into one .. is not possible .. > for sure you can do "nested" mounts .. .but that's most likely not, what > you want to do .. > > if you want to add a GL2 (or any other ESS) ..to an existing (other) > cluster... - you can't preserve ESS's RG definitions... > you need to create the RGs after adding the IO-nodes to the existing > cluster... > > so if you got a new ESS.. (no data on it) .. simply unconfigure cluster > .. .. add the nodes to your existing cluster.. and then start > configuring the RGs > I was under the impression (from post by IBM employees on this list) that you are not allowed to mix GNR, ESS, DSS, classical GPFS, DDN GPFS etc. in the same cluster. Not a technical limitation but a licensing one. JAB. -- Jonathan A. Buzzard Tel: +44141-5483420 HPC System Administrator, ARCHIE-WeSt. University of Strathclyde, John Anderson Building, Glasgow. G4 0NG ___ gpfsug-discuss mailing list gpfsug-discuss at spectrumscale.org https://urldefense.proofpoint.com/v2/url?u=http-3A__gpfsug.org_mailman_listinfo_gpfsug-2Ddiscuss=DwIF-g=jf_iaSHvJObTbx-siA1ZOg=cvpnBBH0j41aQy0RPiG2xRL_M8mTc1izuQD3_PmtjZ8=lEWw7H2AdQxSCu_vbgGHhztL0y7voTATCG_KfbRgHJw=wg5NvwO5OAw-jLCsL-BtSRGisghnRu5F39r_G_gKNKk= ___ gpfsug-discuss mailing list gpfsug-discuss at spectrumscale.org http://gpfsug.org/mailman/listinfo/gpfsug-discuss
Re: [gpfsug-discuss] How to join GNR nodes to a non-GNR cluster
On 03/12/2019 14:54, Olaf Weiser wrote: > Hallo > "merging" 2 different GPFS cluster into one .. is not possible .. > for sure you can do "nested" mounts .. .but that's most likely not, what > you want to do .. > > if you want to add a GL2 (or any other ESS) ..to an existing (other) > cluster... - you can't preserve ESS's RG definitions... > you need to create the RGs after adding the IO-nodes to the existing > cluster... > > so if you got a new ESS.. (no data on it) .. simply unconfigure cluster > .. .. add the nodes to your existing cluster.. and then start > configuring the RGs > I was under the impression (from post by IBM employees on this list) that you are not allowed to mix GNR, ESS, DSS, classical GPFS, DDN GPFS etc. in the same cluster. Not a technical limitation but a licensing one. JAB. -- Jonathan A. Buzzard Tel: +44141-5483420 HPC System Administrator, ARCHIE-WeSt. University of Strathclyde, John Anderson Building, Glasgow. G4 0NG ___ gpfsug-discuss mailing list gpfsug-discuss at spectrumscale.org http://gpfsug.org/mailman/listinfo/gpfsug-discuss
Re: [gpfsug-discuss] How to join GNR nodes to a non-GNR cluster
Hi, Actually, I believe that GNR is not a limiting factor here. mmexportfs and mmimportfs ( man mm??portfs) will export/import GNR configuration as well: "If the specified file system device is a IBM Spectrum Scale RAID-based file system, then all affected IBM Spectrum Scale RAID objects will be exported as well. This includes recovery groups, declustered arrays, vdisks, and any other file systems that are based on these objects. For more information about IBM Spectrum Scale RAID, see IBM Spectrum Scale RAID: Administration. " OTOH, I suspect that due to the version mismatch, it wouldn't work - since I would assume that the cluster config version is to high for the NetApp based cluster. I would also suspect that the filesystem version on the ESS will be different. Regards, Tomer Perry Scalable I/O Development (Spectrum Scale) email: t...@il.ibm.com 1 Azrieli Center, Tel Aviv 67021, Israel Global Tel:+1 720 3422758 Israel Tel: +972 3 9188625 Mobile: +972 52 2554625 From: "Olaf Weiser" To: gpfsug main discussion list Date: 03/12/2019 16:54 Subject:[EXTERNAL] Re: [gpfsug-discuss] How to join GNR nodes to a non-GNR cluster Sent by:gpfsug-discuss-boun...@spectrumscale.org Hallo "merging" 2 different GPFS cluster into one .. is not possible .. for sure you can do "nested" mounts .. .but that's most likely not, what you want to do .. if you want to add a GL2 (or any other ESS) ..to an existing (other) cluster... - you can't preserve ESS's RG definitions... you need to create the RGs after adding the IO-nodes to the existing cluster... so if you got a new ESS.. (no data on it) .. simply unconfigure cluster .. .. add the nodes to your existing cluster.. and then start configuring the RGs From:"Dorigo Alvise (PSI)" To:"gpfsug-discuss@spectrumscale.org" Date:12/03/2019 09:35 AM Subject:[EXTERNAL] [gpfsug-discuss] How to join GNR nodes to a non-GNR cluster Sent by:gpfsug-discuss-boun...@spectrumscale.org Hello everyone, I have: - A NetApp system with hardware RAID - SpectrumScale 4.2.3-13 running on top of the NetApp - A GL2 system with ESS 5.3.2.1 (Spectrum Scale 5.0.2-1) What I need to do is to merge the GL2 in the other GPFS cluster (running on the NetApp) without loosing, of course, the RecoveryGroup configuration, etc. I'd like to ask the experts 1.whether it is feasible, considering the difference in the GPFS versions, architectures differences (x86_64 vs. power) 2.if yes, whether anyone already did something like this and what is the best strategy suggested 3.finally: is there any documentation dedicated to that, or at least inspiring the correct procedure ? Thank you very much, Alvise Dorigo___ gpfsug-discuss mailing list gpfsug-discuss at spectrumscale.org http://gpfsug.org/mailman/listinfo/gpfsug-discuss ___ gpfsug-discuss mailing list gpfsug-discuss at spectrumscale.org https://urldefense.proofpoint.com/v2/url?u=http-3A__gpfsug.org_mailman_listinfo_gpfsug-2Ddiscuss=DwICAg=jf_iaSHvJObTbx-siA1ZOg=mLPyKeOa1gNDrORvEXBgMw=5Ji4Rrk0dQhYpwfSkj-6RPXwgYhhiqqImlaHmuHrOsk=Z0aCyK22UfYZ2VIREnwtIirpmS2fM6a7IrkEUnuWyB8= ___ gpfsug-discuss mailing list gpfsug-discuss at spectrumscale.org http://gpfsug.org/mailman/listinfo/gpfsug-discuss
Re: [gpfsug-discuss] How to join GNR nodes to a non-GNR cluster
Hallo "merging" 2 different GPFS cluster into one .. is not possible .. for sure you can do "nested" mounts .. .but that's most likely not, what you want to do .. if you want to add a GL2 (or any other ESS) ..to an existing (other) cluster... - you can't preserve ESS's RG definitions... you need to create the RGs after adding the IO-nodes to the existing cluster... so if you got a new ESS.. (no data on it) .. simply unconfigure cluster .. .. add the nodes to your existing cluster.. and then start configuring the RGsFrom: "Dorigo Alvise (PSI)" To: "gpfsug-discuss@spectrumscale.org" Date: 12/03/2019 09:35 AMSubject: [EXTERNAL] [gpfsug-discuss] How to join GNR nodes to a non-GNR clusterSent by: gpfsug-discuss-boun...@spectrumscale.orgHello everyone,I have: - A NetApp system with hardware RAID - SpectrumScale 4.2.3-13 running on top of the NetApp - A GL2 system with ESS 5.3.2.1 (Spectrum Scale 5.0.2-1)What I need to do is to merge the GL2 in the other GPFS cluster (running on the NetApp) without loosing, of course, the RecoveryGroup configuration, etc.I'd like to ask the experts1. whether it is feasible, considering the difference in the GPFS versions, architectures differences (x86_64 vs. power)2. if yes, whether anyone already did something like this and what is the best strategy suggested3. finally: is there any documentation dedicated to that, or at least inspiring the correct procedure ?Thank you very much, Alvise Dorigo___gpfsug-discuss mailing listgpfsug-discuss at spectrumscale.orghttp://gpfsug.org/mailman/listinfo/gpfsug-discuss ___ gpfsug-discuss mailing list gpfsug-discuss at spectrumscale.org http://gpfsug.org/mailman/listinfo/gpfsug-discuss
Re: [gpfsug-discuss] How to join GNR nodes to a non-GNR cluster
Hi Dorigo, What do you mean have Spectrum Scale on top of NetApp? Do you create LUNs on NetApp and assign through FCP/SCSI to Linux servers? Abraços / Regards / Saludos, Anderson NobrePower and Storage ConsultantIBM Systems Hardware Client Technical Team – IBM Systems Lab Services Phone: 55-19-2132-4317E-mail: ano...@br.ibm.com - Original message -From: "Dorigo Alvise (PSI)" Sent by: gpfsug-discuss-boun...@spectrumscale.orgTo: "gpfsug-discuss@spectrumscale.org" Cc:Subject: [EXTERNAL] [gpfsug-discuss] How to join GNR nodes to a non-GNR clusterDate: Tue, Dec 3, 2019 11:36 Hello everyone, I have: - A NetApp system with hardware RAID - SpectrumScale 4.2.3-13 running on top of the NetApp - A GL2 system with ESS 5.3.2.1 (Spectrum Scale 5.0.2-1) What I need to do is to merge the GL2 in the other GPFS cluster (running on the NetApp) without loosing, of course, the RecoveryGroup configuration, etc. I'd like to ask the experts whether it is feasible, considering the difference in the GPFS versions, architectures differences (x86_64 vs. power)if yes, whether anyone already did something like this and what is the best strategy suggestedfinally: is there any documentation dedicated to that, or at least inspiring the correct procedure ? Thank you very much, Alvise Dorigo ___gpfsug-discuss mailing listgpfsug-discuss at spectrumscale.orghttp://gpfsug.org/mailman/listinfo/gpfsug-discuss ___ gpfsug-discuss mailing list gpfsug-discuss at spectrumscale.org http://gpfsug.org/mailman/listinfo/gpfsug-discuss
[gpfsug-discuss] How to join GNR nodes to a non-GNR cluster
Hello everyone, I have: - A NetApp system with hardware RAID - SpectrumScale 4.2.3-13 running on top of the NetApp - A GL2 system with ESS 5.3.2.1 (Spectrum Scale 5.0.2-1) What I need to do is to merge the GL2 in the other GPFS cluster (running on the NetApp) without loosing, of course, the RecoveryGroup configuration, etc. I'd like to ask the experts 1. whether it is feasible, considering the difference in the GPFS versions, architectures differences (x86_64 vs. power) 2. if yes, whether anyone already did something like this and what is the best strategy suggested 3. finally: is there any documentation dedicated to that, or at least inspiring the correct procedure ? Thank you very much, Alvise Dorigo ___ gpfsug-discuss mailing list gpfsug-discuss at spectrumscale.org http://gpfsug.org/mailman/listinfo/gpfsug-discuss
