Re: [lustre-discuss] lustre 2.10.1 MOFED 4.1 QDR/FDR mixing

[Jeff Johnson]

Harald,

Yes. You want to ensure your MOFED IB settings and options, as well as lnet
options on the new systems are compatible with the settings on the existing
functional LFS. You want common lnet and OFED config files across your
servers and clients. This can change a bit if you get into lnet routing or
more exotic configurations. For a single, non-routed flat IB fabric
maintain common config files on all systems. Some do this by hand and
others use provisioning environments like Puppet to achieve this.

--Jeff


On Tue, Nov 14, 2017 at 9:57 AM, Harald van Pee 
wrote:

> Hi Jeff,
>
> thanks for your answer.
> Can I be sure that there is no autoprobing which sets any configuration
> differently?
> The options given for mkfs.lustre and in /etc/modprobe.d/lustre.conf
> will be the same, is this enough?
>
> Best
> Harald
>
> On Tuesday 14 November 2017 18:39:49 Jeff Johnson wrote:
> > Harald,
> >
> > As long as your new servers and clients all have the same settings in
> their
> > config files as your currently running configuration you should be fine.
> >
> > --Jeff
> >
> >
> > On Tue, Nov 14, 2017 at 9:24 AM, Harald van Pee 
> >
> > wrote:
> > > Dear all,
> > >
> > > I have installed lustre 2.10.1 from source with MOFED 4.1.
> > > mdt/mgs and oss run on centos 7.4
> > > clients on debian 9 (kernel 4.9)
> > >
> > > our test (cluster 1x mgs/mdt + 1x oss + 1x client) all with mellanox ib
> > > qdr nics
> > > runs without problems on a mellanox fdr switch.
> > > Now we have additional clients and servers with fdr and qdr nics.
> > >
> > > Do I need any special configuration (beside options lnet
> networks=o2ib0)
> > > if I add additional fdr clients and/or servers?
> > >
> > > Was the configuration probed? And does it make a difference if I would
> > > start
> > > with fdr servers and clients and add qdr servers and clients or the
> other
> > > way
> > > around?
> > >
> > > Thanks in advance
> > > Harald
> > >
> > >
> > > ___
> > > lustre-discuss mailing list
> > > lustre-discuss@lists.lustre.org
> > > http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org
>
> ___
> lustre-discuss mailing list
> lustre-discuss@lists.lustre.org
> http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org
>



-- 
--
Jeff Johnson
Co-Founder
Aeon Computing

jeff.john...@aeoncomputing.com
www.aeoncomputing.com
t: 858-412-3810 x1001   f: 858-412-3845
m: 619-204-9061

4170 Morena Boulevard, Suite D - San Diego, CA 92117

High-Performance Computing / Lustre Filesystems / Scale-out Storage
___
lustre-discuss mailing list
lustre-discuss@lists.lustre.org
http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org

Re: [lustre-discuss] lustre 2.10.1 MOFED 4.1 QDR/FDR mixing

[Harald van Pee]

Hi Jeff, 

thanks for your answer.
Can I be sure that there is no autoprobing which sets any configuration 
differently?
The options given for mkfs.lustre and in /etc/modprobe.d/lustre.conf 
will be the same, is this enough?

Best
Harald

On Tuesday 14 November 2017 18:39:49 Jeff Johnson wrote:
> Harald,
> 
> As long as your new servers and clients all have the same settings in their
> config files as your currently running configuration you should be fine.
> 
> --Jeff
> 
> 
> On Tue, Nov 14, 2017 at 9:24 AM, Harald van Pee 
> 
> wrote:
> > Dear all,
> > 
> > I have installed lustre 2.10.1 from source with MOFED 4.1.
> > mdt/mgs and oss run on centos 7.4
> > clients on debian 9 (kernel 4.9)
> > 
> > our test (cluster 1x mgs/mdt + 1x oss + 1x client) all with mellanox ib
> > qdr nics
> > runs without problems on a mellanox fdr switch.
> > Now we have additional clients and servers with fdr and qdr nics.
> > 
> > Do I need any special configuration (beside options lnet networks=o2ib0)
> > if I add additional fdr clients and/or servers?
> > 
> > Was the configuration probed? And does it make a difference if I would
> > start
> > with fdr servers and clients and add qdr servers and clients or the other
> > way
> > around?
> > 
> > Thanks in advance
> > Harald
> > 
> > 
> > ___
> > lustre-discuss mailing list
> > lustre-discuss@lists.lustre.org
> > http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org

___
lustre-discuss mailing list
lustre-discuss@lists.lustre.org
http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org

Re: [lustre-discuss] lustre 2.10.1 MOFED 4.1 QDR/FDR mixing

[Jeff Johnson]

Harald,

As long as your new servers and clients all have the same settings in their
config files as your currently running configuration you should be fine.

--Jeff


On Tue, Nov 14, 2017 at 9:24 AM, Harald van Pee 
wrote:

> Dear all,
>
> I have installed lustre 2.10.1 from source with MOFED 4.1.
> mdt/mgs and oss run on centos 7.4
> clients on debian 9 (kernel 4.9)
>
> our test (cluster 1x mgs/mdt + 1x oss + 1x client) all with mellanox ib qdr
> nics
> runs without problems on a mellanox fdr switch.
> Now we have additional clients and servers with fdr and qdr nics.
>
> Do I need any special configuration (beside options lnet networks=o2ib0)
> if I add additional fdr clients and/or servers?
>
> Was the configuration probed? And does it make a difference if I would
> start
> with fdr servers and clients and add qdr servers and clients or the other
> way
> around?
>
> Thanks in advance
> Harald
>
>
> ___
> lustre-discuss mailing list
> lustre-discuss@lists.lustre.org
> http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org
>



-- 
--
Jeff Johnson
Co-Founder
Aeon Computing

jeff.john...@aeoncomputing.com
www.aeoncomputing.com
t: 858-412-3810 x1001   f: 858-412-3845
m: 619-204-9061

4170 Morena Boulevard, Suite D - San Diego, CA 92117

High-Performance Computing / Lustre Filesystems / Scale-out Storage
___
lustre-discuss mailing list
lustre-discuss@lists.lustre.org
http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org

[lustre-discuss] lustre 2.10.1 MOFED 4.1 QDR/FDR mixing

[Harald van Pee]

Dear all,

I have installed lustre 2.10.1 from source with MOFED 4.1.
mdt/mgs and oss run on centos 7.4
clients on debian 9 (kernel 4.9)

our test (cluster 1x mgs/mdt + 1x oss + 1x client) all with mellanox ib qdr 
nics
runs without problems on a mellanox fdr switch.
Now we have additional clients and servers with fdr and qdr nics.

Do I need any special configuration (beside options lnet networks=o2ib0) 
if I add additional fdr clients and/or servers?

Was the configuration probed? And does it make a difference if I would start 
with fdr servers and clients and add qdr servers and clients or the other way 
around?

Thanks in advance
Harald


___
lustre-discuss mailing list
lustre-discuss@lists.lustre.org
http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org