let me add a few comments from some very successful large installations in Eruope
# InterOP
Even though (as Luis pointed to) , there is no support statement to run intermix DSS/ESS in general, it was ~, and is, and will be, ~ allowed for short term purposes, such as e.g migration.
The reason to not support those DSS/ESS mixed configuration in general is simply driven by the fact, that different release version of DSS/ESS potentially (not in every release, but sometimes) comes with different driver levels, (e.g. MOFED), OS, RDMA-settings, GPFS tuning, etc...
Those changes can have an impact/multiple impacts and therefore, we do not support that in general. Of course -and this would be the advice for every one - if you are faced the need to run a mixed configuration for e.g. a migration and/or e.g. cause of you need to temporary provide space etc... contact you IBM representative and settle to plan that accordingly..
There will be (likely) some additional requirements/dependencies defined like driver versions, OS, and/or Scale versions, but you'll get a chance to run mixed configuration - temporary limited to your specific scenario.
# Monitoring
No doubt, monitoring is essential and absolutely needed. - and/but - IBM wants customers to be very sensitive, what kind of additional software (=workload) gets installed on the ESS-IO servers. BTW, this rule applies as well to any other important GPFS node with special roles (e.g. any other NSD server etc)
But given the fact, that customer's usually manage and monitor their server farms from a central point of control (any 3rd party software), it is common/ best practice , that additionally monitor software(clients/endpoints) has to run on GPFS nodes, so as on ESS nodes too.
If that way of acceptance applies for DSS too, you may want to double check with Lenovo ?!
#additionally GW functions
It would be a hot iron, to general allow routing on IO nodes. Similar to the mixed support approach, the field variety for such a statement would be hard(==impossible) to manage. As we all agree, additional network traffic can (and in fact will) impact GPFS.
In your special case, the expected data rates seems to me more than ok and acceptable to go with your suggested config (as long workloads remain on that level / monitor it accordingly as you are already obviously doing)
Again,to be on the safe side.. contact your IBM representative and I'm sure you 'll find a way..
kind regards....
olaf
----- Original message -----
From: Jonathan Buzzard <[email protected]>
Sent by: [email protected]
To: [email protected]
Cc:
Subject: [EXTERNAL] Re: [gpfsug-discuss] Services on DSS/ESS nodes
Date: Sun, Oct 4, 2020 12:17 PM
On 04/10/2020 10:29, Luis Bolinches wrote:
> Hi
>
> As stated on the same link you can do remote mounts from each other and
> be a supported setup.
>
> “ You can use the remote mount feature of IBM Spectrum Scale to share
> file system data across clusters.”
>
You can, but imagine I have a DSS-G cluster, with 2PB of storage on it
which is quite modest in 2020. It is now end of life and for whatever
reason I decide I want to move to ESS instead.
What any sane storage admin want to do at this stage is set the ESS, add
the ESS nodes to the existing cluster on the DSS-G then do a bit of
mmadddisk/mmdeldisk and sit back while the data is seemlessly moved from
the DSS-G to the ESS. Admittedly this might take a while :-)
Then once all the data is moved a bit of mmdelnode and bingo the storage
has been migrated from DSS-G to ESS with zero downtime.
As that is not allowed for what I presume are commercial reasons (you
could do it in reverse and presumable that is what IBM don't want) then
once you are down the rabbit hole of one type of storage the you are not
going to switch to a different one.
You need to look at it from the perspective of the users. They frankly
could not give a monkeys what storage solution you are using. All they
care about is having usable storage and large amounts of downtime to
switch from one storage type to another is not really acceptable.
JAB.
--
Jonathan A. Buzzard Tel: +44141-5483420
HPC System Administrator, ARCHIE-WeSt.
University of Strathclyde, John Anderson Building, Glasgow. G4 0NG
_______________________________________________
gpfsug-discuss mailing list
gpfsug-discuss at spectrumscale.org
http://gpfsug.org/mailman/listinfo/gpfsug-discuss
_______________________________________________ gpfsug-discuss mailing list gpfsug-discuss at spectrumscale.org http://gpfsug.org/mailman/listinfo/gpfsug-discuss
