Also I found no documentation that advised against having the gateway role on a nsd server. There was advise to not run the gateway role on a CES node. What is the recommendation there. Would a SAN client or shared disk be preferred to keep the latency down.
Thanks Matt On 3/26/20 4:03 AM, Venkateswara R Puvvada wrote: > Most of these recommendations documented in KC, we will add missing > information on number of filesets and inodes per gateway in the next > release. > > https://www.ibm.com/support/knowledgecenter/STXKQY_5.0.4/com.ibm.spectrum.scale.v5r04.doc/bl1ins_gatewaynodefailureafm.htm > > ~Venkat (vpuvv...@in.ibm.com) > > > > From: Matt Weil <mw...@wustl.edu> > To: gpfsug-discuss@spectrumscale.org > Date: 03/25/2020 10:34 PM > Subject: [EXTERNAL] Re: [gpfsug-discuss] AFM gateway node scaling > Sent by: gpfsug-discuss-boun...@spectrumscale.org > ------------------------------------------------------------------------ > > > > thank you thank you... I would like to see that in IBM documentation > somewhere. > > On 3/25/20 11:50 AM, Venkateswara R Puvvada wrote: > Matt, > > It is recommended to have dedicated AFM gateway nodes. Memory and CPU > requirements for AFM gateway node depends on the number of filesets > handled by the node and the inode usage of those filesets. Since AFM > keeps track of changes in the memory, any network disturbance can > cause the memory utilization to go high and which eventually leads to > in-memory queue to be dropped. After the queue is dropped, AFM runs > recovery to recover the lost operations which is expensive as it > involves creating the snapshot, running policy scan, doing readdir > from home/secondary and build the list of lost operations. When the > gateway node goes down, all the filesets handled by that node > distributed to the remaining active gateway nodes. After the gateway > node comes back, filesets are transferred back to the original gateway > node. When designing the gateway node, make sure that it have enough > memory , CPU resources for handling the incoming and outgoing data > based on the bandwidth. Limit the filesets per gateway(ex. less than > 20 filesets per gateway) so that number of AFM recoveries triggered > will be minimal when the queues are lost. Also limit the total number > of inodes handled by the gateway node across all the filesets (ex. > less than 400 million inodes per gateway). AFM gateway nodes are > licensed as server nodes. > > > ~Venkat (_vpuvv...@in.ibm.com_ <mailto:vpuvv...@in.ibm.com>) > > > > From: Matt Weil _<mw...@wustl.edu>_ <mailto:mw...@wustl.edu> > To: _gpfsug-discuss@spectrumscale.org_ > <mailto:gpfsug-discuss@spectrumscale.org> > Date: 03/23/2020 11:39 PM > Subject: [EXTERNAL] [gpfsug-discuss] AFM gateway node scaling > Sent by: _gpfsug-discuss-bounces@spectrumscale.org_ > <mailto:gpfsug-discuss-boun...@spectrumscale.org> > > ------------------------------------------------------------------------ > > > > Hello all, > > Is there any guide and or recommendation as to how to scale this. > > filesets per gateway node? Is it necessary to separate NSD server and > gateway roles. Are dedicated gateway nodes licensed as clients? > > Thanks for any guidance. > > Matt > > _______________________________________________ > gpfsug-discuss mailing list > gpfsug-discuss at spectrumscale.org_ > __http://gpfsug.org/mailman/listinfo/gpfsug-discuss_ > > > > > > _______________________________________________ > gpfsug-discuss mailing list > gpfsug-discuss at spectrumscale.org > _http://gpfsug.org/mailman/listinfo/gpfsug-discuss_ > _______________________________________________ > gpfsug-discuss mailing list > gpfsug-discuss at spectrumscale.org > http://gpfsug.org/mailman/listinfo/gpfsug-discuss > > > > > _______________________________________________ > gpfsug-discuss mailing list > gpfsug-discuss at spectrumscale.org > http://gpfsug.org/mailman/listinfo/gpfsug-discuss
_______________________________________________ gpfsug-discuss mailing list gpfsug-discuss at spectrumscale.org http://gpfsug.org/mailman/listinfo/gpfsug-discuss