if you are looking for a commercial supported solution, our Dataflow product is purpose build for this kind of task. a presentation that covers some high level aspects of it was given by me last year at one of the spectrum scale meetings in the UK --> https://www.spectrumscaleug.org/wp-content/uploads/2019/05/SSUG19UK-Day-1-05-DDN-Optimizing-storage-stacks-for-AI.pdf. its at the end of the deck.
if you want more infos, please let me know and i can get you in contact with the right person. Sven On Wed, Feb 26, 2020 at 7:24 AM Frederick Stock <[email protected]> wrote: > > What sources are you using to help you with configuring AFM? > > Fred > __________________________________________________ > Fred Stock | IBM Pittsburgh Lab | 720-430-8821 > [email protected] > > > > ----- Original message ----- > From: Andi Christiansen <[email protected]> > To: Frederick Stock <[email protected]>, [email protected] > Cc: > Subject: [EXTERNAL] RE: [gpfsug-discuss] AFM Alternative? > Date: Wed, Feb 26, 2020 8:39 AM > > 5.0.4-2.1 (home and cache) > > On February 26, 2020 2:33 PM Frederick Stock <[email protected]> wrote: > > > Andi, what version of Spectrum Scale do you have installed? > > Fred > __________________________________________________ > Fred Stock | IBM Pittsburgh Lab | 720-430-8821 > [email protected] > > > > ----- Original message ----- > From: "Olaf Weiser" <[email protected]> > Sent by: [email protected] > To: [email protected], [email protected] > Cc: [email protected] > Subject: [EXTERNAL] Re: [gpfsug-discuss] AFM Alternative? > Date: Wed, Feb 26, 2020 8:27 AM > > you may consider WatchFolder ... (cluster wider inotify --> kafka) .. and > then you go from there > > > > ----- Original message ----- > From: Andi Christiansen <[email protected]> > Sent by: [email protected] > To: "[email protected]" <[email protected]> > Cc: > Subject: [EXTERNAL] [gpfsug-discuss] AFM Alternative? > Date: Wed, Feb 26, 2020 1:59 PM > > Hi all, > > Does anyone know of an alternative to AFM ? > > We have been working on tuning AFM for a few weeks now and see little to no > improvement.. And now we are searching for an alternative.. So if anyone > knows of a product that can implement with Spectrum Scale i am open to any > suggestions :) > > We have a good mix of files but primarily billions of very small files which > AFM does not handle well on long distances. > > > Best Regards > A. Christiansen > _______________________________________________ > gpfsug-discuss mailing list > gpfsug-discuss at spectrumscale.org > http://gpfsug.org/mailman/listinfo/gpfsug-discuss > > > > _______________________________________________ > gpfsug-discuss mailing list > gpfsug-discuss at spectrumscale.org > http://gpfsug.org/mailman/listinfo/gpfsug-discuss > > > > > > _______________________________________________ > gpfsug-discuss mailing list > gpfsug-discuss at spectrumscale.org > http://gpfsug.org/mailman/listinfo/gpfsug-discuss _______________________________________________ gpfsug-discuss mailing list gpfsug-discuss at spectrumscale.org http://gpfsug.org/mailman/listinfo/gpfsug-discuss
