Re: [gpfsug-discuss] AFM Alternative? Aspera?

2020-02-27 Thread Venkateswara R Puvvada
...@in.ibm.com) From: Chris Schlipalius To: Date: 02/27/2020 05:54 AM Subject:[EXTERNAL] Re: [gpfsug-discuss] AFM Alternative? Aspera? Sent by:gpfsug-discuss-boun...@spectrumscale.org Maybe the following would assist? I do think tarring up files first is best, but you could

Re: [gpfsug-discuss] AFM Alternative? Aspera?

2020-02-26 Thread Chris Schlipalius
Maybe the following would assist? I do think tarring up files first is best, but you could always check out: http://www.redbooks.ibm.com/redpapers/pdfs/redp5527.pdf https://www.spectrumscaleug.org/wp-content/uploads/2019/05/SSSD19DE-Day-2-B02-Integration-of-Spectrum-Scale-and-Aspera-Sync.pdf

Re: [gpfsug-discuss] AFM Alternative?

2020-02-26 Thread Sven Oehme
-discuss@spectrumscale.org > Cc: > Subject: [EXTERNAL] RE: [gpfsug-discuss] AFM Alternative? > Date: Wed, Feb 26, 2020 8:39 AM > > 5.0.4-2.1 (home and cache) > > On February 26, 2020 2:33 PM Frederick Stock wrote: > > > Andi, what version of Spectrum Scale do you have install

Re: [gpfsug-discuss] AFM Alternative?

2020-02-26 Thread Frederick Stock
@spectrumscale.orgCc:Subject: [EXTERNAL] RE: [gpfsug-discuss] AFM Alternative?Date: Wed, Feb 26, 2020 8:39 AM  5.0.4-2.1 (home and cache) On February 26, 2020 2:33 PM Frederick Stock wrote:     Andi, what version of Spectrum Scale do you have installed? Fred__Fred Stock | IBM

Re: [gpfsug-discuss] AFM Alternative?

2020-02-26 Thread Andi Christiansen
rom there - Original message - From: Andi Christiansen Sent by: gpfsug-discuss-boun...@spectrumscale.org To: "gpfsug-discuss@spectrumscale.org" Cc: Subject: [EXTERNAL] [gpfsug-discuss] AFM Alternative? Date: Wed, F

Re: [gpfsug-discuss] AFM Alternative?

2020-02-26 Thread Andi Christiansen
ug-discuss@spectrumscale.org Cc: gpfsug-discuss@spectrumscale.org Subject: [EXTERNAL] Re: [gpfsug-discuss] AFM Alternative? Date: Wed, Feb 26, 2020 8:27 AM   you may consider WatchFolder  ... (cluster wider inotify --> kafka) .. and the

Re: [gpfsug-discuss] AFM Alternative?

2020-02-26 Thread Andi Christiansen
i seem to have read somewhere that packing is just wasting time on zipping as the same file will be transfered so i havent really looked to much into that? and i dont think that is even possible in our use case as that would also mean that we in theory need to store the same data twice

Re: [gpfsug-discuss] AFM Alternative?

2020-02-26 Thread Frederick Stock
cale.orgTo: a...@christiansen.xxx, gpfsug-discuss@spectrumscale.orgCc: gpfsug-discuss@spectrumscale.orgSubject: [EXTERNAL] Re: [gpfsug-discuss] AFM Alternative?Date: Wed, Feb 26, 2020 8:27 AM  you may consider WatchFolder  ... (cluster wider inotify --> kafka) .. and then you go from there     - Original mess

Re: [gpfsug-discuss] AFM Alternative?

2020-02-26 Thread Olaf Weiser
you may consider WatchFolder  ... (cluster wider inotify --> kafka) .. and then you go from there     - Original message -From: Andi Christiansen Sent by: gpfsug-discuss-boun...@spectrumscale.orgTo: "gpfsug-discuss@spectrumscale.org" Cc:Subject: [EXTERNAL] [gpfsug-discuss] AF

Re: [gpfsug-discuss] AFM Alternative?

2020-02-26 Thread Andrew Beattie
Why don’t you look at packaging your small files into larger files which will be handled more effectively. There is no simple way to replicate / move billions of small files, But surely you can build your work flow to package the files up into a zip or tar format which will simplify not only

[gpfsug-discuss] AFM Alternative?

2020-02-26 Thread Andi Christiansen
Hi all, Does anyone know of an alternative to AFM ? We have been working on tuning AFM for a few weeks now and see little to no improvement.. And now we are searching for an alternative.. So if anyone knows of a product that can implement with