Re: [gpfsug-discuss] mmapplypolicy didn't migrate everything it should have - why not?

2017-04-19 Thread Yaron Daniel
Re: [gpfsug-discuss] mmapplypolicy didn't migrate everything it should have - why not? Sent by:gpfsug-discuss-boun...@spectrumscale.org Hey Marc, I?m having some issues where a simple ILM list policy never completes, but I have yet to open a PMR or enable additional logging. But I wa

Re: [gpfsug-discuss] mmapplypolicy didn't migrate everything it should have - why not?

2017-04-19 Thread Bryan Banister
: Wednesday, April 19, 2017 9:37 AM To: gpfsug main discussion list <gpfsug-discuss@spectrumscale.org> Subject: Re: [gpfsug-discuss] mmapplypolicy didn't migrate everything it should have - why not? Well I'm glad we followed Mr. S. Holmes dictum which I'll paraphrase... eliminate the impossible an

Re: [gpfsug-discuss] mmapplypolicy didn't migrate everything it should have - why not?

2017-04-19 Thread Marc A Kaplan
Well I'm glad we followed Mr. S. Holmes dictum which I'll paraphrase... eliminate the impossible and what remains, even if it seems improbable, must hold. BTW - you may want to look at mmclone. Personally, I find the doc and terminology confusing, but mmclone was designed to efficiently

Re: [gpfsug-discuss] mmapplypolicy didn't migrate everything it should have - why not?

2017-04-19 Thread Buterbaugh, Kevin L
Hi All, I think we *may* be able to wrap this saga up… ;-) Dave - in regards to your question, all I know is that the tail end of the log file is “normal” for all the successful pool migrations I’ve done in the past few years. It looks like the hard links were the problem. We have one group

Re: [gpfsug-discuss] mmapplypolicy didn't migrate everything it should have - why not?

2017-04-18 Thread Buterbaugh, Kevin L
Hi Marc, Two things: 1. I have a PMR open now. 2. You *may* have identified the problem … I’m still checking … but files with hard links may be our problem. I wrote a simple Perl script to interate over the log file I had mmapplypolicy create. Here’s the code (don’t laugh, I’m a

Re: [gpfsug-discuss] mmapplypolicy didn't migrate everything it should have - why not?

2017-04-18 Thread Marc A Kaplan
That is a summary message. It says one way or another, the command has dealt with 1.6 million files. For the case under discussion there are no EXTERNAL pools, nor any DELETions, just intra-GPFS MIGRATions. [I] A total of 1620263 files have been migrated, deleted or processed by an EXTERNAL

Re: [gpfsug-discuss] mmapplypolicy didn't migrate everything it should have - why not?

2017-04-18 Thread David D. Johnson
I have an observation, which may merely serve to show my ignorance: Is it significant that the words "EXTERNAL EXEC/script” are seen below? If migrating between storage pools within the cluster, I would expect the PIT engine to do the migration. When doing HSM (off cluster, tape libraries,

Re: [gpfsug-discuss] mmapplypolicy didn't migrate everything it should have - why not?

2017-04-18 Thread Marc A Kaplan
ANYONE else reading this saga? Who uses mmapplypolicy to migrate files within multi-TB file systems? Problems? Or all working as expected? -- Well, again mmapplypolicy "thinks" it has "chosen" 1.6 million files whose total size is 61 Terabytes and migrating those will bring the occupancy

Re: [gpfsug-discuss] mmapplypolicy didn't migrate everything it should have - why not?

2017-04-18 Thread Zachary Giles
ale (né GPFS) From:"Buterbaugh, Kevin L" <kevin.buterba...@vanderbilt.edu> To: gpfsug main discussion list <gpfsug-discuss@spectrumscale.org> Date: 04/17/2017 11:25 AM Subject:Re: [gpfsug-discuss] mmapplypolicy didn't migrate everything it shoul

Re: [gpfsug-discuss] mmapplypolicy didn't migrate everything it should have - why not?

2017-04-18 Thread Buterbaugh, Kevin L
e (né GPFS) From:"Buterbaugh, Kevin L" <kevin.buterba...@vanderbilt.edu<mailto:kevin.buterba...@vanderbilt.edu>> To:gpfsug main discussion list <gpfsug-discuss@spectrumscale.org<mailto:gpfsug-discuss@spectrumscale.org>> Date:04/17/2017 11:25

Re: [gpfsug-discuss] mmapplypolicy didn't migrate everything it should have - why not?

2017-04-17 Thread Buterbaugh, Kevin L
it doesn't "know" about that! Run it again! From:"Buterbaugh, Kevin L" <kevin.buterba...@vanderbilt.edu<mailto:kevin.buterba...@vanderbilt.edu>> To: gpfsug main discussion list <gpfsug-discuss@spectrumscale.org<mailto:gpfsug-discuss@spectrumsc

Re: [gpfsug-discuss] mmapplypolicy didn't migrate everything it should have - why not?

2017-04-17 Thread Marc A Kaplan
Oops... If you want to see the list of what would be migrated '-I test -L 2' If you want to migrate and see each file migrated '-I yes -L 2' I don't recommend -L 4 or higher, unless you want to see the files that do not match your rules. -L 3 will show you all the files that match the

Re: [gpfsug-discuss] mmapplypolicy didn't migrate everything it should have - why not?

2017-04-17 Thread Marc A Kaplan
ilt.edu> To: gpfsug main discussion list <gpfsug-discuss@spectrumscale.org> Date: 04/17/2017 11:25 AM Subject:Re: [gpfsug-discuss] mmapplypolicy didn't migrate everything it shouldhave - why not? Sent by:gpfsug-discuss-boun...@spectrumscale.org Hi Marc, I do

Re: [gpfsug-discuss] mmapplypolicy didn't migrate everything it should have - why not?

2017-04-17 Thread Alex Chekholko
vin.buterba...@vanderbilt.edu <mailto:kevin.buterba...@vanderbilt.edu>> To:gpfsug main discussion list <gpfsug-discuss@spectrumscale.org <mailto:gpfsug-discuss@spectrumscale.org>> Date: 04/16/2017 09:47 AM Subject:[gpfsug-discuss] mmapplypolicy didn't migrate

Re: [gpfsug-discuss] mmapplypolicy didn't migrate everything it should have - why not?

2017-04-17 Thread Buterbaugh, Kevin L
t that! Run it again! From:"Buterbaugh, Kevin L" <kevin.buterba...@vanderbilt.edu<mailto:kevin.buterba...@vanderbilt.edu>> To:gpfsug main discussion list <gpfsug-discuss@spectrumscale.org<mailto:gpfsug-discuss@spectrumscale.org>> Date:

Re: [gpfsug-discuss] mmapplypolicy didn't migrate everything it should have - why not?

2017-04-16 Thread Marc A Kaplan
ing -- it doesn't "know" about that! Run it again! From: "Buterbaugh, Kevin L" <kevin.buterba...@vanderbilt.edu> To: gpfsug main discussion list <gpfsug-discuss@spectrumscale.org> Date: 04/16/2017 09:47 AM Subject: [gpfsug-discuss] mmapplypolicy d

[gpfsug-discuss] mmapplypolicy didn't migrate everything it should have - why not?

2017-04-16 Thread Buterbaugh, Kevin L
Hi All, First off, I can open a PMR for this if I need to. Second, I am far from an mmapplypolicy guru. With that out of the way … I have an mmapplypolicy job that didn’t migrate anywhere close to what it could / should have. From the log file I have it create, here is the part where it