Hi Alex,
not 100% sure about my answer.. but so far as I see it.. it is working, because of the so called "dito resolution " .. In the snaphost's inode .. die pointer to the DA's point the the next (more recent) inode information ..
so accessing a file in a snapshot- "redirects" the request to the origin inode - and there ..the information about compression is given and points to the origin DA
(of course.. only as long nobody changed the file since the snapshot was taken)
From: "Alexander Wolf" <a.wolf-re...@de.ibm.com>
To: gpfsug-discuss@spectrumscale.org
Date: 11/28/2019 07:03 AM
Subject: [EXTERNAL] Re: [gpfsug-discuss] Compression question
Sent by: gpfsug-discuss-boun...@spectrumscale.org
I see the same behavior of mmlsattr on my system (with some post 5.0.4 development build). Funny enough if I look at the file content in the snapshot it gets properly decompressed.
Mit freundlichen Grüßen / Kind regards
| Dr.
Alexander Wolf-Reber Spectrum Scale Release Lead Architect Department M069 / Spectrum Scale Software Development +49-160-90540880 a.wolf-re...@de.ibm.com |
IBM Data Privacy Statement
IBM Deutschland Research &
Development GmbH / Vorsitzende des Aufsichtsrats: Matthias Hartmann / Geschäftsführung:
Dirk Wittkopp
Sitz der Gesellschaft: Böblingen / Registergericht: Amtsgericht Stuttgart,
HRB 243294
----- Original message -----
From: "Luis Bolinches" <luis.bolinc...@fi.ibm.com>
Sent by: gpfsug-discuss-boun...@spectrumscale.org
To: "gpfsug main discussion list" <gpfsug-discuss@spectrumscale.org>
Cc:
Subject: [EXTERNAL] Re: [gpfsug-discuss] Compression question
Date: Thu, Nov 28, 2019 14:00
Which version are you running?
I was involved on a big for compressed file
sets and snapshots that were related to what you see.
--
Cheers
On 28. Nov 2019, at 14.57, Cregan, Bob <b.cre...@imperial.ac.uk>
wrote:
Hi
Sounds logical - except
the snap metadata does not have the compression flag set. So if the inode
now points to a set of compressed blocks how does the client know to decompress
it?
After compression of an existing file we
get in the snap
-bash-4.2$ mmlsattr -L .snapshots/@GMT-2019.11.27-19.30.14/UserGuide_13.06.pdf
file name:
.snapshots/@GMT-2019.11.27-19.30.14/UserGuide_13.06.pdf
metadata replication: 2 max 2
data replication: 1 max 3
immutable:
no
appendOnly:
no
flags:
storage pool name: sata1
fileset name:
userdirs
snapshot name: @GMT-2019.11.27-19.30.14
creation time: Tue
Mar 5 16:16:40 2019
Misc attributes: ARCHIVE
Encrypted:
no
and the original file is definitely
compressed.
-bash-4.2$ mmlsattr -L UserGuide_13.06.pdf
file name:
UserGuide_13.06.pdf
metadata replication: 2 max 2
data replication: 1 max 3
immutable:
no
appendOnly:
no
flags:
storage pool name: sata1
fileset name:
userdirs
snapshot name:
creation time: Tue
Mar 5 16:16:40 2019
Misc attributes: ARCHIVE
COMPRESSION (library z)
Encrypted:
no
Bob
Bob Cregan
HPC Systems Analyst
Information & Communication Technologies
Imperial College London,
South Kensington Campus London, SW7 2AZ
T: 07712388129
E: b.cre...@imperial.ac.uk
W: www.imperial.ac.uk/ict/rcs
<Outlook-1505984389.png>
@imperialRCS
@imperialRSE
<Outlook-1505983882.png>
From: gpfsug-discuss-boun...@spectrumscale.org <gpfsug-discuss-boun...@spectrumscale.org> on behalf of Daniel Kidger <daniel.kid...@uk.ibm.com>
Sent: 28 November 2019 12:30
To: gpfsug-discuss@spectrumscale.org <gpfsug-discuss@spectrumscale.org>
Cc: gpfsug-discuss@spectrumscale.org <gpfsug-discuss@spectrumscale.org>
Subject: Re: [gpfsug-discuss] Compression question
| Caution - This email from daniel.kid...@uk.ibm.com originated outside Imperial |
Alexander,
Can you then confirm then that the inodes
in the snapshot will now point to fewer but compressed blocks ?
Daniel
_________________________________________________________
Daniel Kidger
IBM Technical Sales Specialist
Spectrum Scale, Spectrum NAS and IBM Cloud
Object Store
+44-(0)7818 522 266
daniel.kid...@uk.ibm.com
----- Original message -----
From: "Alexander Wolf" <a.wolf-re...@de.ibm.com>
Sent by: gpfsug-discuss-boun...@spectrumscale.org
To: gpfsug-discuss@spectrumscale.org
Cc:
Subject: [EXTERNAL] Re: [gpfsug-discuss] Compression question
Date: Thu, Nov 28, 2019 12:21
I just tested this. Compressing a file did free up space in the file system. Looks like our compression code does not trigger COW on the snapshot. You can test this yourself by looking into mmlssnapshot -d (please not on a large production fs, this command is expensive).
Mit freundlichen Grüßen / Kind regards
<Image.15749424191280.png>
| <Image.15749424191281.png> | Dr.
Alexander Wolf-Reber Spectrum Scale Release Lead Architect Department M069 / Spectrum Scale Software Development +49-160-90540880 a.wolf-re...@de.ibm.com |
<Image.15749424191282.png>
IBM Data Privacy Statement
IBM Deutschland Research &
Development GmbH / Vorsitzende des Aufsichtsrats: Matthias Hartmann / Geschäftsführung:
Dirk Wittkopp
Sitz der Gesellschaft: Böblingen / Registergericht: Amtsgericht Stuttgart,
HRB 243294
----- Original message -----
From: "Luis Bolinches" <luis.bolinc...@fi.ibm.com>
Sent by: gpfsug-discuss-boun...@spectrumscale.org
To: gpfsug-discuss@spectrumscale.org
Cc: gpfsug-discuss@spectrumscale.org
Subject: [EXTERNAL] Re: [gpfsug-discuss] Compression question
Date: Thu, Nov 28, 2019 11:47
Hi
Same principle COW. The data blocks do not
get modified.
--
Ystävällisin terveisin / Kind regards / Saludos cordiales / Salutations
/ Salutacions
Luis Bolinches
Consultant IT Specialist
Mobile Phone: +358503112585
https://www.youracclaim.com/user/luis-bolinches
"If you always give you will always have" -- Anonymous
----- Original message -----
From: "Cregan, Bob" <b.cre...@imperial.ac.uk>
Sent by: gpfsug-discuss-boun...@spectrumscale.org
To: gpfsug main discussion list <gpfsug-discuss@spectrumscale.org>
Cc:
Subject: [EXTERNAL] Re: [gpfsug-discuss] Compression question
Date: Thu, Nov 28, 2019 12:06 PM
Just to clarify this is SS compression,
so
mmchattr --compression yes <filename>
or an ILM equivalent
So not a regular modification.
Bob
Bob Cregan
HPC Systems Analyst
Information & Communication Technologies
Imperial College London,
South Kensington Campus London, SW7 2AZ
T: 07712388129
E: b.cre...@imperial.ac.uk
W: www.imperial.ac.uk/ict/rcs
<Image.f7e9a6a7-c4ab-4a2c-a979-4513774aa054.png>
@imperialRCS
@imperialRSE
<Image.dbd2fd87-b9f2-4b15-903f-c74c7b7298d3.png>
From: Cregan, Bob
Sent: 28 November 2019 09:43
To: gpfsug main discussion list <gpfsug-discuss@spectrumscale.org>
Subject: Compression question
Hi All,
Can someone answer the following question on compression in a snapshot context? We have tried various experiments and they are inconclusive - too tedious to go into the detail.
What happens to the snapshot when a file is compressed in SS? The logic as I see it is
####### In a non compressed situation ###############
1) create a file,
2) create a snapshot.
3) modify a file in a normal way - the blocks on disk are changed and the old blocks are then written to the snap.
######In a compressed situation ############
1) create a file,
2) create a snapshot.
3) Compress the file. Now IBM says the blocks are rewritten when the file is compressed. So do the old uncompressed blocks go into the snap? If so we now have 2 copies of the file and unless the compression > 50% we have used more space until the snap is deleted.
You get the space back in the end, but if you are in a tight situation then potentially compression might not work for you in the short term.
Thanks
Bob
Bob Cregan
HPC Systems Analyst
Information & Communication Technologies
Imperial College London,
South Kensington Campus London, SW7 2AZ
T: 07712388129
E: b.cre...@imperial.ac.uk
W: www.imperial.ac.uk/ict/rcs
<Image.ba38b5d6-12d6-476a-bcd8-7d647bfab5b3.png>
@imperialRCS
@imperialRSE
<Image.e34493c2-d35b-4fff-a6df-ce87e2ae5ed4.png>
_______________________________________________
gpfsug-discuss mailing list
gpfsug-discuss at spectrumscale.org
http://gpfsug.org/mailman/listinfo/gpfsug-discuss
Ellei edellä ole toisin mainittu: / Unless stated otherwise above:
Oy IBM Finland Ab
PL 265, 00101 Helsinki, Finland
Business ID, Y-tunnus: 0195876-3
Registered in Finland
_______________________________________________
gpfsug-discuss mailing list
gpfsug-discuss at spectrumscale.org
http://gpfsug.org/mailman/listinfo/gpfsug-discuss
_______________________________________________
gpfsug-discuss mailing list
gpfsug-discuss at spectrumscale.org
http://gpfsug.org/mailman/listinfo/gpfsug-discuss
Unless stated otherwise above:
IBM United Kingdom Limited - Registered in England and Wales with number
741598.
Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6
3AU
Ellei edellä ole toisin mainittu: / Unless stated otherwise above:
Oy IBM Finland Ab
PL 265, 00101 Helsinki, Finland
Business ID, Y-tunnus: 0195876-3
Registered in Finland
_______________________________________________
gpfsug-discuss mailing list
gpfsug-discuss at spectrumscale.org
http://gpfsug.org/mailman/listinfo/gpfsug-discuss
_______________________________________________
gpfsug-discuss mailing list
gpfsug-discuss at spectrumscale.org
http://gpfsug.org/mailman/listinfo/gpfsug-discuss
_______________________________________________ gpfsug-discuss mailing list gpfsug-discuss at spectrumscale.org http://gpfsug.org/mailman/listinfo/gpfsug-discuss
