This sounds like a bug to me... (I wouldn't expect mmchattr works on different node than other file access). I would check "mmdiag --iohist verbose" during these slow reads, to see if it gives a hint at what it's doing, versus what it shows during "mmchattr". Maybe one is triggering prefetch, while the other is some kind of random IO ? Also might be worth to try a mmtrace. Compare the traces for
mmtrace start trace="all 0 vnode 1 vnop 1 io 1" cat compressedLargeFile mmtrace stop vs.: mmtrace start trace="all 0 vnode 1 vnop 1 io 1" mmchattr --compress no someLargeFile mmtrace stop (but please make sure that the file wasn't already uncompressed in pagepool in this second run). -jf On Wed, Jan 20, 2021 at 12:59 PM Daniel Kidger <[email protected]> wrote: > I think you need to think about which node the file is being decompressed > on (and if that node has plenty of space in the page pool.) > iirc mmchattr works on one of the 'manager' nodes not necessarily the node > you typed the command on? > Daniel > > _________________________________________________________ > *Daniel Kidger Ph.D.* > IBM Technical Sales Specialist > Spectrum Scale, Spectrum Discover and IBM Cloud Object Storage > > +44-(0)7818 522 266 > [email protected] > > <https://www.youracclaim.com/badges/687cf790-fe65-4a92-b129-d23ae41862ac/public_url> > <https://www.youracclaim.com/badges/8dac4bc0-7b3a-4035-b127-daec8dce9200> > <https://www.youracclaim.com/badges/8153c6a7-3e02-40be-87ee-24e27ae9459c/public_url> > <https://www.youracclaim.com/badges/78197e2c-4277-4ec9-808b-ad6abe1e1b16/public_url> > > > > > ----- Original message ----- > From: Alec <[email protected]> > Sent by: [email protected] > To: [email protected] > Cc: > Subject: [EXTERNAL] [gpfsug-discuss] Spectrum Scale 5 and Reading > Compressed Data > Date: Wed, Jan 20, 2021 11:10 > > We have AIX and Spectrum Scale 5.1 and are compressing older data. > > We can compress data at about 10GB/minute and decompress data wicked fast > using mmchattr, when a user reads data from a compressed file via > application open / read calls.... it moves at about 5MB/s. Normally our > I/O pipeline allows for 2400MB/s on a single file read. > > What can we look at to speed up the read of the compressed data, are there > any tunables that might affect this? > > As it is now if the backup daemon is backing up a compressed file, it can > get stuck for hours, I will go and mmchattr to decompress the file, within > a minute the file is decompressed, and backed up, then I simply recompress > the file once backup has moved on. > > Any advice on how to improve the compressed reads under AIX would be very > helpful. > > Alec > _______________________________________________ > gpfsug-discuss mailing list > gpfsug-discuss at spectrumscale.org > > https://urldefense.proofpoint.com/v2/url?u=http-3A__gpfsug.org_mailman_listinfo_gpfsug-2Ddiscuss&d=DwICAg&c=jf_iaSHvJObTbx-siA1ZOg&r=HlQDuUjgJx4p54QzcXd0_zTwf4Cr2t3NINalNhLTA2E&m=USgQqOp8HDCg0DYjdjSVFvVOwq1rMgRYPP_hoZqgUyI&s=_hdEB3EvWW-8ZzdS1D1roh92-AicdrVMywJwQGlKTIQ&e= > > > > Unless stated otherwise above: > IBM United Kingdom Limited - Registered in England and Wales with number > 741598. > Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU > > _______________________________________________ > gpfsug-discuss mailing list > gpfsug-discuss at spectrumscale.org > http://gpfsug.org/mailman/listinfo/gpfsug-discuss >
_______________________________________________ gpfsug-discuss mailing list gpfsug-discuss at spectrumscale.org http://gpfsug.org/mailman/listinfo/gpfsug-discuss
