All -
No doubt this topic has come up before on the BB: I'd like to ask
about the current capabilities of the various integration programs (in
practice we use only MOSFLM XDS) for reading compressed diffraction
images from synchrotrons. AFAICS XDS has limited support for reading
compressed
compressed diffraction images?
All -
No doubt this topic has come up before on the BB: I'd like to ask about the
current capabilities of the various integration programs (in practice we use
only MOSFLM XDS) for reading compressed diffraction images from
synchrotrons. A...
cluster: the disk I/O
Entering xds gzip at www.ixquick.com came up with
http://www.mpimf-heidelberg.mpg.de/~kabsch/xds/html_doc/xds_parameters.html:
To save space it is allowed to compress the images by using the UNIX compress,
gzip, or bzip2 routines. On data processing XDS will automatically recognize and
expand the
images with the following extensions: .gz
.bz2 .Z .pck and .cbf
-Original Message-
From: CCP4 bulletin board [mailto:ccp...@jiscmail.ac.uk] On Behalf Of Ian
Tickle
Sent: Thursday, May 06, 2010 6:25 AM
To: CCP4BB@JISCMAIL.AC.UK
Subject: [ccp4bb] Processing compressed diffraction
Hi Ian
I've looked briefly at implementing gunzip in Mosflm in the past, but never
really pursued it. It could probably be done when I have some free time, but
who knows when that will be? gzip'ing one of my standard test sets gives around
a 40-50% reduction in size, bzip2 ~60-70%. The speed
Hi Tim thanks for that, sorry yes I missed that page. But I'm still
not clear: is it uncompressing to disk or is it doing it in memory? I
assume the latter: if the former then obviously nothing is gained.
You're right about the compression factor, it's more like a factor of
2 or 3, I should have
Hi Harry
Thanks for the info. Speed of compression is not an issue I think
since compression backing up of the images are done asynchronously
with data collection, and currently backing up easily keeps up, so I
think compression straight to the backup disk would too. As you saw
from my reply
Compression methods such as gzip are unlikely to be optimum for diffraction
images, and AFAIK the methods in CBF are better (I think Jim Pflugrath did some
races a long time ago, and I guess others have too). There is no reason for
data acquisition software ever to write uncompressed images
Message-
From: CCP4 bulletin board [mailto:ccp...@jiscmail.ac.uk] On Behalf Of Ian Tickle
Sent: Thursday, May 06, 2010 08:28 AM
To: CCP4BB@JISCMAIL.AC.UK
Subject: Re: [ccp4bb] Processing compressed diffraction images?
Hi Tim thanks for that, sorry yes I missed that page. But I'm still
yet another compressor to consider : xz
http://tukaani.org/xz/
hth
Something I have been playing with recently that might address your
problem in a way you like is SquashFS:
http://squashfs.sourceforge.net/
SquashFS is a read-only compressed file system. It uses gzip --best,
which is comparable to bzip2 for diffraction images (in my experience).
Basically,
11 matches
Mail list logo
