Re: [ccp4bb] lossy compression of diffraction images

2010-05-09 Thread Frank von Delft
Just looked at the algorithm, how it stores the average non-spot through all the images. What happens with dataset where the non-spot (e.g. background) changes systematically through the dataset, i.e. anisotropic datasets or thin crystals lying flat in a thin loop? How much worse is

Re: [ccp4bb] lossy compression of diffraction images

2010-05-09 Thread George M. Sheldrick
This is a purely historical comment. In the 1970's we were faced with the problem that if, as was then the practice, the reflection data were stored on punched cards with one reflection per card, even small molecule structures could be rather heavy to carry around. One of the innovations

Re: [ccp4bb] lossy compression of diffraction images

2010-05-09 Thread James Holton
Frank von Delft wrote: Just looked at the algorithm, how it stores the average non-spot through all the images. What happens with dataset where the non-spot (e.g. background) changes systematically through the dataset, i.e. anisotropic datasets or thin crystals lying flat in a thin loop?

Re: [ccp4bb] lossy compression of diffraction images

2010-05-09 Thread Herbert J. Bernstein
Dear Colleagues, The main problem with a lossy compression that suppresses weak spots is that those spots may be a tip-off to a misidentified symmetry, so you may wish to keep some faithful copy of the original diffraction image until you are very certain of having the symmetry right. That

Re: [ccp4bb] lossy compression of diffraction images

2010-05-07 Thread H. Raaijmakers
James, caseB was lossy compressed. It is 10% smaller when compressed (gzip, bzip2), so it contains significantly less information. cheers, Hans James Holton schreef: Ian Tickle wrote: I found an old e-mail from James Holton where he suggested lossy compression for diffraction images (as

Re: [ccp4bb] lossy compression of diffraction images

2010-05-07 Thread James Holton
So far I have gotten several votes based on the lossless compression ratio of the images, but, before I reveal the answer to the CCP4BB I remind everyone that the LOSSY compression ratio of the compressed images is 34-fold! So bzip2 and gzip are now incredibly inefficient methods of storage

[ccp4bb] lossy compression of diffraction images

2010-05-06 Thread James Holton
Ian Tickle wrote: I found an old e-mail from James Holton where he suggested lossy compression for diffraction images (as long as it didn't change the F's significantly!) - I'm not sure whether anything came of that! Well, yes, something did come of this But I don't think Gerard