Re: [ccp4bb] ccp4 pack_images program?
Hi Bill I mean to pick this up earlier in the week but somehow it slipped through my net. Sorry. Anyhows: pack_c.c and pack_f.f are not programs but essentially subroutine libraries. You would need to write your own program to actually use them (I'm not sure that there's any documentation btw...) There are copies of pack_c.c and pack_f.f in $CLIBS which appear to have been updated more recently than the ones in $CCP4/x-windows/ipdisp/src (according to our CVS). So these would probably be a better choice to build. You can build them using the pack_c.o and pack_f.o targets in the $CLIBS Makefile, however it does appear that they are built anyway as part of the standard CCP4 library without any additional effort. Hope this helps, best wishes Peter William Scott wrote: Sorry for the repost, but I think my question got lost in the earlier thread. I've found $CCP4/x-windows/ipdisp/src/pack_c.c, pack_f.f and so forth, but they apparently don't build by default, and when I try to, I get You need to make mosflm-bits in the library for the image-packing stuff exit 1 make: *** [$CCP4/lib/libccp4.a(pack_c.o)] Error 1 I have no idea what mosflm-bits refers to and can't seem to find anything. Any suggestions how to build this? It seems it would be good to have available if we are all going to put our images on our web-servers. Thanks. Bill Scott -- ___ Peter J Briggs, [EMAIL PROTECTED] Tel: +44 1925 603826 CCP4, [EMAIL PROTECTED] Fax: +44 1925 603825 http://www.ccp4.ac.uk/ Daresbury Laboratory, Daresbury, Warrington WA4 4AD
Re: [ccp4bb] diffraction images images/jpeg2000
Hi James,
On the gathering of the data from all possible beamline / source /
detector combinations below, I am also keen to get hold of these. To
assist with this I have written a couple of bash shell scripts which
will tar, gzip and split into 128MB chunks data, then reverse this
process to ensure that the images are correctly preserved. I am sure
that people will rise to the challenge of improving them, but they work
for me...
Scripts follow:
packer.sh
This will tar up the source directory to files prefixed with the
destination prefix, and compute md5sums for the resulting files. Usage:
Usage: packer.sh ./path/to/image/dir prefix - prefix.aaa prefix.aab etc
files.
--- packer.sh ---
#!/bin/bash
# check input arguments here
if [ $# -ne 2 ] ; then
echo $0 source destination
exit
fi
export source=${1}
export dest=${2}
echo Packing ${source} to ${dest}
tar cvf - ${source} | bzip2 | split -a 3 -b 128m - ${dest}.
md5sum ${dest}* ${dest}.md5
unpacker
Performs the inverse operation above - checks the md5 sums and unpacks
the original directory structure...
Usage: unpacker prefix ./path/to/destination/dir
unpacker.sh
-
#!/bin/bash
# check input arguments
if [ $# -ne 2 ] ; then
echo $0 source destination
exit
fi
export source=${1}
export dest=${2}
# explain what we are doing
echo Unpacking ${source} to ${dest}
# check the md5sums of the chunks
md5sum -c ${source}.md5
if [ $? -ne 0 ] ; then
exit
fi
# then go ahead and unpack
cat `ls ${source}.* | grep -v md5 | sort` | bunzip2 | tar xvf - -C
${dest}
---
Tested on my OS X intel mac but I think they should work fine on Linux
and perhaps any other modernish UNIX installation, with the GNU
coreutils available.
Cheers,
Graeme
-Original Message-
From: CCP4 bulletin board [mailto:[EMAIL PROTECTED] On Behalf Of
James Holton
Sent: 23 August 2007 18:47
To: CCP4BB@JISCMAIL.AC.UK
Subject: Re: [ccp4bb] diffraction images images/jpeg2000
Well, I know it's not the definitive source of anything, but the
wikipedia entry on JPEG2000 says:
The PNG (Portable Network Graphics) format is still more
space-efficient in the case of images with many pixels of the same
color, and supports special compression features that JPEG 2000 does
not.
So would PNG be better? It does support 16 bit greyscale. Then again,
so does TIFF, and Mar already uses that. Why don't they use the LZW
compression feature of TIFF? The old Mar325 images were compressed
after all. I think only Mar can answer this, but I imagine the choice to
drop compression was because the advantages of compression (a factor or
2 or so in space) are outweighed by the disadvantages (limited speed and
limited compatibility with data processing packages).
How good could lossless compression of diffraction images possibly be?
I just ran an entropy calculation on the 44968 images on /data at the
moment at ALS 8.3.1. I am using a feature of Andy Hammersley's program
FIT2D to compute the entropy. I don't pretend to completely
understand the algorithm, but I do understand that the entropy of the
image reflects the maximum possible compression ratio. For these
images, the theoretical maximum compression ratio ranged from 1.2 to
4.8 with mean 2.7 and standard deviation 0.7. The values for Huffmann
encoding ranged from 0.95 to 4.7 with mean 2.4 and standard deviation
1.0. The correlation coefficient between the Huffmann and theoretical
compression ratio was 0.97. I had a look at a few of the outlier cases.
As one might expect, the best compression ratios are from blank images
(where all the high-order bits are zero). The #1 hardest-to-compress
image had many overloads, clear protein diffraction and a bunch of ice
rings.
So, unless I am missing something, I think the best we are going to get
with lossless compression is about 2.5:1. At least, for individual
frames. Compressing a data set as a video sequence might have
substantial gains since only a few pixels change significantly from
frame-to-frame. Are there any lossless video codecs out there? If so,
can they handle 6144x6144 video?
What about lossy compression? Yes yes, I know it sounds like a
horrible idea to use lossy compression on scientific data, because it
would change the values of that most precious of numbers: Fobs.
However, the question I have never heard a good answer to is HOW MUCH
would it change Fobs? More practically: how much compression can you do
before Fobs changes by more than the value of SIGFobs? Diffraction
patterns are inherently noisy. If you take the same image twice, then
photon counting statistics make sure that no two images are exactly the
same. So which one is right? If the changes in pixel values from a
lossy compression algorithm are always
Re: [ccp4bb] diffraction images images/jpeg2000
Hi James, The old mar345 images were compressed with the pack which Bill is referring to. This is suppoprted in CBFlib. PNG and jpeg2000 may well do better at compression (would like to see the numbers with this) but are likely to be much slower than something customised for use with diffraction images. Anything doing complex mathematical analysis is likely to be slow... On an example set packed with the scripts in another email I got a compression ratio with bzip2 of 3.49:1 for 270 frames. This exceeds the value you quote below, but was from images where some of the detector was unused, where the packing would probably work better. On the question of lossy compression, I think we'd have to ask some data reduction guru's how much the noise would affect the data reduction. I suspect that the main problem is that the noise added would be correlated across the image and would therefore affect the background statistics in a non-trivial way. Although the intensity measurements may not be badly affected the error estimates on them could be... Cheers, Graeme -Original Message- From: CCP4 bulletin board [mailto:[EMAIL PROTECTED] On Behalf Of James Holton Sent: 23 August 2007 18:47 To: CCP4BB@JISCMAIL.AC.UK Subject: Re: [ccp4bb] diffraction images images/jpeg2000 Well, I know it's not the definitive source of anything, but the wikipedia entry on JPEG2000 says: The PNG (Portable Network Graphics) format is still more space-efficient in the case of images with many pixels of the same color, and supports special compression features that JPEG 2000 does not. So would PNG be better? It does support 16 bit greyscale. Then again, so does TIFF, and Mar already uses that. Why don't they use the LZW compression feature of TIFF? The old Mar325 images were compressed after all. I think only Mar can answer this, but I imagine the choice to drop compression was because the advantages of compression (a factor or 2 or so in space) are outweighed by the disadvantages (limited speed and limited compatibility with data processing packages). How good could lossless compression of diffraction images possibly be? I just ran an entropy calculation on the 44968 images on /data at the moment at ALS 8.3.1. I am using a feature of Andy Hammersley's program FIT2D to compute the entropy. I don't pretend to completely understand the algorithm, but I do understand that the entropy of the image reflects the maximum possible compression ratio. For these images, the theoretical maximum compression ratio ranged from 1.2 to 4.8 with mean 2.7 and standard deviation 0.7. The values for Huffmann encoding ranged from 0.95 to 4.7 with mean 2.4 and standard deviation 1.0. The correlation coefficient between the Huffmann and theoretical compression ratio was 0.97. I had a look at a few of the outlier cases. As one might expect, the best compression ratios are from blank images (where all the high-order bits are zero). The #1 hardest-to-compress image had many overloads, clear protein diffraction and a bunch of ice rings. So, unless I am missing something, I think the best we are going to get with lossless compression is about 2.5:1. At least, for individual frames. Compressing a data set as a video sequence might have substantial gains since only a few pixels change significantly from frame-to-frame. Are there any lossless video codecs out there? If so, can they handle 6144x6144 video? What about lossy compression? Yes yes, I know it sounds like a horrible idea to use lossy compression on scientific data, because it would change the values of that most precious of numbers: Fobs. However, the question I have never heard a good answer to is HOW MUCH would it change Fobs? More practically: how much compression can you do before Fobs changes by more than the value of SIGFobs? Diffraction patterns are inherently noisy. If you take the same image twice, then photon counting statistics make sure that no two images are exactly the same. So which one is right? If the changes in pixel values from a lossy compression algorithm are always smaller than that introduced by photon-counting noise, then is lossy compression really such a bad idea? The errors introduced could be small when compared to errors in say, scale factors or bulk solvent parameters. A great deal can be gained in compression ratio if only random noise is removed. I remember the days before MP3 when it was lamented that sampled audio files could never be compressed very well. Even today bzip2 does not work very well at all at compressing sampled audio (about 1.3:1), but mp3 files can be made at a compression ratio of 10:1 over CD-quality audio and we all seem to still enjoy the music. I suppose the best lossy compression is the one that preserves the features of the image you want and throws out the stuff you don't care about. So, in a way, data-reduction programs are probably the best lossy compression we are going to get. Unfortunately,
Re: [ccp4bb] diffraction images images/jpeg2000
Hi Lossy compression should be okay, provided that the errors introduced are smaller than those expected for counting statistics (assuming that the pixels are more-or-less independent) - i.e. less than the square-root of the individual pixel intensities (though I don't see why this can't be extended to the integrated reflection intensities). So it's more important to accurately retain your weak pixel values than your strong ones - an error of ±10 for a pixel in a background count where the background should be 40 is significant, but an error of ±10 for a saturated pixel on most detectors (say, about 64K for a CCD) wouldn't affect anything. On the question of lossy compression, I think we'd have to ask some data reduction guru's how much the noise would affect the data reduction. I suspect that the main problem is that the noise added would be correlated across the image and would therefore affect the background statistics in a non-trivial way. Although the intensity measurements may not be badly affected the error estimates on them could be... Harry -- Dr Harry Powell, MRC Laboratory of Molecular Biology, MRC Centre, Hills Road, Cambridge, CB2 2QH
Re: [ccp4bb] diffraction images images/jpeg2000
Dear all, I think we need to stop and think right here. The errors in pixel values of images are neither Poisson (i.e. forget about taking square roots) nor independent. Our ideas about image statistics are already disastrously poor enough: the last thing we need is to make matters even worse by using compression methods based on those erroneous statistical arguments! With best wishes, Gerard. -- On Fri, Aug 24, 2007 at 01:20:29PM +0100, Harry Powell wrote: Hi Lossy compression should be okay, provided that the errors introduced are smaller than those expected for counting statistics (assuming that the pixels are more-or-less independent) - i.e. less than the square-root of the individual pixel intensities (though I don't see why this can't be extended to the integrated reflection intensities). So it's more important to accurately retain your weak pixel values than your strong ones - an error of ±10 for a pixel in a background count where the background should be 40 is significant, but an error of ±10 for a saturated pixel on most detectors (say, about 64K for a CCD) wouldn't affect anything. On the question of lossy compression, I think we'd have to ask some data reduction guru's how much the noise would affect the data reduction. I suspect that the main problem is that the noise added would be correlated across the image and would therefore affect the background statistics in a non-trivial way. Although the intensity measurements may not be badly affected the error estimates on them could be... Harry -- Dr Harry Powell, MRC Laboratory of Molecular Biology, MRC Centre, Hills Road, Cambridge, CB2 2QH -- === * * * Gerard Bricogne [EMAIL PROTECTED] * * * * Global Phasing Ltd. * * Sheraton House, Castle Park Tel: +44-(0)1223-353033 * * Cambridge CB3 0AX, UK Fax: +44-(0)1223-366889 * * * ===
Re: [ccp4bb] centrosymm structure
oopss...Not science: Proteins: Structure, Function, and Genetics Volume 16, Issue 3 , Pages 301 - 305 (1993) -Original Message- From: CCP4 bulletin board [mailto:[EMAIL PROTECTED] On Behalf Of Soisson, Stephen Michael Sent: Friday, August 24, 2007 10:59 AM To: CCP4BB@JISCMAIL.AC.UK Subject: Re: [ccp4bb] centrosymm structure Jeremy Berg, Rubredoxin. In Science around 1995. Steve -Original Message- From: CCP4 bulletin board [mailto:[EMAIL PROTECTED] On Behalf Of Bernhard Rupp Sent: Friday, August 24, 2007 10:57 AM To: CCP4BB@JISCMAIL.AC.UK Subject: [ccp4bb] centrosymm structure Dear All, there was a paper (quite) a while ago where someone made for the first time a racemic protein mixture, obtained a centrosymmetric structure and solved it (not the 2003 PNAS paper by the Eisenberg grp). Hints appreciated. Thx, br - Bernhard Rupp 001 (925) 209-7429 +43 (676) 571-0536 [EMAIL PROTECTED] [EMAIL PROTECTED] http://www.ruppweb.org/ - People can be divided in three classes: The few who make things happen The many who watch things happen And the overwhelming majority who have no idea what is happening. - -- Notice: This e-mail message, together with any attachments, contains information of Merck Co., Inc. (One Merck Drive, Whitehouse Station, New Jersey, USA 08889), and/or its affiliates (which may be known outside the United States as Merck Frosst, Merck Sharp Dohme or MSD and in Japan, as Banyu - direct contact information for affiliates is available at http://www.merck.com/contact/contacts.html) that may be confidential, proprietary copyrighted and/or legally privileged. It is intended solely for the use of the individual or entity named on this message. If you are not the intended recipient, and have received this message in error, please notify us immediately by reply e-mail and then delete it from your system. -- -- Notice: This e-mail message, together with any attachments, contains information of Merck Co., Inc. (One Merck Drive, Whitehouse Station, New Jersey, USA 08889), and/or its affiliates (which may be known outside the United States as Merck Frosst, Merck Sharp Dohme or MSD and in Japan, as Banyu - direct contact information for affiliates is available at http://www.merck.com/contact/contacts.html) that may be confidential, proprietary copyrighted and/or legally privileged. It is intended solely for the use of the individual or entity named on this message. If you are not the intended recipient, and have received this message in error, please notify us immediately by reply e-mail and then delete it from your system. -- -- Notice: This e-mail message, together with any attachments, contains information of Merck Co., Inc. (One Merck Drive, Whitehouse Station, New Jersey, USA 08889), and/or its affiliates (which may be known outside the United States as Merck Frosst, Merck Sharp Dohme or MSD and in Japan, as Banyu - direct contact information for affiliates is available at http://www.merck.com/contact/contacts.html) that may be confidential, proprietary copyrighted and/or legally privileged. It is intended solely for the use of the individual or entity named on this message. If you are not the intended recipient, and have received this message in error, please notify us immediately by reply e-mail and then delete it from your system. --
Re: [ccp4bb] diffraction images images/jpeg2000
Wow. I don't know about the rest of you, but I got told three times. Gerard is, of course, right about pixel non-independence (think point spread function, among other things), and I wouldn't care to argue statistics with him, but as far as I know (and I could well be wrong) most of the integration programs out there _do_ use counting statistics (i.e. Poisson statistics) at least as a first approximation for the random error in measurement; this may be modified by some detector inefficiency factor (See Borek, Minor Otwinowski, Acta Cryst (2003) D59 2031 - 2038), but it's still there and being used by everyone, nonetheless. Having said that, regarding the storage of images, my personal feeling is that there's no real point in using a lossy compression when there are good lossless systems out there. I also think that almost no-one would ever bother to reprocess deposited images anyway; my guess is that unusual structures would be detected by other means, and that examining the original images would rarely shed light on the problem. I think we need to stop and think right here. The errors in pixel values of images are neither Poisson (i.e. forget about taking square roots) nor independent. Our ideas about image statistics are already disastrously poor enough: the last thing we need is to make matters even worse by using compression methods based on those erroneous statistical arguments! With best wishes, Gerard. -- On Fri, Aug 24, 2007 at 01:20:29PM +0100, Harry Powell wrote: Hi Lossy compression should be okay, provided that the errors introduced are smaller than those expected for counting statistics (assuming that the pixels are more-or-less independent) - i.e. less than the square-root of the individual pixel intensities (though I don't see why this can't be extended to the integrated reflection intensities). So it's more important to accurately retain your weak pixel values than your strong ones - an error of ±10 for a pixel in a background count where the background should be 40 is significant, but an error of ±10 for a saturated pixel on most detectors (say, about 64K for a CCD) wouldn't affect anything. On the question of lossy compression, I think we'd have to ask some data reduction guru's how much the noise would affect the data reduction. I suspect that the main problem is that the noise added would be correlated across the image and would therefore affect the background statistics in a non-trivial way. Although the intensity measurements may not be badly affected the error estimates on them could be... Harry -- Dr Harry Powell, MRC Laboratory of Molecular Biology, MRC Centre, Hills Road, Cambridge, CB2 2QH -- === * * * Gerard Bricogne [EMAIL PROTECTED] * * * * Global Phasing Ltd. * * Sheraton House, Castle Park Tel: +44-(0)1223-353033 * * Cambridge CB3 0AX, UK Fax: +44-(0)1223-366889 * * * === Harry -- Dr Harry Powell, MRC Laboratory of Molecular Biology, MRC Centre, Hills Road, Cambridge, CB2 2QH
Re: [ccp4bb] centrosymm structure
Jeremy Berg, Rubredoxin. In Science around 1995. Steve -Original Message- From: CCP4 bulletin board [mailto:[EMAIL PROTECTED] On Behalf Of Bernhard Rupp Sent: Friday, August 24, 2007 10:57 AM To: CCP4BB@JISCMAIL.AC.UK Subject: [ccp4bb] centrosymm structure Dear All, there was a paper (quite) a while ago where someone made for the first time a racemic protein mixture, obtained a centrosymmetric structure and solved it (not the 2003 PNAS paper by the Eisenberg grp). Hints appreciated. Thx, br - Bernhard Rupp 001 (925) 209-7429 +43 (676) 571-0536 [EMAIL PROTECTED] [EMAIL PROTECTED] http://www.ruppweb.org/ - People can be divided in three classes: The few who make things happen The many who watch things happen And the overwhelming majority who have no idea what is happening. - -- Notice: This e-mail message, together with any attachments, contains information of Merck Co., Inc. (One Merck Drive, Whitehouse Station, New Jersey, USA 08889), and/or its affiliates (which may be known outside the United States as Merck Frosst, Merck Sharp Dohme or MSD and in Japan, as Banyu - direct contact information for affiliates is available at http://www.merck.com/contact/contacts.html) that may be confidential, proprietary copyrighted and/or legally privileged. It is intended solely for the use of the individual or entity named on this message. If you are not the intended recipient, and have received this message in error, please notify us immediately by reply e-mail and then delete it from your system. --
[ccp4bb] Mosflm v 7.0.1 Mac Intel pre-built executable.
Hi folks If you've recently downloaded a pre-built copy of Mosflm version 7.0.1 for Intel Mac (including the universal binary) and been surprised by the need for libgfortran.1.dylib, read on. Otherwise, feel free to ignore this! It seems my attempts to produce a statically linked OS X executable with gfortran weren't as successful as I thought! Although the executable was (in a logical sense) statically linked, the linker put information in the header suggesting that it required the dynamic library. This is wrong and may be viewed as a linker bug. However, I've now sorted out the problem and produced executables which do link the libgfortran statically, and these replace the earlier copies. If you've managed to run Mosflm version 7.0.1 on an Intel Mac there is no need at all to download this executable, so you can ignore this. However, if you've been stymied in your attempt to run it by being told that /sw/lib//gcc4/lib/libgfortran.1.dylib doesn't exist, it is probably worthwhile downloading this new executable. have a nice weekend! Harry -- Dr Harry Powell, MRC Laboratory of Molecular Biology, MRC Centre, Hills Road, Cambridge, CB2 2QH
[ccp4bb] PX Jobs Team Leader, Postdoc, Tech (SGC-Oxford).txt
My group has several vacancies to fill immediately: a Team Leader for Infrastructure and Methods Development; two postdoc positions; and a technician. This is the Protein Crystallography group of the Structural Genomics Consortium, Oxford. For details, please see respectively: Team Leader: http://www.sgc.ox.ac.uk/jobs/07057.html Postdocs:http://www.sgc.ox.ac.uk/jobs/07058.html Technician: http://www.sgc.ox.ac.uk/jobs/07067.html Feel free to contact me for details. The remit of the Structural Genomics Consortium (SGC) is to solve human proteins of medical relevance and place them in the public domain without restrictions; it is funded by a consortium of public and industrial funders, and has independently operating sites in Oxford and Toronto Universities and Karolinska Institutet (Stockholm). The Oxford site solved nearly 200 such structures in its first three years, and has now entered Phase II, with 4 years of funding till June 2011; it now has an increased emphasis on chemical biology and membrane proteins. The Protein Crystallography group collaborates tightly with the 5 other groups to get their purified proteins crystallized and solved -- five per month, to be precise. As importantly, with all the robotics and toys you'd care for, and access not only to stacks of real, interesting samples but also to heaps of structured data accumulated during Phase I, we're ideally positioned for development and testing of crystallographic methods, which will be the major scientific thrust of the group in this phase. Enthusiastic crystallographers, especially those that really enjoy the nuts and bolts of the technique and mucking around with tricks and toys, both crystallization and algorithms, are encouraged to apply. phx.
Re: [ccp4bb] Questions about diffraction
On Friday 24 August 2007 12:22, Michel Fodje wrote: 1. In every description of Braggs' law I've seen, the in-coming waves have to be in phase. Why is that? Given that the sources used for diffraction studies are mostly non-coherent. Think of Bragg's Law as explaining what happens to a single photon that is probabilistically scattered by every atom in the lattice. It's perfectly coherent with itself. This idea should be no stranger than textbook illustrations of the result of sending a single particle through a narrow slit or pinhole. The interference effects follow the expected predictions even for illumination by one particle at a time. -- Ethan A Merritt
Re: [ccp4bb] Questions about diffraction
Michel Fodje wrote: Dear Crystallographers, Here are a few paradoxes about diffraction I would like to get some answers about: ... 3. What happens to the photon energy when waves destructively interfere as mentioned in the text books. Doesn't 'destructive interference' appear to violate the first and second laws of thermodynamics? Besides, since the sources are non-coherent, how come the photon 'waves' don't annihilate each other before reaching the sample? If they were coherent, would we just end up with a single wave any how? With what will it interfere to cause diffraction? For every direction where there is destructive interference and a loss of energy there is a direction where there is constructive interference that piles up energy. If you integrate over all directions energy is conserved. I'm not sure what your concern is about the second law. The radiation is spreading out into space and so entropy increases. Dale Tronrud
Re: [ccp4bb] Questions about diffraction
1. In every description of Braggs' law I've seen, the in-coming waves have to be in phase. Why is that? Given that the sources used for diffraction studies are mostly non-coherent. Think of Bragg's Law as explaining what happens to a single photon that is probabilistically scattered by every atom in the lattice. It's perfectly coherent with itself. Why does the photon not diffract at all angles then, since one 'part' of the photon will never have a different phase from another 'part'? If you are correct that it is a single photon diffracting, it would suggest that at some point the photon is in-coherent with itself thus the need to have a diffraction condition for specific angles. It would make more sense to me in such a case, if we interpreted Braggs' law as the condition under which the photon keeps it's 'internal coherence but instead we talk of constructive and destructive interference! This idea should be no stranger than textbook illustrations of the result of sending a single particle through a narrow slit or pinhole. The interference effects follow the expected predictions even for illumination by one particle at a time. The mathematics works but doesn't necessarily mean the current interpretation of the mathematics has any resemblance to what actually happens in reality.
Re: [ccp4bb] Questions about diffraction
For every direction where there is destructive interference and a loss of energy there is a direction where there is constructive interference that piles up energy. If you integrate over all directions energy is conserved. For the total integrated energy to be conserved, energy will have to be created in certain directions to compensate for the loss in other directions. So in a direction in which the condition is met, the total will have to be more than the sum of the waves in that direction. How about considering the possibility that all photons coming into the sample are diffracted -- just in different directions. So that what is happening is not constructive and destructive interference but a kind sorting of the photons based on a certain property of the photons, maybe the phase.
Re: [ccp4bb] Questions about diffraction
You are just using the coherent fraction of the beam. My point is that Braggs' law as currently understood does not preclude the diffraction from waves which were non-coherent before hitting the sample It is not clear at all how you arrive to that condition. By definition, if two waves are non coherent, you cannot define a phase difference. The phase difference is continuosly changing at random with non coherent waves. if the phase difference between two waves is y, the extra distance the second wave has to travel to again be in phase is y*lambda/2pi if this distance is the same as the 2dsin(theta), the diffraction condition will also be met. My understanding is that coherence has to do with the phase not the wavelength. Only when the wavelengths are also different will the phase difference be changing continuously. No? You do not annihilate energy with interferences, you just spread it differently. Apart for conservation of energy, Thermodynamics is seldom involved in these considerations. My point is that destructive interference implies that energy is destroyed which does not make sense. If as you say the energy is simply spread differently, what would be the mechanism/geometry of this spreading? A complete lack of coherence leads simply to adding intensities of the two waves. On the contrary. complete lack of coherence leads to destructive interference. No? Perfect coherence leads to the amplitudes adding up.
Re: [ccp4bb] Questions about diffraction
For the total integrated energy to be conserved, energy will have to be created in certain directions to compensate for the loss in other directions. So in a direction in which the condition is met, the total will have to be more than the sum of the waves in that direction. How about considering the possibility that all photons coming into the sample are diffracted -- just in different directions. So that what is happening is not constructive and destructive interference but a kind sorting of the photons based on a certain property of the photons, maybe the phase. * I think of it that each photon that happens to be perturb an electron, i.e., Thomson scattering, sends out a spherical wave, which has anisotropy to it, i.e., the wave front is more concentrated in the forward direction. These spherical waves interfere with each other, making the diffraction pattern. Something for you to chew on: how is it that the electrons of the protein, which are presumably not in phase with each other nor in exactly the same place in their orbitals from unit cell to unit cell (maybe they are?) when they scatter the photons, they result in interference? What are the chances that the scattering electrons are exactly in the same place as the electrons in another unit cell, or of the same phase? And would they not need to be in the same place to sub-angstrom precision to scatter coherently? I would suggest two possible answers, neither of which am I entirely satisfied: 1. Something about the crystalline state induces the protein molecules' molecular orbitals to be totally in synch with each other. This seems too miraculous to be true, in a way. Nevertheless, it would account for the data, I think. 2. The scattering electrons are elusive probablistic entities which are really no place at all. This, however, does not solve the problem of the phases (not in the usual sense of finding fourier phases) which is that it seems unlikely that electrons in multiple unit cells should be exactly in phase with each other, something which it seems would be necessary to produce interference. NB this issue came up in a crystallography class several years ago, and I have been ruminating on it, on and off, since then. JPK *** Jacob Keller Northwestern University 6541 N. Francisco #3 Chicago IL 60645 (847)467-4049 [EMAIL PROTECTED] ***
Re: [ccp4bb] Questions about diffraction
On Fri, 24 Aug 2007 14:40:13 -0600 Michel Fodje [EMAIL PROTECTED] wrote: The mathematics works but doesn't necessarily mean the current interpretation of the mathematics has any resemblance to what actually happens in reality. Sure, it does. Crystallography is traditionally derived using classical wave mechanics, but you can take a quantum approach, using the First Born approximation (a single photon scatters elastically from a point source exactly once). If you want to speak about what a single photon does, then you have to resort to that approach. Except in rare instances, the photon interferes only with itself, regardless of how many or how few are present. The particle is the photon, and the wave is the propensity for the photon to appear at a given position on the detector. QM teaches us that the entire experiment, in this case the crystal lattice, has to be taken into account, (which simply means you have to add amplitudes rather than intensities). It is the same thing with a single particle going through a double slit. BOTH slits must be taken into account, as both are possible paths. A crystal is simply a near-infinite diffraction grating in three dimensions, but the physical interpretation is identical. Feynman developed the most intuitive way of looking at this, which is to sum over all possible paths before squaring the wave. Unfortunately, the accompanying mathematical treatment is a bit hairy. Bill
Re: [ccp4bb] Questions about diffraction
Michel Fodje wrote: For every direction where there is destructive interference and a loss of energy there is a direction where there is constructive interference that piles up energy. If you integrate over all directions energy is conserved. For the total integrated energy to be conserved, energy will have to be created in certain directions to compensate for the loss in other directions. So in a direction in which the condition is met, the total will have to be more than the sum of the waves in that direction. How about considering the possibility that all photons coming into the sample are diffracted -- just in different directions. So that what is happening is not constructive and destructive interference but a kind sorting of the photons based on a certain property of the photons, maybe the phase. You seem to be operating under the impression that there are two diffracting waves that later destructively interfere. All constructive and destructive interference occurs at the point of scattering. There is no energy that heads off in a direction that later disappears - Nothing ever went in that direction. You have the same problem with your idea of two waves, out of phase but identical in wavelength, that scatter off an electron. The two waves, if they are coherent, would interfere with each other long before they reach the electron and become a single wave. If they are not coherent they will interact with the scatter independently and produce incoherent diffraction waves, which will sum by intensity independent of phase. I can get into deep trouble with this next point so I hope a physicist jumps on me where I'm wrong. All light sources are coherent to a degree. A laser is pretty much 100% coherent and my pocket flashlight is hardly coherent at all. I seem to recall that there is a parameter called the coherence length that measures the distance within a beam that the light is coherent. The coherence length of a rotating anode X-ray generator is small but unit cells are smaller so there are plenty of unit cells to form a nice diffraction pattern. Your second paragraph is just the Copenhagen Interpretation of the wave function. If you want to think of photons then the diffraction wave we are talking about is the wave function and that function maps the probability of finding a photon. Wave/particle duality says we can look at the experiment either way. Dale Tronrud
Re: [ccp4bb] Questions about diffraction
Here's a fun way to think of it: A photon hits a crystal and will diffract off in a certain direction with the same energy as the original photon. The direction is subject to a probability distribution based on the lattice, with angles at the diffraction conditions being most likely and the broadness of the peaks in the distribution arising from imperfections in the lattice. The photon propagates as this probability distribution and then is forced to select from the distribution because we stuck a detector up. The diffraction pattern we observe is the sum of many such photons interacting with the crystal. I think this is consistent with the math. James Jacob Keller wrote: For the total integrated energy to be conserved, energy will have to be created in certain directions to compensate for the loss in other directions. So in a direction in which the condition is met, the total will have to be more than the sum of the waves in that direction. How about considering the possibility that all photons coming into the sample are diffracted -- just in different directions. So that what is happening is not constructive and destructive interference but a kind sorting of the photons based on a certain property of the photons, maybe the phase. * I think of it that each photon that happens to be perturb an electron, i.e., Thomson scattering, sends out a spherical wave, which has anisotropy to it, i.e., the wave front is more concentrated in the forward direction. These spherical waves interfere with each other, making the diffraction pattern. Something for you to chew on: how is it that the electrons of the protein, which are presumably not in phase with each other nor in exactly the same place in their orbitals from unit cell to unit cell (maybe they are?) when they scatter the photons, they result in interference? What are the chances that the scattering electrons are exactly in the same place as the electrons in another unit cell, or of the same phase? And would they not need to be in the same place to sub-angstrom precision to scatter coherently? I would suggest two possible answers, neither of which am I entirely satisfied: 1. Something about the crystalline state induces the protein molecules' molecular orbitals to be totally in synch with each other. This seems too miraculous to be true, in a way. Nevertheless, it would account for the data, I think. 2. The scattering electrons are elusive probablistic entities which are really no place at all. This, however, does not solve the problem of the phases (not in the usual sense of finding fourier phases) which is that it seems unlikely that electrons in multiple unit cells should be exactly in phase with each other, something which it seems would be necessary to produce interference. NB this issue came up in a crystallography class several years ago, and I have been ruminating on it, on and off, since then. JPK *** Jacob Keller Northwestern University 6541 N. Francisco #3 Chicago IL 60645 (847)467-4049 [EMAIL PROTECTED] *** -- James Stroud UCLA-DOE Institute for Genomics and Proteomics Box 951570 Los Angeles, CA 90095 http://www.jamesstroud.com/
Re: [ccp4bb] Questions about diffraction
Without resorting to a circular argument? You are asking too much. However, this probability distribution is perfectly described by considering a component wave model wherein coherence of the component waves correlates with peaks in the probability distribution--i.e. Bragg's Law. IANAM (I am not a mathematician), but, if pressed, I would posit that one could decompose the fun description just a little bit and consider the lattice not as *groups* of reflecting planes, but as individual planes. In such a case, each single reflecting plane would contribute a probability distribution with an angular dependence. The total probability distribution would then be the sum of the probability distributions for every plane in the lattice. Your next question might be, what's the probability distribution for a single plane. Well, I would imagine that it has a maximum where the angle of incidence equals the angle of reflection and that the phase of a component probability distributions is spatially (i.e. angularly) directly related to the phase of the originating photon. The sum distribution of the reflected photon takes into account the angular phase dependence of its components and so one gets positive and negative interference between component distributions. James Jacob Keller wrote: Yes, but why should the directions of diffraction conditions be most probable (one of your premises)? ==Original message text=== On Fri, 24 Aug 2007 4:54:53 pm CDT James Stroud wrote: Here's a fun way to think of it: A photon hits a crystal and will diffract off in a certain direction with the same energy as the original photon. The direction is subject to a probability distribution based on the lattice, with angles at the diffraction conditions being most likely and the broadness of the peaks in the distribution arising from imperfections in the lattice. The photon propagates as this probability distribution and then is forced to select from the distribution because we stuck a detector up. The diffraction pattern we observe is the sum of many such photons interacting with the crystal. -- James Stroud UCLA-DOE Institute for Genomics and Proteomics Box 951570 Los Angeles, CA 90095 http://www.jamesstroud.com/
Re: [ccp4bb] Questions about diffraction
Would it be taking it too far to suggest that one could go all the way and consider that each electron diffracts not as groups in a plane but as individual electrons and a photon impinging on an electron with with a specific phase will be diffracted in a specific direction. However the lattice arrangement of the electrons will statistically accumulate photons which impinged on electrons on a specific family of planes in one direction at the detector. Such that the crystal is a phase sorter. In which case diffraction is not based on constructive or destructive interference but on conservation of some property of the photon, such as angular momentum? IANAM either. On Fri, 2007-08-24 at 15:36 -0700, James Stroud wrote: Without resorting to a circular argument? You are asking too much. However, this probability distribution is perfectly described by considering a component wave model wherein coherence of the component waves correlates with peaks in the probability distribution--i.e. Bragg's Law. IANAM (I am not a mathematician), but, if pressed, I would posit that one could decompose the fun description just a little bit and consider the lattice not as *groups* of reflecting planes, but as individual planes. In such a case, each single reflecting plane would contribute a probability distribution with an angular dependence. The total probability distribution would then be the sum of the probability distributions for every plane in the lattice. Your next question might be, what's the probability distribution for a single plane. Well, I would imagine that it has a maximum where the angle of incidence equals the angle of reflection and that the phase of a component probability distributions is spatially (i.e. angularly) directly related to the phase of the originating photon. The sum distribution of the reflected photon takes into account the angular phase dependence of its components and so one gets positive and negative interference between component distributions. James Jacob Keller wrote: Yes, but why should the directions of diffraction conditions be most probable (one of your premises)? ==Original message text=== On Fri, 24 Aug 2007 4:54:53 pm CDT James Stroud wrote: Here's a fun way to think of it: A photon hits a crystal and will diffract off in a certain direction with the same energy as the original photon. The direction is subject to a probability distribution based on the lattice, with angles at the diffraction conditions being most likely and the broadness of the peaks in the distribution arising from imperfections in the lattice. The photon propagates as this probability distribution and then is forced to select from the distribution because we stuck a detector up. The diffraction pattern we observe is the sum of many such photons interacting with the crystal.
[ccp4bb] Off Topic:crystallization in the presence of glycerol
Dear All, I am working with a protein that requires 10% glycerol throughout the purification to keep it soluble. I have been very worried that having glycerol in my protein solution when I am trying to crystallize it will prevent me from obtaining crystals. I am curious to see if others have successfully obtained crystals of proteins that required glycerol to keep them soluble and if so what concentration of glycerol could you use? Should I be concerned about this or am I worrying too much? Thanks for your input Rob -- Robert J.Gruninger B.Sc. PhD Student Department of Chemistry Biochemistry University of Lethbridge, 4401 University Dr. Lethbridge, AB, Canada, T1K 3M4 Phone:(403)329-2746 Fax:(403)329-2057
Re: [ccp4bb] Off Topic:crystallization in the presence of glycerol
We have crystallised many things with 10% Glycerol. If % is high enough, there is also often the added bonus in that the crystals are naturally cryoprotected! J Edward Berry [EMAIL PROTECTED] wrote: We've grown crystals of the cytochrome bc1 complex in the presence of glycerol. I think as high as 25% in the initial droplet (protein in 50% glycerol mixed with equal volume of precipitant), but that was diluted somewhat by reverse vapor diffusion. Glycerol tends to increase the solubility in our case, so the PEG concentration we had to use in those experiments was higher than in later glycerol-free setups. Ed Rob Gruninger wrote: Dear All, I am working with a protein that requires 10% glycerol throughout the purification to keep it soluble. I have been very worried that having glycerol in my protein solution when I am trying to crystallize it will prevent me from obtaining crystals. I am curious to see if others have successfully obtained crystals of proteins that required glycerol to keep them soluble and if so what concentration of glycerol could you use? Should I be concerned about this or am I worrying too much? Thanks for your input Rob -- Professor James Whisstock NHMRC Principal Research Fellow / Monash University Senior Logan fellow Department of Biochemistry and Molecular Biology Monash University, Clayton Campus, PO Box 13d, VIC, 3800, Australia +613 9905 3747 (Phone) +613 9905 4699 (Fax) +61 418 170 585 (Mobile)
Re: [ccp4bb] Off Topic:crystallization in the presence of glycerol
I have crystallized a refolded membrane protein in presence of Glycerol. It did not seem to affect crystallizability and speed of crystallization. If you have no luck in presence of glycerol, try to lower the glycerol concentration without compromising on the stability of the protein (buffer exchange). Remember that Glycerol concentration is going to drop when you mix it with reservoir solution, depending upon the volume ratio that you use. Glycerol is known to delay crystallization a bit. There are cases where the quality of crystals are improved. subjective! Good luck Kumar = Original Message From Rob Gruninger [EMAIL PROTECTED] = Dear All, I am working with a protein that requires 10% glycerol throughout the purification to keep it soluble. I have been very worried that having glycerol in my protein solution when I am trying to crystallize it will prevent me from obtaining crystals. I am curious to see if others have successfully obtained crystals of proteins that required glycerol to keep them soluble and if so what concentration of glycerol could you use? Should I be concerned about this or am I worrying too much? Thanks for your input Rob -- Robert J.Gruninger B.Sc. PhD Student Department of Chemistry Biochemistry University of Lethbridge, 4401 University Dr. Lethbridge, AB, Canada, T1K 3M4 Phone:(403)329-2746 Fax:(403)329-2057 Dept. of Biochemistry, Cellular and Molecular Biology, Walters Life Science, # 406, University of Tennessee, TN, Knoxville, USA
