Re: [ccp4bb] Vista
Deena, A lot of this comes down to personal preference at the end of the day..you just have to find what works for you. I primarily use MacOSX and Linux. I also have a Vista partition on my laptop to share Microsoft office files with others in the department that use Windows or Macs..no comment on crystallography applications, but I think Vista is a regression from XP and I do not particularly enjoy using it. Linux has, obviously, been very well supported by software developers for many years...pre-compiled binaries for the popular flavours are mostly available and, with the odd afternoon bashing (no pun intended) your head off the monitor excluded, is fairly logical once you get to grips with it. Fedora/RedHat and Ubuntu are both excellent distributions...I'm currently using Fedora 8 on a (fairly new) Dell XPS M1330 and it mostly worked out-the-box. In my opinion MacOSX falls short in package managers..fink, in my humble opinion, is vastly inferior to yum and apt..although this may be partly down to my inability to dedicate the time required to truly get to grips with the program. Also, I much prefer GNOME (got to have those multiple desktops) over QUARTZ (MacOSX), although with some work you can get GNOME running on a Mac.. Bill Scott provides excellent support for crystallography applications on MacOSX and, I believe, debian-based distros like Ubuntu so you should refer to him on those. Gentoo is also well supported, I think..apologies for not remembering who labors on this and if any of this is repetitious.. Gently fanning the embers, Iain Anastassis Perrakis wrote: flames on Vista or XP, I shamelessly admit that I personally totally fail to see why - ideology left aside - I would ever buy a *laptop* which is not a Mac. Even for the fact that when a Mac is asleep you open the cover and it actually comes up in 1 sec, exactly where it was last evening, its worth it ! I run MS Office (I actually do like Word), iWork (Keynote is what PP might some day be ... Pages is great for eg posters), iLife for home, Coot, O, CCP4, Phenix, ARP/wARP, Solve/Resolve, Phaser, Pymol, Papers (!!! a dream that will not come true for Windows-based scientists - http://mekentosj.com/), R, Adobe and all work. And you also have Xcode, the GUI builder, management tools, and others for development. For the last 4 years I use a G4 as my laptop which is also my desktop with external monitor and keyboard/mouse - I have dropped it a couple of times, have been around the world, still works. I only switched to Macs in 2001 and I am still happy. And I did not even mention that it LOOKS better ;-)) flames off Apologies for not answering the original question, but the discussion was drifting this way. I am now waiting to see what Vista can do that Windows cannot. I am aware of one application I want and does not run on a Mac, and that is the Polar Fitness software for my bike. Argh. Most spyware and virus detection software also do not run in a Mac btw! For a reason: you do not need them dudes ! A. PS And no, I don't work for Apple, I don't get free gifts from Apple, but if Steve is reading and he wants to send me a Mac Air for my kind words, please, go ahead ! ;-)))
Re: [ccp4bb] Vista
Ok, I downloaded and installed CCP4 from the website on Jan 24, left the defaults as they were. Flip -Original Message- From: Scott Pegan [mailto:[EMAIL PROTECTED] Sent: Wednesday, February 20, 2008 22:49 To: [EMAIL PROTECTED] Cc: ccp4bb@jiscmail.ac.uk Subject: Re: [ccp4bb] Vista My attempt was a desktop installation. This was ~1 month ago. Scott On Wed, February 20, 2008 3:34 pm, Flip Hoedemaeker wrote: Strange, CCP4 and Coot run just fine on my Vista laptop... Have not tried all programs though. Flip -Original Message- From: CCP4 bulletin board [mailto:[EMAIL PROTECTED] On Behalf Of Scott Pegan Sent: Wednesday, February 20, 2008 22:04 To: CCP4BB@JISCMAIL.AC.UK Subject: Re: [ccp4bb] Vista Deena, If you are going to run windows, have to use XP. I tried vista, and the software wouldn't work. Scott On Wed, February 20, 2008 2:44 pm, Paul Paukstelis wrote: Or don't buy an OS at all. There are a number of vendors out there that sell OEM notebooks without OSs. I picked up a Compal IF90 with WSXGA+ (1680x1050) for a good price. --paul Yi Zhou wrote: I think we are all using Linux, so just choose the cheaper OS (you have to delete it anyway). Yi On Wed, 2008-02-20 at 15:11 -0500, deena wrote: Hi All, I am about to buy a laptop and find that the XP is twice the price of Vista. Does anyone have positive experience using Vista for crystallographic packages: CCP4, Coot, O, X-plor? or must I still avoid it? Thanks, Deena Deena Abells Oren, PhD Manager, Structural Biology Resource Center Rockefeller University 1230 York Avenue New York, NY 10065-6399 phone: 212- 327-7429 fax: 212-327-7389 -- Paul Paukstelis, Ph.D. Research Associate Institute for Cellular and Molecular Biology The University of Texas at Austin P: 512-471-4778, F: 512-232-3420 [EMAIL PROTECTED] -- Scott D. Pegan, Ph.D. Visiting Senior Research Specialist Center for Pharmaceutical Biotechnology University of Illinois at Chicago -- Scott D. Pegan, Ph.D. Visiting Senior Research Specialist Center for Pharmaceutical Biotechnology University of Illinois at Chicago
Re: [ccp4bb] XDS and overlaps
Engin Ozkan schrieb: Hi everyone, I have been recently relying on XDS quite a bit, but at the same time worrying about how XDS treats overlaps. We had one dataset that both HKL2000 and Mosflm would show to have severe overlaps, as expected due to unit cell parameters and the unfortunate crystal orientation in the loop. We always ended up with completeness percentages in the 70's. XDS can find the same lattice, index and scale the data, but yields a 100% complete mtz (and a nice structure). Without the HKL/Mosflm-like GUI, it is difficult to assess the fate of the overlapped observations in XDS. What I could see with VIEW was that some observations were being divided into several ovals, probably different reflections, but I'm not very certain. So, the basic question is, how does XDS treat overlaps? I could not find in the documentation an answer to this question; the single mention of overlaps I could find tells me that XDS can recognize overlaps, but does not tell me if it rejects them, or divvies them up into separate reflections, and if that is the case, how does it divide them, and how reliable is that? Depending on how it divides the overlaps, could that affect commonly-used intensity stats and distributions? Thanks, Engin Engin, the basic answer is: a) each pixel of the detector is assigned to its nearest reflection in reciprocal space b) some of these pixels will mostly allow the background estimation, others will mostly contribute to the integration area (but as they are transformed into a local coordinate system there is not a 1:1 relationship). At this step, pixels which should be background but are higher than expected (due to overlap) are rejected. c) for each reflection, the background is estimated, and the 3D profile is assembled from the pixels contributing to it d) a comparison is made: for a reflection, is the percentage of its observed profile assembled in c) larger than some constant (called MINPK in XDS.INP)? If the answer is no, this reflection will be discarded (you could call this situation overlap). Among other things, this means that: a) the program does _not_ look around each reflection to detect an overlap situation, it just tries to gather the pixels for each reflection b) as a user, when your crystal-detector distance was chosen too low or the reflections are very broad (resulting in generally strong overlap), you may reduce MINPK down to 50. This will result in more completeness, but you should monitor the quality of the resulting data. Conversely, if you raise MINPK over its default of 75 you will discard more reflections, but the resulting dataset will be a bit cleaner. The reference is W. Kabsch (1988) Evaluation of single-crystal X-ray diffraction data from a position-sensitive detector. J. Appl. Cryst. 21, 916-924. (http://dx.doi.org/10.1107/S0021889888007903) HTH, Kay -- Kay Diederichshttp://strucbio.biologie.uni-konstanz.de email: [EMAIL PROTECTED]Tel +49 7531 88 4049 Fax 3183 Fachbereich Biologie, Universität Konstanz, Box M647, D-78457 Konstanz smime.p7s Description: S/MIME Cryptographic Signature
[ccp4bb] Aurigene Crystallography requirement
Aurigene is looking for young, energetic and committed individuals to work with its Crystallography team. Ideal candidate will have a Ph.D. in the area of protein crystallography with publications in reputed journals. Post doctoral experience in the area is preferable although not essential. Candidates with Masters degree with the relevant experience of solving protein structures and understating protein-ligand interactions are also encouraged to apply. Additionally, knowledge of the scripts, software installation and familiarity in Linux/UNIX is desirable. Please send in your resumes to [EMAIL PROTECTED] with subject title Aurigene-Crystallography-2008-01. *The need is immediate* Aurigene is a Bangalore based Biotech company working in many drug discovery programs in the area of Metabolic syndrome, Oncology, Inflammation etc. For more details visit: www.aurigene.com http://www.aurigene.com/ . Best regards, Ramesh Sistla Ph.D. This email and its attachments are confidential and proprietary to AURIGENE Discovery Technologies and are meant for the intended recipient only. If you are not the intended recipient, do not take any actions based on the contents of this message, except to permanently delete the message and its attachments and inform the sender immediately image001.gif
Re: [ccp4bb] Vista
What virtualisation software are you using? VMWare, virtualbox, xen? Not seeing anything means, you can't even see to prompt when the virtual machines tries to boot from CD/DVD? Tim -- Tim Gruene Institut fuer anorganische Chemie Tammannstr. 4 D-37077 Goettingen GPG Key ID = A46BEE1A On Thu, 21 Feb 2008, Flip Hoedemaeker wrote: This actually pops up a question on a somewhat related issue, I've recently tried to install Fedora 8 on a virtual PC, but I get caught up with the video display being completely warped. Since I don't see anything I cannot even begin to troubleshoot the issue. Has anybody got a solution for this? Flip -Original Message- From: CCP4 bulletin board [mailto:[EMAIL PROTECTED] On Behalf Of Iain Kerr Sent: Thursday, February 21, 2008 09:45 To: CCP4BB@JISCMAIL.AC.UK Subject: Re: [ccp4bb] Vista Deena, A lot of this comes down to personal preference at the end of the day..you just have to find what works for you. I primarily use MacOSX and Linux. I also have a Vista partition on my laptop to share Microsoft office files with others in the department that use Windows or Macs..no comment on crystallography applications, but I think Vista is a regression from XP and I do not particularly enjoy using it. Linux has, obviously, been very well supported by software developers for many years...pre-compiled binaries for the popular flavours are mostly available and, with the odd afternoon bashing (no pun intended) your head off the monitor excluded, is fairly logical once you get to grips with it. Fedora/RedHat and Ubuntu are both excellent distributions...I'm currently using Fedora 8 on a (fairly new) Dell XPS M1330 and it mostly worked out-the-box. In my opinion MacOSX falls short in package managers..fink, in my humble opinion, is vastly inferior to yum and apt..although this may be partly down to my inability to dedicate the time required to truly get to grips with the program. Also, I much prefer GNOME (got to have those multiple desktops) over QUARTZ (MacOSX), although with some work you can get GNOME running on a Mac.. Bill Scott provides excellent support for crystallography applications on MacOSX and, I believe, debian-based distros like Ubuntu so you should refer to him on those. Gentoo is also well supported, I think..apologies for not remembering who labors on this and if any of this is repetitious.. Gently fanning the embers, Iain Anastassis Perrakis wrote: flames on Vista or XP, I shamelessly admit that I personally totally fail to see why - ideology left aside - I would ever buy a *laptop* which is not a Mac. Even for the fact that when a Mac is asleep you open the cover and it actually comes up in 1 sec, exactly where it was last evening, its worth it ! I run MS Office (I actually do like Word), iWork (Keynote is what PP might some day be ... Pages is great for eg posters), iLife for home, Coot, O, CCP4, Phenix, ARP/wARP, Solve/Resolve, Phaser, Pymol, Papers (!!! a dream that will not come true for Windows-based scientists - http://mekentosj.com/), R, Adobe and all work. And you also have Xcode, the GUI builder, management tools, and others for development. For the last 4 years I use a G4 as my laptop which is also my desktop with external monitor and keyboard/mouse - I have dropped it a couple of times, have been around the world, still works. I only switched to Macs in 2001 and I am still happy. And I did not even mention that it LOOKS better ;-)) flames off Apologies for not answering the original question, but the discussion was drifting this way. I am now waiting to see what Vista can do that Windows cannot. I am aware of one application I want and does not run on a Mac, and that is the Polar Fitness software for my bike. Argh. Most spyware and virus detection software also do not run in a Mac btw! For a reason: you do not need them dudes ! A. PS And no, I don't work for Apple, I don't get free gifts from Apple, but if Steve is reading and he wants to send me a Mac Air for my kind words, please, go ahead ! ;-)))
Re: [ccp4bb] Vista
Dear Flip, I have had trouble with video drivers for various flavours of Linux. If if driver is not working then, as you say, you just cannot do anything after booting normally. You need to boot the machine in a safe mode or single user mode (init 1). You can then edit the /etc/X11/XF86Config file, using `vi` of course. If you have documentation on the monitor, that will tell you the correct horizontal sync and vertical refresh rates. Also you need to ensure that you have the correct options in the Screen and Device sections. I have not installed a virtual machine, so there maybe additional considerations of which I have no knowledge. Best of luck,Adam On Thu, 21 Feb 2008, Flip Hoedemaeker wrote: This actually pops up a question on a somewhat related issue, I've recently tried to install Fedora 8 on a virtual PC, but I get caught up with the video display being completely warped. Since I don't see anything I cannot even begin to troubleshoot the issue. Has anybody got a solution for this? Flip -Original Message- From: CCP4 bulletin board [mailto:[EMAIL PROTECTED] On Behalf Of Iain Kerr Sent: Thursday, February 21, 2008 09:45 To: CCP4BB@JISCMAIL.AC.UK Subject: Re: [ccp4bb] Vista Deena, A lot of this comes down to personal preference at the end of the day..you just have to find what works for you. I primarily use MacOSX and Linux. I also have a Vista partition on my laptop to share Microsoft office files with others in the department that use Windows or Macs..no comment on crystallography applications, but I think Vista is a regression from XP and I do not particularly enjoy using it. Linux has, obviously, been very well supported by software developers for many years...pre-compiled binaries for the popular flavours are mostly available and, with the odd afternoon bashing (no pun intended) your head off the monitor excluded, is fairly logical once you get to grips with it. Fedora/RedHat and Ubuntu are both excellent distributions...I'm currently using Fedora 8 on a (fairly new) Dell XPS M1330 and it mostly worked out-the-box. In my opinion MacOSX falls short in package managers..fink, in my humble opinion, is vastly inferior to yum and apt..although this may be partly down to my inability to dedicate the time required to truly get to grips with the program. Also, I much prefer GNOME (got to have those multiple desktops) over QUARTZ (MacOSX), although with some work you can get GNOME running on a Mac.. Bill Scott provides excellent support for crystallography applications on MacOSX and, I believe, debian-based distros like Ubuntu so you should refer to him on those. Gentoo is also well supported, I think..apologies for not remembering who labors on this and if any of this is repetitious.. Gently fanning the embers, Iain Anastassis Perrakis wrote: flames on Vista or XP, I shamelessly admit that I personally totally fail to see why - ideology left aside - I would ever buy a *laptop* which is not a Mac. Even for the fact that when a Mac is asleep you open the cover and it actually comes up in 1 sec, exactly where it was last evening, its worth it ! I run MS Office (I actually do like Word), iWork (Keynote is what PP might some day be ... Pages is great for eg posters), iLife for home, Coot, O, CCP4, Phenix, ARP/wARP, Solve/Resolve, Phaser, Pymol, Papers (!!! a dream that will not come true for Windows-based scientists - http://mekentosj.com/), R, Adobe and all work. And you also have Xcode, the GUI builder, management tools, and others for development. For the last 4 years I use a G4 as my laptop which is also my desktop with external monitor and keyboard/mouse - I have dropped it a couple of times, have been around the world, still works. I only switched to Macs in 2001 and I am still happy. And I did not even mention that it LOOKS better ;-)) flames off Apologies for not answering the original question, but the discussion was drifting this way. I am now waiting to see what Vista can do that Windows cannot. I am aware of one application I want and does not run on a Mac, and that is the Polar Fitness software for my bike. Argh. Most spyware and virus detection software also do not run in a Mac btw! For a reason: you do not need them dudes ! A. PS And no, I don't work for Apple, I don't get free gifts from Apple, but if Steve is reading and he wants to send me a Mac Air for my kind words, please, go ahead ! ;-)))
Re: [ccp4bb] Tough Low-Res MR
Absolutely try Phaser! See Phaser documents for all the nifty combinations. Multiple molecules in MR model; break down molecule by domains etc etc. You can trim down side chains, make hybrid models and what not. All easy to get up and going through the GUI. Once the GUI drove me insane because of some 'hyphen' mark in the file name; sometimes I just prefer to set up scripts. Run to the documentation and examples, which are both pretty darned good. NO! I don't get paid by Randy. However, given the low resolution and the ambiguities you describe, especially of the combination in the ASU and if the two subunits are closely related and of similar lengths, my hunch is that experimental phasing will become necessary at some point. Since you describe you have 2*AB in the ASU, first search for AB as search model. One way for internal check is to feed Phaser only Molecule A and ask to look for 4 molecules. Then, feed only Molecule B and look for 4 molecules. If all these results look the same, I'd be worried and I might think of SelMet-labeling one of the two molecules or something like that. Just some thoughts. Good luck and just download Phaser for starters. Raji -Included Message-- Date: 21-feb-2008 06:40:57 -0500 From: James Stroud [EMAIL PROTECTED] To: CCP4BB@JISCMAIL.AC.UK Subject: [ccp4bb] Tough Low-Res MR Hello All, I have a tough ~3.5 #197; (pushing it) MR problem where I have a solution of sorts, but because I'm working with a heterodimer of two closely related subunits (with two such heterodimers in the ASU) I have a 2**2 possibilities for the arrangement of these subunits in the ASU. Each subunit is composed of two more-or-less independent domains. Basically I'm looking for the best possible software to disambiguate this problem. I usually use CNS for these problems, but I think I may have exhausted its capabilities. I've heard that there have been some advances in MR in recent years, but I haven't kept up with all of the software. Does anyone have suggestions for packages to try? James -- James Stroud UCLA-DOE Institute for Genomics and Proteomics Box 951570 Los Angeles, CA 90095 http://www.jamesstroud.com -End of Included Message--
Re: [ccp4bb] XDS and overlaps
Simon Kolstoe schrieb: Whilst we are on the subject of XDS... I had difficulty processing a data-set in mosflm the other day so on the recommendation of a colleague switched to xds which, with a bit of tweaking, seemed to work really well. I converted the resulting XDS_ASCII.HKL using xdsconv and then f2mtz ready for phaser and refmac. We do it in the same way here. However, my colleague then told me that xds handled missing reflections differently from the usual mosflm/CCP4 route I have honestly not the slightest idea what your colleague was referring to. and thus I had to run the CCP4 program UNIQUE before I tried refinement as apparently refmac does not like reflection files originally processed with xds. As I couldn't In the case of a new project, one should run uniqueify or some other means of assigning reflections to the free set (thin shells come to mind; see earlier discussions on CCP4BB). In the case of an old project, one should transfer the free set of reflections from some master data set to the new dataset. None of this is specific to XDS. HTH, Kay find anything in the literature about this I was wondering whether this advice is still up to date? Thanks, Simon On 21 Feb 2008, at 09:44, Kay Diederichs wrote: Engin Ozkan schrieb: Hi everyone, I have been recently relying on XDS quite a bit, but at the same time worrying about how XDS treats overlaps. We had one dataset that both HKL2000 and Mosflm would show to have severe overlaps, as expected due to unit cell parameters and the unfortunate crystal orientation in the loop. We always ended up with completeness percentages in the 70's. XDS can find the same lattice, index and scale the data, but yields a 100% complete mtz (and a nice structure). Without the HKL/Mosflm-like GUI, it is difficult to assess the fate of the overlapped observations in XDS. What I could see with VIEW was that some observations were being divided into several ovals, probably different reflections, but I'm not very certain. So, the basic question is, how does XDS treat overlaps? I could not find in the documentation an answer to this question; the single mention of overlaps I could find tells me that XDS can recognize overlaps, but does not tell me if it rejects them, or divvies them up into separate reflections, and if that is the case, how does it divide them, and how reliable is that? Depending on how it divides the overlaps, could that affect commonly-used intensity stats and distributions? Thanks, Engin Engin, the basic answer is: a) each pixel of the detector is assigned to its nearest reflection in reciprocal space b) some of these pixels will mostly allow the background estimation, others will mostly contribute to the integration area (but as they are transformed into a local coordinate system there is not a 1:1 relationship). At this step, pixels which should be background but are higher than expected (due to overlap) are rejected. c) for each reflection, the background is estimated, and the 3D profile is assembled from the pixels contributing to it d) a comparison is made: for a reflection, is the percentage of its observed profile assembled in c) larger than some constant (called MINPK in XDS.INP)? If the answer is no, this reflection will be discarded (you could call this situation overlap). Among other things, this means that: a) the program does _not_ look around each reflection to detect an overlap situation, it just tries to gather the pixels for each reflection b) as a user, when your crystal-detector distance was chosen too low or the reflections are very broad (resulting in generally strong overlap), you may reduce MINPK down to 50. This will result in more completeness, but you should monitor the quality of the resulting data. Conversely, if you raise MINPK over its default of 75 you will discard more reflections, but the resulting dataset will be a bit cleaner. The reference is W. Kabsch (1988) Evaluation of single-crystal X-ray diffraction data from a position-sensitive detector. J. Appl. Cryst. 21, 916-924. (http://dx.doi.org/10.1107/S0021889888007903) HTH, Kay --Kay Diederichshttp://strucbio.biologie.uni-konstanz.de email: [EMAIL PROTECTED]Tel +49 7531 88 4049 Fax 3183 Fachbereich Biologie, Universität Konstanz, Box M647, D-78457 Konstanz -- Kay Diederichshttp://strucbio.biologie.uni-konstanz.de email: [EMAIL PROTECTED]Tel +49 7531 88 4049 Fax 3183 Fachbereich Biologie, Universität Konstanz, Box M647, D-78457 Konstanz smime.p7s Description: S/MIME Cryptographic Signature
Re: [ccp4bb] XDS and overlaps
In my experience when going from XDS via some intermediate file to mtz format, XDS uses in some cases a different reciprocal asymmetric unit as mtz uses, which may result in only half of the reflections being used and/or ccp4 programs getting confused. By using UNIQUE, one makes sure that the reflections are mapped to the correct asymmetric unit. It has nothing to do with missing reflections but is in many cases essential. Best regards, Herman -Original Message- From: CCP4 bulletin board [mailto:[EMAIL PROTECTED] On Behalf Of Kay Diederichs Sent: Thursday, February 21, 2008 3:46 PM To: CCP4BB@JISCMAIL.AC.UK Subject: Re: [ccp4bb] XDS and overlaps Simon Kolstoe schrieb: Whilst we are on the subject of XDS... I had difficulty processing a data-set in mosflm the other day so on the recommendation of a colleague switched to xds which, with a bit of tweaking, seemed to work really well. I converted the resulting XDS_ASCII.HKL using xdsconv and then f2mtz ready for phaser and refmac. We do it in the same way here. However, my colleague then told me that xds handled missing reflections differently from the usual mosflm/CCP4 route I have honestly not the slightest idea what your colleague was referring to. and thus I had to run the CCP4 program UNIQUE before I tried refinement as apparently refmac does not like reflection files originally processed with xds. As I couldn't In the case of a new project, one should run uniqueify or some other means of assigning reflections to the free set (thin shells come to mind; see earlier discussions on CCP4BB). In the case of an old project, one should transfer the free set of reflections from some master data set to the new dataset. None of this is specific to XDS. HTH, Kay find anything in the literature about this I was wondering whether this advice is still up to date? Thanks, Simon On 21 Feb 2008, at 09:44, Kay Diederichs wrote: Engin Ozkan schrieb: Hi everyone, I have been recently relying on XDS quite a bit, but at the same time worrying about how XDS treats overlaps. We had one dataset that both HKL2000 and Mosflm would show to have severe overlaps, as expected due to unit cell parameters and the unfortunate crystal orientation in the loop. We always ended up with completeness percentages in the 70's. XDS can find the same lattice, index and scale the data, but yields a 100% complete mtz (and a nice structure). Without the HKL/Mosflm-like GUI, it is difficult to assess the fate of the overlapped observations in XDS. What I could see with VIEW was that some observations were being divided into several ovals, probably different reflections, but I'm not very certain. So, the basic question is, how does XDS treat overlaps? I could not find in the documentation an answer to this question; the single mention of overlaps I could find tells me that XDS can recognize overlaps, but does not tell me if it rejects them, or divvies them up into separate reflections, and if that is the case, how does it divide them, and how reliable is that? Depending on how it divides the overlaps, could that affect commonly-used intensity stats and distributions? Thanks, Engin Engin, the basic answer is: a) each pixel of the detector is assigned to its nearest reflection in reciprocal space b) some of these pixels will mostly allow the background estimation, others will mostly contribute to the integration area (but as they are transformed into a local coordinate system there is not a 1:1 relationship). At this step, pixels which should be background but are higher than expected (due to overlap) are rejected. c) for each reflection, the background is estimated, and the 3D profile is assembled from the pixels contributing to it d) a comparison is made: for a reflection, is the percentage of its observed profile assembled in c) larger than some constant (called MINPK in XDS.INP)? If the answer is no, this reflection will be discarded (you could call this situation overlap). Among other things, this means that: a) the program does _not_ look around each reflection to detect an overlap situation, it just tries to gather the pixels for each reflection b) as a user, when your crystal-detector distance was chosen too low or the reflections are very broad (resulting in generally strong overlap), you may reduce MINPK down to 50. This will result in more completeness, but you should monitor the quality of the resulting data. Conversely, if you raise MINPK over its default of 75 you will discard more reflections, but the resulting dataset will be a bit cleaner. The reference is W. Kabsch (1988) Evaluation of single-crystal X-ray diffraction data from a position-sensitive detector. J. Appl. Cryst. 21, 916-924. (http://dx.doi.org/10.1107/S0021889888007903) HTH, Kay --Kay Diederichshttp://strucbio.biologie.uni-konstanz.de email:
Re: [ccp4bb] XDS and overlaps
Usually, I run CAD first after F2MTZ to make sure that the reflections are in the correct reciprocal asymmetric unit for CCP4 programs. I think, UNIQUE on its own doesn't do this, but the UNIQUEIFY script calls CAD, UNIQUE and FREERFLAG for setting a FreeR_flag column. The latter may or may not be wanted, depending on whether the test-set has been assigned by XDS/XSCALE, already. Best regards, Dirk. Am 21.02.2008 um 16:15 schrieb [EMAIL PROTECTED]: In my experience when going from XDS via some intermediate file to mtz format, XDS uses in some cases a different reciprocal asymmetric unit as mtz uses, which may result in only half of the reflections being used and/or ccp4 programs getting confused. By using UNIQUE, one makes sure that the reflections are mapped to the correct asymmetric unit. It has nothing to do with missing reflections but is in many cases essential. Best regards, Herman -Original Message- From: CCP4 bulletin board [mailto:[EMAIL PROTECTED] On Behalf Of Kay Diederichs Sent: Thursday, February 21, 2008 3:46 PM To: CCP4BB@JISCMAIL.AC.UK Subject: Re: [ccp4bb] XDS and overlaps Simon Kolstoe schrieb: Whilst we are on the subject of XDS... I had difficulty processing a data-set in mosflm the other day so on the recommendation of a colleague switched to xds which, with a bit of tweaking, seemed to work really well. I converted the resulting XDS_ASCII.HKL using xdsconv and then f2mtz ready for phaser and refmac. We do it in the same way here. However, my colleague then told me that xds handled missing reflections differently from the usual mosflm/CCP4 route I have honestly not the slightest idea what your colleague was referring to. and thus I had to run the CCP4 program UNIQUE before I tried refinement as apparently refmac does not like reflection files originally processed with xds. As I couldn't In the case of a new project, one should run uniqueify or some other means of assigning reflections to the free set (thin shells come to mind; see earlier discussions on CCP4BB). In the case of an old project, one should transfer the free set of reflections from some master data set to the new dataset. None of this is specific to XDS. HTH, Kay find anything in the literature about this I was wondering whether this advice is still up to date? Thanks, Simon On 21 Feb 2008, at 09:44, Kay Diederichs wrote: Engin Ozkan schrieb: Hi everyone, I have been recently relying on XDS quite a bit, but at the same time worrying about how XDS treats overlaps. We had one dataset that both HKL2000 and Mosflm would show to have severe overlaps, as expected due to unit cell parameters and the unfortunate crystal orientation in the loop. We always ended up with completeness percentages in the 70's. XDS can find the same lattice, index and scale the data, but yields a 100% complete mtz (and a nice structure). Without the HKL/Mosflm-like GUI, it is difficult to assess the fate of the overlapped observations in XDS. What I could see with VIEW was that some observations were being divided into several ovals, probably different reflections, but I'm not very certain. So, the basic question is, how does XDS treat overlaps? I could not find in the documentation an answer to this question; the single mention of overlaps I could find tells me that XDS can recognize overlaps, but does not tell me if it rejects them, or divvies them up into separate reflections, and if that is the case, how does it divide them, and how reliable is that? Depending on how it divides the overlaps, could that affect commonly-used intensity stats and distributions? Thanks, Engin Engin, the basic answer is: a) each pixel of the detector is assigned to its nearest reflection in reciprocal space b) some of these pixels will mostly allow the background estimation, others will mostly contribute to the integration area (but as they are transformed into a local coordinate system there is not a 1:1 relationship). At this step, pixels which should be background but are higher than expected (due to overlap) are rejected. c) for each reflection, the background is estimated, and the 3D profile is assembled from the pixels contributing to it d) a comparison is made: for a reflection, is the percentage of its observed profile assembled in c) larger than some constant (called MINPK in XDS.INP)? If the answer is no, this reflection will be discarded (you could call this situation overlap). Among other things, this means that: a) the program does _not_ look around each reflection to detect an overlap situation, it just tries to gather the pixels for each reflection b) as a user, when your crystal-detector distance was chosen too low or the reflections are very broad (resulting in generally strong overlap), you may reduce MINPK down to 50. This will result in more completeness, but you should monitor the quality of the resulting data. Conversely, if you raise MINPK over its default
Re: [ccp4bb] XDS and overlaps
I use XDSCONV to make the mtz file using all of the reflections. Then I use uniqueify in CCP4 to make sure the asymmetric unit is correct for CCP4, and tag the test set. Bernie Santarsiero On Thu, February 21, 2008 9:32 am, Dirk Kostrewa wrote: Usually, I run CAD first after F2MTZ to make sure that the reflections are in the correct reciprocal asymmetric unit for CCP4 programs. I think, UNIQUE on its own doesn't do this, but the UNIQUEIFY script calls CAD, UNIQUE and FREERFLAG for setting a FreeR_flag column. The latter may or may not be wanted, depending on whether the test-set has been assigned by XDS/XSCALE, already. Best regards, Dirk. Am 21.02.2008 um 16:15 schrieb [EMAIL PROTECTED]: In my experience when going from XDS via some intermediate file to mtz format, XDS uses in some cases a different reciprocal asymmetric unit as mtz uses, which may result in only half of the reflections being used and/or ccp4 programs getting confused. By using UNIQUE, one makes sure that the reflections are mapped to the correct asymmetric unit. It has nothing to do with missing reflections but is in many cases essential. Best regards, Herman -Original Message- From: CCP4 bulletin board [mailto:[EMAIL PROTECTED] On Behalf Of Kay Diederichs Sent: Thursday, February 21, 2008 3:46 PM To: CCP4BB@JISCMAIL.AC.UK Subject: Re: [ccp4bb] XDS and overlaps Simon Kolstoe schrieb: Whilst we are on the subject of XDS... I had difficulty processing a data-set in mosflm the other day so on the recommendation of a colleague switched to xds which, with a bit of tweaking, seemed to work really well. I converted the resulting XDS_ASCII.HKL using xdsconv and then f2mtz ready for phaser and refmac. We do it in the same way here. However, my colleague then told me that xds handled missing reflections differently from the usual mosflm/CCP4 route I have honestly not the slightest idea what your colleague was referring to. and thus I had to run the CCP4 program UNIQUE before I tried refinement as apparently refmac does not like reflection files originally processed with xds. As I couldn't In the case of a new project, one should run uniqueify or some other means of assigning reflections to the free set (thin shells come to mind; see earlier discussions on CCP4BB). In the case of an old project, one should transfer the free set of reflections from some master data set to the new dataset. None of this is specific to XDS. HTH, Kay find anything in the literature about this I was wondering whether this advice is still up to date? Thanks, Simon On 21 Feb 2008, at 09:44, Kay Diederichs wrote: Engin Ozkan schrieb: Hi everyone, I have been recently relying on XDS quite a bit, but at the same time worrying about how XDS treats overlaps. We had one dataset that both HKL2000 and Mosflm would show to have severe overlaps, as expected due to unit cell parameters and the unfortunate crystal orientation in the loop. We always ended up with completeness percentages in the 70's. XDS can find the same lattice, index and scale the data, but yields a 100% complete mtz (and a nice structure). Without the HKL/Mosflm-like GUI, it is difficult to assess the fate of the overlapped observations in XDS. What I could see with VIEW was that some observations were being divided into several ovals, probably different reflections, but I'm not very certain. So, the basic question is, how does XDS treat overlaps? I could not find in the documentation an answer to this question; the single mention of overlaps I could find tells me that XDS can recognize overlaps, but does not tell me if it rejects them, or divvies them up into separate reflections, and if that is the case, how does it divide them, and how reliable is that? Depending on how it divides the overlaps, could that affect commonly-used intensity stats and distributions? Thanks, Engin Engin, the basic answer is: a) each pixel of the detector is assigned to its nearest reflection in reciprocal space b) some of these pixels will mostly allow the background estimation, others will mostly contribute to the integration area (but as they are transformed into a local coordinate system there is not a 1:1 relationship). At this step, pixels which should be background but are higher than expected (due to overlap) are rejected. c) for each reflection, the background is estimated, and the 3D profile is assembled from the pixels contributing to it d) a comparison is made: for a reflection, is the percentage of its observed profile assembled in c) larger than some constant (called MINPK in XDS.INP)? If the answer is no, this reflection will be discarded (you could call this situation overlap). Among other things, this means that: a) the program does _not_ look around each reflection to detect an overlap situation, it just tries to gather the pixels for each reflection b) as a user, when your
Re: [ccp4bb] Tough Low-Res MR
Hi all, just to second previous statements: Phaser has done wonders for me in a few tough cases, also at low resolution: Eg. various 6-10AA data sets with highly homologous search models. Or a rather bad search model: 1.8AA rmsd over ~50% of the residues. Often up to a point where the solution was correct, however model bias did not allow to get further. Sometimes it turned out to be helpful to increase the numbers of rotation solutions that are considered for the translation search. Cheers Jan ## deCODE biostructures Bainbridge Island 2008/2/21, Roger Rowlett [EMAIL PROTECTED]: James Stroud wrote: Hello All, I have a tough ~3.5 Å (pushing it) MR problem where I have a solution of sorts, but because I'm working with a heterodimer of two closely related subunits (with two such heterodimers in the ASU) I have a 2**2 possibilities for the arrangement of these subunits in the ASU. Each subunit is composed of two more-or-less independent domains. Basically I'm looking for the best possible software to disambiguate this problem. I usually use CNS for these problems, but I think I may have exhausted its capabilities. I've heard that there have been some advances in MR in recent years, but I haven't kept up with all of the software. Does anyone have suggestions for packages to try? James -- James Stroud UCLA-DOE Institute for Genomics and Proteomics Box 951570 Los Angeles, CA 90095 http://www.jamesstroud.com Phaser and EPMR (now Open-EPMR) are among the best, and EPMR may actually work better in some cases without high resolution data. We've solved some really tough nuts to crack with both of these, and they are pretty fast to boot. Cheers, -- Roger S. Rowlett Professor Colgate University Presidential Scholar Department of Chemistry Colgate University 13 Oak Drive Hamilton, NY 13346 tel: (315)-228-7245 ofc: (315)-228-7395 fax: (315)-228-7935 email: [EMAIL PROTECTED]
Re: [ccp4bb] XDS and overlaps
In the past I have used the USF program DATAMAN to pick my Rfree in thin shells. Thus my data goes XDS- XDSCONV- DATAMAN- F2MTZ in order to get a reflection file that refmac and phaser are happy with. Do I then need to run the UNIQUEIFY script selecting the option keep existing FreeR data in order to avoid the reciprocal asymmetric unit problem? If so this problem must only occur occasionally as I have had structures refining with R Rfree's in the very low 20s without performing the UNIQUEIFY step. Simon On 21 Feb 2008, at 15:32, Dirk Kostrewa wrote: Usually, I run CAD first after F2MTZ to make sure that the reflections are in the correct reciprocal asymmetric unit for CCP4 programs. I think, UNIQUE on its own doesn't do this, but the UNIQUEIFY script calls CAD, UNIQUE and FREERFLAG for setting a FreeR_flag column. The latter may or may not be wanted, depending on whether the test-set has been assigned by XDS/XSCALE, already. Best regards, Dirk. Am 21.02.2008 um 16:15 schrieb [EMAIL PROTECTED]: In my experience when going from XDS via some intermediate file to mtz format, XDS uses in some cases a different reciprocal asymmetric unit as mtz uses, which may result in only half of the reflections being used and/or ccp4 programs getting confused. By using UNIQUE, one makes sure that the reflections are mapped to the correct asymmetric unit. It has nothing to do with missing reflections but is in many cases essential. Best regards, Herman -Original Message- From: CCP4 bulletin board [mailto:[EMAIL PROTECTED] On Behalf Of Kay Diederichs Sent: Thursday, February 21, 2008 3:46 PM To: CCP4BB@JISCMAIL.AC.UK Subject: Re: [ccp4bb] XDS and overlaps Simon Kolstoe schrieb: Whilst we are on the subject of XDS... I had difficulty processing a data-set in mosflm the other day so on the recommendation of a colleague switched to xds which, with a bit of tweaking, seemed to work really well. I converted the resulting XDS_ASCII.HKL using xdsconv and then f2mtz ready for phaser and refmac. We do it in the same way here. However, my colleague then told me that xds handled missing reflections differently from the usual mosflm/CCP4 route I have honestly not the slightest idea what your colleague was referring to. and thus I had to run the CCP4 program UNIQUE before I tried refinement as apparently refmac does not like reflection files originally processed with xds. As I couldn't In the case of a new project, one should run uniqueify or some other means of assigning reflections to the free set (thin shells come to mind; see earlier discussions on CCP4BB). In the case of an old project, one should transfer the free set of reflections from some master data set to the new dataset. None of this is specific to XDS. HTH, Kay find anything in the literature about this I was wondering whether this advice is still up to date? Thanks, Simon On 21 Feb 2008, at 09:44, Kay Diederichs wrote: Engin Ozkan schrieb: Hi everyone, I have been recently relying on XDS quite a bit, but at the same time worrying about how XDS treats overlaps. We had one dataset that both HKL2000 and Mosflm would show to have severe overlaps, as expected due to unit cell parameters and the unfortunate crystal orientation in the loop. We always ended up with completeness percentages in the 70's. XDS can find the same lattice, index and scale the data, but yields a 100% complete mtz (and a nice structure). Without the HKL/Mosflm-like GUI, it is difficult to assess the fate of the overlapped observations in XDS. What I could see with VIEW was that some observations were being divided into several ovals, probably different reflections, but I'm not very certain. So, the basic question is, how does XDS treat overlaps? I could not find in the documentation an answer to this question; the single mention of overlaps I could find tells me that XDS can recognize overlaps, but does not tell me if it rejects them, or divvies them up into separate reflections, and if that is the case, how does it divide them, and how reliable is that? Depending on how it divides the overlaps, could that affect commonly-used intensity stats and distributions? Thanks, Engin Engin, the basic answer is: a) each pixel of the detector is assigned to its nearest reflection in reciprocal space b) some of these pixels will mostly allow the background estimation, others will mostly contribute to the integration area (but as they are transformed into a local coordinate system there is not a 1:1 relationship). At this step, pixels which should be background but are higher than expected (due to overlap) are rejected. c) for each reflection, the background is estimated, and the 3D profile is assembled from the pixels contributing to it d) a comparison is made: for a reflection, is the percentage of its observed profile assembled in c) larger than some constant (called MINPK in XDS.INP)? If the
Re: [ccp4bb] XDS and overlaps
doh... of course I also run CAD - which means (according to XDSCONV.LP) that the data would have already been converted into the CCP4-asymmetric unit prior to me getting to DATAMAN to pick my Rfree set. In which case we have solved the reciprocal asymmetric unit problem and thus get back to my original question (which I think has already been answered) as to whether or not there is anything else one needs to do to the data regarding missing reflections before trying to refine the model - I think the consensus so far is no and thus my colleagues advice is either dated or wrong. Simon On 21 Feb 2008, at 17:32, Michele Lunelli wrote: Simon Kolstoe wrote: I converted the resulting XDS_ASCII.HKL using xdsconv and then f2mtz ready for phaser and refmac. Sorry if this is obvious, but you should also run CAD after f2mtz, as reported at the end of the log file XDSCONV.LP. Michele
[ccp4bb] drag-and-drop (custom condition order) option in Crystal Screen Wizard
Hi all, To respond to a users suggestion, I've added drag-and-drop boxes to the crystal screen wizard, so now you can now set up your screening conditions in any order you want (web 2.0-ified if you will). You can also mix and match different kits from different manufacturers in the same tray. It’s written in JavaScript, so you’ll need that enabled (most browsers do by default). As I’ve only just started using this language, there are probably a few bugs in it – so please let me know. The only one I am aware of at the moment is that it doesn’t work in IE6 (I’m trying to fix that). To access it, go to http://www.pageforaday.com/xtalwizard and click on either the 24 Well - Custom Order or 96 Well - Custom Order buttons. I have also included a simple salt crystal predictor, which at the moment might be useful as a quick pre-screen check for metalloprotein buffers. It’s only meant as a guide, and to quench a new crystallographer’s initial enthusiasm for certain hits. As usual, please send me any comments, suggestions, or notification of any bugs. Cheers, AGS _ Helping your favorite cause is as easy as instant messaging. You IM, we give. http://im.live.com/Messenger/IM/Home/?source=text_hotmail_join
Re: [ccp4bb] Vista
I pretty much agree with Tassos, but he neglected to mention that on an Intel Mac you can also run Linux and Windows at native speeds, on the same machine, simultaneously. With Parallels software, you also get the added advantage of OpenGL 3D hardware acceleration, whereas with VMWare, you get 64 bit support, dual-core VMs, and better stability. Both solutions support drag drop as well as copy paste between interleaved application windows on a unified desktop. I've also noticed that virtualized OSes boot much faster than their non-virtualized equivalents, and that fully backing up a virtual computer is a simple copy operation. Of course, you'll spend a more money on software licenses, disk space, and RAM to make this work, but there is nothing on the planet that can compare with the compatibility, portability, and convenience of a MacBook Pro running OS X, Windows, Linux together under Parallels and/or VMware. PS. Me too Steve! Ditto Bill! Ditto Linus! Oh wait... From: CCP4 bulletin board [mailto:[EMAIL PROTECTED] On Behalf Of Anastassis Perrakis Sent: Thursday, February 21, 2008 9:23 AM To: CCP4BB@JISCMAIL.AC.UK Subject: Re: [ccp4bb] Vista flames on Vista or XP, I shamelessly admit that I personally totally fail to see why - ideology left aside - I would ever buy a *laptop* which is not a Mac. Even for the fact that when a Mac is asleep you open the cover and it actually comes up in 1 sec, exactly where it was last evening, its worth it ! I run MS Office (I actually do like Word), iWork (Keynote is what PP might some day be ... Pages is great for eg posters), iLife for home, Coot, O, CCP4, Phenix, ARP/wARP, Solve/Resolve, Phaser, Pymol, Papers (!!! a dream that will not come true for Windows-based scientists - http://mekentosj.com/), R, Adobe and all work. And you also have Xcode, the GUI builder, management tools, and others for development. For the last 4 years I use a G4 as my laptop which is also my desktop with external monitor and keyboard/mouse - I have dropped it a couple of times, have been around the world, still works. I only switched to Macs in 2001 and I am still happy. And I did not even mention that it LOOKS better ;-)) flames off Apologies for not answering the original question, but the discussion was drifting this way. I am now waiting to see what Vista can do that Windows cannot. I am aware of one application I want and does not run on a Mac, and that is the Polar Fitness software for my bike. Argh. Most spyware and virus detection software also do not run in a Mac btw! For a reason: you do not need them dudes ! A. PS And no, I don't work for Apple, I don't get free gifts from Apple, but if Steve is reading and he wants to send me a Mac Air for my kind words, please, go ahead ! ;-)))
Re: [ccp4bb] XDS and overlaps
Hi, Going back to your original question regarding integration with XDS. On Feb 20, 2008, at 11:54 PM, Engin Ozkan wrote: I have been recently relying on XDS quite a bit, but at the same time worrying about how XDS treats overlaps. We had one dataset that both HKL2000 and Mosflm would show to have severe overlaps, as expected due to unit cell parameters and the unfortunate crystal orientation in the loop. We always ended up with completeness percentages in the 70's. XDS can find the same lattice, index and scale the data, but yields a 100% complete mtz (and a nice structure). Without the HKL/Mosflm- like GUI, it is difficult to assess the fate of the overlapped observations in XDS. What I could see with VIEW was that some observations were being divided into several ovals, probably different reflections, but I'm not very certain. So, the basic question is, how does XDS treat overlaps? I could not find in the documentation an answer to this question; the single mention of overlaps I could find tells me that XDS can recognize overlaps, but does not tell me if it rejects them, or divvies them up into separate reflections, and if that is the case, how does it divide them, and how reliable is that? Depending on how it divides the overlaps, could that affect commonly-used intensity stats and distributions? One answer lies in the XDS 3D profiles (already been pointed out by Kay and Bernard). But XDS at least used to be quite liberal with inclusion of overlaps and we had one case quite a while back where Mosflm/DENZO would say ~40% overlaps while XDS happily integrated everything and delivered a complete dataset. The cumulative intensity distribution of this dataset looked strange (would love to give you a screenshot but I can't get hold of that old data in an instant) and the structure didn't refine well. Of course, the dataset should never have been collected like this (and no, I was not guilty) but that was what we had and crystals couldn't be repeated at that time. What worked in the end was to use the DENZO predictions and integrate it using PrOW which performs deconvolution of spatially overlapped spots. After this the cumulative intensity distribution looked normal and the structure refined well. Reference to PrOW: Acta Crystallogr D Biol Crystallogr. 55,1733-41. New processing tools for weak and/or spatially overlapped macromolecular diffraction patterns. by Dominique Bourgeois. Link: http://scripts.iucr.org/cgi-bin/paper?BA0019 Best regards, Martin . B. Martin Hallberg, PhD Assistant professor Department of Cell and Molecular Biology Medical Nobel Institute Karolinska Institutet Nobels v. 3 SE-171 77 Stockholm Sweden
Re: [ccp4bb] Tough Low-Res MR
Dear James -- I have a tough ~3.5 Å (pushing it) MR problem where I have a solution of sorts, but because I'm working with a heterodimer of two closely related subunits (with two such heterodimers in the ASU) I have a 2**2 possibilities for the arrangement of these subunits in the ASU. Each subunit is composed of two more-or-less independent domains. Basically I'm looking for the best possible software to disambiguate this problem. I usually use CNS for these problems, but I think I may have exhausted its capabilities. I've heard that there have been some advances in MR in recent years, but I haven't kept up with all of the software. Does anyone have suggestions for packages to try? I usually run in parallel Phaser and Balbes (ie Molrep), and check which one gives me the best solution (stats map). The wonder with Balbes is that it tries automatically a large batch of possibilities, models. Now, as a personal feeling, although I could solve a couple of structures with Phaser for which I had problems using Molrep, I often get solutions that overlap / crash with the neighboring molecule using Phaser. Obviously, this might be due to my low resolution data (it was around 4 A if I recall correctly), but the fact that Balbes did worked properly for the same data... HTH Kind regards. Leo Chavas Leonard, Ph.D. Research Associate Faculty of Life Sciences The University of Manchester The Michael Smith Building Oxford Road Manchester Lancashire M13 9PT Tel: +44(0)161-275-1586 e-mail: [EMAIL PROTECTED]
Re: [ccp4bb] XDS and overlaps
I should add that inspecting the average three-dimensional profiles in INTEGRATE.LP could give you an indication if you have overlaps or not. The profiles look like slices through a 3D unimodal gaussian. If you have overlaps it will be revealed by bi-/multi-modal features. However, these are average profiles so a small number of overlaps may not be apparent. /Michel On Wed, 2008-02-20 at 14:54 -0800, Engin Ozkan wrote: Hi everyone, I have been recently relying on XDS quite a bit, but at the same time worrying about how XDS treats overlaps. We had one dataset that both HKL2000 and Mosflm would show to have severe overlaps, as expected due to unit cell parameters and the unfortunate crystal orientation in the loop. We always ended up with completeness percentages in the 70's. XDS can find the same lattice, index and scale the data, but yields a 100% complete mtz (and a nice structure). Without the HKL/Mosflm-like GUI, it is difficult to assess the fate of the overlapped observations in XDS. What I could see with VIEW was that some observations were being divided into several ovals, probably different reflections, but I'm not very certain. So, the basic question is, how does XDS treat overlaps? I could not find in the documentation an answer to this question; the single mention of overlaps I could find tells me that XDS can recognize overlaps, but does not tell me if it rejects them, or divvies them up into separate reflections, and if that is the case, how does it divide them, and how reliable is that? Depending on how it divides the overlaps, could that affect commonly-used intensity stats and distributions? Thanks, Engin
Re: [ccp4bb] cns and refmac refinement
Then I used CNS to do annealing, then use refmac to do rigid body refinement. That can be why the R-factors went up. Rigid-body subsequent to simulated annealing will (if anything) undo the refinement ... Anastassis Perrakis wrote: Just to point out that I have missed the NCS presence; Lijun is *perfectly* correct; even more I should emphasize the point and say that at 3.0 A, Thou Must Use NCS (if I say again anything about Gospels and Kleywegt we end up with more facebook groups ...) I think there is no reason at this resolution edge (~3A), making bonds possibly resolvable, that NCS should not be used, By the way, it is a dimer and I didnot use NCS restarin during refinement. Below is from refmac log: I would however disagree with Lijun thats its instability. Of course the 180 cycles are of no use at all (10 cycles in Refmac are more than enough in 99% of cases), but Rfree shoots up from 32.6 to 36.2 in 12 cycles, which is clearly an indication of some sort of pathology, and not instability. Well, tight NCS is the best starting point in this case. Tassos
Re: [ccp4bb] Does NCS bias a randomly-chosen test set (even if not enforced)? [ccp4bb] an over refined structure
Dale Tronrud wrote: In summary, this argument depends on two assertions that you can argue with me about: 1) When a parameter is being used to fit the signal it was designed for, the resulting model develops predictive power and can lower both the working and free R. When a signal is perturbing the value of a parameter for which is was not designed, it is unlikely to improve its predictive power and the working R will tend to drop, but the free R will not (and may rise). 2) If the unmodeled signal in the data set is a property in real space and has the same symmetry as the molecule in the unit cell, the inappropriate fitting of parameters will be systematic with respect to that symmetry and the presence of a reflection in the working set will tend to cause its symmetry mate in the test set to be better predicted despite the fact that this predictive power does not extend to reflections that are unrelated by symmetry. This bias will occur for any kind of error as long as that error obeys the symmetry of the unit cell in real space. Well, I've had time now to think about this, and I find myself agreeing with point 1 and most of point 2 (that the unmodeled signal has the same symmetry as the model), and I still would argue that NCS symmetry does not bias the free set if it is not enforced. Sorry to be so persistent, and I'm sure 95% of readers will want to stop here, but: I don't see that the symmetry of unmodeled signal will cause a free reflection whose symmetry mate is working to be better predicted than another free reflection which is not sym-related to a working reflection, or than in the case where there is no symmetry. I think this assertion comes from noting that in the final refined structure the sym-related Fc's will be correlated, and since the Fo's are also correlated, and since the sign of |Fo-Fc| is correlated because of the symmetry of the un-modeled signal, this correlation of sym-related Fc's results in an artificial reduction of |Fo-Fc| at test reflections. That might be true of the correlation between sym-related Fc's were perfect. I want to argue that the correlation between Fc's results only from the approach of the test Fc to the F of the true (symmetrical) structure, i.e. the correlation follows from, rather than contributes to, the decrease of |Fo-Fc| at test reflections. The decrease in |Fo-Fc| at test reflections results only from the approach of the electron density of the model to that of the real structure (and the fact that the Fo's are good estimates of the diffraction pattern of the real structure), and this is exactly what the Free-R is supposed to measure. Let me describe the refinement process in perhaps oversimplified steps which make my argument clear, and then we may want to argue about the individual assumptions. In my view it all depends on what is driving what. Briefly, the need to minimize |Fo-Fc| at working reflections drives the structural changes, the resulting approach of the structure to the true structure drives the decrease in |Fo-Fc| at free reflections, and it is only this reduction in |Fo-Fc| at free reflections which brings about the correlation between free Fc and sym-related working Fc. You cannot then turn around and say the correlation between sym-related free and working Fc biases the |Fo-Fc|Free. To elaborate: The individual structural changes are driven by the need to minimize |Fo-Fc| at the working reflections. The refinement program reduces |Fo-Fc| at working reflections by a combination of (1) appropriate structural changes which actually make the model closer to the true structure, (2)Inappropriate structural changes which happen to reduce |Fo-Fc| by accounting for some of the un-modeled signal, but not in a way that resembles the real structure, and (3) fitting the noise in the measurements. The reduction in |Fo-Fc| at free reflections is driven by the fact that the changes make the model a better approximation to the electron density of the real structure, driving the Fc closer to the theoretical F's of the true structure, of which the Fo are a good approximation. This is mainly due to changes of type 1 above, appropriate modeling of the structure. The inappropriate movement of atoms into density may also improve free |Fo-Fc| at least at low resolution, so give a smaller decrease in Rfree than in Rwork. And fitting the noise will in general move the structure away from the true structure and so tend to increase |Fo-Fc|free. The point, which we may need to argue about, is that the only force driving the reduction of |Fo-Fc| at free reflections is the approach of the model electron density to that of the true structure. Finally, the correlation of free Fc with working Fc is driven by the approach of both to to their respective Fo, together with the fact that the Fo are highly correlated. The correlation can never be better than the correlation of free Fc to Fo, which we said in the previous step is due to improvement
Re: [ccp4bb] XDS and overlaps
In this case, XDS-processed data is clearly twinned, if one were to believe moments and the cumulative intensity distribution calculated by truncate (and everyone should - channelling Dr. Dodson). Why I'm worried about XDS separating relatively overlapped spots is the funny intensity stats that may result from this (as George Sheldrick and Martin Hallberg has pointed out)? The incomplete HKL2000-processed data (70%) still has intermediate values for the intensity stats, leaning towards twinning. This somewhat agrees with twinning, but not overzealous data integration. Engin P.S. If twinning (perfect in this case) is taken into account structure refines to 20/26% R at 2.8 Ang., unlike in Martin's case. Without twinning, numbers are 29/33%. Quoting George M. Sheldrick [EMAIL PROTECTED]: All SHELX programs and XPREP are also indifferent to the asu choice and to whether the data have been merged or not (even SHELX-76). It is CCP4 historical baggage and high time it was eliminated. On the official thread of this discussion, my impression is that 3D integration programs (like XDS) are able to handle overlapping reflections better than 2D integration programs, as one would expect. One simple test is the mean value of |E^2-1|; if is is too small, you either have twinning or reflection overlap. Unfortunately and surprisingly XDS often fails this test (especially if the data have been flattened with XSCALE), George On Thu, 21 Feb 2008, Peter Zwart wrote: vloeken in de kerk or use phenix, which is indifferent to format and asu choice. /vloeken in de kerk P Prof. George M. Sheldrick FRS Dept. Structural Chemistry, University of Goettingen, Tammannstr. 4, D37077 Goettingen, Germany Tel. +49-551-39-3021 or -3068 Fax. +49-551-39-2582
Re: [ccp4bb] XDS and overlaps
That was _supposed_ to say I'm not sure ... ! On 21 Feb 2008, at 21:01, Phil Evans wrote: On 21 Feb 2008, at 18:24, George M. Sheldrick wrote: All SHELX programs and XPREP are also indifferent to the asu choice and to whether the data have been merged or not (even SHELX-76). It is CCP4 historical baggage and high time it was eliminated. I'm sure that many if any of the CCP4 programs if any actually care about which asymmetric unit is used. It only matters when you are collecting together different datasets, eg native derivative, and this handled automatically by CAD. Other people may have evidence otherwise Phil
Re: [ccp4bb] possible solutions for those troobleshooting ccp4 and ccp4i...
Hi, Was this an install from executables, or from source? I have just installed Ubuntu Gutsy on my laptop (everything including wireless worked out of the box - I think Ubuntu is great for people who want to simply use computers). I installed CCP4 6.0.2 with the automatic installer from the CCP4 website, using the redhat executables, and used the 'everything' option (why compile when someone else has already done this for you?). After I put the setup command in my .cshrc (using tcsh - old habit), the system is slightly broken: python is installed by CCP4 in /usr/local/lib and this is version 2.4 Some of the system administration interfaces (like for printing) expect python 2.5 and fail with a 'libglade.so missing' error when python 2.4 takes over. This took me quite a while to discover. I have not fully worked out whether python 2.4 is essential for ccp4: after deleting it from /usr/local/bin everything I have used (interface included) still seems to work. Johan James Thompson wrote: re: possible solutions for those troobleshooting ccp4 and ccp4i. On Ubuntu 7.10 - Gusty: A necessary shared lib file for ccp4 6.0.2 called libblas.so.2 was not present on my fresh system. So provide a link to libblas.so.3.0 On any system: Perhaps all your new jobs in ccp4i report a job status of STARTING forever and ever there are no errors in the session log, no ability to view com files, no output files. If none of the troubleshooting tips on the CCP4 site are helpful, see if you installed CRANK after CCP4 with regards to your PATH. In my new Ubuntu installation, I found an empty ccp4ish dummy file inside CRANK's bin directory that had me dummy guessing wrong for hours. Kind regards, James Thompson Department of Physiology and Biomedical Engineering E-mail thompson.james(at)mayo.edu -- + Dr. Johan P. Turkenburg X-ray facilities manager York Structural Biology Laboratory University of York Phone (+) 44 1904 328251 York YO10 5DD UK Fax (+) 44 1904 328266 +
Re: [ccp4bb] XDS and overlaps
There is of course nothing stopping you from having both severe overlap problems and twinning :-( In this case, XDS-processed data is clearly twinned, if one were to believe moments and the cumulative intensity distribution calculated by truncate (and everyone should - channelling Dr. Dodson). Why I'm worried about XDS separating relatively overlapped spots is the funny intensity stats that may result from this (as George Sheldrick and Martin Hallberg has pointed out)? The incomplete HKL2000-processed data (70%) still has intermediate values for the intensity stats, leaning towards twinning. This somewhat agrees with twinning, but not overzealous data integration. What happens if you play with spot parameters (to increase the number of reflections identified as overlaps) in Denzo? This will reduce the completeness even more but might be a worthwhile exercise to figure out what the problem really is. Can you get cleaner intensity statistics this way? Similarly, you can play with the MINPK parameter in XDS (as suggested by Kay). Best regards, Martin . B. Martin Hallberg, PhD Assistant professor Department of Cell and Molecular Biology Medical Nobel Institute Karolinska Institutet Nobels v. 3 SE-171 77 Stockholm Sweden
Re: [ccp4bb] XDS and overlaps
Hi, While on the subject, a related matter that may be of relevance: surprisingly many people do not remove the outliers after XDS processing (via using the REMOVE.HKL file) and this, in certain cases, has its effects on the intensity distribution and 'refinability'. Petri On Feb 21, 2008, at 11:44 AM, Kay Diederichs wrote: Engin Ozkan schrieb: Hi everyone, I have been recently relying on XDS quite a bit, but at the same time worrying about how XDS treats overlaps. We had one dataset that both HKL2000 and Mosflm would show to have severe overlaps, as expected due to unit cell parameters and the unfortunate crystal orientation in the loop. We always ended up with completeness percentages in the 70's. XDS can find the same lattice, index and scale the data, but yields a 100% complete mtz (and a nice structure). Without the HKL/ Mosflm-like GUI, it is difficult to assess the fate of the overlapped observations in XDS. What I could see with VIEW was that some observations were being divided into several ovals, probably different reflections, but I'm not very certain. So, the basic question is, how does XDS treat overlaps? I could not find in the documentation an answer to this question; the single mention of overlaps I could find tells me that XDS can recognize overlaps, but does not tell me if it rejects them, or divvies them up into separate reflections, and if that is the case, how does it divide them, and how reliable is that? Depending on how it divides the overlaps, could that affect commonly-used intensity stats and distributions? Thanks, Engin Engin, the basic answer is: a) each pixel of the detector is assigned to its nearest reflection in reciprocal space b) some of these pixels will mostly allow the background estimation, others will mostly contribute to the integration area (but as they are transformed into a local coordinate system there is not a 1:1 relationship). At this step, pixels which should be background but are higher than expected (due to overlap) are rejected. c) for each reflection, the background is estimated, and the 3D profile is assembled from the pixels contributing to it d) a comparison is made: for a reflection, is the percentage of its observed profile assembled in c) larger than some constant (called MINPK in XDS.INP)? If the answer is no, this reflection will be discarded (you could call this situation overlap). Among other things, this means that: a) the program does _not_ look around each reflection to detect an overlap situation, it just tries to gather the pixels for each reflection b) as a user, when your crystal-detector distance was chosen too low or the reflections are very broad (resulting in generally strong overlap), you may reduce MINPK down to 50. This will result in more completeness, but you should monitor the quality of the resulting data. Conversely, if you raise MINPK over its default of 75 you will discard more reflections, but the resulting dataset will be a bit cleaner. The reference is W. Kabsch (1988) Evaluation of single-crystal X-ray diffraction data from a position-sensitive detector. J. Appl. Cryst. 21, 916-924. (http://dx.doi.org/10.1107/S0021889888007903) HTH, Kay -- Kay Diederichshttp://strucbio.biologie.uni-konstanz.de email: [EMAIL PROTECTED]Tel +49 7531 88 4049 Fax 3183 Fachbereich Biologie, Universität Konstanz, Box M647, D-78457 Konstanz --- Petri Kursula, Ph.D. Academy Research Fellow Docent of Neurobiochemistry Department of Biochemistry University of Oulu Oulu, Finland [EMAIL PROTECTED] cc.oulu.fi/~pkursula www.biochem.oulu.fi/kursula ---
Re: [ccp4bb] XDS and overlaps
Dear all, I've written http://strucbio.biologie.uni-konstanz.de/xdswiki/index.php/INTEGRATE hoping that this will explain how XDS works (the last part focuses on the overlap question). Caveat: this is written to the best of my knowledge. I would also like to draw your attention to the article Quality Control in XDSwiki. This would benefit _a_ _lot_ if people who have/had problems with data from XDS could donate their raw data (to be put on our FTP server, like the PulG data), with the goal of having a proper data reduction (pinpointing and solving problems!) and evaluation in the wiki. I believe that by moving from anecdotal evidence and hearsay to an evidence-based evaluation of algorithms (with publicly accessible data), everybody would benefit. best, Kay Engin Ozkan schrieb: In this case, XDS-processed data is clearly twinned, if one were to believe moments and the cumulative intensity distribution calculated by truncate (and everyone should - channelling Dr. Dodson). Why I'm worried about XDS separating relatively overlapped spots is the funny intensity stats that may result from this (as George Sheldrick and Martin Hallberg has pointed out)? The incomplete HKL2000-processed data (70%) still has intermediate values for the intensity stats, leaning towards twinning. This somewhat agrees with twinning, but not overzealous data integration. Engin P.S. If twinning (perfect in this case) is taken into account structure refines to 20/26% R at 2.8 Ang., unlike in Martin's case. Without twinning, numbers are 29/33%. Quoting George M. Sheldrick [EMAIL PROTECTED]: All SHELX programs and XPREP are also indifferent to the asu choice and to whether the data have been merged or not (even SHELX-76). It is CCP4 historical baggage and high time it was eliminated. On the official thread of this discussion, my impression is that 3D integration programs (like XDS) are able to handle overlapping reflections better than 2D integration programs, as one would expect. One simple test is the mean value of |E^2-1|; if is is too small, you either have twinning or reflection overlap. Unfortunately and surprisingly XDS often fails this test (especially if the data have been flattened with XSCALE), George On Thu, 21 Feb 2008, Peter Zwart wrote: vloeken in de kerk or use phenix, which is indifferent to format and asu choice. /vloeken in de kerk P Prof. George M. Sheldrick FRS Dept. Structural Chemistry, University of Goettingen, Tammannstr. 4, D37077 Goettingen, Germany Tel. +49-551-39-3021 or -3068 Fax. +49-551-39-2582
