Re: [ccp4bb] program to calculate electron density at x,y,z (SUMMARY)
On Tue, 2011-04-12 at 22:54 -0400, Edward A. Berry wrote: What about doing the Fourier summation at the precise location requested, in order to not calculate the map or interpolate at all? Input would be the mtz file rather than map file. eab One advantage of map interpolation is speed - my code (admittedly not optimized for speed) is about 6x slower than sftools/mapman combination (the advantage of fast fourier transform and explicit use of symmetry). But these days it is only 20ms per atom on a 40,000 reflection dataset (on a modest 2.33GHz Core2 Q8200 CPU). -- Hurry up before we all come back to our senses! Julian, King of Lemurs
Re: [ccp4bb] program to calculate electron density at x,y,z (SUMMARY)
Dear Tim, On Mon, 2011-04-11 at 10:44 +0200, Tim Gruene wrote: since you pointed it out I wonder if there is any reasonable (i.e. w.r.t. data error/ resolution) difference between the interpolated values and the calculated value. I actually doubt that That should depend on the quality of interpolation primarily (i.e. grid used, resolution, etc). With a 1.9A test case the discrepancy runs somewhere around 3%, which likely is within experimental error. This is using sftools to calculate the map and mapman peek command, all with defaults. Direct calculation versus interpolation looks like this http://tinyurl.com/67f79oe For the record, getting this via mapman is not entirely trivial. Atoms should be inside the map (expanding map to the full unit cell and shifting all the atoms by symmetry inside the (0,1) range was my solution). Also, it fails for some atoms (~0.5% in this case) with Spline interpolation error, in which case the output pdb file has ** instead of B-factor (perhaps switching to interpolation can fix that). Also for the record, I believe there is a bug in sftools that messes up the expansion to P1, but only for some space groups/circumstances. For example, another test case in P3121 looks like this (~30% average discrepancy) http://tinyurl.com/69wqhvp but upon closer inspection, the maps after expansion into P1 (or those calculated with sftools FFT command) show a lot of noise. Cheers, Ed -- Hurry up before we all come back to our senses! Julian, King of Lemurs
Re: [ccp4bb] program to calculate electron density at x,y,z (SUMMARY)
On Fri, 2011-04-08 at 18:06 -0700, Pavel Afonine wrote: phenix.map_value_at_point map_coeffs.mtz label=2FOFC point=1 2 3 point=4 5 6 Cool. Afaiu, this is interpolation. A useful extension would be automatic picking of (x,y,z) from a pdb-file (a la mapman), although a determined person can definitely come up with a script that converts a pdb file into a list of point statements. -- Hurry up before we all come back to our senses! Julian, King of Lemurs
Re: [ccp4bb] program to calculate electron density at x,y,z (SUMMARY)
Hi Ed, yes, this is the eight-point interpolation, but since you can select to choose very small grid step for the map calculation (grid_step parameter), I hope this should be ok. If necessary, I can add an option so it will give you the map value at the closest grid point instead of interpolation or even both (although I guess the latter would be too much). In the next build (dev-728 and up) it will be possible to use a PDB file as a source of points. Pavel. On Tue, Apr 12, 2011 at 9:37 AM, Ed Pozharski epozh...@umaryland.eduwrote: On Fri, 2011-04-08 at 18:06 -0700, Pavel Afonine wrote: phenix.map_value_at_point map_coeffs.mtz label=2FOFC point=1 2 3 point=4 5 6 Cool. Afaiu, this is interpolation. A useful extension would be automatic picking of (x,y,z) from a pdb-file (a la mapman), although a determined person can definitely come up with a script that converts a pdb file into a list of point statements. -- Hurry up before we all come back to our senses! Julian, King of Lemurs
Re: [ccp4bb] program to calculate electron density at x,y,z (SUMMARY)
Pavel Afonine wrote: Hi Ed, yes, this is the eight-point interpolation, but since you can select to choose very small grid step for the map calculation (grid_step parameter), I hope this should be ok. If necessary, I can add an option so it will give you the map value at the closest grid point instead of interpolation or even both (although I guess the latter would be too much). What about doing the Fourier summation at the precise location requested, in order to not calculate the map or interpolate at all? Input would be the mtz file rather than map file. eab In the next build (dev-728 and up) it will be possible to use a PDB file as a source of points. Pavel. On Tue, Apr 12, 2011 at 9:37 AM, Ed Pozharski epozh...@umaryland.edu mailto:epozh...@umaryland.edu wrote: On Fri, 2011-04-08 at 18:06 -0700, Pavel Afonine wrote: phenix.map_value_at_point map_coeffs.mtz label=2FOFC point=1 2 3 point=4 5 6 Cool. Afaiu, this is interpolation. A useful extension would be automatic picking of (x,y,z) from a pdb-file (a la mapman), although a determined person can definitely come up with a script that converts a pdb file into a list of point statements. -- Hurry up before we all come back to our senses! Julian, King of Lemurs
Re: [ccp4bb] program to calculate electron density at x,y,z (SUMMARY)
That is exactly what HYDENS is doing. A good interpolation with small grid steps should be equally good but with current computers and just a few hundred or even thousand points to evaluate, a classical Fourier summation is pretty fast and, for me, easier to program than a proper cubic-spline interpolation. Bart On 11-04-12 08:54 PM, Edward A. Berry wrote: Pavel Afonine wrote: Hi Ed, yes, this is the eight-point interpolation, but since you can select to choose very small grid step for the map calculation (grid_step parameter), I hope this should be ok. If necessary, I can add an option so it will give you the map value at the closest grid point instead of interpolation or even both (although I guess the latter would be too much). What about doing the Fourier summation at the precise location requested, in order to not calculate the map or interpolate at all? Input would be the mtz file rather than map file. eab In the next build (dev-728 and up) it will be possible to use a PDB file as a source of points. Pavel. On Tue, Apr 12, 2011 at 9:37 AM, Ed Pozharski epozh...@umaryland.edu mailto:epozh...@umaryland.edu wrote: On Fri, 2011-04-08 at 18:06 -0700, Pavel Afonine wrote: phenix.map_value_at_point map_coeffs.mtz label=2FOFC point=1 2 3 point=4 5 6 Cool. Afaiu, this is interpolation. A useful extension would be automatic picking of (x,y,z) from a pdb-file (a la mapman), although a determined person can definitely come up with a script that converts a pdb file into a list of point statements. -- Hurry up before we all come back to our senses! Julian, King of Lemurs -- Bart Hazes (Associate Professor) Dept. of Medical Microbiology Immunology University of Alberta 1-15 Medical Sciences Building Edmonton, Alberta Canada, T6G 2H7 phone: 1-780-492-0042 fax:1-780-492-7521
Re: [ccp4bb] program to calculate electron density at x,y,z (SUMMARY)
Hi Ed, yes, this is one of possible ways of doing this. Although I doubt it will make any (significant) difference in practice compared to other options. All mentioned methods should normally result in similar values. Pavel. On Tue, Apr 12, 2011 at 7:54 PM, Edward A. Berry ber...@upstate.edu wrote: Pavel Afonine wrote: Hi Ed, yes, this is the eight-point interpolation, but since you can select to choose very small grid step for the map calculation (grid_step parameter), I hope this should be ok. If necessary, I can add an option so it will give you the map value at the closest grid point instead of interpolation or even both (although I guess the latter would be too much). What about doing the Fourier summation at the precise location requested, in order to not calculate the map or interpolate at all? Input would be the mtz file rather than map file. eab In the next build (dev-728 and up) it will be possible to use a PDB file as a source of points. Pavel. On Tue, Apr 12, 2011 at 9:37 AM, Ed Pozharski epozh...@umaryland.edu mailto:epozh...@umaryland.edu wrote: On Fri, 2011-04-08 at 18:06 -0700, Pavel Afonine wrote: phenix.map_value_at_point map_coeffs.mtz label=2FOFC point=1 2 3 point=4 5 6 Cool. Afaiu, this is interpolation. A useful extension would be automatic picking of (x,y,z) from a pdb-file (a la mapman), although a determined person can definitely come up with a script that converts a pdb file into a list of point statements. -- Hurry up before we all come back to our senses! Julian, King of Lemurs
Re: [ccp4bb] program to calculate electron density at x,y,z (SUMMARY)
Dear Ed, since you pointed it out I wonder if there is any reasonable (i.e. w.r.t. data error/ resolution) difference between the interpolated values and the calculated value. I actually doubt that Cheers, Tim On Fri, Apr 08, 2011 at 12:07:21PM -0400, Ed Pozharski wrote: Thanks to everyone for their suggestions. The closest solution was 1. Expand dataset in P1 using SFTOOLS (keyword EXPAND) 2. Write it out in text file (WRITE data.hkl format(3i5,2f16.3) col col1 col2) 3. Use program HYDENS (Bart Hazes) It should be noted that the current version of HYDENS only works in orthorhombic spacegroups (although there may be a workaround of pre-calculating fractional coordinates with coordconv and then setting the unit cell to 1x1x1). This was the only solution that actually calculates electron density at the atomic positions. The rest of options were all to calculate the map and then interpolate it to the select (x,y,z). Several options exist for that (in the order they were received): 1. MAPMAN (PEEK keyword) (Gerard Kleywegt, James Holton, Edward Berry) 2. CCTBX (Pavel Afonin) 3. COOT (density_at_point, probably uses clipper libs) (Paul Emsley) 4. CHIMERA (Values at Atom Positions) (Eric Pettersen) On Fri, 2011-04-01 at 11:16 -0400, Ed Pozharski wrote: I need to calculate the electron density values for a list of spatial locations (e.g. atom positions in a model) using an mtz-file that already contains map coefficients. To write my own code may be easier than I think (if one can manipulate mtz columns, isn't the only problem left how to incorporate symmetry-related reflections?), but I would need an alternative at least for troubleshooting purposes. So, Does anyone know of a software tool that can calculate point electron density for every atom in a structure? If I would have to bring a dependency into this, the best choice for me would be clipper libs. Thanks in advance, Ed. -- Hurry up before we all come back to our senses! Julian, King of Lemurs -- -- Tim Gruene Institut fuer anorganische Chemie Tammannstr. 4 D-37077 Goettingen phone: +49 (0)551 39 22149 GPG Key ID = A46BEE1A signature.asc Description: Digital signature
Re: [ccp4bb] program to calculate electron density at x,y,z (SUMMARY)
Thanks to everyone for their suggestions. The closest solution was 1. Expand dataset in P1 using SFTOOLS (keyword EXPAND) 2. Write it out in text file (WRITE data.hkl format(3i5,2f16.3) col col1 col2) 3. Use program HYDENS (Bart Hazes) It should be noted that the current version of HYDENS only works in orthorhombic spacegroups (although there may be a workaround of pre-calculating fractional coordinates with coordconv and then setting the unit cell to 1x1x1). This was the only solution that actually calculates electron density at the atomic positions. The rest of options were all to calculate the map and then interpolate it to the select (x,y,z). Several options exist for that (in the order they were received): 1. MAPMAN (PEEK keyword) (Gerard Kleywegt, James Holton, Edward Berry) 2. CCTBX (Pavel Afonin) 3. COOT (density_at_point, probably uses clipper libs) (Paul Emsley) 4. CHIMERA (Values at Atom Positions) (Eric Pettersen) On Fri, 2011-04-01 at 11:16 -0400, Ed Pozharski wrote: I need to calculate the electron density values for a list of spatial locations (e.g. atom positions in a model) using an mtz-file that already contains map coefficients. To write my own code may be easier than I think (if one can manipulate mtz columns, isn't the only problem left how to incorporate symmetry-related reflections?), but I would need an alternative at least for troubleshooting purposes. So, Does anyone know of a software tool that can calculate point electron density for every atom in a structure? If I would have to bring a dependency into this, the best choice for me would be clipper libs. Thanks in advance, Ed. -- Hurry up before we all come back to our senses! Julian, King of Lemurs
Re: [ccp4bb] program to calculate electron density at x,y,z (SUMMARY)
Hi Ed, thanks for nice summary! Just a quick update while on this subject: (using nightly build dev-724 an up) you will be able to get density value at a given point using just one command: phenix.map_value_at_point map_coeffs.mtz label=2FOFC point=1 2 3 point=4 5 6 where you can specify as many points as you wish. You can request to use either sigma or volume scaled map, as well as you can specify the grid_step for map finess. You can use a parameter file if the number of points is too large to be specified in the command line. Pavel. On Fri, Apr 8, 2011 at 9:07 AM, Ed Pozharski epozh...@umaryland.edu wrote: Thanks to everyone for their suggestions. The closest solution was 1. Expand dataset in P1 using SFTOOLS (keyword EXPAND) 2. Write it out in text file (WRITE data.hkl format(3i5,2f16.3) col col1 col2) 3. Use program HYDENS (Bart Hazes) It should be noted that the current version of HYDENS only works in orthorhombic spacegroups (although there may be a workaround of pre-calculating fractional coordinates with coordconv and then setting the unit cell to 1x1x1). This was the only solution that actually calculates electron density at the atomic positions. The rest of options were all to calculate the map and then interpolate it to the select (x,y,z). Several options exist for that (in the order they were received): 1. MAPMAN (PEEK keyword) (Gerard Kleywegt, James Holton, Edward Berry) 2. CCTBX (Pavel Afonin) 3. COOT (density_at_point, probably uses clipper libs) (Paul Emsley) 4. CHIMERA (Values at Atom Positions) (Eric Pettersen) On Fri, 2011-04-01 at 11:16 -0400, Ed Pozharski wrote: I need to calculate the electron density values for a list of spatial locations (e.g. atom positions in a model) using an mtz-file that already contains map coefficients. To write my own code may be easier than I think (if one can manipulate mtz columns, isn't the only problem left how to incorporate symmetry-related reflections?), but I would need an alternative at least for troubleshooting purposes. So, Does anyone know of a software tool that can calculate point electron density for every atom in a structure? If I would have to bring a dependency into this, the best choice for me would be clipper libs. Thanks in advance, Ed. -- Hurry up before we all come back to our senses! Julian, King of Lemurs
Re: [ccp4bb] program to calculate electron density at x,y,z
On Apr 1, 2011, at 4:00 PM, Ed Pozharski wrote: I need to calculate the electron density values for a list of spatial locations (e.g. atom positions in a model) using an mtz-file that already contains map coefficients. To write my own code may be easier than I think (if one can manipulate mtz columns, isn't the only problem left how to incorporate symmetry-related reflections?), but I would need an alternative at least for troubleshooting purposes. So, Does anyone know of a software tool that can calculate point electron density for every atom in a structure? If I would have to bring a dependency into this, the best choice for me would be clipper libs. Hi Ed, You'd have to convert the map to ccp4 or XPLOR format, but once you've done that it's pretty straightforward to get the values at atom positions using Chimera, pretty much as described here: http://plato.cgl.ucsf.edu/pipermail/chimera-users/2011-March/006202.html except skipping the part where you average them across a residue. --Eric Eric Pettersen UCSF Computer Graphics Lab
Re: [ccp4bb] program to calculate electron density at x,y,z
Ed Pozharski wrote: I need to calculate the electron density values for a list of spatial locations (e.g. atom positions in a model) using an mtz-file that already contains map coefficients. To write my own code may be easier than I think (if one can manipulate mtz columns, isn't the only problem left how to incorporate symmetry-related reflections?), but I would need an alternative at least for troubleshooting purposes. So, Does anyone know of a software tool that can calculate point electron density for every atom in a structure? fft to calculate the map, then mapman with the peek value command. Give it a pdb file (fek.pdb) with the coordinates of the atoms, it returns a pdb file with electron density at those points in the B-factor column. Several options for interpolating map values to the chosen point, iirc. Something like: setenv MAPSIZE 500 /data/trp/berry/usf/rave/lx_mapman -b eof re m1 d.map CCP4 !norm m1 !pick level 5.2 !pick peaks m1 danopeaks.pdb pdb peek value m1 fek.pdb fepkhght.pdb int quit eof If I would have to bring a dependency into this, the best choice for me would be clipper libs. Thanks in advance, Ed. -- I'd jump in myself, if I weren't so good at whistling. Julian, King of Lemurs
Re: [ccp4bb] program to calculate electron density at x,y,z
Hi Ed, if you are familiar with CCTBX then map_value = map_data.eight_point_interpolation(site_fractional) Also, there is a similar method that will just give you the density value at closes grid point. Let me know if interested, and I can send you a 10 lines Python script-example that will do it. Pavel. On Fri, Apr 1, 2011 at 8:16 AM, Ed Pozharski epozh...@umaryland.edu wrote: I need to calculate the electron density values for a list of spatial locations (e.g. atom positions in a model) using an mtz-file that already contains map coefficients. To write my own code may be easier than I think (if one can manipulate mtz columns, isn't the only problem left how to incorporate symmetry-related reflections?), but I would need an alternative at least for troubleshooting purposes. So, Does anyone know of a software tool that can calculate point electron density for every atom in a structure? If I would have to bring a dependency into this, the best choice for me would be clipper libs. Thanks in advance, Ed. -- I'd jump in myself, if I weren't so good at whistling. Julian, King of Lemurs
Re: [ccp4bb] program to calculate electron density at x,y,z
Hi Ed, I wrote a short program name HYDENS that takes a PDB file and an H K L amplitude phase file for a full hemisphere of data. You can make the latter from an MTZ with sftools. The program is on my website at http://129.128.24.248/highlights.html. There is a linux executable as well as the source code that should compile with any standard fortran compiler. Bart On 11-04-01 09:16 AM, Ed Pozharski wrote: I need to calculate the electron density values for a list of spatial locations (e.g. atom positions in a model) using an mtz-file that already contains map coefficients. To write my own code may be easier than I think (if one can manipulate mtz columns, isn't the only problem left how to incorporate symmetry-related reflections?), but I would need an alternative at least for troubleshooting purposes. So, Does anyone know of a software tool that can calculate point electron density for every atom in a structure? If I would have to bring a dependency into this, the best choice for me would be clipper libs. Thanks in advance, Ed. -- Bart Hazes (Associate Professor) Dept. of Medical Microbiology Immunology University of Alberta 1-15 Medical Sciences Building Edmonton, Alberta Canada, T6G 2H7 phone: 1-780-492-0042 fax:1-780-492-7521
