Re: [ccp4bb] Fine Phi Slicing

[Graeme Winter]

James,

On

On 20 Jul 2017, at 19:06, James Holton 
> wrote:

 In my experience you need at least an average of 1 photon/pixel/image before 
even the best data processing algorithms start to fall over.

I do not agree… both XDS and DIALS will (today) to pretty well with images 
which are a factor of 25 weaker than this (i.e. around .04 counts/pixel on 
average)

https://zenodo.org/record/49559

I think is the right set, it was measured a while back.

Background estimation was the bigger problem with this, since almost all of the 
pixels are 0…

Have a play, data like this are quite entertaining.

Cheers Graeme

-- 
This e-mail and any attachments may contain confidential, copyright and or 
privileged material, and are for the use of the intended addressee only. If you 
are not the intended addressee or an authorised recipient of the addressee 
please notify us of receipt by returning the e-mail and do not use, copy, 
retain, distribute or disclose the information in or attached to the e-mail.
Any opinions expressed within this e-mail are those of the individual and not 
necessarily of Diamond Light Source Ltd. 
Diamond Light Source Ltd. cannot guarantee that this e-mail or any attachments 
are free from viruses and we cannot accept liability for any damage which you 
may sustain as a result of software viruses which may be transmitted in or with 
the message.
Diamond Light Source Limited (company no. 4375679). Registered in England and 
Wales with its registered office at Diamond House, Harwell Science and 
Innovation Campus, Didcot, Oxfordshire, OX11 0DE, United Kingdom

Re: [ccp4bb] Fine Phi Slicing

[Keller, Jacob]

Based on this, a vision for the future:

A warehouse filled with sealed-tube, top-hat-profiled sources, super-accurate 
goniostats, and Dectris detectors, a robot running back and forth from a 
central dewar to place the crystals, all images 1-bit, intensities measured as 
probabilities; a day when crystal-frying will finally come to an end, or will 
be used routinely (as RIP) as the sure-fire way to solve crystal structures.

JPK



-Original Message-
From: CCP4 bulletin board [mailto:CCP4BB@JISCMAIL.AC.UK] On Behalf Of James 
Holton
Sent: Thursday, July 20, 2017 2:07 PM
To: CCP4BB@JISCMAIL.AC.UK
Subject: Re: [ccp4bb] Fine Phi Slicing

An important aspect of fine phi slicing that has not been mentioned yet (and 
took me a long time to figure out) is the impact of read-out time.  
Traditionally, read-out time is simply a delay that makes the overall data 
collection take longer, but with so-called "shutterless" data collection the 
read-out time can have a surprising impact on data quality.  It's 2 ms on my 
Pilatus3 S 6M.  This doesn't sound like much, and indeed 2 ms is also the 
timing jitter of my x-ray shutter, which had not been a problem with CCD 
detectors for 15 years.  The difference is that with so-called "shutterless" 
data collection not only can appreciable intensity fall into this 2 ms hole, 
but none of the data processing programs have a way to "correct" for it.  What 
you end up with is Rmerge/Rmeas values of 15-30% in the lowest-angle bin, and 
correspondingly low overall I/sigma.  At first,  I couldn't even solve lysozyme 
by S-SAD!  This had been an easy task with the Q315r I had just replaced.  The 
difference turned out to be "noise" coming from this read-out gap.

The 2 ms gap between images is only important if it is comparable to the time 
it takes a relp to transit the Ewald sphere.  At 1 deg/s and mosaic spread of 
0.5 deg this is 0.002 deg of missing data, or about 1% error in integrated 
intensity.  This is fine for most applications.  But if you are turning at 25 
deg/s with a room-temperature crystal of mosaicity
0.05 deg, then you could loose the spot entirely in a 2 ms read-out gap (100% 
error).  This is one of several arguments for fine phi slicing, where you make 
sure that every spot is not just observed, but split over
2-3 images.  This also helps the pile-up correction Gerd already mentioned.  
What is often overlooked, however, is that the error due to read-out gap is 
only relevant to partials.  Fulls don't experience it at all, so wide phi 
slicing is practically immune to it.  But with fine phi slicing everything is a 
partial, and 100% of the spots are going to take on read-out-gap error. So, 
what is the solution?  Slow down.

The problem with slowing down the spindle, of course, is radiation damage.  If 
you've got a flux of 1e12 photons/s into a 100x100 micron beam spot and 1 A 
wavelength you are dosing metal-free protein crystals at about 50 kGy/s.  Most 
room-temperature crystals can endure no more than 200 kGy, so they will live 
for about 4 seconds in this beam.  A detector framing at 25 Hz will only get 
100 images, no matter what the spindle speed. The decision then: is it better 
to get 100 deg at 1 
deg/image?  or 2.5 deg with fine phi slicing?   That is, if the mosaic 
spread is 0.05 deg, you can do no more than 0.025 deg/image and still barely 
qualify as "fine phi slicing". The 25 Hz framing rate dictates no less than 40 
ms exposures, and that means turning the spindle at (0.025 deg/ 0.04 s) = 0.625 
deg/s. Thus, we cover 2.5 deg in the 4 seconds before the crystal dies.  That's 
just algebra.  The pragmatic consequence is the difference between getting a 
complete dataset from one crystal and needing to merge 40 crystals.

Of course, you can attenuate 40x and get 100 fine-sliced degrees, but that will 
take 40x more beam time.  The images will also be 40x weaker.  
In my experience you need at least an average of 1 photon/pixel/image before 
even the best data processing algorithms start to fall over.  You can actually 
calculate photons/pixel/image beforehand if you know your flux and how thick 
your sample is:

photons/pixel = 1.2e-5*flux*exposure*thickness/pixels

where flux is in photons/s, exposure in seconds, thickness in microns and 
1.2e-5 comes from the NIST elastic scattering cross section of light atoms (C, 
N, O are all ~ 0.2 cm^2/g), the rough density of protein crystals (1.2 g/cm^3), 
and the fact that about half of all scattered photons land on a flat detector 
at typical distance from the sample, or: 
0.2*1.2*(1e-4 cm/um)/2 = 1.155e-5

So, if your flux is 1e12 photons/s and your sample is 100 um thick you will get 
~1 photon/pixel on a 6M in about 5 ms.  That corresponds to a framing rate of 
200 Hz.  If the detector can't go that fast, you need to attenuate.  Note this 
is the total sample thickness, including the stuff around the crystal.  Air 
scatter counts as 1 micr

Re: [ccp4bb] Fine Phi Slicing

[James Holton]

An important aspect of fine phi slicing that has not been mentioned yet 
(and took me a long time to figure out) is the impact of read-out time.  
Traditionally, read-out time is simply a delay that makes the overall 
data collection take longer, but with so-called "shutterless" data 
collection the read-out time can have a surprising impact on data 
quality.  It's 2 ms on my Pilatus3 S 6M.  This doesn't sound like much, 
and indeed 2 ms is also the timing jitter of my x-ray shutter, which  
had not been a problem with CCD detectors for 15 years.  The difference 
is that with so-called "shutterless" data collection not only can 
appreciable intensity fall into this 2 ms hole, but none of the data 
processing programs have a way to "correct" for it.  What you end up 
with is Rmerge/Rmeas values of 15-30% in the lowest-angle bin, and 
correspondingly low overall I/sigma.  At first,  I couldn't even solve 
lysozyme by S-SAD!  This had been an easy task with the Q315r I had just 
replaced.  The difference turned out to be "noise" coming from this 
read-out gap.


The 2 ms gap between images is only important if it is comparable to the 
time it takes a relp to transit the Ewald sphere.  At 1 deg/s and mosaic 
spread of 0.5 deg this is 0.002 deg of missing data, or about 1% error 
in integrated intensity.  This is fine for most applications.  But if 
you are turning at 25 deg/s with a room-temperature crystal of mosaicity 
0.05 deg, then you could loose the spot entirely in a 2 ms read-out gap 
(100% error).  This is one of several arguments for fine phi slicing, 
where you make sure that every spot is not just observed, but split over 
2-3 images.  This also helps the pile-up correction Gerd already 
mentioned.  What is often overlooked, however, is that the error due to 
read-out gap is only relevant to partials.  Fulls don't experience it at 
all, so wide phi slicing is practically immune to it.  But with fine phi 
slicing everything is a partial, and 100% of the spots are going to take 
on read-out-gap error. So, what is the solution?  Slow down.


The problem with slowing down the spindle, of course, is radiation 
damage.  If you've got a flux of 1e12 photons/s into a 100x100 micron 
beam spot and 1 A wavelength you are dosing metal-free protein crystals 
at about 50 kGy/s.  Most room-temperature crystals can endure no more 
than 200 kGy, so they will live for about 4 seconds in this beam.  A 
detector framing at 25 Hz will only get 100 images, no matter what the 
spindle speed. The decision then: is it better to get 100 deg at 1 
deg/image?  or 2.5 deg with fine phi slicing?   That is, if the mosaic 
spread is 0.05 deg, you can do no more than 0.025 deg/image and still 
barely qualify as "fine phi slicing". The 25 Hz framing rate dictates no 
less than 40 ms exposures, and that means turning the spindle at (0.025 
deg/ 0.04 s) = 0.625 deg/s. Thus, we cover 2.5 deg in the 4 seconds 
before the crystal dies.  That's just algebra.  The pragmatic 
consequence is the difference between getting a complete dataset from 
one crystal and needing to merge 40 crystals.


Of course, you can attenuate 40x and get 100 fine-sliced degrees, but 
that will take 40x more beam time.  The images will also be 40x weaker.  
In my experience you need at least an average of 1 photon/pixel/image 
before even the best data processing algorithms start to fall over.  You 
can actually calculate photons/pixel/image beforehand if you know your 
flux and how thick your sample is:


photons/pixel = 1.2e-5*flux*exposure*thickness/pixels

where flux is in photons/s, exposure in seconds, thickness in microns 
and 1.2e-5 comes from the NIST elastic scattering cross section of light 
atoms (C, N, O are all ~ 0.2 cm^2/g), the rough density of protein 
crystals (1.2 g/cm^3), and the fact that about half of all scattered 
photons land on a flat detector at typical distance from the sample, or: 
0.2*1.2*(1e-4 cm/um)/2 = 1.155e-5


So, if your flux is 1e12 photons/s and your sample is 100 um thick you 
will get ~1 photon/pixel on a 6M in about 5 ms.  That corresponds to a 
framing rate of 200 Hz.  If the detector can't go that fast, you need to 
attenuate.  Note this is the total sample thickness, including the stuff 
around the crystal.  Air scatter counts as 1 micron of sample thickness 
for every mm of air between the beamstop and collimator.  So, in a way, 
the beam properties and detector properties can be matched.  What is 
counter-intuitive is that a 25 Hz detector is sub-optimal for a 100 
micron beam with flux 1e12 photons/s.  Certain fluxaholics would already 
call that a "weak" beam, so why does getting a faster detector mean you 
should attenuate it?


It would seem that fine slicing requires throwing away a lot of beam, 
unless you only want a few degrees from every crystal. Maybe there is a 
better way?


I have experimented with 1-deg images with no attenuation and room 
temperature crystals and the results are surprisingly good. 

Re: [ccp4bb] Fine Phi Slicing

[Diana Tomchick]

And the instrument makers (e.g., Rigaku and Bruker) currently sell multi-axis 
goniostats for in-house data collection--and not just for small-molecule 
purposes.

Diana

P.S. Ron, we still call 'em goniometer heads and goniostats at UT Southwestern.

> On Jul 14, 2017, at 1:00 PM, "stenk...@u.washington.edu" 
>  wrote:
>
> I'm not understanding much of this discussion, but I've noticed the use of 
> words reminiscent of old issues concerning omega-two theta scans on 
> four-circle goniostats (see Stout and Jensen, pages 168-173, second edition).
>
> p.s.  In my upbringing, crystals were placed on goniometer heads so they 
> could be placed on goniometers or goniostats.
>
>> On Fri, 14 Jul 2017, Harry Powell wrote:
>>
>> hi folks
>> Just my two ha'porth - the small molecule crystallographers have been doing 
>> multi-orientation data collections
>> since they moved from point detectors to area detectors in the early 1990's, 
>> for the very reasons that Gerard
>> gives (their cusps are huge compared to ours...). Since they were perfectly 
>> used to using multi-axis goniostats*,
>> this wasn't a big psychological jump for them.
>> * I prefer calling the thing that the crystals sit on a "goniostat" because 
>> a "goniometer" is correctly something
>> for measuring angles (however, a "positioning goniometer" appears to be a 
>> specialised kind of goniostat);
>> wikipedia tells me that crystallographers seem to be the only group of 
>> people who confuse the two (but I didn't
>> read the article very carefully so IMWBW).
>> On 14 Jul 2017, at 15:15, Gerard Bricogne wrote:
>>  Dear Leo,
>>  What seems to have happened is that an existing thread where fine
>>  phi (actually: omega!) slicing was discussed, among many other things,
>>  digressed into a discussion of data collection protocols using more
>>  than one instrumental setting (either using a 2-theta motion of the
>>  detector, or a chi reorientation of the crystal). Briefly, my two
>>  cents on that topic: a 2-theta movement may help use different pixels
>>  on the detector, and could be valuable in filling the wide horizontal
>>  gaps on a Pilatus or Eiger, but it will leave the cusp in the same
>>  place and therefore will not fill it. Reorienting the crystal, on the
>>  other hand, can help cure all the known ills of single-sweep datasets
>>  (gaps and cusp in particular).
>>  On the matter of multi-orientation data collection, the idea and
>>  the practice go back (at least, in my memory) to Alan Wonacott, the
>>  co-creator of the Arndt-Wonacott rotation camera in the early 1970's.
>>  It was all done with gonio arcs. As each crystal had to be aligned
>>  manually in order to continue data collection where the previous one
>>  had left off, these arcs were in constant use, and there was always an
>>  extra cusp-filling collection at the end. Nowadays data collection has
>>  speeded up so much, and has become so dominated by automation, that
>>  multi-axis goniometry has been sidestepped because using it properly
>>  would have had to involve non-automated steps that are difficult to
>>  standardise (a notable exception being the protocol with 8 different
>>  values of Chi, using the PRIGo goniometer on the PX-III beamline at
>>  the SLS, that has been "instrumental" in enabling large structures to
>>  be experimentally phased by native SAD at 6keV).
>>  It is great to see that there are many developments underway in
>>  both hardware and software, leading gradually towards a reinstatement
>>  of multi-orientation data collection as an off-the-shelf option for
>>  those who are prepared to spend a bit more time to reliably get much
>>  better data. The Proxima-1 beamline scientists at SOLEIL have always
>>  been among the believers that the time would come when these efforts
>>  would bear fruit, and what my group has been able to do in this area
>>  owes a great deal to them.
>>  With best wishes,
>>   Gerard.
>>  --
>>  On Fri, Jul 14, 2017 at 01:18:35PM +, CHAVAS Leonard wrote:
>>Reading back my email, when I mentioned 'just introduced', it is 
>> not giving justice to
>>the reality and those who came up with the concept. I should have 
>> mentioned 'just
>>reminded us', as the concept has been introduced quite a long 
>> time ago and few tens of
>>communications. It is therefore a reminder that when coming to 
>> the will to collect good,
>>clean and complete data, things aren't as simple as they would 
>> seem. Automation at our
>>favourite beamlines do help by providing much more time thinking 
>> properly of the
>>necessary strategies when coming to these difficult crystals so 
>> important to our hearts.
>>Sorry again for the confusion. No 

Re: [ccp4bb] Fine Phi Slicing

[stenkamp]

I'm not understanding much of this discussion, but I've noticed the use of 
words reminiscent of old issues concerning omega-two theta scans on four-circle 
goniostats (see Stout and Jensen, pages 168-173, second edition).

p.s.  In my upbringing, crystals were placed on goniometer heads so they could 
be placed on goniometers or goniostats.

On Fri, 14 Jul 2017, Harry Powell wrote:


hi folks
Just my two ha'porth - the small molecule crystallographers have been doing 
multi-orientation data collections
since they moved from point detectors to area detectors in the early 1990's, 
for the very reasons that Gerard
gives (their cusps are huge compared to ours...). Since they were perfectly 
used to using multi-axis goniostats*,
this wasn't a big psychological jump for them.

* I prefer calling the thing that the crystals sit on a "goniostat" because a 
"goniometer" is correctly something
for measuring angles (however, a "positioning goniometer" appears to be a 
specialised kind of goniostat);
wikipedia tells me that crystallographers seem to be the only group of people 
who confuse the two (but I didn't
read the article very carefully so IMWBW).


On 14 Jul 2017, at 15:15, Gerard Bricogne wrote:

  Dear Leo,

  What seems to have happened is that an existing thread where fine
  phi (actually: omega!) slicing was discussed, among many other things,
  digressed into a discussion of data collection protocols using more
  than one instrumental setting (either using a 2-theta motion of the
  detector, or a chi reorientation of the crystal). Briefly, my two
  cents on that topic: a 2-theta movement may help use different pixels
  on the detector, and could be valuable in filling the wide horizontal
  gaps on a Pilatus or Eiger, but it will leave the cusp in the same
  place and therefore will not fill it. Reorienting the crystal, on the
  other hand, can help cure all the known ills of single-sweep datasets
  (gaps and cusp in particular).

  On the matter of multi-orientation data collection, the idea and
  the practice go back (at least, in my memory) to Alan Wonacott, the
  co-creator of the Arndt-Wonacott rotation camera in the early 1970's.
  It was all done with gonio arcs. As each crystal had to be aligned
  manually in order to continue data collection where the previous one
  had left off, these arcs were in constant use, and there was always an
  extra cusp-filling collection at the end. Nowadays data collection has
  speeded up so much, and has become so dominated by automation, that
  multi-axis goniometry has been sidestepped because using it properly
  would have had to involve non-automated steps that are difficult to
  standardise (a notable exception being the protocol with 8 different
  values of Chi, using the PRIGo goniometer on the PX-III beamline at
  the SLS, that has been "instrumental" in enabling large structures to
  be experimentally phased by native SAD at 6keV).

  It is great to see that there are many developments underway in
  both hardware and software, leading gradually towards a reinstatement
  of multi-orientation data collection as an off-the-shelf option for
  those who are prepared to spend a bit more time to reliably get much
  better data. The Proxima-1 beamline scientists at SOLEIL have always
  been among the believers that the time would come when these efforts
  would bear fruit, and what my group has been able to do in this area
  owes a great deal to them.


  With best wishes,

   Gerard.

  --
  On Fri, Jul 14, 2017 at 01:18:35PM +, CHAVAS Leonard wrote:
Reading back my email, when I mentioned 'just introduced', it is 
not giving justice to
the reality and those who came up with the concept. I should have 
mentioned 'just
reminded us', as the concept has been introduced quite a long time 
ago and few tens of
communications. It is therefore a reminder that when coming to the 
will to collect good,
clean and complete data, things aren't as simple as they would 
seem. Automation at our
favourite beamlines do help by providing much more time thinking 
properly of the
necessary strategies when coming to these difficult crystals so 
important to our hearts.


Sorry again for the confusion. No hurt feelings I hope.

Cheers, leo


-

Leonard Chavas

-

Synchrotron SOLEIL

Proxima-I

L'Orme des Merisiers

Saint-Aubin - BP 48

91192 Gif-sur-Yvette Cedex

France

-

Phone:  +33 169 359 746

Mobile: +33 644 321 614

E-mail: leonard.cha...@synchrotron-soleil.fr

-


  On 14 Jul 2017, at 14:07, CHAVAS Leonard
  

Re: [ccp4bb] Fine Phi Slicing

[Harry Powell]

hi folks

Just my two ha'porth - the small molecule crystallographers have been doing 
multi-orientation data collections since they moved from point detectors to 
area detectors in the early 1990's, for the very reasons that Gerard gives 
(their cusps are huge compared to ours...). Since they were perfectly used to 
using multi-axis goniostats*, this wasn't a big psychological jump for them.

* I prefer calling the thing that the crystals sit on a "goniostat" because a 
"goniometer" is correctly something for measuring angles (however, a 
"positioning goniometer" appears to be a specialised kind of goniostat); 
wikipedia tells me that crystallographers seem to be the only group of people 
who confuse the two (but I didn't read the article very carefully so IMWBW).


On 14 Jul 2017, at 15:15, Gerard Bricogne wrote:

> Dear Leo,
> 
> What seems to have happened is that an existing thread where fine
> phi (actually: omega!) slicing was discussed, among many other things,
> digressed into a discussion of data collection protocols using more
> than one instrumental setting (either using a 2-theta motion of the
> detector, or a chi reorientation of the crystal). Briefly, my two
> cents on that topic: a 2-theta movement may help use different pixels
> on the detector, and could be valuable in filling the wide horizontal
> gaps on a Pilatus or Eiger, but it will leave the cusp in the same
> place and therefore will not fill it. Reorienting the crystal, on the
> other hand, can help cure all the known ills of single-sweep datasets
> (gaps and cusp in particular).
> 
> On the matter of multi-orientation data collection, the idea and
> the practice go back (at least, in my memory) to Alan Wonacott, the
> co-creator of the Arndt-Wonacott rotation camera in the early 1970's.
> It was all done with gonio arcs. As each crystal had to be aligned
> manually in order to continue data collection where the previous one
> had left off, these arcs were in constant use, and there was always an
> extra cusp-filling collection at the end. Nowadays data collection has
> speeded up so much, and has become so dominated by automation, that
> multi-axis goniometry has been sidestepped because using it properly
> would have had to involve non-automated steps that are difficult to
> standardise (a notable exception being the protocol with 8 different
> values of Chi, using the PRIGo goniometer on the PX-III beamline at
> the SLS, that has been "instrumental" in enabling large structures to
> be experimentally phased by native SAD at 6keV).
> 
> It is great to see that there are many developments underway in
> both hardware and software, leading gradually towards a reinstatement
> of multi-orientation data collection as an off-the-shelf option for
> those who are prepared to spend a bit more time to reliably get much
> better data. The Proxima-1 beamline scientists at SOLEIL have always
> been among the believers that the time would come when these efforts
> would bear fruit, and what my group has been able to do in this area
> owes a great deal to them.
> 
> 
> With best wishes,
> 
>  Gerard.
> 
> --
> On Fri, Jul 14, 2017 at 01:18:35PM +, CHAVAS Leonard wrote:
>> Reading back my email, when I mentioned 'just introduced', it is not giving 
>> justice to the reality and those who came up with the concept. I should have 
>> mentioned 'just reminded us', as the concept has been introduced quite a 
>> long time ago and few tens of communications. It is therefore a reminder 
>> that when coming to the will to collect good, clean and complete data, 
>> things aren't as simple as they would seem. Automation at our favourite 
>> beamlines do help by providing much more time thinking properly of the 
>> necessary strategies when coming to these difficult crystals so important to 
>> our hearts.
>> 
>> Sorry again for the confusion. No hurt feelings I hope.
>> Cheers, leo
>> 
>> -
>> Leonard Chavas
>> - 
>> Synchrotron SOLEIL
>> Proxima-I
>> L'Orme des Merisiers
>> Saint-Aubin - BP 48
>> 91192 Gif-sur-Yvette Cedex
>> France
>> - 
>> Phone:  +33 169 359 746
>> Mobile: +33 644 321 614
>> E-mail: leonard.cha...@synchrotron-soleil.fr
>> -
>> 
>>> On 14 Jul 2017, at 14:07, CHAVAS Leonard 
>>>  wrote:
>>> 
>>> Just to comment on what Graeme just introduced. We (and I know we are not 
>>> the first ones and not the only ones) are pushing our user community 
>>> towards this procedure as a standard: lowering the transmission (less 
>>> juicy, yet...) and getting few data with various chi. It does help greatly 
>>> in getting fully complete data, with no loss in resolution. Just fantastic!
>>> 
>>> Cheers, leo
>>> 
>>> -
>>> Leonard Chavas
>>> - 
>>> Synchrotron SOLEIL
>>> Proxima-I
>>> L'Orme des Merisiers
>>> Saint-Aubin - BP 48
>>> 91192 Gif-sur-Yvette Cedex
>>> France
>>> - 
>>> Phone:  +33 169 359 746
>>> Mobile: +33 644 321 614
>>> E-mail: leonard.cha...@synchrotron-soleil.fr
>>> -

Re: [ccp4bb] Fine Phi Slicing

[Gerard Bricogne]

Dear Leo,

 What seems to have happened is that an existing thread where fine
phi (actually: omega!) slicing was discussed, among many other things,
digressed into a discussion of data collection protocols using more
than one instrumental setting (either using a 2-theta motion of the
detector, or a chi reorientation of the crystal). Briefly, my two
cents on that topic: a 2-theta movement may help use different pixels
on the detector, and could be valuable in filling the wide horizontal
gaps on a Pilatus or Eiger, but it will leave the cusp in the same
place and therefore will not fill it. Reorienting the crystal, on the
other hand, can help cure all the known ills of single-sweep datasets
(gaps and cusp in particular).

 On the matter of multi-orientation data collection, the idea and
the practice go back (at least, in my memory) to Alan Wonacott, the
co-creator of the Arndt-Wonacott rotation camera in the early 1970's.
It was all done with gonio arcs. As each crystal had to be aligned
manually in order to continue data collection where the previous one
had left off, these arcs were in constant use, and there was always an
extra cusp-filling collection at the end. Nowadays data collection has
speeded up so much, and has become so dominated by automation, that
multi-axis goniometry has been sidestepped because using it properly
would have had to involve non-automated steps that are difficult to
standardise (a notable exception being the protocol with 8 different
values of Chi, using the PRIGo goniometer on the PX-III beamline at
the SLS, that has been "instrumental" in enabling large structures to
be experimentally phased by native SAD at 6keV).

 It is great to see that there are many developments underway in
both hardware and software, leading gradually towards a reinstatement
of multi-orientation data collection as an off-the-shelf option for
those who are prepared to spend a bit more time to reliably get much
better data. The Proxima-1 beamline scientists at SOLEIL have always
been among the believers that the time would come when these efforts
would bear fruit, and what my group has been able to do in this area
owes a great deal to them.


 With best wishes,
 
  Gerard.

--
On Fri, Jul 14, 2017 at 01:18:35PM +, CHAVAS Leonard wrote:
> Reading back my email, when I mentioned 'just introduced', it is not giving 
> justice to the reality and those who came up with the concept. I should have 
> mentioned 'just reminded us', as the concept has been introduced quite a long 
> time ago and few tens of communications. It is therefore a reminder that when 
> coming to the will to collect good, clean and complete data, things aren't as 
> simple as they would seem. Automation at our favourite beamlines do help by 
> providing much more time thinking properly of the necessary strategies when 
> coming to these difficult crystals so important to our hearts.
> 
> Sorry again for the confusion. No hurt feelings I hope.
> Cheers, leo
> 
> -
> Leonard Chavas
> - 
> Synchrotron SOLEIL
> Proxima-I
> L'Orme des Merisiers
> Saint-Aubin - BP 48
> 91192 Gif-sur-Yvette Cedex
> France
> - 
> Phone:  +33 169 359 746
> Mobile: +33 644 321 614
> E-mail: leonard.cha...@synchrotron-soleil.fr
> -
> 
> > On 14 Jul 2017, at 14:07, CHAVAS Leonard 
> >  wrote:
> > 
> > Just to comment on what Graeme just introduced. We (and I know we are not 
> > the first ones and not the only ones) are pushing our user community 
> > towards this procedure as a standard: lowering the transmission (less 
> > juicy, yet...) and getting few data with various chi. It does help greatly 
> > in getting fully complete data, with no loss in resolution. Just fantastic!
> > 
> > Cheers, leo
> > 
> > -
> > Leonard Chavas
> > - 
> > Synchrotron SOLEIL
> > Proxima-I
> > L'Orme des Merisiers
> > Saint-Aubin - BP 48
> > 91192 Gif-sur-Yvette Cedex
> > France
> > - 
> > Phone:  +33 169 359 746
> > Mobile: +33 644 321 614
> > E-mail: leonard.cha...@synchrotron-soleil.fr
> > -
> > 
> >> On 14 Jul 2017, at 07:36, Graeme Winter  
> >> wrote:
> >> 
> >> Jacob
> >> 
> >> If you have a complete 360 deg data set and your sample is still alive, 
> >> and you have a multi-axis gonio, I would recommend rotating the crystal 
> >> about the beam (ideally by ~ maximum scattering 2-theta angle) and 
> >> collecting again. This would record your blind region as well as moving 
> >> the reflections to different pixels, and (as a bonus) also will move 
> >> reflections out from the tile join regions into somewhere they can be 
> >> measured, which would not happen for small 2-theta shift.
> >> 
> >> See http://scripts.iucr.org/cgi-bin/paper?BA0020 Figure 16 as excellent 
> >> illustration of this.
> >> 
> >> Biggest risk with this is getting *moving* shadows on the data on the 
> >> second run, as an effective 45-50 degree chi shift (say) will usually be a 
> >> pretty wide opening 

Re: [ccp4bb] Fine Phi Slicing

[CHAVAS Leonard]

Reading back my email, when I mentioned 'just introduced', it is not giving 
justice to the reality and those who came up with the concept. I should have 
mentioned 'just reminded us', as the concept has been introduced quite a long 
time ago and few tens of communications. It is therefore a reminder that when 
coming to the will to collect good, clean and complete data, things aren't as 
simple as they would seem. Automation at our favourite beamlines do help by 
providing much more time thinking properly of the necessary strategies when 
coming to these difficult crystals so important to our hearts.

Sorry again for the confusion. No hurt feelings I hope.
Cheers, leo

-
Leonard Chavas
- 
Synchrotron SOLEIL
Proxima-I
L'Orme des Merisiers
Saint-Aubin - BP 48
91192 Gif-sur-Yvette Cedex
France
- 
Phone:  +33 169 359 746
Mobile: +33 644 321 614
E-mail: leonard.cha...@synchrotron-soleil.fr
-

> On 14 Jul 2017, at 14:07, CHAVAS Leonard 
>  wrote:
> 
> Just to comment on what Graeme just introduced. We (and I know we are not the 
> first ones and not the only ones) are pushing our user community towards this 
> procedure as a standard: lowering the transmission (less juicy, yet...) and 
> getting few data with various chi. It does help greatly in getting fully 
> complete data, with no loss in resolution. Just fantastic!
> 
> Cheers, leo
> 
> -
> Leonard Chavas
> - 
> Synchrotron SOLEIL
> Proxima-I
> L'Orme des Merisiers
> Saint-Aubin - BP 48
> 91192 Gif-sur-Yvette Cedex
> France
> - 
> Phone:  +33 169 359 746
> Mobile: +33 644 321 614
> E-mail: leonard.cha...@synchrotron-soleil.fr
> -
> 
>> On 14 Jul 2017, at 07:36, Graeme Winter  wrote:
>> 
>> Jacob
>> 
>> If you have a complete 360 deg data set and your sample is still alive, and 
>> you have a multi-axis gonio, I would recommend rotating the crystal about 
>> the beam (ideally by ~ maximum scattering 2-theta angle) and collecting 
>> again. This would record your blind region as well as moving the reflections 
>> to different pixels, and (as a bonus) also will move reflections out from 
>> the tile join regions into somewhere they can be measured, which would not 
>> happen for small 2-theta shift.
>> 
>> See http://scripts.iucr.org/cgi-bin/paper?BA0020 Figure 16 as excellent 
>> illustration of this.
>> 
>> Biggest risk with this is getting *moving* shadows on the data on the second 
>> run, as an effective 45-50 degree chi shift (say) will usually be a pretty 
>> wide opening angle for a kappa gonio. XDS and DIALS both have mechanisms to 
>> deal with this, and automated processing packages are able to apply these 
>> given a reasonable understanding of the beamline.
>> 
>> Also saves building 2-theta axes which can handle 92 kg ;o)
>> 
>> Cheers Graeme
>> 
>> On 13 Jul 2017, at 21:00, Keller, Jacob 
>> > wrote:
>> 
>> I thought there was a new paper from the Pilatus people saying fine slicing 
>> is worth it even beyond the original 1/2 mosaicity rule?
>> 
>> I would think, actually, more gains would made by doing light exposures at, 
>> say, 1/3 mosaicity, collecting 360 deg, then shifting the detector in 2theta 
>> by a degree or two to shift uniformly the spots to new pixels, maybe 
>> accompanied by a kappa change. One would have to remember about the 
>> two-theta when processing, however!
>> 
>> JPK
>> 
>> -Original Message-
>> From: CCP4 bulletin board [mailto:CCP4BB@JISCMAIL.AC.UK] On Behalf Of Gerd 
>> Rosenbaum
>> Sent: Thursday, July 13, 2017 3:40 PM
>> To: CCP4BB@JISCMAIL.AC.UK
>> Subject: Re: [ccp4bb] weird diffraction pattern
>> 
>> Dear Gerard,
>> 
>>  my "sound like a sales person" was meant as poking a little fun - nothing 
>> serious, of course.
>> 
>> I and our users like our not-so-new-anymore Pilatus3 6M. It's a great 
>> detector in many ways. But, there is a lot of hype that this detector solves 
>> all-problem, for instance fine slicing that is claimed to be only possible 
>> with a pixel array detector. People get carried away and use
>> 0.01 degree slices even as the mosaicity of their sample is, say, 0.3 
>> degree. Slicing beyond 1/3 of the mosaicity will gain you very little - only 
>> more frames, more processing time.
>> 
>> This discourse is already drifting away from the original topic of the 
>> thread so I will comment on the other arguments  you made like resolution in 
>> a private e-mail.
>> 
>> Best regards,
>> 
>> Gerd
>> 
>> On 13.07.2017 14:00, Gerard Bricogne wrote:
>> Dear Gerd,
>> 
>>I can assure you that I have no shares in Dectris nor any
>> commecial connections with them. What I do have is a lot of still
>> vivid memories of CCD images, with their wooly point-spread function
>> that was affected by fine-grained spatial variability as well as by
>> irredicible inaccuracies in the geometric corrections required to try
>> and undo the 

Re: [ccp4bb] Fine Phi Slicing

[CHAVAS Leonard]

Attenuation... cut the beam with primary slits! We do not use attenuators, only 
for getting very very low when performing energy scans. Else, cutting the flux 
at the source is somehow much more reliable. Not mentioning scattering coming 
from the attenuators as well...

Cheers, leo

-
Leonard Chavas
- 
Synchrotron SOLEIL
Proxima-I
L'Orme des Merisiers
Saint-Aubin - BP 48
91192 Gif-sur-Yvette Cedex
France
- 
Phone:  +33 169 359 746
Mobile: +33 644 321 614
E-mail: leonard.cha...@synchrotron-soleil.fr
-

> On 14 Jul 2017, at 13:02, Keller, Jacob <kell...@janelia.hhmi.org> wrote:
> 
> Hi Graeme,
> 
> I see your point about the blind region and also the tile lines. But 2-theta 
> would have the advantage of also shifting the low-res spots to entirely new 
> pixels, which would be harder through rotation. Also, wouldn't rotating about 
> the beam axis shift the spots to variable degrees across rotation space, with 
> some angles (+/- 90 deg) negligibly shifted?
> 
> Further, does it give anyone pause: Graeme makes a subtle implication that 
> most samples die before collecting 360 degrees, which I think may be true. 
> What can be done about this tragic lack of attenuation? One possibility is to 
> model the radiation damage in refinement, but wouldn't it make a lot more 
> sense to have a lot of good attenuators installed by default (or use 
> sealed-tube sources!).
> 
> JPK
> 
> 
> 
> -Original Message-
> From: graeme.win...@diamond.ac.uk [mailto:graeme.win...@diamond.ac.uk] 
> Sent: Friday, July 14, 2017 1:37 AM
> To: Keller, Jacob <kell...@janelia.hhmi.org>
> Cc: ccp4bb@jiscmail.ac.uk
> Subject: Re: [ccp4bb] Fine Phi Slicing
> 
> Jacob
> 
> If you have a complete 360 deg data set and your sample is still alive, and 
> you have a multi-axis gonio, I would recommend rotating the crystal about the 
> beam (ideally by ~ maximum scattering 2-theta angle) and collecting again. 
> This would record your blind region as well as moving the reflections to 
> different pixels, and (as a bonus) also will move reflections out from the 
> tile join regions into somewhere they can be measured, which would not happen 
> for small 2-theta shift.
> 
> See http://scripts.iucr.org/cgi-bin/paper?BA0020 Figure 16 as excellent 
> illustration of this.
> 
> Biggest risk with this is getting *moving* shadows on the data on the second 
> run, as an effective 45-50 degree chi shift (say) will usually be a pretty 
> wide opening angle for a kappa gonio. XDS and DIALS both have mechanisms to 
> deal with this, and automated processing packages are able to apply these 
> given a reasonable understanding of the beamline.
> 
> Also saves building 2-theta axes which can handle 92 kg ;o)
> 
> Cheers Graeme
> 
> On 13 Jul 2017, at 21:00, Keller, Jacob 
> <kell...@janelia.hhmi.org<mailto:kell...@janelia.hhmi.org>> wrote:
> 
> I thought there was a new paper from the Pilatus people saying fine slicing 
> is worth it even beyond the original 1/2 mosaicity rule?
> 
> I would think, actually, more gains would made by doing light exposures at, 
> say, 1/3 mosaicity, collecting 360 deg, then shifting the detector in 2theta 
> by a degree or two to shift uniformly the spots to new pixels, maybe 
> accompanied by a kappa change. One would have to remember about the two-theta 
> when processing, however!
> 
> JPK
> 
> -Original Message-
> From: CCP4 bulletin board [mailto:CCP4BB@JISCMAIL.AC.UK] On Behalf Of Gerd 
> Rosenbaum
> Sent: Thursday, July 13, 2017 3:40 PM
> To: CCP4BB@JISCMAIL.AC.UK<mailto:CCP4BB@JISCMAIL.AC.UK>
> Subject: Re: [ccp4bb] weird diffraction pattern
> 
> Dear Gerard,
> 
>   my "sound like a sales person" was meant as poking a little fun - nothing 
> serious, of course.
> 
> I and our users like our not-so-new-anymore Pilatus3 6M. It's a great 
> detector in many ways. But, there is a lot of hype that this detector solves 
> all-problem, for instance fine slicing that is claimed to be only possible 
> with a pixel array detector. People get carried away and use
> 0.01 degree slices even as the mosaicity of their sample is, say, 0.3 degree. 
> Slicing beyond 1/3 of the mosaicity will gain you very little - only more 
> frames, more processing time.
> 
> This discourse is already drifting away from the original topic of the thread 
> so I will comment on the other arguments  you made like resolution in a 
> private e-mail.
> 
> Best regards,
> 
> Gerd
> 
> On 13.07.2017 14:00, Gerard Bricogne wrote:
> Dear Gerd,
> 
> I can assure you that I have no shares in Dectris nor any commecial 
> connections with them. What I do have is a lot of still vivid memories of CCD 
> images,

Re: [ccp4bb] Fine Phi Slicing

[CHAVAS Leonard]

Just to comment on what Graeme just introduced. We (and I know we are not the 
first ones and not the only ones) are pushing our user community towards this 
procedure as a standard: lowering the transmission (less juicy, yet...) and 
getting few data with various chi. It does help greatly in getting fully 
complete data, with no loss in resolution. Just fantastic!

Cheers, leo

-
Leonard Chavas
- 
Synchrotron SOLEIL
Proxima-I
L'Orme des Merisiers
Saint-Aubin - BP 48
91192 Gif-sur-Yvette Cedex
France
- 
Phone:  +33 169 359 746
Mobile: +33 644 321 614
E-mail: leonard.cha...@synchrotron-soleil.fr
-

> On 14 Jul 2017, at 07:36, Graeme Winter  wrote:
> 
> Jacob
> 
> If you have a complete 360 deg data set and your sample is still alive, and 
> you have a multi-axis gonio, I would recommend rotating the crystal about the 
> beam (ideally by ~ maximum scattering 2-theta angle) and collecting again. 
> This would record your blind region as well as moving the reflections to 
> different pixels, and (as a bonus) also will move reflections out from the 
> tile join regions into somewhere they can be measured, which would not happen 
> for small 2-theta shift.
> 
> See http://scripts.iucr.org/cgi-bin/paper?BA0020 Figure 16 as excellent 
> illustration of this.
> 
> Biggest risk with this is getting *moving* shadows on the data on the second 
> run, as an effective 45-50 degree chi shift (say) will usually be a pretty 
> wide opening angle for a kappa gonio. XDS and DIALS both have mechanisms to 
> deal with this, and automated processing packages are able to apply these 
> given a reasonable understanding of the beamline.
> 
> Also saves building 2-theta axes which can handle 92 kg ;o)
> 
> Cheers Graeme
> 
> On 13 Jul 2017, at 21:00, Keller, Jacob 
> > wrote:
> 
> I thought there was a new paper from the Pilatus people saying fine slicing 
> is worth it even beyond the original 1/2 mosaicity rule?
> 
> I would think, actually, more gains would made by doing light exposures at, 
> say, 1/3 mosaicity, collecting 360 deg, then shifting the detector in 2theta 
> by a degree or two to shift uniformly the spots to new pixels, maybe 
> accompanied by a kappa change. One would have to remember about the two-theta 
> when processing, however!
> 
> JPK
> 
> -Original Message-
> From: CCP4 bulletin board [mailto:CCP4BB@JISCMAIL.AC.UK] On Behalf Of Gerd 
> Rosenbaum
> Sent: Thursday, July 13, 2017 3:40 PM
> To: CCP4BB@JISCMAIL.AC.UK
> Subject: Re: [ccp4bb] weird diffraction pattern
> 
> Dear Gerard,
> 
>   my "sound like a sales person" was meant as poking a little fun - nothing 
> serious, of course.
> 
> I and our users like our not-so-new-anymore Pilatus3 6M. It's a great 
> detector in many ways. But, there is a lot of hype that this detector solves 
> all-problem, for instance fine slicing that is claimed to be only possible 
> with a pixel array detector. People get carried away and use
> 0.01 degree slices even as the mosaicity of their sample is, say, 0.3 degree. 
> Slicing beyond 1/3 of the mosaicity will gain you very little - only more 
> frames, more processing time.
> 
> This discourse is already drifting away from the original topic of the thread 
> so I will comment on the other arguments  you made like resolution in a 
> private e-mail.
> 
> Best regards,
> 
> Gerd
> 
> On 13.07.2017 14:00, Gerard Bricogne wrote:
> Dear Gerd,
> 
> I can assure you that I have no shares in Dectris nor any
> commecial connections with them. What I do have is a lot of still
> vivid memories of CCD images, with their wooly point-spread function
> that was affected by fine-grained spatial variability as well as by
> irredicible inaccuracies in the geometric corrections required to try
> and undo the distortions introduced by the fiber-optic taper. By
> comparison the pixel-array detectors have a very regular structure, so
> that slight deviations from exact registering of the modules can be
> calibrated with high accuracy, making it possible to get very small
> residuals between calculated and observed spot positions. That, I
> certainly never saw with CCD images.
> 
> I do think that asking for the image width was a highly
> pertinent question in this case, that had not been asked. As a
> specialist you might know how to use a CCD to good effect in
> fine-slicing mode, but it is amazing how many people there are still
> out there who are told to use 0.5 or even 1.0 degree image widths.
> 
> Compensating the poor PSF of a CCD by fine slicing in the
> angular dimension is a tall order. With a Pilatus at 350mm from the
> crystal, the angular separation between 174-micron pixels is 0.5 milliradian.
> To achieve that separation in the angular (rotation) dimension, the
> equivalent image width would have to be 0.03 degree. For an EIGER the
> numbers become 75 microns, hence 0.21 milliradian i.e. 0.012 

Re: [ccp4bb] Fine Phi Slicing

[Keller, Jacob]

Hi Graeme,

I see your point about the blind region and also the tile lines. But 2-theta 
would have the advantage of also shifting the low-res spots to entirely new 
pixels, which would be harder through rotation. Also, wouldn't rotating about 
the beam axis shift the spots to variable degrees across rotation space, with 
some angles (+/- 90 deg) negligibly shifted?

Further, does it give anyone pause: Graeme makes a subtle implication that most 
samples die before collecting 360 degrees, which I think may be true. What can 
be done about this tragic lack of attenuation? One possibility is to model the 
radiation damage in refinement, but wouldn't it make a lot more sense to have a 
lot of good attenuators installed by default (or use sealed-tube sources!).

JPK



-Original Message-
From: graeme.win...@diamond.ac.uk [mailto:graeme.win...@diamond.ac.uk] 
Sent: Friday, July 14, 2017 1:37 AM
To: Keller, Jacob <kell...@janelia.hhmi.org>
Cc: ccp4bb@jiscmail.ac.uk
Subject: Re: [ccp4bb] Fine Phi Slicing

Jacob

If you have a complete 360 deg data set and your sample is still alive, and you 
have a multi-axis gonio, I would recommend rotating the crystal about the beam 
(ideally by ~ maximum scattering 2-theta angle) and collecting again. This 
would record your blind region as well as moving the reflections to different 
pixels, and (as a bonus) also will move reflections out from the tile join 
regions into somewhere they can be measured, which would not happen for small 
2-theta shift.

See http://scripts.iucr.org/cgi-bin/paper?BA0020 Figure 16 as excellent 
illustration of this.

Biggest risk with this is getting *moving* shadows on the data on the second 
run, as an effective 45-50 degree chi shift (say) will usually be a pretty wide 
opening angle for a kappa gonio. XDS and DIALS both have mechanisms to deal 
with this, and automated processing packages are able to apply these given a 
reasonable understanding of the beamline.

Also saves building 2-theta axes which can handle 92 kg ;o)

Cheers Graeme

On 13 Jul 2017, at 21:00, Keller, Jacob 
<kell...@janelia.hhmi.org<mailto:kell...@janelia.hhmi.org>> wrote:

I thought there was a new paper from the Pilatus people saying fine slicing is 
worth it even beyond the original 1/2 mosaicity rule?

I would think, actually, more gains would made by doing light exposures at, 
say, 1/3 mosaicity, collecting 360 deg, then shifting the detector in 2theta by 
a degree or two to shift uniformly the spots to new pixels, maybe accompanied 
by a kappa change. One would have to remember about the two-theta when 
processing, however!

JPK

-Original Message-
From: CCP4 bulletin board [mailto:CCP4BB@JISCMAIL.AC.UK] On Behalf Of Gerd 
Rosenbaum
Sent: Thursday, July 13, 2017 3:40 PM
To: CCP4BB@JISCMAIL.AC.UK<mailto:CCP4BB@JISCMAIL.AC.UK>
Subject: Re: [ccp4bb] weird diffraction pattern

Dear Gerard,

   my "sound like a sales person" was meant as poking a little fun - nothing 
serious, of course.

I and our users like our not-so-new-anymore Pilatus3 6M. It's a great detector 
in many ways. But, there is a lot of hype that this detector solves 
all-problem, for instance fine slicing that is claimed to be only possible with 
a pixel array detector. People get carried away and use
0.01 degree slices even as the mosaicity of their sample is, say, 0.3 degree. 
Slicing beyond 1/3 of the mosaicity will gain you very little - only more 
frames, more processing time.

This discourse is already drifting away from the original topic of the thread 
so I will comment on the other arguments  you made like resolution in a private 
e-mail.

Best regards,

Gerd

On 13.07.2017 14:00, Gerard Bricogne wrote:
Dear Gerd,

 I can assure you that I have no shares in Dectris nor any commecial 
connections with them. What I do have is a lot of still vivid memories of CCD 
images, with their wooly point-spread function that was affected by 
fine-grained spatial variability as well as by irredicible inaccuracies in the 
geometric corrections required to try and undo the distortions introduced by 
the fiber-optic taper. By comparison the pixel-array detectors have a very 
regular structure, so that slight deviations from exact registering of the 
modules can be calibrated with high accuracy, making it possible to get very 
small residuals between calculated and observed spot positions. That, I 
certainly never saw with CCD images.

 I do think that asking for the image width was a highly pertinent question 
in this case, that had not been asked. As a specialist you might know how to 
use a CCD to good effect in fine-slicing mode, but it is amazing how many 
people there are still out there who are told to use 0.5 or even 1.0 degree 
image widths.

 Compensating the poor PSF of a CCD by fine slicing in the angular 
dimension is a tall order. With a Pilatus at 350mm from the crystal, the 
angular separation between 174-micron pixels is 0.5 milli

Re: [ccp4bb] Fine Phi Slicing

[Graeme Winter]

Jacob

If you have a complete 360 deg data set and your sample is still alive, and you 
have a multi-axis gonio, I would recommend rotating the crystal about the beam 
(ideally by ~ maximum scattering 2-theta angle) and collecting again. This 
would record your blind region as well as moving the reflections to different 
pixels, and (as a bonus) also will move reflections out from the tile join 
regions into somewhere they can be measured, which would not happen for small 
2-theta shift.

See http://scripts.iucr.org/cgi-bin/paper?BA0020 Figure 16 as excellent 
illustration of this.

Biggest risk with this is getting *moving* shadows on the data on the second 
run, as an effective 45-50 degree chi shift (say) will usually be a pretty wide 
opening angle for a kappa gonio. XDS and DIALS both have mechanisms to deal 
with this, and automated processing packages are able to apply these given a 
reasonable understanding of the beamline.

Also saves building 2-theta axes which can handle 92 kg ;o)

Cheers Graeme

On 13 Jul 2017, at 21:00, Keller, Jacob 
> wrote:

I thought there was a new paper from the Pilatus people saying fine slicing is 
worth it even beyond the original 1/2 mosaicity rule?

I would think, actually, more gains would made by doing light exposures at, 
say, 1/3 mosaicity, collecting 360 deg, then shifting the detector in 2theta by 
a degree or two to shift uniformly the spots to new pixels, maybe accompanied 
by a kappa change. One would have to remember about the two-theta when 
processing, however!

JPK

-Original Message-
From: CCP4 bulletin board [mailto:CCP4BB@JISCMAIL.AC.UK] On Behalf Of Gerd 
Rosenbaum
Sent: Thursday, July 13, 2017 3:40 PM
To: CCP4BB@JISCMAIL.AC.UK
Subject: Re: [ccp4bb] weird diffraction pattern

Dear Gerard,

   my "sound like a sales person" was meant as poking a little fun - nothing 
serious, of course.

I and our users like our not-so-new-anymore Pilatus3 6M. It's a great detector 
in many ways. But, there is a lot of hype that this detector solves 
all-problem, for instance fine slicing that is claimed to be only possible with 
a pixel array detector. People get carried away and use
0.01 degree slices even as the mosaicity of their sample is, say, 0.3 degree. 
Slicing beyond 1/3 of the mosaicity will gain you very little - only more 
frames, more processing time.

This discourse is already drifting away from the original topic of the thread 
so I will comment on the other arguments  you made like resolution in a private 
e-mail.

Best regards,

Gerd

On 13.07.2017 14:00, Gerard Bricogne wrote:
Dear Gerd,

 I can assure you that I have no shares in Dectris nor any
commecial connections with them. What I do have is a lot of still
vivid memories of CCD images, with their wooly point-spread function
that was affected by fine-grained spatial variability as well as by
irredicible inaccuracies in the geometric corrections required to try
and undo the distortions introduced by the fiber-optic taper. By
comparison the pixel-array detectors have a very regular structure, so
that slight deviations from exact registering of the modules can be
calibrated with high accuracy, making it possible to get very small
residuals between calculated and observed spot positions. That, I
certainly never saw with CCD images.

 I do think that asking for the image width was a highly
pertinent question in this case, that had not been asked. As a
specialist you might know how to use a CCD to good effect in
fine-slicing mode, but it is amazing how many people there are still
out there who are told to use 0.5 or even 1.0 degree image widths.

 Compensating the poor PSF of a CCD by fine slicing in the
angular dimension is a tall order. With a Pilatus at 350mm from the
crystal, the angular separation between 174-micron pixels is 0.5 milliradian.
To achieve that separation in the angular (rotation) dimension, the
equivalent image width would have to be 0.03 degree. For an EIGER the
numbers become 75 microns, hence 0.21 milliradian i.e. 0.012 degree.

 Hence my advice, untainted by any commercial agenda :-) .


 With best wishes,

  Gerard.

--
On Thu, Jul 13, 2017 at 01:25:08PM -0500, Gerd Rosenbaum wrote:
Dear Gerard,

you sound like a sales person for Dectris. Fine slicing is perfectly
fine with CCD detectors - it takes a bit longer because of the step
scan instead of continuous scan. The read noise issue is often
overstated compared to the sample induced scatter background. If for
fine slicing at 0.05 degree or less the diffraction peaks go too
close to the read noise make a longer exposure - signal goes up,
ratio signal to sample-induced-BG less, as for any fine slicing, same read 
noise.

It would be helpful to analyze the dense spot packing along layer
lines if we knew the wavelength and the sample-to-detector distance
(assuming this is a 300 mm detector) and the 

Re: [ccp4bb] Fine Phi Slicing

[Gerd Rosenbaum]

Hi Fred,

fine slicing does not alleviate the count RATE limitation of photon 
counting detectors because fine slicing does not reduce the 
instantaneous photon flux on the detector when you cross the diffraction 
maximum. Fine slicing does help if you push the maximum counts per pixel 
per frame to the counter register limit. On integrating detectors, like 
CCDs, there is practically no count RATE limit. They do have a charge 
(~photons) per pixel per frame limit, as well, which is mostly much 
lower than for the photon counting detectors - about 1/10 even after 
taking into account the diffraction spot covers many more pixels.


Different from integrating detectors where you only have to watch the 
overflow, for counting detectors you have to watch for exceeding either 
the count rate limit or the total count limit. The former is not an easy 
task because you will see only the count per exposure. Divide that by 
the exposure time you get the AVERAGE rate, not the peak rate.


The count rate limit, very short exposure time (using high flux) and the 
1-pixel point spread function work against each other. Exposure time = 
0.01 s, count rate limit 1e6 /sec (Pilatus2), 1-2 pixel per spot => 
10-20k counts per spot maximum for the strongest peak.


Dectris has come up with an ingenious hardware and software in the 
Pilatus3 pushing the rate for reasonable dead time correction to over 
1e7 counts/sec so even with 10 ms exposures weak reflections can be well 
recorded besides strong reflections.


Gerd

On 13.07.2017 15:34, Dyda wrote:

I could be wrong here, but isn't the case that fine slicing is an option
with a CCD and a necessity on a PAD b/c of dead time and/or counter dynamic 
range
issues?

(no current and/or former financial ties to any manufacturer)

Fred
***
Fred Dyda, Ph.D.   Phone:301-402-4496
Laboratory of Molecular BiologyFax: 301-496-0201
DHHS/NIH/NIDDK e-mail:fred.d...@nih.gov
Bldg. 5. Room 303
Bethesda, MD 20892-0560  URGENT message e-mail: 2022476...@mms.att.net
Google maps coords: 39.000597, -77.102102
http://www2.niddk.nih.gov/NIDDKLabs/IntramuralFaculty/DydaFred
***

Re: [ccp4bb] Fine Phi Slicing

[Dyda]

I could be wrong here, but isn't the case that fine slicing is an option
with a CCD and a necessity on a PAD b/c of dead time and/or counter dynamic 
range 
issues?

(no current and/or former financial ties to any manufacturer)

Fred
***
Fred Dyda, Ph.D.   Phone:301-402-4496
Laboratory of Molecular BiologyFax: 301-496-0201
DHHS/NIH/NIDDK e-mail:fred.d...@nih.gov  
Bldg. 5. Room 303 
Bethesda, MD 20892-0560  URGENT message e-mail: 2022476...@mms.att.net
Google maps coords: 39.000597, -77.102102
http://www2.niddk.nih.gov/NIDDKLabs/IntramuralFaculty/DydaFred
***

[ccp4bb] Fine Phi Slicing

[Keller, Jacob]

I thought there was a new paper from the Pilatus people saying fine slicing is 
worth it even beyond the original 1/2 mosaicity rule?

I would think, actually, more gains would made by doing light exposures at, 
say, 1/3 mosaicity, collecting 360 deg, then shifting the detector in 2theta by 
a degree or two to shift uniformly the spots to new pixels, maybe accompanied 
by a kappa change. One would have to remember about the two-theta when 
processing, however!

JPK

-Original Message-
From: CCP4 bulletin board [mailto:CCP4BB@JISCMAIL.AC.UK] On Behalf Of Gerd 
Rosenbaum
Sent: Thursday, July 13, 2017 3:40 PM
To: CCP4BB@JISCMAIL.AC.UK
Subject: Re: [ccp4bb] weird diffraction pattern

Dear Gerard,

my "sound like a sales person" was meant as poking a little fun - nothing 
serious, of course.

I and our users like our not-so-new-anymore Pilatus3 6M. It's a great detector 
in many ways. But, there is a lot of hype that this detector solves 
all-problem, for instance fine slicing that is claimed to be only possible with 
a pixel array detector. People get carried away and use
0.01 degree slices even as the mosaicity of their sample is, say, 0.3 degree. 
Slicing beyond 1/3 of the mosaicity will gain you very little - only more 
frames, more processing time.

This discourse is already drifting away from the original topic of the thread 
so I will comment on the other arguments  you made like resolution in a private 
e-mail.

Best regards,

Gerd

On 13.07.2017 14:00, Gerard Bricogne wrote:
> Dear Gerd,
>
>   I can assure you that I have no shares in Dectris nor any 
> commecial connections with them. What I do have is a lot of still 
> vivid memories of CCD images, with their wooly point-spread function 
> that was affected by fine-grained spatial variability as well as by 
> irredicible inaccuracies in the geometric corrections required to try 
> and undo the distortions introduced by the fiber-optic taper. By 
> comparison the pixel-array detectors have a very regular structure, so 
> that slight deviations from exact registering of the modules can be 
> calibrated with high accuracy, making it possible to get very small 
> residuals between calculated and observed spot positions. That, I 
> certainly never saw with CCD images.
>
>   I do think that asking for the image width was a highly 
> pertinent question in this case, that had not been asked. As a 
> specialist you might know how to use a CCD to good effect in 
> fine-slicing mode, but it is amazing how many people there are still 
> out there who are told to use 0.5 or even 1.0 degree image widths.
>
>   Compensating the poor PSF of a CCD by fine slicing in the 
> angular dimension is a tall order. With a Pilatus at 350mm from the 
> crystal, the angular separation between 174-micron pixels is 0.5 milliradian.
> To achieve that separation in the angular (rotation) dimension, the 
> equivalent image width would have to be 0.03 degree. For an EIGER the 
> numbers become 75 microns, hence 0.21 milliradian i.e. 0.012 degree.
>
>   Hence my advice, untainted by any commercial agenda :-) .
>   
>   
>   With best wishes,
>   
>Gerard.
>
> --
> On Thu, Jul 13, 2017 at 01:25:08PM -0500, Gerd Rosenbaum wrote:
>> Dear Gerard,
>>
>> you sound like a sales person for Dectris. Fine slicing is perfectly 
>> fine with CCD detectors - it takes a bit longer because of the step 
>> scan instead of continuous scan. The read noise issue is often 
>> overstated compared to the sample induced scatter background. If for 
>> fine slicing at 0.05 degree or less the diffraction peaks go too 
>> close to the read noise make a longer exposure - signal goes up, 
>> ratio signal to sample-induced-BG less, as for any fine slicing, same read 
>> noise.
>>
>> It would be helpful to analyze the dense spot packing along layer 
>> lines if we knew the wavelength and the sample-to-detector distance 
>> (assuming this is a 300 mm detector) and the rotation width - as you 
>> pointed out. That would help to distinguish between multiple crystals 
>> (my guess) and lattice translocation disorder. Fine slicing is 
>> definitely needed to figure out what the diffraction pattern at 120 
>> degree could tell you in terms of strong anisotropy .
>>
>> Best regard.
>>
>> Gerd
>>
>> On 13.07.2017 08:20, Gerard Bricogne wrote:
>>> Dear Tang,
>>>
>>>   I noticed that your diffraction images seem to have been 
>>> recorded on a 3x3 CCD detector. With this type of detector, fine 
>>> slicing is often discouraged (because of the readout noise), and yet 
>>> with the two long cell axes you have, any form of thick (or only 
>>> semi-fine) slicing would result in spot overlaps.
>>>
>>>   What, then, was your image width? Would you have access to a 
>>> beamline with a Pilatus detector so that you could collect 
>>> fine-sliced data?
>>>
>>>   I would tend to agree with Herman that your crystals might be 
>>> cursed with lattice translocation