On 22 June 2013 19:39, Douglas Theobald <dtheob...@brandeis.edu> wrote:
> > So I'm no detector expert by any means, but I have been assured by those > who are that there are non-Poissonian sources of noise --- I believe mostly > in the readout, when photon counts get amplified. Of course this will > depend on the exact type of detector, maybe the newest have only Poisson > noise. > Sorry for delay in responding, I've been thinking about it. It's indeed possible that the older detectors had non-Poissonian noise as you say, but AFAIK all detectors return _unsigned_ integers (unless possibly the number is to be interpreted as a flag to indicate some error condition, but then obviously you wouldn't interpret it as a count). So whatever the detector AFAIK it's physically impossible for it to return a negative number that is to be interpreted as a photon count (of course the integration program may interpret the count as a _signed_ integer but that's purely a technical software issue). I think we're all at least agreed that, whatever the true distribution of Ispot (and Iback) is, it's not in general Gaussian, except as an approximation in the limit of large Ispot and Iback (with the proviso that under this approximation Ispot & Iback can never be negative). Certainly the assumption (again AFAIK) has always been that var(count) = count and I think I'm right in saying that only a Poisson distribution has that property? No, its just terminology. For you, Iobs is defined as Ispot-Iback, and > that's fine. (As an aside, assuming the Poisson model, this Iobs will have > a Skellam distribution, which can take negative values and asymptotically > approaches a Gaussian.) The photons contributed to Ispot from Itrue will > still be Poisson. Let's call them something besides Iobs, how about Ireal? > Then, the Poisson model is > > Ispot = Ireal + Iback' > > where Ireal comes from a Poisson with mean Itrue, and Iback' comes from a > Poisson with mean Iback_true. The same likelihood function follows, as > well as the same points. You're correct that we can't directly estimate > Iback', but I assume that Iback (the counts around the spot) come from the > same Poisson with mean Iback_true (as usual). > > So I would say, sure, you have defined Iobs, and it has a Skellam > distribution, but what, if anything, does that Iobs have to do with Itrue? > My point still holds, that your Iobs is not a valid estimate of Itrue when > Ispot<Iback. Iobs as an estimate of Itrue requires unphysical assumptions, > namely that photon counts can be negative. It is impossible to > derive Ispot-Iback as an estimate for Itrue (when Ispot<Iback) *unless* you > make that unphysical assumption (like the Gaussian model). > Please note that I have never claimed that Iobs = Ispot - Iback is to be interpreted as an estimate of Itrue, indeed quite the opposite: I agree completely that Iobs has little to do with Itrue when Iobs is negative. In fact I don't believe anyone else is claiming that Iobs is to be interpreted as an estimate of Itrue either, so maybe this is the source of the misunderstanding? Certainly for me Ispot - Iback is merely the difference between the two measurements, nothing more. Maybe if we called it something other than Iobs (say Idiff), or even avoided giving it a name altogether that would avoid any further confusion? Perhaps this whole discussion has been merely about terminology? > I'm also puzzled as to your claim that Iback' is not Poisson. I don't > think your QM argument is relevant, since we can imagine what we would have > detected at the spot if we'd blocked the reflection, and that # of photon > counts would be Poisson. That is precisely the conventional logic behind > estimating Iback' with Iback (from around the spot), it's supposedly a > reasonable control. It doesn't matter that in reality the photons are > indistinguishable --- that's exactly what the probability model is for. > I'm not clear how you would "block the reflection"? How could you do that without also blocking the background under it? A large part of the background comes from the TDS which is coming from the same place that the Bragg diffraction is coming from, i.e. the crystal. I know of no way of stopping the Bragg diffraction without also stopping the TDS (or vice versa). Indeed the theory shows that there is in reality no distinction between Bragg diffraction and TDS; they are just components of the total scattering that we find convenient to imagine as separate in the dynamical model of scattering (see http://people.cryst.bbk.ac.uk/~tickle/iucr99/s61.html for the relevant equations). Any given photon "experiences" the whole crystal on its way from the source to the detector (in fact it experiences more than that: it traverses all possible trajectories simultaneously, it's just that the vast majority cancel by destructive interference). The resulting wave function of the photon only collapses to a single point on hitting the detector, with a frequency proportional to the square of the wave function at that point, so it's meaningless to talk about the trajectory of an individual photon or whether it "belongs" to Ireal or Iback'. You can't talk about the error distribution of the experimental measurements of some quantity if it's a physical impossibility to design an experiment to measure it! It can of course have a probability distribution derived from prior knowledge of the properties of crystals, but that's not a Poisson, it's a Wilson (exponential) distribution. Is that what you're thinking of? According to QM the only real quantities are the observables (or functions of observables); in this case only Ispot, Iback and Ispot-Iback (and any other functions of Ispot & Iback that might be relevant) are physically meaningful quantities, all else is mere speculation, i.e. part of the model. As I understand it the reason you are suggesting an alternate way of estimating Itrue is that you have a fundamental objection to the F & W algorithm? However I'm not clear precisely what you find objectionable? Perhaps it would be useful to go through F & W in detail and identify where the problem (if any) lies? We can say that the total likelihood of J (= Itrue) given Is (= Ispot) and Ib (= Iback) is equal to the prior probability density of J given only knowledge of the crystal (i.e. the estimated no of atoms from which we can calculate E(J)), multiplied by the joint probability density of Is and Ib given J and its SD (assumed equal to the SD of Is-Ib): P(J | Is,Ib) = P(J | E(J)) P(Is,Ib | J,sdJ) The only function of Is and Ib that's relevant to the joint distribution of Is and Ib given J and sdJ, P(Is,Ib | J), is the difference Is-Ib (at least for large Is and Ib: I don't know what happens if they are small). Note that it's perfectly proper to talk about P(Is-Ib | J) in this context: it's the distribution of the difference you expect to observe given any J. So the above can be rewritten as: P(J | Is-Ib) = P(J | E(J)) P(Is-Ib | J,sdJ) P(Is-Ib | J,sdJ) is just the Gaussian error distribution of Is-Ib making use of the Gaussian approximation of the Poisson. Finally, integrating out J to get the expectation of J (or of F=sqrt(J)) completes the F-W procedure. As indicated earlier there are good reasons to postpone this until after merging equivalents (which is exactly what we do now). So what's wrong with that? Cheers -- Ian
