Hi James What the Poisson distribution tells you is that if the true count is N then the expectation and variance are also N. That's not the same thing as saying that for an observed count N the expectation and variance are N. Consider all those cases where the observed count is exactly zero. That can arise from any number of true counts, though as you noted larger values become increasingly unlikely. However those true counts are all >= 0 which means that the mean and variance of those true counts must be positive and non-zero. From your results they are both 1 though I haven't been through the algebra to prove it.
So what you are saying seems correct: for N observed counts we should be taking the best estimate of the true value and variance as N+1. For reasonably large N the difference is small but if you are concerned with weak images it might start to become significant. Cheers -- Ian On Tue, 12 Oct 2021 at 17:56, James Holton <jmhol...@lbl.gov> wrote: > All my life I have believed that if you're counting photons then the > error of observing N counts is sqrt(N). However, a calculation I just > performed suggests its actually sqrt(N+1). > > My purpose here is to understand the weak-image limit of data > processing. Question is: for a given pixel, if one photon is all you > got, what do you "know"? > > I simulated millions of 1-second experiments. For each I used a "true" > beam intensity (Itrue) between 0.001 and 20 photons/s. That is, for > Itrue= 0.001 the average over a very long exposure would be 1 photon > every 1000 seconds or so. For a 1-second exposure the observed count (N) > is almost always zero. About 1 in 1000 of them will see one photon, and > roughly 1 in a million will get N=2. I do 10,000 such experiments and > put the results into a pile. I then repeat with Itrue=0.002, > Itrue=0.003, etc. All the way up to Itrue = 20. At Itrue > 20 I never > see N=1, not even in 1e7 experiments. With Itrue=0, I also see no N=1 > events. > Now I go through my pile of results and extract those with N=1, and > count up the number of times a given Itrue produced such an event. The > histogram of Itrue values in this subset is itself Poisson, but with > mean = 2 ! If I similarly count up events where 2 and only 2 photons > were seen, the mean Itrue is 3. And if I look at only zero-count events > the mean and standard deviation is unity. > > Does that mean the error of observing N counts is really sqrt(N+1) ? > > I admit that this little exercise assumes that the distribution of Itrue > is uniform between 0.001 and 20, but given that one photon has been > observed Itrue values outside this range are highly unlikely. The > Itrue=0.001 and N=1 events are only a tiny fraction of the whole. So, I > wold say that even if the prior distribution is not uniform, it is > certainly bracketed. Now, Itrue=0 is possible if the shutter didn't > open, but if the rest of the detector pixels have N=~1, doesn't this > affect the prior distribution of Itrue on our pixel of interest? > > Of course, two or more photons are better than one, but these days with > small crystals and big detectors N=1 is no longer a trivial situation. > I look forward to hearing your take on this. And no, this is not a trick. > > -James Holton > MAD Scientist > > ######################################################################## > > To unsubscribe from the CCP4BB list, click the following link: > https://www.jiscmail.ac.uk/cgi-bin/WA-JISC.exe?SUBED1=CCP4BB&A=1 > > This message was issued to members of www.jiscmail.ac.uk/CCP4BB, a > mailing list hosted by www.jiscmail.ac.uk, terms & conditions are > available at https://www.jiscmail.ac.uk/policyandsecurity/ > ######################################################################## To unsubscribe from the CCP4BB list, click the following link: https://www.jiscmail.ac.uk/cgi-bin/WA-JISC.exe?SUBED1=CCP4BB&A=1 This message was issued to members of www.jiscmail.ac.uk/CCP4BB, a mailing list hosted by www.jiscmail.ac.uk, terms & conditions are available at https://www.jiscmail.ac.uk/policyandsecurity/
