Dear Kay, Yes, I agree. I am sorry I was too focused on the errors. The question I was trying to address is how useful frequentist error estimates are when the observed counts are 0. I guess both frequentists and Bayesians agree on the definition of descriptive statistics when the entire population is known. The disagreement comes from the uncertainty either because we do not see the future or somehow we are prevented to see the entire population. It is a bit artificial example, but perhaps we only see 100 pixels of the detector and we try to describe how the rest of, say, 999900 unknown pixels might look like. Based on the mean of 0 I have to conclude that the rest of the pixels must also have 0 counts since there is no range for errors in the rate parameter. And it does not matter if I initially observe 100, 10, or 2 pixels with 0 photons. If this is the frequentist interpretation of the error based on 0 counts then I am not sure how useful it is.
Best wishes, Gergely -----Original Message----- From: Kay Diederichs <[email protected]> Sent: den 16 oktober 2021 09:48 To: [email protected]; Gergely Katona <[email protected]> Subject: Re: am I doing this right? Dear Gergely, with " 10 x 10 patch of pixels ", I believe James means that he observes 100 neighbouring pixels each with 0 counts. Thus the frequentist view can be taken, and results in 0 as the variance, right? best, Kay On Fri, 15 Oct 2021 21:07:26 +0000, Gergely Katona <[email protected]> wrote: >Dear James, > >Uniform distribution sounds like “I have no idea”, but a uniform distribution >does not go from -inf to +inf. If I believe that every count from 0 to 65535 >has the same probability, then I also expect counts with an average of 32768 >on the image. It is not an objective belief in the end and probably not a very >good idea for an X-ray experiment if the number of observations are small. >Concerning which variance is the right one, the frequentist view requires >frequencies to be observed. In the absence of frequencies, there is no error >estimate. Bayesians at least can determine a single distribution as an answer >without observations and that will be their prior belief of the variance. >Again, I would avoid a uniform a priori distribution for the variance. For a >Poisson distribution the convenient conjugate prior is the gamma distribution. >It can control the magnitude of k and strength of belief with its location and >scale parameter, respectively. > >Best wishes, > >Gergely > >Gergely Katona, Professor, Chairman of the Chemistry Program Council >Department of Chemistry and Molecular Biology, University of Gothenburg >Box 462, 40530 Göteborg, Sweden >Tel: +46-31-786-3959 / M: +46-70-912-3309 / Fax: +46-31-786-3910 >Web: http://katonalab.eu, Email: [email protected] > >From: CCP4 bulletin board <[email protected]> On Behalf Of James >Holton >Sent: 15 October, 2021 18:06 >To: [email protected] >Subject: Re: [ccp4bb] am I doing this right? > >Well I'll be... > >Kay Diederichs pointed out to me off-list that the k+1 expectation and >variance from observing k photons is in "Bayesian Reasoning in Data Analysis: >A Critical Introduction" by Giulio D. Agostini. Granted, that is with a >uniform prior, which I take as the Bayesean equivalent of "I have no idea". > >So, if I'm looking to integrate a 10 x 10 patch of pixels on a weak detector >image, and I find that area has zero counts, what variance shall I put on that >observation? Is it: > >a) zero >b) 1.0 >c) 100 > >Wish I could say there are no wrong answers, but I think at least two >of those are incorrect, > >-James Holton >MAD Scientist >On 10/13/2021 2:34 PM, Filipe Maia wrote: >I forgot to add probably the most important. James is correct, the expected >value of u, the true mean, given a single observation k is indeed k+1 and k+1 >is also the mean square error of using k+1 as the estimator of the true mean. > >Cheers, >Filipe > >On Wed, 13 Oct 2021 at 23:17, Filipe Maia ><[email protected]<mailto:[email protected]>> wrote: >Hi, > >The maximum likelihood estimator for a Poisson distributed variable is equal >to the mean of the observations. In the case of a single observation, it will >be equal to that observation. As Graeme suggested, you can calculate the >probability mass function for a given observation with different Poisson >parameters (i.e. true means) and see that function peaks when the parameter >matches the observation. > >The root mean squared error of the estimation of the true mean from a single >observation k seems to be sqrt(k+2). Or to put it in another way, mean squared >error, that is the expected value of (k-u)**2, for an observation k and a true >mean u, is equal to k+2. > >You can see some example calculations at >https://colab.research.google.com/drive/1eoaNrDqaPnP-4FTGiNZxMllP7SFHkQ >uS?usp=sharing > >Cheers, >Filipe > >On Wed, 13 Oct 2021 at 17:14, Winter, Graeme (DLSLtd,RAL,LSCI) ><[email protected]<mailto:[email protected]>> > wrote: >This rang a bell to me last night, and I think you can derive this from >first principles > >If you assume an observation of N counts, you can calculate the >probability of such an observation for a given Poisson rate constant X. >If you then integrate over all possible value of X to work out the >central value of the rate constant which is most likely to result in an >observation of N I think you get X = N+1 > >I think it is the kind of calculation you can perform on a napkin, if >memory serves > >All the best Graeme > > >On 13 Oct 2021, at 16:10, Andrew Leslie - MRC LMB ><[email protected]<mailto:[email protected]>> wrote: > >Hi Ian, James, > > I have a strong feeling that I have seen this result > before, and it was due to Andy Hammersley at ESRF. I’ve done a literature > search and there is a paper relating to errors in analysis of counting > statistics (se below), but I had a quick look at this and could not find the > (N+1) correction, so it must have been somewhere else. I Have cc’d Andy on > this Email (hoping that this Email address from 2016 still works) and maybe > he can throw more light on this. What I remember at the time I saw this was > the simplicity of the correction. > >Cheers, > >Andrew > >Reducing bias in the analysis of counting statistics data Hammersley, >AP<https://www.webofscience.com/wos/author/record/2665675> (Hammersley, >AP) Antoniadis, >A<https://www.webofscience.com/wos/author/record/13070551> (Antoniadis, >A) NUCLEAR INSTRUMENTS & METHODS IN PHYSICS RESEARCH SECTION >A-ACCELERATORS SPECTROMETERS DETECTORS AND ASSOCIATED EQUIPMENT Volume >394 >Issue >1-2 >Page >219-224 >DOI >10.1016/S0168-9002(97)00668-2 >Published >JUL 11 1997 > > >On 12 Oct 2021, at 18:55, Ian Tickle ><[email protected]<mailto:[email protected]>> wrote: > > >Hi James > >What the Poisson distribution tells you is that if the true count is N then >the expectation and variance are also N. That's not the same thing as saying >that for an observed count N the expectation and variance are N. Consider all >those cases where the observed count is exactly zero. That can arise from any >number of true counts, though as you noted larger values become increasingly >unlikely. However those true counts are all >= 0 which means that the mean >and variance of those true counts must be positive and non-zero. From your >results they are both 1 though I haven't been through the algebra to prove it. > >So what you are saying seems correct: for N observed counts we should be >taking the best estimate of the true value and variance as N+1. For >reasonably large N the difference is small but if you are concerned with weak >images it might start to become significant. > >Cheers > >-- Ian > > >On Tue, 12 Oct 2021 at 17:56, James Holton ><[email protected]<mailto:[email protected]>> wrote: >All my life I have believed that if you're counting photons then the >error of observing N counts is sqrt(N). However, a calculation I just >performed suggests its actually sqrt(N+1). > >My purpose here is to understand the weak-image limit of data >processing. Question is: for a given pixel, if one photon is all you >got, what do you "know"? > >I simulated millions of 1-second experiments. For each I used a "true" >beam intensity (Itrue) between 0.001 and 20 photons/s. That is, for >Itrue= 0.001 the average over a very long exposure would be 1 photon >every 1000 seconds or so. For a 1-second exposure the observed count >(N) is almost always zero. About 1 in 1000 of them will see one photon, >and roughly 1 in a million will get N=2. I do 10,000 such experiments >and put the results into a pile. I then repeat with Itrue=0.002, >Itrue=0.003, etc. All the way up to Itrue = 20. At Itrue > 20 I never >see N=1, not even in 1e7 experiments. With Itrue=0, I also see no N=1 >events. >Now I go through my pile of results and extract those with N=1, and >count up the number of times a given Itrue produced such an event. The >histogram of Itrue values in this subset is itself Poisson, but with >mean = 2 ! If I similarly count up events where 2 and only 2 photons >were seen, the mean Itrue is 3. And if I look at only zero-count events >the mean and standard deviation is unity. > >Does that mean the error of observing N counts is really sqrt(N+1) ? > >I admit that this little exercise assumes that the distribution of >Itrue is uniform between 0.001 and 20, but given that one photon has >been observed Itrue values outside this range are highly unlikely. The >Itrue=0.001 and N=1 events are only a tiny fraction of the whole. So, >I wold say that even if the prior distribution is not uniform, it is >certainly bracketed. Now, Itrue=0 is possible if the shutter didn't >open, but if the rest of the detector pixels have N=~1, doesn't this >affect the prior distribution of Itrue on our pixel of interest? > >Of course, two or more photons are better than one, but these days with >small crystals and big detectors N=1 is no longer a trivial situation. >I look forward to hearing your take on this. And no, this is not a trick. > >-James Holton >MAD Scientist > >####################################################################### ># > >To unsubscribe from the CCP4BB list, click the following link: >https://www.jiscmail.ac.uk/cgi-bin/WA-JISC.exe?SUBED1=CCP4BB&A=1 > >This message was issued to members of >www.jiscmail.ac.uk/CCP4BB<http://www.jiscmail.ac.uk/CCP4BB>, a mailing >list hosted by www.jiscmail.ac.uk<http://www.jiscmail.ac.uk/>, terms & >conditions are available at >https://www.jiscmail.ac.uk/policyandsecurity/ > >________________________________ > >To unsubscribe from the CCP4BB list, click the following link: >https://www.jiscmail.ac.uk/cgi-bin/WA-JISC.exe?SUBED1=CCP4BB&A=1 > > >________________________________ > >To unsubscribe from the CCP4BB list, click the following link: >https://www.jiscmail.ac.uk/cgi-bin/WA-JISC.exe?SUBED1=CCP4BB&A=1 > > > > >-- > >This e-mail and any attachments may contain confidential, copyright and or >privileged material, and are for the use of the intended addressee only. If >you are not the intended addressee or an authorised recipient of the addressee >please notify us of receipt by returning the e-mail and do not use, copy, >retain, distribute or disclose the information in or attached to the e-mail. >Any opinions expressed within this e-mail are those of the individual and not >necessarily of Diamond Light Source Ltd. >Diamond Light Source Ltd. cannot guarantee that this e-mail or any attachments >are free from viruses and we cannot accept liability for any damage which you >may sustain as a result of software viruses which may be transmitted in or >with the message. >Diamond Light Source Limited (company no. 4375679). Registered in >England and Wales with its registered office at Diamond House, Harwell >Science and Innovation Campus, Didcot, Oxfordshire, OX11 0DE, United >Kingdom > > >________________________________ > >To unsubscribe from the CCP4BB list, click the following link: >https://www.jiscmail.ac.uk/cgi-bin/WA-JISC.exe?SUBED1=CCP4BB&A=1 > >________________________________ > >To unsubscribe from the CCP4BB list, click the following link: >https://www.jiscmail.ac.uk/cgi-bin/WA-JISC.exe?SUBED1=CCP4BB&A=1 > > >________________________________ > >To unsubscribe from the CCP4BB list, click the following link: >https://www.jiscmail.ac.uk/cgi-bin/WA-JISC.exe?SUBED1=CCP4BB&A=1 > >####################################################################### ># > >To unsubscribe from the CCP4BB list, click the following link: >https://www.jiscmail.ac.uk/cgi-bin/WA-JISC.exe?SUBED1=CCP4BB&A=1 > >This message was issued to members of www.jiscmail.ac.uk/CCP4BB, a >mailing list hosted by www.jiscmail.ac.uk, terms & conditions are >available at https://www.jiscmail.ac.uk/policyandsecurity/ ######################################################################## To unsubscribe from the CCP4BB list, click the following link: https://www.jiscmail.ac.uk/cgi-bin/WA-JISC.exe?SUBED1=CCP4BB&A=1 This message was issued to members of www.jiscmail.ac.uk/CCP4BB, a mailing list hosted by www.jiscmail.ac.uk, terms & conditions are available at https://www.jiscmail.ac.uk/policyandsecurity/
