I wasn't specific enough I guess, The null hypothesis is that there is no expected difference between the 2 choices rather than A will not be preferred to B. A cheaper audio cable may provide more or less bass, and that may or may not be preferred.
So if you consistently prefer the less expensive option, you are able to detect a difference. This is different than a medical trial null hypothesis where you are in fact testing whether the treatment is better than placebo. ----- Original Message ----- From: 'Pascal Jasmin' via Programming <[email protected]> To: "[email protected]" <[email protected]> Cc: Sent: Wednesday, February 25, 2015 1:39 AM Subject: Re: [Jprogramming] Tea-tasting for non-statisticians sorry did not test, (0.025 < 4 : 'binomialprob 0.5, x, (-:x) + | (-:x) - y' ) ----- Original Message ----- From: 'Pascal Jasmin' via Programming <[email protected]> To: "[email protected]" <[email protected]> Cc: Sent: Wednesday, February 25, 2015 1:36 AM Subject: Re: [Jprogramming] Tea-tasting for non-statisticians binomialprob returns the probability that the number of successes will be >: given amount. This is not quite the same as confidence intervals. The latter (for 95% CI) is that the number of successes is outside 2 standard deviations from the mean (0.5) (0.025 < 4 : 'binomialprob 0.5, x, x + | x-y' ) where 1 is reject. ----- Original Message ----- From: Ian Clark <[email protected]> To: [email protected] Cc: Sent: Tuesday, February 24, 2015 9:46 PM Subject: [Jprogramming] Tea-tasting for non-statisticians Addon 'stats/base/distribution' defines the verb: binomialprob. Am I using it correctly? Please cast a beady eye over my train of thought, as I've set it out below... I've written an app in J to administer a double-blind test which reruns the classic experiment described by David Salsburg in "The Lady Tasting Tea". But in place of pre- and post-lactary tea, I play a snatch of one of two soundfiles in a series of 10 trials to ascertain if she really can tell the difference. >From her answers I compute 2 numbers: N=: number of trials (typically 10 or 20) s=: number of successes. I have also set up an adjustable parameter: CONFIDENCE=: 0.95 NB. (the 95% confidence limit) Instead of using binomialprob directly, I define 2 verbs: pH0=: 4 : 'binomialprob 0.5,x,y' rejectH0=: 4 : '(1-CONFIDENCE) > x pH0 y' (p=. N pH0 s) is the likelihood of s arising at random under the "null hypothesis" (H0), viz that she's just guessing with probability=0.5 of success. (N rejectH0 s) returns 1 iff p is too low, as determined by CONFIDENCE, implying the null hypothesis (H0) can safely be rejected. This Boolean value triggers one of 2 messages: 1 --> You can tell the difference. 0 --> You're just guessing. That's straining the epistemology, I know. But I don't expect a non-statistician to make much sense of a statistically kosher message, such as: 0 --> This program has decided that the (null) hypothesis that your results have arisen by pure guesswork cannot be safely rejected on the evidence alone of these 10 trials. There's gratifying interest in the music/audiophile community in such a sound-test, if it's packaged up and made easy-to-use. Questions like: "can you hear any improvement if you use an optical cable instead of a USB one?" come up all the time. And, as I've discovered for myself, even a strong impression of improvement may not stand up to this sort of scrutiny. It's the placebo effect. So not only do I need to assure the soundness of the statistical theory and its J implementation, plus my use of it, but also publish a proper write-up (in Jwiki) which makes sense to a non-statistician. This after all is *the* foundation experiment in the history of Statistics. But AFAICS its treatment in Wikipedia leaves much to be desired. See for instance: https://en.wikipedia.org/wiki/Binomial_test ---------------------------------------------------------------------- For information about J forums see http://www.jsoftware.com/forums.htm ---------------------------------------------------------------------- For information about J forums see http://www.jsoftware.com/forums.htm ---------------------------------------------------------------------- For information about J forums see http://www.jsoftware.com/forums.htm ---------------------------------------------------------------------- For information about J forums see http://www.jsoftware.com/forums.htm
