Re: [UAI] Has anyone else noticed how odd many frequentist techniques are?
Hi Rich,
If you are looking for a forum where these issues are frequently discussed,
I recommend Andrew Gelman's blog: http://andrewgelman.com
If you are looking for formal sources, there are the references cited in
Kevin's attachment (in addition to his book, of course). In particular, if
you are aiming to write something on the topic I recommend perusing the
book by Jaynes (and his papers more generally).
Regards,
Konrad
On Sat, Sep 27, 2014 at 12:44 PM, Richard E Neapolitan <
richard.neapoli...@northwestern.edu> wrote:
> Thanks, Kevin,
> Well, I guess they are not too well-known. I asked my mentor on Bayesian
> stats, Sandy Zabell (prominant Bayesian statistician), about it. Although
> he agreed with me, he did not really have references stating how
> "pathological" these frequentists techniques are.
>
> I will tell Sandy about your book. He still teachs stats at NU.
> Best,
> Rich
>
>
>
> On 9/27/2014 1:08 PM, Kevin Murphy wrote:
>
> Yes, these problems are very well known. I am attaching a brief summary
> ( from my textbook <http://www.cs.ubc.ca/%7Emurphyk/MLbook/index.html>) of
> some of the most famous "pathologies of frequentist statistics" (cited
> references can be found in the bibliography here
> <http://www.cs.ubc.ca/%7Emurphyk/MLbook/pml-bib.pdf>). There are several
> more pathologies, but I didn't want to go overboard :)
>
> Kevin
>
> PS. A very nice practical book for teaching undergrad stats from a
> Bayesian POV is this:
>
> @book{Kruschke10,
> title = {{Doing Bayesian Data Analysis: A Tutorial Introduction with R and
> BUGS}},
> author = "J. Kruschke",
> year = 2010,
> publisher = "Academic Press"
> }
>
>
>
>
> On Fri, Sep 26, 2014 at 1:59 PM, Richard E Neapolitan <
> richard.neapoli...@northwestern.edu> wrote:
>
>> Dear Colleagues,
>>
>> Since I converted to Bayesian statistics in the late 1980's, I have not
>> looked at most frequentist methods. However, every time I look at them
>> again, I notice how apparently preposterous many of them are.
>>
>> First that was the Bonferroni correction, which makes me update my belief
>> about the results of an experiment based on how many other experiments I
>> happen to conduct with it (and which of course implicitly assigns a low
>> prior probability). One researcher even told me once that he has students
>> first conduct fewer experiments so a finding has a better chance of being
>> significant. I just walked away scratching my head.
>>
>> Now, in the process of designing a small test for a student, I noticed
>> that two-tailed hypothesis testing is completely unreasonable. Along with
>> the one-tailed test, it gives me decision rules which enable me to reject
>> the hypothesis that the mean is less than or equal to 0, but not reject the
>> hypothesis that it equals 0. The explanation is wrapped up in a story about
>> the question asked and long run behavior with other similar experiments,
>> that are not even run. So two people can walk away from the same experiment
>> with different updated beliefs about whether the mean is 0, not based on
>> their prior beliefs, but based on the question they happened to ask. In
>> general, hypothesis testing does not seem to be the way to go. We should
>> simply compute confidence intervals or posterior probability intervals.
>>
>> The Bayesian's world is so much simpler. She updates her belief solely on
>> her prior beliefs and the data. There is no story that leads to strange
>> results.
>>
>> All this matters, especially in medical applications, because so many
>> studies are deemed significant or not significant based on the enigmatic
>> p-value and the Bonferroni correction. I like to say that in medicine
>> for every study there is an equal and opposite study.
>>
>> I am writing this because I wonder who else has noticed these oddities? I
>> never read about them. I simply observed them independently. I find it
>> curious that they have persisted for so long, and more is not said about
>> them.
>>
>> Best,
>> Rich
>>
>>
>> --
>> Richard E. Neapolitan, Ph.D., Professor
>> Division of Health and Biomedical Informatics
>> Department of Preventive Medicine
>> Northwestern University Feinberg School of Medicine
>> 750 N. Lake Shore Drive, 11th floor
>> Chicago IL 60611
>>
>>
>> ___
>> uai mailing list
>> uai@ENGR.ORST.EDU
>> https://secure.engr.oregonstate.edu/mailman/listinfo/uai
>>
>>
>
>
Re: [UAI] A perplexing problem - Version 2
On Mon, 23 Feb 2009, Francisco Javier Diez wrote:
> Konrad Scheffler wrote:
> > I agree this is problematic - the notion of calibration (i.e. that you can
> > say P(S|"70%") = .7) does not really make sense in the subjective Bayesian
> > framework where different individuals are working with different priors,
> > because different individuals will have different posteriors and they can't
> > all be equal to 0.7.
>
> I apologize if I have missed your point, but I think it does make sense. If
> different people have different posteriors, it means that some people will
> agree that the TWC reports are calibrated, while others will disagree.
I think this is another way of saying the same thing - if you define the
concept of calibration such that people will, depending on their priors,
disagree over whether the reports are calibrated then it is still
problematic to prescribe calibration in the problem formulation - because
this will mean different things to different people. Unless you take
"TWC is calibrated" to mean "everyone has the same prior as TWC", which I
don't think was the intention in the original question.
In my opinion the source of confusion here is the use of a subjective
Bayesian framework (i.e. one where the prior is not explicitly stated and
is assumed to be different for different people). If instead we use an
objective Bayesian framework where all priors are stated explicitly, the
difficulties disappear.
> Who is right? In the case of unrepeatable events, this question would not make
> sense, because it is not possible to determine the "true" probability, and
> therefore whether a person or a model is calibrated or not is a subjective
> opinion (of an external observer).
>
> However, in the case of repeatable events--and I acknowledge that
> repeatability is a fuzzy concept--, it does make sense to speak of an
> objective probability, which can be identified with the relative frequency.
> Subjective probabilities that agree with the objective probability (frequency)
> can be said to be correct and models that give the correct probability for
> each scenario will be considered to be calibrated.
>
> If we accept that "snow" is a repeatable event, the all the individuals should
> agree on the same probability. If it is not, P(S|"70%") may be different for
> each individual because having different priors and perhaps different
> likelihoods or even different structures in their models.
I strongly disagree with this. The ("true") relative frequency is not the
same thing as the correct posterior. One can imagine a situation where the
correct posterior (calculated from the available information) is very far
from the relative frequency which one would obtain given the opportunity
to perform exhaustive experiments.
Probabilities (in any variant of the Bayesian framework) do not describe
reality directly, they describe what we know about reality (typically in
the absence of complete information).
> Coming back to the main problem, I agree again with Peter Szolovits in making
> the distinction between likelihood and posterior probability.
>
> a) If I take the TWC forecast as the posterior probability returned by a
> calibrated model (the TWC's model), then I accept that the probability of snow
> is 70%.
>
> b) However, if I take "70% probability of snow" as a finding to be introduced
> in my model, then I should combine my prior with the likelihood ratio
> associated with this finding, and after some computation I will arrive at
> P(S|"70%") = 0.70. [Otherwise, I would be incoherent with my assumption that
> the model used by the TWC is calibrated.]
>
> Of course, if I think that the TWC's model is calibrated, I do not need to
> build a model of TWC's reports that will return as an output the same
> probability estimate that I introduce as an input.
>
> Therefore I see no contradiction in the Bayesian framework.
But this argument only considers the case where your prior is identical
to TWC's prior. If your prior were _different_ from theirs (the more
interesting case) then you would not agree that they are calibrated.
___
uai mailing list
uai@ENGR.ORST.EDU
https://secure.engr.oregonstate.edu/mailman/listinfo/uai
Re: [UAI] A perplexing problem - Last Version
Dear Paul, Bayesian inference is still appropriate for both problems. There are two issues here: 1) the subjectivist Bayesian viewpoint is confusing because it does not make it explicit on which information you are conditioning when setting up your prior - it becomes much clearer if you use an objective Bayesian framework (see below). 2) You are describing a situation where your two sources of information are dependent, but you are not quantifying the dependency. As Jean-Louis points out, the problem becomes simple if you are prepared to make an independence assumption (but I think this avoids the difficulty you are asking about: "In part they are using the same background knowledge that Analyst A has"). Below I give the full solution (which unfortunately is only useful if you can quantify the dependencies - for this you need a model of how the analysts go about calculating their reported probabilities). I'll use "X" as a shorthand for the statement "X is at location Y". Let's assume all analysts give their statements as a numerical value which quantifies their confidence that X is true. Let's call the value provided by the spectral analyst B and that provided by the chemical analyst C. Let I denote the information available to analyst A before reading the reports, so that her prior for X is P(X|I). We want to know P(X|BCI), which can be written as follows: P(X|BCI) = P(B|XCI)P(X|CI)/P(B|CI) where P(X|CI) = P(C|XI)P(X|I)/[P(C|XI)P(X|I)+P(C|(not X)I)P((not X)|I)] and P(B|CI) = P(B|XCI)P(X|CI) + P(B|(not X)CI)P((not X)|CI). The quantities P(X|I), P((not X)|I), P(C|XI) and P(C|(not X)I) are straightforward, but instead of P(B|XI) and P(B|(not X)I) we need to know P(B|XCI) and P(B|(not X)CI). Once these are known the answer follows. Hope this is useful, Konrad On Thu, 19 Feb 2009, Lehner, Paul E. wrote: > Austin, Jean-Lous, Konrad, Peter > > Thank you for your responses. They are very helpful. > > Your consensus view seems to be that when receiving evidence in the form > of a single calibrated judgment, one should not update personal > judgments by using Bayes rule. This seems incoherent (from a strict > Bayesian perspective) unless perhaps one explicitly represents the > overlap of knowledge with the source of the calibrated judgment (which > may not be practical.) > > Unfortunately this is the conclusion I was afraid we would reach, > because it leads me to be concerned that I have been giving some bad > advice about applying Bayesian reasoning to some very practical > problems. > > Here is a simple example. > > Analyst A is trying to determine whether X is at location Y. She has > two principal evidence items. The first is a report from a spectral > analyst that concludes "based on the match to the expected spectral > signature I conclude with high confidence that X is at location Y". > The second evidence is a report from a chemical analyst who asserts, > "based on the expected chemical composition that is typically associated > with X, I conclude with moderate confidence that X is at location Y." > How should analyst A approach her analysis? > > Previously I would have suggested something like this. "Consider each > evidence item in turn. Assume that X is at location Y. What are the > chances that you would receive a 'high confidence' report from the > spectral analyst, ... a report of 'moderate confidence' from the > chemical analyst. Now assume X is not a location Y, " In other > words I would have lead the analyst toward some simple instantiation of > Bayes inference. > > But clearly the spectral and chemical analyst are using more than just > the sensor data to make their confidence assessments. In part they are > using the same background knowledge that Analyst A has. Furthermore > both the spectral and chemical analysts are good at their job, their > confidence judgments are reasonably calibrated. This is just like the > TWC problem only more complex. > > So if Bayesian inference is inappropriate for the TWC problem, is it also > inappropriate here? Is my advice bad? > > Paul > > > From: uai-boun...@engr.orst.edu [mailto:uai-boun...@engr.orst.edu] On Behalf > Of Lehner, Paul E. > Sent: Monday, February 16, 2009 11:40 AM > To: uai@ENGR.ORST.EDU > Subject: Re: [UAI] A perplexing problem - Version 2 > > UAI members > > Thank you for your many responses. You've provided at least 5 distinct > answers which I summarize below. > (Answer 5 below is clearly correct, but leads me to a new quandary.) > > > > Answer 1: "70% chance of snow" is just a label and conceptually should be > treated as "XYZ". In other words don't be fooled by the semantics inside the > quotes. > > > > My response: Technically correct, but intuitively unappealing. Although I > often council people on how often intuition is misleading, I just couldn't > ignore my intuition on this one. > > > >
Re: [UAI] A perplexing problem - Version 2
I agree this is problematic - the notion of calibration (i.e. that you can
say P(S|"70%") = .7) does not really make sense in the subjective Bayesian
framework where different individuals are working with different priors,
because different individuals will have different posteriors and they
can't all be equal to 0.7. Instead, you need a notion of calibration with
respect to a particular prior.
Hopefully the TWC forecasts are calibrated with respect to their own prior
(otherwise they are reporting something other than what they believe). In
this case your subjective posterior P(S|"70%") will only be equal to .7 if
your prior happens to be identical to theirs.
Hope this helps,
Konrad
> Consider the following revised version.
>
>
> The TWC problem
>
> 1. Question: What is the chance that it will snow next Monday?
>
> 2. My subjective prior: 5%
>
> 3. Evidence: The Weather Channel (TWC) says there is a "70% chance of
> snow" on Monday.
>
> 4. TWC forecasts of snow are calibrated.
>
>
> Notice that I did not justify by subjective prior with a base rate.
>
> >From P(S)=.05 and P(S|"70%") = .7 I can deduce that P("70%"|S)/P("70%"|~S) =
> >44.33. So now I can "deduce" from my prior and evidence odds that
> >P(S|"70%") = .7. But this seems silly. Suppose my subjective prior was
> >20%. Then P("70%"|S)/P("70%"|~S) = 9.3 and again I can "deduce"
> >P(S|"70%")=.7.
>
> My latest quandary is that it seems odd that my subjective conditional
> probability of the evidence should depend on my subjective prior. This may
> be coherent, but is too counter intuitive for me to easily accept. It would
> also suggest that when receiving a single evidence item in the form of a
> judgment from a calibrated source, my posterior belief does not depend on my
> prior belief. In effect, when forecasting snow, one should ignore priors
> and listen to The Weather Channel.
>
> Is this correct? If so, does this bother anyone else?
___
uai mailing list
uai@ENGR.ORST.EDU
https://secure.engr.oregonstate.edu/mailman/listinfo/uai
Re: [UAI] A perplexing problem
Hi Paul,
Your calculation is correct, but the numbers in the example are odd. If
TWC really only manage to predict snow 10% of the time (90% false negative
rate), you would be right not to assign much value to their predictions
(you do assign _some_, hence the seven-fold increase from your prior to
your posterior, but with prediction performance like that TWC cannot
possibly think there is really a 70% chance of snow).
Change the 10% true positives to 90%, and your posterior goes up to 82.6%
- much more believable.
Also, it's important not to think the figure of 70% has any bearing on the
problem. I appreciate that you put it in as a red herring to challenge the
students, but be aware that it may also lead to confusion.
Konrad
On Fri, 13 Feb 2009, Lehner, Paul E. wrote:
> I was working on a set of instructions to teach simple
> two-hypothesis/one-evidence Bayesian updating. I came across a problem that
> perplexed me. This can't be a new problem so I'm hoping someone will clear
> things up for me.
>
> The problem
>
> 1. Question: What is the chance that it will snow next Monday?
>
> 2. My prior: 5% (because it typically snows about 5% of the days during
> the winter)
>
> 3. Evidence: The Weather Channel (TWC) says there is a "70% chance of
> snow" on Monday.
>
> 4. TWC forecasts of snow are calibrated.
>
> My initial answer is to claim that this problem is underspecified. So I add
>
>
> 5. On winter days that it snows, TWC forecasts "70% chance of snow"
> about 10% of the time
>
> 6. On winter days that it does not snow, TWC forecasts "70% chance of
> snow" about 1% of the time.
>
> So now from P(S)=.05; P("70%"|S)=.10; and P("70%"|S)=.01 I apply Bayes rule
> and deduce my posterior probability to be P(S|"70%") = .3448.
>
> Now it seems particularly odd that I would conclude there is only a 34%
> chance of snow when TWC says there is a 70% chance. TWC knows so much more
> about weather forecasting than I do.
>
> What am I doing wrong?
>
>
>
> Paul E. Lehner, Ph.D.
> Consulting Scientist
> The MITRE Corporation
> (703) 983-7968
> pleh...@mitre.org
>
___
uai mailing list
uai@ENGR.ORST.EDU
https://secure.engr.oregonstate.edu/mailman/listinfo/uai
Re: [UAI] Computation with Imprecise Probabilities--The problem of Vera's age
Dear Prof Zadeh, Perhaps you could elucidate what you mean by "cointensive"? (I assume this is explained in detail in your paper, but I also assume that one purpose of your post here is to convince people that it will be worth investing the time to read the paper.) Also, what do you understand under "probability"? Your distinction between "elasticity of meaning" and "probability of meaning" sounds very similar to the distinction between the Bayesian and frequentist interpretations of probability (as I understand "elasticity of meaning", the former encapsulates it while the latter does not - perhaps you can convince me otherwise). Regards, Konrad ---- Dr Konrad Scheffler Computer Science Division Dept of Mathematical Sciences University of Stellenbosch +27-21-808-4306 http://www.cs.sun.ac.za/~kscheffler/ On Mon, 21 Jul 2008, Lotfi A. Zadeh wrote: > Dear Dr. Mitola: > > Thank you for your constructive comment and for bringing the works of George > Lakoff, Johnson and Rhor, Jackendoff and Tom Ziemke to the attention of the > UAI community. I am very familiar with the work of George Lakoff, my good > friend, and am familiar with the work of Jackendoff. > > The issue that you raise---context-dependence of meaning---is of basic > importance. In natural languages, meaning is for the most part > context-dependent. In synthetic languages, meaning is for the most part > context-free. Context-dependence serves an important purpose, namely, > reduction in the number of words in the vocabulary. Note that such words as > small, near, tall and young are even more context-dependent than the words and > phrases cited in your comment. > > In the examples given in my message, the information set, I, and the question, > q, are described in a natural language. To come up with an answer to the > question, it is necessary to precisiate the meaning of propositions in I. To > illustrate, in the problem of Vera's age, it is necessary to precisiate the > meaning of "mother's age at birth of a child is usually between approximately > twenty and approximately forty." Precisiation should be cointensive in the > sense that the meaning of the result of precisiation should be close to the > meaning of the object of precisiation (Zadeh 2008 > <http://dx.doi.org/10.1016/j.ins.2008.02.012>). The issue of cointensive > precisiation is not addressed in the literature of cognitive linguistics nor > in the literature of computational linguistics. What is needed for this > purpose is a fuzzy logic-based approach to precisiation of meaning (Zadeh 2004 > <http://www.aaai.org/ojs/index.php/aimagazine/article/view/1778/1676>). In > Precisiated Natural Language (PNL) it is the elasticity of meaning rather than > the probability of meaning that plays a pivotal role. What this means is that > the meaning of words can be stretched, with context governing elasticity. It > is this concept that is needed to deal with context-dependence and, more > particularly, with computation with imprecise probabilities, e.g., likely and > usually, which are described in a natural language. > > In computation with imprecise probabilities, the first step involves > precisiation of the information set, I. Precisiation of I can be carried out > in various ways, leading to various models of I. A model, M, of I is > associated with two metrics: (a) cointension; and (b) computational > complexity. In general, the higher the cointension, the higher the > computational complexity is. A good model of I involves a compromise. > > In the problem of Vera's age, I consists of three propositions. p_1 : Vera > has a daughter in the mid-thirties; p_2 : Vera has a son in the mid-twenties; > and p_3 (world knowledge): mother's age at the birth of her child is usually > between approximately 20 and approximately 40. The simplest and the least > cointensive model, M_1 , is one in which mid-thirties is precisiated as 30; > mid-twenties is precisiated as 20; approximately 20 is precisiated as 20; > approximately 40 is precisiated as 40; and usually is precisiated as always. > In this model, p_1 precisiates as: Vera has a 35 year old daughter; p_2 > precisiates as: Vera has a 25 year old son; and p_3 precisiates as mother's > age at the birth of her child varies from 20 to 40. Precisiated p_1 > constrains the age of Vera as the interval [55, 75]. Since p2 is not > independent of p_1 , precisiated p_2 constrains the age of Vera as the > interval [55, 65]. Conjunction (fusion) of the two constraints leads to the > answer: Vera's age lies in the interval [55, 65]. Note that the lower bound is > determined by the lower b
[UAI] Studentships available in evolutionary modelling at Stellenbosch University
The National Bioinformatics Network (NBN) of South Africa has awarded funds for a project in evolutionary modelling at the Computer Science Division, Department of Mathematical Sciences, Stellenbosch University, South Africa. The project, which has ties with researchers at the University of Cape Town, University of California San Diego, and Stanford University, focusses on using ideas from machine learning and probabilistic modelling to model the evolution of recombining protein-coding sequences, with application to studying the evolution of HIV. We are seeking postgraduate students to fill one PhD and one MSc position (available immediately); more positions are likely to become available for students wanting to start in January 2009. The ideal candidate will have a strong background in a mathematical science (e.g. Computer Science, Applied Mathematics, Engineering, Bioinformatics, Mathematics, Statistics; knowledge of machine learning/probabilistic modelling is a plus); however, applicants with a background in other subjects (e.g. Genetics, Biochemistry, Molecular Biology) will also be considered, provided they have strong computer programming skills. Successful candidates will be expected to complete coursework in a variety of bioinformatics topics offered by the NBN. Remuneration is according to the highly competitive NBN scales. With roots going back to 1866, Stellenbosch University is one of the oldest universities in South Africa. It is a medium-sized comprehensive public university, situated in a classic university town and surrounded by the magnificent mountain scenery of the Jonkershoek Valley. Stellenbosch University is proud of being recognized as a research driven university with over 33% percent of its enrolments at the postgraduate level and the highest publication productivity in the country. If you meet the requirements above and would like to get involved in an exciting and challenging research project, with the potential to impact on the important South African HIV research arena, please send a complete academic CV (which should include information on your most advanced computer programming project to date) and a covering letter to Dr Konrad Scheffler ([EMAIL PROTECTED]). Alternatively, please get in touch by e-mail or phone (021 808 4306) to request more information about the project. Dr Konrad Scheffler Computer Science Division Dept of Mathematical Sciences University of Stellenbosch +27-21-808-4306 http://www.cs.sun.ac.za/~kscheffler/ ___ uai mailing list uai@ENGR.ORST.EDU https://secure.engr.oregonstate.edu/mailman/listinfo/uai
Re: [UAI] Determinism verses chance
Hi Marcus, Indeed it is not a novel line of thought - you will find many related ideas in the work of Jaynes, which proposes a form of (objective) probability theory without the concept of randomness. I have also seen arguments for interpretations of quantum theory without the concept of randomness. I do find the terminology used by both yourself and Rich somewhat distressing: from the Bayesian point of view there is absolutely no implication of randomness when a system is described probabilistically. Further, you use the term "objective probabilities" to denote something pertaining to the physical world, implying that a Bayesian interpretation of probability is necessarily subjective. However, objective Bayesianism claims that there is no reason to assume that what you term "outcome" probabilities need to be subjective. One can interpret the term "probability" as referring to a degree of belief about some hypothesis (how "likely" it is to be true) conditional on a specific knowledge state. In this interpretation, "objective probability" simply means that the way in which this probability is assigned/calculated is objective, as of course it needs to be in a scientific approach. Notice that starting from this point of view there is no need to introduce any concept of randomness or nondeterminism, let alone hypothesize that such a thing applies to the real world. For a detailed exposition of this approach, see "Probability Theory, the Logic of Science", by E.T. Jaynes (Cambridge University Press, 2003). In particular, chapter 10 contains arguments very similar to those made by you. regards, Konrad On Sat, 12 Aug 2006, Marcus Hutter wrote: > Hi Rich, > > > plagued many of us, at least since Laplace: Is the universe > > deterministic or is there something truly probabilistic going on > > Here is an argument that the *belief* in a truly random > universe, i.e. the belief in objective probabilities, is > "unscientific", independent of whether it's actually true or not. > > The assumption that an event occurs with some objective > probability expresses the opinion that the occurrence of an > individual stochastic event has no explanation, i.e. is inherently > impossible to predict for sure. One central goal of science is to > *explain* things. Often we do not have an explanation (yet) that > is acceptable, but to say that "something can principally not be > explained" means to stop even *trying* to find an explanation. > >From a distance, tossing a coin looks objectively random, but > looking at it closer the outcome is just subjectively unknown due > to most observers' lack of knowledge of the initial conditions and > external influences on the coin during its throw. When knowing the > exact initial conditions and the exact equations of motion, > classical physics is predictable (this includes chaotic systems). > Physicists claim that quantum mechanics is truly random, and there > is indeed quite some evidence to suggest this, but experiments > cannot exclude the possibility that quantum events are only > pseudo-random (Juergen Schmidhuber stresses this fact in several > of his recent papers, see > http://www.idsia.ch/~juergen/computeruniverse.html). It seems > safer and more honest to say that with our current technology and > understanding we can only determine (subjective) outcome > probabilities. If a sufficiently large community of people arrive > at the same subjective probabilities from their prior knowledge, > one may want to call these probabilities objective. For instance, > for most people (those with no special equipment and education) a > fair coin comes up head in 50% of the cases. And for *all* > people so far, if they measure the spin of one photon in a > para-positronium decay, it is up in 50% of the cases. > On one hand, we have to abandon objective probabilities because > their assumption seems unscientific, but on the other hand, their > assumption is very convenient. Without objective probabilities > there would be no (objective) unbiased coins, dice, > MDPs, radioactive decays, etc. Maybe one should admit > a gray scale of more or less subjective probabilities. > > I describe this argument in my book. M. Hutter, Universal > Artificial Intelligence, 2005 > http://www.hutter1.net/ai/uaibook.htm > I don't know to what extend, > if at all, this is a novel line of thought. > > Cheers, > > Marcus > > _ > Marcus Hutter > [EMAIL PROTECTED] > http://www.hutter1.net > > -- > > > Echte DSL-Flatrate dauerhaft für 0,- Euro*. Nur noch kurze Zeit! > "Feel free" mit GMX DSL: http://www.gmx.net/de/go/dsl > ___ > uai mailing list > uai@ENGR.ORST.EDU > https://secure.engr.oregonstate.edu/mailman/listinfo/uai > ___ uai mailing list uai@ENGR.ORST.EDU https://secure.engr.oregonstate.edu/mailman/listinfo/uai
Re: [UAI] A test problem involving imprecise probabilities
Hmm, no takers on this one yet? I'll rephrase the problem in a way that makes more sense to me (since the original contains words I don't know the meaning of): X and Y are unknown variables taking values in the set (1, 2, ..., n). The entries in the joint probability matrix, P, are unknown and of the form aij, where the aij take values in the unit interval and add up to unity. What is the marginal probability distribution of X in each of the following cases? a) For each aij we are given a fixed interval, with the distribution of aij being uniform inside and zero outside this interval. (I assume the intention here was that the width of the interval is known?) b) (I'll leave the translation of this one to a fuzzy specialist.) Stated this way, case (a) is unproblematic. It's worth adding that using a finite interval with zero probability density outside the interval will often be a bad thing to do in practical problems and is not recommended if you have any say in the problem design. Instead, a distribution that is nonzero throughout the unit interval will avoid nonsensical results in cases where the correct value of aij is outside the interval. Replacing the hard distribution in (a) with a soft distribution that is small but nonzero outside the given interval is easily done and the solution remains unproblematic. Konrad On Thu, 22 Sep 2005, Lotfi Zadeh wrote: > > X and Y are random variables taking values in the set (1, 2, ..., > n). The entries in the joint probability matrix, P, are of the form > "approximately aij," where the aij take values in the unit interval > and add up to unity. What is the marginal probability distribution of > X? Two special cases: (a) "approximately aij," is interpreted as an > interval centering on aij; and (b) "approximately aij," is interpreted > as a fuzzy triangular number centering on aij. > > Warm regards to all, > > Lotfi > ___ > uai mailing list > uai@ENGR.ORST.EDU > https://secure.engr.oregonstate.edu/mailman/listinfo/uai > ___ uai mailing list uai@ENGR.ORST.EDU https://secure.engr.oregonstate.edu/mailman/listinfo/uai
Re: [UAI] Is it a paradox?
On Tue, 12 Jul 2005 [EMAIL PROTECTED] wrote: > Consider the following line of reasoning. Let p be the proposition > "Ronald was born in New York." From p, we can infer q: Ronald was born > in the United States. > From q, we can infer r: It is possible that Ronald > was born in New Jersey. That's not an inference from q. Prior to learning q, r was the description of our state of knowledge regarding whether Ronald was born in New Jersey, and learning q does not change this. (At least not if your description of your state of knowledge is non-quantitative as presented here.) > On the other hand, from p we can infer s: It is > not possible that Ronald was born in New Jersey. We have arrived at a > contradiction. What is wrong? Note: To answer the question, familiarity > with modal logic is not needed. Nothing is wrong. On learning p our state of knowledge changes. From p and q our inference is different than from q alone. There is no paradox - the only way a problem will arise is if you use a reasoning framework that cannot accommodate the way beliefs change as knowledge is added. Konrad ___ uai mailing list uai@ENGR.ORST.EDU https://secure.engr.oregonstate.edu/mailman/listinfo/uai
Re: [UAI] Is it luck or is it skill - my resolution
Hi Rich,
In your analysis you present a frequentist and a Bayesian approach,
arguing that the paradox exists only for the frequentist case. Fair
enough. I would just like to point out that the frequentist approach
(orthodox hypothesis testing) is even more problematic than that, in that
it effectively makes assumptions it claims not to:
In the frequentist exposition, you state: "I have no idea whether my
population includes clairvoyants (or at least I do not want to impose my
prior beliefs)." You then give us an example of a circumstance under which
you would reject the null hypothesis. However, from your example we can
calculate bounds on your prior belief that a randomly chosen individual is
clairvoyant:
P(clairvoyant) < 0.5. (To explain your default belief in the null hypothesis).
P(clairvoyant) > 1/10001 (approximately .0001). (Otherwise it would be
irrational to reject the null hypothesis on observing success - the
alternative would still be less likely.)
If you are willing to use the commonly used p value threshold of 0.01, we
get a stronger bound: P(clairvoyant) > 1/101.
Here I am assuming that you are willing to believe a hypothesis whenever
it has probability > 0.5; if instead you prefer to build in a "grey area"
where you do not accept any beliefs, the bounds on your prior again become
more stringent.
So despite the explicit denial, this method does impose your prior
beliefs.
regards,
Konrad
On Tue, 28 Jun 2005, Rich Neapolitan wrote:
> I thank all those who responded to my query and discussed the matter with
> me. Here is my resolution.
>
> First, I'll re-describe the problem using some numbers and terminology
> provided by Francisco Javier Diez. Suppose there is some task such that
> P(success) = .0001 if someone is not clairvoyant and P(success) = 1 if
> someone is clairvoyant. I have no idea whether my population includes
> clairvoyants (or at least I do not want to impose my prior beliefs). Mike
> claims he is one. My null hypothesis is that he is not. When he succeeds a
> very unlikely event has occurred (.0001) if the null hypothesis is true. So
> I reject that hypothesis and believe Mike probably is one. Next I have
> 10,000 people making claims they are clairvoyants. My null hypothesis is
> that none are. If the null hypothesis is true, the probability of at least
> one succeeding is
> 1-(.)^10,000 = .63. So if Mike alone succeeds I have no reason to
> reject the null hypothesis. I need quite few people succeeding to reject
> it. So I have little reason for believing Mike or anyone else in the group
> is clairvoyant.
>
> There is no way out of this if we insist on obtaining our beliefs from
> hypothesis testing. However, if as I.J. Good said, we don't sweep our prior
> beliefs under the carpet, we can solve the problem using Bayes' Theorem.
> Suppose we believe that there is a .01 probability some individual (say
> Mike) is clairvoyant. Then
>
> P(clairvoyant|success)
> =
> P{success|clairvoyant)P(clairvoyant)/[P{success|clairvoyant)P(clairvoyant)
> + P{success|not clairvoyant)P(not clairvoyant)
> =1 x .01 / [1x .01 + .0001 x .99] = .99.
>
> So when Mike succeeds we believe he is probably a clairvoyant regardless of
> how many other people attempt the task or succeed.
>
> In applications to situations like Buffet predicting stock performance I
> think with a little analysis we can formulate reasonable priors, etc. and
> analyze the problem this second way. In applications like coin tossing we
> can also assign extremely small priors to someone having special ability.
> Actually out of a large group I could see where someone could have some
> talent for forcing heads. So I really mean a random experiment in which we
> control for all known tricks. There still could be some very small
> probability that someone has psychic ability.
>
> Rich
>
> ___
> uai mailing list
> uai@ENGR.ORST.EDU
> https://secure.engr.oregonstate.edu/mailman/listinfo/uai
>
___
uai mailing list
uai@ENGR.ORST.EDU
https://secure.engr.oregonstate.edu/mailman/listinfo/uai
