--- In FairfieldLife@yahoogroups.com, new.morning <[EMAIL PROTECTED]> wrote: > > --- In FairfieldLife@yahoogroups.com, off_world_beings <no_reply@> > wrote: > > > > Even though I do not consider stock markets a measure of anything > > significant, do you really thingk that 30 year Statistician Maestro > > Maxwell Rainforth > > He was not listed as having done the study. > > > did not account for this layman's mistake? > > Don't kid yourself...he accounted for it. > > > > OffWorld > > But OK then. If you are sure. > > Even though this "layman's mistake" is made by many. Its common to > assume normality of distrubutions without testing specifically for > such. Often, results are not too distorted. Except as, stated, when > dealing with extremes at the tail of the distribution. > > > So if my hypothesis is incorrect as to why their analysis was so far > off, by a factor of 5000, then perhaps you can explain why they could > be so far off. They "found" a 1/50,00 chance of a market rise such as > the currently one, when such naturally occurs about 5 out of any 52 > starting weeks, on average in any given year. That is, a 12.9% gain or > better for a 17 weeks period it has occurred on average 4.98 times > year, going back to 1960.
If their mistake is not incorrectly assuming normality (gaussian) of the distribution (at the extreme tails) then, it must be overspecification, the other reason I cited. Overspecification, over qualifiying the event in ways not significant. For example, instead of testing for 13% rises in the S&P 500 over 17 weeks, one could have tested 13% rises in the S&P 500 over 17 weeks where there is a pattern that there are weekly declines in the 3rd, 5th and 7th weeks of the series. That is the pattern of the recent rise, but is absolutely unimportant to the premise being tested: if the 13% rises in the S&P 500 over 17 weeks is "rare" or if it naturally occurs periodically. The occurence of 13% rises in the S&P 500 over 17 weeks with weekly declines in the 3rd, 5th and 7th weeks of the series is indeed much rarer than the over 220 times the simpler 13% rises in the S&P 500 over 17 weeks has occurred since 1960. Perhaps the overspecified event happened only this time, First time ever. So it would a 1/2500 or so probability, not 1/55,000. But of course such overspecified analysis is quite flawed since the pattern of weekly declines in the 3rd, 5th and 7th weeks is totally inconsequential to the event of interest: a 13% rise in 17 weeks. (Unless one is working on some esoteric prime number sequence pattern :) ) But even if they overspecified the pattern, as in the example above, to arrive at a 1/55,000 chance of occurance, they would have to be testing for the pattern over 55,000 weeks, about 1000 years! Preposterous. The S&P 500 was started in 1957. However, Their conclusion is equivalent to saying the recent 13% rise in 17 weeks has only happpened once in the last 1000 years. The fact that this pattern has actually occurred over 220 times since 1960 puts a small dent in their the credibility of their analysis, rigor and objectivity. So overspecification, while it may have been a contributing flaw in their analysis, doesn't seem to be the whole problem. I can duplicate their findings by assuming normality of the distribution at the extreme tails. And partly by overspecification. And I suppose simply by dropping 4 zeros "by mistake". So the ball is in your court Off. Tell me your hypotheses of how such a gross error of analysis could be committed. First, I assume you have a statistics background. Tell us a bit about it. Is that the subject you teach at the arts school? > > --- In FairfieldLife@yahoogroups.com, new.morning <no_reply@> > > wrote: > > > > > > A recent MUM press release states tha "advanced econometric models" > > > show something like a 1/50,000 chance for normal occurance of > > recent > > > market increases since the start of IA -- thus opening the door for > > > claims of some special new effect. > > > > > > A well-known mistake that can be made in studies of financial > > markets > > > is to using analysis and models that assume a normal distribution. > > > Distributions of the returns for most financial markets, certainly > > the > > > S&P 500 -- the proxy for all US markets -- has "fat tails" and a > > > lower peak than a normal distribution. More like the top half of a > > > somewhat flattened circle -- aka an elipse, than the classic "bell" > > > shape of a normal (aka guassian) distribution. > > > > > > For some analysis, in the main body of the distriubtion, the > > > difference is not to substantial. However, when analyzing the > > tails, > > > the extreme events, the normal distribution extremely > > underestimates > > > the occurence of extreme gains or losses, that is the top or > > bottom > > > 1% or more accutely, the top .1%. > > > > > > Such laxness in using normal distribution assumptiosn when modeling > > > financial markets is one reason for the famous, dramatic collapse > > of > > > the hedge fund LTC -- Long Term Capital -- built and managed by > > some > > > of the best academic minds on Wall St. They simply underestimated > > the > > > probability of "the perfect storm". Instead of being a one in 500 > > or > > > so year event, it occurs far more "frequently" -- and did so on > > their > > > watch, collapsing 100 billion dollar portfolios to nothing within > > weeks. > > > > > > That MUM's recent analysis may be falling into similar trap > > occurred > > > to me this morning when doing some calculations to answer Turq's > > question. > > > > > > Looking at just 13 years of S&P 500 data, assuming a normal > > > distribution -- as is the case when applying most tests of > > > significance in statistics, the top return day, is overestimated in > > > its rarity, by a factor of 15,000. The third top day by a factor of > > > 5000. The top 10th day by a factor of 250. The rarity of the top 80 > > > days are overstated by assuming the S&P 500 is normally > > distributed, > > > though the factor of overstatement declines rapidly from the top > > down > > > to the 80th highest daily return. The same occurs on the left tail, > > > the lowest, or most negative returns. And in the middle of the > > > distribution, the "rarity" of returns is underestimated. > > > > > > MUM was trying to assess how rare is the recent run-up in the S&P > > > 500. While they did not disclose what tests or methods they used, > > it > > > is likely they used methods that assume a normal distribution -- as > > > most methods in most statistical software do. And such loose > > > assumptions of normality works ok, except when dealing with the > > very > > > extremes of the "tails" of financial distributions. > > > > > > They are looking at the rarity of the returns over 13+ weeks, the > > same > > > principle applies: A normal distribution will underestimate the > > > frequency of extreme events -- certainly the top 1 or 2, or 10 or > > so. > > > In other words, it will deem such a tail event much rarer than it > > > really is, Thus, the 1/50,000 probability might be overestimated > > by a > > > factor of 1000 or even 10,000. > > > > > > On top of that, if the MUM researchers over specificed > > the "pattern" > > > of the last 13 week rise, that is, over qualified it in ways that > > are > > > not essential to its "description", this too would make the recent > > > rise appear statistically much rare than it is. > > > > > > Thus, the caveat to always eye-ball the underlying data and > > results. > > > Does looking at the recent rise in the context of the last 4-5 > > years > > > seem quite unique? No, similar reversals have occured about once > > every > > > year. > > > > > > Perhaps the MUM press release should states that "advanced > > econometric > > > models", run by not so advanced econometricians, showed ... the > > > following misleading results." > > > > > >
