Re: Meta question: disappearing posts (was Re: calculating a self.value, self.randomnum = normalvariate(x, y))
Ross Ridge rri...@csclub.uwaterloo.ca wrote: I'm not sure what MIME would have to do with it, but Piet van Oostrum's problem is almost certainly as result of the python.org mail to news gateway mangling the References header. The missing postings he's looking for don't actually exist. Just go up the thread one more posting and you'll find the message that was being replied to. In article h1s5l1$79...@panix3.panix.com, Aahz a...@pythoncraft.com wrote: While that's also a bug in Mailman (I have a long-standing to-do item to fix that), there are also plenty of posts that simply aren't showing up in c.l.py. Well, the message IDs that Piet van Oostrum gave are symptomatic of the References header bug and and just like he described, my newsreader also shows Dennis Lee Bieber always replying to posts that don't exist. Other messages might be getting eaten by the gateway, but the missing posts that Piet is complaining about almost certainly never existed. Ross Ridge -- l/ // Ross Ridge -- The Great HTMU [oo][oo] rri...@csclub.uwaterloo.ca -()-/()/ http://www.csclub.uwaterloo.ca/~rridge/ db // -- http://mail.python.org/mailman/listinfo/python-list
Re: Meta question: disappearing posts (was Re: calculating a self.value, self.randomnum = normalvariate(x, y))
In article h1o1dk$hm...@rumours.uwaterloo.ca, Ross Ridge rri...@csclub.uwaterloo.ca wrote: Aahz a...@pythoncraft.com wrote: Piet van Oostrum p...@cs.uu.nl wrote: I notice that I see several postings on news:comp.lang.python that are replies to other postings that I don't see. As stated previously, my suspicion is that at least some is caused by a problem with MIME messages and the mail-news gateway on python.org. I'm not sure what MIME would have to do with it, but Piet van Oostrum's problem is almost certainly as result of the python.org mail to news gateway mangling the References header. The missing postings he's looking for don't actually exist. Just go up the thread one more posting and you'll find the message that was being replied to. While that's also a bug in Mailman (I have a long-standing to-do item to fix that), there are also plenty of posts that simply aren't showing up in c.l.py. As I said, I'm pretty sure (based on what was happening with c.l.py.announce) that it's some kind of weird problem with the mail-news gateway with MIME posts. -- Aahz (a...@pythoncraft.com) * http://www.pythoncraft.com/ as long as we like the same operating system, things are cool. --piranha -- http://mail.python.org/mailman/listinfo/python-list
Re: Meta question: disappearing posts (was Re: calculating a self.value, self.randomnum = normalvariate(x, y))
Piet van Oostrum p...@cs.uu.nl wrote: I notice that I see several postings on news:comp.lang.python that are replies to other postings that I don't see. Aahz a...@pythoncraft.com wrote: As stated previously, my suspicion is that at least some is caused by a problem with MIME messages and the mail-news gateway on python.org. I'm not sure what MIME would have to do with it, but Piet van Oostrum's problem is almost certainly as result of the python.org mail to news gateway mangling the References header. The missing postings he's looking for don't actually exist. Just go up the thread one more posting and you'll find the message that was being replied to. Ross Ridge -- l/ // Ross Ridge -- The Great HTMU [oo][oo] rri...@csclub.uwaterloo.ca -()-/()/ http://www.csclub.uwaterloo.ca/~rridge/ db // -- http://mail.python.org/mailman/listinfo/python-list
Meta question: disappearing posts (was Re: calculating a self.value, self.randomnum = normalvariate(x, y))
I notice that I see several postings on news:comp.lang.python that are replies to other postings that I don't see. Examples are the postings by Dennis Lee Bieber wlfr...@ix.netcom.com that I am replying to (but I break the thread on purpose). For example the posting with Message-ID: mailman.1883.1245538633.8015.python-l...@python.org references: 77e831100906192220y5536d9d2oe5ca2dcc59084...@mail.gmail.com which is not present on my news server. I have been wondering why these disappear, and I noticed the following in the Dennis Lee Bieber posting: On Sat, 20 Jun 2009 11:48:21 -0600, Vincent Davis vinc...@vincentdavis.net declaimed the following in gmane.comp.python.general: So apparently some of these come through gmane.comp.python.general. So my question is: would this be the cause of these disappearing postings? Are postings on gmane.comp.python.general not relayed to comp.lang.python? Are they relayed to the python mailing list? I find it quite disturbing that sometimes only half of a discussion is visible. -- Piet van Oostrum p...@cs.uu.nl URL: http://pietvanoostrum.com [PGP 8DAE142BE17999C4] Private email: p...@vanoostrum.org -- http://mail.python.org/mailman/listinfo/python-list
Re: Meta question: disappearing posts (was Re: calculating a self.value, self.randomnum = normalvariate(x, y))
On Sun, Jun 21, 2009 at 5:25 AM, Piet van Oostrump...@cs.uu.nl wrote: I notice that I see several postings on news:comp.lang.python that are replies to other postings that I don't see. Examples are the postings by Dennis Lee Bieber wlfr...@ix.netcom.com that I am replying to (but I As addressed in an earlier thread, Mr. Bieber chooses to set a Usenet header (X-Noarchive) in his postings that suppresses their permanent archiving (and often the archiving of replies to his posts). I would direct you to the thread, but it looks like it wasn't archived. :P Cheers, Chris -- http://blog.rebertia.com -- http://mail.python.org/mailman/listinfo/python-list
Re: Meta question: disappearing posts (was Re: calculating a self.value, self.randomnum = normalvariate(x, y))
Piet van Oostrum wrote: I notice that I see several postings on news:comp.lang.python that are replies to other postings that I don't see. Examples are the postings by Dennis Lee Bieber wlfr...@ix.netcom.com that I am replying to (but I break the thread on purpose). For example the posting with Message-ID: mailman.1883.1245538633.8015.python-l...@python.org references: 77e831100906192220y5536d9d2oe5ca2dcc59084...@mail.gmail.com which is not present on my news server. I have been wondering why these disappear, and I noticed the following in the Dennis Lee Bieber posting: On Sat, 20 Jun 2009 11:48:21 -0600, Vincent Davis vinc...@vincentdavis.net declaimed the following in gmane.comp.python.general: So apparently some of these come through gmane.comp.python.general. I am posting and reading thru gmane and generally see no problem. Sometimes I do see replies before the posting being replied. Sometimes certain posts get 'echoed' about week after the original posting date. If a reply gets echoed, but not the original, and one missed both originally, that can look weird. tjr So my question is: would this be the cause of these disappearing postings? Are postings on gmane.comp.python.general not relayed to comp.lang.python? Are they relayed to the python mailing list? I find it quite disturbing that sometimes only half of a discussion is visible. -- http://mail.python.org/mailman/listinfo/python-list
Re: Meta question: disappearing posts (was Re: calculating a self.value, self.randomnum = normalvariate(x, y))
Piet van Oostrum wrote: I notice that I see several postings on news:comp.lang.python that are replies to other postings that I don't see. I see the same problem. I suspect it's because of over-vigorous spam filtering from Usenet providers. Some even block everything from anyone using Google Groups. It's quite frustrating, to have perfectly valid Python-related posts go missing while dozens of posts offering to sell well-known brands of shoes and watches are delivered. -- Steven -- http://mail.python.org/mailman/listinfo/python-list
Re: Meta question: disappearing posts (was Re: calculating a self.value, self.randomnum = normalvariate(x, y))
Dennis Lee Bieber wrote: unless one is reading from a server that interprets X-no-archive to mean delete before reading. Can't be too careful with security. Destroy it, memorize it and then read it! -- Greg -- http://mail.python.org/mailman/listinfo/python-list
Re: Meta question: disappearing posts (was Re: calculating a self.value, self.randomnum = normalvariate(x, y))
In article m21vpddb9y.fsf...@cs.uu.nl, Piet van Oostrum p...@cs.uu.nl wrote: I notice that I see several postings on news:comp.lang.python that are replies to other postings that I don't see. As stated previously, my suspicion is that at least some is caused by a problem with MIME messages and the mail-news gateway on python.org. -- Aahz (a...@pythoncraft.com) * http://www.pythoncraft.com/ as long as we like the same operating system, things are cool. --piranha -- http://mail.python.org/mailman/listinfo/python-list
Re: calculating a self.value, self.randomnum = normalvariate(x, y)
While there are a lot of valid ways to do it, anything you do will change the outcome of the probability anyway. I'm assuming you are just looking to clamp the values. Try this: http://codepad.org/NzlmSMN9 (it runs the code, too) == # Clamp a normal distribution outcome import random class applicant(): def __init__(self, x, y): self.randomnum = clamp(random.normalvariate(x, y), 0, 100) def clamp(input, min=0, max=100): Clamps the input between min and max. if input min, returns min min input max, returns input input max, returns max Default: min = 0, max = 100. if input min: return min elif input max: return max else: return input if __name__ == __main__: for num in range(10): print applicant(random.randint(0,100), random.randint(0,100)).randomnum == Or you could just use randint() if you only wanted a linear distribution. PS: Thanks, btw, new to python myself also, and looking into this was cool. :] Best regards, Ching-Yun Xavier Ho, Technical Artist Contact Information Mobile: (+61) 04 3335 4748 Skype ID: SpaXe85 Email: cont...@xavierho.com Website: http://xavierho.com/ On Sat, Jun 20, 2009 at 3:20 PM, Vincent Davis vinc...@vincentdavis.netwrote: I currently have something like this. class applicant(): def __int__(self, x, y): self.randomnum = normalvariate(x, y) then other stuff x, y are only used to calculate self.randomnum and this seems to work. But I want self.randomnum to be 0 = randomnum = 100. The only way I can thing of to do this is is with a while statement and that seems more complicated than necessary. I would really like to keep it on one line. How would I do that? Thanks Vincent Davis -- http://mail.python.org/mailman/listinfo/python-list -- http://mail.python.org/mailman/listinfo/python-list
Re: calculating a self.value, self.randomnum = normalvariate(x, y)
# Clamp a normal distribution outcome import random class applicant(): def __init__(self, x, y): self.randomnum = clamp(random.normalvariate(x, y), 0, 100) def clamp(input, min=0, max=100): Clamps the input between min and max. if input min, returns min min input max, returns input input max, returns max Default: min = 0, max = 100. if input min: return min elif input max: return max else: return input if __name__ == __main__: for num in range(10): print applicant(random.randint(0,100), random.randint(0,100)).randomnum Why not have the def clamp inside the class? I would prefer to keep everything I need for the class together. I am new to classes but I have to say things like if __name__ == __main__: have no intuitive meaning to me. It is true I don't know what __name__ and __main__ do and I can look it up but I don't even have a guess based on the names and usage. I am Now not sure if that is what I want or If I want to redraw from the distribution. I am wanting to simulate test scores. My option see to be to draw from a normal (I don't want linear) distribution and scale it to 0-100 or clamp it as you (Xavier) suggested or draw from the distribution again (this is what I was thinking) I think this is still what I want but I should look up the implications of each. The problem I have with the clamp is that the tails in the sample could be large. Thanks Vincent -- http://mail.python.org/mailman/listinfo/python-list
Re: calculating a self.value, self.randomnum = normalvariate(x, y)
Vincent Davis wrote: I currently have something like this. class applicant(): def __int__(self, x, y): self.randomnum = normalvariate(x, y) then other stuff x, y are only used to calculate self.randomnum and this seems to work. But I want self.randomnum to be 0 = randomnum = 100. The only way I can thing of to do this is is with a while statement and that seems more complicated than necessary. I would really like to keep it on one line. How would I do that? Thanks Vincent Davis There are at least two interpretations to your requirement, both of which change the distribution. So you have to specify what the real distribution is that you want. 1) clamp - see Xavier's response. This will give you more values at exactly 100 and/or exactly 0 than normalvariate would specify, 2) discard - this will throw away any values outside the range, and try again. And in this case, a loop is essential, plus some mechanism to handle the case where the loop would never terminate. For example, suppose x is 1. -- http://mail.python.org/mailman/listinfo/python-list
Re: calculating a self.value, self.randomnum = normalvariate(x, y)
Vincent Davis wrote: I currently have something like this. class applicant(): def __int__(self, x, y): self.randomnum = normalvariate(x, y) then other stuff x, y are only used to calculate self.randomnum and this seems to work. But I want self.randomnum to be 0 = randomnum = 100. Then it isn't a normal variate. The only way I can thing of to do this is is with a while statement and that seems more complicated than necessary. Why? It's a perfectly simple procedure: def __int__(self, x, y): x = -1 while not 0 = x = 100: x = normalvariate(x, y) # do other stuff That is the correct way to truncate a normal distribution. Alternatively, truncate values past the end-points, but this will distort the random distribution significantly: x = max(0, min(100, normalvariate(x, y))) You probably don't want that, as it will cause far more 0 and 100 values than you would otherwise expect. I would really like to keep it on one line. How would I do that? Why? Is the enter key on your keyboard broken? -- Steven -- http://mail.python.org/mailman/listinfo/python-list
Re: calculating a self.value, self.randomnum = normalvariate(x, y)
def __int__(self, x, y): x = -1 while not 0 = x = 100: x = normalvariate(x, y) # do other stuff That is the correct way to truncate a normal distribution. Thanks for the response. But why would you set the mean to -1 to begin? x = max(0, min(100, normalvariate(x, y))) That is an awesome way of shorthanding clamp. Best regards, Ching-Yun Xavier Ho, Technical Artist Contact Information Mobile: (+61) 04 3335 4748 Skype ID: SpaXe85 Email: cont...@xavierho.com Website: http://xavierho.com/ -- http://mail.python.org/mailman/listinfo/python-list
Re: calculating a self.value, self.randomnum = normalvariate(x, y)
Vincent Davis wrote: # Clamp a normal distribution outcome I don't know who you are quoting -- you should give attribution to them. def clamp(input, min=0, max=100): ... if input min: return min elif input max: return max else: return input An easier way to do this: return min(100, max(0, input)) but again, I stress that this will strongly distort the random distribution. It's probably not what you want. Why not have the def clamp inside the class? Why bother? I would prefer to keep everything I need for the class together. But you don't. You have the random.normalvariate in a completely different module. I'm sure you have other operations like +, - etc as built-in functions. Not everything is inside the class. I am new to classes but I have to say things like if __name__ == __main__: have no intuitive meaning to me. It is true I don't know what __name__ and __main__ do and I can look it up but I don't even have a guess based on the names and usage. When you import a module with the line: import mymodule Python automatically creates a variable mymodule.__name__ and sets it to the string __mymodule__. When you run a module as a script, by calling it from the shell, using (for example): $ python mymodule.py Python automatically creates the variable mymodule.__name__ as before, but this time sets its value to the string __main__. So the construction: if __name__ == __main__: do_stuff_here() is a way to include code that will only be executed when running the module as a script, not when it is imported as a module. I am Now not sure if that is what I want or If I want to redraw from the distribution. I am wanting to simulate test scores. My option see to be to draw from a normal (I don't want linear) distribution and scale it to 0-100 or clamp it as you (Xavier) suggested or draw from the distribution again (this is what I was thinking) I think this is still what I want but I should look up the implications of each. The problem I have with the clamp is that the tails in the sample could be large. Strictly speaking, you want a different distribution, not normal. Possibly the F-distribution? Anyway, whatever it is, unless you need very accurate results, a truncated normal distribution shouldn't be *too* far off: close enough for government work. Clamping will not be very good: it will result in an excess of 0 and 100 scores. Imagine a graph that looks vaguely like this: 000: ** 010: * 020: ** 030: 040: *** 050: * 060: ** 070: 080: * 090: ** 100: * That's what you'll get by clamping (possibly exaggerated for effect). Truncating with a while loop will result in something closer to this: 000: * 010: * 020: ** 030: 040: *** 050: * 060: ** 070: 080: * 090: ** 100: * which is far less distorted. -- Steven -- http://mail.python.org/mailman/listinfo/python-list
Re: calculating a self.value, self.randomnum = normalvariate(x, y)
Quoting Steven, Truncating with a while loop will result in something closer to this: 000: * 010: * 020: ** 030: 040: *** 050: * 060: ** 070: 080: * 090: ** 100: * which is far less distorted. That is why I was thinking of a while loop. Strictly speaking, you want a different distribution, not normal. Possibly the F-distribution? Why do you suggest a F dist, I have to admit I am not really sure how most test score are distributed. I was thinking it would be a truncated skewed normal. i.e. scores range from 0-100, mean is 80. For my I realize this is venturing more into statistics. Thanks Vincent Davis 720-301-3003 -- http://mail.python.org/mailman/listinfo/python-list
Re: calculating a self.value, self.randomnum = normalvariate(x, y)
Quoting Dennis Lee Bieber limitedNormal ( 75, 20 ) computed statistics: mu = 75.5121294828 sigma = 8.16374859991 Note how computing the input sigma such that 3*sigma does not exceed boundaries results in a narrow bell curve (hmm, and for this set, no one scored 95-100) retryNormal ( 75, 20 ) computed statistics: mu = 73.283826412 sigma = 16.9151951316 The retry model produces a skew, but perhaps somewhat unnatural; a real skew should still have relatively few entries in the 95-100 bin, whereas this is still rather symmetrical about the mean. Compare the 45, 65, and 80 bins between these two, those show most of the retried values that otherwise clipped to 100 below. clippedNormal ( 75, 20 ) computed statistics: mu = 75.3240108464 sigma = 18.1008966385 Note the unnatural peak of grades in the 95-100 range resulting from clipping out of range values into the range. * See, the full results below* Wow thanks for this Dennis, I am actually trying to simulate the scores on the step 1 medical boards exam. The actual distribution is not readily available. I like your limitedNormal approach. I think this is the way for me to go. I can come up with some reasonable mean and sd numbers from actual results then I this approach seems the best to simulate those results. I should actualy know more about this as I do a lot of stats but mostly regressions. I need to look up ome more about the F dist, but I think your limited approuch is the way to go. Trying to combine learning python and simulating the medical residency application process has been interesting. Here is a graph of past test results, I relize they are not on a 0 - 100 score but they is easy to address [image: step1_score_distribution_custom.GIF] Thanks Vincent Davis 720-301-3003 On Sat, Jun 20, 2009 at 7:43 PM, Dennis Lee Bieber wlfr...@ix.netcom.comwrote: I must be bored today... Expanded variation: -=-=-=-=-=- Random Scores import random import numpy def limitedNormal(mu, sigma=None): returns a random score from a normal (bell) distribution in which the mean, mu, is supplied by the caller, and in which the standard deviation, sigma, is computed such that 3-sigma does not drop below 0 [for mu 50] or rise above 100 [for mu 50] sigma is shown as a parameter but is not used -- it permits using the same arguments for all three *Normal() methods if mu 50.0: sigma = mu / 3.0 else: sigma = (100.0 - mu) / 3.0 return random.normalvariate(mu, sigma) def clippedNormal(mu, sigma): returns a random score from a normal distribution in which the mean, mu, and standard deviation, sigma, are supplied by the caller. the result is clipped to the range 0..100 return max(0.0, min(100.0, random.normalvariate(mu, sigma))) def retryNormal(mu, sigma): returns a random score from a normal distribution in which the mean, mu, and the standard deviation, sigma, are supplied by the caller. if the result falls outside the range 0..100 a new score is generated. extremely large sigma, or mu close to the range end points will cause slowness, as many results are thrown out and retried score = -1 while not (0.0 = score = 100.0): score = random.normalvariate(mu, sigma) return score def clippedGamma(mu, B): returns a random score from a gamma distribution in which the shape, a, is computed as the mean, mu, as from a normal distribution divided by the B, rate. Both mu and B are supplied by the caller. as the gamma distribution has a long tail to the right, for mu 50 the result is computed as 100 - gamma(100-mu) to reflect the desired skewness results are clipped to the boundaries 0..100 as there is no easy way to compute B to limit results, as is done for sigma in limitedNormal() NOTE: while the mean of the results will approach mu, the peak of the curve will be to the right (for mu50) or left (for mu50) relative to a normal curve if mu 50.0: return max(0.0, min(100.0, random.gammavariate(mu / B, B))) else: return 100.0 - max(0.0, min(100.0, random.gammavariate((100.0 - mu) / B, B))) def retryGamma(mu, B): returns a random score from a gamma distribution in which the shape, a, is computed as the mean, mu, as from a normal distribution divided by the B, rate. Both mu and B are supplied by the caller. as the gamma distribution has a long tail to the right, for mu 50 the result is computed as
