Re: Question about text in Python

[Hendrik van Rooyen]

"Steven D'Aprano" <[EMAIL PROTECTED]> wrote:

8<  

> This is not a call to turn a blind eye for plagiarism, or to do students
> homework for them. It's a plea for common-sense. We're not bound by
> university guidelines, or universities' over-broad definition of
> plagiarism, and we don't have to live by them. We are ethically bound not
> to do student's homework for them -- but that doesn't mean we're bound to
> refuse to answer their reasonable questions, or to treat those who are
> looking for help as frauds _just because they are a student_.

I agree - It is not fraud if the OP says he is a student, and asks for help
on something that puzzles him.

It would be fraud if he gets a "full answer" here, and represents it as
"his own work".

It would also be stupid, as its a sure fire way of failing your
examinations.

Getting clarification on some point here is no different from looking it 
up in a library, except that its more risky - some wag may lie to the 
poor bugger...

- Hendrik


-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Mastering Python (OT)

[Hendrik van Rooyen]

 "Steve Holden" <[EMAIL PROTECTED]> wrote:

> Speaking of which, here's a limerick To read it you need to know not 
> only that Hampshire is colloquially know as Hants, but also that 
> Salisbury's ancient Roman name is Sarum.
> 
> There once was a young man of Salisbury
> Whose manners were most halisbury-scalisbury
> He visited Hampshire
> Without any pampshire
> Till somebody told him to walisbury.
> 
> try-running-a-spell-checker-on-that-ly y'rs  - steve

Nice one! - Thanks Steve

- Hendrik

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: socket read timeout

[Hendrik van Rooyen]

 "Steve Holden" <[EMAIL PROTECTED]> wrote:

> 
> Right, but collisions are *so* twentieth-century, aren't they. With a 
> properly-implemented switched infrastructure Ethernet interfaces can 
> transmit and receive at the same time.

This is true, while "A" and "B" are not simultaneously trying to address
"C" - Then you need something like store and forward, on the fly...

: - )better known as "routing"...

Some (most?) of the little switches I have seen are too dumb even to 
allow  "A" to talk to "B" while "C" is talking to "D"  - they just broadcast 
the first "talker"'s message to all the "listeners" - little better than 
active hubs, destroying the end point's hardware capability to talk and
listen at the same time.

I think the keywords here are "properly implemented" - its actually not a 
trivial problem, as the switch has to know or learn who is where, and set 
up paths accordingly, in real time.  This is hard to do without store and
forward.

- Hendrik


-- 
http://mail.python.org/mailman/listinfo/python-list

Re: socket read timeout

[Hendrik van Rooyen]

 "Bryan Olson" <[EMAIL PROTECTED]> wrote:


> Steve Holden wrote:
> > Hendrik van Rooyen wrote:
> >> Are sockets full duplex?
> >>
> > Yes. But you have to use non-blocking calls in your application to use 
> > them as full-duplex in your code.
> 
> Hmmm... I'm missing something. Suppose I have one thread (or
> process) reading from a blocking-mode socket while another is
> writing to it? What stops it from being full duplex?
> 

Elsewhere in this thread I wrote about my experience with a serial port,
where I can show that the "file handler" only does the write once the 
blocking read completes - and the point at issue is if sockets are the same.

We regularly get questions about "my stuff does not come out" on
the group, and I wondered whether this effect is the underlying cause.

But I don't know about the sockets case, which is why I asked.

You raise an interesting point about a different process - my serial
experience is using threads. I have never tried mixing processes
on a serial port.  Haven't a clue if its possible, or if the behaviour
will be different.

- Hendrik

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: socket read timeout

[Hendrik van Rooyen]

 "Steve Holden" <[EMAIL PROTECTED]>


> Hendrik van Rooyen wrote:
> >  <[EMAIL PROTECTED]> wrote:
> >
> >
> >> hg> My issue with that is the effect on write: I only want a timeout on
> >> hg> read ...  but anyway ...
> >>
> >> So set a long timeout when you want to write and short timeout when you
want
> >> to read.
> >>
> >
> > Are sockets full duplex?
> >
> Yes. But you have to use non-blocking calls in your application to use
> them as full-duplex in your code.

This seems to bear out the scenario I have described elsewhere in this
thread - I think its caused by the file handlers, but I don't  _know_  it.

>
> > I know Ethernet isn't.
> >
> Don't know much, then, do you? ;-)

No not really - I easily get confused by such things as collisions...

: - )

- Hendrik

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Mastering Python (OT)

[Hendrik van Rooyen]

"Steve Holden" <[EMAIL PROTECTED]>  wrote:


> Hendrik van Rooyen wrote:

> > It comes out something like "Chum-lee", with the ch like chicken...
> > 
> > (that's what I have heard -  but who knows - It may have been 
> > a regional dialect, a case of the blind leading the blind, or 
> > someone pulling the piss..)
> > 
> You have been correctly informed. It's one of the least intuitive names 
> in the English language.

Oh No! - don't tell me there is worse - this is already enough to drive 
a saint to drink!

I will have to move to "Hants"...

: - ) 

- Hendrik

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: socket read timeout

[Hendrik van Rooyen]

"Jean-Paul Calderone" <[EMAIL PROTECTED]> wrote:

> On Thu, 29 Mar 2007 07:29:35 +0200, Hendrik van Rooyen
<[EMAIL PROTECTED]> wrote:

> >Are sockets full duplex?
>
> Uh, yes.

The reason I asked is that I have noticed that other file like objects
(on Suse 10 Linux and Python 2.4 out of the box) aren't  really:

If you start a blocking read in one thread on a serial port, and you
then issue a write in another thread, the characters are not transmitted
until the read completes.  It is as if the "driver" gets busy in the read
and queues the write - I think a lot of the "My output does not come
out till I close the programme" type of questions on this group is
caused by this.   And calling flush makes no difference under these
circumstances.- you have to unblock and use try - except on the read
as well.

So I was wondering if sockets are the same, as I haven't messed
with them much- just set up simple client server stuff in a
"Monkey see, Monkey do" way.

> >I know Ethernet isn't.
>
> Not that this is relevant, but unless you're using a hub, ethernet _is_
> full duplex.

: - ) Yes the UTP allows point to point full duplex, as there are separate
receive and transmit pairs.

But the protocol is not called Carrier Sense Multiple Access with
Collision Detection for nothing. - there can be no collisions in a true
full duplex channel. And its kind of difficult to swap rx and tx pairs
if there are more than two endpoints - you need a central point, and then
the "slaves" can't hear each other unless the central point passes the
messages on, or does some switching functions.

Reason I mentioned Ethernet is that in most cases (between machines)
its the underlying technology and if sockets were therefore not actually
full duplex it would not be surprising.

Is it actually possible to do what I have described for the serial port
case on a socket and have the message transmitted before one is
received?  Using a blocking receive?

- Hendrik

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Weird behavior in search in a list

[Hendrik van Rooyen]

"Su Y" <[EMAIL PROTECTED]> wrote:

> I want find the first number in extend[] which is larger than num, so
> I wrote:
> def find(num):
> count=0
> for elem in extend:
> if elem count+=1
> return count
> 
> I found that if extend[] is monotonous, like [1.1, 2.3, 3.2, 4.5,
> 5.6],
> it works fine: find(4) returns 3, extend[3] is 4.5.
> But, if extend[] is not monotonous, like [1.1, 2.3, 3.2, 4.5, 5.6,
> 4.6, 3.4, 2.1, 0.3],
> find(4) returns 6, extend[6] is 3.4!
> 
> what's going on here? I really can't understand

Hint: extend[0]  is1.1
Hint: extend[7] is 2.1
Hint: 2.1 is less than 4

You have to stop counting and come out of the loop when you find the 
first one - what your function is doing is counting the elements less than
num, not finding the first one that is.

hth - Hendrik


-- 
http://mail.python.org/mailman/listinfo/python-list

Re: with timeout(...):

[Hendrik van Rooyen]

 "Nick Craig-Wood" <[EMAIL PROTECTED]> wrote:

> I'd like there to be something which works well enough for day to day
> use.  Ie doesn't ever wreck the internals of python.  It could have
> some caveats like "may not timeout during C functions which haven't
> released the GIL" and that would still make it very useable.

I second this (or third or whatever if my post is slow).
It is tremendously useful to start something and to be told it has timed
out by a call, rather than to have to unblock the i/o yourself and
to "busy-loop" to see if its successful.  And from what I can see
the select functionality is not much different from busy looping...

- Hendrik

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: with timeout(...):

[Hendrik van Rooyen]

 "Nick Craig-Wood" <[EMAIL PROTECTED]> wrote:

> Well, yes there are different levels of potential reliability with
> different implementation strategies for each!

Gadzooks!  Foiled again by the horses for courses argument.

; - )

- Hendrik

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Mastering Python

[Hendrik van Rooyen]

 "Dennis Lee Bieber" <[EMAIL PROTECTED]> wrote:


> On Wed, 28 Mar 2007 07:55:20 +0200, "Hendrik van Rooyen"
> <[EMAIL PROTECTED]> declaimed the following in comp.lang.python:

> > Pretty obvious of course, as is the pronounciation of the
> > name:  "Cholmondely"
> >
> Is that a scottish "Ch" (as in LoCH Lomond), plain hard "Ch" (as in
> CHristmas) or a soft "Ch" (as in CHicken)?

It comes out something like "Chum-lee", with the ch like chicken...

(that's what I have heard -  but who knows - It may have been 
a regional dialect, a case of the blind leading the blind, or 
someone pulling the piss..)

- Hendrik

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: socket read timeout

[Hendrik van Rooyen]

 <[EMAIL PROTECTED]> wrote:


> 
> hg> My issue with that is the effect on write: I only want a timeout on
> hg> read ...  but anyway ...
> 
> So set a long timeout when you want to write and short timeout when you want
> to read.
> 

Are sockets full duplex?

I know Ethernet isn't.

- Hendrik


-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Mastering Python

[Hendrik van Rooyen]

 "Bruno Desthuilliers" <[EMAIL PROTECTED]> wrote:


>Dennis Lee Bieber a écrit :
>> On Wed, 21 Mar 2007 21:40:51 +0100, Bruno Desthuilliers
>> <[EMAIL PROTECTED]> declaimed the following in
>> comp.lang.python:
>>>
>> For future reference, and I hope you don't mind the lesson,

>I don't.

>> the past
>> tense of "bind" is "bound"

>err... I knew that, of course.

If one were to apply Dennis' ..ind to ..ound rule in reverse,
then the present tense of the verb "hound"
will be "hind" - bound to be..

English is such a a marvellously logical, consistent language.

For instance, there is a disease of the lungs called phthisis,
which is pronounced something like: "tie-sis"...

Pretty obvious of course, as is the pronounciation of the
name:  "Cholmondely"

- Hendrik


-- 
http://mail.python.org/mailman/listinfo/python-list

Re: with timeout(...):

[Hendrik van Rooyen]

 "Nick Craig-Wood" <[EMAIL PROTECTED]> wrote:


> Hendrik van Rooyen <[EMAIL PROTECTED]> wrote:

> 
> >  But would be useful to be able to do without messing with
> >  threads and GUI and imports. 
> >  Could be hard to implement as the interpreter would have 
> >  to be assured of getting control back periodically, so a 
> >  ticker interrupt routine is called for - begins to sound more 
> >  like a kernel function to me.  
> >  Isn't there something available that could be got at via ctypes?
> 
> I think if we aren't executing python bytecodes (ie are blocked in the
> kernel or running in some C extension) then we shouldn't try to
> interrupt.  It may be possible - under unix you'd send a signal -
> which python would act upon next time it got control back to the
> interpreter, but I don't think it would buy us anything except a whole
> host of problems!

Don't the bytecodes call underlying OS functions? - so is there not a case
where a particular bytecode could block, or all they all protected by
time outs?
Embedded code would handle this sort of thing by interrupting anyway
and trying to clear the mess up afterward - if the limit switch does not
appear after some elapsed time, while you are moving the 100 ton mass,
you should abort and alarm, regardless of anything else...
And if the limit switch sits on a LAN device, the OS timeouts could be
wholly inappropriate...

- Hendrik

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: with timeout(...):

[Hendrik van Rooyen]

 "Diez B. Roggisch" <[EMAIL PROTECTED]> wrote:


> Nick Craig-Wood wrote:
> 
> > Did anyone write a contextmanager implementing a timeout for
> > python2.5?
> > 
> > I'd love to be able to write something like
> > 
> > with timeout(5.0) as exceeded:
> > some_long_running_stuff()
> > if exceeded:
> > print "Oops - took too long!"
> > 
> > And have it work reliably and in a cross platform way!
> 
> Cross platform isn't the issue here - reliability though is. To put it
> simple: can't be done that way. You could of course add a timer to the
> python bytecode core, that would "jump back" to a stored savepoint or
> something like that.
> 
> But to make that work reliably, it has to be ensured that no sideeffects
> occur while being in some_long_running_stuff. which doesn't only extend to
> python itself, but also external modules and systems (file writing, network
> communications...). Which can't be done, unless you use a time-machine.
> Which I'd take as an personal insult, because in that rolled-back timeframe
> I will be possibly proposing to my future wife or something...
> 

how does the timed callback in the Tkinter stuff work - in my experience so 
far it seems that it does the timed callback quite reliably...

probably has to do with the fact that the mainloop runs as a stand alone 
process, and that you set the timer up when you do the "after" call.

so it probably means that to emulate that kind of thing you need a 
separate "thread" that is in a loop to monitor the timer's expiry, that 
somehow gains control from the "long running stuff" periodically...

so Diez is probably right that the way to go is to put the timer in the
python interpreter loop, as its the only thing around that you could 
more or less trust to run all the time.

But then it will not read as nice as Nick's wish, but more like this:

id = setup_callback(error_routine, timeout_in_milliseconds)
long_running_stuff_that_can_block_on_IO(foo, bar, baz)
cancel_callback(id)
print "Hooray it worked !! "
sys.exit()

def error_routine():
print "toughies it took too long - your chocolate is toast"
attempt_at_recovery_or_explanation(foo, bar, baz)

Much more ugly.
But would be useful to be able to do without messing with
threads and GUI and imports. 
Could be hard to implement as the interpreter would have 
to be assured of getting control back periodically, so a 
ticker interrupt routine is called for - begins to sound more 
like a kernel function to me.  
Isn't there something available that could be got at via ctypes?

- Hendrik




-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Join strings - very simple Q.

[Hendrik van Rooyen]

 "Steven D'Aprano" <[EMAIL PROTECTED]> wrote:


> On Fri, 23 Mar 2007 13:15:29 -0700, John Machin wrote:
> 
> > OK, I'll bite: This was "new" in late 2000 when Python 2.0 was
> > released. Where have you been in the last ~6.5 years?
> 
> Western civilization is 6,000 years old. Anything after 1850 is "new".
> 
> *wink*

another nibble:

What happened in 1850 to make it the demarcation line?

I would have thought that the atom bomb round about 1945/6 would
be a crisper marker..

- Hendrik

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: List to string

[Hendrik van Rooyen]

 "Steven D'Aprano" <[EMAIL PROTECTED]> wrote:

On Tue, 20 Mar 2007 13:01:36 +0100, Bruno Desthuilliers wrote:

8< --- confusion about left and right 

It gets worse.

When you work on a lathe,
a "right hand cutting tool"
has its cutting edge 
on the left...

And the worse part is that
its for good reason.

- Hendrik

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Real Time Embedded Systems Monitor in Python?

[Hendrik van Rooyen]

"frikk" <[EMAIL PROTECTED]> wrote:
> 
> I am looking into a project for the company I work for.  Essentially
> it involves setting up a real time monitor / signal injector in
> between a CPU board and a system controller.  The system controller
> sends signals (message packets) to the CPU board.  We would like to
> create an environment where we can modify signals, inject new
> messages, drop signals, etc.  This would simulate communication
> failures and message dropouts to see how the CPU board responds.  The
> application monitor would use a COM port to send and receive
> messages.  The most important part about this monitor is that
> absolutely no messages get delayed or dropped due to inturrupts or lag
> on the PC that the monitor is running on. What would be the expected
> sampling time range that I could expect to handle?

What speed is the serial port running at?   I have run a port at 115200 Baud
(albeit not carrying back to back packets), on a pentium II running Linux.

If you are not trying to do something insane like a Fast Fourier Transform in
real time, then Python should do the job for you - I would suggest you just
try it, starting with the smallest subset of commands - should take no longer 
than about a week to get a prototype running.

When you say "in between" do you mean in one port and out of another?
So that you have to handle an upstream and downstream serial port?

The sampling time range will be directly (more or less, given some spillage
for turn around times) related to the baud rate, unless its a sort of sliding 
window full duplex protocol.   (I assume there is some sort of error
detection and recovery scheme - these normally waste about 10% of the 
available bandwidth)

> 
> I have seen similar applications written for other projects that we
> have which were done in Visual Basic. I assume that if VB is up to the
> task, Python should be as well.  What kind of libraries am I looking
> at? I will probably use wxWindows, but what about for the serial ports
> and packet timing?

Start off with a basic character interface, and use the serial module,
solving the protocol hassles first.
You can add the pretty pictures afterwards.

hth - Hendrik


-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Timeout to readline()/readlines()

[Hendrik van Rooyen]

"Horta" <[EMAIL PROTECTED]> wrote:

>  Sometimes, when I do an os.popen*(), the process executed by the
> command hangs, and the script stops forever on the readline()/
> readlines() calls. I found that I can use select, but I'm thinking...
> if, after a sellect() call returns, the stdout (for example) has more
> than one line? or if it has just some characters, no newline, and the
> process just hangs?
> 
>   I just want a readline(timeout) and readlines(timeout) like
> functions. Do I need to create my owns or there're already
> implemented?

you have to unblock the file and use try - except.
look at the fcntl module

- Hendrik

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: number generator

[Hendrik van Rooyen]

 "Steven D'Aprano" <[EMAIL PROTECTED]> wrote:

> 
> Consider the distance between each paid of consecutive poles. The sum of
> the distances must add up to the distance from the first to the last, and
> if there are two fixed poles plus five in between, there are five
> distances.

No there are six distances - the fifth pole has to fit in the same hole as the 
last fixed one for there to be five.

- Hendrik

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: number generator

[Hendrik van Rooyen]

"Gabriel Genellina" <[EMAIL PROTECTED]> wrote:

>You can't have two different sets with four equal numbers - it's not a
>very difficult thing, it's impossible to distinguish because they're
>identical!
>Given 4 numbers in the set, the 5th is uniquely determined. By example:
>12, 3, 10, 18 *must* end with 7. The 5th number is not "random". Any
>randomness analysis should include only the first 4 numbers (or any other
>set of 4).
>In other words, there are only 4 degrees of freedom. In the fence analogy,
>you only have to choose where to place 4 poles; the 5th is fixed at the
>end.

Yes - the requirement of "5 random numbers" is in a sense in conflict
with the requirement of "that add up to 50" - because for the fifth number,
given that you have chosen the other 4, you have to "get lucky" and
choose the "right one" if you are doing it randomly...

What my question was about was whether some sort of cluster
analysis applied to the results of repeated application of two algorithms
would reveal the way in which the numbers were chosen - the first one being
choose five numbers, see if they add up to 50, repeat if not, repeat till say
1000 samples generated, and the second being choose four numbers,
see if they add to less than 50, repeat if not, else make the fifth the
difference
between the sum and 50, repeat till 1000 instances generated...

Intuitively, the second one will run a LOT faster, all other things being equal,
and I was simply wondering if one could tell the difference afterwards -
in the first case all the numbers clearly come from the same population,
while in the second case four of them do, and the fifth could possibly
be from a different population, as it was "forced" to make up the difference.

So I am not so very sure that the set of fifth numbers of the second run
would actually be indistinguishable from that of the first.

- Hendrik




-- 
http://mail.python.org/mailman/listinfo/python-list

Re: number generator

[Hendrik van Rooyen]

"Steven D'Aprano" <[EMAIL PROTECTED]> wrote:

> Easy-peasey. Just collate the individual numbers from the lists, and graph
> their frequencies. You will get quite different distributions. Both are
> "random", but in different ways.
> 
8<  code -

Thanks - I think too complex - was thinking about means and std devs and stuff
for each column, and chi squared tests of significance...

: - )

- Hendrik




-- 
http://mail.python.org/mailman/listinfo/python-list

Re: number generator

[Hendrik van Rooyen]

"Duncan Smith" <[EMAIL PROTECTED]> wrote:


> Yes, if the generating processes yield numbers from different
> probability mass functions.  You could simply look at the likelihood
> ratio.  Otherwise, the likelihood ratio will be 1.
> 
 I was thinking about the same random number generator being used in the two 
disparate ways.

- Hendrik

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Signed zeros: is this a bug?

[Hendrik van Rooyen]

 "Paddy" <[EMAIL PROTECTED]> wrote

> 
> Hey, I'm still learnin'. Sweet!
> 
contrary to popular belief, the answer to life,
the universe, happiness and everything is
not 42, but the above.

- Hendrik

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: number generator

[Hendrik van Rooyen]

 "Nick Craig-Wood" <[EMAIL PROTECTED]> wrote:

> Paul Rubin  wrote:
> >  The fencepost method still seems to be simplest:
> > 
> >  t = sorted(random.sample(xrange(1,50), 4))
> >  print [(j-i) for i,j in zip([0]+t, t+[50])]
> 
> Mmm, nice.
> 
> Here is another effort which is easier to reason about the
> distribution produced but not as efficient.
> 
> def real(N, M):
> while 1:
> t = [ random.random() for i in range(N) ]
> factor = M / sum(t)
> t = [ int(round(x * factor)) for x in t]
> if sum(t) == M:
> break
> print "again"
> assert len(t) == N
> assert sum(t) == M
> return t
> 
> It goes round the while loop on average 0.5 times.
> 
> If 0 isn't required then just test for it and go around the loop again
> if found.  That of course skews the distribution in difficult to
> calculate ways!
> 

I have been wondering about the following as this thread unrolled:

Is it possible to devise a test that can distinguish between sets
of:

- five random numbers that add to 50, and
- four random numbers and a fudge number that add to 50?

My stats are way too small and rusty to attempt to answer 
the question, but it seems intuitively a very difficult thing.

- Hendrik

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Need help with a string plz! (newbie)

[Hendrik van Rooyen]

"Grant Edwards" <[EMAIL PROTECTED]> wrote:


> On 2007-03-10, Hendrik van Rooyen <[EMAIL PROTECTED]> wrote:
> ><[EMAIL PROTECTED]> wrote:
> >
> >> Oh, thanks for the advice then. And as for Grant..look forward to
> >> seeing more of your posts.
> >
> > YOW! - some recognition at last!
> 
> :)
> 
> I see somebody pays attention to sigs -- which, BTW, are old quotes
> from the Zippy the Pinhead comic strip. www.zippythepinhead.com
> 

Thanks for that - I was wondering where you got hold of the skewed
thinking which I enjoy reading so much.

- Hendrik

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: distributed queue?

[Hendrik van Rooyen]

 "Paul Rubin"  wrote:

> Bjoern Schliessmann <[EMAIL PROTECTED]> writes:
> > (Why does everyone think that "concurrency" equals "usage of
> > multiple threads"?)
> 
> Well, it doesn't necessarily, but that's easiest a lot of the time.
> 
> > Try Twisted for your networking needs. 
> 
> I should try to understand Twisted better one of these days, but it's
> much more confusing than threads.  Also, the function I want to
> parallelize does blocking operations (database lookups), so in Twisted
> I'd have to figure out some way to do them asynchronously.

I would think of making 'pullers' in the remote machines in front of 
whatever it is you are making parallel to get the next thing to do, 
from a 'queue server' in the originating machine to distribute the 
stuff.

I am not sure if Pyro can help you as I have only read about it and 
not used it but I think its worth a look. If it were a one on one
setup I would not hesitate to recommend it but I can't remember 
if it is any good for one to many scenarios.

- Hendrik


-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Python in a desktop environment

[Hendrik van Rooyen]

 "David Cramer" <[EMAIL PROTECTED]> wrote:

> If you had an application that you were about to begin development on
> which you wanted to be cross platform (at least Mac and Windows),
> would you suggest using c++ and Python?
> 
> I'm asking because we were originally thinking about doing c# but
> after attending PyCon this year I'm reconsidering. We are already
> using Python for the website and I figure a c++ backend w/ a Python
> GUI may work really well, and would be pretty easy to port.
> 
> Any opinions?
 
Why drag in the C++ ? 
What do you intend to do in it that you can't do in python?
Seems unnecessarily complex to me.
And on a desktop, speed issues are kind of irrelevant - Python
is snappy enough on modern hardware for most things.

My style would be to have the gui connect via pipes to the
local backend as a separate process, with a third process if there
are any communications needed to off box dbs.   But that is just
the way I think - YMMV

- Hendrik

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Need help with a string plz! (newbie)

[Hendrik van Rooyen]

<[EMAIL PROTECTED]> wrote:

> Oh, thanks for the advice then. And as for Grant..look forward to
> seeing more of your posts.

YOW! - some recognition at last!

- Hendrik

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: persistent fifo queue class

[Hendrik van Rooyen]

"David Bear" <[EMAIL PROTECTED]> wrote:
> Diez B. Roggisch wrote:

> > Why don't you use a DB for that? If you want pickles, use a  blob
> > column. But all the rest - a defined protocol, stable server,
> > transactions - you get for free.
> > 
> > Diez
> 
> Thanks for the suggestion. I did think of this. Indeed the final destination
> of the data is in a db. However, the postsgresql server is on a separate
> box. It will be connected via a private lan. I was worried that possible
> network disruptions would cause either the web application to hang -- or
> data to just get lost. I was thinking along the lines is a message queue
> architecture, where the web app would push data directly onto a queue --
> and then a worker app would dequeue the data and handle it by sending it to
> the db server or otherwise.
> 
Have you looked at Pyro?

I think it could help to tie the boxes on the local LAN together,
with a 'Pusher' on the first box and a 'Popper' on the box where the 
final DB lives.  And Diez' DB could provide the persistence on the first
box.
The 'push' and 'pop' above are a bit of a misnomer, as Pyro will allow 
you to have a 'puller' on the final DB box to front end the DB.
I think a "double DB" architecture like that will be very robust as
Diez' DB can provide a "transactions queued, in process, and 
completed" log.

- Hendrik 



-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Wrong exist status for os.system, os.poepen, etc.

[Hendrik van Rooyen]

"Paolo Pantaleo" <[EMAIL PROTECTED]> wrote:
> $./x
> $echo $?
> 20
> 
> $ python
> 
> give the following commands:
> 
> >>> import os
> >>> os.system("/tmp/x")
> 5120
> 

256 times 20 is 5120
this is a big/little endian little bug

- Hendrik


-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Is every number in a list in a range?

[Hendrik van Rooyen]

 "Steven D'Aprano"  wrote:

8< --- some code with heavy reliance on Booleans and
'is' ---

All Right.
This has convinced me.
Some Australians do have a sense of humour.

- Hendrik



-- 
http://mail.python.org/mailman/listinfo/python-list

Re: How to Read Bytes from a file

[Hendrik van Rooyen]

 "Bart Ogryczak" <[EMAIL PROTECTED]> wrote:


> On Mar 5, 10:51 am, Piet van Oostrum <[EMAIL PROTECTED]> wrote:
> > > "Bart Ogryczak" <[EMAIL PROTECTED]> (BO) wrote:
> > >BO> Any system with 8-bit bytes, which would mean any system made after
> > >BO> 1965. I'm not aware of any Python implementation for UNIVAC, so I
> > >BO> wouldn't worry ;-)
> >
> > 1965? I worked with non-8-byte machines (CDC) until the beginning of the
> > 80's. :=( In fact in that time the institution where Guido worked also had
such
> > a machine, but Python came later.
>
> Right, I should have written 'designed' not 'made'. UNIVACs also have
> been produced until early 1980s. Anyway, I'd call it
> paleoinformatics ;-)

The correct term is: "Data Processing", or DP for short.

- Hendrik

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: How to Read Bytes from a file

[Hendrik van Rooyen]

 "Piet van Oostrum" <[EMAIL PROTECTED]> wrote:

> > "Bart Ogryczak" <[EMAIL PROTECTED]> (BO) wrote:
>
> >BO> Any system with 8-bit bytes, which would mean any system made after
> >BO> 1965. I'm not aware of any Python implementation for UNIVAC, so I
> >BO> wouldn't worry ;-)
>
> 1965? I worked with non-8-byte machines (CDC) until the beginning of the
> 80's. :=( In fact in that time the institution where Guido worked also had
such
> a machine, but Python came later.

Those behemoths were EXPENSIVE - so it made a lot of sense to keep using
them until the point that it became obvious even to an accountant that the
maintenance cost was no longer worth it...

Would actually not surprise me if there were still a few around, doing
electricity
accounts or something.

- Hendrik

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: pop method question

[Hendrik van Rooyen]

 "Alex Martelli" <[EMAIL PROTECTED]> wrote:

> Raymond Hettinger <[EMAIL PROTECTED]> wrote:
>...
> > The notion that "pop" is only defined for stack operations is somewhat
> > pedantic.
> 
> Worse: it's totally wrong.  It's also defined for eyes, as a musical
> genre, as a kind of soda, as an avant-garde artistic movement of the
> '50s, for baloons, as a parent of the male persuasion, for email
> reading, and moreover it's often used to refer to Persistent Organic
> Pollutants or Points Of Presence -- not forgetting weasels, either.
> 
One should never forget the pawning of weasels.

Then there is the contrary meaning - instead of "take out" it can 
mean "put in" - Just pop it in the oven, there's a dear...

- Hendrik

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: How to Read Bytes from a file

[Hendrik van Rooyen]

<[EMAIL PROTECTED]> wrote:

> Thanks Bart.  That's perfect.  The other suggestion was to precompute
> count1 for all possible bytes, I guess that's 0-256, right?

0 to 255 inclusive, actually - that is 256 numbers...

The largest number representable in a byte is 255

eight bits, of value 128,64,32,16,8,4,2,1

Their sum is 255...

And then there is zero.

- Hendrik

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Dialog with a process via subprocess.Popen blocks forever

[Hendrik van Rooyen]

 <[EMAIL PROTECTED]> wrote:

8<--
> The C programm gets its "commands" from its stdin and sends its state
> to stdout. Thus I have some kind of dialog over stdin.
> 
> So, once I start the C Program from the shell, I immediately get its
> output in my terminal. If I start it from a subprocess in python and
> use python's sys.stdin/sys.stdout as the subprocess' stdout/stdin I
> also get it immediately.

so why don't you just write to your stdout and read from your stdin?

> 
> BUT If I use PIPE for both (so I can .write() on the stdin and .read()

This confuses me - I assume you mean write to the c program's stdin?

> from the subprocess' stdout stream (better: file descriptor)) reading
> from the subprocess stdout blocks forever. If I write something onto
> the subprocess' stdin that causes it to somehow proceed, I can read
> from its stdout.

This sounds like the c program is getting stuck waiting for input...

> 
> Thus a useful dialogue is not possible.
> 

If you are both waiting for input, you have a Mexican standoff...

And if you are using threads, and you have issued a .read() on
a file, then a .write() to the same file, even followed by a .flush()
will not complete until after the completion of the .read().

So in such a case you have to unblock the file, and do the .read() in
a try - except clause, to "free up" the "file driver" so that the .write()
can complete.

But I am not sure if this is in fact your problem, or if it is just normal
synchronisation hassles...

- Hendrik

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Dialog with a process via subprocess.Popen blocks forever

[Hendrik van Rooyen]

 <[EMAIL PROTECTED]> wrote:



> Hi,
> 
> Thanks for your answer. I had a look into the fcntl module and tried
> to unlock the output-file, but
> 
> >>> fcntl.lockf(x.stdout, fcntl.LOCK_UN)
> Traceback (most recent call last):
>   File "", line 1, in 
> IOError: [Errno 9] Bad file descriptor
> 
> I wonder why it does work with the sys.stdin It's really a pity, it's
> the first time python does not work as expected. =/
> 
> Flushing the stdin did not help, too.

its block, not lock, and one uses file.flush() after using file.write(),
so the stdin is the wrong side - you have to push, you can't pull.. 

Here is the unblock function I use  - it comes from the internet,
possibly from this group, but I have forgotten who wrote it.

# Some magic to make a file non blocking - from the internet

def unblock(f):
"""Given file 'f', sets its unblock flag to true."""

fcntl.fcntl(f.fileno(), fcntl.F_SETFL, os.O_NONBLOCK)

hope this helps - note that the f is not the file's name but the
thing you get when you write :

f = open(...

- Hendrik

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Writing an interpreter for language similar to python!!

[Hendrik van Rooyen]

"luvsat" <[EMAIL PROTECTED]> wrote:


> Hello all,
>
> I am new to python and working on a project that involves designing a
> new language. The grammar of the language is very much inspired from
> python as in is supports nearly all the statements and expressions
> that are supported by python.

8<-

This post begs the following questions:

- Why make a new language, when
- It is going to be an inferior subset of Python -
- What can the motivation be to do this instead of contributing to the python
effort?

Dont forget about GNU Bison, if you persist...

- Hendrik




-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Dialog with a process via subprocess.Popen blocks forever

[Hendrik van Rooyen]

<[EMAIL PROTECTED]> wrote:


> Is it possible to read to and write to the std streams of a
> subprocess? What am I doing wrong?

I think this problem lies deeper - there has been a lot of
complaints about blocking and data getting stuck in pipes
and sockets...

I have noticed that the Python file objects seem to be 
inherently half duplex, but I am not sure if it is python
or the underlying OS. (Suse 10 in my case)

You can fix it by unblocking using the fcntl module, 
but then all your accesses have to be in try - except
clauses.

It may be worth making some sort of FAQ on this
subject, as it appears from time to time.

The standard advice has been to use file.flush()
after file.write(), but if you are threading and 
have called file.read(n), then the flushing does 
not help - this is why I say that the file object
seems to be inherently half duplex.

It makes perfect sense, of course, if the file is a
real disk file, as you have to finish the read before
you can move the heads to do the write- but for 
pipes, sockets and RS-232 serial lines it does not
make so much sense.

Does anybody know where it comes from - 
Python, the various OSses, or C?

- Hendrik

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Running Python scripts from BASH

[Hendrik van Rooyen]

"Ishpeck" <[EMAIL PROTECTED]> wrote:

8<--- a bash problem -

If it were Python, the advice would have been to use the 
print statement to figure out what the content of the 
variables were.

As it is Bash, you may have to stoop to something like
echo to see what is in $i...

hth - Hendrik

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Nested Parameter Definitions

[Hendrik van Rooyen]

"Arnaud Delobelle" <[EMAIL PROTECTED]> wrote:


> On Feb 25, 6:00 pm, "Paddy" <[EMAIL PROTECTED]> wrote:
> > I blogged on finding a new-to-me feature of Python, in that you are
> > allowed to nnest parameter definitions:
> >
> > >>> def x ((p0, p1), p2):
> >
> > ... return p0,p1,p2
> > ...>>> x(('Does', 'this'), 'work')
> >
> > ('Does', 'this', 'work')
> 
> Reminds me of LeLisp! It had a similar feature.  IIRC you could write
> for example (I think 'df' was LeLisp for 'defun'):
> (df mycar (a . b) a)
> or
> (df mylist L L)
> or
> (df mycaadr (a (b . c) . e) b)
> 
> I didn't know that this was possible in python and it does surprise
> me.  It feels at odd with the python philosophy.
> 

Not at all - it much nicer than you think, and there is no "nesting" 
involved - Penguins carry their eggs on their feet.

The original function definition describes a function that has a two element
tuple as a first parameter, and something else as a second one.
The first two names provide a means of accessing the elements of the
tuple, instead of using slicing.

look at this:

>>> def f((a,b),c):
 return a,b,c

>>> tup = ('hi','there')
>>> f(tup,'foo')
('hi', 'there', 'foo')
>>> lis = ['hi','there']
>>> f(lis,'foo')
('hi', 'there', 'foo')
>>> d,e,f = f(lis,42)
>>> print d,e,f
hi there 42
>>> e
'there'
>>> s = 'go'
>>> f(s,'back')
('g', 'o', 'back')
>>>  

Long live duck typing...

- Hendrik


-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Help on object scope?

[Hendrik van Rooyen]

 "hg" <[EMAIL PROTECTED]> wrote:


> [EMAIL PROTECTED] wrote:
> 
> > Hello everybody,
> > 
> > I have a (hopefully) simple question about scoping in python. I have a
> > program written as a package, with two files of interest. The two
> > files are /p.py and /lib/q.py

Make a third file for all the system wide globals, say param.py:

global r
r = 42

> > 
> > My file p.py looks like this:
> > 
> > ---
> > 
> > from lib import q
> > 
> > def main():

from param import *

> >   global r
> >   r = q.object1()
> >   s = q.object2()
> > 
> > if __name__ == "__main__":
> >   main()
> > 
> > ---
> > 
> > My file q.py in the subdirectory lib looks like this:

from param import *

> > 
> > class object1:
> >   t = 3
> > 
> > class object2:
> >   print r.t
> > 
> > ---
> > 
> > Python gives me an error, saying it can't recognize global name r.
> > However I define r as global in the top-level main definition! Can
> > anyone suggest how I can get around this, if I want to define and bind
> > global names inside of main() which are valid in all sub-modules?
> > 
> > Thanks very much for your help!
> 
> Might be wrong, but globals can only be global to the module they're
> declared in.

Correct.

> 
> I suggest you find another way such as passing your object as a parameter

or make the third file and import it everywhere you need them...

- Hendrik

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Help on Dict

[Hendrik van Rooyen]

 "James Stroud" <[EMAIL PROTECTED]> wrote:


> Clement wrote:
> > Can any body tell how Dict is implemented in python... plz tell what
> > datastructure that uses
> > 
> 
> I think it uses a dict.

"Groan!" - this answer is like Microsoft documentation - 
while technically correct, it is useless...  : - )

The key of a dict entry is subjected to a bit of magic called 
a "Hashing algorithm" that yields a "bin" in which the data
is kept.

So to get hold of a dict entry always takes a similar time.

And its the quickest form of random access based on
a non integer key that you can get.

hth - Hendrik

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Convert to binary and convert back to strings

[Hendrik van Rooyen]

 "Paul Rubin" <http://[EMAIL PROTECTED]> wrote:

> "Hendrik van Rooyen" <[EMAIL PROTECTED]> writes:
> > s = 'some string that needs a bcc appended'
> > ar = array.array('B',s)
> > bcc = 0
> > for x in ar[:]:
> > bcc ^= x
> > ar.append(bcc)
> > s=ar.tostring()
> 
> Untested:
> 
> import operator
> s = 'some string that needs a bcc appended'
> ar = array.array('B',s)
> s += chr(reduce(operator.xor, ar))
> 

Yikes! - someday soon I am going to read the docs on
what reduce does...

Won't this be slow because of the double function call on each char?

- Hendrik

 

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Convert to binary and convert back to strings

[Hendrik van Rooyen]

 "Steven D'Aprano" <[EMAIL PROTECTED]> wrote:


> On Thu, 22 Feb 2007 08:18:07 +0200, Hendrik van Rooyen wrote:
>
> > I would xor each char in it with 'U' as a mild form of obfuscation...
>
> I've often wished this would work.
>
> >>> 'a' ^ 'U'
> Traceback (most recent call last):
>   File "", line 1, in ?
> TypeError: unsupported operand type(s) for ^: 'str' and 'str'
>
> instead of the more verbose
>
> >>> chr(ord('a') ^ ord('U'))
> '4'

you are not alone in this - to do something simple like calculating a BCC on a
string, or a checksum like at the end of a line in an Intel hex file is a bit of
a pain
in Python.

>
>
> > Look at the array module to get things you can xor, or use ord() on
> > each byte, and char()
>
> Arrays don't support XOR any more than strings do. What's the advantage to
> using the array module if you still have to jump through hoops to get it
> to work?

I think you will have less function calls, but I may be wrong:

s = 'some string that needs a bcc appended'
ar = array.array('B',s)
bcc = 0
for x in ar[:]:
bcc ^= x
ar.append(bcc)
s=ar.tostring()

>
> ''.join([chr(ord(c) ^ 85) for c in text])
>
> is probably about as simple as you can get.
>

This is nice and compact.

It would be very nice if you could just use a single char string like
an int and apply the operators to it - the Python way seems so left-
handed - make it an int, do the work, make it back into a string -
and all the time we are working on essentially a one byte value...

- Hendrik

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: is it possible to remove the ':' symbol in the end of linesstarting with 'if', 'while' etc?

[Hendrik van Rooyen]

 "Diez B. Roggisch" <[EMAIL PROTECTED]> wrote:

> [EMAIL PROTECTED] wrote:

8<  request to remove colon --

> Won't happen. There have been plenty of discussions about this, and while
> technically not necessary, the colon is usually considered "optically
> pleasing". So - it will stay. 

It is also used by text editors to make indentation of the next line happen...

- Hendrik

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Convert to binary and convert back to strings

[Hendrik van Rooyen]

 "Harlin Seritt" <[EMAIL PROTECTED]> wrote:


> Hi...
> 
> I would like to take a string like 'supercalifragilisticexpialidocius'
> and write it to a file in binary forms -- this way a user cannot read
> the string in case they were try to open in something like ascii text
> editor. I'd also like to be able to read the binary formed data back
> into string format so that it shows the original value. Is there any
> way to do this in Python?
> 

I would xor each char in it with 'U' as a mild form of obfuscation...

Look at the array module to get things you can xor, or use ord() on
each byte, and char()

Note that char(ord('U')) is 'U', and ord('U') is the equivalent of 0x55,
or five sixteens plus five - 85.

If its ascii just writing it out as binary is useless for what you want to do.

This will invert every alternate bit, producing apparent gibberish.

HTH - Hendrik

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Weird result returned from adding floats depending on order I add them

[Hendrik van Rooyen]

 "joanne matthews (RRes-Roth)" <[EMAIL PROTECTED]> wrote:

8<

> Can anyone tell me whats going on
> and how I can avoid the problem. Thanks

Don't know about the first question.

Would avoid it by using ints and asking for percentages...

- Hendrik


-- 
http://mail.python.org/mailman/listinfo/python-list

Re: f---ing typechecking

[Hendrik van Rooyen]

 "Nick Craig-Wood" <[EMAIL PROTECTED]> wrote:

> 
> Ie
> 
> x += a
> 
> does not equal
> 
> x = x + a
> 
> which it really should for all types of x and a

One would hope so , yes.

However, I think that the first form is supposed to update in place, 
while the second is free to bind a new thing to x

> 
> (That is the kind of statement about which I'm sure someone will post
> a perfectly reasonable counterexample ;-)
> 

I don't think its reasonable - its just an accident of implementation..

- Hendrik

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: How to test if one dict is subset of another?

[Hendrik van Rooyen]

 "Jay Tee" <[EMAIL PROTECTED]> wrote:

> Hi,
> 
> I have some code that does, essentially, the following:
> 
> - gather information on tens of thousands of items (in this case, jobs
> running on a
>  compute cluster)
> - store the information as a list (one per job) of Job items
> (essentially wrapped
>  dictionaries mapping attribute names to values)
> 
> and then does some computations on the data.  One of the things the
> code needs to do, very often, is troll through the list and find jobs
> of a certain class:
> 
> for j in jobs:
>if (j.get('user') == 'jeff' and j.get('state')=='running') :
>   do_something()
> 
> This operation is ultimately the limiting factor in the performance.
> What I would like to try, if it is possible, is instead do something
> like this:
> 

When you are gathering the data and building the lists - why do you not 
simultaneously build a dict of running jobs keyed by the Jeffs?

- Hendrik

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Help Required for Choosing Programming Language

[Hendrik van Rooyen]

 "Bruno Desthuilliers" <[EMAIL PROTECTED]> wrote:


>Stef Mientki a écrit :
>(snip)
>> I've been using Python for just 2 months, and didn't try any graphical
>> design,
>
>So how can you comment on GUI programming with Python ?

I think we have a language problem here (no pun intended)

When Stef says "Gui Programming" he means using something like
Delphi or Boa to do the Graphical Layout, while on this group it
normally means writing the python code to make your own windows
etc., using Tkinter or better...

There is no doubt that the first approach gets you going faster,
albeit true that you have more flexibility with the second.

The learning curves are also different, the first approach feeling less
painful, as you seem to make progress from the start, and you don't
have to worry about questions like : "whats a frame/toplevel/mainloop/etc.?"

So from Stef's perspective he is right when he claims that Python's
"Gui Programming" is poor - in the standard library it is non existent,
as there are no Delphi-, Glade- or Boa-like tools available.

And one can argue that something like Boa or the WX.. packages are
not Python, as they are not included in the standard library...

- Hendrik


-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Approaches of interprocess communication

[Hendrik van Rooyen]

 "exhuma.twn" <[EMAIL PROTECTED]> wrote:

> Hi all,
> 
> Supposing you have two separate processes running on the same box,
> what approach would you suggest to communicate between those two
> processes.

8< -- sockets,webservices,CORBA,shared memory ---

> Supposing both processes are written in Python, is there any other way
> to achieve this? To me, shared memory sound the most suited approach.
> But as said, I am still fuzzy in this area. Where can I find more
> information on this subject?

Using named pipes works for me

Also look at Pyro, albeit not aimed at being in the same box.

- Hendrik



-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Pep 3105: the end of print?

[Hendrik van Rooyen]

"Steve Holden" <[EMAIL PROTECTED]> wrote:

> Jean-Paul Calderone wrote:

> >
> > I think some people are confused that the language "Python 3.x" has "Python"
> > in its name, since there is already a language with "Python" in its name,
> > with which it is not compatible.
> >
> Right. Let's call Python 3.0 something different but related. How about
> "snake oil"? ;-)

I kind of like this idea - it means that the initiates can go on caravan tours
of America, selling it as a panacea for all ills, IT related or not...

A whole new industry.

- Hendrik

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: builtin set literal

[Hendrik van Rooyen]

 "Schüle Daniel" <[EMAIL PROTECTED]> wrote:


>
> > {:} for empty dict and {} for empty set don't look too much atrocious
> > to me.
>
> this looks consistent to me

I disagree. What would be consistent would be to follow the pattern,
and use a different set of delimiters.

Python uses () for tuples, [] for lists, {} for dictionaries.
To be consistent, sets need something else, so you can see at a
glance what you are dealing with.

About all that is left is <>, ugly as it is.

The other way is the start of the slippery slide into alphabet soup.

- Hendrik

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: builtin set literal

[Hendrik van Rooyen]

 "Paul Rubin"  wrote:



> Steven Bethard <[EMAIL PROTECTED]> writes:
> > Yes, a lot of people liked this approach, but it was rejected due to
> > gratuitous breakage. While Python 3.0 is not afraid to break backwards
> > compatibility, it tries to do so only when there's a very substantial
> > advantage. I guess enough people felt that having a shortcut for set()
> > was less important than keeping the current spelling of dict() the same.
>
> There's even a sentiment in some pythonistas to get rid of the [] and {}
> notations for lists and dicts, using list((1,2,3)) and dict((1,2),(3,4))
> for [1,2,3] and {1:2, 3:4} respectively.

YUK!

Moving in the wrong direction to bracketmania!

If you are going to use only one kind of brackets, use [] - on most keyboards,
you don't have to press the shift key - think of the numberless hours of total
time
saved by this simple reform...

It will also give Python a very distinctive "look" - unlike any other language.

- Hendrik

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: how do "real" python programmers work?

[Hendrik van Rooyen]

 "Tim Golden" <[EMAIL PROTECTED]> wrote:

> Tyrrell, Wendy wrote:
> 
> (Well, nothing)
> 
8<--
> Your organisation seems to deal with partnerships between
> business and education; are you looking to promote the use
> of programming in schools? Or is there something else you're
> after?
> 
> (I'm above-averagely curious because I work in Camden and
> I'm involved in Youth work in Ealing).

oh!  - The line has caught a fish.  And a fine one too...

- Hendrik

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Enter Enter... troubles

[Hendrik van Rooyen]

 Sorin Schwimmer  wrote:

8<-
>   def handler(self,event):
>self.aButton.unbind('')
>self.aButton.unbind('')
8<-
>
>The idea is to prevent a fast user (like my boss) to press repeatedly the
"enter"
>key and create havoc with self.property and localVar. But it doesn't work: my
>boss manages to start at least twice the handler (which, among other things
>creates a Toplevel, so I end up with a messy screen).

I would have separate handlers for the two keys - should be slightly faster..

I generally use configure to change the button's command to a do nothing thing
to sort out the multiple click maniacs.

hth - Hendrik

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Segmentation faults using threads

[Hendrik van Rooyen]

 "Mathias" <[EMAIL PROTECTED]> wrote:


> Does someone have experience with threading in python - are there 
> non-threadsafe functions I should know about?

how do your threads communicate with one another - are there any
globals that are accessed from different threads?

strange this - you should get an exception, not a segment fault if its
in the python bits..

- Hendrik


-- 
http://mail.python.org/mailman/listinfo/python-list

Re: searching a list of lists as a two-dimensional array?

[Hendrik van Rooyen]

"John Machin" <[EMAIL PROTECTED]> wrote:

> On Feb 13, 4:57 pm, "Hendrik van Rooyen" <[EMAIL PROTECTED]> wrote:
> >  "John Machin" <[EMAIL PROTECTED]> wrote:
> >
> > > Now for the algorithm: all of that testing to see if you are about to
> > > sail off the end of the world is a bit ugly and slow. You can use bit-
> > > bashing, as Paul suggested, even though it's on Steven D'Aprano's list
> > > of 6 deadly sins :-)
> >
> > Thou shallt not bit - bash
> >
> > What are the other five?
> 
> """
> ... regexes have their place, together with pointer arithmetic, bit
> manipulations, reverse polish notation and goto. The problem is when
> people use them inappropriately ...
> """
> 

I find this unsatisfactory - I was hoping for a five commandment
style of thing, setting out proper prohibitions against evil stuff.

It seems that I am a closet fundamentalist.

A sixth, btw is:

Thou shallt not use globals.

- Hendrik



-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Help with Optimization of Python software: real-time audiocontroller

[Hendrik van Rooyen]

"Jean-Paul Calderone" <[EMAIL PROTECTED]> wrote:

> Yep.  There are even some existing Python applications which deal with
> sound and manage to work with ~20ms samples.

I agree. Python is not *that* slow...

I have written serial port communications that send and catch one
character at a time (raw, unbuffered i/o) - and they seem to work 
well at 38400 baud, sending and receiving almost four characters
a millisecond, continuously, back to back,  without any stress

So to "get there" every twenty millis should not be a problem,
unless the machine is a real dog. (i.e. older than seven years)

Would not try to handle individual samples at 44 KiloHerz in 
python though...

- Hendrik

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Tkinter and Tile

[Hendrik van Rooyen]

"Eric Brunel" <[EMAIL PROTECTED]> wrote:

FYI, changes done in tcl/tk are usually quite rapidly integrated in
Tkinter. For example, for the "panedwindow" widget, introduced in tk8.4
(first version out in the end of 2002), a Tkinter wrapper was available in
Python 2.3 (first version out mid-2003). So I don't doubt that the Tile
extension package will be integrated in Tkinter, as soon as it is
available in an official tcl/tk release.

Do you imagine that the bizarre behaviour of  "ButtonRelease" will be fixed too?

- Hendrik


-- 
http://mail.python.org/mailman/listinfo/python-list

Re: searching a list of lists as a two-dimensional array?

[Hendrik van Rooyen]

 "John Machin" <[EMAIL PROTECTED]> wrote:

> Now for the algorithm: all of that testing to see if you are about to
> sail off the end of the world is a bit ugly and slow. You can use bit-
> bashing, as Paul suggested, even though it's on Steven D'Aprano's list
> of 6 deadly sins :-)

Thou shallt not bit - bash

What are the other five?

- Hendrik


-- 
http://mail.python.org/mailman/listinfo/python-list

Re: pygame and python 2.5

[Hendrik van Rooyen]

"Ben Sizer" <[EMAIL PROTECTED]> wrote:


> On Feb 10, 8:42 am, Steve Holden <[EMAIL PROTECTED]> wrote:
> > Hendrik van Rooyen wrote:
> > > <[EMAIL PROTECTED]> wrote:
> > > "Ben Sizer" <[EMAIL PROTECTED]> wrote:
> >
> > >> Ben> Python extensions written in C require recompilation for each
new
> > >> Ben> version of Python, due to Python limitations.
> >
> > >> Can you propose a means to eliminate this limitation?
> >
> > > Yes.   - Instead of calling something, send it a message...
> >
> > I suppose you are proposing to use the ISO 1.333 generic
> > message-passing interface for this? The one that doesn't actually call a
> > function to pass a message?
>
> I'm assuming you're being facetious here..?

Please see my reply to Steve - and Yes, I believe he was oulling the oiss...

>
> Of course, functions get called at the ends of the message passing
> process, but those functions can stay the same across versions while
> the messages themselves change. The important part is reducing the
> binary interface between the two sides to a level where it's trivial
> to guarantee that part of the equation is safe.
>
> eg.
> Instead of having PySomeType_FromLong(long value) exposed to the API,
> you could have a PyAnyObject_FromLong(long value, char*
> object_type_name). That function can return NULL and set up an
> exception if it doesn't understand the object you asked for, so Python
> versions earlier than the one that implement the type you want will
> just raise an exception gracefully rather than not linking.
>
> The other issue comes with interfaces that are fragile by definition -
> eg. instead of returning a FILE* from Python to the extension,  return
> the file descriptor and create the FILE* on the extension side with
> fdopen.

This sort of thing is exactly what is wrong with the whole concept of
an API...
Its very difficult, if not impossible, to guarantee that *my stuff* and
*your stuff* will work together over time.

Whereas if *my stuff* just publishes a message format, *anything* that
can make up the message can interact with it - but it requires *my stuff*
to be independently executable, and it needs a message passing
mechanism that will stand the test of time.

And it can create a whole new market of "Mini Appliances" each of
which has *your stuff* inside them...

- Hendrik


-- 
http://mail.python.org/mailman/listinfo/python-list

Re: pygame and python 2.5

[Hendrik van Rooyen]

"Steve Holden" <[EMAIL PROTECTED]>

> Hendrik van Rooyen wrote:
> > <[EMAIL PROTECTED]> wrote:
> > "Ben Sizer" <[EMAIL PROTECTED]> wrote:
> > 
> > 
> >> Ben> Python extensions written in C require recompilation for each new
> >> Ben> version of Python, due to Python limitations.
> >>
> >> Can you propose a means to eliminate this limitation?
> >>
> > 
> > Yes.   - Instead of calling something, send it a message...
> > 
> I suppose you are proposing to use the ISO 1.333 generic 
> message-passing interface for this? The one that doesn't actually call a 
> function to pass a message?
> 

Actually I am not aware that this ISO standard exists.

My feeling about ISO standards in general are such that 
I would rather have *anything* else than an ISO standard.
These feelings are mainly caused by the frustration of trying
to decipher standards written in standardese, liberally sprinkled
with non standard acronyms...

Its very interesting to learn that you can pass a message without
doing a call - when I next need to entertain a children's party as a
magician I will endeavour to incorporate it into my act.  Thanks 
for the tip.

But more seriously, the concept is that you should couple
as loosely as possible, to prevent exactly the kind of trouble
that this thread talks about.  Just as keyword parameters are
more robust than positional parameters, the concept of doing
something similar to putting a dict on a queue, is vastly more 
future-proof than hoping that your call will *get through*
when tomorrow's compiler buggers around with the calling 
convention.

One tends to forget that calling also builds messages on 
the stack.

When you boil it right down, all you need for a minimalistic
interface are four message types:

- get something's value
- set something to a value
- return the requested value
- do something  

This is not the most minimalistic interface, but its a nice 
compromise.

The advantages of such loose coupling are quite obvious when 
you think about them, as you don't care where the worker sits -
on the other side of the world over the internet, or in the same 
room in another box, or in the same box on another processor, 
or on the same processor in another process, or in the same 
process in another thread...

The problem with all this, of course, is the message passing 
mechanism, as it is not trivial to implement something that
will address all the cases. A layered approach (ISO ?  ; - )  )
could do it...

But Skip asked how to sort it, and this would be 
My Way.   (TM circa 1960 F Sinatra)

- Hendrik


-- 
http://mail.python.org/mailman/listinfo/python-list

Re: pygame and python 2.5

[Hendrik van Rooyen]

<[EMAIL PROTECTED]> wrote:
"Ben Sizer" <[EMAIL PROTECTED]> wrote:


> Ben> Python extensions written in C require recompilation for each new
> Ben> version of Python, due to Python limitations.
> 
> Can you propose a means to eliminate this limitation?
> 

Yes.   - Instead of calling something, send it a message...

- Hendrik


-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Calling J from Python

[Hendrik van Rooyen]

 "Dennis Lee Bieber" <[EMAIL PROTECTED]> wrote:

> On Thu, 8 Feb 2007 10:55:17 +0200, "Hendrik van Rooyen"
> <[EMAIL PROTECTED]> declaimed the following in comp.lang.python:
> 
> >  I am under the impression that Loki had a daughter called Hel ...
> >
> One of his few "normal" offspring... After all, Loki also was
> Sleipnir's* /mother/! And probably related to Fenrir (aka, the Fenris
> Wolf) as well.
> 
> 
> * Odin's eight-legged horse

We should both be more careful - using the past tense like that is 
disrespectful, and there are no guarantees that the Norse gods are dead.

Vindictive lot, they were..   uuhhh are, I mean.

- Hendrik

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Simple Interpolation in Numpy?

[Hendrik van Rooyen]

 LAPI, VINCENT J, ATTLABS  wrote:


>Hi,
>Please bear with me as I am new to Python and have not done any programming in
about 20 years. I am >attempting to do a simple interpolation of  a line's
intermediate points given the x,y coordinates of the line's two >endpoints
within an Active State Python script that I am working with. Is there a simple
way to do this simple >interpolation in the Active state Python 2.4 that I have
or do I need to get Numeric Python? And where do I get >it?

You don't need anything fancy, the bulk standard python should do this:

untested:

ep0 = (x0,y0)# known endpoints
ep1 = (x1,y1)

dx = x1-x0# calculate span
dy = y1-y0

y = y0 + dy*(x-x0)/dx# to give y from known x

x = x0 + dx*(y-y0)/dy# to give x from known y

The interactive interpreter is your friend.
(you activate it by typing "Python" at the command prompt)
(don't type the quotes..)
Use it for playing with this kind of stuff.

And other stuff too - it answers most questions of the:

"can I do this? "  and
"what happens when I ..."   variety.

Also read the tutorial - if you have done programming
twenty years ago, you will take to it like a duck to water.

It really works much easier than a teletype...

hth - Hendrik

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Calling J from Python

[Hendrik van Rooyen]

"Tina I" <[EMAIL PROTECTED]> wrote:


Gosi wrote:
> On Feb 7, 3:46 pm, Marc 'BlackJack' Rintsch <[EMAIL PROTECTED]> wrote:
>> In <[EMAIL PROTECTED]>, Gosi wrote:
>>> I like to use J for many things and I think that combining Python and
>>> J is a hell of a good mixture.
>> I was able to follow this sentence up to and including the word "hell"...
:-)
>>
>> Ciao,
>> Marc 'BlackJack' Rintsch
>
>
> That is a start.
>
> "Hell" is also what most example start with as in "Hello something"
> Hell in northern countries is very cold.
> Hell in middle east is very hot.
> I do not know which is your Hell hot or cold.
> Hell o veröld
>
It's also a village in Norway: http://en.wikipedia.org/wiki/Hell,_Norway
:D

 I am under the impression that Loki had a daughter called Hel ...

- Hendrik


-- 
http://mail.python.org/mailman/listinfo/python-list

Re: The reliability of python threads

[Hendrik van Rooyen]

 "Carl J. Van Arsdall" <[EMAIL PROTECTED]> wrote:
> Hendrik van Rooyen wrote:
> >  "Carl J. Van Arsdall" <[EMAIL PROTECTED]> wrote:

8< ---

> >   
> Yea, I do some of that too.  I use that with conditional print 
> statements to stderr when i'm doing my validation against my test 
> cases.  But I could definitely do more of them.  The thing will be 

When I read this - I thought - probably your stuff is working 
perfectly - on your test cases - you could try to send it some
random data and to see what happens - seeing as you have a test 
server, throw the kitchen sink at it.

Possibly "random" here means something that "looks like" data
but that is malformed in some way. Kind of try to "trick" the 
system to get it to break reliably.

I'm sorry I can't be more specific - it sounds so weak, and you
probably already have test cases that "must fail" but I don't 
know how to put it any better...

- Hendrik


-- 
http://mail.python.org/mailman/listinfo/python-list

Re: The reliability of python threads

[Hendrik van Rooyen]

 "Carl J. Van Arsdall" <[EMAIL PROTECTED]> wrote:

> Right, I wasn't coming here to get someone to debug my app, I'm just 
> looking for ideas.  I constantly am trying to find new ways to improve 
> my software and new ways to reduce bugs, and when i get really stuck, 
> new ways to track bugs down.  The exception won't mean much, but I can 
> say that the error appears to me as bad data.  I do checks prior to 
> performing actions on any data, if the data doesn't look like what it 
> should look like, then the system flags an exception.
> 
> The problem I'm having is determining how the data went bad.  In 
> tracking down the problem a couple guys mentioned that problems like 
> that usually are a race condition.  From here I examined my code, 
> checked out all the locking stuff, made sure it was good, and wasn't 
> able to find anything.  Being that there's one lock and the critical 
> sections are well defined, I'm having difficulty.  One idea I have to 

Are you 100% rock bottom gold plated guaranteed sure that there is
not something else that is also critical that you just haven't realised is?

This stuff is never obvious before the fact - and always seems stupid
afterward, when you have found it.  Your best (some would say only)
weapon is your imagination, fueled by scepticism...

> try and get a better understanding might be to check data before its 
> stored.  Again, I still don't know how it would get messed up nor can I 
> reproduce the error on my own. 
> 
> Do any of you think that would be a good practice for trying to track 
> this down? (Check the data after reading it, check the data before 
> saving it)

Nothing wrong with doing that to find a bug - not as a general 
practice, of course - that would be too pessimistic.

In hard to find bugs - doing anything to narrow the time and place
of the error down is fair game - the object is to get you to read
some code that you *know works* with new eyes...

I build in a global boolean variable that I call trace, and when its on
I do all sort of weird stuff, giving a running commentary (either by
print or in some log like file) of what the programme is doing, 
like read this, wrote that, received this, done that here, etc.
A bare useful minimum is a "we get here" indicator like the routine
name, but the data helps a lot too.

Compared to an assert, it does not stop the execution, and you
could get lucky by cross correlating such "traces" from different
threads. - or better, if you use a queue or a pipe for the "log", 
you might see the timing relationships directly.

But this in itself is fraught with danger, as you can hit file size 
limits, or slow the whole thing down to unusability.

On the other hand it does not generate the volume that a genuine 
trace does, it is easier to read, and you can limit it to the bits that
you are currently suspicious of.

Programming is such fun...

hth - Hendrik



-- 
http://mail.python.org/mailman/listinfo/python-list

Re: OT Annoying Habits (Was: when format strings attack)

[Hendrik van Rooyen]

"Steven D'Aprano" <[EMAIL PROTECTED]> wrote:

> Or perhaps I should say:
> 
> .snoitnevnoc
> hsilgnE tpada )ylbissop revenehw( dluohs ew os dna ,naitraM ton ,puorgswen
> egaugnal hsilgnE na no er'ew ,segaugnal hcus era ereht fi neve tuB

First I thought it was Welsh or Cornish or something.

Then it was like being in my first year of school again-
reading letter by letter.  Never realised how difficult it is.

I suppose it will improve with practice.

- Hendrik

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: How can I create a linked list in Python?

[Hendrik van Rooyen]

"Jorgen Grahn" <[EMAIL PROTECTED]> wrote:

> 
> FWIW, I oppose the idea (paraphrased from further up the thread) that linked
> lists and other data structures are obsolete and dying concepts, obsoleted
> by Python and other modern languages.
> 
> 99% of the time. a Python list is the right tool for the job, but that's
> only because we have CPU cycles to spare and the 'n' in our 'O(n)' is
> limited. You cannot call yourself a computer scientist without understanding
> things like linked lists.  No other data structure has the same
> characteristics (good and bad) as that one. Or those two, really.

+1

The concept of storing a pointer that points to the "next thing" is so basic
that it will never go away.  One meets it all time in chained buffer blocks, 
in tag sorts, etc...

And if you add a pointer to the "previous thing" too, then adding or taking
something out of what could become a ring is a constant effort.  Until you
run out of memory.

Ye Olde Universal Controlle Blocke:

- pointer to the start of data block
- length of allocated data block
- pointer to next control block
- pointer to previous control block
- next in pointer into data block
- next out pointer into data block
- optional length of data
- optional here starts or ends the lesson indicator

errrm... thats about it, unless you want a fast index too:

- pointer to first control block
- pointer to second control block
- pointer to third control block
...

Isn't it nice of python to hide all this stuff?

- Hendrik


-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Would a Dutch speaker please check this wiki page please?

[Hendrik van Rooyen]

"Martin P. Hellwig" <[EMAIL PROTECTED]> wrote:


> Stef Mientki wrote:
> > [EMAIL PROTECTED] wrote:
> >> Got a note about a new page on the Python Wiki:
> >>
> >>> "Wade" == Wade McDaniel <[EMAIL PROTECTED]> writes:
> >>
> >> http://wiki.python.org/moin/Selcuk_Altun
> >>
> >> I suspect it's junk since it doesn't seem to mention Python and the 
> >> website
> >> it mentions doesn't seem to exist.  Still, just in case...
> > The only (incorrect) Dutch sentence on the whole site is:
> >   "Welkom op me site www.keriwar.nl",
> > which should have been
> >   "Welkom op mijn site www.keriwar.nl"
> > but that could be explained, while his name "Selcuk Altun" is not 
> > orginated from the Netherlands.
> > I just expect that he is trying to setup a wiki,
> > but that's too soon to give it a judgement.
> > 
> > cheers,
> > Stef Mientki
> >>
> >> Thx,
> >>
> >> Skip
> 
> Could be Afrikaans too, but the page is gone now so I can't check.

No its not Afrikaans - the "me" would have had to be "my" and "site"
would have been "webwerf" - literally "web yard"...

Some Flemish dialect, perhaps?

- Hendrik



-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Class data members in C

[Hendrik van Rooyen]

"Nick Maclaren" <[EMAIL PROTECTED]> wrote:


> 
> In article <[EMAIL PROTECTED]>,
> "Hendrik van Rooyen" <[EMAIL PROTECTED]> writes:
> |> > 
> |> > Hmm.  The extensions documentation describes how to add instance
> |> > members to a class (PyMemberDef), but I want to add a class member.
> |> > Yes, this is constant for all instances of the class.
> |> 
> |> When? - at time of defining base class, between class definition and
> |> first instance, at time of creating instance, or after instance creation?
> |> 
> |> After instance creation is not easy, I think you would have to add 
> |> to each instance... 
> 
> Oh, one of the first two - I am not bonkers!  Changing a class after
> instance creation is guaranteed to cause confusion, if nothing else.
> 
*grin* - its also just about impossible to do, if you try to imagine
what you would have to do to all instances when you effectively
change the __init__ method.  And some people expect it to happen
automagically...

Sorry I can't be of more help. All of the docs I have read essentially
talk about changing instances.  From a practical point of view I would 
say: "Just redefine it".  But that is not very helpful. Specially if you are
mucking about in C, as your title suggests...



It has never struck me before that the root word for
"madness" and the slang for "copulation" is the same.
I wonder if there is some sort of a message here?



- Hendrik

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: asyncore/asynchat and terminator string

[Hendrik van Rooyen]

 "David Hirschfield" <[EMAIL PROTECTED]> wrote:

8< --- problems on syncing up in serial comms -

I have seen people address this with success by using stuff like:

"XXHEADERXX" as a "here starts the lesson" identifier, with no 
trouble, on a high volume newsfeed.

If you assume that any binary value is possible and equally likely, 
then the problem looks hopeless, but in practice if you do this kind 
of thing, followed by a length, which automatically points to the 
next "instance" of the "here starts the lesson" in a stream, then 
it becomes extremely unlikely to occur in the wild.  If you also
repeat the length at the end, then you can "scan Backwards" 
through the stream. And if its not like that, its a "false sync"...

The other way is to take a leaf out of the bit oriented protocols'
book, and to "frame" the "packets" between (possibly repetitious)
occurrences of a character that is guaranteed not to occur in the
data, known as a "flag" character.

You do this by a process that is called "escaping" the occurrences 
of the flag character in the data with yet another "escape char", 
that also needs special treatment if it happens in the data...

Escaping can be accomplished by replacing each instance of the
"Poison Characters" by an instance of the escape char, followed 
by the bitwise inversion of the poison char.  Unescaping has to
do the reverse.

Using a tilde "~" as a flag char, and the ordinary slash as an 
escape char works well.

So a packet with both poison chars in it will look like this:

"~~ordinary chars followed by tilde" + "/\x81"+
"ordinary followed by slash" +"/\xd0" + "somestuff~~"

So you sync up by looking for a tilde followed by a 
non tilde, and the end is when you hit a tilde again.

To unescape, you look for slashes and invert the chars
following them.

Also Yahoo for "netstrings"

If you want to be Paranoid, then you also implement message
numbers, to make sure you don't lose packets, and for hyper
paranoia, put in a BCC or LRC to make sure what you send
off is received...

If you are running over Ethernet, the last lot is not warranted,
as its done anyway - but on a serial port, where you are on 
your own, it makes sense.

HTH - Hendrik

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Class data members in C

[Hendrik van Rooyen]

"Nick Maclaren" <[EMAIL PROTECTED]> wrote:


> 
> Hmm.  The extensions documentation describes how to add instance
> members to a class (PyMemberDef), but I want to add a class member.
> Yes, this is constant for all instances of the class.
> 
> Any pointers?

When? - at time of defining base class, between class definition and
first instance, at time of creating instance, or after instance creation?

After instance creation is not easy, I think you would have to add 
to each instance... 

But then, I may be wrong - I often am.

- Hendrik

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Maths error

[Hendrik van Rooyen]

"Nick Maclaren" <[EMAIL PROTECTED]> wrote:

[Tim Roberts]
> |> Actually, this is a very well studied part of computer science called
> |> "interval arithmetic".  As you say, you do every computation twice, once to
> |> compute the minimum, once to compute the maximum.  When you're done, you
> |> can be confident that the true answer lies within the interval.

>
> The problem with it is that it is an unrealistically pessimal model,
> and there are huge classes of algorithm that it can't handle at all;
> anything involving iterative convergence for a start.  It has been
> around for yonks (I first dabbled with it 30+ years ago), and it has
> never reached viability for most real applications.  In 30 years, it
> has got almost nowhere.
>
> Don't confuse interval methods with interval arithmetic, because you
> don't need the latter for the former, despite the claims that you do.
>
> |> For people just getting into it, it can be shocking to realize just how
> |> wide the interval can become after some computations.
>
> Yes.  Even when you can prove (mathematically) that the bounds are
> actually quite tight :-)

This sounds like one of those pesky:
"but you should be able to do better" - kinds of things...

- Hendrik

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Maths error

[Hendrik van Rooyen]

"Dennis Lee Bieber" <[EMAIL PROTECTED]>wrote:


> On Sun, 14 Jan 2007 07:18:11 +0200, "Hendrik van Rooyen"
> <[EMAIL PROTECTED]> declaimed the following in comp.lang.python:
> 
> > 
> > I recall an SF character known as "Slipstick Libby",
> > who was supposed to be a Genius - but I forget
> > the setting and the author.
> >
> Robert Heinlein. Appears a few of the Lazarus Long books.
>  
> > It is something that has become quietly extinct, and
> > we did not even notice.
> >
> And get collector prices --
> http://www.sphere.bc.ca/test/sruniverse.html

Thanks Dennis - Fascinating site !

- Hendrik

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Maths error

[Hendrik van Rooyen]

 "Tim Peters" <[EMAIL PROTECTED]> wrote:


> [Nick Maclaren]
> >> ...
> >> Yes, but that wasn't their point.  It was that in (say) iterative
> >> algorithms, the error builds up by a factor of the base at every
> >> step. If it wasn't for the fact that errors build up, almost all
> >> programs could ignore numerical analysis and still get reliable
> >> answers! 
> >>
> >> Actually, my (limited) investigations indicated that such an error
> >> build-up was extremely rare - I could achieve it only in VERY
> >> artificial programs.  But I did find that the errors built up faster
> >> for higher bases, so that a reasonable rule of thumb is that 28
> >> digits with a decimal base was comparable to (say) 80 bits with a
> >> binary base. 
> 
> [Hendrik van Rooyen]
> > I would have thought that this sort of thing was a natural consequence 
> > of rounding errors - if I round (or worse truncate) a binary, I can be 
> > off by at most one, with an expectation of a half of a least 
> > significant digit, while if I use hex digits, my expectation is around
> > eight, and for decimal around five... 
> 
> Which, in all cases, is a half ULP at worst (when rounding -- as 
> everyone does now).
> 
> > So it would seem natural that errors would propagate 
> > faster on big base systems, AOTBE, but this may be 
> > a naive view.. 
> 
> I don't know of any current support for this view.  It the bad old days, 
> such things were often confused by architectures that mixed non-binary 
> bases with "creative" rounding rules (like truncation indeed), and it 
> could be hard to know where to "pin the blame".
> 
> What you will still see stated is variations on Kahan's telegraphic 
> "binary is better than any other radix for error analysis (but not very 
> much)", listed as one of two techincal advantages for binary fp in:
> 
> http://www.cs.berkeley.edu/~wkahan/MktgMath.pdf
> 
> It's important to note that he says "error analysis", not "error 
> propagation" -- regardless of base in use, rounding is good to <= 1/2 
> ULP.  A fuller elementary explanation of this can be found in David 
> Goldberg's widely available "What Every Computer Scientist Should Know 
> About Floating-Point", in its "Relative Error and Ulps" section.  The 
> short course is that rigorous forward error analysis of fp algorithms is 
> usually framed in terms of relative error:  given a computed 
> approximation x' to the mathematically exact result x, what's the 
> largest possible absolute value of the mathematical
> 
>r = (x'-x)/x
> 
> (the relative error of x')?  This framework gets used because it's more-
> or-less tractable, starting by assuming inputs are exact (or not, in 
> which case you start by bounding the inputs' relative errors), then 
> successively computing relative errors for each step of the algorithm.  
> Goldberg's paper, and Knuth volume 2, contain many introductory examples 
> of rigorous analysis using this approach.
> 
> Analysis of relative error generally goes along independent of FP base.  
> It's at the end, when you want to transform a statement about relative 
> error into a statement about error as measured by ULPs (units in the 
> last place), where the base comes in strongly.  As Goldberg explains, 
> the larger the fp base the sloppier the relative-error-converted-to-ULPs 
> bound is -- but this is by a constant factor independent of the 
> algorithm being analyzed, hence Kahan's "... better ... but not very 
> much".  In more words from Goldberg:
> 
> Since epsilon [a measure of relative error] can overestimate the
> effect of rounding to the nearest floating-point number by the
> wobble factor of B [the FP base, like 2 for binary or 10 for
> decimal], error estimates of formulas will be tighter on machines
> with a small B.
> 
> When only the order of magnitude of rounding error is of interest,
> ulps and epsilon may be used interchangeably, since they differ by
> at most a factor of B.
> 
> So that factor of B is irrelevant to most apps most of the time.  For a 
> combination of an fp algorithm + set of inputs near the edge of giving 
> gibberish results, of course it can be important.  Someone using 
> Python's decimal implementation has an often very effective workaround 
> then, short of writing a more robust fp algorithm:  just boost the 
> precision.
> 

Thanks Tim, for taking the trouble. - really nice explanation.

My basic error of thinking ( ? - more like gut feel ) was that the
bigger bases somehow lose "more bits" at every round, 
forgetting that half a microvolt is still half a microvolt, whether
it is rounded in binary, decimal, or hex...

- Hendrik

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: General Question About Python

[Hendrik van Rooyen]

 "Torabisu" <[EMAIL PROTECTED]> wrote:


> 
> Hendrik van Rooyen wrote:

> > What do you want done? - I am only a thousand miles away...
> >
> 
> If I can just get my Python teleporter sorted out, distance will be no
> problem...  A little buggy at the moment though...  Poor John, I told
> him not to test it but oh well.
 
OMG!  - I have been feeding that unexpected heap of minced meat
to the cat...

- Hendrik

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Maths error

[Hendrik van Rooyen]

"Nick Maclaren" <[EMAIL PROTECTED]> wrote:

> The "cheap" means "cheap in hardware" - it needs very little logic,
> which is why it was used on the old, discrete-logic, machines.
> 
> I have been told by hardware people that implementing IEEE 754 rounding
> and denormalised numbers needs a horrific amount of logic - which is
> why only IBM do it all in hardware.  And the decimal formats are
> significantly more complicated.
> 
> What I don't know is how much precision this approximation loses when
> used in real applications, and I have never found anyone else who has
> much of a clue, either.
> 
I would suspect that this is one of those questions which are simple
to ask, but horribly difficult to answer - I mean - if the hardware has 
thrown it away, how do you study it - you need somehow two
different parallel engines doing the same stuff, and comparing the 
results, or you have to write a big simulation, and then you bring 
your simulation errors into the picture - There be Dragons...

- Hendrik

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Maths error

[Hendrik van Rooyen]

 "Dennis Lee Bieber" <[EMAIL PROTECTED]> wrote:


> {My 8th grade teacher was a bit worried at seeing me with a slipstick
> ; and my HighSchool Trig/Geometry teacher only required 3 significant
> digits for answers -- even though half the class had calculators by
> then}

LOL - I haven't seen the word "slipstick" for yonks...

I recall an SF character known as "Slipstick Libby",
who was supposed to be a Genius - but I forget
the setting and the author.

It is something that has become quietly extinct, and
we did not even notice.

We should start a movement for reviving them -
on grounds of their "greenness" - they use no
batteries...

Fat chance.

- Hendrik


-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Is there a way to protect a piece of critical code?

[Hendrik van Rooyen]

 "robert" <[EMAIL PROTECTED]> wrote:


> Hendrik van Rooyen wrote:

> > So far I have only used dicts to pass functions around
> > in a relatively unimaginative static jump table like way...
> 
> 
> Probably one has just to see that one can a pass a function object 
> (or any callable) around as any other object.
> Similar to a function address in assembler/C but very comfortable 
> and with the comfort of closures (which automatically hold the 
> status of local variables):
> 
> def f():
>  print "hello"
> 
> def g(func):
>  print "I'll do it ..."
>  func()
>  print "done."
> 
> 
> def run(x):
>  g(f)
>  a="local variable\n"
>  def h():
>  b="inner local"
>  print "inner function"
>  print x,a,b
>  g(h)
>  g(lambda:sys.stdout.write(a))
> 
> run(1)
> 
>  From there its just natural to not pass dead objects through an 
> inter-thread queue, but just code as it or even a "piece of 
> critical code" ...
> A small step in thought, but a big step in effect - soon 
> eliminating bunches of worries about queues, pop-races/None 
> objects, protocol, serialization, critical sections, thousands of 
> locks etc.

Thanks - this simplicity takes a bit of getting used to - I am used 
to passing what are effective entry point pointers around while
building state machines that run under interrupt - effectively
dynamically changing vectors for ticker based routines.  But
on the low level, trying to pass data at the same time is a bit
of a pain, as you have to look after it yourself, so the 
temptation is great to only use globals...  
This passing_the_function_along_with_its_data is neat...

*sigh* now if only it were possible to do it over a serial link,
in the same way as through a queue (I know about Pyro, but
this does not seem quite the same thing )

I suppose its not really possible - because while I suspect only 
"pointers" get passed through say a queue, you would have to
send the actual code along as well over a link - or you need a 
mirror of the code on both sides, and a way to translate 
addresses - which I suppose is why Pyro looks like it does.

I am working on something similar on a *very* small scale 
at the moment, in the context of control of physical things.
Hence the sigh.

- Hendrik

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Rational Numbers

[Hendrik van Rooyen]

From: "Nick Maclaren" <[EMAIL PROTECTED]> wrote:

> Financial calculations need decimal FIXED-point, with a precisely
> specified precision.  It is claimed that decimal FLOATING-point
> helps with providing that, but that claim is extremely dubious.
> I can explain the problem in as much detail as you want, but would
> very much rather not.

Ok I will throw in a skewed ball at this point - use integer arithmetic,
and work in tenths of cents or pennies or whatever, and don't be too 
lazy to do your own print formatting...

To represent $121.23 as a float is just asking for trouble when you
are trying to deal with something that can take a value like:

$123 456 789 012 345 678 901.07  - and you are worried about the 
seven cents for accounting purposes...

Remember that its not only money that is measured like that, but 
debt too...

-Hendrik

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Maths error

[Hendrik van Rooyen]

 "Nick Maclaren" <[EMAIL PROTECTED]> wrote:
>
> In article <[EMAIL PROTECTED]>,
> "Hendrik van Rooyen" <[EMAIL PROTECTED]> writes:
> |>
> |> I would have thought that this sort of thing was a natural consequence
> |> of rounding errors - if I round (or worse truncate) a binary, I can be off
> |> by at most one, with an expectation of a half of a least significant digit,
> |> while if I use hex digits, my expectation is around eight, and for decimal
> |> around five...
> |>
> |> So it would seem natural that errors would propagate
> |> faster on big base systems, AOTBE, but this may be
> |> a naive view..
>
> Yes, indeed, and that is precisely why the "we must use binary" camp won
> out.  The problem was that computers of the early 1970s were not quite
> powerful enough to run real applications with simulated floating-point
> arithmetic.  I am one of the half-dozen people who did ANY actual tests
> on real numerical code, but there may have been some work since!

*grin* - I was around at that time, and some of the inappropriate habits
almost forced by the lack of processing power still linger in my mind,
like - "Don't use division if you can possibly avoid it, - its EXPENSIVE!"
- it seems so silly nowadays.

>
> Nowadays, it would be easy, and it would make quite a good PhD.  The
> points to look at would be the base and the rounding rules (including
> IEEE rounding versus probabilistic versus last bit forced[*]).  We know
> that the use or not of denormalised numbers and the exact details of
> true rounding make essentially no difference.
>
> In a world ruled by reason rather than spin, this investigation
> would have been done before claiming that decimal floating-point is an
> adequate replacement for binary for numerical work, but we don't live
> in such a world.  No matter.  Almost everyone in the area agrees that
> decimal floating-point isn't MUCH worse than binary, from a numerical
> point of view :-)
>

As an old slide rule user - I can agree with this - if you know the order
of the answer, and maybe two points after the decimal, it will tell you
if the bridge will fall down or not. Having an additional fifty decimal
places of accuracy does not really add any real information in these
cases.  Its nice of course if its free, like it has almost become - but
I think people get mesmerized by the numbers, without giving any
thought to what they mean - which is probably why we often see
threads complaining about the "error" in the fifteenth decimal place..
>
> [*] Assuming signed magnitude, calculate the answer truncated towards
> zero but keep track of whether it is exact.  If not, force the last
> bit to 1.  An old, cheap approximation to rounding.
>
This is not so cheap - its good solid reasoning in my book -
after all, "something" is a lot more than "nothing" and should
not be thrown away...

- Hendrik

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: General Question About Python

[Hendrik van Rooyen]

"Torabisu" <[EMAIL PROTECTED]> wrote:


> 
> billie wrote:
> > Torabisu wrote:
> >
> > > Its quite weird, we're looking for Python skills but are battling to
> > > find at the moment...  Normally Python on its own will probably not
> > > land you a job, but the last two companies I've worked for are doing
> > > indepth Python development, so hopefully the tables are turning a bit.
> >
> > I can tell the same for Italy.
> > Where are you from?
> 
> Cape Town, South Africa.  There is a strong Unix following here, but
> more on the System Administration side, not really the software
> development side with regards to Python.  Python more being used for
> task automation and shell scripting.

What do you want done? - I am only a thousand miles away...

- Hendrik

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: maximum number of threads

[Hendrik van Rooyen]

 "William Heymann" <[EMAIL PROTECTED]> wrote:


> So you know I tried this on ubuntu edgy 64bit edition on a dual 2218 opteron 
> system with 8G of ram and I got
> 
> 
> Exception raised: can't start new thread
> Biggest number of threads: 32274
> 

This almost looks as if the number of threads is a sixteen bit signed int... 

- Hendrik

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Maths error

[Hendrik van Rooyen]

"Nick Maclaren" <[EMAIL PROTECTED]> wrote:

> Yes, but that wasn't their point.  It was that in (say) iterative
> algorithms, the error builds up by a factor of the base at every step.
> If it wasn't for the fact that errors build up, almost all programs
> could ignore numerical analysis and still get reliable answers!
> 
> Actually, my (limited) investigations indicated that such an error
> build-up was extremely rare - I could achieve it only in VERY artificial
> programs.  But I did find that the errors built up faster for higher
> bases, so that a reasonable rule of thumb is that 28 digits with a decimal
> base was comparable to (say) 80 bits with a binary base.
> 

I would have thought that this sort of thing was a natural consequence
of rounding errors - if I round (or worse truncate) a binary, I can be off
by at most one, with an expectation of a half of a least significant digit,
while if I use hex digits, my expectation is around eight, and for decimal
around five...

So it would seem natural that errors would propagate 
faster on big base systems, AOTBE, but this may be 
a naive view.. 

- Hendrik

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Is there a way to protect a piece of critical code?

[Hendrik van Rooyen]

"robert" <[EMAIL PROTECTED]> wrote:


> pushing data objects through an inter-thread queue is a major source for
trouble - as this thread shows again.
> Everybody builds up a new protocol and worries about Empty/Full,
Exception-handling/passing, None-Elements, ...
> I've noticed that those troubles disappear when a functional queue is used -
which is very easy with a functional language like Python.
> For example with
http://aspn.activestate.com/ASPN/Cookbook/Python/Recipe/491281
>
>
> One would just use a  cq=CallQueue()
>
> On the producer side one would just write the functional code one wants to
execute in a target thread:
>
> cq.call( what_i_want_do_func )
>
>
> The consumer/receiver thread would just do (periodically) a non-blocking
>
> cq.receive()
>
>
> => Without any object fumbling, protocol worries and very fast.
>
> And note: This way - working with functional jobs - one can also "protect a
piece of critical code" most naturally and specifically for certain threads
without spreading locks throughout the code.
> Even things which are commonly claimed "forbidden" (even when using lots of
locks) can be magically done in perfect order and effectively this way. Think of
worker threads doing things in the GUI or in master / database owner threads
etc.
>
> Similarly discrete background thread jobs can be used in a functional style
this way:
> http://aspn.activestate.com/ASPN/Cookbook/Python/Recipe/491280
> ( an alternative for the laborious OO-centric threading.Thread which mostly is
a lazy copy from Java )
> or for higher job frequencies by using "default consumer threads" as also
shown in the 1st example of
> http://aspn.activestate.com/ASPN/Cookbook/Python/Recipe/491281
>

Thank you - had a (very) quick look and I will return to it
later - It is not immediately obvious to my assembler
programmer's mentality - looks like in the one case
the thread starts up, does its job and then dies, and in
the other its a sort of "remote" daemon like engine,
that you can "tell what to do", from "here"...

Both concepts seem nice and I will try to wrap my head
around them properly.

So far I have only used dicts to pass functions around
in a relatively unimaginative static jump table like way...

Thanks.

- Hendrik

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Urgent Openings with the World's Leading Software Company !!!

[Hendrik van Rooyen]

"Software Hiring" <[EMAIL PROTECTED]> wrote:


> If you're interested in relocating to India for such a good
> opportunity and be a part of the World's Best Software Company and if
> you have good skill sets  in:
> 
> C, C++, C#, Java (for .Net Platform or Visual Studio)
> 
> OR
> 
> C, C++, Java, Product Development (for MS RFID Infrastructure
> PathFinder - Adapters)
> 
> OR
> 
> C, C++, driver or kernel, Linux or Unix (for Windows Serviceability)
> 

not much to do with python

*closes eyes, and imagines the sing-song accent of the Indians
contrasting with his guttural Afrikaans accent *

*shudders*

I think I will give this one a miss - it would be worse than
relocating to Wales...

- Hendrik

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Is there a way to protect a piece of critical code?

[Hendrik van Rooyen]

 "Paul Rubin"  wrote:


> Paul Rubin  writes:
> > def get_from_queue(queue):
> >try:
> >   return queue.get(block=False)
> >except Queue.Empty:
> >   return QUEUE_IS_EMPTY
> 
> Alternatively:
> 
>  def get_from_queue(queue):
> try:
>return (queue.get(block=False), True)
> except Queue.Empty:
>return (None, False)
> 
> This is maybe a nicer interface (no special sentinel value needed).
> You'd use
> 
>  value, nonempty = get_from_queue(queue)
>   
> if nonempty is true then the item is valid.
> 
Hey, this is even nicer - thanks!

- Hendrik 

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Is there a way to protect a piece of critical code?

[Hendrik van Rooyen]

"Paul Rubin" <http://[EMAIL PROTECTED]> wrote:


> "Hendrik van Rooyen" <[EMAIL PROTECTED]> writes:
> > am aware of Queue module - the same app uses it for something else.
> > I dont like too many try -- excepts in the code - I find they confuse
> > me when I try to read it later - and in this case I cannot block on waiting
for
> > the queue to fill.
>
> Do you have multiple threads reading from the same queue?  If not then
> you can use queue.empty() to see whether the queue is empty.  If yes,
> queue.empty isn't reliable (it can return nonempty, and then another
> thread empties out the queue before you get a chance to read it).  But
> you could always wrap the queue:
>
> QUEUE_IS_EMPTY = object()   # global sentinel
>
> def get_from_queue(queue):
>try:
>   return queue.get(block=False)
>except Queue.Empty:
>   return QUEUE_IS_EMPTY
>
> Maybe it's worth adding a method like that to the Queue module in the stdlib.
>
 There is only one reader.
I like this its clever, thanks

- Hendrik

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Is there a way to protect a piece of critical code?

[Hendrik van Rooyen]

 "Steve Holden" <[EMAIL PROTECTED]> wrote:


> Hendrik van Rooyen wrote:
> > Hi,
> > 
> > I would like to do the following as one atomic operation:
> > 
> > 1) Append an item to a list
> > 2) Set a Boolean indicator
> > 
> > It would be almost like getting and holding the GIL,
> > to prevent a thread swap out between the two operations.
> > - sort of the inverted function than for which the GIL
> > seems to be used, which looks like "let go", get control
> > back via return from blocking I/O, and then "re - acquire"
> > 
> > Is this "reversed" usage possible?
> > Is there some way to prevent thread swapping?
> > 
> This seems to me to be a typical example of putting the cart before the 
> horse. Therefore, please don't think that what follows is directed 
> specifically at you: it's directed at everybody who thinks that their 
> problem is something other than it really is (of course, my extensive 
> experience on c.l.py plus my well-known psychic powers uniquely qualify 
> me to explain to you that you don't understand your own problem).

Yes of course - I see this happening all the time here 
- have probably done it myself...

> 
> > The question arises in the context of a multi threaded
> > environment where the list is used as a single producer,
> > single consumer queue - I can solve my problem in various
> > ways, of which this is one, and I am curious as to if it is 
> > possible to prevent a thread swap from inside the thread.
> > 
> Of course you will know what they say about curiosity [1].
> 

My understanding about it is not as advanced as yours- 
I was under the possibly false impression that it had
something vaguely to do with felines.

> You don't say what the Boolean indicator is for. My natural inclination 
> is to assume it's to say whether there's anything in the list. The 
> Twisted crew can tell you this is a terrible mistake,. What you should 
> really do is define a function that waits until there is something to 
> put on the list and then returns a deferred that will eventually 
> indicate whether the insertion was successful [2].
> 

How did you ever guess that it was exactly what I in my 
misguided zeal to make the code understandable for myself did?

As for the Theatrical Twisted way of saying:

"Don't call us, we'll call you ! " 

I don't reeaally have the time to wait...

> But your *actual* problem appears to be the introduction of critical 
> sections into your program, a question about which computer scientists 
> have written for over forty years now, albeit in the guise of 
> discussions about how to get a good meal [3].
> 

H - this can get philosophical - Most days, I write in assembler,
on bare metal small systems - and would ya believe - no matter how
I squirm - some bloody piece of code always turns out to be
critical - the hard part is to recognise that it is so when you write it,
Instead of having it bite you in the arse whan you are not looking.

> I could go on, but I am realising as I write that less and less of this 
> is really relevant to you. In short, please don't try to reinvent the 
> wheel when there are wheelwrights all around and a shop selling spare 
> wheels just around the corner. Python is already replete with ways to 
> implement critical sections and thread-safe queuing mechanisms [5].
> 

Now I know that a Youngster like you would not believe me when I tell
you that I know about queues and locks, so I won't bother.

> I could, of course, say
> 
> http://www.justfuckinggoogleit.com/search.pl?query=python+atomic+operation
> 
> but that would seem rude, which is against the tradition of c.l.py. 
> Besides which the answers aren't necessarily as helpful as what's been 
> posted on this thread, so I'll content myself with saying that one's can 
> often be better spent R'ing TFM than posting on this newsgroup, but that 
> while Google /may/ be your friend it's not as good a friend as this 
> newsgroup.
> 

Rudeness is against the tradition?
But bluntness is ok?

> If it isn't obvious that this post was meant more to amuse regular 
> readers than inform and/or chastise someone who isn't (yet) one then 
> please accept my apologies. Fortunately I don't normally go on like this 
> more than once a year, so now it's hey ho for 2008 [6].
> 

Happy New Year Steve - I for one have missed your posts lately.
I was beginning to feel you don't love me anymore...

8<-- some explanatory stuff --

*sigh* - I guess all this is just a nice way of saying "no!" to the
question.

- Hendrik



-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Is there a way to protect a piece of critical code?

[Hendrik van Rooyen]

"Paul Rubin" <http://[EMAIL PROTECTED]> wrote:


> "Hendrik van Rooyen" <[EMAIL PROTECTED]> writes:
> > I would like to do the following as one atomic operation:
> > 
> > 1) Append an item to a list
> > 2) Set a Boolean indicator
> 
> You could do it with locks as others have suggested, but maybe you
> really want the Queue module.
> 
> 
Please see my reply to Robert - am aware of queue, using it in fact.

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Is there a way to protect a piece of critical code?

[Hendrik van Rooyen]

"robert" <[EMAIL PROTECTED]> wrote:

> Hendrik van Rooyen wrote:
> > Hi,
> >
> > I would like to do the following as one atomic operation:
> >
> > 1) Append an item to a list
> > 2) Set a Boolean indicator
>
>
> I doubt you have to worry at all about this in such simple single-single
queue - if there is not a much more complex condition upon the insert order.
> And what should the indicator tell? that a new element is there?

Yes -that is what I was using it for, and I got bitten - It would fail after
five or so hours
of running quite happily, because the consumer section, that tested the bool,
would try to pop from an empty queue - the consumer section also cleared the
boolean when the queue length was zero...

A classic case of fuzzy thinking...

>
> The list itself tells its the length, its guaranteed to be increased _after_
.append()
> And you can .pop(0) just so - catching/retring at Key/IndexError at least.

I changed to testing the length of the queue to solve the problem,
like I said I was just curious to see if the code could be protected.

>
> List .append() and .pop() will be atomic in any Python though its not
mentioned explicitely - otherwise it would be time to leave Python.
>
> There is also Queue.Queue - though it has unneccessary overhead for most
purposes.
>
>
am aware of Queue module - the same app uses it for something else.
I dont like too many try -- excepts in the code - I find they confuse
me when I try to read it later - and in this case I cannot block on waiting for
the queue to fill.

> A function to block Python interpreter thread switching in such VHL language
would be nice for reducing the need for spreading locks in some cases (huge
code - little critical sections). Yet your example is by far not a trigger for
this. I also requested that once. Implementation in non-C-Pythons may be
difficult.
>
>
probably true - but even here - the problem would be solved by blocking
thread switching in the one thread, whereas using locks (or blocks as I would
tend to call them) requires fiddling in both threads.

> Generally there is also technique for optimistic unprotected execution of
critical sections - basically using an atomic counter and you need to provide
code for unrolling half executions. Search Google.
>
>
ok thanks will do

- Hendrik

-- 
http://mail.python.org/mailman/listinfo/python-list

Is there a way to protect a piece of critical code?

[Hendrik van Rooyen]

Hi,

I would like to do the following as one atomic operation:

1) Append an item to a list
2) Set a Boolean indicator

It would be almost like getting and holding the GIL,
to prevent a thread swap out between the two operations.
- sort of the inverted function than for which the GIL
seems to be used, which looks like "let go", get control
back via return from blocking I/O, and then "re - acquire"

Is this "reversed" usage possible?
Is there some way to prevent thread swapping?

The question arises in the context of a multi threaded
environment where the list is used as a single producer,
single consumer queue - I can solve my problem in various
ways, of which this is one, and I am curious as to if it is 
possible to prevent a thread swap from inside the thread.

- Hendrik

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Why less emphasis on private data?

[Hendrik van Rooyen]

"Steven D'Aprano" <[EMAIL PROTECTED]> wrote:

> On Tue, 09 Jan 2007 10:27:56 +0200, Hendrik van Rooyen wrote:
>
> > "Steven D'Aprano" <[EMAIL PROTECTED]> wrote:
> >
> >
> >> On Mon, 08 Jan 2007 13:11:14 +0200, Hendrik van Rooyen wrote:
> >>
> >> > When you hear a programmer use the word "probability" -
> >> > then its time to fire him, as in programming even the lowest
> >> > probability is a certainty when you are doing millions of
> >> > things a second.
> >>
> >> That is total and utter nonsense and displays the most appalling
> >> misunderstanding of probability, not to mention a shocking lack of common
> >> sense.
> >
> > Really?
> >
> > Strong words.
> >
> > If you don't understand you need merely ask, so let me elucidate:
> >
> > If there is some small chance of something occurring at run time that can
> > cause code to fail - a "low probability" in all the accepted senses of the
> > word - and a programmer declaims - "There is such a low probability of
> > that occurring and its so difficult to cater for that I won't bother"
> > - then am I supposed to congratulate him on his wisdom and outstanding
> > common sense?
> >
> > Hardly. - If anything can go wrong, it will. - to paraphrase Murphy's law.
> >
> > To illustrate:
> > If there is one place in any piece of code that is critical and not
protected,
> > even if its in a relatively rarely called routine, then because of the high
> > speed of operations, and the fact that time is essentially infinite,
>
> Time is essentially infinite? Do you really expect your code will still be
> in use fifty years from now, let alone a billion years?

My code does not suffer from bit rot, so it should outlast the hardware...

But seriously - for the sort of mistakes we make as programmers - it does
not actually need infinite time for the lightning to strike - most things that
will actually run overnight are probably stable - and if it takes say a week
of running for the bug to raise its head - it is normally a very difficult
problem to find and fix. A case in point - One of my first postings to
this newsgroup concerned an intermittent failure on a serial port - It was
never resolved in a satisfactory manner - eventually I followed my gut
feel, made some changes, and it seems to have gone away - but I expect
it to bite me anytime - I don't actually *know* that its fixed, and there is
not, as a corollary to your sum below here, any real way to know for
certain.

>
> I know flowcharts have fallen out of favour in IT, and rightly so -- they
> don't model modern programming techniques very well, simply because modern
> programming techniques would lead to a chart far too big to be practical.

I actually like drawing data flow diagrams, even if they are sketchy, primitive
ones, to try to model the inter process communications (where a "process"
may be just a python thread) - I find it useful to keep an overall perspective.

> But for the sake of the exercise, imagine a simplified flowchart of some
> program, one with a mere five components, such that one could take any of
> the following paths through the program:
>
> START -> A -> B -> C -> D -> E
> START -> A -> C -> B -> D -> E
> START -> A -> C -> D -> B -> E
> ...
> START -> E -> D -> C -> B -> A
>
> There are 5! (five factorial) = 120 possible paths through the program.
>
> Now imagine one where there are just fifty components, still quite a
> small program, giving 50! = 3e64 possible paths. Now suppose that there is
> a bug that results from following just one of those paths. That would
> match your description of "lowest probability" -- any lower and it would
> be zero.
>
> If all of the paths are equally likely to be taken, and the program takes
> a billion different paths each millisecond, on average it would take about
> 1.5e55 milliseconds to hit the bug -- or about 5e44 YEARS of continual
> usage. If every person on Earth did nothing but run this program 24/7, it
> would still take on average almost sixty million billion billion billion
> years to discover the bug.

In something with just 50 components it is, I believe, better to try to
inspect the quality in, than to hope that random testing will show up
errors - But I suppose this is all about design, and about avoiding
doing known no - nos.

>
> But of course in reality some paths are more likely than others. If the
> bug happens to exist in a path that is executed often, or if it exists
> in many paths, then the bug will be found qui

Re: Colons, indentation and reformatting. (2)

[Hendrik van Rooyen]

 "Jorgen Grahn" <[EMAIL PROTECTED]>wrote:

> On 8 Jan 2007 23:57:29 -0800, Paddy <[EMAIL PROTECTED]> wrote:
> >
> > OK, whilst colons are not sufficient to re-format a completely
> > mis-indented file. I'm thinking that they are sufficient for
> > reformatting most pasted code blocks when refactoring say?
> 
> Let's put it this way: if the formatter can assume the original code is
> valid (i.e. has the intended indentation) then it can do all kinds of nifty
> things to it.

This is true - and I think it will only fail if the "entry" point in the pasted
code is "further to the right" than where it has to fit in to the original
code - i.e. if you "run out of space" to the left.  - but in that case
you really are hacking, and you are in urgent need of some slashing...

- Hendrik

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Bitwise expression

[Hendrik van Rooyen]

 "Gigs_" <[EMAIL PROTECTED]> wrote:


> Now is all clearer thanks to [EMAIL PROTECTED] and Hendrick van Rooyen

Contrary to popular belief in the English speaking world - 

>>> "c" in "Hendrik"
False
>>> 

There is no "c" in "Hendrik"

: - )- Hendrik

-- 
http://mail.python.org/mailman/listinfo/python-list