date:20091124

On 22 Nov, 22:38, Patrick Stinson patrickstinson.li...@gmail.com
wrote:

 Has anyone every tried wrapping the CPython lib into a daemon with an
 RPC mechanism in order to move the GIL out of the process?

 I imagine this is how the multiprocessing module works.

It does not.


-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Where to put the error handing test?

2009-11-24 Thread Dave Angel


Peng Yu wrote:

On Mon, Nov 23, 2009 at 9:44 PM, Lie Ryan lie.1...@gmail.com wrote:
  

Peng Yu wrote:


Suppose that I have function f() that calls g(), I can put a test on
the argument 'x' in either g() or f(). I'm wondering what is the
common practice.

My thought is that if I put the test in g(x), the code of g(x) is
safer, but the test is not necessary when g() is called by h().

If I put the test in f(), then g() becomes more efficient when other
code call g() and guarantee x will pass the test even though the test
code in not in g(). But there might be some caller of g() that pass an
'x' that might not pass the test, if there were the test in g().
  

Typically, you test for x as early as possible, e.g. just after user input
(or file or url load or whatever). After that test, you can (or should be
able to) assume that all function calls will always be called with the
correct argument. This is the ideal situation, it's not always easy to do.

In any case though, don't optimize early.



Let's suppose that g() is refactored out from f() and is call by not
only f() but other functions, and g() is likely to be called by new
functions.

If I don't optimize early, I should put the test in g(), rather than f(), right?

  
Your question is so open-ended as to be unanswerable.  All we should do 
in this case is supply some guidelines so you can guess which one might 
apply in your particular case.


You could be referring to a test that triggers alternate handling.  Or 
you could be referring to a test that notices bad input by a user, or 
bad data from an untrusted source.  Or you could be referring to a test 
that discovers bugs in your code.  And there are variations of these, 
depending on whether your user is also writing code (eval, or even 
import of user-supplied mixins), etc.


The first thing that's needed in the function g() is a docstring, 
defining what inputs it expects, and what it'll do with them.  Then if 
it gets any input that doesn't meet those requirements, it might throw 
an exception.  Or it might just get an arbitrary result.  That's all up 
to the docstring.  Without any documentation, nothing is correct.


Functions that are only called by trusted code need not have explicit 
tests on their inputs, since you're writing it all.  Part of debugging 
is catching those cases where f () can pass bad data to g().  If it's 
caused because bad data is passed to f(), then you have a bug in that 
caller.  Eventually, you get to the user.  If the bad data comes from 
the user, it should be caught as soon as possible, and feedback supplied 
right then.


assert() ought to be the correct way to add tests in g() that test 
whether there's such a bug in f().  Unfortunately, in CPython it 
defaults to debug mode, so scripts that are run will execute those tests 
by default.  Consequently, people leave them out, to avoid slowing down 
code.




It comes down to trust.  If you throw the code together without a test 
suite, you'll be a long time finding all the bugs in non-trivial code.  
So add lots of defensive tests throughout the code, and pretend that's 
equivalent to a good test system.  If you're writing a library to be 
used by others, then define your public interfaces with exceptions for 
any invalid code, and write careful documentation describing what's 
invalid.  And if you're writing an end-user application, test their 
input as soon as you get it, so none of the rest of the application ever 
gets invalid data.



DaveA
--
http://mail.python.org/mailman/listinfo/python-list

Re: xmlrpc idea for getting around the GIL

On 23 Nov, 22:02, Patrick Stinson patrickstinson.li...@gmail.com
wrote:

 What I meant was that I am *not allowed* to make calls to the CPython
 API from the threads I currently have because these threads are high
 priority and are never allowed to make blocking calls. Fortunately,
 each of these threads can have a completely separate interpreter,

This seems confused. What would they do with the interpreter if they
cannot call the CPython API?


 My question was whether or not anyone has done anything like this in
 C/C++.

I have programmed parallel numerical software in Python, including on-
line signal processing. I have yet to find the GIL gets in my way.

Usually people complaining about the GIL does not know what they are
talking about.



-- 
http://mail.python.org/mailman/listinfo/python-list

Re: xmlrpc idea for getting around the GIL

2009-11-24 Thread Daniel Fetchinson

icating) the multiprocessing module would be ideal.
The problem is that the OP has a embedded application running threads.
multiprocssing doesn't help there.

that's right. I cannot make CPython calls from my original C-based
threads.

It's quite possible to do that. A thread started from C can make
calls to Python if it first calls PyGILState_Ensure, although you'd
have to make sure that the Python interpreter has been previously
initialized. See PEP 311 for details.

http://www.python.org/dev/peps/pep-0311/

I also suggest that if you want people to be more receptive to write
your replies after the quoted text. That is the custom in this
newsgroup/mailing list.

What I meant was that I am *not allowed* to make calls to the CPython
API from the threads I currently have because these threads are high
priority and are never allowed to make blocking calls. Fortunately,
each of these threads can have a completely separate interpreter, so
my idea was to create a daemon process for each thread. This daemon
would contain the libpython symbols and would make the CPython calls.
This would keep my current threads from having to contend over the
single GIL.

My question was whether or not anyone has done anything like this in
C/C++. This situation is slightly unique in that I am trying to
maintain separate interpreters within a single app (which is currently
kind of hacked out using a single interpreter, but it's ugly), but I
could see how this sort of thing would be useful for other C/C++ apps
that implement an embedded scripting engine.

My reference to multiprocessing was based on the idea that the library
hides the details fo the process management, shared memory, and rpc
mechanisms. Again, I can't use multiprocessing because it runs *in*
python I need this to be implemented *outside* of python to avoid
acquiring the GIL. complex, I know.

Naturally, the most intimidating part of perusing this kind of idea is
the overhead of the processess management and the RPC. Further,
libpython executes callable objects using C function pointers, and I
can't think of a way that this would be able to re-gain access to the
original app's respective functions if the interpreter was living in
another processes.

That's not to mention the impossibility of debugging.

Damn you, Gil.

By the way, you might be interested in this thread as well:

http://groups.google.com/group/comp.lang.python/browse_thread/thread/2d537ad8df9dab67/cc4cb2b493c98170

HTH,
Daniel

--
Psss, psss, put it down! - http://www.cafepress.com/putitdown
--
http://mail.python.org/mailman/listinfo/python-list

Re: xmlrpc idea for getting around the GIL

On 24 Nov, 12:15, Daniel Fetchinson fetchin...@googlemail.com wrote:

 By the way, you might be interested in this thread as well:

 http://groups.google.com/group/comp.lang.python/browse_thread/thread/...

It is Windows specific, and it does not work with extension modules.

Yes we can embed multiple interpreters just by making multiple copies
of Python26.dll. It's an ugly hack and not one I'd recommend for
production servers.


-- 
http://mail.python.org/mailman/listinfo/python-list

pointless musings on performance

2009-11-24 Thread mk


#!/usr/local/bin/python

import timeit


def pythonic():
nonevar = None
zerovar = 0
for x in range(100):
if nonevar:
pass
if zerovar:
pass

def unpythonic():
nonevar = None
zerovar = 0
for x in range(100):
if nonevar is not None:
pass
if zerovar  0:
pass

for f in [pythonic, unpythonic]:
print f.func_name, timeit.timeit(f, number=10)



# ./t.py
pythonic 2.13092803955
unpythonic 2.82064604759

Decidedly counterintuitive: are there special optimizations for if 
nonevar: type of statements in cpython implementation?



regards,
mk


--
http://mail.python.org/mailman/listinfo/python-list

Re: pointless musings on performance

2009-11-24 Thread MRAB


mk wrote:

#!/usr/local/bin/python

import timeit


def pythonic():
nonevar = None
zerovar = 0
for x in range(100):
if nonevar:
pass
if zerovar:
pass

def unpythonic():
nonevar = None
zerovar = 0
for x in range(100):
if nonevar is not None:
pass
if zerovar  0:
pass

for f in [pythonic, unpythonic]:
print f.func_name, timeit.timeit(f, number=10)



# ./t.py
pythonic 2.13092803955
unpythonic 2.82064604759

Decidedly counterintuitive: are there special optimizations for if 
nonevar: type of statements in cpython implementation?



In what way is it counterintuitive? In 'pythonic' the conditions are
simpler, less work is being done, therefore it's faster.
--
http://mail.python.org/mailman/listinfo/python-list

Re: pointless musings on performance

2009-11-24 Thread Rob Williscroft

mk wrote in news:mailman.915.1259064240.2873.python-l...@python.org in 
comp.lang.python:

 
 def pythonic():
 
 def unpythonic():
 
 
 Decidedly counterintuitive: are there special optimizations for if 
 nonevar: type of statements in cpython implementation?
 

from dis import dis

dis( unpythonic )

18  31 LOAD_FAST0 (nonevar)
 34 LOAD_CONST   0 (None)
 37 COMPARE_OP   9 (is not)
 40 JUMP_IF_FALSE4 (to 47)

dis( pythonic )

11  31 LOAD_FAST0 (nonevar)
 34 JUMP_IF_FALSE4 (to 41)
 
Rob.
 

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Where to put the error handing test?

On Mon, 23 Nov 2009 21:15:48 -0600, Peng Yu wrote:

 Suppose that I have function f() that calls g(), I can put a test on the
 argument 'x' in either g() or f(). I'm wondering what is the common
 practice.
 
 My thought is that if I put the test in g(x), the code of g(x) is safer,
 but the test is not necessary when g() is called by h().

If the function g requires the test, then put it in g. If it does not 
require the test, then don't put it in g.

If the test is only required by f, then it belongs in f.



-- 
Steven
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: pointless musings on performance

2009-11-24 Thread Chris Rebert

On Tue, Nov 24, 2009 at 4:31 AM, Rob Williscroft r...@freenet.co.uk wrote:
 mk wrote in news:mailman.915.1259064240.2873.python-l...@python.org in
 comp.lang.python:


 def pythonic():

 def unpythonic():


 Decidedly counterintuitive: are there special optimizations for if
 nonevar: type of statements in cpython implementation?


 from dis import dis

 dis( unpythonic )

 18          31 LOAD_FAST                0 (nonevar)
             34 LOAD_CONST               0 (None)
             37 COMPARE_OP               9 (is not)
             40 JUMP_IF_FALSE            4 (to 47)

 dis( pythonic )

 11          31 LOAD_FAST                0 (nonevar)
             34 JUMP_IF_FALSE            4 (to 41)

In other words, CPython doesn't happen to optimize `if nonevar is not
None` as much as it theoretically could (which would essentially
require a JUMP_IF_NONE opcode). Since CPython isn't known for doing
fancy optimizations, this isn't surprising.

Cheers,
Chris
--
http://blog.rebertia.com
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Any elegant way to construct the complete $k$-partite graph in Python?

2009-11-24 Thread Malte Helmert

Paul Miller wrote:
 On Mon, 23 Nov 2009 19:57:05 -0800, Richard Thomas wrote:
 
 Not sure exactly how you're representing graphs, this seems like the
 simplest way of listing the edges.

 def complete_partite(*sizes):
 total = sum(sizes)
 nodes, edges = range(total), []
 for group in xrange(len(sizes)):
 low = sum(sizes[:group-1])
 high = sum(sizes[:group])

I think this has a conceptual off-by-one error. Add

   print group, low, high

to see what I mean (especially the first iteration). It still works, but
I think this would be clearer:

   low = sum(sizes[:group])
   high = sum(sizes[:group + 1])

or to avoid doing essentially the same summation twice:

   low = sum(sizes[:group])
   high = low + sizes[group]

 edges.extend((i, j) for i in xrange(low, high)
 for j in xrange(high, total))
 return nodes, edges

Here's a variant that uses a running total instead of recomputing the
sum in every iteration, thus getting rid of xrange(len(...)).

def complete_partite(*sizes):
total = sum(sizes)
nodes, edges = range(total), []
curr_total = 0
for size in sizes:
edges.extend((i, j) for i in xrange(curr_total, curr_total+size)
for j in xrange(curr_total+size, total))
curr_total += size
return nodes, edges

Finally, here is a variant that is a bit shorter because it produces the
edges in a different way and hence gets rid of the need for knowing the
total up front and uses total as running total instead. It has the
drawback of not generating the edges in ascending order though, so I
think the previous one is nicer:

def complete_partite(*sizes):
total, edges = 0, []
for size in sizes:
edges.extend((i, j) for i in xrange(total)
for j in xrange(total, total + size))
total += size
return range(total), edges

Finally, here's a variation on the same theme:

def complete_partite(*sizes):
nodes, edges = [], []
for size in sizes:
partition = xrange(len(nodes), len(nodes) + size)
edges.extend((i, j) for i in nodes for j in partition)
nodes.extend(partition)
return nodes, edges

Malte

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: UnicodeDecodeError? Argh! Nothing works! I'm tired and hurting and...

On Mon, 23 Nov 2009 22:06:29 +0100, Alf P. Steinbach wrote:


 6. Googling, yes, it seems Thunderbird has a habit of forgetting
 mails. But they're really there after all. It's just the index that's
 screwed up.
[...]
 And I'm hesitant to just delete index file, hoping that it'll rebuild.

Right-click on the mailbox and choose Rebuild Index.

If you're particularly paranoid, and you probably should be, make a 
backup copy of the entire mail folder first.

http://kb.mozillazine.org/Compacting_folders
http://kb.mozillazine.org/Recover_messages_from_a_corrupt_folder
http://kb.mozillazine.org/Disappearing_mail


Good grief, it's about six weeks away from 2010 and Thunderbird still 
uses mbox as it's default mail box format. Hello, the nineties called, 
they want their mail formats back! Are the tbird developers on crack or 
something? I can't believe that they're still using that crappy format.

No, I tell a lie. I can believe it far too well.



-- 
Steven
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Beginning Question about Python functions, parameters...

2009-11-24 Thread Peter Otten

Terry Reedy wrote:

 astral orange wrote:
 
 As far as the program. I did add print statements such as print
 (MyNames) and got back:
 
 {'middle': {}, 'last': {'Smith': ['John Larry Smith']}, 'first': {}}
 
 Hmmm, as I understood the code, either that should be ... 'last': {} ...
 before the first store(), as you seem to be thinking below, or after the
 first store(),
 
 {'middle': {'Larry': ['John Larry Smith'],
   'last':   {'Smith': ['John Larry Smith'],
   'first':  {'John'  ['John Larry Smith']}
 
 or the same with [['John','Larry','Smith']] for each entry (do not
 remember exactly what was stored. Maybe a typo.

That's a bug in the store() function

# as posted
def store(data, full_name):
names = full_name.split()
if len(names) == 2: names.insert(1, '')
labels = 'first', 'middle', 'last'
for label, name in zip(labels, names):
people = lookup(data, label, name)
if people:
people.append(full_name)
else:
data[label][name] = [full_name]

# what was probably intended
def store(data, full_name):
names = full_name.split()
if len(names) == 2: names.insert(1, '')
labels = 'first', 'middle', 'last'
for label, name in zip(labels, names):
people = lookup(data, label, name)
if people:
people.append(full_name)
else:
data[label][name] = [full_name]


Peter

-- 
http://mail.python.org/mailman/listinfo/python-list

Recall: How to log messages _only once_ from all modules ?

2009-11-24 Thread Barak, Ron

Barak, Ron would like to recall the message, How to log messages _only once_ 
from all modules ?.
-- 
http://mail.python.org/mailman/listinfo/python-list

How to log messages _only once_ from all modules ?

2009-11-24 Thread Barak, Ron

Hi,

I'm trying to add the logging module to my application, but I seem to be 
missing something.
My application (a wxPython one) has a main script that calls various helper 
classes.
I want the log messages from all modules to go to one central log file.

When I implement logging, I think that due to preparation, I get the same 
message more than once.

Here's an example:

$ cat -n server.py
 1  import logging
 2  import logging.handlers
 3
 4  class Server():
 5  def __init__(self):
 6  self.client_logger = logging.getLogger(client)
 7  self.client_logger.setLevel(logging.DEBUG)
 8  h = logging.FileHandler(client.log)
 9  h.setLevel(logging.DEBUG)
10  formatter = logging.Formatter(%(asctime)s %(name)-12s 
%(levelname)-8s %(message)s)
11  h.setFormatter(formatter)
12  self.client_logger.addHandler(h)
13
14  def util(self):
15  self.client_logger.warning('This message comes from Server 
module')

$ cat -n client.py
 1  import logging
 2  import logging.handlers
 3  from server import Server
 4
 5  class Client():
 6  def __init__(self):
 7  self.client_logger = logging.getLogger(client)
 8  self.client_logger.setLevel(logging.DEBUG)
 9  h = logging.FileHandler(client.log)
10  h.setLevel(logging.DEBUG)
11  formatter = logging.Formatter(%(asctime)s %(name)-12s 
%(levelname)-8s %(message)s)
12  h.setFormatter(formatter)
13  self.client_logger.addHandler(h)
14
15  def client_test(self):
16  self.client_logger.warning(This message comes from Client 
module)
17
18  if __name__ == __main__:
19  ser = Server()
20  cli = Client()
21  ser.util()
22  cli.client_test()
$ rm client.log ; python client.py ; cat client.log
2009-11-24 14:40:39,762 client   WARNING  This message comes from Server 
module
2009-11-24 14:40:39,762 client   WARNING  This message comes from Server 
module
2009-11-24 14:40:39,762 client   WARNING  This message comes from Client 
module
2009-11-24 14:40:39,762 client   WARNING  This message comes from Client 
module
Googling and reading http://docs.python.org/library/logging.html didn't 
enlighten me.

Could you suggest what should I change in the above scripts so that the log 
messages would appear only once ?

Thanks,
Ron.

inline: image001.jpginline: image003.jpg

client.log
Description: client.log


client.py
Description: client.py


server.py
Description: server.py
-- 
http://mail.python.org/mailman/listinfo/python-list

How to log messages _only once_ from all modules ?

2009-11-24 Thread Barak, Ron

Hi,

I'm trying to add the logging module to my application, but I seem to be 
missing something.
My application (a wxPython one) has a main script that calls various helper 
classes.
I want the log messages from all modules to go to one central log file.

When I implement logging, I think that due to preparation, I get the same 
message more than once.

Here's an example:

$ cat -n server.py
 1  import logging
 2  import logging.handlers
 3
 4  class Server():
 5  def __init__(self):
 6  self.client_logger = logging.getLogger(client)
 7  self.client_logger.setLevel(logging.DEBUG)
 8  h = logging.FileHandler(client.log)
 9  h.setLevel(logging.DEBUG)
10  formatter = logging.Formatter(%(asctime)s %(name)-12s 
%(levelname)-8s %(message)s)
11  h.setFormatter(formatter)
12  self.client_logger.addHandler(h)
13
14  def util(self):
15  self.client_logger.warning('This message comes from Server 
module')

$ cat -n client.py
 1  import logging
 2  import logging.handlers
 3  from server import Server
 4
 5  class Client():
 6  def __init__(self):
 7  self.client_logger = logging.getLogger(client)
 8  self.client_logger.setLevel(logging.DEBUG)
 9  h = logging.FileHandler(client.log)
10  h.setLevel(logging.DEBUG)
11  formatter = logging.Formatter(%(asctime)s %(name)-12s 
%(levelname)-8s %(message)s)
12  h.setFormatter(formatter)
13  self.client_logger.addHandler(h)
14
15  def client_test(self):
16  self.client_logger.warning(This message comes from Client 
module)
17
18  if __name__ == __main__:
19  ser = Server()
20  cli = Client()
21  ser.util()
22  cli.client_test()
$ rm client.log ; python client.py ; cat client.log
2009-11-24 14:40:39,762 client   WARNING  This message comes from Server 
module
2009-11-24 14:40:39,762 client   WARNING  This message comes from Server 
module
2009-11-24 14:40:39,762 client   WARNING  This message comes from Client 
module
2009-11-24 14:40:39,762 client   WARNING  This message comes from Client 
module
Googling and reading http://docs.python.org/library/logging.html didn't 
enlighten me.

Could you suggest what should I change in the above scripts so that the log 
messages would appear only once ?

Thanks,
Ron.



server.py
Description: server.py


client.py
Description: client.py


client.log
Description: client.log
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Go versus Brand X

Le Mon, 23 Nov 2009 15:30:16 -0600, Robert Kern a écrit :
 particularly constrained environments like editors that may not be 
 extensible at all.

I'm not really an expert on this, but I think most good editors /are/ 
extensible (through plugins, scripts or other things).

 You can get away with just that and have something people recognize as
 syntax highlighting, yes. But if it is possible to highlight local
 variables, globals, and types differently, that *is* useful. And you
 will even see some syntax highlighters doing more advanced things like
 that even for Python (though mostly with heuristics).

I suppose it's a matter of taste. I don't expect syntax highlighting to 
do anything else than make the source code more readable and make some 
important things stick out (comments, keywords etc.). It's probably the 
same debate as text editor vs. full IDE. Users of text editors view 
programming as a literary practice where they manipulate text, while 
users of IDEs view programming as bringing technologies together through 
specialized tools.

Interestingly, we don't know how easy to parse Go is. We just have to 
trust their word on that, but perhaps Python is easier to parse (while 
being less ugly).

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: pointless musings on performance

2009-11-24 Thread mk


MRAB wrote:

In what way is it counterintuitive? In 'pythonic' the conditions are
simpler, less work is being done, therefore it's faster.


But the pythonic condition is more general: nonevar or zerovar can be 
'', 0, or None. So I thought it was more work for interpreter to compare 
those, while I thought that is not None is translated to one, more 
low-level and faster action. Apparently not.


As Rob pointed out (thanks):

11  31 LOAD_FAST0 (nonevar)
 34 JUMP_IF_FALSE4 (to 41)

I'm no good at py compiler or implementation internals and so I have no 
idea what bytecode JUMP_IF_FALSE is actually doing.


Regards,
mk


--
http://mail.python.org/mailman/listinfo/python-list

Re: xmlrpc idea for getting around the GIL

2009-11-24 Thread Brian Blais


On Nov 24, 2009, at 5:58 , sturlamolden wrote:


I have programmed parallel numerical software in Python, including on-
line signal processing. I have yet to find the GIL gets in my way.

Usually people complaining about the GIL does not know what they are
talking about.




I'd love to know which tools/libraries/approach you have found most  
profitable, and which were a waste of time.  Seems as if many people  
feel the GIL gets in the way, but perhaps they've been trying methods  
of parallelization that just aren't as effective.  What do you do?   
Do you have any sample code?



bb


--
Brian Blais
bbl...@bryant.edu
http://web.bryant.edu/~bblais



-- 
http://mail.python.org/mailman/listinfo/python-list

Re: How to log messages _only once_ from all modules ?

2009-11-24 Thread Soltys


Barak, Ron pisze:

Hi,

I'm trying to add the logging module to my application, but I seem to be 
missing something.
My application (a wxPython one) has a main script that calls various helper 
classes.
I want the log messages from all modules to go to one central log file.

When I implement logging, I think that due to preparation, I get the same 
message more than once.

Here's an example:

$ cat -n server.py
 1  import logging
 2  import logging.handlers
 3
 4  class Server():
 5  def __init__(self):
 6  self.client_logger = logging.getLogger(client)
 7  self.client_logger.setLevel(logging.DEBUG)
 8  h = logging.FileHandler(client.log)
 9  h.setLevel(logging.DEBUG)
10  formatter = logging.Formatter(%(asctime)s %(name)-12s 
%(levelname)-8s %(message)s)
11  h.setFormatter(formatter)
12  self.client_logger.addHandler(h)
13
14  def util(self):
15  self.client_logger.warning('This message comes from Server 
module')

$ cat -n client.py
 1  import logging
 2  import logging.handlers
 3  from server import Server
 4
 5  class Client():
 6  def __init__(self):
 7  self.client_logger = logging.getLogger(client)
 8  self.client_logger.setLevel(logging.DEBUG)
 9  h = logging.FileHandler(client.log)
10  h.setLevel(logging.DEBUG)
11  formatter = logging.Formatter(%(asctime)s %(name)-12s 
%(levelname)-8s %(message)s)
12  h.setFormatter(formatter)
13  self.client_logger.addHandler(h)
14
15  def client_test(self):
16  self.client_logger.warning(This message comes from Client 
module)
17
18  if __name__ == __main__:
19  ser = Server()
20  cli = Client()
21  ser.util()
22  cli.client_test()
$ rm client.log ; python client.py ; cat client.log
2009-11-24 14:40:39,762 client   WARNING  This message comes from Server 
module
2009-11-24 14:40:39,762 client   WARNING  This message comes from Server 
module
2009-11-24 14:40:39,762 client   WARNING  This message comes from Client 
module
2009-11-24 14:40:39,762 client   WARNING  This message comes from Client 
module
Googling and reading http://docs.python.org/library/logging.html didn't 
enlighten me.

Could you suggest what should I change in the above scripts so that the log 
messages would appear only once ?

Thanks,
Ron.




Have a look at http://docs.python.org/library/logging.html#logger-objects
First thing mentioned is Logger.propagate which is, what I believe, you're
looking for ;)

--
Soltys

Free software is a matter of liberty not price
--
http://mail.python.org/mailman/listinfo/python-list

Re: How to log messages _only once_ from all modules ?

2009-11-24 Thread Ron Barak

On Nov 24, 3:45 pm, Soltys sol...@noabuse.com wrote:
 Barak, Ron pisze:





  Hi,

  I'm trying to add the logging module to my application, but I seem to be 
  missing something.
  My application (a wxPython one) has a main script that calls various helper 
  classes.
  I want the log messages from all modules to go to one central log file.

  When I implement logging, I think that due to preparation, I get the same 
  message more than once.

  Here's an example:

  $ cat -n server.py
   1  import logging
   2  import logging.handlers
   3
   4  class Server():
   5  def __init__(self):
   6  self.client_logger = logging.getLogger(client)
   7  self.client_logger.setLevel(logging.DEBUG)
   8  h = logging.FileHandler(client.log)
   9  h.setLevel(logging.DEBUG)
  10  formatter = logging.Formatter(%(asctime)s %(name)-12s 
  %(levelname)-8s %(message)s)
  11  h.setFormatter(formatter)
  12  self.client_logger.addHandler(h)
  13
  14  def util(self):
  15  self.client_logger.warning('This message comes from Server 
  module')

  $ cat -n client.py
   1  import logging
   2  import logging.handlers
   3  from server import Server
   4
   5  class Client():
   6  def __init__(self):
   7  self.client_logger = logging.getLogger(client)
   8  self.client_logger.setLevel(logging.DEBUG)
   9  h = logging.FileHandler(client.log)
  10  h.setLevel(logging.DEBUG)
  11  formatter = logging.Formatter(%(asctime)s %(name)-12s 
  %(levelname)-8s %(message)s)
  12  h.setFormatter(formatter)
  13  self.client_logger.addHandler(h)
  14
  15  def client_test(self):
  16  self.client_logger.warning(This message comes from Client 
  module)
  17
  18  if __name__ == __main__:
  19  ser = Server()
  20  cli = Client()
  21  ser.util()
  22  cli.client_test()
  $ rm client.log ; python client.py ; cat client.log
  2009-11-24 14:40:39,762 client   WARNING  This message comes from 
  Server module
  2009-11-24 14:40:39,762 client   WARNING  This message comes from 
  Server module
  2009-11-24 14:40:39,762 client   WARNING  This message comes from 
  Client module
  2009-11-24 14:40:39,762 client   WARNING  This message comes from 
  Client module
  Googling and readinghttp://docs.python.org/library/logging.htmldidn't 
  enlighten me.

  Could you suggest what should I change in the above scripts so that the log 
  messages would appear only once ?

  Thanks,
  Ron.

 Have a look athttp://docs.python.org/library/logging.html#logger-objects
 First thing mentioned is Logger.propagate which is, what I believe, you're
 looking for ;)

 --
 Soltys

 Free software is a matter of liberty not price- Hide quoted text -

 - Show quoted text -

Hi Soltys,
I actually tried that, without any noticeable effects, viz.:

$ cat server.py
import logging
import logging.handlers

class Server():
def __init__(self):
self.client_logger = logging.getLogger(client)
self.client_logger.setLevel(logging.DEBUG)
h = logging.FileHandler(client.log)
h.setLevel(logging.DEBUG)
formatter = logging.Formatter(%(asctime)s %(name)-12s %
(levelname)-8s %(message)s)
h.setFormatter(formatter)
self.client_logger.addHandler(h)
self.client_logger.propagate = 0

def util(self):
self.client_logger.warning('This message comes from Server
module')

$ cat client.py
import logging
import logging.handlers
from server import Server

class Client():
def __init__(self):
self.client_logger = logging.getLogger(client)
self.client_logger.setLevel(logging.DEBUG)
h = logging.FileHandler(client.log)
h.setLevel(logging.DEBUG)
formatter = logging.Formatter(%(asctime)s %(name)-12s %
(levelname)-8s %(message)s)
h.setFormatter(formatter)
self.client_logger.addHandler(h)
self.client_logger.propagate = 0

def client_test(self):
self.client_logger.warning(This message comes from Client
module)

if __name__ == __main__:
ser = Server()
cli = Client()
ser.util()
cli.client_test()

$ rm client.log ; python client.py ; cat client.log
2009-11-24 16:06:35,710 client   WARNING  This message comes from
Server module
2009-11-24 16:06:35,710 client   WARNING  This message comes from
Server module
2009-11-24 16:06:35,710 client   WARNING  This message comes from
Client module
2009-11-24 16:06:35,710 client   WARNING  This message comes from
Client module

$

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Line-continuation Anti-Idiom and with statement

2009-11-24 Thread Neil Cerutti

On 2009-11-23, Terry Reedy tjre...@udel.edu wrote:
 Neil Cerutti wrote:
 Unfortunately, the new nested with statement (which I also read
 about today) forces me into this anti-idiom. When replacing an
 appearance of contextlib.nested with the 3K with statement, I
 ended up with an unexpected syntax error.
 
 with (open(roster_path, 'r') as roster_file,
   open(disb_path, 'w') as out_file,
   open(report_path, 'w') as report_file):
 
 The result was:
 
   File C:\project\codxml.py, line 184
 with (open(roster_path, 'r') as roster_file,
   ^
 SyntaxError: invalid syntax

 Right. The first open paren is illegal.

 I believe that '\ \n' would always be harmless or a SyntexError
 outside of expressons. I believe 'subtly wrong' only applies
 within expressions. 

 So I would not call \ continuation an anti-pattern outside
 expressions. So you might suggest that the whole entry specify
 expression context to begin with. To me, your example shows why
 blanket condemnation is wrong.

 The HOWTOs are not scripture.

I like your suggestion. Changing the title of the anti-idiom to
Using Backslash to Continue Expressions seems like a good fix.
I submitted it as issue 7391.

I've done a search of the PEP's, Python-Dev, and Python-Ideas,
but I couldn't find much official discussion. The change was made
because contextlib.nested was not semantically equivalent to
actual nested with statments.

GvR noted in Python Ideas that he liked the idea of making the
new syntax parallel to the import statement sytax variant:

import module  [as name] ( ,  [as name] )*

So that's where the 'multi-with' statement found its model.

-- 
Neil Cerutti
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: IDE+hg

2009-11-24 Thread Gerhard Häring

Rhodri James wrote:
 On Mon, 23 Nov 2009 19:20:27 -, NiklasRTZ nikla...@gmail.com wrote:
 
 Dear experts,
 Since no py IDE I found has easy hg access. IDEs PIDA and Eric claim
 Mercurial support not found i.e. buttons to clone, commit and push to
 repositories to define dev env dvcs, editor and deployment all in 1.
 
 I don't really understand this urge to cram everything into a single
 program, since that inevitably leads to compromises that will compromise
 just how much of Mercurial's useful and interesting functionality you
 can get at.  Still, if you really must, Emacs (and presumably vim) seems
 to be capable of working with most source control systems.

I prefer the commandline tools, too.

FWIW, Eclipse supports Mercurial through
http://www.vectrace.com/mercurialeclipse/

-- Gerhard


-- 
http://mail.python.org/mailman/listinfo/python-list

Re: pointless musings on performance

2009-11-24 Thread Rob Williscroft

mk wrote in news:mailman.923.1259070092.2873.python-l...@python.org in 
comp.lang.python:

 MRAB wrote:
 In what way is it counterintuitive? In 'pythonic' the conditions are
 simpler, less work is being done, therefore it's faster.
 
 But the pythonic condition is more general: nonevar or zerovar can be 
 '', 0, or None. So I thought it was more work for interpreter to compare 
 those, while I thought that is not None is translated to one, more 
 low-level and faster action. Apparently not.
 
 As Rob pointed out (thanks):
 
 11  31 LOAD_FAST0 (nonevar)
   34 JUMP_IF_FALSE4 (to 41)
 
 I'm no good at py compiler or implementation internals and so I have no 
 idea what bytecode JUMP_IF_FALSE is actually doing.

IIUC it implements:

http://docs.python.org/3.1/reference/expressions.html#boolean-operations

In the context of Boolean operations, and also when expressions are used 
by control flow statements, the following values are interpreted as false: 
False, None, numeric zero of all types, and empty strings and containers 
(including strings, tuples, lists, dictionaries, sets and frozensets). All 
other values are interpreted as true. User-defined objects can customize 
their truth value by providing a __bool__() method.

In particular its implementing ... Boolean operation ... used by 
control flow ..., all in one handy op code.

Rob.
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: How to log messages _only once_ from all modules ?

2009-11-24 Thread Soltys


Ron Barak pisze:

On Nov 24, 3:45 pm, Soltys sol...@noabuse.com wrote:

Barak, Ron pisze:






Hi,
I'm trying to add the logging module to my application, but I seem to be 
missing something.
My application (a wxPython one) has a main script that calls various helper 
classes.
I want the log messages from all modules to go to one central log file.
When I implement logging, I think that due to preparation, I get the same 
message more than once.
Here's an example:
$ cat -n server.py
 1  import logging
 2  import logging.handlers
 3
 4  class Server():
 5  def __init__(self):
 6  self.client_logger = logging.getLogger(client)
 7  self.client_logger.setLevel(logging.DEBUG)
 8  h = logging.FileHandler(client.log)
 9  h.setLevel(logging.DEBUG)
10  formatter = logging.Formatter(%(asctime)s %(name)-12s 
%(levelname)-8s %(message)s)
11  h.setFormatter(formatter)
12  self.client_logger.addHandler(h)
13
14  def util(self):
15  self.client_logger.warning('This message comes from Server 
module')
$ cat -n client.py
 1  import logging
 2  import logging.handlers
 3  from server import Server
 4
 5  class Client():
 6  def __init__(self):
 7  self.client_logger = logging.getLogger(client)
 8  self.client_logger.setLevel(logging.DEBUG)
 9  h = logging.FileHandler(client.log)
10  h.setLevel(logging.DEBUG)
11  formatter = logging.Formatter(%(asctime)s %(name)-12s 
%(levelname)-8s %(message)s)
12  h.setFormatter(formatter)
13  self.client_logger.addHandler(h)
14
15  def client_test(self):
16  self.client_logger.warning(This message comes from Client 
module)
17
18  if __name__ == __main__:
19  ser = Server()
20  cli = Client()
21  ser.util()
22  cli.client_test()
$ rm client.log ; python client.py ; cat client.log
2009-11-24 14:40:39,762 client   WARNING  This message comes from Server 
module
2009-11-24 14:40:39,762 client   WARNING  This message comes from Server 
module
2009-11-24 14:40:39,762 client   WARNING  This message comes from Client 
module
2009-11-24 14:40:39,762 client   WARNING  This message comes from Client 
module
Googling and readinghttp://docs.python.org/library/logging.htmldidn't enlighten 
me.
Could you suggest what should I change in the above scripts so that the log 
messages would appear only once ?
Thanks,
Ron.

Have a look athttp://docs.python.org/library/logging.html#logger-objects
First thing mentioned is Logger.propagate which is, what I believe, you're
looking for ;)

--
Soltys

Free software is a matter of liberty not price- Hide quoted text -

- Show quoted text -


Hi Soltys,
I actually tried that, without any noticeable effects, viz.:

$ cat server.py
import logging
import logging.handlers

class Server():
def __init__(self):
self.client_logger = logging.getLogger(client)
self.client_logger.setLevel(logging.DEBUG)
h = logging.FileHandler(client.log)
h.setLevel(logging.DEBUG)
formatter = logging.Formatter(%(asctime)s %(name)-12s %
(levelname)-8s %(message)s)
h.setFormatter(formatter)
self.client_logger.addHandler(h)
self.client_logger.propagate = 0

def util(self):
self.client_logger.warning('This message comes from Server
module')

$ cat client.py
import logging
import logging.handlers
from server import Server

class Client():
def __init__(self):
self.client_logger = logging.getLogger(client)
self.client_logger.setLevel(logging.DEBUG)
h = logging.FileHandler(client.log)
h.setLevel(logging.DEBUG)
formatter = logging.Formatter(%(asctime)s %(name)-12s %
(levelname)-8s %(message)s)
h.setFormatter(formatter)
self.client_logger.addHandler(h)
self.client_logger.propagate = 0

def client_test(self):
self.client_logger.warning(This message comes from Client
module)

if __name__ == __main__:
ser = Server()
cli = Client()
ser.util()
cli.client_test()

$ rm client.log ; python client.py ; cat client.log
2009-11-24 16:06:35,710 client   WARNING  This message comes from
Server module
2009-11-24 16:06:35,710 client   WARNING  This message comes from
Server module
2009-11-24 16:06:35,710 client   WARNING  This message comes from
Client module
2009-11-24 16:06:35,710 client   WARNING  This message comes from
Client module

$



Rename logger in server.py to server:
self.client_logger = logging.getLogger(server)

Rerun, you should get sth. like this:
$ cat client.log
2009-11-24 16:06:54,990 server   WARNING  This message comes from Server 
module
2009-11-24 16:06:54,990 client   WARNING  This message comes from Client 
module



--
Soltys

Free software is a matter of liberty not price
--

Re: pointless musings on performance


Hello,

Le Tue, 24 Nov 2009 14:41:19 +0100, mk a écrit :
 
 As Rob pointed out (thanks):
 
 11  31 LOAD_FAST0 (nonevar)
   34 JUMP_IF_FALSE4 (to 41)
 
 I'm no good at py compiler or implementation internals and so I have no
 idea what bytecode JUMP_IF_FALSE is actually doing.

It tries to evaluate the op of the stack (here nonevar) in a boolean 
context (which theoretically involves calling __nonzero__ on the type) 
and then jumps if the result is False (rather than True).

You are totally right that it does /more/ than is not None, but since 
it is executed as a single opcode rather than a sequence of several 
opcodes, the additional work it has to do is compensated (in this case) 
by the smaller overhead in bytecode interpretation.

As someone pointed out, the Python interpreter could grow CISC-like 
opcodes so as to collapse is not None (or generically is not 
constant) into a single JUMP_IF_IS_NOT_CONST opcode. Actually, it is 
the kind of optimizations wpython does (http://code.google.com/p/
wpython/).

Regards

Antoine.

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: IDE+hg

2009-11-24 Thread Richard Riley

Gerhard Häring g...@ghaering.de writes:

 Rhodri James wrote:
 On Mon, 23 Nov 2009 19:20:27 -, NiklasRTZ nikla...@gmail.com wrote:
 
 Dear experts,
 Since no py IDE I found has easy hg access. IDEs PIDA and Eric claim
 Mercurial support not found i.e. buttons to clone, commit and push to
 repositories to define dev env dvcs, editor and deployment all in 1.
 
 I don't really understand this urge to cram everything into a single
 program, since that inevitably leads to compromises that will
 compromise

Huh? Cram what? Nothing is crammed into anything. The IDE/Editor is
merely programmed to hook into the external tools

 just how much of Mercurial's useful and interesting functionality you
 can get at.  Still, if you really must, Emacs (and presumably vim) seems
 to be capable of working with most source control systems.

 I prefer the commandline tools, too.

 FWIW, Eclipse supports Mercurial through
 http://www.vectrace.com/mercurialeclipse/

 -- Gerhard

Why would you prefer the command line tools in a shell when the same
tools can be used in a way which makes navigating the output so much
easier? It strikes me as a kind of intransigence. it's a common
misconception that IDEs use their own tools all the time. They
don't. They integrate the very same tools. e.g Why the hell would I drop
to a command line to diff a file with a back version in GIT when I can
do the same in the buffer in emacs with a single hot key? Why would I
pipe the output of compile into a file then open that file when a single
hot key can fire off the SAME compiler and then list the errors in an
emacs buffer and another hot key can take me directly to the source
lines in question? Living in the past has its mements, but really.

e.g I have pylint working live in python buffers. Big time
saver. Similar with C.
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: How to log messages _only once_ from all modules ?

2009-11-24 Thread Ron Barak

On Nov 24, 5:08 pm, Soltys sol...@noabuse.com wrote:
 Ron Barak pisze:





  On Nov 24, 3:45 pm, Soltys sol...@noabuse.com wrote:
  Barak, Ron pisze:

  Hi,
  I'm trying to add the logging module to my application, but I seem to be 
  missing something.
  My application (a wxPython one) has a main script that calls various 
  helper classes.
  I want the log messages from all modules to go to one central log file.
  When I implement logging, I think that due to preparation, I get the same 
  message more than once.
  Here's an example:
  $ cat -n server.py
   1  import logging
   2  import logging.handlers
   3
   4  class Server():
   5  def __init__(self):
   6  self.client_logger = logging.getLogger(client)
   7  self.client_logger.setLevel(logging.DEBUG)
   8  h = logging.FileHandler(client.log)
   9  h.setLevel(logging.DEBUG)
  10  formatter = logging.Formatter(%(asctime)s %(name)-12s 
  %(levelname)-8s %(message)s)
  11  h.setFormatter(formatter)
  12  self.client_logger.addHandler(h)
  13
  14  def util(self):
  15  self.client_logger.warning('This message comes from 
  Server module')
  $ cat -n client.py
   1  import logging
   2  import logging.handlers
   3  from server import Server
   4
   5  class Client():
   6  def __init__(self):
   7  self.client_logger = logging.getLogger(client)
   8  self.client_logger.setLevel(logging.DEBUG)
   9  h = logging.FileHandler(client.log)
  10  h.setLevel(logging.DEBUG)
  11  formatter = logging.Formatter(%(asctime)s %(name)-12s 
  %(levelname)-8s %(message)s)
  12  h.setFormatter(formatter)
  13  self.client_logger.addHandler(h)
  14
  15  def client_test(self):
  16  self.client_logger.warning(This message comes from 
  Client module)
  17
  18  if __name__ == __main__:
  19  ser = Server()
  20  cli = Client()
  21  ser.util()
  22  cli.client_test()
  $ rm client.log ; python client.py ; cat client.log
  2009-11-24 14:40:39,762 client   WARNING  This message comes from 
  Server module
  2009-11-24 14:40:39,762 client   WARNING  This message comes from 
  Server module
  2009-11-24 14:40:39,762 client   WARNING  This message comes from 
  Client module
  2009-11-24 14:40:39,762 client   WARNING  This message comes from 
  Client module
  Googling and 
  readinghttp://docs.python.org/library/logging.htmldidn'tenlighten me.
  Could you suggest what should I change in the above scripts so that the 
  log messages would appear only once ?
  Thanks,
  Ron.
  Have a look athttp://docs.python.org/library/logging.html#logger-objects
  First thing mentioned is Logger.propagate which is, what I believe, you're
  looking for ;)

  --
  Soltys

  Free software is a matter of liberty not price- Hide quoted text -

  - Show quoted text -

  Hi Soltys,
  I actually tried that, without any noticeable effects, viz.:

  $ cat server.py
  import logging
  import logging.handlers

  class Server():
  def __init__(self):
  self.client_logger = logging.getLogger(client)
  self.client_logger.setLevel(logging.DEBUG)
  h = logging.FileHandler(client.log)
  h.setLevel(logging.DEBUG)
  formatter = logging.Formatter(%(asctime)s %(name)-12s %
  (levelname)-8s %(message)s)
  h.setFormatter(formatter)
  self.client_logger.addHandler(h)
  self.client_logger.propagate = 0

  def util(self):
  self.client_logger.warning('This message comes from Server
  module')

  $ cat client.py
  import logging
  import logging.handlers
  from server import Server

  class Client():
  def __init__(self):
  self.client_logger = logging.getLogger(client)
  self.client_logger.setLevel(logging.DEBUG)
  h = logging.FileHandler(client.log)
  h.setLevel(logging.DEBUG)
  formatter = logging.Formatter(%(asctime)s %(name)-12s %
  (levelname)-8s %(message)s)
  h.setFormatter(formatter)
  self.client_logger.addHandler(h)
  self.client_logger.propagate = 0

  def client_test(self):
  self.client_logger.warning(This message comes from Client
  module)

  if __name__ == __main__:
  ser = Server()
  cli = Client()
  ser.util()
  cli.client_test()

  $ rm client.log ; python client.py ; cat client.log
  2009-11-24 16:06:35,710 client   WARNING  This message comes from
  Server module
  2009-11-24 16:06:35,710 client   WARNING  This message comes from
  Server module
  2009-11-24 16:06:35,710 client   WARNING  This message comes from
  Client module
  2009-11-24 16:06:35,710 client   WARNING  This message comes from
  Client module

  $

 Rename logger in server.py to server:
 self.client_logger =

Re: xmlrpc idea for getting around the GIL

Le Tue, 24 Nov 2009 02:53:30 -0800, sturlamolden a écrit :
 On 22 Nov, 22:38, Patrick Stinson patrickstinson.li...@gmail.com
 wrote:
 
 Has anyone every tried wrapping the CPython lib into a daemon with an
 RPC mechanism in order to move the GIL out of the process?
 
 I imagine this is how the multiprocessing module works.
 
 It does not.

Actually, it is how multiprocessing works under Windows (for lack of the 
fork() function), except that it uses pickle by default (but it does have 
xmlrpc support).

Regards

Antoine.


-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Where to put the error handing test?

2009-11-24 Thread Peng Yu

On Tue, Nov 24, 2009 at 12:27 AM, alex23 wuwe...@gmail.com wrote:
 On Nov 24, 1:15 pm, Peng Yu pengyu...@gmail.com wrote:
 Suppose that I have function f() that calls g(), I can put a test on
 the argument 'x' in either g() or f(). I'm wondering what is the
 common practice.

 If I put the test in f(), then g() becomes more efficient when other
 code call g() and guarantee x will pass the test even though the test
 code in not in g(). But there might be some caller of g() that pass an
 'x' that might not pass the test, if there were the test in g().

 What you should try to do is make each function as self-contained as
 possible. f() shouldn't have to know what is a valid argument for g(),
 that's the responsibility of g(). What f() needs to know is how to
 deal with any problems that arise while using g().

This may not always be possible, because g() might call a third party
software, that I don't have the complete knowledge of. What would you
do if this case?

Another scenario:

Suppose that f_1(),...,f_n(), g() are in a package, where g() is an
internal function that the end users are not suppose to call, and
f_1(),...,f_n() are the functions that the end users may call.

Since all the f_1 ... f_n functions knows g(), they can be
programmed to guarantee not to pass any arguments that can not be
handled by g(). In this case, I think it is reasonable to move the
test code from g()? Is it the general accepted practice?

 As a very rough example:

    def g(x):
        try:
            assert isinstance(x, int)
        except AssertionError:
            raise TypeError, excepted int, got %s % type(x)
        # ... function code goes here

    def f(x):
        try:
            g(x)
        except TypeError:
            # handle the problem here
        # ... function code goes here

 My thought is that if I put the test in g(x), the code of g(x) is
 safer, but the test is not necessary when g() is called by h().

 This sounds strange to me. Are you stating that h() can pass values to
 g() that would be illegal for f() to pass? That sounds like a very
 dangerous design...you want each function's behaviour to be as
 consistent and predictable as it possibly can.

You misunderstood me.

h() doesn't pass any illegal arguments to g(). If I put the test code
in g(), it would be a waste of run time when h() calls g(). In this
case, and under the condition that g() is an internal function of a
package as I mentioned above, I think I should move the test code from
g() to f(). What do you think?
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Beginning Question about Python functions, parameters...

2009-11-24 Thread astral orange

On Nov 23, 10:37 pm, r rt8...@gmail.com wrote:
 On Nov 23, 11:19 am, astral orange 457r0...@gmail.com wrote:



  Hi, I am trying to teach myself Python and have a good book to help me
  but I am stuck on something and I would like for someone to explain
  the following piece of code for me and what it's actually doing.
  Certain parts are very clear but once it enters the def store(data,
  full_name):  function and the def lookup()... function things
  get a little confusing for me. Specifically, lines 103-108 *and* Lines
  110-111.

  Lastly, I am not sure how to print the results I've put into this
  program either, the book I'm reading doesn't tell me. As you can tell,
  I am a beginner and I don't truly understand everything that is going
  on here...a lot, but not all

  Here is the code:

   92 def init(data):
   93     data['first'] = {}
   94     data['middle'] = {}
   95     data['last'] = {}
   96
   97 def store(data, full_name):
   98     names = full_name.split()
  100     if len(names) == 2: names.insert(1, '')
  101     labels = 'first', 'middle', 'last'
  103     for label, name in zip(labels, names):
  104         people = lookup(data, label, name)
  105     if people:
  106         people.append(full_name)
  107     else:
  108         data[label][name] = [full_name]
  109
  110 def lookup(data, label, name):
  111     return data[label].get(name)
  112
  113
  114 MyNames = {}
  115 init(MyNames)
  116 store(MyNames, 'John Larry Smith')
  117 lookup(MyNames, 'middle', 'Smith')

 This is a horrible example to show noobs. I think the OP could better
 understand this as a class EVEN though the OP may or may not know what
 a class *is* yet.

  class Name():

         def __init__(self, first, middle, last):
                 self.first = first
                 self.middle = middle
                 self.last = last

  name1 = Name('Guido', 'van', 'Rossum')
  name1.first
 'Guido'
  name1.middle
 'van'
  name1.last
 'Rossum'
  print name1

 __main__.Name instance at 0x029BFD78

 This time we add a __str__ method, the result will speak louder than
 words!!

  class Name():

         def __init__(self, first, middle, last):
                 self.first = first
                 self.middle = middle
                 self.last = last
         def __str__(self):
                 return '%s %s %s' %(self.first, self.middle, self.last)

  name2 = Name('Terry', 'J', 'Reedy')
  name2.first
 'Terry'
  name2.middle
 'J'
  name2.last
 'Reedy'
  print name2

 Terry J Reedy

 See the difference in the print statements. Now lets have some real
 fun and access each sub name by index haha!

  class Name():

         def __init__(self, first, middle, last):
                 self.first = first
                 self.middle = middle
                 self.last = last
         def __str__(self):
                 return '%s %s %s' %(self.first, self.middle, self.last)
         def __len__(self):
                 return 3
         def __getitem__(self, item):
                 if item == 0:
                         return self.first
                 elif item == 1:
                         return self.middle
                 elif item == 2:
                         return self.last
                 else:
                         raise IndexError(Index must be in range 0, 2)

  name = Name('Joe', 'blow', 'scripter')
  name[0]
 'Joe'
  name[1]
 'blow'
  name[2]
 'scripter'
  len(name)

 3

 WOW, thats more info in a few lines than any tut i ever seen! I wish i
 could have seen that in my initial days, could have save some
 countless hours of confusion!!! Maybe i am in the wrong line of work?

 Should i keep going...?

Yeah, I don't think the example in the book is the best for someone
starting out. I still
am not getting certain parts of the program so I think I'll move on
in hopes that it will
*not* came back to haunt me and the book (along with the online
tutorial) will help me grab
more of the basics of Python programming.

As for the class Name(): example above? Even though I haven't seen
exactly what purpose 'self' serves
yet I can follow and understand what is going on very easily, that
helps out tremendously.
Very clearly written...Thank you!

And thanks again to everyone...
-- 
http://mail.python.org/mailman/listinfo/python-list

Need help to understand a getattr syntax.

2009-11-24 Thread NMarcu

Hello all,
I need some help to understand a getattr syntax. The syntax is:

try:
getattr(self, command)(cursor, row)
except Exception, e:
print Action failed : '%s' % command
raise e

I don't understand why is not like this: a = getattr(), and what are
the scope of the second (): (cursor, row)

Thanks.
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Need help to understand a getattr syntax.

2009-11-24 Thread Diez B. Roggisch

NMarcu wrote:

 Hello all,
 I need some help to understand a getattr syntax. The syntax is:
 
 try:
 getattr(self, command)(cursor, row)
 except Exception, e:
 print Action failed : '%s' % command
 raise e
 
 I don't understand why is not like this: a = getattr(), and what are
 the scope of the second (): (cursor, row)

The attribute in question is a callable, a method most probably. So 

 getattr(self, command)

returns a reference to the method, and the following

 (cursor, row)

calls it with cursor and row as parameters.

You could rewrite it as 

  method = getattr(self, command)
  print method
  method(cursor, row)

Diez
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: pointless musings on performance

2009-11-24 Thread Neil Cerutti

On 2009-11-24, Antoine Pitrou solip...@pitrou.net wrote:
 It tries to evaluate the op of the stack (here nonevar) in a
 boolean context (which theoretically involves calling
 __nonzero__ on the type) 

...or __bool__ in Py3K.

-- 
Neil Cerutti
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: pointless musings on performance

Le Tue, 24 Nov 2009 15:11:29 +, Antoine Pitrou a écrit :
 Hello,
 
 Le Tue, 24 Nov 2009 14:41:19 +0100, mk a écrit :
 
 As Rob pointed out (thanks):
 
 11  31 LOAD_FAST0 (nonevar)
   34 JUMP_IF_FALSE4 (to 41)
 
 I'm no good at py compiler or implementation internals and so I have no
 idea what bytecode JUMP_IF_FALSE is actually doing.
 
 It tries to evaluate the op of the stack (here nonevar)

I meant the top of the stack obviously.


-- 
http://mail.python.org/mailman/listinfo/python-list

Re: IDE+hg

2009-11-24 Thread rustom

On Nov 24, 8:13 pm, Richard Riley rileyrg...@gmail.com wrote:
 Gerhard Häring g...@ghaering.de writes:
  Rhodri James wrote:
  On Mon, 23 Nov 2009 19:20:27 -, NiklasRTZ nikla...@gmail.com wrote:

  Dear experts,
  Since no py IDE I found has easy hg access. IDEs PIDA and Eric claim
  Mercurial support not found i.e. buttons to clone, commit and push to
  repositories to define dev env dvcs, editor and deployment all in 1.

  I don't really understand this urge to cram everything into a single
  program, since that inevitably leads to compromises that will
  compromise

 Huh? Cram what? Nothing is crammed into anything. The IDE/Editor is
 merely programmed to hook into the external tools

  just how much of Mercurial's useful and interesting functionality you
  can get at.  Still, if you really must, Emacs (and presumably vim) seems
  to be capable of working with most source control systems.

  I prefer the commandline tools, too.

  FWIW, Eclipse supports Mercurial through
 http://www.vectrace.com/mercurialeclipse/

  -- Gerhard

 Why would you prefer the command line tools in a shell when the same
 tools can be used in a way which makes navigating the output so much
 easier? It strikes me as a kind of intransigence. it's a common
 misconception that IDEs use their own tools all the time. They
 don't. They integrate the very same tools. e.g Why the hell would I drop
 to a command line to diff a file with a back version in GIT when I can
 do the same in the buffer in emacs with a single hot key? Why would I
 pipe the output of compile into a file then open that file when a single
 hot key can fire off the SAME compiler and then list the errors in an
 emacs buffer and another hot key can take me directly to the source
 lines in question? Living in the past has its mements, but really.

 e.g I have pylint working live in python buffers. Big time
 saver. Similar with C.

I sometimes think that the amount of time I spend tweaking emacs to
save my time is more than the time I spend on anything else :-)

But more seriously:
I tried to use emacs with git recently -- it was a sorry experience.
The git.el that comes with git is broken (on windows)
vc was too old for git like systems
dvc is a joke (its supposedly generic for all Distributed Version
Systems -- but everything is couched in terms of tla.
TLA! For heavens sake!
magit would not run on windows and to use egg http://github.com/bogolisk/egg
I must read magit docs.
Finally I decided to stay with what Ive used for the last 25 years --
the shell

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Where to put the error handing test?

2009-11-24 Thread Peng Yu

On Tue, Nov 24, 2009 at 4:58 AM, Dave Angel da...@ieee.org wrote:
 Peng Yu wrote:

 On Mon, Nov 23, 2009 at 9:44 PM, Lie Ryan lie.1...@gmail.com wrote:


 Peng Yu wrote:


 Suppose that I have function f() that calls g(), I can put a test on
 the argument 'x' in either g() or f(). I'm wondering what is the
 common practice.

 My thought is that if I put the test in g(x), the code of g(x) is
 safer, but the test is not necessary when g() is called by h().

 If I put the test in f(), then g() becomes more efficient when other
 code call g() and guarantee x will pass the test even though the test
 code in not in g(). But there might be some caller of g() that pass an
 'x' that might not pass the test, if there were the test in g().


 Typically, you test for x as early as possible, e.g. just after user
 input
 (or file or url load or whatever). After that test, you can (or should be
 able to) assume that all function calls will always be called with the
 correct argument. This is the ideal situation, it's not always easy to
 do.

 In any case though, don't optimize early.


 Let's suppose that g() is refactored out from f() and is call by not
 only f() but other functions, and g() is likely to be called by new
 functions.

 If I don't optimize early, I should put the test in g(), rather than f(),
 right?



 Your question is so open-ended as to be unanswerable.  All we should do in
 this case is supply some guidelines so you can guess which one might apply
 in your particular case.

 You could be referring to a test that triggers alternate handling.  Or you
 could be referring to a test that notices bad input by a user, or bad data
 from an untrusted source.  Or you could be referring to a test that
 discovers bugs in your code.  And there are variations of these, depending
 on whether your user is also writing code (eval, or even import of
 user-supplied mixins), etc.

 The first thing that's needed in the function g() is a docstring, defining
 what inputs it expects, and what it'll do with them.  Then if it gets any
 input that doesn't meet those requirements, it might throw an exception.  Or
 it might just get an arbitrary result.  That's all up to the docstring.
  Without any documentation, nothing is correct.

 Functions that are only called by trusted code need not have explicit tests
 on their inputs, since you're writing it all.  Part of debugging is catching
 those cases where f () can pass bad data to g().  If it's caused because bad
 data is passed to f(), then you have a bug in that caller.  Eventually, you
 get to the user.  If the bad data comes from the user, it should be caught
 as soon as possible, and feedback supplied right then.

I'll still confused by the guideline that an error should be caught as
early as possible.

Suppose I have the following call chain

f1() -- f2() -- f3() -- f4()

The input in f1() might cause an error in f4(). However, this error
can of cause be caught by f1(), whenever I want to do so. In the worst
case, I could duplicate the code of f2 and f3, and the test code in f4
to f1(), to catch the error in f1 rather than f4. But I don't think
that this is what you mean.

Then the problem is where to put the test code more effectively. I
would consider 'whether it is obvious to test the condition in the
give function' as the guideline. However, it might be equal obvious to
test the same thing two functions, for example, f1 and f4.

In this case, I thought originally that I should put the test code in
f1 rather than f4, if f1, f2, f3 and f4 are all the functions that I
have in the package that I am making. But it is possible that some
time later I added the function f5(),...,f10() that calls f4(). Since
f4 doesn't have the test code, f5(),...,f10() should have the same
test code. This is clearly a redundancy to the code. If I move the
test code to f4(), there is a redundancy of the code between f1 and
f4.

I'm wondering how you would solve the above problem?

 assert() ought to be the correct way to add tests in g() that test whether
 there's such a bug in f().  Unfortunately, in CPython it defaults to debug
 mode, so scripts that are run will execute those tests by default.
  Consequently, people leave them out, to avoid slowing down code.



 It comes down to trust.  If you throw the code together without a test
 suite, you'll be a long time finding all the bugs in non-trivial code.  So
 add lots of defensive tests throughout the code, and pretend that's
 equivalent to a good test system.  If you're writing a library to be used by
 others, then define your public interfaces with exceptions for any invalid
 code, and write careful documentation describing what's invalid.  And if
 you're writing an end-user application, test their input as soon as you get
 it, so none of the rest of the application ever gets invalid data.

Having the test code for any function and any class (even the ones
that are internal in the package) is basically what I am doing.
However, if I

fixing xml output format

2009-11-24 Thread wadi wadi

 Hi all,

I am creating some xml output using minidom and saving it to a file using
doc.writexml()
The output however is as follows:

?xml version=1.0 ?
info
Author
bill catman
/Author
/info


Is there a way to save xml output in a file such as xml is formatted the
right way? I mean with the right indentation and the elements values with no
new lines?

?xml version=1.0 ?
info
  Authorbill catman/Author
/info

Note: writexml() with parameters and toprettyxml() are not giving the
desired output.

Thanks.
-- 
http://mail.python.org/mailman/listinfo/python-list

csv and mixed lists of unicode and numbers

2009-11-24 Thread Sibylle Koczian


Hello,

I want to put data from a database into a tab separated text file. This
looks like a typical application for the csv module, but there is a
snag: the rows I get from the database module (kinterbasdb in this case)
contain unicode objects and numbers. And of course the unicode objects
contain lots of non-ascii characters.

If I try to use csv.writer as is, I get UnicodeEncodeErrors. If I use
the UnicodeWriter from the module documentation, I get TypeErrors with
the numbers. (I'm using Python 2.6 - upgrading to 3.1 on this machine
would cause other complications.)

So do I have to process the rows myself and treat numbers and text
fields differently? Or what's the best way?

Here is a small example:


#!/usr/bin/env python
# -*- coding: utf-8 -*-

import csv, codecs, cStringIO
import tempfile

cData = [u'Ärger', u'Ödland', 5, u'Süßigkeit', u'élève', 6.9, u'forêt']

class UnicodeWriter:

A CSV writer which will write rows to CSV file f,
which is encoded in the given encoding.


def __init__(self, f, dialect=csv.excel, encoding=utf-8, **kwds):
# Redirect output to a queue
self.queue = cStringIO.StringIO()
self.writer = csv.writer(self.queue, dialect=dialect, **kwds)
self.stream = f
self.encoder = codecs.getincrementalencoder(encoding)()

def writerow(self, row):
self.writer.writerow([s.encode(utf-8) for s in row])
# Fetch UTF-8 output from the queue ...
data = self.queue.getvalue()
data = data.decode(utf-8)
# ... and reencode it into the target encoding
data = self.encoder.encode(data)
# write to the target stream
self.stream.write(data)
# empty queue
self.queue.truncate(0)

def writerows(self, rows):
for row in rows:
self.writerow(row)

def writewithcsv(outfile, datalist):
wrt = csv.writer(outfile, dialect=csv.excel)
wrt.writerow(datalist)

def writeunicode(outfile, datalist):
wrt = UnicodeWriter(outfile)
wrt.writerow(datalist)

def main():
with tempfile.NamedTemporaryFile() as csvfile:
print CSV file:, csvfile.name
print Try with csv.writer
try:
writewithcsv(csvfile, cData)
except UnicodeEncodeError as e:
print e
print Try with UnicodeWriter
writeunicode(csvfile, cData)
print Ready.

if __name__ == __main__:
main()


##

Hoping for advice,

Sibylle
--
http://mail.python.org/mailman/listinfo/python-list

Re: pointless musings on performance

2009-11-24 Thread Paul Boddie

On 24 Nov, 16:11, Antoine Pitrou solip...@pitrou.net wrote:


[JUMP_IF_FALSE]

 It tries to evaluate the op of the stack (here nonevar) in a boolean
 context (which theoretically involves calling __nonzero__ on the type)
 and then jumps if the result is False (rather than True).

[...]

 As someone pointed out, the Python interpreter could grow CISC-like
 opcodes so as to collapse is not None (or generically is not
 constant) into a single JUMP_IF_IS_NOT_CONST opcode.

Of course, JUMP_IF_FALSE is already quite CISC-like, whereas testing
if something is not None could involve some fairly RISC-like
instructions: just compare the address of an operand with the address
of None. As you point out, a lot of this RISC vs. CISC analysis (and
inferences drawn from Python bytecode analysis) is somewhat academic:
the cost of the JUMP_IF_FALSE instruction is likely to be minimal in
the context of all the activity going on to evaluate the bytecodes.

I imagine that someone (or a number of people) must have profiled the
Python interpreter and shown how much time goes on the individual
bytecode implementations and how much goes on the interpreter's own
housekeeping activities. It would be interesting to see such figures.

Paul
-- 
http://mail.python.org/mailman/listinfo/python-list

howto send signal to a OS daemon on linux

2009-11-24 Thread bheemesh v

Hello,

Greetings.

I am a newbie to python programming.
Can anybody please help me with options to send a user defined signal to a
OS daemon (example a daemon snmpd).

I tried finding options to get pid of a daemon (process) by passing it's
name.
Once a pid is available thenwe can use os.kill(signal_type, signalhandler);

But how to get pid of a process by passing it's name and then how to send
signal whichis userdefind.

Kindly let me know.
Thanks in advance.

Best Regards,
Bheemesh
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: xmlrpc idea for getting around the GIL

On 24 Nov, 16:13, Antoine Pitrou solip...@pitrou.net wrote:

  Has anyone every tried wrapping the CPython lib into a daemon with an
  RPC mechanism in order to move the GIL out of the process?

  I imagine this is how the multiprocessing module works.

  It does not.

 Actually, it is how multiprocessing works under Windows (for lack of the
 fork() function), except that it uses pickle by default (but it does have
 xmlrpc support).

Windows does not have daemons, so this is obviously incorrect. (There
are something called Windows Services, but multiprocessing does not
use them.)

Multiprocessing on Windows uses the subprocess module.











-- 
http://mail.python.org/mailman/listinfo/python-list

Re: IDE+hg

2009-11-24 Thread Günther Dietrich

NiklasRTZ nikla...@gmail.com wrote:

Since no py IDE I found has easy hg access.

Obviously, you didn't try Eclipse with PyDev (http://www.pydev.org) 
and Mercurial Eclipse (http://www.vectrace.com/mercurialeclipse/) 
plugins.
This combination is also available stuffed into one package as 
'EasyEclipse for Python' (http://www.easyeclipse.org).

Both, pure Eclipse with plugins installed by hand, and EasyEclipse, are 
very convenient for python development.



Best regards,

Günther
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: csv and mixed lists of unicode and numbers

2009-11-24 Thread Benjamin Kaplan

On Tue, Nov 24, 2009 at 11:42 AM, Sibylle Koczian nulla.epist...@web.de wrote:
 Hello,

 I want to put data from a database into a tab separated text file. This
 looks like a typical application for the csv module, but there is a
 snag: the rows I get from the database module (kinterbasdb in this case)
 contain unicode objects and numbers. And of course the unicode objects
 contain lots of non-ascii characters.

 If I try to use csv.writer as is, I get UnicodeEncodeErrors. If I use
 the UnicodeWriter from the module documentation, I get TypeErrors with
 the numbers. (I'm using Python 2.6 - upgrading to 3.1 on this machine
 would cause other complications.)

 So do I have to process the rows myself and treat numbers and text
 fields differently? Or what's the best way?

 Here is a small example:

 
 #!/usr/bin/env python
 # -*- coding: utf-8 -*-

 import csv, codecs, cStringIO
 import tempfile

 cData = [u'Ärger', u'Ödland', 5, u'Süßigkeit', u'élève', 6.9, u'forêt']

 class UnicodeWriter:
    
    A CSV writer which will write rows to CSV file f,
    which is encoded in the given encoding.
    

    def __init__(self, f, dialect=csv.excel, encoding=utf-8, **kwds):
        # Redirect output to a queue
        self.queue = cStringIO.StringIO()
        self.writer = csv.writer(self.queue, dialect=dialect, **kwds)
        self.stream = f
        self.encoder = codecs.getincrementalencoder(encoding)()

    def writerow(self, row):
        self.writer.writerow([s.encode(utf-8) for s in row])

try doing [s.encode(utf-8) if isinstance(s,unicode) else s for s in row]
That way, you'll only encode the unicode strings


        # Fetch UTF-8 output from the queue ...
        data = self.queue.getvalue()
        data = data.decode(utf-8)
        # ... and reencode it into the target encoding
        data = self.encoder.encode(data)
        # write to the target stream
        self.stream.write(data)
        # empty queue
        self.queue.truncate(0)

    def writerows(self, rows):
        for row in rows:
            self.writerow(row)

 def writewithcsv(outfile, datalist):
    wrt = csv.writer(outfile, dialect=csv.excel)
    wrt.writerow(datalist)

 def writeunicode(outfile, datalist):
    wrt = UnicodeWriter(outfile)
    wrt.writerow(datalist)

 def main():
    with tempfile.NamedTemporaryFile() as csvfile:
        print CSV file:, csvfile.name
        print Try with csv.writer
        try:
            writewithcsv(csvfile, cData)
        except UnicodeEncodeError as e:
            print e
        print Try with UnicodeWriter
        writeunicode(csvfile, cData)
    print Ready.

 if __name__ == __main__:
    main()


 ##

 Hoping for advice,

 Sibylle
 --
 http://mail.python.org/mailman/listinfo/python-list

-- 
http://mail.python.org/mailman/listinfo/python-list

Python-URL! - weekly Python news and links (Nov 24)

2009-11-24 Thread Cameron Laird

QOTW:  ... it's generally accepted that COM sucks rocks through
straws, so
explore alternatives when they're available ;-) - Chris Withers
http://groups.google.com/group/comp.lang.python/msg/29577c851ceed167


From nothing to a complete working program - Peter Otten on
stepwise
refinement:

http://groups.google.com/group/comp.lang.python/t/f6f44b646af5b09e/8f59b2585da524a1?#8f59b2585da524a1

Handling whitespace in command line arguments:
http://groups.google.com/group/comp.lang.python/t/9a828279953b45a2/

Recognizing hex arguments in the command line:
http://groups.google.com/group/comp.lang.python/t/31d4c9386291c/

A pipeline of Python programs:
http://groups.google.com/group/comp.lang.python/t/cc06520602ae3f42/

Calling Python functions from Excel
http://groups.google.com/group/comp.lang.python/t/83aa60666c555d87/

The scope of interactive commands:   =20
http://groups.google.com/group/comp.lang.python/t/3f0d7607ed5a4a78/

List comprehensions and slice assignments - which are the
corresponding
operations for dictionaries?
http://groups.google.com/group/comp.lang.python/t/7aa443ac48f58851/

The precise semantics of [:]=20
http://groups.google.com/group/comp.lang.python/t/84b5ec30cdd26cde/

The 'with' statement doesn't allow () for implicit line
continuation:
http://comments.gmane.org/gmane.comp.python.general/645508

Grant Edwards on the best way to get help from this group :)

http://groups.google.com/group/comp.lang.python/t/b8a0c32cae495522/21e80ac383745d88?#21e80ac383745d88

Finding the root cause of slowness when sorting certain objects:
http://groups.google.com/group/comp.lang.python/t/44d80224360e085/

The fastest alternative to list.extend()
http://groups.google.com/group/comp.lang.python/t/614bfc36a09d9ab7/

A library for bijective mappings:
http://groups.google.com/group/comp.lang.python/t/785d100681f7d101/

GUI builders reviewed:
http://groups.google.com/group/comp.lang.python/t/3db5b18d77974b8/

A long thread started two weeks ago: is Python not scalable enough
for
Google?
http://groups.google.com/group/comp.lang.python/t/ceef2ae6b4472b61/



Everything Python-related you want is probably one or two clicks away
in
these pages:

Python.org's Python Language Website is the traditional
center of Pythonia
http://www.python.org
Notice especially the master FAQ
http://www.python.org/doc/FAQ.html

PythonWare complements the digest you're reading with the
marvelous daily python url
 http://www.pythonware.com/daily

Just beginning with Python?  This page is a great place to start:
http://wiki.python.org/moin/BeginnersGuide/Programmers

The Python Papers aims to publish the efforts of Python
enthusiasts:
http://pythonpapers.org/
The Python Magazine is a technical monthly devoted to Python:
http://pythonmagazine.com

Readers have recommended the Planet site:
http://planet.python.org

comp.lang.python.announce announces new Python software.  Be
sure to scan this newsgroup weekly.
http://groups.google.com/group/comp.lang.python.announce/topics

Python411 indexes podcasts ... to help people learn Python ...
Updates appear more-than-weekly:
http://www.awaretek.com/python/index.html

The Python Package Index catalogues packages.
http://www.python.org/pypi/

Much of Python's real work takes place on Special-Interest Group
mailing lists
http://www.python.org/sigs/

Python Success Stories--from air-traffic control to on-line
match-making--can inspire you or decision-makers to whom you're
subject with a vision of what the language makes practical.
http://www.pythonology.com/success

The Python Software Foundation (PSF) has replaced the Python
Consortium as an independent nexus of activity.  It has official
responsibility for Python's development and maintenance.
http://www.python.org/psf/
Among the ways you can support PSF is with a donation.
http://www.python.org/psf/donations/

The Summary of Python Tracker Issues is an automatically generated
report summarizing new bugs, closed ones, and patch submissions.
http://search.gmane.org/?author=status%40bugs.python.orggroup=gmane.com
p.python.develsort=date

Although unmaintained since 2002, the Cetus collection of Python
hyperlinks retains a few gems.
http://www.cetus-links.org/oo_python.html

Python FAQTS
http://python.faqts.com/

The Cookbook is a collaborative effort to capture useful and
interesting recipes.
http://code.activestate.com/recipes/langs/python/

Many Python conferences around the world are in preparation.
Watch this

Re: attributes, properties, and accessors -- philosophy

2009-11-24 Thread Ethan Furman


Bruno Desthuilliers wrote:

Ethan Furman a écrit :

The problem I have with properties is my typing.  I'll end up 
assigning to an attribute, but get the spelling slightly wrong 
(capitalized, or missing an underscore -- non-obvious things when 
bug-hunting), so now I have an extra attribute which of course has 
zero effect on what I'm trying to do and I start getting wierd results 
like viewing deleted records when I *know* I set useDeleted = False... 
30 minutes later I notice it was /supposed/ to be use_deleted.  *sigh*


So -- to keep myself out of trouble -- I have started coding such 
things as, for example:


result = table.use_deleted()   # returns True or False
table.use_deleted(False)   # skip deleted records

instead of

result = table.use_deleted
table.use_deleted = False

My question:  is this [ severely | mildly | not at all ] un-pythonic?



Definitly and totally unpythonic. The first solution to your problem is 
to stick to standard naming conventions. If this is not enough, Chris 
pointed you to really useful tools. Also, you can override __setattr__ 
to catch such errors - at least during the coding/debug phase.


Good tools to know about, and a consistent naming pattern also makes 
life easier (which I have since done ;).


Let's head towards murkier waters (at least murkier to me -- hopefully 
they can be easily clarified):  some of the attributes are read-only, 
such as record count; others are not directly exposed, but still 
settable, such as table version; and still others require a small amount 
of processing... at which point do I switch from simple attribute access 
to method access?


~Ethan~
--
http://mail.python.org/mailman/listinfo/python-list

Re: xmlrpc idea for getting around the GIL

Le Tue, 24 Nov 2009 09:27:24 -0800, sturlamolden a écrit :
 
 Windows does not have daemons, so this is obviously incorrect. (There
 are something called Windows Services, but multiprocessing does not use
 them.)

This is nitpicking. Technically it might not be a daemon but it's used as 
such.
The important point is that it does use a (custom) form of RPC through 
marshalling, which is what the original question was about.

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: UnicodeDecodeError? Argh! Nothing works! I'm tired and hurting and...

2009-11-24 Thread Chris Jones

On Tue, Nov 24, 2009 at 08:02:09AM EST, Steven D'Aprano wrote:

 Good grief, it's about six weeks away from 2010 and Thunderbird still 
 uses mbox as it's default mail box format. Hello, the nineties called, 
 they want their mail formats back! Are the tbird developers on crack or 
 something? I can't believe that they're still using that crappy format.
 
 No, I tell a lie. I can believe it far too well.

:-)

I realize that's somewhat OT, but what mail box format do you recommend,
and why?

Thanks,

CJ
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: python bijection

2009-11-24 Thread Joshua Bronson

Hey Raymond,

Thanks for your thoughtful reply! I think your idea for a class-
generation approach in the spirit of namedtuple is brilliant; looking
forward to coding this up and seeing how it feels to use it.

(By the way, it occurred to me that bijection is perhaps the wrong
term to use for this data structure; really it's just an injective
mapping, as it has no idea whether the function whose mappings it
contains is also surjective. (Unless we take the domain, codomain, and
range of the function being modeled to be exactly defined by the state
of the mapping at any given time. But it feels more correct to me to
interpret the mapping as a sampling of some underlying function, where
the sampling can change but the function stays the same.) So I'm
thinking of renaming the class injectivedict or idict instead of
bijection. Is that crazy?)

More responses inline:

On Nov 21, 9:22 pm, Raymond Hettinger pyt...@rcn.com wrote:
 * The idea of using __call__ for looking-up inverse values was
 inspired.  That is useable, clean, and easy to remember; however, as
 discussed below, there are issues though with its actual use in real
 code.

Totally agree the call syntax has issues. Did you happen to see
Terry's suggestion to use slice syntax instead? Now *that* was
inspired. It's also much better because it works for setitem and
delitem too. I replaced the call syntax with the slice syntax on
Friday night -- would be interested to hear whether you think it's an
improvement.


 * Am not excited by the inverse iterators.  With just a regular
 mapping you can write:

         for a, b in m.items() ...   # consider either a or b be the
 key and the other to be the value

   That meets all of the needs that would have been served by
 iter_inverse_keys() or iter_inverse_values() or whatnot.  The mirrored
 API doesn't really provide much in the way of value added.

Hm, the one value I see the latest version of ``inverted`` adding (may
not have been in the version you saw) is that you can pass it either a
mapping, an iterable, or any object implementing an __inverted__
method. So in one case it's just syntax sugar for writing [(v, k) for
(k, v) in d.items()], but in other cases it's providing some
abstraction.

snip much good feedback and ideas /

 Hope these ideas help.  The ultimate success of the Bijection code
 will depend on its clarity, simplicity, and speed.  Experiment with
 various approaches to find-out which looks the best in real code.  It
 cannot be error-prone or it is doomed.  Also, it should not introduce
 much overhead processing or else people will avoid it.  The API should
 be trivially simple so that people remember how to use it months after
 seeing it for the first time.

Thank you for the sage advice.

Best,
Josh
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: attributes, properties, and accessors -- philosophy

2009-11-24 Thread Chris Rebert

On Tue, Nov 24, 2009 at 9:39 AM, Ethan Furman et...@stoneleaf.us wrote:
 Bruno Desthuilliers wrote:
 Ethan Furman a écrit :
 The problem I have with properties is my typing.  I'll end up assigning
 to an attribute, but get the spelling slightly wrong (capitalized, or
 missing an underscore -- non-obvious things when bug-hunting), so now I have
 an extra attribute which of course has zero effect on what I'm trying to do
 and I start getting wierd results like viewing deleted records when I *know*
 I set useDeleted = False... 30 minutes later I notice it was /supposed/ to
 be use_deleted.  *sigh*

 So -- to keep myself out of trouble -- I have started coding such things
 as, for example:

 result = table.use_deleted()       # returns True or False
 table.use_deleted(False)           # skip deleted records

 instead of

 result = table.use_deleted
 table.use_deleted = False

 My question:  is this [ severely | mildly | not at all ] un-pythonic?


 Definitly and totally unpythonic. The first solution to your problem is to
 stick to standard naming conventions. If this is not enough, Chris pointed
 you to really useful tools. Also, you can override __setattr__ to catch such
 errors - at least during the coding/debug phase.

 Good tools to know about, and a consistent naming pattern also makes life
 easier (which I have since done ;).

 Let's head towards murkier waters (at least murkier to me -- hopefully they
 can be easily clarified):  some of the attributes are read-only, such as
 record count; others are not directly exposed, but still settable, such as
 table version; and still others require a small amount of processing... at
 which point do I switch from simple attribute access to method access?

Thanks to the magic of properties, the end-user-programmer need not
know which you're using:

http://docs.python.org/library/functions.html#property

Cheers,
Chris
--
http://blog.rebertia.com
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: pointless musings on performance

Le Tue, 24 Nov 2009 08:58:40 -0800, Paul Boddie a écrit :
 As you
 point out, a lot of this RISC vs. CISC analysis (and inferences drawn
 from Python bytecode analysis) is somewhat academic: the cost of the
 JUMP_IF_FALSE instruction is likely to be minimal in the context of all
 the activity going on to evaluate the bytecodes.

Sorry, I have trouble parsing your sentence. Do you mean bytecode 
interpretation overhead is minimal compared to the cost of actual useful 
work, or the contrary?
(IMO both are wrong by the way)

 I imagine that someone (or a number of people) must have profiled the
 Python interpreter and shown how much time goes on the individual
 bytecode implementations and how much goes on the interpreter's own
 housekeeping activities.

Well the one problem is that it's not easy to draw a line. Another 
problem is that it depends on the workload. If you are compressing large 
data or running expensive regular expressions the answer won't be the 
same as if you compute a Mandelbrot set in pure Python.

One data point is that the computed gotos option in py3k generally 
makes the interpreter faster by ~15%. Another data point I've heard is 
that people who have tried a very crude form of Python-to-C compilation 
(generating the exact C code corresponding to a function or method, using 
Python's C API and preserving dynamicity without attempting to be clever) 
have apparently reached speedups of up to 50% (in other words, twice as 
fast). So you could say that the interpretation overhead is generally 
between 15% and 50%.


-- 
http://mail.python.org/mailman/listinfo/python-list

Python-URL! - weekly Python news and links (Nov 24)

2009-11-24 Thread Gabriel Genellina

QOTW:  ... it's generally accepted that COM sucks rocks through straws, so
explore alternatives when they're available ;-) - Chris Withers
http://groups.google.com/group/comp.lang.python/msg/29577c851ceed167


From nothing to a complete working program - Peter Otten on stepwise
refinement:

http://groups.google.com/group/comp.lang.python/t/f6f44b646af5b09e/8f59b2585da524a1?#8f59b2585da524a1

Handling whitespace in command line arguments:
http://groups.google.com/group/comp.lang.python/t/9a828279953b45a2/

Recognizing hex arguments in the command line:
http://groups.google.com/group/comp.lang.python/t/31d4c9386291c/

A pipeline of Python programs:
http://groups.google.com/group/comp.lang.python/t/cc06520602ae3f42/

Calling Python functions from Excel
http://groups.google.com/group/comp.lang.python/t/83aa60666c555d87/

The scope of interactive commands:   =20
http://groups.google.com/group/comp.lang.python/t/3f0d7607ed5a4a78/

List comprehensions and slice assignments - which are the corresponding
operations for dictionaries?
http://groups.google.com/group/comp.lang.python/t/7aa443ac48f58851/

The precise semantics of [:]=20
http://groups.google.com/group/comp.lang.python/t/84b5ec30cdd26cde/

The 'with' statement doesn't allow () for implicit line continuation:
http://comments.gmane.org/gmane.comp.python.general/645508

Grant Edwards on the best way to get help from this group :)

http://groups.google.com/group/comp.lang.python/t/b8a0c32cae495522/21e80ac383745d88?#21e80ac383745d88

Finding the root cause of slowness when sorting certain objects:
http://groups.google.com/group/comp.lang.python/t/44d80224360e085/

The fastest alternative to list.extend()
http://groups.google.com/group/comp.lang.python/t/614bfc36a09d9ab7/

A library for bijective mappings:
http://groups.google.com/group/comp.lang.python/t/785d100681f7d101/

GUI builders reviewed:
http://groups.google.com/group/comp.lang.python/t/3db5b18d77974b8/

A long thread started two weeks ago: is Python not scalable enough for
Google?
http://groups.google.com/group/comp.lang.python/t/ceef2ae6b4472b61/



Everything Python-related you want is probably one or two clicks away in
these pages:

Python.org's Python Language Website is the traditional
center of Pythonia
http://www.python.org
Notice especially the master FAQ
http://www.python.org/doc/FAQ.html

PythonWare complements the digest you're reading with the
marvelous daily python url
 http://www.pythonware.com/daily

Just beginning with Python?  This page is a great place to start:
http://wiki.python.org/moin/BeginnersGuide/Programmers

The Python Papers aims to publish the efforts of Python enthusiasts:
http://pythonpapers.org/
The Python Magazine is a technical monthly devoted to Python:
http://pythonmagazine.com

Readers have recommended the Planet site:
http://planet.python.org

comp.lang.python.announce announces new Python software.  Be
sure to scan this newsgroup weekly.
http://groups.google.com/group/comp.lang.python.announce/topics

Python411 indexes podcasts ... to help people learn Python ...
Updates appear more-than-weekly:
http://www.awaretek.com/python/index.html

The Python Package Index catalogues packages.
http://www.python.org/pypi/

Much of Python's real work takes place on Special-Interest Group
mailing lists
http://www.python.org/sigs/

Python Success Stories--from air-traffic control to on-line
match-making--can inspire you or decision-makers to whom you're
subject with a vision of what the language makes practical.
http://www.pythonology.com/success

The Python Software Foundation (PSF) has replaced the Python
Consortium as an independent nexus of activity.  It has official
responsibility for Python's development and maintenance.
http://www.python.org/psf/
Among the ways you can support PSF is with a donation.
http://www.python.org/psf/donations/

The Summary of Python Tracker Issues is an automatically generated
report summarizing new bugs, closed ones, and patch submissions. 

http://search.gmane.org/?author=status%40bugs.python.orggroup=gmane.comp.python.develsort=date

Although unmaintained since 2002, the Cetus collection of Python
hyperlinks retains a few gems.
http://www.cetus-links.org/oo_python.html

Python FAQTS
http://python.faqts.com/

The Cookbook is a collaborative effort to capture useful and
interesting recipes.
http://code.activestate.com/recipes/langs/python/

Many

Re: csv and mixed lists of unicode and numbers

2009-11-24 Thread Peter Otten

Sibylle Koczian wrote:

 I want to put data from a database into a tab separated text file. This
 looks like a typical application for the csv module, but there is a
 snag: the rows I get from the database module (kinterbasdb in this case)
 contain unicode objects and numbers. And of course the unicode objects
 contain lots of non-ascii characters.
 
 If I try to use csv.writer as is, I get UnicodeEncodeErrors. If I use
 the UnicodeWriter from the module documentation, I get TypeErrors with
 the numbers. (I'm using Python 2.6 - upgrading to 3.1 on this machine
 would cause other complications.)
 
 So do I have to process the rows myself and treat numbers and text
 fields differently? Or what's the best way?

I'd preprocess the rows as I tend to prefer the simplest approach I can come 
up with. Example:

def recode_rows(rows, source_encoding, target_encoding):
def recode(field):
if isinstance(field, unicode):
return field.encode(target_encoding)
elif isinstance(field, str):
return unicode(field, source_encoding).encode(target_encoding)
return unicode(field).encode(target_encoding)

return (map(recode, row) for row in rows)

rows = [[1.23], [uäöü], [uÄÖÜ.encode(latin1)], [1, 2, 3]]
writer = csv.writer(sys.stdout)
writer.writerows(recode_rows(rows, latin1, utf-8))

The only limitation I can see: target_encoding probably has to be a superset 
of ASCII.

Peter

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: pointless musings on performance

2009-11-24 Thread Tim Wintle

On Tue, 2009-11-24 at 18:25 +, Antoine Pitrou wrote:
 Le Tue, 24 Nov 2009 08:58:40 -0800, Paul Boddie a écrit :
  As you
  point out, a lot of this RISC vs. CISC analysis (and inferences
 drawn
  from Python bytecode analysis) is somewhat academic: the cost of the
  JUMP_IF_FALSE instruction is likely to be minimal in the context of
 all the activity going on to evaluate the bytecodes.
 
 Sorry, I have trouble parsing your sentence. Do you mean bytecode 
 interpretation overhead is minimal compared to the cost of actual
 useful work, or the contrary?
 (IMO both are wrong by the way)

Out of interest - has anyone else spotted that the call to
PyObject_IsTrue in the XXX_JUMP_IF_ blocks performs two unnecessary
pointer comparisons?

 ceval.c 
if (w == Py_True) {
Py_DECREF(w);
FAST_DISPATCH();
}
if (w == Py_False) {
Py_DECREF(w);
JUMPTO(oparg);
FAST_DISPATCH();
}
err = PyObject_IsTrue(w);
Py_DECREF(w);
.
.
.
==

 object.c 
PyObject_IsTrue(PyObject *v)
{
Py_ssize_t res;
if (v == Py_True)
return 1;
if (v == Py_False)
return 0;
.
.
.
==

Would it be worth in-lining the remaining part of PyObject_IsTrue in
ceval?

 Another data point I've heard is that people who have tried a very
 crude form of Python-to-C compilation (generating the exact C code
 corresponding to a function or method, using Python's C API and
 preserving dynamicity without attempting to be clever) have apparently
 reached speedups of up to 50% (in other words, twice as fast).

That's roughly what I get with Cython - which does exactly that.

Tim

-- 
http://mail.python.org/mailman/listinfo/python-list

Raw strings as input from File?

2009-11-24 Thread utabintarbo

I have a log file with full Windows paths on a line. eg:
K:\A\B\C\10xx\somerandomfilename.ext-/a1/b1/c1/10xx
\somerandomfilename.ext ; txx; 11/23/2009 15:00:16 ; 1259006416

As I try to pull in the line and process it, python changes the \10
to a \x08. This is before I can do anything with it. Is there a way
to specify that incoming lines (say, when using .readlines() ) should
be treated as raw strings?

TIA
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: attributes, properties, and accessors -- philosophy

2009-11-24 Thread Lie Ryan


Ethan Furman wrote:


Good tools to know about, and a consistent naming pattern also makes 
life easier (which I have since done ;).


Let's head towards murkier waters (at least murkier to me -- hopefully 
they can be easily clarified):  some of the attributes are read-only, 
such as record count; others are not directly exposed, but still 
settable, such as table version; and still others require a small amount 
of processing... at which point do I switch from simple attribute access 
to method access?


~Ethan~


method accessor is not pythonic, use property

property can be read-only, write-only (!), and it can process data 
before returning and setting the real attributes.

--
http://mail.python.org/mailman/listinfo/python-list

Re: Raw strings as input from File?

2009-11-24 Thread MRAB


utabintarbo wrote:

I have a log file with full Windows paths on a line. eg:
K:\A\B\C\10xx\somerandomfilename.ext-/a1/b1/c1/10xx
\somerandomfilename.ext ; txx; 11/23/2009 15:00:16 ; 1259006416

As I try to pull in the line and process it, python changes the \10
to a \x08. This is before I can do anything with it. Is there a way
to specify that incoming lines (say, when using .readlines() ) should
be treated as raw strings?


.readlines() doesn't change the \10 in a file to \x08 in the string
it returns.

Could you provide some code which shows your problem?
--
http://mail.python.org/mailman/listinfo/python-list

Re: Raw strings as input from File?

2009-11-24 Thread Carsten Haese

utabintarbo wrote:
 I have a log file with full Windows paths on a line. eg:
 K:\A\B\C\10xx\somerandomfilename.ext-/a1/b1/c1/10xx
 \somerandomfilename.ext ; txx; 11/23/2009 15:00:16 ; 1259006416
 
 As I try to pull in the line and process it, python changes the \10
 to a \x08.

Python does no such thing. When Python reads bytes from a file, it
doesn't interpret or change those bytes in any way. Either there is
something else going on here that you're not telling us, or the file
doesn't contain what you think it contains. Please show us the exact
code you're using to process this file, and show us the exact contents
of the file you're processing.

--
Carsten Haese
http://informixdb.sourceforge.net

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: How to log messages _only once_ from all modules ?

2009-11-24 Thread Vinay Sajip

On Nov 24, 3:14 pm, Ron Barak rbar...@gmail.com wrote:
 Many thanks Soltys, that did the trick (actually, no need for setting
 propagate to 0), namely,

[snip]

It might work for now, but instantiating a handler in an instance
constructor can be an anti-pattern. If you ever create multiple client
instances, for example, you would create multiple handlers and add
those to the logger (which is essentially persistent for the lifetime
of the process), which could result in multiple messages, corrupted
output or exceptions (in the general case).

It's generally better to declare module-level loggers via

logger = logging.getLogger(__name__)

or, if better granularity is wanted, loggers with names prefixed by
__name__ + '.' - and handler set up should typically be done in one
place in such a way as to not inadvertently create handlers multiple
times. A common pattern is to just add handlers to the root logger
(e.g. the basicConfig() API does this) - other loggers automatically
get to use them, under normal circumstances.

Regards,

Vinay Sajip
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: csv and mixed lists of unicode and numbers

2009-11-24 Thread Sibylle Koczian

Peter Otten schrieb:
 I'd preprocess the rows as I tend to prefer the simplest approach I can come 
 up with. Example:
 
 def recode_rows(rows, source_encoding, target_encoding):
 def recode(field):
 if isinstance(field, unicode):
 return field.encode(target_encoding)
 elif isinstance(field, str):
 return unicode(field, source_encoding).encode(target_encoding)
 return unicode(field).encode(target_encoding)
 
 return (map(recode, row) for row in rows)
 

For this case isinstance really seems to be quite reasonable. And it was
silly of me not to think of sys.stdout as file object for the example!

 rows = [[1.23], [uäöü], [uÄÖÜ.encode(latin1)], [1, 2, 3]]
 writer = csv.writer(sys.stdout)
 writer.writerows(recode_rows(rows, latin1, utf-8))
 
 The only limitation I can see: target_encoding probably has to be a superset 
 of ASCII.
 

Coping with umlauts and accents is quite enough for me.

This problem really goes away with Python 3 (tried it on another
machine), but something else changes too: in Python 2.6 the
documentation for the csv module explicitly says If csvfile is a file
object, it must be opened with the ‘b’ flag on platforms where that
makes a difference. The documentation for Python 3.1 doesn't have this
sentence, and if I do that in Python 3.1 I get for all sorts of data,
even for a list with only one integer literal:

TypeError: must be bytes or buffer, not str

I don't really understand that.

Regards,
Sibylle
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Inserting Unicode text with MySQLdb in Python 2.4-2.5?

2009-11-24 Thread Keith Hughitt

Hi John,

Thanks for the suggestions: I have finally been able to get it
working :)

In anyone else runs into the same problem, here is some example code
that works in Python 2.4+:

= Begin Example ==

#!/usr/bin/env python
#-*- coding:utf-8 -*-
import sys
def main():
import MySQLdb, getpass

admin = raw_input(Database admin: )
pw = getpass.getpass(Password: )
db = MySQLdb.connect(use_unicode=True, charset = utf8,
user=admin, passwd=pw)

cursor = db.cursor()
try:
cursor.execute(DROP DATABASE IF EXISTS unicode_test;)
cursor.execute(CREATE DATABASE unicode_test DEFAULT CHARACTER
SET utf8;)
except:
print 

cursor.execute('''
CREATE TABLE `unicode_test`.`test` (
  `id`  SMALLINT unsigned NOT NULL AUTO_INCREMENT,
  `name`VARCHAR(255) NOT NULL,
   PRIMARY KEY (`id`), INDEX (`id`)
) DEFAULT CHARSET=utf8;''')

cursor.execute(INSERT INTO `unicode_test`.`test` VALUES (NULL,
'%s'); % uÅngström)

# Test 1
print Just printing: %s % 'Ångström'

# Test 2
cursor.execute(SELECT name FROM unicode_test.test;)
print From database: %s % cursor.fetchone()[0]

# Test 3 (Manual)
print 'To verify manually: mysql -u %s -p -e SELECT name FROM
unicode_test.test' % admin

if __name__ == '__main__':
sys.exit(main())

= End Example 

Thanks all for taking the time to help!

Best,
Keith
-- 
http://mail.python.org/mailman/listinfo/python-list

Creating a drop down filter in Admin site

2009-11-24 Thread Jase

Hoping someone could help me out. I am updating an admin site and
added a couple list_filters. They are rather long so I wanted to
turn them into a drop down filter. Can this be done? Any suggestions?

Thanks,

Jase
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Newsgroup for beginners

2009-11-24 Thread bruno.desthuilli...@gmail.com

On 20 nov, 20:42, Ethan Furman et...@stoneleaf.us wrote:
 Aahz wrote:
  In article hdt6tb$9d...@reader1.panix.com,
  Grant Edwards  inva...@invalid.invalid wrote:

 You've really got to try pretty hard to create one.  But if you
 want to, here's how to do it:

 1) Start by complaining that your program doesn't work because
    of a bug in Python.

  [...]

  Post of the month!

 I'll second that!  I really needed a good laugh.  Many thanks!

So I'll thrice it - FWIW it indeed made it's way to the weekly python
(thanks the python-url team), but deserves much more than that. I was
so hilarious my son came out of it's room and even tried to read the
post by himself - I just wasn't able to calm down and explain him what
this was about.

Grant, if we ever meet, remind me to pay you a couple beers. Cheers !
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Where to put the error handing test?

2009-11-24 Thread Lie Ryan


Peng Yu wrote:

On Tue, Nov 24, 2009 at 4:58 AM, Dave Angel da...@ieee.org wrote:


I'll put an extra emphasis on this:
Your question is so open-ended as to be unanswerable.  




I'll still confused by the guideline that an error should be caught as
early as possible.


but not too early. Errors must be RAISED as early as possible. Errors 
must be CAUGHT as soon as there is enough information to handle the errors.



Suppose I have the following call chain

f1() -- f2() -- f3() -- f4()

The input in f1() might cause an error in f4(). However, this error
can of cause be caught by f1(), whenever I want to do so. In the worst
case, I could duplicate the code of f2 and f3, and the test code in f4
to f1(), to catch the error in f1 rather than f4. But I don't think
that this is what you mean.


Why would f1() have faulty data? Does it come from external input? Then 
the input must be invalidated at f1(). Then f2(), f3(), and f4() does 
not require any argument checking since it is assumed that f1() calls 
them with good input.


Of course, there is cases where data validation may not be on f1. For 
example, if f1() is a function that receives keyboard input for a 
filename f1 validates whether the filename is valid. Then f2 might open 
the file and validate that the file is of the expected type (if f2 
expect a .csv file, but given an .xls file, f2 would scream out an 
error). f3 received rows/lines of data from f2 and extracts 
fields/columns from the rows; but some of the data might be faulty and 
these must be filtered out before the data is transformed by the f4. f4 
assumes f3 cleans the row and so f4 is does not do error-checking. The 
transformed data then is rewritten back by f3. Now there is an f5 which 
creates new data, the new data needs to be transformed by f4 as well and 
since f5 creates a new data, there is no way for it to create invalid 
data. Now assume f6 merges new data from another csv file; f6's data 
will also be transformed by f4, f6 can use the same validation as in f3 
but why not turn f2,f3 instead to open the other file? Now f7 will merge 
data from multiple f3 streams and transforms by f4. Then comes f8 which 
creates new data from user input, the user-inputted data will need some 
checking; and now we have trouble since the data validation is inside 
f3. But we can easily factor out the validation part into f9 and call f9 
from f3 and f8.


The example case still checks for problem as early as possible. It 
checks for problem when it is possible to determine whether a particular 
condition is problematic. The as early as possible guideline does not 
mean f1, a filename input function, must check whether the csv fields 
contains valid data. But f2, the file opening function, must be given a 
valid filename; f3, the file parser, must be given a valid file object 
with the proper type; f4, the transformer, must be given a valid row to 
transform; etc. f2 should not need to check it is given a valid 
filename, it's f1 job to validate it; f3 should not need to check 
whether the file object is of the proper type; f4 should not need to 
check it is given a valid row tuple; and so on...



Then the problem is where to put the test code more effectively. I
would consider 'whether it is obvious to test the condition in the
give function' as the guideline. However, it might be equal obvious to
test the same thing two functions, for example, f1 and f4.

In this case, I thought originally that I should put the test code in
f1 rather than f4, if f1, f2, f3 and f4 are all the functions that I
have in the package that I am making. But it is possible that some
time later I added the function f5(),...,f10() that calls f4(). Since
f4 doesn't have the test code, f5(),...,f10() should have the same
test code. This is clearly a redundancy to the code. If I move the
test code to f4(), there is a redundancy of the code between f1 and
f4.


I can't think of a good example where such redundancy would happen and 
there is no other, better way to walk around it. Could you provide a 
CONCRETE example? One that might happen, instead one that could 
theoretically happen.

--
http://mail.python.org/mailman/listinfo/python-list

Re: Raw strings as input from File?

2009-11-24 Thread utabintarbo

On Nov 24, 3:27 pm, MRAB pyt...@mrabarnett.plus.com wrote:

 .readlines() doesn't change the \10 in a file to \x08 in the string
 it returns.

 Could you provide some code which shows your problem?

Here is the code block I have so far:
for l in open(CONTENTS, 'r').readlines():
f = os.path.splitext(os.path.split(l.split('-')[0]))[0]
if f in os.listdir(DIR1) and os.path.isdir(os.path.join(DIR1,f)):
shutil.rmtree(os.path.join(DIR1,f))
if f in os.listdir(DIR2) and os.path.isdir(os.path.join(DIR2,f)):
shutil.rmtree(os.path.join(DIR2,f))

I am trying to find dirs with the basename of the initial path less
the extension in both DIR1 and DIR2

A minimally obfuscated line from the log file:
K:\sm\SMI\des\RS\Pat\10DJ\121.D5-30\1215B-B-D5-BSHOE-MM.smz-/arch_m1/
smi/des/RS/Pat/10DJ/121.D5-30\1215B-B-D5-BSHOE-MM.smz ; t9480rc ;
11/24/2009 08:16:42 ; 1259068602

What I get from the debugger/python shell:
'K:\\sm\\SMI\\des\\RS\\Pat\x08DJQ.D5-30Q5B-B-D5-BSHOE-MM.smz-/arch_m1/
smi/des/RS/Pat/10DJ/121.D5-30/1215B-B-D5-BSHOE-MM.smz ; t9480rc ;
11/24/2009 08:16:42 ; 1259068602'

TIA

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Creating a drop down filter in Admin site

2009-11-24 Thread Jon Clements

On Nov 24, 9:08 pm, Jase jasewar...@gmail.com wrote:
 Hoping someone could help me out. I am updating an admin site and
 added a couple list_filters. They are rather long so I wanted to
 turn them into a drop down filter. Can this be done? Any suggestions?

 Thanks,

 Jase

I'm guessing you mean Django - You may well get advice here, but the
Django Users google group and the associated documentation of
djangoproject and how to customise the admin interface is your best
start.

hth,
Jon.
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Securing a multiprocessing.BaseManager connection via SSL

2009-11-24 Thread Xavier Sanz

Hi Jonas

I've having same need so i found a solution but you need to hack a bit

I am using python 2.6.3 but i suppose it could be applicable to your
case.

First of all u need to edit /usr/lib/python2.6/lib/python2.6/
multiprocessing/connection.py

This is mine:

#
# A higher level module for using sockets (or Windows named pipes)
#
# multiprocessing/connection.py
#
# Copyright (c) 2006-2008, R Oudkerk --- see COPYING.txt
#

__all__ = [ 'Client', 'Listener', 'Pipe' ]

import os
import sys
import socket
import errno
import time
import tempfile
import itertools
# Added by Xavi
import ssl

import _multiprocessing
from multiprocessing import current_process, AuthenticationError
from multiprocessing.util import get_temp_dir, Finalize, sub_debug,
debug
from multiprocessing.forking import duplicate, close


#
#
#

BUFSIZE = 8192

_mmap_counter = itertools.count()

default_family = 'AF_INET'
families = ['AF_INET']

if hasattr(socket, 'AF_UNIX'):
default_family = 'AF_UNIX'
families += ['AF_UNIX']

if sys.platform == 'win32':
default_family = 'AF_PIPE'
families += ['AF_PIPE']

#
#
#

def arbitrary_address(family):
'''
Return an arbitrary free address for the given family
'''
if family == 'AF_INET':
return ('localhost', 0)
elif family == 'AF_UNIX':
return tempfile.mktemp(prefix='listener-', dir=get_temp_dir())
elif family == 'AF_PIPE':
return tempfile.mktemp(prefix=r'\\.\pipe\pyc-%d-%d-' %
   (os.getpid(), _mmap_counter.next()))
else:
raise ValueError('unrecognized family')


def address_type(address):
'''
Return the types of the address

This can be 'AF_INET', 'AF_UNIX', or 'AF_PIPE'
'''
if type(address) == tuple:
return 'AF_INET'
elif type(address) is str and address.startswith(''):
return 'AF_PIPE'
elif type(address) is str:
return 'AF_UNIX'
else:
raise ValueError('address type of %r unrecognized' % address)

#
# Public functions
#

class Listener(object):
'''
Returns a listener object.

This is a wrapper for a bound socket which is 'listening' for
connections, or for a Windows named pipe.
'''
def __init__(self, address=None, family=None, backlog=1,
authkey=None, sslsock=True):
family = family or (address and address_type(address)) \
 or default_family
address = address or arbitrary_address(family)

if family == 'AF_PIPE':
self._listener = PipeListener(address, backlog)
else:
self._listener = SocketListener(address, family, backlog,
sslsock)
if authkey is not None and not isinstance(authkey, bytes):
raise TypeError, 'authkey should be a byte string'

self._authkey = authkey

def accept(self):
'''
Accept a connection on the bound socket or named pipe of
`self`.

Returns a `Connection` object.
'''
c = self._listener.accept()
if self._authkey:
deliver_challenge(c, self._authkey)
answer_challenge(c, self._authkey)
return c

def close(self):
'''
Close the bound socket or named pipe of `self`.
'''
return self._listener.close()

address = property(lambda self: self._listener._address)
last_accepted = property(lambda self:
self._listener._last_accepted)


def Client(address, family=None, authkey=None, sslsock=True):
'''
Returns a connection to the address of a `Listener`
'''
family = family or address_type(address)
if family == 'AF_PIPE':
c = PipeClient(address)
else:
c = SocketClient(address,sslsock)

if authkey is not None and not isinstance(authkey, bytes):
raise TypeError, 'authkey should be a byte string'

if authkey is not None:
answer_challenge(c, authkey)
deliver_challenge(c, authkey)

return c


if sys.platform != 'win32':

def Pipe(duplex=True):
'''
Returns pair of connection objects at either end of a pipe
'''
if duplex:
s1, s2 = socket.socketpair()
c1 = _multiprocessing.Connection(os.dup(s1.fileno()))
c2 = _multiprocessing.Connection(os.dup(s2.fileno()))
s1.close()
s2.close()
else:
fd1, fd2 = os.pipe()
c1 = _multiprocessing.Connection(fd1, writable=False)
c2 = _multiprocessing.Connection(fd2, readable=False)

return c1, c2

else:

from ._multiprocessing import win32

def Pipe(duplex=True):
'''
Returns pair of connection objects at either end of a pipe
'''
address = arbitrary_address('AF_PIPE')
if duplex:
openmode = win32.PIPE_ACCESS_DUPLEX
access = win32.GENERIC_READ | win32.GENERIC_WRITE
obsize, ibsize = BUFSIZE, BUFSIZE
else:
openmode = win32.PIPE_ACCESS_INBOUND

Re: Raw strings as input from File?

2009-11-24 Thread Jon Clements

On Nov 24, 9:20 pm, utabintarbo utabinta...@gmail.com wrote:
 On Nov 24, 3:27 pm, MRAB pyt...@mrabarnett.plus.com wrote:



  .readlines() doesn't change the \10 in a file to \x08 in the string
  it returns.

  Could you provide some code which shows your problem?

 Here is the code block I have so far:
 for l in open(CONTENTS, 'r').readlines():
     f = os.path.splitext(os.path.split(l.split('-')[0]))[0]
     if f in os.listdir(DIR1) and os.path.isdir(os.path.join(DIR1,f)):
         shutil.rmtree(os.path.join(DIR1,f))
         if f in os.listdir(DIR2) and os.path.isdir(os.path.join(DIR2,f)):
                 shutil.rmtree(os.path.join(DIR2,f))

 I am trying to find dirs with the basename of the initial path less
 the extension in both DIR1 and DIR2

 A minimally obfuscated line from the log file:
 K:\sm\SMI\des\RS\Pat\10DJ\121.D5-30\1215B-B-D5-BSHOE-MM.smz-/arch_m1/
 smi/des/RS/Pat/10DJ/121.D5-30\1215B-B-D5-BSHOE-MM.smz ; t9480rc ;
 11/24/2009 08:16:42 ; 1259068602

 What I get from the debugger/python shell:
 'K:\\sm\\SMI\\des\\RS\\Pat\x08DJQ.D5-30Q5B-B-D5-BSHOE-MM.smz-/arch_m1/
 smi/des/RS/Pat/10DJ/121.D5-30/1215B-B-D5-BSHOE-MM.smz ; t9480rc ;
 11/24/2009 08:16:42 ; 1259068602'

 TIA

j...@jon-desktop:~/pytest$ cat log.txt
K:\sm\SMI\des\RS\Pat\10DJ\121.D5-30\1215B-B-D5-BSHOE-MM.smz-/arch_m1/
smi/des/RS/Pat/10DJ/121.D5-30\1215B-B-D5-BSHOE-MM.smz ; t9480rc ;
11/24/2009 08:16:42 ; 1259068602

 log = open('/home/jon/pytest/log.txt', 'r').readlines()
 log
['K:\\sm\\SMI\\des\\RS\\Pat\\10DJ\\121.D5-30\\1215B-B-D5-BSHOE-MM.smz-
/arch_m1/\n', 'smi/des/RS/Pat/10DJ/121.D5-30\\1215B-B-D5-BSHOE-
MM.smz ; t9480rc ;\n', '11/24/2009 08:16:42 ; 1259068602\n']

See -- it's not doing anything :)

Although, Pat\x08DJQ.D5-30Q5B-B-D5-BSHOE-MM.smz and Pat
\x08DJQ.D5-30Q5B-B-D5-BSHOE-MM.smz seem to be fairly different -- are
you sure you're posting the correct output!?

Jon.
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Raw strings as input from File?

2009-11-24 Thread Jon Clements

On Nov 24, 9:50 pm, Jon Clements jon...@googlemail.com wrote:
 On Nov 24, 9:20 pm, utabintarbo utabinta...@gmail.com wrote:
[snip]
 Although, Pat\x08DJQ.D5-30Q5B-B-D5-BSHOE-MM.smz and Pat
 \x08DJQ.D5-30Q5B-B-D5-BSHOE-MM.smz seem to be fairly different -- are
 you sure you're posting the correct output!?


Ugh... let's try that...

Pat\10DJ\121.D5-30\1215B-B-D5-BSHOE-MM.smz
Pat\x08DJQ.D5-30Q5B-B-D5-BSHOE-MM.smz

Jon.
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: csv and mixed lists of unicode and numbers

2009-11-24 Thread Peter Otten

Sibylle Koczian wrote:

 This problem really goes away with Python 3 (tried it on another
 machine), but something else changes too: in Python 2.6 the
 documentation for the csv module explicitly says If csvfile is a file
 object, it must be opened with the ‘b’ flag on platforms where that
 makes a difference. The documentation for Python 3.1 doesn't have this
 sentence, and if I do that in Python 3.1 I get for all sorts of data,
 even for a list with only one integer literal:
 
 TypeError: must be bytes or buffer, not str

Read the documentation for open() at

http://docs.python.org/3.1/library/functions.html#open

There are significant changes with respect to 2.x; you won't even get a file 
object anymore:

 open(tmp.txt, w)
_io.TextIOWrapper name='tmp.txt' encoding='UTF-8'
 _.write(yadda)
5
 open(tmp.dat, wb)
_io.BufferedWriter name='tmp.dat'
 _.write(yadda)
Traceback (most recent call last):
  File stdin, line 1, in module
TypeError: write() argument 1 must be bytes or buffer, not str
 open(tmp.dat, wb).write(byadda)
5

If you specify the b flag in 3.x the write() method expects bytes, not 
str. The translation of newlines is now controlled by the newline 
argument.

Peter

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: pointless musings on performance

2009-11-24 Thread Benjamin Peterson

Tim Wintle tim.wintle at teamrubber.com writes:
 Out of interest - has anyone else spotted that the call to
 PyObject_IsTrue in the XXX_JUMP_IF_ blocks performs two unnecessary
 pointer comparisons?

I doubt two pointer comparisons will make much of a difference.

 Would it be worth in-lining the remaining part of PyObject_IsTrue in
 ceval?

Inlining by hand is prone to error and maintainability problems.




-- 
http://mail.python.org/mailman/listinfo/python-list

Re: pointless musings on performance


Chris Rebert wrote:

On Tue, Nov 24, 2009 at 4:31 AM, Rob Williscroft r...@freenet.co.uk wrote:

mk wrote in news:mailman.915.1259064240.2873.python-l...@python.org in
comp.lang.python:


def pythonic():
def unpythonic():



Decidedly counterintuitive: are there special optimizations for if
nonevar: type of statements in cpython implementation?


from dis import dis

dis( unpythonic )

18  31 LOAD_FAST0 (nonevar)
34 LOAD_CONST   0 (None)
37 COMPARE_OP   9 (is not)
40 JUMP_IF_FALSE4 (to 47)

dis( pythonic )

11  31 LOAD_FAST0 (nonevar)
34 JUMP_IF_FALSE4 (to 41)


In other words, CPython doesn't happen to optimize `if nonevar is not
None` as much as it theoretically could (which would essentially
require a JUMP_IF_NONE opcode). Since CPython isn't known for doing
fancy optimizations, this isn't surprising.


There is a limit of 256 bytecodes. I believe there are currently fewer. 
It would seem that adding bytecodes that combine current pairs would 
speed the interpreter, depending on how often the combined pair is used. 
And people have looked for frequent pairs across a corpus of code, and 
proposed additions.


However, additional bytecodes enlarge the basic bytecode eval loop, 
possibly (and sometimes actually) leading to to more processor cache 
misses and slower execution. At least some proposed non-essential 
additions have been rejected for the reason.


Also, even though CPython-specific bytecodes are outside the language 
definition, and could theoretically be revamped every minor (x.y) 
release, there is a cost to change. Rewrite the ast to bytecode 
compiler, rewrite dis, rewrite the dis doc, and burden anyone concerned 
with multiple versions to remember. So in practice, change is minimized 
and unassigned bytecodes are left open for the future.


Optimizations that make better use of a fix set of bytecodes are a 
different topic. Guido is conservative because historically, compilier 
optimizations are too often not exactly equivalent to naive, obviously 
correct code in some overlooked corner case, leading to subtle, 
occasional bugs. Of course, byte code changes could mess-up current 
optimizations that are performed.


Terry Jan Reedy

--
http://mail.python.org/mailman/listinfo/python-list

Re: Securing a multiprocessing.BaseManager connection via SSL

2009-11-24 Thread Xavier Sanz

I recommend u to test it before use it in production environment.

On 24 nov, 22:45, Xavier Sanz x.san...@gmail.com wrote:
 Hi Jonas

 I've having same need so i found a solution but you need to hack a bit

 I am using python 2.6.3 but i suppose it could be applicable to your
 case.

 First of all u need to edit 
 /usr/lib/python2.6/lib/python2.6/multiprocessing/connection.py

 This is mine:

 #
 # A higher level module for using sockets (or Windows named pipes)
 #
 #multiprocessing/connection.py
 #
 # Copyright (c) 2006-2008, R Oudkerk --- see COPYING.txt
 #

 __all__ = [ 'Client', 'Listener', 'Pipe' ]

 import os
 import sys
 import socket
 import errno
 import time
 import tempfile
 import itertools
 # Added by Xavi
 import ssl

 import _multiprocessing
 frommultiprocessingimport current_process, AuthenticationError
 frommultiprocessing.util import get_temp_dir, Finalize, sub_debug,
 debug
 frommultiprocessing.forking import duplicate, close

 #
 #
 #

 BUFSIZE = 8192

 _mmap_counter = itertools.count()

 default_family = 'AF_INET'
 families = ['AF_INET']

 if hasattr(socket, 'AF_UNIX'):
     default_family = 'AF_UNIX'
     families += ['AF_UNIX']

 if sys.platform == 'win32':
     default_family = 'AF_PIPE'
     families += ['AF_PIPE']

 #
 #
 #

 def arbitrary_address(family):
     '''
     Return an arbitrary free address for the given family
     '''
     if family == 'AF_INET':
         return ('localhost', 0)
     elif family == 'AF_UNIX':
         return tempfile.mktemp(prefix='listener-', dir=get_temp_dir())
     elif family == 'AF_PIPE':
         return tempfile.mktemp(prefix=r'\\.\pipe\pyc-%d-%d-' %
                                (os.getpid(), _mmap_counter.next()))
     else:
         raise ValueError('unrecognized family')

 def address_type(address):
     '''
     Return the types of the address

     This can be 'AF_INET', 'AF_UNIX', or 'AF_PIPE'
     '''
     if type(address) == tuple:
         return 'AF_INET'
     elif type(address) is str and address.startswith(''):
         return 'AF_PIPE'
     elif type(address) is str:
         return 'AF_UNIX'
     else:
         raise ValueError('address type of %r unrecognized' % address)

 #
 # Public functions
 #

 class Listener(object):
     '''
     Returns a listener object.

     This is a wrapper for a bound socket which is 'listening' for
     connections, or for a Windows named pipe.
     '''
     def __init__(self, address=None, family=None, backlog=1,
 authkey=None, sslsock=True):
         family = family or (address and address_type(address)) \
                  or default_family
         address = address or arbitrary_address(family)

         if family == 'AF_PIPE':
             self._listener = PipeListener(address, backlog)
         else:
             self._listener = SocketListener(address, family, backlog,
 sslsock)
         if authkey is not None and not isinstance(authkey, bytes):
             raise TypeError, 'authkey should be a byte string'

         self._authkey = authkey

     def accept(self):
         '''
         Accept a connection on the bound socket or named pipe of
 `self`.

         Returns a `Connection` object.
         '''
         c = self._listener.accept()
         if self._authkey:
             deliver_challenge(c, self._authkey)
             answer_challenge(c, self._authkey)
         return c

     def close(self):
         '''
         Close the bound socket or named pipe of `self`.
         '''
         return self._listener.close()

     address = property(lambda self: self._listener._address)
     last_accepted = property(lambda self:
 self._listener._last_accepted)

 def Client(address, family=None, authkey=None, sslsock=True):
     '''
     Returns a connection to the address of a `Listener`
     '''
     family = family or address_type(address)
     if family == 'AF_PIPE':
         c = PipeClient(address)
     else:
         c = SocketClient(address,sslsock)

     if authkey is not None and not isinstance(authkey, bytes):
         raise TypeError, 'authkey should be a byte string'

     if authkey is not None:
         answer_challenge(c, authkey)
         deliver_challenge(c, authkey)

     return c

 if sys.platform != 'win32':

     def Pipe(duplex=True):
         '''
         Returns pair of connection objects at either end of a pipe
         '''
         if duplex:
             s1, s2 = socket.socketpair()
             c1 = _multiprocessing.Connection(os.dup(s1.fileno()))
             c2 = _multiprocessing.Connection(os.dup(s2.fileno()))
             s1.close()
             s2.close()
         else:
             fd1, fd2 = os.pipe()
             c1 = _multiprocessing.Connection(fd1, writable=False)
             c2 = _multiprocessing.Connection(fd2, readable=False)

         return c1, c2

 else:

     from ._multiprocessing import win32

     def Pipe(duplex=True):
         '''
         Returns pair of connection objects at either end of a pipe
         '''
         address =

Re: Beginning Question about Python functions, parameters...


Peter Otten wrote:

Terry Reedy wrote:



remember exactly what was stored. Maybe a typo.


That's a bug in the store() function

# as posted
def store(data, full_name):
names = full_name.split()
if len(names) == 2: names.insert(1, '')
labels = 'first', 'middle', 'last'
for label, name in zip(labels, names):
people = lookup(data, label, name)
if people:
people.append(full_name)
else:
data[label][name] = [full_name]

# what was probably intended
def store(data, full_name):
names = full_name.split()
if len(names) == 2: names.insert(1, '')
labels = 'first', 'middle', 'last'
for label, name in zip(labels, names):
people = lookup(data, label, name)
if people:
people.append(full_name)
else:
data[label][name] = [full_name]


(Note indent error, which I overlooked)
It is a nuisance when people post untested code, without identifying it 
as such. Publishing untested but trivial-to-test code like this (one 
print statement needed), especially in a book for learners, is really bad.


(I am trying to do better in the book I am working on.)

tjr

--
http://mail.python.org/mailman/listinfo/python-list

Re: Perl conversion to python...

2009-11-24 Thread Hans Mulder


J Kenneth King wrote:

Benjamin Schollnick bscholln...@gmail.com writes:



select(undef, undef, undef, 0.01);



# select() is a system call in Perl..
# insert Python equivalent here


That would be:

  time.sleep(0.01)

Perl has a sleep() built-in, but it truncates its argument to an int.
If you want to sleep less than a second in Perl, you have to use select
as shown.

Not so in Python.  Python's time.sleep() takes a float argument and
calls some platform-dependent function that provides sleeping with
sub-second accuracy.  On some platforms, it ends up calling the
C level select() function.

Keep in mind that in both languages, your program may end up sleeping
longer than it should due to scheduling or other activity on the system.


Hope this helps,

-- HansM

--
http://mail.python.org/mailman/listinfo/python-list

Re: UnicodeDecodeError? Argh! Nothing works! I'm tired and hurting and...

On Tue, 24 Nov 2009 13:19:10 -0500, Chris Jones wrote:

 On Tue, Nov 24, 2009 at 08:02:09AM EST, Steven D'Aprano wrote:
 
 Good grief, it's about six weeks away from 2010 and Thunderbird still
 uses mbox as it's default mail box format. Hello, the nineties called,
 they want their mail formats back! Are the tbird developers on crack or
 something? I can't believe that they're still using that crappy format.
 
 No, I tell a lie. I can believe it far too well.
 
 :-)
 
 I realize that's somewhat OT, but what mail box format do you recommend,
 and why?

maildir++

http://en.wikipedia.org/wiki/Maildir

Corruption is less likely, if there is corruption you'll only lose a 
single message rather than potentially everything in the mail folder[*], 
at a pinch you can read the emails using a text editor or easily grep 
through them, and compacting the mail folder is lightning fast, there's 
no wasted space in the mail folder, and there's no need to mangle lines 
starting with From  in the body of the email.

The only major downside is that because you're dealing with potentially 
thousands of smallish files, it *may* have reduced performance on some 
older file systems that don't deal well with lots of files. These days, 
that's not a real issue.

Oh yes, and people using Windows can't use maildir because (1) it doesn't 
allow colons in names, and (2) it doesn't have atomic renames. Neither of 
these are insurmountable problems: an implementation could substitute 
another character for the colon, and while that would be a technical 
violation of the standard, it would still work. And the lack of atomic 
renames would simply mean that implementations have to be more careful 
about not having two threads writing to the one mailbox at the same time.




[*] I'm assuming normal oops there's a bug in the mail client code 
corruption rather than I got drunk and started deleting random files and 
directories corruption.



-- 
Steven
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Where to put the error handing test?

On Tue, 24 Nov 2009 10:14:19 -0600, Peng Yu wrote:

 I'll still confused by the guideline that an error should be caught as
 early as possible.

I think you are confused by what that means. It means the error should be 
caught as early as possible in *time*, not in the function chain.

In Python, errors are always generated as soon as they occur. But imagine 
a system where you have a function that returns -1 on error and some 
integer on success. Then it is possible for that -1 error code to be 
passed along from one function to another to another to another before 
finally causing a fatal error, by which time you have no idea where it 
came from. That's what should be prohibited.



 Suppose I have the following call chain
 
 f1() -- f2() -- f3() -- f4()
 
 The input in f1() might cause an error in f4(). 

Assuming that none of the functions have side-effects, then f4 is the 
right place to catch the error. To do otherwise means that f1 needs to 
know all the possible things that can go wrong in f2, and f3, and f4.


 However, this error can
 of cause be caught by f1(), whenever I want to do so. In the worst case,
 I could duplicate the code of f2 and f3, and the test code in f4 to
 f1(), to catch the error in f1 rather than f4. But I don't think that
 this is what you mean.
 
 Then the problem is where to put the test code more effectively. I would
 consider 'whether it is obvious to test the condition in the give
 function' as the guideline. However, it might be equal obvious to test
 the same thing two functions, for example, f1 and f4.

You're not a mathematician, are you?

For each function, define its domain: the values it must work with. 
Suppose f1 should return a result for (e.g.) all integers, but f2 only 
returns a result for *positive* integers.

Then it is very simple: f2 will raise an error if given zero or negative 
values, and so f1 must either:

* avoid that error by handling zero or negative separately; or
* catch the error and then handle zero or negative separately.

Simplified example:

def f2(x):
if x  0:
return x+1
raise ValueError


def f1(x):  # version 1
if x  0:
return f2(x)
else:
return something else

# or

def f1(x):  # version 2
try:
return f2(x)
except ValueError:
return something else


Version two is the preferred way to do it, since that means f1 doesn't 
need to know what the domain of f2 is. But if f2 has side-effects 
(usually a bad idea) then version one is better.


But what if f1 and f2 have the same domain?

Then it is very simple: put the error checking wherever it makes sense. 
If f2 is public, put the error checking there, and then f1 can avoid 
duplicating the error checking.

But what if f1 has the more restrictive domain? Then put the error 
checking in f1.


-- 
Steven
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: csv and mixed lists of unicode and numbers


Sibylle Koczian wrote:

Peter Otten schrieb:
I'd preprocess the rows as I tend to prefer the simplest approach I can come 
up with. Example:


def recode_rows(rows, source_encoding, target_encoding):
def recode(field):
if isinstance(field, unicode):
return field.encode(target_encoding)
elif isinstance(field, str):
return unicode(field, source_encoding).encode(target_encoding)
return unicode(field).encode(target_encoding)

return (map(recode, row) for row in rows)



For this case isinstance really seems to be quite reasonable. And it was
silly of me not to think of sys.stdout as file object for the example!


rows = [[1.23], [uäöü], [uÄÖÜ.encode(latin1)], [1, 2, 3]]
writer = csv.writer(sys.stdout)
writer.writerows(recode_rows(rows, latin1, utf-8))

The only limitation I can see: target_encoding probably has to be a superset 
of ASCII.




Coping with umlauts and accents is quite enough for me.

This problem really goes away with Python 3 (tried it on another
machine), but something else changes too: in Python 2.6 the
documentation for the csv module explicitly says If csvfile is a file
object, it must be opened with the ‘b’ flag on platforms where that
makes a difference. The documentation for Python 3.1 doesn't have this
sentence, and if I do that in Python 3.1 I get for all sorts of data,
even for a list with only one integer literal:

TypeError: must be bytes or buffer, not str

I don't really understand that.


In Python 3, a file opened in 'b' mode is for reading and writing bytes 
with no encoding/decoding. I believe cvs works with files in text mode 
as it returns and expects strings/text for reading and writing. Perhaps 
the cvs doc should say must not be opened in 'b' mode. Not sure.


tjr

--
http://mail.python.org/mailman/listinfo/python-list

Re: Python-URL! - weekly Python news and links (Nov 24)

2009-11-24 Thread Erik Max Francis


Bruno Desthuilliers wrote:

Cameron Laird a écrit :


Grant Edwards on the best way to get help from this group :)

http://groups.google.com/group/comp.lang.python/t/b8a0c32cae495522/21e80ac383745d88?#21e80ac383745d88


This one really deserves a POTM award !-)


It is a bit surprising to see on IRC how often people are asking for 
help but trying desperately to either 1. not provide enough information 
so that anyone could possibly help, or (even weirder) 2. argue with the 
people trying to help with them or insist they didn't really need the 
help anyway once they get the answer they're looking for.  The former is 
surely just laziness, but there's something psychological going on with 
the latter.


--
Erik Max Francis  m...@alcyone.com  http://www.alcyone.com/max/
 San Jose, CA, USA  37 18 N 121 57 W  AIM/Y!M/Skype erikmaxfrancis
  Nine worlds I remember.
   -- Icelandic Edda of Snorri Sturluson
--
http://mail.python.org/mailman/listinfo/python-list

Re: Raw strings as input from File?