Re: suggestions please "what should i watch for/guard against' in a file upload situation?"

2010-10-07 Thread MRAB

On 06/10/2010 21:01, Martin Gregorie wrote:

On Wed, 06 Oct 2010 09:02:21 -0700, geekbuntu wrote:


in general, what are things i would want to 'watch for/guard against' in
a file upload situation?

i have my file upload working (in the self-made framework @ work without
any concession for multipart form uploads), but was told to make sure
it's cleansed and cannot do any harm inside the system.


Off the top of my head, and assuming that you get passed the exact
filename that the user entered:

- The user may need to use an absolute pathname to upload a file
   that isn't in his current directory, so retain only the basename
   by discarding the rightmost slash and everything to the left of it:
 /home/auser/photos/my_photo.jpg   ===>  my_photo.jpg
 c:\My Photos\My Photo.jpg ===>  My Photo.jpg

- If your target system doesn't like spaces in names or you want to be
   on the safe side there, replace spaces in the name with underscores:
 My Photo.jpg ===> My_Photo.jpg

- reject any filenames that could cause the receiving system to do
   dangerous things, e.g. .EXE or .SCR if the upload target is Windows.
   This list will be different for each upload target, so make it
   configurable.

   You can't assume anything about else about the extension.
   .py .c .txt and .html are all valid in the operating systems I use
   and so are their capitalised equivalents.


A whitelist is better than a blacklist; instead of rejecting what you
know could be dangerous, accept what you know _isn't_ dangerous.


- check whether the file already exists. You need
   rules about what to do if it exists (do you reject the upload,
   silently overwrite, or alter the name, e.g. by adding a numeric
   suffix to make the name unique:

  my_photo.jpg  ===>   my_photo-01.jpg

- run the application in your upload target directory and put the
   uploaded file there or, better, into a configured uploads directory
   by prepending it to the file name:

 my_photo.jpg   ===>   /home/upload_user/uploads/my_photo.jpg

- make sure you document the process so that a user can work out
   what has happened to his file and why if you have to reject it
   or alter its name.


not sure but any suggestions or examples are most welcome :)


There's probably something I've forgotten, but that list should get you
going.


Maximum file size, perhaps?
--
http://mail.python.org/mailman/listinfo/python-list


Re: list parameter of a recursive function

2010-10-07 Thread Chris Rebert
On Wed, Oct 6, 2010 at 10:25 PM, TP  wrote:
> Diez B. Roggisch wrote:
>
>> Back to your example: your solution is perfectly fine, although a bit
>> costly and more error-prone if you happen to forget to create a copy.
>> A safer alternative for these cases is using tuples, because they are
>> immutable.
>
> Thanks Diez for your explanation.
> The problem with tuples is that it is not easy to modify them: in my case, I
> need to append a string to the tuple at each recursive step.
> So, it seems to me that I need to write a loop to modify the tuple, because
> on a simple example:
>
 a=("foo", "bar")
 b=(a[0],a[1],"toto")
 b
> (u'foo', u'bar', u'toto')
>
> I do not find any other means to obtain that result for b. With lists, I can
> use ".extend()".
> Am I right?

Nope, sorry:
>>> a = ("foo", "bar")
>>> s = "toto"
>>> b = a + (s,)
>>> b
('foo', 'bar', 'toto')

Cheers,
Chris
--
http://blog.rebertia.com
-- 
http://mail.python.org/mailman/listinfo/python-list


Re: list parameter of a recursive function

2010-10-07 Thread Diez B. Roggisch
TP  writes:

> Diez B. Roggisch wrote:
>
>> Back to your example: your solution is perfectly fine, although a bit
>> costly and more error-prone if you happen to forget to create a copy.
>> A safer alternative for these cases is using tuples, because they are
>> immutable.
>
> Thanks Diez for your explanation.
> The problem with tuples is that it is not easy to modify them: in my case, I 
> need to append a string to the tuple at each recursive step.
> So, it seems to me that I need to write a loop to modify the tuple, because 
> on a simple example:
>
 a=("foo", "bar")
 b=(a[0],a[1],"toto")
 b
> (u'foo', u'bar', u'toto')
>
> I do not find any other means to obtain that result for b. With lists, I can 
> use ".extend()".
> Am I right?

Yes and no - again. Of course you cannot use extend. But you can
concatenate:

b = a + ("toto",)

Note the trailing comma. A common misconception is to think that tuples
are build using parethesis. They aren't. They are build using the
comma-operator, and needed for disambiguation, e.g. in this case:

foo(a, (b, c))

Diez
-- 
http://mail.python.org/mailman/listinfo/python-list


Re: mantissa and exponent in base 10

2010-10-07 Thread Robert Kern

On 10/6/10 5:54 PM, Steven D'Aprano wrote:

I want the mantissa and decimal exponent of a float, in base 10:

mantissa and exponent of 1.2345e7
=>  (1.2345, 7)

(0.12345, 8) would also be acceptable.


The math module has a frexp() function, but it produces a base-2 exponent:


math.frexp(1.2345e7)

(0.73581933975219727, 24)

Have I missed a built-in or math function somewhere?


|7> def frexp10(x):
..> exp = int(math.log10(x))
..> return x / 10**exp, exp
..>

|8> frexp10(1.2345e7)
(1.2344, 7)

--
Robert Kern

"I have come to believe that the whole world is an enigma, a harmless enigma
 that is made terrible by our own mad attempt to interpret it as though it had
 an underlying truth."
  -- Umberto Eco

--
http://mail.python.org/mailman/listinfo/python-list


Re: list parameter of a recursive function

2010-10-07 Thread Terry Reedy

On 10/6/2010 3:22 PM, TP wrote:

Hi,

I have a function f that calls itself recursively. It has a list as second
argument, with default argument equal to None (and not [], as indicated at:
http://www.ferg.org/projects/python_gotchas.html#contents_item_6 )


This sort of function is an exception. If you add the line below, so 
that you can use just one list, you should just say 'some_list = []' in 
the header.




This is the outline of my function:

def f ( argument, some_list = None ):

if some_list == None:
   some_list = []
[...]
# creation of a new_argument
# the function is called recursively only if some condition is respected
if some_condition:
   some_list.append( elem )
   f( new_argument, some_list )

 some_list.pop()

# otherwise, we have reached a leaf of the a branch of the recursive tree
# (said differently, terminal condition has been reached for this branch)
print "Terminal condition"

The problem is that when the terminal condition is reached, we return back
to some other branch of the recursive tree, but some_list has the value
obtained in the previous branch!


Uhless you pop items off the stack as you back up!

This is assuming that you do not need to keep the leaf value of the list 
as the funcions works up and back down to the next leaf, and so on. Even 
if you do, it might still be better to copy the working buffer just once 
when you hit the leaf.



So, it seems that there is only one some_list list for all the recursive
tree.
To get rid of this behavior, I have been compelled to do at the beginning of
f:

import copy from copy
some_list = copy( some_list )

I suppose this is not a surprise to you: I am compelled to create a new
some_list with the same content.
So, if I am right, all is happening as if the parameters of a function are
always passed by address to the function. Whereas in C, they are always
passed by copy (which gives relevance to pointers).


C sometimes copies values and sometimes passes addresses (pointers, 
references). It does the latter for arrays and functions. I think the 
rule has varied for structs. C and Python have quite different 
name-object-value models.


Python calls functions by binding argument objects to parameter names.
Unlike C assignment, Python name binding never copies objects.

CPython implements binding by directly associating name and object 
address. It also does not use a garbage collector that move objects 
around in memory. Other implementation do differently.


--
Terry Jan Reedy

--
http://mail.python.org/mailman/listinfo/python-list


Re: suggestions please "what should i watch for/guard against' in a file upload situation?"

2010-10-07 Thread Terry Reedy

On 10/6/2010 12:02 PM, geekbuntu wrote:

in general, what are things i would want to 'watch for/guard against'
in a file upload situation?

i have my file upload working (in the self-made framework @ work
without any concession for multipart form uploads), but was told to
make sure it's cleansed and cannot do any harm inside the system.

my checklist so far is basically to check the extension - ensure it
has 3 places, ensure it's in the allowed list (like jpg gif etc...).

not sure what else i could do to guard against anything bad
happening.  maybe the file name itself could cause greif?

not sure but any suggestions or examples are most welcome :)


I am not sure whether anyone mentioned limiting the file size, checking 
the incoming header, and aborting an upload if it goes over anyway. Most 
sites do not want 10 gigabyte files ;-).


--
Terry Jan Reedy

--
http://mail.python.org/mailman/listinfo/python-list


Re: hashkey/digest for a complex object

2010-10-07 Thread Terry Reedy

On 10/6/2010 2:58 PM, kj wrote:


These objects are non-mutable once they are created,


See below.


like to use a two-step comparison for equality, based on the
assumption that I can compute (either at creation time, or as needed
and memoized) a hashkey/digest for each object.  The test for
equality of two of these objects would first compare their hashkeys.
If they are different, the two objects are declared different; if
they match, then a more stringent test for equality is performed.


I believe Python strings do this (cache the hash). Equality comparison 
can check length, hashes, and only then chars.



So the problem is to come up with a reasonable hashkey for each of
these objects.  The objects have two significant attributes, and
two of these objects should be regarded as equal if these attributes
are "the same" in both.  The first attribute is a simple dictionary
whose keys are integers and values are strings.  The second attribute
is more complicated.  It is a tree structure, represented as a
dictionary of dictionaries of dictionaries... until we get to the
leaf elements, which are frozensets of strings.  The keys at every
level of this structure are strings.  E.g. a simple example of such
an attribute would look like:

{'A': {'a': set(['1', '2', '3']),
'b': set(['4', '5'])},
  'B': set(['6', '7', '8'])}


If these two attributes, and hence the dicts, are public, then your 
instances are mutable.


--
Terry Jan Reedy

--
http://mail.python.org/mailman/listinfo/python-list


Re: help!!!

2010-10-07 Thread Emile van Sebille

On 10/6/2010 12:08 PM Diez B. Roggisch said...

  writes:


plz can u convert this cpp file into python i need that badly as soon as 
possible... I am new to python. I just wanna learn it


For such an aspiring student of the art of computer programming, I have
the strange feeling of lack-of-effort-showing here. Do I have to lose my
faith in youth?



Oh come now -- isn't being lazy a primary programmer's attribute?

:)

Emile

--
http://mail.python.org/mailman/listinfo/python-list


Re: [Python-ideas] [Python-Dev] Inclusive Range

2010-10-07 Thread Terry Reedy

On 10/6/2010 7:14 AM, Antoon Pardon wrote:


That right-hand-half-open intervals (i.e. a<= i<  b, equivalently [a,
b) ), which are what Python uses, are to be preferred.
(See aforelinked PDF: http://www.cs.utexas.edu/users/EWD/ewd08xx/EWD831.PDF)


This specifically discusses subsequences of 'natural numbers'.


The problem is that the slice notation is sometimes handy in situations where
an open interval doesn't allow easily to mark what you want.

For instance I have at one time implemted a Tree. This is a dict like structure
but it allows to visit the keys in order. Because there is an order, slice
notation can make sense. e.g. if T is a tree with names as keys, T['bea':'mike']
is a subtree where we have for each key that 'bea'<= key<  'mike'.



But what if I wanted a subtree where 'mike' was still included, but nothing 
further?
Or what if the keys were floats or tuples and I wanted an inclusive upper 
boundary?


Strings and tuples are not natural numbers, but do have least members 
('' and ()), so the bottom end had better be closed. Since substracting 
strings and tuples in meaningless, the argument of having stop-start == 
number of items does not apply. Floats can be subtracted, but the result 
in not a count of included items. So write the .__getitem__ method of 
*your* class how it best fits your use-cases.


Just be aware that an inclusive upper boundary means that s[a:b]+s[b:c] 
will no longer be s[a:c] because b will be duplicated. Let me also point 
out integer slices for builtins are adjusted to that they refer to 
actual slice positions. This is harder to do with strings;-). Also, 
suppose you want all strings beginning with 'a' or 'b'. With an open 
upper end, Tree['a':'c'] will do that. With closed upper end, you would 
need Tree['a':'bbb'] or somesuch.



And what if you needed the reverse sequence. If you start with inclusive limit,
the reverse of a<= item<= b is b>= item>= a. If the second limit is to be
exclusive the reverse of a<= item<  b becomes (b - 1)>= item>  (a - 1).


Slices with positive strides can be defined and understood in terms of 
slice positions before the first item, between all successive pairs of 
items and after the last. By this same definition and understanding, 
s[b:a:-1] should be reversed(s[a:b]), which is what many expect. It is 
not because the definition is given in terms of the translation into 
indexes and blindly applied to the case of negative strides without the 
needed adjustment. I consider this a design mistake, at least for many 
uses. (Extended slices, if not extended slicing, were added for 
numerical Python and they may have wanted the current definition for 
negative strides.)


--
Terry Jan Reedy

--
http://mail.python.org/mailman/listinfo/python-list


Opening a webpage in the background via webbrowser.open()

2010-10-07 Thread python
Python 2.7 (32-bit/Windows): Is there a way to use
webbrowser.open() to open a web page in the default browser, but
in the background, so that the application making the
webbrowser.open() call remains the active application?

Thank you,
Malcolm
-- 
http://mail.python.org/mailman/listinfo/python-list


ConFoo spam?

2010-10-07 Thread Chris Withers

Hi All,

Is it just me or does the mailing of just about every single 
python-based project mailing list with a 90% form email advertising a 
conference that only has one python track *and* clashes with PyCon feel 
just a bit like spam?


I know it's enough to put me off even considering going to ConFoo, 
whatever it is...


Chris

--
Simplistix - Content Management, Batch Processing & Python Consulting
   - http://www.simplistix.co.uk
--
http://mail.python.org/mailman/listinfo/python-list


Re: 64 bit offsets?

2010-10-07 Thread jay thompson
As nice as it would be to use 64bit offsets I am instead mmapping the file
in 1GB chunks and getting the results I need. I would still be interested in
a 64bit solution though.

jt

On Wed, Oct 6, 2010 at 2:41 PM, jay thompson wrote:

> Hello everyone,
>
> I'm trying to extract some data from a large memory mapped file (the
> largest is ~30GB) with re.finditer() and re.start(). Pythons regular
> expression module is great but the size of re.start() is 32bits (signed so I
> can really only address 2GB).  I was wondering if any here had some
> suggestions on how to get the long offsets I need. btw... I can't break up
> the file because the pattern I'm looking for can occur anywhere and on any
> boundry.
>
> Also, is seek() limited to 32bit addresses?
>
> this is what I have in python 2.7 AMD64:
>
>
> with open(file_path, 'r+b') as file:
>
> file_map = mmap.mmap(file.fileno(), 0, access=mmap.ACCESS_READ)
> file_map.seek(0)
>
> pattern = re.compile("pattern")
>
> for iii in re.finditer(pattern, file_map):
>
> offset = iii.start()
>
> write_to_sqlite(offset)
>
>
>
>
>
> --
> "It's quite difficult to remind people that all this stuff was here for a
> million years before people. So the idea that we are required to manage it
> is ridiculous. What we are having to manage is us."   ...Bill Ballantine,
> marine biologist.
>
>


-- 
"It's quite difficult to remind people that all this stuff was here for a
million years before people. So the idea that we are required to manage it
is ridiculous. What we are having to manage is us."   ...Bill Ballantine,
marine biologist.
-- 
http://mail.python.org/mailman/listinfo/python-list


64 bit offsets?

2010-10-07 Thread jay thompson
Hello everyone,

I'm trying to extract some data from a large memory mapped file (the largest
is ~30GB) with re.finditer() and re.start(). Pythons regular expression
module is great but the size of re.start() is 32bits (signed so I can really
only address 2GB).  I was wondering if any here had some suggestions on how
to get the long offsets I need. btw... I can't break up the file because the
pattern I'm looking for can occur anywhere and on any boundry.

Also, is seek() limited to 32bit addresses?

this is what I have in python 2.7 AMD64:


with open(file_path, 'r+b') as file:

file_map = mmap.mmap(file.fileno(), 0, access=mmap.ACCESS_READ)
file_map.seek(0)

pattern = re.compile("pattern")

for iii in re.finditer(pattern, file_map):

offset = iii.start()

write_to_sqlite(offset)





-- 
"It's quite difficult to remind people that all this stuff was here for a
million years before people. So the idea that we are required to manage it
is ridiculous. What we are having to manage is us."   ...Bill Ballantine,
marine biologist.
-- 
http://mail.python.org/mailman/listinfo/python-list


Re: list parameter of a recursive function

2010-10-07 Thread TP
Seebs wrote:

> On 2010-10-07, TP  wrote:
>> Diez B. Roggisch wrote:
>>> A safer alternative for these cases is using tuples, because they are
>>> immutable.
> 
>> The problem with tuples is that it is not easy to modify them:
> 
> This is probably the best post-and-response I've seen in the last couple
> of months.

:-) Yes, they are immutable. I badly expressed the fact that the facilities 
of lists to create new lists are not available with tuples.

But my sentence was funny, I have to admit it :-) I hope that this 
conversation will not be included in the "fortunes"!!

-- 
python -c "print ''.join([chr(154 - ord(c)) for c in '*9(9&(18%.\
9&1+,\'Z4(55l4('])"

"When a distinguished but elderly scientist states that something is
possible, he is almost certainly right. When he states that something is
impossible, he is very probably wrong." (first law of AC Clarke)
-- 
http://mail.python.org/mailman/listinfo/python-list


[ANN] pywebkit - python bindings for webkit DOM (alpha)

2010-10-07 Thread Luke Kenneth Casson Leighton
i've been kindly sponsored by http://www.samurai.com.br to create
direct python bindings to webkit's DOM:
http://www.gnu.org/software/pythonwebkit/

the significance of this project is that it makes python a peer of
javascript when it comes to manipulating HTML through DOM functions
(including gaining access to the full features of HTML5 for example).
anything that can be done with javascript (such as
getElementsByTagName, appendChild etc.) can be done with python, in a
declarative programming style (not as < script language="python" >).
that means that the powerful features of HTML5 that were formerly
pretty much exclusively available to javascript programmers are now
equally available to python programmers, and webkit is known as being
especially suited to embedded environments.

anyone who has been following pywebkitgtk or pyjamas development will
be aware that the previous python webkit DOM bindings, written in
2008, were based on top of glib/gobject bindings and used
python-gobject to autogenerate them.  whilst this approach worked, its
future was made slightly awkward when the development of the gobject
bindings were kindly taken over by free software developers who have
been working hard to incorporate the full features of the original
gobject bindings into webkit for almost a year, now.  the pythonwebkit
project therefore begins again, with *direct* access to webkit DOM
functions and properties instead of going via an intermediate layer
such as gobject.

the source code for the pywebkitgtk project has been directly
incorporated into the pythonwebkit project, making it a much simpler
prospect to build.  that does not mean that it's easy - just easier!
for build instructions, please read the instructions at
http://www.gnu.org/software/pythonwebkit/ which recommend reading of
the original webkit build instructions on the original webkit
developer web site, with specific advice on the additional python
requirements.  (please note that the ongoing development focus is on
linux and embedded linux systems: windows and macosx developers are
very much "on their own" as far as build procedures are concerned, but
macosx developers are best advised to start from darwinports and to go
from there.  windows developers: good luck.  send us a postcard from
whatever loony bin you end up in, if they allow you access to
crayons).

for those people interested in pyjamas (http://pyjs.org), pyjamas
desktop has already been updated accordingly, and is already useable
with pywebkit.

please note that this is a very early announcement, with quite a lot
still left to do, but whilst there are known issues, the project is
definitely at the "working" state and so is worthy of an announcement
if anyone is interested in testing and contributing or just seeing
what the fuss is about.

l.

p.s. it's worthwhile pointing out, for anyone who is interested, that
if you wish to access a browser engine's DOM model on windows, it is
much easier to use python-COM bindings to MSHTML than it is to try
building webkit with python bindings on windows - but should you ever
succeed in building pythonwebkit on windows, please do get in touch.
if however you See The Light and realise in good time that it's a
train coming straight at you, and decide to use python COM instead, i
recommend looking at pyjd/mshtml.py and associated code in the pyjamas
project - http://pyjs.org - and beginning from there.  compiling
webkit (and xulrunner) on windows really is that bad, and
incorporating python into the mix really is likely to push your sanity
off the edge of a cliff.  if you succeed however, there are quite a
lot of people who will be extremely grateful for your sacrifice, and
who will come visit you and bring you flowers and children's colouring
picture books on a regular basis.
-- 
http://mail.python.org/mailman/listinfo/python-list


Re: mantissa and exponent in base 10

2010-10-07 Thread Jason Swails
On Wed, Oct 6, 2010 at 6:54 PM, Steven D'Aprano <
steve-remove-t...@cybersource.com.au> wrote:

> I want the mantissa and decimal exponent of a float, in base 10:
>
> mantissa and exponent of 1.2345e7
>

Perhaps not the prettiest, but you can always use string manipulations:

def frexp_10(decimal):
   parts = ("%e" % decimal).split('e')
   return float(parts[0]), int(parts[1])

You could also create your own mathematical function to do it

def frexp_10(decimal):
   import math
   logdecimal = math.log10(decimal)
   return 10 ** (logdecimal - int(logdecimal)), int(logdecimal)

Testing the timings on those 2 functions for the number 1.235323e+09 on my
machine had the math-version at about 3x faster than the string version
(admittedly it was a single conversion... who knows what the general answer
is).  The math module was loaded at the top of the program, so that didn't
add to the time of the math-execution.

Good luck!
Jason

=> (1.2345, 7)
>
> (0.12345, 8) would also be acceptable.
>
>
> The math module has a frexp() function, but it produces a base-2 exponent:
>
> >>> math.frexp(1.2345e7)
> (0.73581933975219727, 24)
>
> Have I missed a built-in or math function somewhere?
>
>
>
> --
> Steven
> --
> http://mail.python.org/mailman/listinfo/python-list
>



-- 
Jason M. Swails
Quantum Theory Project,
University of Florida
Ph.D. Graduate Student
352-392-4032
-- 
http://mail.python.org/mailman/listinfo/python-list


no .pyc

2010-10-07 Thread Tim Hanson
I feel a little silly.  I am learning Python by reading _Learning_Python_ by 
Mark Lutz.  I made it all the way to "Hello World" before my first question. 
:-)

The program isn't too complicated.
print "Hello World"
print 2**100

I saved it to a directory for which I have rw permissions that I moved to with 
cd ~/prog.  I set the execute bit.  I started the file with #!/usr/bin/python.  
I think I did everything right, and the program ran properly, both as a shell 
command (python foo.py) and as a standalone (./foo.py). 

No *.pyc.  .py is there.  What am I doing wrong?

Oh, yeah:  2.6.5, Ubuntu 10.04.
-- 
http://mail.python.org/mailman/listinfo/python-list


how to write an xml file without dom.ext?

2010-10-07 Thread hackingKK

Hello all.
I need to create an xml file.
I am using dom.minidom module.
It works fine as long as the xml tree is created.
But I get the import error for dom.ext.
I searched through the python docs but can't find a solution.
I am pritty sure that there is way to write the file to disk without 
using the ext module.
Since there are so many software doing this with python 2.6 I am sure it 
works.
So can some one tell me if there is a way to avoide ext and prittyprint 
and still write a file to the disk?


Happy hacking.
Krishnakant.




--
http://mail.python.org/mailman/listinfo/python-list


Re: no .pyc

2010-10-07 Thread News123
On 10/07/2010 11:55 AM, Tim Hanson wrote:
> I feel a little silly.  I am learning Python by reading _Learning_Python_ by 
> Mark Lutz.  I made it all the way to "Hello World" before my first question. 
> :-)
> 
> The program isn't too complicated.
> print "Hello World"
> print 2**100
> 
> I saved it to a directory for which I have rw permissions that I moved to 
> with 
> cd ~/prog.  I set the execute bit.  I started the file with 
> #!/usr/bin/python.  
> I think I did everything right, and the program ran properly, both as a shell 
> command (python foo.py) and as a standalone (./foo.py). 
> 
> No *.pyc.  .py is there.  What am I doing wrong?

Nothing's wrong.

The 'main' script (the one you called on the command line will not be
converted to .pyc)

if you had main.py and mysubmodule.py and if main.py had imported
mysubmodule.py, then you would have seen mysubmodule.pyc
> 
> Oh, yeah:  2.6.5, Ubuntu 10.04.

-- 
http://mail.python.org/mailman/listinfo/python-list


Re: how to write an xml file without dom.ext?

2010-10-07 Thread Nitin Pawar
why not just convert it to string with print pretty and then normal write to
a file


On Thu, Oct 7, 2010 at 3:36 PM, hackingKK  wrote:

> Hello all.
> I need to create an xml file.
> I am using dom.minidom module.
> It works fine as long as the xml tree is created.
> But I get the import error for dom.ext.
> I searched through the python docs but can't find a solution.
> I am pritty sure that there is way to write the file to disk without using
> the ext module.
> Since there are so many software doing this with python 2.6 I am sure it
> works.
> So can some one tell me if there is a way to avoide ext and prittyprint and
> still write a file to the disk?
>
> Happy hacking.
> Krishnakant.
>
>
>
>
> --
> http://mail.python.org/mailman/listinfo/python-list
>



-- 
Nitin Pawar
-- 
http://mail.python.org/mailman/listinfo/python-list


Re: hashkey/digest for a complex object

2010-10-07 Thread kj
In  Terry Reedy 
 writes:

>If these two attributes, and hence the dicts, are public, then your 
>instances are mutable.

I guess I should have written "immutable among consenting adults."

As far as I know, Python does not support private attributes, so
I guess the dicts are public no matter what I do.  I suppose that
I can implement "frozendict", but I can't think of any way to
enforce the immutability of these "frozendicts" that would not be
trivial to circumvent (it better be, in fact, otherwise I wouldn't
be able to initialize the damn things).

If you had something else in mind, please let me know.

~kj
-- 
http://mail.python.org/mailman/listinfo/python-list


Re: how to write an xml file without dom.ext?

2010-10-07 Thread hackingKK

On Thursday 07 October 2010 03:49 PM, Nitin Pawar wrote:
why not just convert it to string with print pretty and then normal 
write to a file




Can you give an example.

happy hacking.
Krishnakant.

On Thu, Oct 7, 2010 at 3:36 PM, hackingKK > wrote:


Hello all.
I need to create an xml file.
I am using dom.minidom module.
It works fine as long as the xml tree is created.
But I get the import error for dom.ext.
I searched through the python docs but can't find a solution.
I am pritty sure that there is way to write the file to disk
without using the ext module.
Since there are so many software doing this with python 2.6 I am
sure it works.
So can some one tell me if there is a way to avoide ext and
prittyprint and still write a file to the disk?

Happy hacking.
Krishnakant.




-- 
http://mail.python.org/mailman/listinfo/python-list





--
Nitin Pawar



-- 
http://mail.python.org/mailman/listinfo/python-list


Re: hashkey/digest for a complex object

2010-10-07 Thread kj
In  de...@web.de (Diez B. Roggisch) writes:

>kj  writes:

>> The short version of this question is: where can I find the algorithm
>> used by the tuple class's __hash__ method?

>Surprisingly, in the source:

>http://google.com/codesearch/p?hl=de#-2BKs-LW4I0/trunk/python/src/Objects/tupleobject.c&q=python%20tuplehash&sa=N&cd=1&ct=rc

Thanks to you, and to all who pointed me to the place in the source
where this is.

How exactly did you search for this?  Taking a hint from the url
above, I went to Google Code Search and searched for "python tuple
hash" (minus the quotes, of course), which produced a ton of
irrelevant stuff (almost 80K hits).  Searching for "python tuple
hash lang:c" cut down the number of hits to ~8K, but still too much
to wade through.  Clearly I need a smarter search strategy (one
that does not include foreknowledge of the name of the actual
function in the C source, of course).

~kj
-- 
http://mail.python.org/mailman/listinfo/python-list


Re: how to write an xml file without dom.ext?

2010-10-07 Thread Nitin Pawar
import xml.dom.minidom
import os
xml = xml.dom.minidom.parse(xml_fname) # or
xml.dom.minidom.parseString(xml_string)
pretty_xml_as_string = xml.toprettyxml()
file = open("newfile", 'w')
file.write(pretty_xml_as_string)
file.close()


   1.





On Thu, Oct 7, 2010 at 4:16 PM, hackingKK  wrote:

>  On Thursday 07 October 2010 03:49 PM, Nitin Pawar wrote:
>
> why not just convert it to string with print pretty and then normal write
> to a file
>
>
>  Can you give an example.
>
> happy hacking.
> Krishnakant.
>
>  On Thu, Oct 7, 2010 at 3:36 PM, hackingKK  wrote:
>
>> Hello all.
>> I need to create an xml file.
>> I am using dom.minidom module.
>> It works fine as long as the xml tree is created.
>> But I get the import error for dom.ext.
>> I searched through the python docs but can't find a solution.
>> I am pritty sure that there is way to write the file to disk without using
>> the ext module.
>> Since there are so many software doing this with python 2.6 I am sure it
>> works.
>> So can some one tell me if there is a way to avoide ext and prittyprint
>> and still write a file to the disk?
>>
>> Happy hacking.
>> Krishnakant.
>>
>>
>>
>>
>> --
>> http://mail.python.org/mailman/listinfo/python-list
>>
>
>
>
> --
> Nitin Pawar
>
>
>


-- 
Nitin Pawar
-- 
http://mail.python.org/mailman/listinfo/python-list


Re: if the else short form

2010-10-07 Thread BartC

On Thu, 07 Oct 2010 01:36:33 +0100, BartC wrote:



However,  as I mentioned, one problem here is having to evaluate all the
items in the list before selecting one:

...

x = {1 : fna(), 2 : fnb(), 3 : fnc()}.get(i, "None Of The Above")


"Mel"  wrote in message
news:i8j56e$ub...@speranza.aioe.org...

x = {1: fna, 2: fnb, 3: fnc}.get(i, lambda: "None Of The Above") ()


"Steven D'Aprano"  wrote in message
news:4cad1817$0$29996$c3e8da3$54964...@news.astraweb.com...

x = {1 : fna, 2 : fnb, 3 : fnc}.get(i, "None Of The Above")()


Well, these both kind of work (although the lambda thing is needed in the
second version, when i is out of range).

But you're both taking my example too literally; it was just my way of
testing whether or not each item was in fact evaluated.

In general, each item can be any arbitrary expression, which may or may not
include function calls.

To make this work as expected, I think 'lambda' is needed in front of every
expression, with the expressions written normally (so fn() instead of fn).

But with the syntax becoming more cumbersome, unintuitive, and with unknown
overheads to set up the lambda expressions (is it still necessary to
construct all dictionary entries?), it might be better to just use ordinary
conditional statements.

--
Bartc 


--
http://mail.python.org/mailman/listinfo/python-list


Re: mantissa and exponent in base 10

2010-10-07 Thread Dave Angel

 On 2:59 PM, Steven D'Aprano wrote:

I want the mantissa and decimal exponent of a float, in base 10:

mantissa and exponent of 1.2345e7
=>  (1.2345, 7)

(0.12345, 8) would also be acceptable.


The math module has a frexp() function, but it produces a base-2 exponent:


math.frexp(1.2345e7)

(0.73581933975219727, 24)

Have I missed a built-in or math function somewhere?


First point is that this is a conversion, since the float is stored 
internally using a binary exponent.  So anything you do with logs might 
just give roundoff errors, one way or another.  I'd therefore suggest 
converting the number explicitly, using whatever approach is appropriate 
to the further processing of the number.


If you want the version that will display, then convert it to a string, 
and parse that.


If you want to do further processing, use the decimal module to convert 
it.  Once it's converted, there's an as_tuple() method that will give 
you an exact representation of the original value.  That may not be the 
same mantissa as you'd get with the original, and for numbers like 
9.  or whatever, this could be significant.


DaveA
--
http://mail.python.org/mailman/listinfo/python-list


Re: SysLogHandler message formatting

2010-10-07 Thread Vinay Sajip
On Oct 6, 6:56 pm, "Dustin C. Hatch"  wrote:

> 
> My question, therefore, is where does this problem lie? Is it a bug in
> Metalog that it doesn't properly parse the message, or is it a bug in
> SysLogHandler that it doesn't properly format it? I guess it could
> also be that if one wants to use the SysLogHandler, then that
> particular format string must be used to ensure compatibility with all
> syslog daemons. I found a couple of examples[2][3] of people

Hi Dustin,

Thanks for the detailed report. I tried posting a response a couple of times,
but Google appears to have swallowed it ... trying again. Sorry if it results in
multiple responses.

The SysLogHandler aims to work within RFC 5424, though that does provide for
some flexibility in message formats. The SysLogHandler sends formatted
message, as you've observed, with the "formatted message" part encoded in UTF-8
with a prepended BOM if it's Unicode.

The existing SysLogHandler code seems to work OK with syslog, syslog-ng and
rsyslog - perhaps these are more forgiving than Metalog in the way they parse
messages, or perhaps it's a Metalog configuration issue. I'd try posting this
issue to the Metalog forum / mailing list to see what feedback they can give. If
you do think it's a bug in SysLogHandler then please create an issue on
bugs.python.org and assign it to me, so I can take a look at it; however, I
think using an appropriate Formatter will ensure interoperability in any
particular scenario. I don't especially want to hard-code any format into
SysLogHandler, since a Formatter can be used to set the format flexibly.

Regards,

Vinay Sajip

-- 
http://mail.python.org/mailman/listinfo/python-list


Re: how to write an xml file without dom.ext?

2010-10-07 Thread Diez B. Roggisch
hackingKK  writes:

> Hello all.
> I need to create an xml file.
> I am using dom.minidom module.
> It works fine as long as the xml tree is created.
> But I get the import error for dom.ext.
> I searched through the python docs but can't find a solution.
> I am pritty sure that there is way to write the file to disk without
> using the ext module.
> Since there are so many software doing this with python 2.6 I am sure
> it works.
> So can some one tell me if there is a way to avoide ext and
> prittyprint and still write a file to the disk?

By not using minidom at all.

from xml.etree import ElementTree as et

doc = et.fromstring("")
print et.tostring(doc)

Diez
-- 
http://mail.python.org/mailman/listinfo/python-list


Re: ConFoo spam?

2010-10-07 Thread Diez B. Roggisch
Chris Withers  writes:

> Hi All,
>
> Is it just me or does the mailing of just about every single
> python-based project mailing list with a 90% form email advertising a
> conference that only has one python track *and* clashes with PyCon
> feel just a bit like spam?
>
> I know it's enough to put me off even considering going to ConFoo,
> whatever it is...

The post appeared on TurboGears lists as well - but I actually consider
it on-topic. We get enough other conference calls here, sometimes even
without an actual python-track, but instead catering to specific
scientific communities that might or might not use python. 

I don't consider this spam. It is certainly less annoying than the
umpteenth GIL discussion...

Diez
-- 
http://mail.python.org/mailman/listinfo/python-list


Re: hashkey/digest for a complex object

2010-10-07 Thread Diez B. Roggisch
kj  writes:

> In  de...@web.de (Diez B. Roggisch) writes:
>
>>kj  writes:
>
>>> The short version of this question is: where can I find the algorithm
>>> used by the tuple class's __hash__ method?
>
>>Surprisingly, in the source:
>
>>http://google.com/codesearch/p?hl=de#-2BKs-LW4I0/trunk/python/src/Objects/tupleobject.c&q=python%20tuplehash&sa=N&cd=1&ct=rc
>
> Thanks to you, and to all who pointed me to the place in the source
> where this is.
>
> How exactly did you search for this?  Taking a hint from the url
> above, I went to Google Code Search and searched for "python tuple
> hash" (minus the quotes, of course), which produced a ton of
> irrelevant stuff (almost 80K hits).  Searching for "python tuple
> hash lang:c" cut down the number of hits to ~8K, but still too much
> to wade through.  Clearly I need a smarter search strategy (one
> that does not include foreknowledge of the name of the actual
> function in the C source, of course).

I tried codesearch first. Not satisfied after 30 seconds with the
results, I did the most straight forward thing - I downloaded and
un-packed the python source... and took a look. 

>From that I learned the tuplehash function name.

Diez
-- 
http://mail.python.org/mailman/listinfo/python-list


[ANN] pywebkit - python bindings for webkit DOM (alpha)

2010-10-07 Thread lkcl
From: l...@lkcl.net
To: python-list@python.org

i've been kindly sponsored by http://www.samurai.com.br to create
direct python bindings to webkit's DOM:
http://www.gnu.org/software/pythonwebkit/

the significance of this project is that it makes python a peer of
javascript when it comes to manipulating HTML through DOM functions
(including gaining access to the full features of HTML5 for example).
anything that can be done with javascript (such as
getElementsByTagName, appendChild etc.) can be done with python, in a
declarative programming style (not as < script language="python" >).
that means that the powerful features of HTML5 that were formerly
pretty much exclusively available to javascript programmers are now
equally available to python programmers, and webkit is known as being
especially suited to embedded environments.

anyone who has been following pywebkitgtk or pyjamas development will
be aware that the previous python webkit DOM bindings, written in
2008, were based on top of glib/gobject bindings and used
python-gobject to autogenerate them.  whilst this approach worked, its
future was made slightly awkward when the development of the gobject
bindings were kindly taken over by free software developers who have
been working hard to incorporate the full features of the original
gobject bindings into webkit for almost a year, now.  the pythonwebkit
project therefore begins again, with *direct* access to webkit DOM
functions and properties instead of going via an intermediate layer
such as gobject.

the source code for the pywebkitgtk project has been directly
incorporated into the pythonwebkit project, making it a much simpler
prospect to build.  that does not mean that it's easy - just easier!
for build instructions, please read the instructions at
http://www.gnu.org/software/pythonwebkit/ which recommend reading of
the original webkit build instructions on the original webkit
developer web site, with specific advice on the additional python
requirements.  (please note that the ongoing development focus is on
linux and embedded linux systems: windows and macosx developers are
very much "on their own" as far as build procedures are concerned, but
macosx developers are best advised to start from darwinports and to go
from there.  windows developers: good luck.  send us a postcard from
whatever loony bin you end up in, if they allow you access to
crayons).

for those people interested in pyjamas (http://pyjs.org), pyjamas
desktop has already been updated accordingly, and is already useable
with pywebkit.

please note that this is a very early announcement, with quite a lot
still left to do, but whilst there are known issues, the project is
definitely at the "working" state and so is worthy of an announcement
if anyone is interested in testing and contributing or just seeing
what the fuss is about.

l.

p.s. it's worthwhile pointing out, for anyone who is interested, that
if you wish to access a browser engine's DOM model on windows, it is
much easier to use python-COM bindings to MSHTML than it is to try
building webkit with python bindings on windows - but should you ever
succeed in building pythonwebkit on windows, please do get in touch.
if however you See The Light and realise in good time that it's a
train coming straight at you, and decide to use python COM instead, i
recommend looking at pyjd/mshtml.py and associated code in the pyjamas
project - http://pyjs.org - and beginning from there.  compiling
webkit (and xulrunner) on windows really is that bad, and
incorporating python into the mix really is likely to push your sanity
off the edge of a cliff.  if you succeed however, there are quite a
lot of people who will be extremely grateful for your sacrifice, and
who will come visit you and bring you flowers and children's colouring
picture books on a regular basis.
-- 
http://mail.python.org/mailman/listinfo/python-list


[ANN] pywebkit - python bindings for webkit DOM (alpha)

2010-10-07 Thread lkcl
i've been kindly sponsored by http://www.samurai.com.br to create
direct python bindings to webkit's DOM:
http://www.gnu.org/software/pythonwebkit/

the significance of this project is that it makes python a peer of
javascript when it comes to manipulating HTML through DOM functions
(including gaining access to the full features of HTML5 for example).
anything that can be done with javascript (such as
getElementsByTagName, appendChild etc.) can be done with python, in a
declarative programming style (not as < script language="python" >).
that means that the powerful features of HTML5 that were formerly
pretty much exclusively available to javascript programmers are now
equally available to python programmers, and webkit is known as being
especially suited to embedded environments.

anyone who has been following pywebkitgtk or pyjamas development will
be aware that the previous python webkit DOM bindings, written in
2008, were based on top of glib/gobject bindings and used
python-gobject to autogenerate them.  whilst this approach worked, its
future was made slightly awkward when the development of the gobject
bindings were kindly taken over by free software developers who have
been working hard to incorporate the full features of the original
gobject bindings into webkit for almost a year, now.  the pythonwebkit
project therefore begins again, with *direct* access to webkit DOM
functions and properties instead of going via an intermediate layer
such as gobject.

the source code for the pywebkitgtk project has been directly
incorporated into the pythonwebkit project, making it a much simpler
prospect to build.  that does not mean that it's easy - just easier!
for build instructions, please read the instructions at
http://www.gnu.org/software/pythonwebkit/ which recommend reading of
the original webkit build instructions on the original webkit
developer web site, with specific advice on the additional python
requirements.  (please note that the ongoing development focus is on
linux and embedded linux systems: windows and macosx developers are
very much "on their own" as far as build procedures are concerned, but
macosx developers are best advised to start from darwinports and to go
from there.  windows developers: good luck.  send us a postcard from
whatever loony bin you end up in, if they allow you access to
crayons).

for those people interested in pyjamas (http://pyjs.org), pyjamas
desktop has already been updated accordingly, and is already useable
with pywebkit.

please note that this is a very early announcement, with quite a lot
still left to do, but whilst there are known issues, the project is
definitely at the "working" state and so is worthy of an announcement
if anyone is interested in testing and contributing or just seeing
what the fuss is about.

l.

p.s. it's worthwhile pointing out, for anyone who is interested, that
if you wish to access a browser engine's DOM model on windows, it is
much easier to use python-COM bindings to MSHTML than it is to try
building webkit with python bindings on windows - but should you ever
succeed in building pythonwebkit on windows, please do get in touch.
if however you See The Light and realise in good time that it's a
train coming straight at you, and decide to use python COM instead, i
recommend looking at pyjd/mshtml.py and associated code in the pyjamas
project - http://pyjs.org - and beginning from there.  compiling
webkit (and xulrunner) on windows really is that bad, and
incorporating python into the mix really is likely to push your sanity
off the edge of a cliff.  if you succeed however, there are quite a
lot of people who will be extremely grateful for your sacrifice, and
who will come visit you and bring you flowers and children's colouring
picture books on a regular basis.
-- 
http://mail.python.org/mailman/listinfo/python-list


Re: hashkey/digest for a complex object

2010-10-07 Thread kj
In <87pqvmp611@web.de> de...@web.de (Diez B. Roggisch) writes:

>I tried codesearch first. Not satisfied after 30 seconds with the
>results, I did the most straight forward thing - I downloaded and
>un-packed the python source... and took a look. 

>From that I learned the tuplehash function name.

You must be at least somewhat familiar with the Python source.
Everytime I've peeked into it I just feel lost, but it's clearly
something I need to master sooner or later...  I can't wait for
the next one of those occasional introductions to the Python
internals at our local Python club.

Thanks,

~kj
-- 
http://mail.python.org/mailman/listinfo/python-list


Re: hashkey/digest for a complex object

2010-10-07 Thread Diez B. Roggisch
kj  writes:

> In <87pqvmp611@web.de> de...@web.de (Diez B. Roggisch) writes:
>
>>I tried codesearch first. Not satisfied after 30 seconds with the
>>results, I did the most straight forward thing - I downloaded and
>>un-packed the python source... and took a look. 
>
>>From that I learned the tuplehash function name.
>
> You must be at least somewhat familiar with the Python source.
> Everytime I've peeked into it I just feel lost, but it's clearly
> something I need to master sooner or later...  I can't wait for
> the next one of those occasional introductions to the Python
> internals at our local Python club.

No, I'm not the tiniest bit. I just followed my instincts in looking
into the "Objects" folder, because that's where I suspected the
definition of objects to be

And grep has been proven useful in these cases as well.

Diez
-- 
http://mail.python.org/mailman/listinfo/python-list


Re: hashkey/digest for a complex object

2010-10-07 Thread MRAB

On 07/10/2010 11:42, kj wrote:

In  Terry 
Reedy  writes:


If these two attributes, and hence the dicts, are public, then your
instances are mutable.


I guess I should have written "immutable among consenting adults."

As far as I know, Python does not support private attributes, so
I guess the dicts are public no matter what I do.  I suppose that
I can implement "frozendict", but I can't think of any way to
enforce the immutability of these "frozendicts" that would not be
trivial to circumvent (it better be, in fact, otherwise I wouldn't
be able to initialize the damn things).


You would initialise them by creating them from a list of tuples (or an
iterable which yields tuples), like with a dict:

>>> dict([(1, "foo"), (2, "bar")])
{1: 'foo', 2: 'bar'}
--
http://mail.python.org/mailman/listinfo/python-list


Data source path of a lyer file

2010-10-07 Thread Ahmed, Shakir
Hi,

Is there any way that I can read the data source path of a .lyr file
using Arc Object gp processor.

 

Thanks

sa

 

 

 

-- 
http://mail.python.org/mailman/listinfo/python-list


Re: ConFoo spam?

2010-10-07 Thread Yannick Gingras
On October 7, 2010, Chris Withers wrote:
> Hi All,
> 
> Is it just me or does the mailing of just about every single 
> python-based project mailing list with a 90% form email advertising a 
> conference that only has one python track and clashes with PyCon feel 
> just a bit like spam?
> 
> I know it's enough to put me off even considering going to ConFoo, 
> whatever it is...

[I'm sending this message again because the first time, I was not
subscribed to the list and my reply seems to be withheld.  I hope that
the list moderator will see this and that you won't receive it twice.]

Hello Chris and others, 
  I'm sorry if our call for speakers was too broad.  A few weeks ago,
PyCon launched its call for speakers with a very massive coverage on
great many mailing lists, including the Montréal-Python one.  We
understand that PyCon it THE Python conference and we actually
appreciated that our low volume mailing list was considered important
enough to receive the PyCon call for speakers.

Inspired by that massive out-reach effort by the PyCon program
committee, we decided to launch the ConFoo CFP with great glory too.
A total of 24 lists were contacted, most of them related to Web
development and a few where Python core hackers can be found on
because Raymond Hettinger's talk last year was a great success.

Should we only have contacted the Web related lists?  Should we
refrain from contacting any lists?  I admit that we don't know what
the right answer is but we will take the feedback from the community
and adjust our strategy for next year.

Regarding the clash with PyCon, we are also very torn.  ConFoo does
not have a lot of leverage to negotiate with the hotel so we had to
book a date as early as possible to get a decent rate and we did it
with PyCon 2010 dates in mind.  When the news came out that PyCon
would clash with ConFoo in a very significant manner, we,
Montréal-Python, had the option to cancel the Python track or to look
for a compromise, which was to ensure that all the Python talks would
happen before the first day of PyCon talks.  We'll do better next year
but this is the best that we could do this year.

By the way, ConFoo and Montréal-Python are non-profits, we do it
because we want to show how Python is a viable Web development
plate-form; we won't make money out of it.

Again, I'm sorry if our call for speakers was too broad and we'll do
better next year.

Best regards, 

-- 
Yannick Gingras
http://montrealpython.org -- lead organizer


signature.asc
Description: This is a digitally signed message part.
-- 
http://mail.python.org/mailman/listinfo/python-list


Re: 64 bit offsets?

2010-10-07 Thread MRAB

On 06/10/2010 22:41, jay thompson wrote:

Hello everyone,

I'm trying to extract some data from a large memory mapped file (the
largest is ~30GB) with re.finditer() and re.start(). Pythons regular
expression module is great but the size of re.start() is 32bits (signed
so I can really only address 2GB).  I was wondering if any here had some
suggestions on how to get the long offsets I need. btw... I can't break
up the file because the pattern I'm looking for can occur anywhere and
on any boundry.

Also, is seek() limited to 32bit addresses?

this is what I have in python 2.7 AMD64:


with open(file_path, 'r+b') as file:

 file_map = mmap.mmap(file.fileno(), 0, access=mmap.ACCESS_READ)
 file_map.seek(0)

 pattern = re.compile("pattern")

 for iii in re.finditer(pattern, file_map):

 offset = iii.start()

 write_to_sqlite(offset)


I would've thought that a 64-bit version of Python would have 64-bit
offsets. Is that not the case?
--
http://mail.python.org/mailman/listinfo/python-list


Re: ConFoo spam?

2010-10-07 Thread Yannick Gingras
On October 7, 2010, Chris Withers wrote:
> Hi All,
> 
> Is it just me or does the mailing of just about every single 
> python-based project mailing list with a 90% form email advertising a 
> conference that only has one python track and clashes with PyCon feel 
> just a bit like spam?
> 
> I know it's enough to put me off even considering going to ConFoo, 
> whatever it is...

Hello Chris and others, 
  I'm sorry if our call for speakers was too broad.  A few weeks ago,
PyCon launched its call for speakers with a very massive coverage on
great many mailing lists, including the Montréal-Python one.  We
understand that PyCon it THE Python conference and we actually
appreciated that our low volume mailing list was considered important
enough to receive the PyCon call for speakers.

Inspired by that massive out-reach effort by the PyCon program
committee, we decided to launch the ConFoo CFP with great glory too.
A total of 24 lists were contacted, most of them related to Web
development and a few where Python core hackers can be found on
because Raymond Hettinger's talk last year was a great success.

Should we only have contacted the Web related lists?  Should we
refrain from contacting any lists?  I admit that we don't know what
the right answer is but we will take the feedback from the community
and adjust our strategy for next year.

Regarding the clash with PyCon, we are also very torn.  ConFoo does
not have a lot of leverage to negotiate with the hotel so we had to
book a date as early as possible to get a decent rate and we did it
with PyCon 2010 dates in mind.  When the news came out that PyCon
would clash with ConFoo in a very significant manner, we,
Montréal-Python, had the option to cancel the Python track or to look
for a compromise, which was to ensure that all the Python talks would
happen before the first day of PyCon talks.  We'll do better next year
but this is the best that we could do this year.

By the way, ConFoo and Montréal-Python are non-profits, we do it
because we want to show how Python is a viable Web development
plate-form; we won't make money out of it.

Again, I'm sorry if our call for speakers was too broad and we'll do
better next year.

Best regards, 

-- 
Yannick Gingras
http://montrealpython.org -- lead organizer


signature.asc
Description: This is a digitally signed message part.
-- 
http://mail.python.org/mailman/listinfo/python-list


Python Programmer needed in the LA area

2010-10-07 Thread Capstick, Antony H
I am looking for an experienced Person or Company located in Southern 
California to develop and implement a graphical track map operating in MS 
Windows that will be used to display graphical information for a new Disney 
attraction.

Implementer Qualifications
General requirements:

* Experience developing graphical user interfaces for Windows (or 
cross-platform tool sets which support Windows)

* Ability to develop custom "widgets" for graphical user interfaces - 
in addition to the standard widgets (text boxes, list boxes, etc), custom 
widgets must be used by the track map application.

* Experience with simple animation of widgets on-screen, including the 
ability to translate and rotate bitmaps with masks to any position within a 
window

* Experience with developing network-enabled applications specifically 
using UDP and custom protocols.

* Experience using Excel.

If WDI existing environment is used, additional requirements include:

* Experience using Python as a primary development language on the 
Windows platform

* Ability to understand partially documented Python code
Experience using the wxWidgets cross-platform graphical user interface 
development library, and preferably the wxPython bindings for Python.

Please contact Antony Capstick at
 818-544-3230


-- 
http://mail.python.org/mailman/listinfo/python-list


Re: mantissa and exponent in base 10

2010-10-07 Thread C or L Smith
I just sent a similar suggestion to tutor: check out the %g format.

>>> print '%g' % 1.2345e7
1.2345e+07
>>> print '%g' % 1.2345e-7
1.2345e-07
>>> print '%g' % 1.2345
1.2345

>>> def me(n, sigfigs = 4):
...  s = ('%.'+'%ig' % sigfigs) % n # check docs for a better way?
...  if 'e' in s: m, e = s.split('e')
...  else: m, e = s, 0
...  m, e = float(m), float(e)
...  if m >= 10:
...   m /= 100
...   e += 2
...  elif m >= 1:
...   m /= 10
...   e += 1
...  return m, e
... 
>>> me(9.999)
(0.10001, 2.0)
>>> me(12345)
(0.12351, 5.0)
>>> me(1.2345e-9)
(0.12351, -8.0)
>>> me(1.2345e-9, 2)
(0.12, -8.0)

Best regards,
 Chris Smith
-- 
http://mail.python.org/mailman/listinfo/python-list


Re: Eclipse/PyDev - BOM Lexical Error

2010-10-07 Thread Diez B. Roggisch
Lawrence D'Oliveiro  writes:

> In message <87d3rorf2f@web.de>, Diez B. Roggisch wrote:
>
>> Lawrence D'Oliveiro  writes:
>> 
>>> What exactly is the point of a BOM in a UTF-8-encoded file?
>> 
>> It's a marker like the "coding: utf-8" in python-files. It tells the
>> software aware of it that the content is UTF-8.
>
> But if the software is aware of it, then why does it need to be told?

Let me rephrase: windows editors such as notepad recognize the BOM, and
then assume (hopefully rightfully so) that the rest of the file is text
in utf-8 encoding.

So it is similar to the coding-header in Python.

>
>> Naming it "BOM" is obviously stupid, but that's the way it is called.
>
> It is in fact a Unicode BOM character, and I can understand why it’s called 
> that. What I’m trying to understand is why you need to put one in a UTF-8-
> encoded file.

I hope that's clear now. It says "I'm a UTF-8 file".

Diez
-- 
http://mail.python.org/mailman/listinfo/python-list


Re: ConFoo spam?

2010-10-07 Thread Jack Diederich
On Thu, Oct 7, 2010 at 7:56 AM, Diez B. Roggisch  wrote:
> Chris Withers  writes:
>>
>> Is it just me or does the mailing of just about every single
>> python-based project mailing list with a 90% form email advertising a
>> conference that only has one python track *and* clashes with PyCon
>> feel just a bit like spam?
>
> The post appeared on TurboGears lists as well - but I actually consider
> it on-topic. We get enough other conference calls here, sometimes even
> without an actual python-track, but instead catering to specific
> scientific communities that might or might not use python.
>
> I don't consider this spam. It is certainly less annoying than the
> umpteenth GIL discussion...

I don't consider it spam either and I subscribe to half the lists they
sent it to.  People announce and talk about things of no interest to
me every day on python-list.  You get used to it. [For the record
ConFoo /does/ interest me, but I can't take a week and a half off to
do that plus PyCon].

-Jack
-- 
http://mail.python.org/mailman/listinfo/python-list


Re: Python Programmer needed in the LA area

2010-10-07 Thread Jack Diederich
On Thu, Oct 7, 2010 at 11:55 AM, Capstick, Antony H
 wrote:
> I am looking for an experienced Person .. to develop and implement a 
> graphical track map operating in MS
> Windows that will be used to display graphical information for a new Disney
> attraction.

1) http://www.python.org/community/jobs/ is the appropriate place for this.

2) Comic sans?  Wow.

-Jack
-- 
http://mail.python.org/mailman/listinfo/python-list


Ordering tests in a testsuite

2010-10-07 Thread Ulrich Eckhardt
Hello!

I'm currently working on a testsuite using Python's unittest library. This
works all good and fine, but there's one thing where I haven't seen an
elegant solution to yet, and that is the ordering. Currently, it takes all
classes and orders them alphabetically and then takes all test functions
therein and runs those alphabetically, too. However, sometimes it doesn't
make sense to run test_bar() if test_foo() already failed, because they
basically build upon each other. However, test_bar() is still run first,
and test_foo() second.

What I sometimes do is to simply number them, like test_1_foo() and
test_2_bar(), but that seems ugly. I'm not even looking for a way to
express complicated relations between tests, but is there a less ugly way
to explicitly order them?

Cheers!

Uli

-- 
Sator Laser GmbH
Geschäftsführer: Thorsten Föcking, Amtsgericht Hamburg HR B62 932

-- 
http://mail.python.org/mailman/listinfo/python-list


Re: mantissa and exponent in base 10

2010-10-07 Thread Jason Swails
On Thu, Oct 7, 2010 at 11:51 AM, C or L Smith  wrote:

> I just sent a similar suggestion to tutor: check out the %g format.
>
> >>> print '%g' % 1.2345e7
> 1.2345e+07
> >>> print '%g' % 1.2345e-7
> 1.2345e-07
> >>> print '%g' % 1.2345
> 1.2345
>
> >>> def me(n, sigfigs = 4):
> ...  s = ('%.'+'%ig' % sigfigs) % n # check docs for a better way?
>

s = ('%%%ig' % sigfigs) % n # double-% cancels the %

...  if 'e' in s: m, e = s.split('e')
> ...  else: m, e = s, 0
> ...  m, e = float(m), float(e)
> ...  if m >= 10:
> ...   m /= 100
> ...   e += 2
> ...  elif m >= 1:
> ...   m /= 10
> ...   e += 1
> ...  return m, e
> ...
> >>> me(9.999)
> (0.10001, 2.0)
> >>> me(12345)
> (0.12351, 5.0)
> >>> me(1.2345e-9)
> (0.12351, -8.0)
> >>> me(1.2345e-9, 2)
> (0.12, -8.0)
>
> Best regards,
>  Chris Smith
> --
> http://mail.python.org/mailman/listinfo/python-list
>



-- 
Jason M. Swails
Quantum Theory Project,
University of Florida
Ph.D. Graduate Student
352-392-4032
-- 
http://mail.python.org/mailman/listinfo/python-list


Re: Ordering tests in a testsuite

2010-10-07 Thread Peter Otten
Ulrich Eckhardt wrote:

> I'm currently working on a testsuite using Python's unittest library. This
> works all good and fine, but there's one thing where I haven't seen an
> elegant solution to yet, and that is the ordering. Currently, it takes all
> classes and orders them alphabetically and then takes all test functions
> therein and runs those alphabetically, too. However, sometimes it doesn't
> make sense to run test_bar() if test_foo() already failed, because they
> basically build upon each other. However, test_bar() is still run first,
> and test_foo() second.
> 
> What I sometimes do is to simply number them, like test_1_foo() and
> test_2_bar(), but that seems ugly. I'm not even looking for a way to
> express complicated relations between tests, but is there a less ugly way
> to explicitly order them?

You can pass unittest.main() a custom test loader which in turn can define a 
custom sortTestMethodsUsing() method. Here's a fancy example:

import unittest

def _toposort(data):
# adapted from http://code.activestate.com/recipes/577413-topological-
sort/
for k, v in data.items():
v.discard(k) # Ignore self dependencies
extra_items_in_deps = reduce(set.union, data.values()) - 
set(data.keys())
data.update((item, set()) for item in extra_items_in_deps)
while True:
ordered = set(item for item,dep in data.items() if not dep)
if not ordered:
break
for item in sorted(ordered):
yield item
data = dict((item, (dep - ordered)) for item,dep in data.items()
if item not in ordered)

def _get_name(func):
try:
return func.__name__
except AttributeError:
pass
assert isinstance(func, str)
return func

class Cmp(object):
def __init__(self):
self._deps = {}
def depends(self, f, names):
k = _get_name(f)
assert k not in self._deps
self._deps[k] =  set(_get_name(n) for n in names)
return f
def make_loader(self):
lookup = dict((name, index) for index, name
  in enumerate(_toposort(self._deps)))
def compare(a, b):
return cmp(lookup[a], lookup[b])
class TestLoader(unittest.TestLoader):
pass
loader = TestLoader()
loader.sortTestMethodsUsing = compare
return loader
def depends_on(self, *names):
def d(f):
return self.depends(f, names)
return d


c = Cmp()
depends_on = c.depends_on

class A(unittest.TestCase):
@depends_on("test_beta")
def test_alpha(self):
pass
@depends_on("test_gamma")
def test_beta(self):
pass
def test_gamma(self):
pass
@depends_on(test_gamma)
def test_delta(self):
pass
@depends_on(test_alpha, test_beta)
def test_epsilon(self):
pass

if __name__ == "__main__":
unittest.main(testLoader=c.make_loader())

Peter

-- 
http://mail.python.org/mailman/listinfo/python-list


Re: 64 bit offsets?

2010-10-07 Thread jay thompson
I'm not sure if it is limited to 32 bit addresses or if it's only re.start()
that is limited to 'em.

jt

-

"It's quite difficult to remind people that all this stuff was here for a
million years before people. So the idea that we are required to manage it
is ridiculous. What we are having to manage is us." ...Bill Ballantine,
marine biologist.

On 2010-10-07 8:42 AM, "MRAB"  wrote:

On 06/10/2010 22:41, jay thompson wrote:
>
> Hello everyone,
>
> I'm trying to extract some data fro...
I would've thought that a 64-bit version of Python would have 64-bit
offsets. Is that not the case?
-- 
http://mail.python.org/mailman/listinfo/python-list
-- 
http://mail.python.org/mailman/listinfo/python-list


how to test for atomicity/mutability/hashability?

2010-10-07 Thread kj




I want to implement a test t() that will return True if its two
arguments are "completely" different.  By this I mean that they
don't share any "non-atomic" component.  E.g., if

a = [0, 1]
b = [0, 1]
c = [2, 3]
d = [2, 3]

A = (a, c, 0)
B = (a, d, 1)
C = (b, d, 0)

The desired test t() would yield:

t(A, B) -> False (A and B share the mutable component a)
t(A, C) -> True (a =!= c, b =!= d, and 0 is not mutable)
t(B, C) -> False (B and C share the mutable component d)

(=!= is shorthand with "is not identical to".)

It would facilitate the implementation of t() to have a simple test
for mutability.  Is there one?

Thanks!

~kj
-- 
http://mail.python.org/mailman/listinfo/python-list


Re: how to test for atomicity/mutability/hashability?

2010-10-07 Thread Chris Rebert
On Thu, Oct 7, 2010 at 12:13 PM, kj  wrote:

> It would facilitate the implementation of t() to have a simple test
> for mutability.  Is there one?

Non-default hashability is an approximate heuristic:

def is_immutable(x):
try:
hash(x)
except TypeError:
return False
else:
klass = type(x)
return klass is object or klass.__hash__ is not object.__hash__

Cheers,
Chris
--
http://blog.rebertia.com
-- 
http://mail.python.org/mailman/listinfo/python-list


Re: hashkey/digest for a complex object

2010-10-07 Thread Arnaud Delobelle
kj  writes:

> The short version of this question is: where can I find the algorithm
> used by the tuple class's __hash__ method?
>
> Now, for the long version of this question, I'm working with some
> complext Python objects that I want to be able to compare for
> equality easily.
>
> These objects are non-mutable once they are created, so I would
> like to use a two-step comparison for equality, based on the
> assumption that I can compute (either at creation time, or as needed
> and memoized) a hashkey/digest for each object.  The test for
> equality of two of these objects would first compare their hashkeys.
> If they are different, the two objects are declared different; if
> they match, then a more stringent test for equality is performed.
>
> So the problem is to come up with a reasonable hashkey for each of
> these objects.  The objects have two significant attributes, and
> two of these objects should be regarded as equal if these attributes
> are "the same" in both.  The first attribute is a simple dictionary
> whose keys are integers and values are strings.  The second attribute
> is more complicated.  It is a tree structure, represented as a
> dictionary of dictionaries of dictionaries... until we get to the
> leaf elements, which are frozensets of strings.  The keys at every
> level of this structure are strings.  E.g. a simple example of such
> an attribute would look like:
>
> {'A': {'a': set(['1', '2', '3']),
>'b': set(['4', '5'])},
>  'B': set(['6', '7', '8'])}
>
> I'm looking for a good algorithm for computing a hash key for
> something like this?  (Basically I'm looking for a good way to
> combine hashkeys.)

You could do something like this:


deep_methods = {   
list: lambda f, l: tuple(map(f, l)),
dict: lambda f, d: frozenset((k, f(v)) for k, v in d.items()),
set: lambda f, s: frozenset(map(f, s)),
# Add more if needed
}

def apply_method(f, obj):
try:
method = deep_methods[type(obj)]
except KeyError:
return obj
return method(f, obj)

def deepfreeze(obj):
"""Return a 'hashable version' of an object
return apply_method(deepfreeze, obj)

def deephash(obj):
"""Return hash(deepfreeze(obj)) without deepfreezing"""
return hash(apply_method(deephash, obj))

# Example of deepfreezable object:
obj = [1, "foo", {(2, 4): {7, 5, 4}, "bar": "baz"}]


>>> deepfreeze(obj)
(1, 'foo', frozenset({('bar', 'baz'), ((2, 4), frozenset({4, 5, 7}))}))
>>> deephash(obj)
1341422540
>>> hash(deepfreeze(obj))
1341422540


-- 
Arnaud
-- 
http://mail.python.org/mailman/listinfo/python-list


del() function - cannot find documentation

2010-10-07 Thread mafeusek
Hallo ML.

there is following python script:

mine = { 1: "sd", 2: "mk" }
del(mine[1])
print mine

the problem is I cannot find any information about del() function in
python 2.7 documentation.
is it a documentation bug or I misunderstand something about del()?

best regards,
Pawel


-- 
http://mail.python.org/mailman/listinfo/python-list


Re: del() function - cannot find documentation

2010-10-07 Thread Chris Rebert
On Thu, Oct 7, 2010 at 1:12 PM,   wrote:
> Hallo ML.
>
> there is following python script:
>
> mine = { 1: "sd", 2: "mk" }
> del(mine[1])
> print mine
>
> the problem is I cannot find any information about del() function in
> python 2.7 documentation.
> is it a documentation bug or I misunderstand something about del()?

del is a *statement*, *not* a function:
http://docs.python.org/reference/simple_stmts.html#the-del-statement

Cheers,
Chris
--
http://blog.rebertia.com
-- 
http://mail.python.org/mailman/listinfo/python-list


Re: how to test for atomicity/mutability/hashability?

2010-10-07 Thread Arnaud Delobelle
kj  writes:

> I want to implement a test t() that will return True if its two
> arguments are "completely" different.  By this I mean that they
> don't share any "non-atomic" component.  E.g., if
>
> a = [0, 1]
> b = [0, 1]
> c = [2, 3]
> d = [2, 3]
>
> A = (a, c, 0)
> B = (a, d, 1)
> C = (b, d, 0)
>
> The desired test t() would yield:
>
> t(A, B) -> False (A and B share the mutable component a)
> t(A, C) -> True (a =!= c, b =!= d, and 0 is not mutable)
> t(B, C) -> False (B and C share the mutable component d)
>
> (=!= is shorthand with "is not identical to".)
>
> It would facilitate the implementation of t() to have a simple test
> for mutability.  Is there one?
>
> Thanks!
>
> ~kj

I think defining mutability is subject to opinion, but here is a first
approximation.


def mutable(obj):
return obj.__hash__ is None or type(obj).__hash__ == object.__hash__

def t(l1, l2):
return not any(mutable(x) and x is y for x, y in zip(l1, l2))

a = [0, 1]
b = [0, 1]
c = [2, 3]
d = [2, 3]

A = (a, c, 0)
B = (a, d, 1)
C = (b, d, 0)


>>> t(A, B)
False
>>> t(A, C)
True
>>> t(B, C)
False

-- 
Arnaud
-- 
http://mail.python.org/mailman/listinfo/python-list


Re: how to test for atomicity/mutability/hashability?

2010-10-07 Thread Chris Rebert
On Thu, Oct 7, 2010 at 1:46 PM, Arnaud Delobelle  wrote:
> kj  writes:
>> I want to implement a test t() that will return True if its two
>> arguments are "completely" different.  By this I mean that they
>> don't share any "non-atomic" component.  E.g., if
>>
>> a = [0, 1]
>> b = [0, 1]
>> c = [2, 3]
>> d = [2, 3]
>>
>> A = (a, c, 0)
>> B = (a, d, 1)
>> C = (b, d, 0)
>>
>> The desired test t() would yield:
>>
>> t(A, B) -> False (A and B share the mutable component a)
>> t(A, C) -> True (a =!= c, b =!= d, and 0 is not mutable)
>> t(B, C) -> False (B and C share the mutable component d)
>>
>> (=!= is shorthand with "is not identical to".)
>>
>> It would facilitate the implementation of t() to have a simple test
>> for mutability.  Is there one?
>>
>> Thanks!
>>
>> ~kj
>
> I think defining mutability is subject to opinion, but here is a first
> approximation.
>
>
> def mutable(obj):
>    return obj.__hash__ is None or type(obj).__hash__ == object.__hash__

Corner case (I think):
>>> a = object()
>>> a.b = "c"
Traceback (most recent call last):
  File "", line 1, in 
AttributeError: 'object' object has no attribute 'b'
>>> mutable(a)
True

Cheers,
Chris
-- 
http://mail.python.org/mailman/listinfo/python-list


Re: 64 bit offsets?

2010-10-07 Thread MRAB

On 07/10/2010 20:12, jay thompson wrote:

I'm not sure if it is limited to 32 bit addresses or if it's only
re.start() that is limited to 'em.

jt


From what I can tell, Microsoft compilers (I'm assuming you're using
Windows) have a 32-bit 'int' type for both 32-bit and 64-bit builds,
and the re module uses Py_BuildValue("i", ...), which according to the
docs uses the C 'int' type, so yes, it's 32 bits even in 64-bit Python.
:-(

I don't have a 64-bit system, but I'll see if I can build a 64-bit
version of the regex module (http://pypi.python.org/pypi/regex).


-

"It's quite difficult to remind people that all this stuff was here for
a million years before people. So the idea that we are required to
manage it is ridiculous. What we are having to manage is us." ...Bill
Ballantine, marine biologist.


On 2010-10-07 8:42 AM, "MRAB" mailto:pyt...@mrabarnett.plus.com>> wrote:

On 06/10/2010 22:41, jay thompson wrote:
>
> Hello everyone,
>
> I'm trying to extract some data fro...

I would've thought that a 64-bit version of Python would have 64-bit
offsets. Is that not the case?
--
http://mail.python.org/mailman/listinfo/python-list




--
http://mail.python.org/mailman/listinfo/python-list


Re: how to test for atomicity/mutability/hashability?

2010-10-07 Thread Arnaud Delobelle
Chris Rebert  writes:

> On Thu, Oct 7, 2010 at 1:46 PM, Arnaud Delobelle  wrote:
[...]
>> I think defining mutability is subject to opinion, but here is a first
>> approximation.
>>
>>
>> def mutable(obj):
>>    return obj.__hash__ is None or type(obj).__hash__ == object.__hash__
>
> Corner case (I think):
 a = object()
 a.b = "c"
> Traceback (most recent call last):
>   File "", line 1, in 
> AttributeError: 'object' object has no attribute 'b'
 mutable(a)
> True

Ah true.  There was also the problem that e.g. mutable(tuple) was False.

How about this second approximation:

def mutable(obj):
h = type(obj).__hash__
return not h or h is object.__hash__ and type(obj) is not object

-- 
Arnaud
-- 
http://mail.python.org/mailman/listinfo/python-list


Re: how to test for atomicity/mutability/hashability?

2010-10-07 Thread Christian Heimes
Am 07.10.2010 22:02, schrieb Chris Rebert:
> On Thu, Oct 7, 2010 at 12:13 PM, kj  wrote:
> 
>> It would facilitate the implementation of t() to have a simple test
>> for mutability.  Is there one?
> 
> Non-default hashability is an approximate heuristic:

Except that every user defined class is hashable and mutable by default ;)

>>> class Example(object):
... pass
...
>>> example = Example()
>>> hash(example)
139810551284624
>>> example.__dict__
{}
>>> example.egg = "spam"
>>> example.__dict__
{'egg': 'spam'}

-- 
http://mail.python.org/mailman/listinfo/python-list


harmful str(bytes)

2010-10-07 Thread Hallvard B Furuseth
I've been playing a bit with Python3.2a2, and frankly its charset
handling looks _less_ safe than in Python 2.

The offender is bytes.__str__: str(b'foo') == "b'foo'".
It's often not clear from looking at a piece of code whether
some data is treated as strings or bytes, particularly when
translating from old code.  Which means one cannot see from
context if str(s) or "%s" % s will produce garbage.

With 2. conversion Unicode <-> string the equivalent operation did
not silently produce garbage: it raised UnicodeError instead.  With old
raw Python strings that was not a problem in applications which did not
need to convert any charsets, with python3 they can break.

I really wish bytes.__str__ would at least by default fail.

-- 
Hallvard
-- 
http://mail.python.org/mailman/listinfo/python-list


Re: Ordering tests in a testsuite

2010-10-07 Thread Ben Finney
Ulrich Eckhardt  writes:

> However, sometimes it doesn't make sense to run test_bar() if
> test_foo() already failed, because they basically build upon each
> other.

That's a mistake. If the success of ‘test_bar’ depends on the result of
‘test_foo’, then it's not an independent test and therefore isn't a unit
test.

Which is a good reason for re-ordering tests; it's even a good idea to
randomise the order in which they're run, but AFAIK the ‘unittest’
library doesn't do that yet.

If you have state needed by multiple tests, that's what the fixtures are
for: define a ‘setUp’ (and, if useful, ‘tearDown’) to handle common
fixtures needed by tests in the test class, and ensure they're in a
known state before each test.

-- 
 \ “Science is a way of trying not to fool yourself. The first |
  `\ principle is that you must not fool yourself, and you are the |
_o__)   easiest person to fool.” —Richard P. Feynman, 1964 |
Ben Finney
-- 
http://mail.python.org/mailman/listinfo/python-list


frozendict (v0.1)

2010-10-07 Thread kj










































Following a suggestion from MRAB, I attempted to implement a
frozendict class.  My implementation took a lot more work than
something this simple should take, and it still sucks.  So I'm
hoping someone can show me a better way.  Specifically, I'm hoping
that there is a "recipe" for building off standard classes that
cover all the bases with a minimum of tedious repetitive work.

Here's my little monster:

ass frozendict():
_DESTRUCTIVE = set(('__delitem__ __setitem__ clear pop popitem setdefault '
'update').split())

_NON_DESTRUCTIVE = set(('__contains__ __format__ __getitem__ __hash__ '
'__init__ __iter__ __len__ __repr__ __sizeof__ '
'__str__ copy fromkeys get has_key items iteritems '
'iterkeys itervalues keys values'.split()))

_COMPARISONS = set(('__cmp__ __eq__ __ge__ __gt__ __le__ __lt__ '
'__ne__').split())


def __init__(self, iterable=(), **kwargs):
self._dict = dict(iterable, **kwargs)


def __hash__(self):
return hash(tuple(self.items()))


def __getattr__(self, attrib):
class_ = self.__class__
dict_ = self._dict
if attrib in class_._COMPARISONS:
return lambda x: dict_.__getattribute__(attrib)(x._dict)
elif attrib in class_._NON_DESTRUCTIVE:
return dict_.__getattribute__(attrib)
else:
if attrib in class_._DESTRUCTIVE:
raise TypeError("'%s' object is not mutable" % class_.__name__)
else:
raise AttributeError("'%s' object has no attribute '%s'" %
 (class_.__name__, attrib))



I didn't implement this as a subclass of dict to avoid having to
write a dumb little "blocking" method for every destructive dict
method.  (I couldn't figure out how to write a loop to define these
overriding methods programmatically, because their signatures are
all over the place.)

I didn't implement it as a subclass of object with an internal dict
delegate, because I couldn't figure a reasonable way to pass certain
object methods to the delegate (since in this case frozendict.__getattr__
wouldn't be called).

The handling of comparison methods is particularly horrific and
inefficient.

If "Beautiful is better than ugly", I sure how there's another way
that is a lot more beautiful than this one.

TIA!

~kj

-- 
http://mail.python.org/mailman/listinfo/python-list


Re: harmful str(bytes)

2010-10-07 Thread Arnaud Delobelle
Hallvard B Furuseth  writes:

> I've been playing a bit with Python3.2a2, and frankly its charset
> handling looks _less_ safe than in Python 2.
>
> The offender is bytes.__str__: str(b'foo') == "b'foo'".
> It's often not clear from looking at a piece of code whether
> some data is treated as strings or bytes, particularly when
> translating from old code.  Which means one cannot see from
> context if str(s) or "%s" % s will produce garbage.
>
> With 2. conversion Unicode <-> string the equivalent operation did
> not silently produce garbage: it raised UnicodeError instead.  With old
> raw Python strings that was not a problem in applications which did not
> need to convert any charsets, with python3 they can break.
>
> I really wish bytes.__str__ would at least by default fail.

I think you misunderstand the purpose of str().  It is to provide a
(unicode) string representation of an object and has nothing to do with
converting it to unicode:

>>> b = b"\xc2\xa3"
>>> str(b)
"b'\\xc2\\xa3'"


If you want to *decode* a bytes string, use its decode method and you
get a unicode string (if your bytes string is a valid encoding):

>>> b = b"\xc2\xa3"
>>> b.decode('utf8')
'£'
>>> b.decode('ascii')
Traceback (most recent call last):
  File "", line 1, in 
UnicodeDecodeError: 'ascii' codec can't decode byte 0xc2 in position 0: ordinal 
not in range(128)


If you want to *encode* a (unicode) string, use its encode method and you
get a bytes string (provided your string can be encoded using the given
encoding):

>>> s="€"
>>> s.encode('utf8')
b'\xe2\x82\xac'
>>> s.encode('ascii')
Traceback (most recent call last):
  File "", line 1, in 
UnicodeEncodeError: 'ascii' codec can't encode character '\u20ac' in position 
0: ordinal not in range(128)

-- 
Arnaud
-- 
http://mail.python.org/mailman/listinfo/python-list


Re: frozendict (v0.1)

2010-10-07 Thread Arnaud Delobelle
kj  writes:

> Following a suggestion from MRAB, I attempted to implement a
> frozendict class.  My implementation took a lot more work than
> something this simple should take, and it still sucks.  So I'm
> hoping someone can show me a better way.  Specifically, I'm hoping
> that there is a "recipe" for building off standard classes that
> cover all the bases with a minimum of tedious repetitive work.
>
> Here's my little monster:
>
> class frozendict():
[...]
> def __hash__(self):
> return hash(tuple(self.items()))

There is a problem with this hash function stemming from the fact that
the hash value of a tuple is different depending on the order of its
elements, i.e. hash((1, 2)) is not equal to hash((2, 1)).

Now, if there is a hash collision in the keys of the dictionary, then
the order of enumeration of its items will depend on the order in which
they were inserted into the dictionary.  Here is a simple example below 

>>> h = hash('a') # So h and 'a' have the same hash value
>>> d = {'a':1, h:2}
>>> d1 = {'a':1, h:2}
>>> d2 = {h:2, 'a':1}
>>> d1 == d2 # The dictionaries are equal...
True
>>> tuple(d1.items()) # but their items are enumerated in...
(('a', 1), (-468864544, 2))
>>> tuple(d2.items()) # different orders!
((-468864544, 2), ('a', 1))

A simple fix is to use hash(frozenset(self.items())) instead.

-- 
Arnaud
-- 
http://mail.python.org/mailman/listinfo/python-list


Re: how to test for atomicity/mutability/hashability?

2010-10-07 Thread Chris Rebert
On Thu, Oct 7, 2010 at 2:28 PM, Christian Heimes  wrote:
> Am 07.10.2010 22:02, schrieb Chris Rebert:
>> On Thu, Oct 7, 2010 at 12:13 PM, kj  wrote:
>> 
>>> It would facilitate the implementation of t() to have a simple test
>>> for mutability.  Is there one?
>>
>> Non-default hashability is an approximate heuristic:
>
> Except that every user defined class is hashable and mutable by default ;)
>
 class Example(object):
> ...     pass
> ...
 example = Example()
 hash(example)
> 139810551284624
 example.__dict__
> {}
 example.egg = "spam"
 example.__dict__
> {'egg': 'spam'}

Hence exactly why I said *non-default* hashability (i.e. the object is
hashable, but does not use the default implementation which you point
out exists):

>>> is_immutable(Example())
False

Not that the heuristic I suggested is infallible or anything though.

Cheers,
Chris
--
http://blog.rebertia.com
-- 
http://mail.python.org/mailman/listinfo/python-list


Re: frozendict (v0.1)

2010-10-07 Thread Arnaud Delobelle
Oops I sent off my reply before I had finished!

kj  writes:

> Following a suggestion from MRAB, I attempted to implement a
> frozendict class.  My implementation took a lot more work than
> something this simple should take, and it still sucks.  So I'm
> hoping someone can show me a better way.  Specifically, I'm hoping
> that there is a "recipe" for building off standard classes that
> cover all the bases with a minimum of tedious repetitive work.
>
> Here's my little monster:
>
> ass frozendict():
> _DESTRUCTIVE = set(('__delitem__ __setitem__ clear pop popitem setdefault 
> '
> 'update').split())
[...]
> def __hash__(self):
> return hash(tuple(self.items()))

Also, you'll probably want to cache your hash value.

>
>
> def __getattr__(self, attrib):
> class_ = self.__class__
> dict_ = self._dict
> if attrib in class_._COMPARISONS:
> return lambda x: dict_.__getattribute__(attrib)(x._dict)
> elif attrib in class_._NON_DESTRUCTIVE:
> return dict_.__getattribute__(attrib)
> else:
> if attrib in class_._DESTRUCTIVE:
> raise TypeError("'%s' object is not mutable" % 
> class_.__name__)
> else:
> raise AttributeError("'%s' object has no attribute '%s'" %
>  (class_.__name__, attrib))
>
>
>
> I didn't implement this as a subclass of dict to avoid having to
> write a dumb little "blocking" method for every destructive dict
> method.  (I couldn't figure out how to write a loop to define these
> overriding methods programmatically, because their signatures are
> all over the place.)

You could subclass dict and have something like:

class frozendict(dict):
DESTRUCTIVE = ...
for meth in DESTRUCTIVE:
exec """def %s(s, *a, **k): raise TypeError("not mutable")"""
del DESTRUCTIVE
...

-- 
Arnaud
-- 
http://mail.python.org/mailman/listinfo/python-list


Many newbie questions regarding python

2010-10-07 Thread Rogério Brito
Hi there.

I am used to some languages like C, but I am just a complete newbie with Python
and, while writing some small snippets, I had encountered some problems, with
which I would sincerely appreciate any help, since I appreciate this language to
write my "running pseudocode in" and I am seriously thinking of using it to
teach some algorithms classes.

1 - The first issue that I am having is that I don't seem to be able to, say,
use something that would be common for people writing programs in C: defining a
one-dimensional vector and only initializing it when needed.

For instance, in C, I would write something like:

int v[20];
for (i = 0; i < 20; i++)
v[i] = 0;

Note that I only define the vector v (and its size) at the beginning but
initialize it latter during the code per-se.

My first try to write it in Python was something like this:

v = []
for i in range(20):
v[i] = 0

Unfortunately, this doesn't work, as I get an index out of bounds when trying to
index the v list. Of course, the main difference between the two snippets is
that, in C, I declared v to have 20 positions, while in python I initialized it
to be the empty list and, indeed, it has an empty set of indexes.

What is the Pythonic way of writing code like this? So far, I have found many
alternatives and I would like to write code that others in the Python community
would find natural to read. Some of the things that crossed my mind:

v = [0 for i in range(20)]

v = [0] * 20

v = []
for i in range(20): v.append(0)

What should I prefer? Any other alternative?

If possible, I would like to simply declare the list and fill it latter in my
program, as lazily as possible (this happens notoriously when one is using a
technique of programming called dynamic programming where initializing all
positions of a table may take too much time in comparison to the filling of the
array).


2 - If I declare a class with some member variables, is is strictly necessary
for me to qualify those members in a method in that class? For instance, if I
define:

class C:
f = 1
def g(self):
return f

I get an annoying message when I try to call the g method in an object of type
C, telling me that there's no global symbol called f. If I make g return self.f
instead, things work as expected, but the code loses some readability.

Is there any way around this or is that simply "a matter of life"?

I have some other questions, but I will save them for latter.

Please, keep in mind that I am a newbie in Python. Despite that, I am enjoying
the little that I know.


Thank you very much in advance,

-- 
Rogério Brito : rbr...@{ime.usp.br,gmail.com} : GPG key 4096R/BCFC
http://rb.doesntexist.org : Packages for LaTeX : algorithms.berlios.de
DebianQA: http://qa.debian.org/developer.php?login=rbrito%40ime.usp.br
-- 
http://mail.python.org/mailman/listinfo/python-list


Re: Eclipse/PyDev - BOM Lexical Error

2010-10-07 Thread Lawrence D'Oliveiro
In message <87hbgyosdc@web.de>, Diez B. Roggisch wrote:

> Lawrence D'Oliveiro  writes:
> 
>> In message <87d3rorf2f@web.de>, Diez B. Roggisch wrote:
>>
>>> Lawrence D'Oliveiro  writes:
>>> 
 What exactly is the point of a BOM in a UTF-8-encoded file?
>>> 
>>> It's a marker like the "coding: utf-8" in python-files. It tells the
>>> software aware of it that the content is UTF-8.
>>
>> But if the software is aware of it, then why does it need to be told?
> 
> Let me rephrase: windows editors such as notepad recognize the BOM, and
> then assume (hopefully rightfully so) that the rest of the file is text
> in utf-8 encoding.

But they can only recognize it as a BOM if they assume UTF-8 encoding to 
begin with. Otherwise it could be interpreted as some other coding.

-- 
http://mail.python.org/mailman/listinfo/python-list


Re: Many newbie questions regarding python

2010-10-07 Thread MRAB

On 08/10/2010 00:10, Rogério Brito wrote:

Hi there.

I am used to some languages like C, but I am just a complete newbie with Python
and, while writing some small snippets, I had encountered some problems, with
which I would sincerely appreciate any help, since I appreciate this language to
write my "running pseudocode in" and I am seriously thinking of using it to
teach some algorithms classes.

1 - The first issue that I am having is that I don't seem to be able to, say,
use something that would be common for people writing programs in C: defining a
one-dimensional vector and only initializing it when needed.

For instance, in C, I would write something like:

int v[20];
for (i = 0; i<  20; i++)
 v[i] = 0;

Note that I only define the vector v (and its size) at the beginning but
initialize it latter during the code per-se.

My first try to write it in Python was something like this:

v = []
for i in range(20):
 v[i] = 0

Unfortunately, this doesn't work, as I get an index out of bounds when trying to
index the v list. Of course, the main difference between the two snippets is
that, in C, I declared v to have 20 positions, while in python I initialized it
to be the empty list and, indeed, it has an empty set of indexes.

What is the Pythonic way of writing code like this? So far, I have found many
alternatives and I would like to write code that others in the Python community
would find natural to read. Some of the things that crossed my mind:

 v = [0 for i in range(20)]

 v = [0] * 20

 v = []
 for i in range(20): v.append(0)

What should I prefer? Any other alternative?

If possible, I would like to simply declare the list and fill it latter in my
program, as lazily as possible (this happens notoriously when one is using a
technique of programming called dynamic programming where initializing all
positions of a table may take too much time in comparison to the filling of the
array).


Python doesn't have declarations. The code:

v = []

simply creates an empty list and binds the name 'v' to it. If you want 
to create a list containing 20 zeros then:


v = [0] * 20

is the Pythonic way. Which one you do depends on the particular problem
you're working on; do whichever makes the most sense.


2 - If I declare a class with some member variables, is is strictly necessary
for me to qualify those members in a method in that class? For instance, if I
define:

class C:
 f = 1
 def g(self):
 return f

I get an annoying message when I try to call the g method in an object of type
C, telling me that there's no global symbol called f. If I make g return self.f
instead, things work as expected, but the code loses some readability.

Is there any way around this or is that simply "a matter of life"?


The name 'f' in that case is an attribute of the class itself. If you
want it to be an attribute of an instance of the class then do
something like this:

class C:
def __init__(self):
self.f = 1
def g(self):
return self.f

You should write Python code in the Python idiom and not try to look
for the Python equivalent of a C idiom. In other words, don't try to
write a C program in Python! You might want to read "The Zen of
Python". Just type:

help("this")

at the Python prompt.
--
http://mail.python.org/mailman/listinfo/python-list


Re: Many newbie questions regarding python

2010-10-07 Thread Andreas Waldenburger
On Thu, 07 Oct 2010 20:10:14 -0300 Rogério Brito 
wrote:

> I am used to some languages like C, but I am just a complete newbie
> with Python and, while writing some small snippets, I had encountered
> some problems, with which I would sincerely appreciate any help,
> since I appreciate this language to write my "running pseudocode in"
> and I am seriously thinking of using it to teach some algorithms
> classes.
> 
Let me congratulate you on your choice.


> 1 - The first issue that I am having is that I don't seem to be able
> to, say, use something that would be common for people writing
> programs in C: defining a one-dimensional vector and only
> initializing it when needed.
> [snip]
I could be cheeky here and ask "Do you really need this? REALLY?". And
I will: Really? ;)

> 
> Note that I only define the vector v (and its size) at the beginning
> but initialize it latter during the code per-se.
> 
> [snip]
> 
> What is the Pythonic way of writing code like this? So far, I have
> found many alternatives and I would like to write code that others in
> the Python community would find natural to read. Some of the things
> that crossed my mind:
> 
> v = [0 for i in range(20)]
> 
Pretty good.


> v = [0] * 20
> 
This will break on you if you replace the zeros with mutable objects.
Try it, you'll be unpleasantly surprised.

I guess it's OK for numbers, though. Or tuples, if need be.


> v = []
> for i in range(20): v.append(0)
> 
That's the older (pre list comp) way, I guess. No need for it anymore,
unless the initialization is more complicated than this.


> 
> If possible, I would like to simply declare the list and fill it
> latter in my program, as lazily as possible (this happens notoriously
> when one is using a technique of programming called dynamic
> programming where initializing all positions of a table may take too
> much time in comparison to the filling of the array).
> 
Nah, don't worry too much about performance. You can do that when your
program is too slow.

I know that this isn't too satisfying, but seriously: Try to write
programs for humans, not for the computer. (I'm talking about Python
here; in C it's certainly par for the course to try to think like a
computer does.)


> 2 - If I declare a class with some member variables, is is strictly
> necessary for me to qualify those members in a method in that class?
> For instance, if I define:
> 
> class C:
> f = 1
> def g(self):
> return f
> 
> I get an annoying message when I try to call the g method in an
> object of type C, telling me that there's no global symbol called f.
> If I make g return self.f instead, things work as expected, but the
> code loses some readability.
> 
No it doesn't.

(... pedagogical pause ...)

Python kind of forces you to be *very* explicit about which f you mean.
And this is a good thing. If you mean the global f, then say f. If you
mean the instance attribute of the current instance then say self.f and
if you mean the class attribute then say C.f (or, if you fear you're
going to be renaming C a lot, self.__class__.f).

Note here that self.f refers to the f as accessible by the
specific *instance* of the class, *not* the class attribute C.f. If you
don't change f on the instance, then they'll coincide, otherwise they
won't. Example (warning: not Python 3 compatible!):

[wilde...@localhost ~]$ ipython
Python 2.5.2 (r252:60911, Sep 30 2008, 15:41:38)
Type "copyright", "credits" or "license" for more information.

IPython 0.8.4 -- An enhanced Interactive Python.
? -> Introduction and overview of IPython's features.
%quickref -> Quick reference.
help  -> Python's own help system.
object?   -> Details about 'object'. ?object also works, ?? prints more.

In [1]: class C:
   ...: f = 1
   ...: def g(self):
   ...: return self.f
   ...:
   ...:

In [2]: c = C();

In [3]: print c.g()
1

In [4]: C.f = 2

In [5]: print c.g()
2

In [6]: c.f = 3

In [7]: print c.g()
3

In [8]: print C.f
2



> Is there any way around this or is that simply "a matter of life"?
> 
Yes.


> I have some other questions, but I will save them for latter.
> 
Keep 'em coming. But be prepared for hearing "get used to it" many a
time. Or, from the more harsh folks, "don't try to write C in
Python". :)


> Please, keep in mind that I am a newbie in Python. Despite that, I am
> enjoying the little that I know.
> 
Really? I never enjoy knowing just a little. ;)



-- 
To reach me via email, replace INVALID with the country code of my home 
country.  But if you spam me, I'll be one sour kraut.

-- 
http://mail.python.org/mailman/listinfo/python-list


Re: Many newbie questions regarding python

2010-10-07 Thread Andreas Waldenburger
On Fri, 08 Oct 2010 00:46:41 +0100 MRAB 
wrote:

> In other words, don't try to write a C program in Python!

Man, I'm good. :D

/W

-- 
To reach me via email, replace INVALID with the country code of my home 
country.  But if you spam me, I'll be one sour kraut.

-- 
http://mail.python.org/mailman/listinfo/python-list


Re: Many newbie questions regarding python

2010-10-07 Thread Tim Harig
On 2010-10-07, Rogério Brito  wrote:
> 1 - The first issue that I am having is that I don't seem to be able to, say,
> use something that would be common for people writing programs in C: defining 
> a
> one-dimensional vector and only initializing it when needed.
>
> For instance, in C, I would write something like:
>
> int v[20];
> for (i = 0; i < 20; i++)
> v[i] = 0;
>
> Note that I only define the vector v (and its size) at the beginning but
> initialize it latter during the code per-se.

You are reserving enough space to be defined as an array of 20 integers.
Note that after you declare it, you *can* access those memory locations
even before you initialize them.  They therefore still contain something
even if you have not yet defined what that something is.  Note that
uninitialized variables are a common C error.

Python is dynamic. You never need to allocate memory for something, you
simply assign something and the system takes care of the allocation for
you.  Before something is assigned, it cannot be addressed.  Nothing takes
up memory before it is assigned.

> Unfortunately, this doesn't work, as I get an index out of bounds when trying 
> to
> index the v list. Of course, the main difference between the two snippets is
> that, in C, I declared v to have 20 positions, while in python I initialized 
> it
> to be the empty list and, indeed, it has an empty set of indexes.
>
> What is the Pythonic way of writing code like this? So far, I have found many
> alternatives and I would like to write code that others in the Python 
> community
> would find natural to read. Some of the things that crossed my mind:
>
> v = [0 for i in range(20)]
>
> v = [0] * 20
>
> v = []
> for i in range(20): v.append(0)
>
> What should I prefer? Any other alternative?

It really would help to know what you are trying to with v.  If we knew
that, we might be able to make better suggestions.  Usually in Python,
there is no need to initialize elements in a list as you can simply add
them as you need them.

> If possible, I would like to simply declare the list and fill it latter in my
> program, as lazily as possible (this happens notoriously when one is using a
> technique of programming called dynamic programming where initializing all
> positions of a table may take too much time in comparison to the filling of 
> the
> array).

What I *think* you *might* be looking for is a kind of sparse list.  You
can achieve this by using a dictionary with numeric keys rather then a
list.  Not that when you do this, you recreate the C problem of trying to
access something that has not been allocated.  You either need to make sure
that a key exists before trying to access it or catch and handleKeyError.

> 2 - If I declare a class with some member variables, is is strictly necessary
> for me to qualify those members in a method in that class? For instance, if I
> define:
>
> class C:
> f = 1
> def g(self):
> return f

Note that you have assigned f to the class object and not to the instance
objects.  This is a common pitfall for new Python programmers.  I suggest
assigning f in the constructor unless you are really sure that you want to
assign it to the class object which will be shared among the instance
objects.

> I get an annoying message when I try to call the g method in an object of type
> C, telling me that there's no global symbol called f. If I make g return 
> self.f
> instead, things work as expected, but the code loses some readability.
>
> Is there any way around this or is that simply "a matter of life"?

Each member functions has its own scope.  The object itself is not part of
this scope.  Therefore, you must use the self object reference to access
other members of the object containing the function.

I don't really see the readability problem; but, you could work around it
by creating a reference to the member at the start of the function:

def g(self):
f = self.f

For this to work, f must be mutable.  If f is not mutable then you can wrap
f inside of a class in such a way that it is mutable.  Do this at your own
risk.  In doing this, you are starting to play with fire.
-- 
http://mail.python.org/mailman/listinfo/python-list


Re: frozendict (v0.1)

2010-10-07 Thread kj
In <87hbgxlk67@gmail.com> Arnaud Delobelle  writes:

>A simple fix is to use hash(frozenset(self.items())) instead.

Thanks for pointing out the hash bug.  It was an oversight: I meant
to write

def __hash__(self):
return hash(sorted(tuple(self.items(

I imagine that frozenset is better than sorted(tuple(...)) here,
but it's not obvious to me why.

At any rate, using your suggestions in this and your other post,
the current implementation of frozendict stands at:

class frozendict(dict):
for method in ('__delitem__ __setitem__ clear pop popitem setdefault '
   'update').split():
exec """
def %s(self, *a, **k):
cn = self.__class__.__name__
raise TypeError("'%%s' object is not mutable" %% cn)
""" % method

def __hash__(self):
return hash(frozenset(self.items()))

...which is a lot nicer!

Thanks!

~kj
-- 
http://mail.python.org/mailman/listinfo/python-list


Re: Many newbie questions regarding python

2010-10-07 Thread Steven D'Aprano
On Thu, 07 Oct 2010 20:10:14 -0300, Rogério Brito wrote:


> What is the Pythonic way of writing code like this? So far, I have
> found many alternatives and I would like to write code that others in
> the Python community would find natural to read. Some of the things
> that crossed my mind:
>
> v = [0 for i in range(20)]

Absolutely not. Such a code snippet is very common, in fact I've done it 
myself, but it is a "hammer solution" -- to a small boy with a hammer, 
everything looks like a nail that needs hammering. Writing such a list 
comprehension is a "list comp solution".


> v = [0] * 20

Yes, this is the solution. But given your earlier explanation that you 
want to lazily initialise v, I don't think it applies, because it 
initialises it all at once. (Of course, such initialisation is cheap, but 
it does happen all at once.)

If you are writing this:

v = []  # declare v as an empty list
do_something()
do_something_else()
v = [0]*20  # now initialise v before using it
do_something_with(v)


then such declarations are not necessary and are discouraged in Python. 
Having an unused, empty variable v floating around doing nothing is 
pointless in Python. Just write:

do_something()
do_something_else()
v = [0]*20  # create v before using it
do_something_with(v)



> v = []
> for i in range(20): v.append(0)

I would never write that literally. It's as bad as the list comp, only it 
takes up more space. It too also fails to be lazy initialisation. 
However, using append is the correct way when you have a situation where 
you need to dynamically grow the list, e.g.:


v = []
for i in range(20):
v.append(0)
do_something_with(v)



Two more alternatives:

v = list(range(20))  # in Python 2.x you can leave out the call to list()

or

v = []
v.extend(range(20))

both initialise v to [0, 1, 2, 3, ... , 19] instead of [0, 0, ..., 0].




> 2 - If I declare a class with some member variables, is is strictly
> necessary for me to qualify those members in a method in that class? For
> instance, if I define:

In Python, we generally refer to "attributes" rather than "members".

 
> class C:
> f = 1
> def g(self):
> return f

By the way, you have created a class attribute f which is shared by all 
instances. You probably want:

class C:
def __init__(self):
self.f = 1
def g(self):
return self.f


 
> I get an annoying message when I try to call the g method in an object
> of type C, telling me that there's no global symbol called f.

No you don't. You get a message that there is no global NAME called f.

You might think I'm being pedantic, but I'm not. If you're thinking in 
terms of C language, you probably think that there is a symbol table 
created by the compiler so that Python can look at a reference "f" and 
say "oh, that's a member variable" at compile time, and only the correct 
value needs to be looked up at runtime. But that's not Python's execution 
model. *Everything* in Python is looked up dynamically at runtime in 
namespaces. (The only exceptions are statements.) So when you write "f", 
Python *cannot* know at compile time whether it is a local variable, a 
non-local, a global, an attribute (member) or something else. It must 
look the name up in one or more namespaces at runtime.


(Of course, you might already know this, in which case, yes, I'm just 
being pedantic *grins* )



> If I make
> g return self.f instead, things work as expected, but the code loses
> some readability.

On the contrary, it increases readability, because it explicitly tells 
the reader that you are accessing an attribute f rather than a local 
variable.

Remember that Python uses nested namespaces. Inside the method C.g above 
the namespaces that are searched are:

local variables
any non-local (nested) functions or closures (none in this example)
global variables
built-ins

in that order. Each namespace is independent, and Python will never try 
to guess that f is an attribute of the instance or class unless you 
explicitly tell it so. For example, an attribute instance.len will never 
block access to the built-in function len().

(Actually, there is one exception... when a class statement is executed 
for the first time, creating the class, local variables of the class 
block are identified as attributes. This is a case of practicality beats 
purity, since otherwise Python would need extra syntax for creating 
methods and class attributes.)


Attributes have their own search order:

instance attributes
class attributes
attributes of any base classes

When you refer to instance.attribute, the namespaces are searched in that 
order for the name "attribute". The locals and globals are not searched.



Hope this helps,



-- 
Steven
-- 
http://mail.python.org/mailman/listinfo/python-list


Re: frozendict (v0.1)

2010-10-07 Thread Steven D'Aprano
On Fri, 08 Oct 2010 00:23:30 +, kj wrote:

> In <87hbgxlk67@gmail.com> Arnaud Delobelle 
> writes:
> 
>>A simple fix is to use hash(frozenset(self.items())) instead.
> 
> Thanks for pointing out the hash bug.  It was an oversight: I meant to
> write
> 
> def __hash__(self):
> return hash(sorted(tuple(self.items(
> 
> I imagine that frozenset is better than sorted(tuple(...)) here, but
> it's not obvious to me why.


Because it's always better to use a well-written, fast, efficient, 
correct, well-tested wheel than to invent your own slow, incorrect 
wheel :)



-- 
Steven
-- 
http://mail.python.org/mailman/listinfo/python-list


Re: Ordering tests in a testsuite

2010-10-07 Thread Steven D'Aprano
On Fri, 08 Oct 2010 08:35:12 +1100, Ben Finney wrote:

> Ulrich Eckhardt  writes:
> 
>> However, sometimes it doesn't make sense to run test_bar() if
>> test_foo() already failed, because they basically build upon each
>> other.
> 
> That's a mistake. If the success of ‘test_bar’ depends on the result of
> ‘test_foo’, then it's not an independent test and therefore isn't a unit
> test.


I don't think that is what Ulrich means to imply. I don't think he means 
that test_bar uses the result of test_foo, but only that if test_foo 
fails then test_bar has no hope of succeeding.

To give an example, suppose you have:

class MyClass:
def __init__(self, cheese):
self.cheese = cheese

class MyTests(unittest.TestCase):
def test_create(self):
# Test that instances can be created correctly.
self.assert_raises(TypeError, MyClass)
x = MyClass("cheddar")  # will fail if MyClass can't be created
def test_cheese(self):
# Test that instances have a cheese attribute.
x = MyClass("swiss")
self.assertEquals(x.cheese, "swiss")


If test_create fails, then so will test_cheese, not because the later 
test is dependent on the first, but because creation is more fundamental 
than attribute access. I think the OP wants to skip test_cheese in the 
even that test_create fails.



-- 
Steven
-- 
http://mail.python.org/mailman/listinfo/python-list


Re: toy list processing problem: collect similar terms

2010-10-07 Thread sln
On Wed, 06 Oct 2010 10:52:19 -0700, s...@netherlands.com wrote:

>On Sat, 25 Sep 2010 21:05:13 -0700 (PDT), Xah Lee  wrote:
>
>>here's a interesting toy list processing problem.
>>
>>I have a list of lists, where each sublist is labelled by
>>a number. I need to collect together the contents of all sublists
>>sharing
>>the same label. So if I have the list
>>
>>((0 a b) (1 c d) (2 e f) (3 g h) (1 i j) (2 k l) (4 m n) (2 o p) (4 q
>>r) (5 s t))
>>
>>where the first element of each sublist is the label, I need to
>>produce:
>>
>>output:
>>((a b) (c d i j) (e f k l o p) (g h) (m n q r) (s t))
>>
>[snip]
>
>>anyone care to give a solution in Python, Perl, javascript, or other
>>lang? am guessing the scheme solution can be much improved... perhaps
>>using some lib but that seems to show scheme is pretty weak if the lib
>>is non-standard.
>>
>
>Crossposting to Lisp, Python and Perl because the weird list of lists looks
>like Lisp or something else, and you mention other languages so I'm throwing
>this out for Perl.
>
>It appears this string you have there is actually list syntax in another 
>language.
>If it is, its the job of the language to parse the data out. Why then do you
>want to put it into another language form? At runtime, once the data is in 
>variables,
>dictated by the syntax, you can do whatever data manipulation you want
>(combining arrays, etc..).
>
>So, in the spirit of a preprocessor, given that the text is balanced, with 
>proper closure,
>ie:   ( (data) (data) )is ok.
>  ( data (data) )  is not ok.
>
>the below does simple text manipulation, joining like labeled sublists, 
>without going into
>the runtime guts of internalizing the data itself. Internally, this is too 
>simple.
>

If not preprocessor, then ...
The too simple, order independent, id independent, Perl approach.

-sln
-

use strict;
use warnings;
use Data::Dump 'dump';

my @inp = ([0,'a','b'],[1,'c','d'],[2,'e','f'],[3,'g','h'],
   [1,'i','j'],[2,'k','l'],[4,'m','n'],[2,'o','p'],
   [4,'q','r'],[5,'s','t']);

my ($cnt, @outp, %hs) = (0);

for my $ref (@inp) {
   $hs{ $$ref[0] } or $hs{ $$ref[0] } = $cnt++;
   push @{$outp[ $hs{ $$ref[0] } ] }, @{$ref}[ 1 .. $#{$ref} ];
}

dump @outp;

__END__

(
  ["a", "b"],
  ["c", "d", "i", "j"],
  ["e", "f", "k", "l", "o", "p"],
  ["g", "h"],
  ["m", "n", "q", "r"],
  ["s", "t"],
)

-- 
http://mail.python.org/mailman/listinfo/python-list


ANN: Roundup Issue Tracker 1.4.16 released

2010-10-07 Thread Richard Jones
I'm proud to release version 1.4.16 of Roundup which introduces some
minor features and, as usual, fixes some bugs:

Features:

- allow trackers to override the classes used to render properties in
  templating per issue2550659 (thanks Ezio Melotti)
- new mailgw configuration item "subject_updates_title": If set to "no"
  a changed subject in a reply to an issue will not update the issue
  title with the changed subject. Thanks to Arkadiusz Kita and Peter
  Funk for requesting the feature and discussing the implementation.
  http://thread.gmane.org/gmane.comp.bug-tracking.roundup.user/10169
- new rdbms config item sqlite_timeout makes the previously hard-coded
  timeout of 30 seconds configurable. This is the time a client waits
  for the locked database to become free before giving up. Used only for
  SQLite backend.
- new mailgw config item unpack_rfc822 that unpacks message attachments
  of type message/rfc822 and attaches the individual parts instead of
  attaching the whole message/rfc822 attachment to the roundup issue.

Fixed:

- fixed reporting of source missing warnings
- relevant tests made locale independent, issue2550660 (thanks
  Benni Bärmann for reporting).
- fix for incorrect except: syntax, issue2550661 (thanks Jakub Wilk)
- No longer use the root logger, use a logger with prefix "roundup",
  see http://thread.gmane.org/gmane.comp.bug-tracking.roundup.devel/5356
- improve handling of '>' when URLs are converted to links, issue2550664
  (thanks Ezio Melotti)
- fixed registration, issue2550665 (thanks Timo Paulssen)
- make sorting of multilinks in the web interface more robust, issue2550663
- Fix charset of first text-part of outgoing multipart messages, thanks Dirk
  Geschke for reporting, see
  http://thread.gmane.org/gmane.comp.bug-tracking.roundup.user/10223
- Fix handling of incoming message/rfc822 attachments. These resulted in
  a weird mail usage error because the email module threw a TypeError
  which roundup interprets as a Reject exception. Fixes issue2550667.
  Added regression tests for message/rfc822 attachments with and without
  configured unpacking (mailgw unpack_rfc822, see Features above)
  Thanks to Benni Bärmann for reporting.
- Allow search_popup macro to work with all db classes, issue2550567
  (thanks John Kristensen)
- lower memory footprint for (journal-) import

If you're upgrading from an older version of Roundup you *must* follow
the "Software Upgrade" guidelines given in the maintenance documentation.

Roundup requires python 2.3 or later (but not 3+) for correct operation.

To give Roundup a try, just download (see below), unpack and run::

roundup-demo

Release info and download page:
 http://cheeseshop.python.org/pypi/roundup
Source and documentation is available at the website:
 http://roundup.sourceforge.net/
Mailing lists - the place to ask questions:
 http://sourceforge.net/mail/?group_id=31577


About Roundup
=

Roundup is a simple-to-use and -install issue-tracking system with
command-line, web and e-mail interfaces. It is based on the winning design
from Ka-Ping Yee in the Software Carpentry "Track" design competition.

Note: Ping is not responsible for this project. The contact for this
project is rich...@users.sourceforge.net.

Roundup manages a number of issues (with flexible properties such as
"description", "priority", and so on) and provides the ability to:

(a) submit new issues,
(b) find and edit existing issues, and
(c) discuss issues with other participants.

The system will facilitate communication among the participants by managing
discussions and notifying interested parties when issues are edited. One of
the major design goals for Roundup that it be simple to get going. Roundup
is therefore usable "out of the box" with any python 2.3+ (but not 3+)
installation. It doesn't even need to be "installed" to be operational,
though an install script is provided.

It comes with two issue tracker templates (a classic bug/feature tracker and
a minimal skeleton) and four database back-ends (anydbm, sqlite, mysql
and postgresql).
-- 
http://mail.python.org/mailman/listinfo/python-list


Re: del() function - cannot find documentation

2010-10-07 Thread Tim Roberts
Chris Rebert  wrote:
>On Thu, Oct 7, 2010 at 1:12 PM,   wrote:
>>
>> there is following python script:
>>
>> mine = { 1: "sd", 2: "mk" }
>> del(mine[1])
>> print mine
>>
>> the problem is I cannot find any information about del() function in
>> python 2.7 documentation.
>> is it a documentation bug or I misunderstand something about del()?
>
>del is a *statement*, *not* a function:
>http://docs.python.org/reference/simple_stmts.html#the-del-statement

This kind of confusion is, in my opinion, the primary reason why
parentheses should never be used with "del" and "return", as we so commonly
see.
-- 
Tim Roberts, t...@probo.com
Providenza & Boekelheide, Inc.
-- 
http://mail.python.org/mailman/listinfo/python-list


Re: frozendict (v0.1)

2010-10-07 Thread Arnaud Delobelle
kj  writes:

> In <87hbgxlk67@gmail.com> Arnaud Delobelle  writes:
>
>>A simple fix is to use hash(frozenset(self.items())) instead.
>
> Thanks for pointing out the hash bug.  It was an oversight: I meant
> to write
>
> def __hash__(self):
> return hash(sorted(tuple(self.items(

But sorted returns a list!

so you need rather: hash(tuple(sorted(self.items(

However, it still won't work because some keys may be incomparable.

E.g., try with {1:'a', 1j:'b'}

-- 
Arnaud
-- 
http://mail.python.org/mailman/listinfo/python-list