Re: [Python-Dev] Let's update CObject API so it is safe and regular!

[Hrvoje Niksic]

Larry Hastings wrote:
If we're adding type information, then please make it a Python object 
rather than a C string.  That way the creator and the consumer can use 
a richer API to query the "type", such as by calling its methods or by 
inspecting it in some other way.


I'm not writing my patch that way; it would be too cumbersome for what 
is ostensibly an easy, light-weight API.   If you're going that route

> you might as well create a real PyTypeObject for the blob you're
> passing in.

Well, that's exactly the point, given a PyObject* tag, you can add any 
kind of type identification you need, including some Python type.  (It 
is assumed that the actual pointer you're passing is not a PyObject 
itself, of course, otherwise you wouldn't need PyCObject at all.)


I have no desire to compete with your patch, it was a suggestion for 
(what I see as) improvement.

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Package Management - thoughts from the peanut gallery

[Tarek Ziadé]

Guys,

I have taken the commitment to lead these tasks and synchronize the people
that are willing to help on this.

We are working on several tasks and PEPS to make things happen since the
summit.

There's no public roadmap yet on when things will be done (because there's
no 100% certitude yet on what shall be done).
But that it will probably be too late to see it happen in 3.1. Python 2.7
will be our target.

The tasks discussed so far are:

- version definition (http://wiki.python.org/moin/DistutilsVersionFight)
- egg.info standardification (PEP 376)
- metadata enhancement (rewrite PEP 345)
- static metadata definition work  (*)
- creation of a network of OS packager people
- PyPI mirroring (PEP 381)

Each one of this task has a leader, except the one with (*). I just got back
from travelling, and I will reorganize
http://wiki.python.org/moin/Distutils asap to it is up-to-date.

If you want to work on one of this task or feel there's a new task you can
start, please, join Distutils SIG or contact me,

Regards
Tarek

On Fri, Apr 3, 2009 at 6:55 AM, Stephen J. Turnbull wrote:

> Chris Withers writes:
>
>  > Personally I feel all of the above are perfectly possible, and can't see
>  > anyone being left unhappy by them. I'm sure I've missed something then,
>  > otherwise why not make it happen?
>
> Labor shortage.
>
> We will need a PEP, the PEP will need a sample implementation, and
> a proponent.  Who's gonna bell the cat?
> ___
> Python-Dev mailing list
> Python-Dev@python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> http://mail.python.org/mailman/options/python-dev/ziade.tarek%40gmail.com
>



-- 
Tarek Ziadé | Association AfPy | www.afpy.org
Blog FR | http://programmation-python.org
Blog EN | http://tarekziade.wordpress.com/
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] UnicodeDecodeError bug in distutils

[Tarek Ziadé]

On Fri, Apr 3, 2009 at 2:25 AM, Ben Finney  wrote:
> "Phillip J. Eby"  writes:
>
>> However, there's currently no standard, as far as I know, for what
>> encoding the PKG-INFO file should use.
>
> Who would define such a standard?

PEP 376 where we can explain that all files in egg-info should be in a
specific encoding

>  My vote goes for “default is UTF-8”.

+1

>
>> Meanwhile, the 'register' command accepts Unicode, but is broken in
>> handling it. […]

how so ?

Tarek
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Should the io-c modules be put in their own directory?

[Antoine Pitrou]

Alexandre Vassalotti  peadrop.com> writes:
> 
> I just noticed that the new io-c modules were merged in the py3k
> branch (I know, I am kind late on the news—blame school work). Anyway,
> I am just wondering if it would be a good idea to put the io-c modules
> in a sub-directory (like sqlite), instead of scattering them around in
> the Modules/ directory.

Welcome back!

I have no particular opinion on this. I suggest waiting for Benjamin's advice
and following it :-)

(unless the FLUFL wants to chime in)

Benjamin-makes-boring-decisions-easy'ly yrs,

Antoine.


___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PyDict_SetItem hook

[Antoine Pitrou]

Thomas Wouters  python.org> writes:
> 
> 
> Pystone is pretty much a useless benchmark. If it measures anything, it's the
speed of the bytecode dispatcher (and it doesn't measure it particularly well.)
PyBench isn't any better, in my experience.

I don't think pybench is useless. It gives a lot of performance data about
crucial internal operations of the interpreter. It is of course very little
real-world, but conversely makes you know immediately where a performance
regression has happened. (by contrast, if you witness a regression in a
high-level benchmark, you still have a lot of investigation to do to find out
where exactly something bad happened)

Perhaps someone should start maintaining a suite of benchmarks, high-level and
low-level; we currently have them all scattered around (pybench, pystone,
stringbench, richard, iobench, and the various Unladen Swallow benchmarks; not
to mention other third-party stuff that can be found in e.g. the Computer
Language Shootout).

I also know Gregory P. Smith had emitted the idea of plotting benchmark figures
for each new revision of trunk or py3k (and, perhaps, other implementations),
but I don't know if he's willing to do it himself :-)

Regards

Antoine.


___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Let's update CObject API so it is safe and regular!

[Kristján Valur Jónsson]

Here's one from EVE, where the DB module creates raw data, for our Crowsets, 
and then hands it over to another module for consumption (actual creation of 
the CRow and CrowDescriptor objects:

BluePy raw(PyCObject_FromVoidPtr(&mColumnList, 0));
if (!raw)
return 0;
return PyObject_CallMethod(blueModule, "DBRowDescriptor", "O", raw.o);
This is done for performance reasons to avoid data duplication.  Of course it 
implies tight coupling of the modules.

In our FreeType wrapper system, we also use it to wrap pointers to FreeType 
structs:

template 
struct Wrapper : public T
{
...
PyObject *Wrap() {if (!sMap.size())Init(); return 
PyCObject_FromVoidPtrAndDesc(this, &sMap, 0);}
};

It is quite useful to pass unknown and opaque stuff around with, really, and 
makes certain things possible that otherwise wouldn't be.
We live with the type unsafety, of course.

In fact, I don't think we ever use a CObject to expose an API.

Kristj'an

-Original Message-
From: python-dev-bounces+kristjan=ccpgames@python.org 
[mailto:python-dev-bounces+kristjan=ccpgames@python.org] On Behalf Of Guido 
van Rossum
Sent: 2. apríl 2009 17:19
To: Jim Fulton
Cc: Python-Dev@python.org
Subject: Re: [Python-Dev] Let's update CObject API so it is safe and regular!

On Thu, Apr 2, 2009 at 6:22 AM, Jim Fulton  wrote:
> The original use case for CObjects was to export an API from a module, in
> which case, you'd be importing the API from the module.

I consider this the *only* use case. What other use cases are there?


___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

[Python-Dev] sequence slice that wraps, bug or intention?

[Ulrich Eckhardt]

Hi!

I just stumbled across something in Python 2.6 where I'm not sure if it is by 
design or a fault:

x = 'abdc'
x[-3:-3] -> ''
x[-3:-2] -> 'b'
x[-3:-1] -> 'bc'
x[-3: 0] -> ''

The one that actually bothers me here is the last one, I would have expected 
it to yield 'bcd' instead, because otherwise I don't see a way to specify a 
slice that starts with a negative index but still includes the last element.

Similarly, I would expect x[-1,1] to yield 'ca' or at least raise an error, 
but not to return an empty string.

Bug?

Uli

-- 
Sator Laser GmbH
Geschäftsführer: Thorsten Föcking, Amtsgericht Hamburg HR B62 932

**
Sator Laser GmbH, Fangdieckstraße 75a, 22547 Hamburg, Deutschland
Geschäftsführer: Thorsten Föcking, Amtsgericht Hamburg HR B62 932
**
   Visit our website at 
**
Diese E-Mail einschließlich sämtlicher Anhänge ist nur für den Adressaten 
bestimmt und kann vertrauliche Informationen enthalten. Bitte benachrichtigen 
Sie den Absender umgehend, falls Sie nicht der beabsichtigte Empfänger sein 
sollten. Die E-Mail ist in diesem Fall zu löschen und darf weder gelesen, 
weitergeleitet, veröffentlicht oder anderweitig benutzt werden.
E-Mails können durch Dritte gelesen werden und Viren sowie nichtautorisierte 
Änderungen enthalten. Sator Laser GmbH ist für diese Folgen nicht 
verantwortlich.
**

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] sequence slice that wraps, bug or intention?

[Paul Moore]

2009/4/3 Ulrich Eckhardt :
> Hi!
>
> I just stumbled across something in Python 2.6 where I'm not sure if it is by
> design or a fault:
>
> x = 'abdc'
> x[-3:-3] -> ''
> x[-3:-2] -> 'b'
> x[-3:-1] -> 'bc'
> x[-3: 0] -> ''
>
> The one that actually bothers me here is the last one, I would have expected
> it to yield 'bcd' instead, because otherwise I don't see a way to specify a
> slice that starts with a negative index but still includes the last element.
>
> Similarly, I would expect x[-1,1] to yield 'ca' or at least raise an error,
> but not to return an empty string.
>
> Bug?

Feature. Documented behaviour, even
(http://docs.python.org/reference/expressions.html#id5 section
"Slicings").

This question is more appropriate for python-list (comp.lang.python)
as it is about using Python, rather than the development of the Python
interpreter itself (although I can see that your uncertainty as to
whether this was a bug might have led you to think this was a more
appropriate list). You should first confirm on python-list that a
given behaviour is a bug, and if it is, post it to the tracker, rather
than to python-dev.

In this case, the behaviour is fine. As regards your point "I don't
see a way to specify a slice that starts with a negative index but
still includes the last element" what you want is x[-3:].

If you want to discuss this further, please do so on python-list.

Paul.
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Getting values stored inside sets

[Paul Moore]

2009/4/3 Hrvoje Niksic :
> I've stumbled upon an oddity using sets.  It's trivial to test if a value is
> in the set, but it appears to be impossible to retrieve a stored value,
> other than by iterating over the whole set.  Let me describe a concrete use
> case.
>
> Imagine a set of objects identified by some piece of information, such as a
> "key" slot (guaranteed to be constant for any particular element).  The
> object could look like this:
>
> class Element(object):
>    def __init__(self, key):
>        self.key = key
>    def __eq__(self, other):
>        return self.key == other
>    def __hash__(self):
>        return hash(self.key)
>    # ...
>
> Now imagine a set "s" of such objects.  I can add them to the set:
>
 s = set()
 s.add(Element('foo'))
 s.add(Element('bar'))
>
> I can test membership using the keys:
>
 'foo' in s
> True
 'blah' in s
> False
>
> But I can't seem to find a way to retrieve the element corresponding to
> 'foo', at least not without iterating over the entire set.  Is this an
> oversight or an intentional feature?  Or am I just missing an obvious way to
> do this?

My instinct is that it's intentional. I'd say that you're abusing
__eq__ here. If you can say "x in s" and then can't use x as if it
were the actual item inserted into s, then are they really "equal"?

Using a dict seems like the correct answer. I certainly don't think
it's worth complicating the set interface to cover this corner case.

Paul.
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

[Python-Dev] Getting values stored inside sets

[Hrvoje Niksic]

I've stumbled upon an oddity using sets.  It's trivial to test if a 
value is in the set, but it appears to be impossible to retrieve a 
stored value, other than by iterating over the whole set.  Let me 
describe a concrete use case.


Imagine a set of objects identified by some piece of information, such 
as a "key" slot (guaranteed to be constant for any particular element). 
 The object could look like this:


class Element(object):
def __init__(self, key):
self.key = key
def __eq__(self, other):
return self.key == other
def __hash__(self):
return hash(self.key)
# ...

Now imagine a set "s" of such objects.  I can add them to the set:

>>> s = set()
>>> s.add(Element('foo'))
>>> s.add(Element('bar'))

I can test membership using the keys:

>>> 'foo' in s
True
>>> 'blah' in s
False

But I can't seem to find a way to retrieve the element corresponding to 
'foo', at least not without iterating over the entire set.  Is this an 
oversight or an intentional feature?  Or am I just missing an obvious 
way to do this?


I know I can work around this by changing the set of elements to a dict 
that maps key -> element, but this feels unsatisfactory.  It's 
redundant, as the element already contains all the necessary 
information, and the set already knows how to use it, and the set must 
remember the original elements anyway, to be able to iterate over them, 
so why not allow one to retrieve them?  Secondly, the data structure I 
need conceptually *is* a set of elements, so it feels wrong to 
pigeonhole it into a dict.


This wasn't an isolated case, we stumbled on this several times while 
trying to use sets.  In comparison, STL sets don't have this limitation.


If this is not possible, I would like to propose either that set's 
__getitem__ translates key to value, so that s['foo'] would return the 
first element, or, if this is considered ugly, an equivalent method, 
such as s.get('foo').

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Getting values stored inside sets

[Terry Reedy]

Hrvoje Niksic wrote:
I've stumbled upon an oddity using sets.  It's trivial to test if a 
value is in the set, but it appears to be impossible to retrieve a 
stored value, 


Set elements, by definition, do not have keys or position by which to 
grab.  When they do, use a dict or list.


other than by iterating over the whole set.  Let me 
describe a concrete use case.


Imagine a set of objects identified by some piece of information, such 
as a "key" slot (guaranteed to be constant for any particular element). 
 The object could look like this:


class Element(object):
def __init__(self, key):
self.key = key
def __eq__(self, other):
return self.key == other
def __hash__(self):
return hash(self.key)
# ...

Now imagine a set "s" of such objects.  I can add them to the set:

 >>> s = set()
 >>> s.add(Element('foo'))
 >>> s.add(Element('bar'))

I can test membership using the keys:

 >>> 'foo' in s
True
 >>> 'blah' in s
False

But I can't seem to find a way to retrieve the element corresponding to 
'foo', at least not without iterating over the entire set.  Is this an 
oversight or an intentional feature?  Or am I just missing an obvious 
way to do this?


Use a dict, like you did.


I know I can work around this by changing the set of elements to a dict 
that maps key -> element, but this feels unsatisfactory.


Sorry, that is the right way.

>  It's
redundant, as the element already contains all the necessary 
information,


Records in a database have all the information of the record, but we 
still put out fields for indexes.


tjr

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] unittest package

[Olemis Lang]

On Thu, Apr 2, 2009 at 6:07 PM, Barry Warsaw  wrote:
> On Apr 2, 2009, at 4:58 PM, Michael Foord wrote:
>
>> The unittest module is around 1500 lines of code now, and the tests are
>> 3000 lines.
>>
>> It would be much easier to maintain as a package rather than a module.
>> Shall I work on a suggested structure or are there objections in principle?
>
> +1/jfdi :)
>

I remember that something like this was discussed some time ago ...
perhaps the ideas mentionned that time might be valuable ... AFAICR
somebody provided an example ... ;)

+1 for unittest as a package ...

BTW ... Q: Does it means that there will be subpkgs for specific (...
yet standard ...) pkgs ?

If this is the case, and there is a space for a unittest.doctest pkg
(... or whatever ... the name may be different ;) ... and inclusion is
Ok ... and so on ... I wonder ...

Q: Is it possible that dutest module [1]_ be considered ... to live in
stdlib ... ?

The module integrates doctest + unittest ... without needing a plugin
architecture or anything like that, just unittest + doctest... (... in
fact, sometimes I dont really get the idea for having plugins in
testing framews for what can be done following unittest philosophy ...
but anyway ... this is a long OT thread ... and I dont even think to
continue ... was just a brief comment ...)

Classes
=
- DocTestLoader allows to load (using unittest-style) TestCases which
check the match made for doctests. It provides integration with
TestProgram, supports building complex TestSuites in a more natural
way, and eases the use of specialized instances of TestCases built out
of doctest examples.

- A few classes so as to allow reporting the individual results of
each and every interactive example executed during the test run. A
separate entry is created in the corresponding TestResult instance
containing the expected value and the actual result.

- PackageTestLoader class (acting as a decorator ... design pattern ;)
loads all the tests found throughout a package hierarchy using another
loader . The later is used to retrieve the tests found in modules
matching a specified pattern.

- dutest.main is an alias for dutest.VerboseTestProgram. This class
fixes a minor bug (... IMO) I found while specifying different
verbosity levels from the command line to unittest.TestProgram.

These are the classes right now, but some others (e.g.
DocTestScriptLoader ... to load doctests out of test scripts ...)
might be helpful as well ... ;o)

Download from PyPI
===
dutest-0.2.2.win32.exe  MS Windows installer  any 76KB  28
dutest-0.2.2-py2.5.egg  Python Egg  2.5  17KB   93
dutest-0.2.2.zip  Sourceany   13KB
 47

PS: Random thoughts ...

.. [1] dutest 0.2.2
  (http://pypi.python.org/pypi/dutest)

.. [2] "Doctest and unittest... now they'll live happily together", O.
Lang (2008) The Python Papers, Volume 3, Issue 1, pp. 31:51
 (http://ojs.pythonpapers.org/index.php/tpp/article/view/56/51)

-- 
Regards,

Olemis.

Blog ES: http://simelo-es.blogspot.com/
Blog EN: http://simelo-en.blogspot.com/

Featured article:
No me gustan los templates de Django ...
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Package Management - thoughts from the peanut gallery

[Olemis Lang]

2009/4/3 Tarek Ziadé :
> Guys,
>
> The tasks discussed so far are:
>
> - version definition (http://wiki.python.org/moin/DistutilsVersionFight)
> - egg.info standardification (PEP 376)
> - metadata enhancement (rewrite PEP 345)
> - static metadata definition work  (*)

Looks fine ... and very useful ... ;)

> - creation of a network of OS packager people
> - PyPI mirroring (PEP 381)
>

Wow !

BTW ... I see nothing about removing dist_* commands from distutils ...

Q: Am I wrong or it seems they will remain in stdlib ?

-- 
Regards,

Olemis.

Blog ES: http://simelo-es.blogspot.com/
Blog EN: http://simelo-en.blogspot.com/

Featured article:
Comandos : Pipe Viewer ... ¿Qué está pasando por esta tubería?
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Package Management - thoughts from the peanut gallery

[Tarek Ziadé]

On Fri, Apr 3, 2009 at 2:56 PM, Olemis Lang  wrote:
>
> BTW ... I see nothing about removing dist_* commands from distutils ...
>
> Q: Am I wrong or it seems they will remain in stdlib ?

Ok, beware that what I am writing here is for the long term. There are no plans
yet to remove things right now. Maybe some things for 3.1, as long as
it is clearly
defined and non-controversial. And this is not the most urgent thing
to take care of.

So,
Some commands are not really used by the OS packagers, whether because
these commands don't provide what packagers need, whether because
they are unable to let the packagers configure them the way they would like to.

Packagers still need to tell us why and how to make things better. Some people
like Toshio or Matthias are already helping a lot on this. We are making
a lot of progress since the summit to share our point of views.

So I'd put this task under "creation of a network of OS packager
people" (them+others)

And in detail :

1/ define with them the precise usage of Distutils commands in each OS community

2/ define if there's a leading project that could take care of
building OS-dependant
   packages, using packages built by/with Distutils

4/ see what needs to be done in Distutils to let these projects play
with Python packages
   whithout pain.

5/ finally, see what could be externalized/removed from Distutils in
favor of these third-party projects.

This is roughly what Guido was talking about when he said we would
remove things like bdist_rpm
from the stdlib : it's too OS-specific for the stdlib to do a good job
in this area.

To discuss this plan in details, let's move to Distutils-SIG

Cheers
Tarek


--
Tarek Ziadé | Association AfPy | www.afpy.org
Blog FR | http://programmation-python.org
Blog EN | http://tarekziade.wordpress.com/
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Getting values stored inside sets

[Steven D'Aprano]

On Fri, 3 Apr 2009 11:22:02 pm Paul Moore wrote:

> I'd say that you're abusing __eq__ here. If you can say "x in s" 
> and then can't use x as if it were the actual item inserted into 
> s, then are they really "equal"? 

That's hardly unusual in Python.

>>> alist = [0, 1, 2, 3, 4]
>>> 3.0 in alist
True
>>> alist[3.0]
Traceback (most recent call last):
  File "", line 1, in 
TypeError: list indices must be integers


Besides, there's a concrete use-case for retrieving the actual object 
inside the set. You can ensure that you only have one instance of any 
object with a particular value by using a cache like this:

_cache = {}
def cache(obj):
if obj in _cache: return _cache[obj]
_cache[obj] = obj
return obj


Arguably, it would be neater if the cache was a set rather than a dict, 
thus saving one pointer per item, but of course that would rely on a 
change on set behaviour.



-- 
Steven D'Aprano
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Getting values stored inside sets

[Antoine Pitrou]

Steven D'Aprano  pearwood.info> writes:
> 
> That's hardly unusual in Python.
> 
> >>> alist = [0, 1, 2, 3, 4]
> >>> 3.0 in alist
> True
> >>> alist[3.0]
> Traceback (most recent call last):
>   File "", line 1, in 
> TypeError: list indices must be integers

Your example is wrong:

>>> alist = [0, 1, 2, 3, 4]
>>> alist.index(3.0)
3
>>> alist[alist.index(3.0)]
3

Regards

Antoine.


___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Getting values stored inside sets

[Steven D'Aprano]

On Sat, 4 Apr 2009 02:07:28 am Antoine Pitrou wrote:

> Your example is wrong:

Of course it is. The perils of posting at 2am, sorry.

Nevertheless, the principle still holds. There's nothing in Python that 
prohibits two objects from being equal, but without them being 
interchangeable. As poorly written as my example was, it still holds: I 
just need to add a level of indirection.

>>> alist = [100, 111, 102, 103, 105, 104, 106, 108]
>>> indices_of_odd_numbers = [alist.index(n) for n in alist if n%2]
>>> if Decimal('3') in indices_of_odd_numbers:
... print alist[Decimal('3')]
...
Traceback (most recent call last):
  File "", line 2, in 
TypeError: list indices must be integers


Python does not promise that if x == y, you can use y anywhere you can 
use x. Nor should it. Paul's declaration of abuse of __eq__ is 
unfounded.



-- 
Steven D'Aprano
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Getting values stored inside sets

[Sebastian Rittau]

Hello,

On Fri, Apr 03, 2009 at 02:07:02PM +0200, Hrvoje Niksic wrote:

> But I can't seem to find a way to retrieve the element corresponding to  
> 'foo', at least not without iterating over the entire set.  Is this an  
> oversight or an intentional feature?  Or am I just missing an obvious  
> way to do this?

I am missing a simple way to retrieve the "first" element of any
iterable in python that matches a certain condition anyway. Something
like this:

  def first(iter, cb):
  for el in iter:
  if cb(el):
  return el
  raise IndexError()

Or (shorter, but potentially slower):

  def first(iter, cb):
  return [el for el in iter if cb(el)][0]

To be used like this:

  my_el = first(my_set, lambda el: el == "foobar")

This is something I need from time to time and this also seems to solve
your problem.

 - Sebastian
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PyDict_SetItem hook

[Thomas Wouters]

On Fri, Apr 3, 2009 at 11:27, Antoine Pitrou  wrote:

> Thomas Wouters  python.org> writes:
> >
> >
> > Pystone is pretty much a useless benchmark. If it measures anything, it's
> the
> speed of the bytecode dispatcher (and it doesn't measure it particularly
> well.)
> PyBench isn't any better, in my experience.
>
> I don't think pybench is useless. It gives a lot of performance data about
> crucial internal operations of the interpreter. It is of course very little
> real-world, but conversely makes you know immediately where a performance
> regression has happened. (by contrast, if you witness a regression in a
> high-level benchmark, you still have a lot of investigation to do to find
> out
> where exactly something bad happened)


Really? Have you tried it? I get at least 5% noise between runs without any
changes. I have gotten results that include *negative* run times. And yes, I
tried all the different settings for calibration runs and timing mechanisms.
The tests in PyBench are not micro-benchmarks (they do way too much for
that), they don't try to minimize overhead or noise, but they are also not
representative of real-world code. That doesn't just mean "you can't infer
the affected operation from the test name", but "you can't infer anything."
You can just be looking at differently borrowed runtime. I have in the past
written patches to Python that improved *every* micro-benchmark and *every*
real-world measurement I made, except PyBench. Trying to pinpoint the
slowdown invariably lead to tests that did too much in the measurement loop,
introduced too much noise in the "calibration" run or just spent their time
*in the measurement loop* on doing setup and teardown of the test. Collin
and Jeffrey have seen the exact same thing since starting work on Unladen
Swallow.

So, sure, it might be "useful" if you have 10% or more difference across the
board, and if you don't have access to anything but pybench and pystone.


> Perhaps someone should start maintaining a suite of benchmarks, high-level
> and
> low-level; we currently have them all scattered around (pybench, pystone,
> stringbench, richard, iobench, and the various Unladen Swallow benchmarks;
> not
> to mention other third-party stuff that can be found in e.g. the Computer
> Language Shootout).


That's exactly what Collin proposed at the summits last week. Have you seen
http://code.google.com/p/unladen-swallow/wiki/Benchmarks
 ? Please feel free to suggest more benchmarks to add :)

-- 
Thomas Wouters 

Hi! I'm a .signature virus! copy me into your .signature file to help me
spread!
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Getting values stored inside sets

[Amaury Forgeot d'Arc]

Hi,

On Fri, Apr 3, 2009 at 17:45, Sebastian Rittau
 wrote:
> I am missing a simple way to retrieve the "first" element of any
> iterable in python that matches a certain condition anyway. Something
> like this:
>
>  def first(iter, cb):
>      for el in iter:
>          if cb(el):
>              return el
>      raise IndexError()
>
> Or (shorter, but potentially slower):
>
>  def first(iter, cb):
>      return [el for el in iter if cb(el)][0]
>
> To be used like this:
>
>  my_el = first(my_set, lambda el: el == "foobar")
>
> This is something I need from time to time and this also seems to solve
> your problem.

def first(iter, cb):
return itertools.ifilter(cb, iter).next()

-- 
Amaury Forgeot d'Arc
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] issue5578 - explanation

[Chris Withers]

Guido van Rossum wrote:

But anyways this is moot, the bug was only about exec in a class body
*nested inside a function*.

Indeed, I just hate seeing execs and it was an interesting mental exercise
to try and get rid of the above one ;-)

Assuming it breaks no tests, would there be objection to me committing the
above change to the Python 3 trunk?


That's up to Benjamin. Personally, I live by "if it ain't broke, don't
fix it." :-)


Anything using an exec is broken by definition ;-)

Benjamin?

cheers,

Chris

--
Simplistix - Content Management, Zope & Python Consulting
   - http://www.simplistix.co.uk
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Package Management - thoughts from the peanut gallery

[Olemis Lang]

On Fri, Apr 3, 2009 at 8:36 AM, Tarek Ziadé  wrote:
> On Fri, Apr 3, 2009 at 2:56 PM, Olemis Lang  wrote:
>>
>> BTW ... I see nothing about removing dist_* commands from distutils ...
>>
>> Q: Am I wrong or it seems they will remain in stdlib ?
>
> This is roughly what Guido was talking about when he said we would
> remove things like bdist_rpm
> from the stdlib : it's too OS-specific for the stdlib to do a good job
> in this area.
>
> To discuss this plan in details, let's move to Distutils-SIG
>

understood ... ;)

-- 
Regards,

Olemis.

Blog ES: http://simelo-es.blogspot.com/
Blog EN: http://simelo-en.blogspot.com/

Featured article:
Comandos : Pipe Viewer ... ¿Qué está pasando por esta tubería?
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PyDict_SetItem hook

[Collin Winter]

On Fri, Apr 3, 2009 at 2:27 AM, Antoine Pitrou  wrote:
> Thomas Wouters  python.org> writes:
>>
>>
>> Pystone is pretty much a useless benchmark. If it measures anything, it's the
> speed of the bytecode dispatcher (and it doesn't measure it particularly 
> well.)
> PyBench isn't any better, in my experience.
>
> I don't think pybench is useless. It gives a lot of performance data about
> crucial internal operations of the interpreter. It is of course very little
> real-world, but conversely makes you know immediately where a performance
> regression has happened. (by contrast, if you witness a regression in a
> high-level benchmark, you still have a lot of investigation to do to find out
> where exactly something bad happened)
>
> Perhaps someone should start maintaining a suite of benchmarks, high-level and
> low-level; we currently have them all scattered around (pybench, pystone,
> stringbench, richard, iobench, and the various Unladen Swallow benchmarks; not
> to mention other third-party stuff that can be found in e.g. the Computer
> Language Shootout).

Already in the works :)

As part of the common standard library and test suite that we agreed
on at the PyCon language summit last week, we're going to include a
common benchmark suite that all Python implementations can share. This
is still some months off, though, so there'll be plenty of time to
bikeshed^Wrationally discuss which benchmarks should go in there.

Collin
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Package Management - thoughts from the peanut gallery

[Chris Withers]

Tarek Ziadé wrote:

I have taken the commitment to lead these tasks and synchronize the people
that are willing to help on this.


Good, I'm one of those people, sadly my only help may be to ask "how is 
this bit going to be done?".



The tasks discussed so far are:

- version definition (http://wiki.python.org/moin/DistutilsVersionFight)
- egg.info standardification (PEP 376)
- metadata enhancement (rewrite PEP 345)
- static metadata definition work  (*)


These all seem to be a subset of the last one, right?


- creation of a network of OS packager people


This would be useful...


- PyPI mirroring (PEP 381)


I don't see why PyPI isn't just ported to GAE with an S3 data storage 
bit and be done with it... Offline mirrors for people behind firewalls 
already have solutions out there...



Each one of this task has a leader, except the one with (*). I just got back
from travelling, and I will reorganize
http://wiki.python.org/moin/Distutils asap to it is up-to-date.


Cool, is this the focal point to track your activities?


If you want to work on one of this task or feel there's a new task you can
start, please, join Distutils SIG or contact me,


Well, I think my "big list" breaks down roughly as tasks, of which I 
think the stuff you're already doing will hopefully take care of the 
first 2, but what about the rest. If labour shortage is all that's 
stopping this, then let me know ;-)


Chris

--
Simplistix - Content Management, Zope & Python Consulting
   - http://www.simplistix.co.uk
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PyDict_SetItem hook

[Martin v. Löwis]

> I think it's worse to give the poor guy the run around
> by making him run lots of random benchmarks.

"the poor guy" works for Wingware (a company you may have
heard of) and has contributed to Python at several occasions.
His name is John Ehresmann.

> In the end, someone will run a timeit or have a specific
> case that shows the full effect.  All of the respondents so far seem to
> have a clear intuition that hook is right in the middle of a critical
> path.  Their intuition matches what I learned by spending a month 
> trying to find ways to optimize dictionaries.

Ok, so add me as a respondent who thinks that this deserves to be
added despite being in the critical path. I doubt it will be noticeable
in practice.

> Am surprised that there has been no discussion of why this should be in
> the default build (as opposed to a compile time option).

Because, as a compile time option, it will be useless. It's not targeted
for people who want to work on the Python VM (who are the primary users
of compile time options), but for people developing Python applications.

> AFAICT, users have not previously requested a hook like this.

That's because debugging Python in general is in a sad state (which, in
turn, is because you can get very far with just print calls).

> Also, there has been no discussion for an overall strategy
> for monitoring containers in general.  Lists and tuples will
> both defy this approach because there is so much code
> that accesses the arrays directly. 

Dicts are special because they are used to implement namespaces.
Watchpoints is an incredibly useful debugging aid.

> Am not sure whether the
> setitem hook would work for other implementations either.

I can't see why it shouldn't.

> If my thoughts on the subject bug you, I'll happily
> withdraw from the thread.  I don't aspire to be a
> source of negativity.  I just happen to think this proposal isn't a good
> idea.

As somebody who has worked a lot on performance, I'm puzzled how
easily you judge a the performance impact of a patch without having
seen any benchmarks. If I have learned anything about performance, it
is this: never guess the performance aspects of code without
benchmarking.

Regards,
Martin
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Should the io-c modules be put in their own directory?

[Martin v. Löwis]

>> I just noticed that the new io-c modules were merged in the py3k
>> branch (I know, I am kind late on the news—blame school work). Anyway,
>> I am just wondering if it would be a good idea to put the io-c modules
>> in a sub-directory (like sqlite), instead of scattering them around in
>> the Modules/ directory.
> 
> Welcome back!
> 
> I have no particular opinion on this. I suggest waiting for Benjamin's advice
> and following it :-)

I would suggest to leave it as is:
a) never change a running system
b) flat is better than nested

Martin
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

[Python-Dev] Package Management - thoughts from the peanut gallery

[Olemis Lang]

On Fri, Apr 3, 2009 at 11:20 AM, Chris Withers  wrote:
> Tarek Ziadé wrote:
>
>> - PyPI mirroring (PEP 381)
>
> I don't see why PyPI isn't just ported to GAE with an S3 data storage bit
> and be done with it... Offline mirrors for people behind firewalls already
> have solutions out there...
>

-1 ... IMHO ...

--
Regards,

Olemis.

Blog ES: http://simelo-es.blogspot.com/
Blog EN: http://simelo-en.blogspot.com/

Featured article:
No me gustan los templates de Django ...

-- 
Regards,

Olemis.

Blog ES: http://simelo-es.blogspot.com/
Blog EN: http://simelo-en.blogspot.com/

Featured article:
Comandos : Pipe Viewer ... ¿Qué está pasando por esta tubería?
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Getting values stored inside sets

[Martin v. Löwis]

> I've stumbled upon an oddity using sets.  It's trivial to test if a
> value is in the set, but it appears to be impossible to retrieve a
> stored value, other than by iterating over the whole set. 

Of course it is. That's why it is called a set: it's an unordered
collection of objects, keyed by nothing.

If you have a set of elements, and you check "'foo' in s", then
you should be able just to use the string 'foo' itself for whatever
you want to do with it - you have essentially created a set of
strings. If you think that 'foo' and Element('foo') are different
things, you should not implement __eq__ in a way that they are
considered equal.

Regards,
Martin
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PyDict_SetItem hook

[Antoine Pitrou]

Thomas Wouters  python.org> writes:
> 
> Really? Have you tried it? I get at least 5% noise between runs without any
changes. I have gotten results that include *negative* run times.

That's an implementation problem, not an issue with the tests themselves.
Perhaps a better timing mechanism could be inspired from the timeit module.
Perhaps the default numbers of iterations should be higher (many subtests run
in less than 100ms on a modern CPU, which might be too low for accurate
measurement). Perhaps the so-called "calibration" should just be disabled.
etc.

> The tests in PyBench are not micro-benchmarks (they do way too much for
that),

Then I wonder what you call a micro-benchmark. Should it involve direct calls
to
low-level C API functions?

> but they are also not representative of real-world code.

Representativity is not black or white. Is measuring Spitfire performance
representative of the Genshi templating engine, or str.format-based templating?
Regardless of the answer, it is still an interesting measurement.

> That doesn't just mean "you can't infer the affected operation from the test
name"

I'm not sure what you mean by that. If you introduce an optimization to make
list comprehensions faster, it will certainly show up in the list
comprehensions subtest, and probably in none of the other tests. Isn't it enough
in terms of specificity?

Of course, some optimizations are interpreter-wide, and then the breakdown into
individual subtests is less relevant.

> I have in the past written patches to Python that improved *every*
micro-benchmark and *every* real-world measurement I made, except PyBench.

Well, I didn't claim that pybench measures /everything/. That's why we have
other benchmarks as well (stringbench, iobench, whatever).
It does test a bunch of very common operations which are important in daily use
of Python. If some important operation is missing, it's possible to add a new
test.

Conversely, someone optimizing e.g. list comprehensions and trying to measure
the impact using a set of so-called "real-world benchmarks" which don't involve
any list comprehension in their critical path will not see any improvement in
those "real-world benchmarks". Does it mean that the optimization is useless?
No, certainly not. The world is not black and white.

> That's exactly what Collin proposed at the summits last week. Have you seen
http://code.google.com/p/unladen-swallow/wiki/Benchmarks

Yes, I've seen. I haven't tried it, I hope it can be run without installing the
whole unladen-swallow suite?

These are the benchmarks I've had a tendency to use depending on the issue at
hand: pybench, richards, stringbench, iobench, binary-trees (from the Computer
Language Shootout). And various custom timeit runs :-)

Cheers

Antoine.


___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PyDict_SetItem hook

[John Ehresman]

Just want to reply quickly because I'm traveling -- I appreciate the 
feedback from Raymond and others.  Part of the reason I created an issue 
with a proof of concept patch is to get this kind of feedback.  I also 
agree that this shouldn't go in if it slows things down noticeably.


I will do some benchmarking and look at the dtrace patches next week to 
see if there is some sort of more systematic way of adding these types 
of hooks.


John
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Package Management - thoughts from the peanut gallery

[Chris Withers]

Olemis Lang wrote:

On Fri, Apr 3, 2009 at 11:20 AM, Chris Withers  wrote:

Tarek Ziadé wrote:


- PyPI mirroring (PEP 381)

I don't see why PyPI isn't just ported to GAE with an S3 data storage bit
and be done with it... Offline mirrors for people behind firewalls already
have solutions out there...


-1 ... IMHO ...


For what reason?

Chris

--
Simplistix - Content Management, Zope & Python Consulting
   - http://www.simplistix.co.uk
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Getting values stored inside sets

[Paul Moore]

2009/4/3 Steven D'Aprano :
> Python does not promise that if x == y, you can use y anywhere you can
> use x. Nor should it. Paul's declaration of abuse of __eq__ is
> unfounded.

Sorry, I was trying to simplify what I was saying, and simplified it
to the point where it didn't make sense :-) Martin (quoted below)
explained what I was trying to say far more clearly.

2009/4/3 "Martin v. Löwis" :
> If you have a set of elements, and you check "'foo' in s", then
> you should be able just to use the string 'foo' itself for whatever
> you want to do with it - you have essentially created a set of
> strings. If you think that 'foo' and Element('foo') are different
> things, you should not implement __eq__ in a way that they are
> considered equal.

-- in particular, if you're using things in sets (which are *all
about* equality, insofar as that's how "duplicates" are defined) you
should ensure that your definition of __eq__ respects the idea that
equal objects are duplicates (ie, interchangeable). Otherwise, a dict
is the appropriate data structure.

Actually, given the definition in the original post,

class Element(object):
   def __init__(self, key):
   self.key = key
   def __eq__(self, other):
   return self.key == other
   def __hash__(self):
   return hash(self.key)

as far as I can tell, equality is *only* defined between Elements and
keys - not even between 2 elements! So with that definition, there
could be many Elements in a set, all equal to the same key. Which is
completely insane.

In fact, Python seems to be doing something I don't understand:

>>> class Element(object):
...def __init__(self, key, id):
...self.key = key
...self.id = id
...def __eq__(self, other):
...print "Calling __eq__ for %s" % self.id
...return self.key == other
...def __hash__(self):
...return hash(self.key)
...
>>> a = Element('k', 'a')
>>> b = Element('k', 'b')
>>> a == b
Calling __eq__ for a
Calling __eq__ for b
True
>>> a == a
Calling __eq__ for a
Calling __eq__ for a
True
>>>

Why does __eq__ get called twice in these cases? Why does a == b, as
that means a.key == b, and clearly a.key ('k') does *not* equal b. Or
are there some further options being tried, in str,__eq__ or
object.__eq__? The documentation doesn't say so... Specifically,
there's nothing saying that a "reversed" version is tried.

Paul.
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PyDict_SetItem hook

[M.-A. Lemburg]

On 2009-04-03 18:06, Thomas Wouters wrote:
> On Fri, Apr 3, 2009 at 11:27, Antoine Pitrou  wrote:
> 
>> Thomas Wouters  python.org> writes:
>>>
>>> Pystone is pretty much a useless benchmark. If it measures anything, it's
>> the
>> speed of the bytecode dispatcher (and it doesn't measure it particularly
>> well.)
>> PyBench isn't any better, in my experience.
>>
>> I don't think pybench is useless. It gives a lot of performance data about
>> crucial internal operations of the interpreter. It is of course very little
>> real-world, but conversely makes you know immediately where a performance
>> regression has happened. (by contrast, if you witness a regression in a
>> high-level benchmark, you still have a lot of investigation to do to find
>> out
>> where exactly something bad happened)
> 
> 
> Really? Have you tried it? I get at least 5% noise between runs without any
> changes. I have gotten results that include *negative* run times. 

On which platform ? pybench 2.0 works reasonably well on Linux and
Windows, but of course can't do better than the timers available for
those platforms. If you have e.g. NTP running and it uses wall clock
timers, it is possible that you get negative round times. If you don't
and still get negative round times, you have to change the test
parameters (see below).

> And yes, I
> tried all the different settings for calibration runs and timing mechanisms.
> The tests in PyBench are not micro-benchmarks (they do way too much for
> that), they don't try to minimize overhead or noise,

That is not true. They were written as micro-benchmarks and adjusted
to have a high signal-noise ratio. For some operations this isn't easy
to do, but I certainly tried hard to get the overhead low (note that the
overhead is listed in the output).

That said, please keep in mind that the settings in pybench were last
adjusted some years ago to have the tests all run in more or less the
same wall clock time. CPUs have evolved a lot since then and this shows.

> but they are also not
> representative of real-world code.

True and they never were meant for that, since I was frustrated by
other benchmarks at the time and the whole approach in general.

Each of the tests checks one specific aspect of Python. If your
application happens to use a lot of dictionary operations, you'll
be mostly interested in those. If you do a lot of simple arithmetic,
there's another test for that.

On top of that the application is written to be easily extensible,
so it's easy to add new tests specific to whatever application space
you're after.

> That doesn't just mean "you can't infer
> the affected operation from the test name", but "you can't infer anything."
> You can just be looking at differently borrowed runtime. I have in the past
> written patches to Python that improved *every* micro-benchmark and *every*
> real-world measurement I made, except PyBench. Trying to pinpoint the
> slowdown invariably lead to tests that did too much in the measurement loop,
> introduced too much noise in the "calibration" run or just spent their time
> *in the measurement loop* on doing setup and teardown of the test. 

pybench calibrates itself to remove that kind of noise from the output.
Each test has a .calibrate() method which does all the setup and
tear down minus the actual benchmark operations.

If you get wrong numbers, try adjusting the parameters and add more
"packets" of operations. Don't forget to adjust the version number to
not compare apples and orange, though.

Perhaps it's time to readjust the pybench parameters to todays
CPUs.

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Apr 03 2009)
>>> Python/Zope Consulting and Support ...http://www.egenix.com/
>>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...http://python.egenix.com/

2009-03-19: Released mxODBC.Connect 1.0.1  http://python.egenix.com/

::: Try our new mxODBC.Connect Python Database Interface for free ! 


   eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
   Registered at Amtsgericht Duesseldorf: HRB 46611
   http://www.egenix.com/company/contact/
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Package Management - thoughts from the peanut gallery

[Tarek Ziadé]

On Fri, Apr 3, 2009 at 6:20 PM, Chris Withers  wrote:
> Tarek Ziadé wrote:
>>
>> I have taken the commitment to lead these tasks and synchronize the people
>> that are willing to help on this.
>
> Good, I'm one of those people,

Great !

>  sadly my only help may be to ask "how is this
> bit going to be done?".

I'll work on the wiki this week end for that

>
>> The tasks discussed so far are:
>>
>> - version definition (http://wiki.python.org/moin/DistutilsVersionFight)
>> - egg.info standardification (PEP 376)
>> - metadata enhancement (rewrite PEP 345)
>> - static metadata definition work  (*)
>
> These all seem to be a subset of the last one, right?

Sorry I used "task" I should have used "topics".

We are trying to have a list of well-defined, isolated tasks. Theses tasks
are built upon the discussions we have in these topics.

The last topic (static metadata) might generate new tasks and/or
complete existing tasks.

>> - PyPI mirroring (PEP 381)
>
> I don't see why PyPI isn't just ported to GAE with an S3 data storage bit
> and be done with it... Offline mirrors for people behind firewalls already
> have solutions out there...

GAE+S3 is just an implementation imho. We still need a mirroring protocol
ala CPAN and features in client softwares to use them. (as defined in 381)

>
>> Each one of this task has a leader, except the one with (*). I just got
>> back
>> from travelling, and I will reorganize
>> http://wiki.python.org/moin/Distutils asap to it is up-to-date.
>
> Cool, is this the focal point to track your activities?

Exactly. And Distutils-SIG is the mailing list to discuss in ;)

>
>> If you want to work on one of this task or feel there's a new task you can
>> start, please, join Distutils SIG or contact me,
>
> Well, I think my "big list" breaks down roughly as tasks, of which I think
> the stuff you're already doing will hopefully take care of the first 2, but
> what about the rest. If labour shortage is all that's stopping this, then
> let me know ;-)
>

Please discuss these new points in Distutils-SIG

Cheers
Tarek
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Package Management - thoughts from the peanut gallery

[Michael Foord]

Chris Withers wrote:

Olemis Lang wrote:
On Fri, Apr 3, 2009 at 11:20 AM, Chris Withers 
 wrote:

Tarek Ziadé wrote:


- PyPI mirroring (PEP 381)
I don't see why PyPI isn't just ported to GAE with an S3 data 
storage bit
and be done with it... Offline mirrors for people behind firewalls 
already

have solutions out there...


-1 ... IMHO ...


For what reason?


GAE does suffer from blackouts - which is the problem we are attempting 
to solve with mirroring.


I don't see why we should tie vital Python infrastructure to the 
proprietary APIs of a single vendor and outsource delivery entirely to 
them. If we have the manpower to do this ourselves it seems better to do 
it and retain control.


Added to which GAE is a commercial service and beyond a certain level 
bandwidth / cycles needs paying for. This may not be an issue in itself 
(either Google may waive charges or the PSF may be willing to pay).


Michael



Chris




--
http://www.ironpythoninaction.com/
http://www.voidspace.org.uk/blog


___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Getting values stored inside sets

[Raymond Hettinger]

Hrvoje Niksic wrote:
I've stumbled upon an oddity using sets.  It's trivial to test if a 
value is in the set, but it appears to be impossible to retrieve a 
stored value, 


See:  http://code.activestate.com/recipes/499299/


Raymond
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Getting values stored inside sets

[Alexander Belopolsky]

I just want to add a link to a 2.5 year old discussion on this  issue:
.  In that discussion I disagreed
with Martin and argued that "interning is a set
operation and it is unfortunate that set API does not support it directly."


On Fri, Apr 3, 2009 at 12:43 PM, "Martin v. Löwis"  wrote:
>> I've stumbled upon an oddity using sets.  It's trivial to test if a
>> value is in the set, but it appears to be impossible to retrieve a
>> stored value, other than by iterating over the whole set.
>
> Of course it is. That's why it is called a set: it's an unordered
> collection of objects, keyed by nothing.
>
> If you have a set of elements, and you check "'foo' in s", then
> you should be able just to use the string 'foo' itself for whatever
> you want to do with it - you have essentially created a set of
> strings. If you think that 'foo' and Element('foo') are different
> things, you should not implement __eq__ in a way that they are
> considered equal.
>
> Regards,
> Martin
> ___
> Python-Dev mailing list
> Python-Dev@python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: 
> http://mail.python.org/mailman/options/python-dev/alexander.belopolsky%40gmail.com
>
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PyDict_SetItem hook

[Collin Winter]

On Fri, Apr 3, 2009 at 9:43 AM, Antoine Pitrou  wrote:
> Thomas Wouters  python.org> writes:
>>
>> Really? Have you tried it? I get at least 5% noise between runs without any
> changes. I have gotten results that include *negative* run times.
>
> That's an implementation problem, not an issue with the tests themselves.
> Perhaps a better timing mechanism could be inspired from the timeit module.
> Perhaps the default numbers of iterations should be higher (many subtests run
> in less than 100ms on a modern CPU, which might be too low for accurate
> measurement). Perhaps the so-called "calibration" should just be disabled.
> etc.
>
>> The tests in PyBench are not micro-benchmarks (they do way too much for
> that),
>
> Then I wonder what you call a micro-benchmark. Should it involve direct calls
> to
> low-level C API functions?

I agree that a suite of microbenchmarks is supremely useful: I would
very much like to be able to isolate, say, raise statement
performance. PyBench suffers from implementation defects that in its
current incarnation make it unsuitable for this, though:
- It does not effectively isolate component performance as it claims.
When I was working on a change to BINARY_MODULO to make string
formatting faster, PyBench would report that floating point math got
slower, or that generator yields got slower. There is a lot of random
noise in the results.
- We have observed overall performance swings of 10-15% between runs
on the same machine, using the same Python binary. Using the same
binary on the same unloaded machine should give as close an answer to
0% as possible.
- I wish PyBench actually did more isolation.
Call.py:ComplexPythonFunctionCalls is on my mind right now; I wish it
didn't put keyword arguments and **kwargs in the same microbenchmark.
- In experimenting with gcc 4.4's FDO support, I produced a training
load that resulted in a 15-30% performance improvement (depending on
benchmark) across all benchmarks. Using this trained binary, PyBench
slowed down by 10%.
- I would like to see PyBench incorporate better statistics for
indicating the significance of the observed performance difference.

I don't believe that these are insurmountable problems, though. A
great contribution to Python performance work would be an improved
version of PyBench that corrects these problems and offers more
precise measurements. Is that something you might be interested in
contributing to? As performance moves more into the wider
consciousness, having good tools will become increasingly important.

Thanks,
Collin
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PyDict_SetItem hook

[Michael Foord]

Collin Winter wrote:

On Fri, Apr 3, 2009 at 2:27 AM, Antoine Pitrou  wrote:
  

Thomas Wouters  python.org> writes:


Pystone is pretty much a useless benchmark. If it measures anything, it's the
  

speed of the bytecode dispatcher (and it doesn't measure it particularly well.)
PyBench isn't any better, in my experience.

I don't think pybench is useless. It gives a lot of performance data about
crucial internal operations of the interpreter. It is of course very little
real-world, but conversely makes you know immediately where a performance
regression has happened. (by contrast, if you witness a regression in a
high-level benchmark, you still have a lot of investigation to do to find out
where exactly something bad happened)

Perhaps someone should start maintaining a suite of benchmarks, high-level and
low-level; we currently have them all scattered around (pybench, pystone,
stringbench, richard, iobench, and the various Unladen Swallow benchmarks; not
to mention other third-party stuff that can be found in e.g. the Computer
Language Shootout).



Already in the works :)

As part of the common standard library and test suite that we agreed
on at the PyCon language summit last week, we're going to include a
common benchmark suite that all Python implementations can share. This
is still some months off, though, so there'll be plenty of time to
bikeshed^Wrationally discuss which benchmarks should go in there.
  
Where is the right place for us to discuss this common benchmark and 
test suite?


As the benchmark is developed I would like to ensure it can run on 
IronPython.


The test suite changes will need some discussion as well - Jython and 
IronPython (and probably PyPy) have almost identical changes to tests 
that currently rely on deterministic finalisation (reference counting) 
so it makes sense to test changes on both platforms and commit a single 
solution.


Michael


Collin
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/fuzzyman%40voidspace.org.uk
  



--
http://www.ironpythoninaction.com/
http://www.voidspace.org.uk/blog


___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Getting values stored inside sets

[R. David Murray]

On Fri, 3 Apr 2009 at 17:57, Paul Moore wrote:

In fact, Python seems to be doing something I don't understand:


class Element(object):

...def __init__(self, key, id):
...self.key = key
...self.id = id
...def __eq__(self, other):
...print "Calling __eq__ for %s" % self.id
...return self.key == other
...def __hash__(self):
...return hash(self.key)
...

a = Element('k', 'a')
b = Element('k', 'b')
a == b

Calling __eq__ for a
Calling __eq__ for b
True

a == a

Calling __eq__ for a
Calling __eq__ for a
True




Why does __eq__ get called twice in these cases? Why does a == b, as
that means a.key == b, and clearly a.key ('k') does *not* equal b. Or
are there some further options being tried, in str,__eq__ or
object.__eq__? The documentation doesn't say so... Specifically,
there's nothing saying that a "reversed" version is tried.


a == b

So, python calls a.__eq__(b)

Now, that function does:

a.key == b

Since b is an object with an __eq__ method, python calls
b.__eq__(a.key).

That function does:

a.key == b.key

ie: the OP's code is inefficient :)

--David
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PyDict_SetItem hook

[Collin Winter]

On Fri, Apr 3, 2009 at 10:28 AM, Michael Foord
 wrote:
> Collin Winter wrote:
>> As part of the common standard library and test suite that we agreed
>> on at the PyCon language summit last week, we're going to include a
>> common benchmark suite that all Python implementations can share. This
>> is still some months off, though, so there'll be plenty of time to
>> bikeshed^Wrationally discuss which benchmarks should go in there.
>>
>
> Where is the right place for us to discuss this common benchmark and test
> suite?
>
> As the benchmark is developed I would like to ensure it can run on
> IronPython.
>
> The test suite changes will need some discussion as well - Jython and
> IronPython (and probably PyPy) have almost identical changes to tests that
> currently rely on deterministic finalisation (reference counting) so it
> makes sense to test changes on both platforms and commit a single solution.

I believe Brett Cannon is the best person to talk to about this kind
of thing. I don't know that any common mailing list has been set up,
though there may be and Brett just hasn't told anyone yet :)

Collin
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PyDict_SetItem hook

[Antoine Pitrou]

Collin Winter  gmail.com> writes:
> 
> - I wish PyBench actually did more isolation.
> Call.py:ComplexPythonFunctionCalls is on my mind right now; I wish it
> didn't put keyword arguments and **kwargs in the same microbenchmark.

Well, there is a balance to be found between having more subtests and keeping a
reasonable total running time :-)
(I have to plead guilty for ComplexPythonFunctionCalls, btw)

> - I would like to see PyBench incorporate better statistics for
> indicating the significance of the observed performance difference.

I see you already have this kind of measurement in your perf.py script, would it
be easy to port it?

We could also discuss making individual tests longer (by changing the default
"warp factor").


___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Getting values stored inside sets

[Paul Moore]

2009/4/3 R. David Murray :
> a == b
>
> So, python calls a.__eq__(b)
>
> Now, that function does:
>
> a.key == b
>
> Since b is an object with an __eq__ method, python calls
> b.__eq__(a.key).

That's the bit I can't actually find documented anywhere.

Ah, looking again I see that I misread the section describing the rich
comparison methods:

"""
There are no swapped-argument versions of these methods (to be used
when the left argument does not support the operation but the right
argument does); rather, __lt__() and __gt__() are each other’s
reflection, __le__() and __ge__() are each other’s reflection, and
__eq__() and __ne__() are their own reflection.
"""

I read that as meaning that no "reversed" version was called, whereas
it actually means that __eq__ is its own reversed version - and so
gets called both times.

Thanks for helping me clear that up!

Paul.
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PyDict_SetItem hook

[Michael Foord]

Collin Winter wrote:

On Fri, Apr 3, 2009 at 10:28 AM, Michael Foord
 wrote:
  

Collin Winter wrote:


As part of the common standard library and test suite that we agreed
on at the PyCon language summit last week, we're going to include a
common benchmark suite that all Python implementations can share. This
is still some months off, though, so there'll be plenty of time to
bikeshed^Wrationally discuss which benchmarks should go in there.

  

Where is the right place for us to discuss this common benchmark and test
suite?

As the benchmark is developed I would like to ensure it can run on
IronPython.

The test suite changes will need some discussion as well - Jython and
IronPython (and probably PyPy) have almost identical changes to tests that
currently rely on deterministic finalisation (reference counting) so it
makes sense to test changes on both platforms and commit a single solution.



I believe Brett Cannon is the best person to talk to about this kind
of thing. I don't know that any common mailing list has been set up,
though there may be and Brett just hasn't told anyone yet :)

Collin
  

Which begs the question of whether we *should* have a separate mailing list.

I don't think we discussed this specific point in the language summit - 
although it makes sense. Should we have a list specifically for the test 
/ benchmarking or would a more general implementations-sig be appropriate?


And is it really Brett who sets up mailing lists? My understanding is 
that he is pulling out of stuff for a while anyway, so that he can do 
Java / Phd type things... ;-)


Michael

--
http://www.ironpythoninaction.com/
http://www.voidspace.org.uk/blog


___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PyDict_SetItem hook

[Collin Winter]

On Fri, Apr 3, 2009 at 10:50 AM, Antoine Pitrou  wrote:
> Collin Winter  gmail.com> writes:
>>
>> - I wish PyBench actually did more isolation.
>> Call.py:ComplexPythonFunctionCalls is on my mind right now; I wish it
>> didn't put keyword arguments and **kwargs in the same microbenchmark.
>
> Well, there is a balance to be found between having more subtests and keeping 
> a
> reasonable total running time :-)
> (I have to plead guilty for ComplexPythonFunctionCalls, btw)

Sure, there's definitely a balance to maintain. With perf.py, we're
going down the road of having different tiers of benchmarks: the
default set is the one we pay the most attention to, with other
benchmarks available for benchmarking certain specific subsystems or
workloads (like pickling list-heavy input data). Something similar
could be done for PyBench, giving the user the option of increasing
the level of detail (and run-time) as appropriate.

>> - I would like to see PyBench incorporate better statistics for
>> indicating the significance of the observed performance difference.
>
> I see you already have this kind of measurement in your perf.py script, would 
> it
> be easy to port it?

Yes, it should be straightforward to incorporate these statistics into
PyBench. In the same directory as perf.py, you'll find test_perf.py
which includes tests for the stats functions we're using.

Collin
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] issue5578 - explanation

[Steve Holden]

Chris Withers wrote:
> Guido van Rossum wrote:
 But anyways this is moot, the bug was only about exec in a class body
 *nested inside a function*.
>>> Indeed, I just hate seeing execs and it was an interesting mental
>>> exercise
>>> to try and get rid of the above one ;-)
>>>
>>> Assuming it breaks no tests, would there be objection to me
>>> committing the
>>> above change to the Python 3 trunk?
>>
>> That's up to Benjamin. Personally, I live by "if it ain't broke, don't
>> fix it." :-)
> 
> Anything using an exec 

that can be done in some other (more pythonic way)

> is broken by definition ;-)
> 
> Benjamin?
> 
We've just had a fairly clear demonstration that small semantic changes
to the language can leave unexpected areas borked.

regards
 Steve
-- 
Steve Holden   +1 571 484 6266   +1 800 494 3119
Holden Web LLC http://www.holdenweb.com/
Want to know? Come to PyCon - soon! http://us.pycon.org/

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 382: Namespace Packages

[Martin v. Löwis]

> Perhaps we could add something like a sys.namespace_packages that would
> be updated by this mechanism?  Then, pkg_resources could check both that
> and its internal registry to be both backward and forward compatible.

I could see no problem with that, so I have added this to the PEP.

Thanks for the feedback,

Martin
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 382: Namespace Packages

[Martin v. Löwis]

Chris Withers wrote:
> Martin v. Löwis wrote:
>> I propose the following PEP for inclusion to Python 3.1.
>> Please comment.
> 
> Would this support the following case:
> 
> I have a package called mortar, which defines useful stuff:
> 
> from mortar import content, ...
> 
> I now want to distribute large optional chunks separately, but ideally
> so that the following will will work:
> 
> from mortar.rbd import ...
> from mortar.zodb import ...
> from mortar.wsgi import ...
> 
> Does the PEP support this? 

That's the primary purpose of the PEP. You can do this today already
(see the zope package, and the reference to current techniques in the
PEP), but the PEP provides a cleaner way.

In each chunk (which the PEP calls portion), you had a structure like
this:

mortar/
mortar/rbd.pkg (contains just "*")
mortar/rbd.py

or

mortar/
mortar/zobd.pkg
mortar/zobd/
mortar/zobd/__init__.py
mortar/zobd/backends.py

As a site effect, you can also do "import mortar", but that would just
give you the (nearly) empty namespace package, whose only significant
contents is the variable __path__.

Regards,
Martin
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 382: Namespace Packages

[Martin v. Löwis]

> I'd like to extend the proposal to Python 2.7 and later.

I don't object, but I also don't want to propose this, so
I added it to the discussion.

My (and perhaps other people's) concern is that 2.7 might
well be the last release of the 2.x series. If so, adding
this feature to it would make 2.7 an odd special case for
users and providers of third party tools.

> That's going to slow down Python package detection a lot - you'd
> replace an O(1) test with an O(n) scan.

I question that claim. In traditional Unix systems, the file system
driver performs a linear search of the directory, so it's rather
O(n)-in-kernel vs. O(n)-in-Python. Even for advanced file systems,
you need at least O(log n) to determine whether a specific file is
in a directory. For all practical purposes, the package directory
will fit in a single disk block (containing a single .pkg file, and
one or few subpackages), making listdir complete as fast as stat.

> Wouldn't it be better to stick with a simpler approach and look for
> "__pkg__.py" files to detect namespace packages using that O(1) check ?

Again - this wouldn't be O(1). More importantly, it breaks system
packages, which now again have to deal with the conflicting file names
if they want to install all portions into a single location.

> This would also avoid any issues you'd otherwise run into if you want
> to maintain this scheme in an importer that doesn't have access to a list
> of files in a package directory, but is well capable for the checking
> the existence of a file.

Do you have a specific mechanism in mind?

Regards,
Martin

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 382: Namespace Packages

[Martin v. Löwis]

> Note that there is no such thing as a "defining namespace package" --
> namespace package contents are symmetrical peers.

With the PEP, a "defining package" becomes possible - at most one
portion can define an __init__.py.

I know that the current mechanisms don't support it, and it might
not be useful in general, but now there is a clean way of doing it,
so I wouldn't exclude it. Distribution-wise, all distributions
relying on the defining package would need to require (or
install_require, or depend on) it.

> The above are also true for using only a '*' in .pkg files -- in that
> event there are no sys.path changes.  (Frankly, I'm doubtful that
> anybody is using extend_path and .pkg files to begin with, so I'd be
> fine with a proposal that instead used something like '.nsp' files that
> didn't even need to be opened and read -- which would let the directory
> scan stop at the first .nsp file found.

That would work for me as well. Nobody at PyCon could remember where
.pkg files came from.

> I believe the PEP does this as well, IIUC.

Correct.

>> * It's possible to have a defining package dir and add-one package
>> dirs.
> 
> Also possible in the PEP, although the __init__.py must be in the first
> such directory on sys.path.

I should make it clear that this is not the case. I envision it to work
this way: import zope
- searches sys.path, until finding either a directory zope, or a file
  zope.{py,pyc,pyd,...}
- if it is a directory, it checks for .pkg files. If it finds any,
  it processes them, extending __path__.
- it *then* checks for __init__.py, taking the first hit anywhere
  on __path__ (just like any module import would)
- if no .pkg was found, nor an __init__.py, it proceeds with the next
  sys.path item (skipping the directory entirely)

Regards,
Martin
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PyDict_SetItem hook

[Jeffrey Yasskin]

On Fri, Apr 3, 2009 at 12:28 PM, Michael Foord
 wrote:
> Collin Winter wrote:
>>
>> On Fri, Apr 3, 2009 at 2:27 AM, Antoine Pitrou 
>> wrote:
>>
>>>
>>> Thomas Wouters  python.org> writes:
>>>

 Pystone is pretty much a useless benchmark. If it measures anything,
 it's the

>>>
>>> speed of the bytecode dispatcher (and it doesn't measure it particularly
>>> well.)
>>> PyBench isn't any better, in my experience.
>>>
>>> I don't think pybench is useless. It gives a lot of performance data
>>> about
>>> crucial internal operations of the interpreter. It is of course very
>>> little
>>> real-world, but conversely makes you know immediately where a performance
>>> regression has happened. (by contrast, if you witness a regression in a
>>> high-level benchmark, you still have a lot of investigation to do to find
>>> out
>>> where exactly something bad happened)
>>>
>>> Perhaps someone should start maintaining a suite of benchmarks,
>>> high-level and
>>> low-level; we currently have them all scattered around (pybench, pystone,
>>> stringbench, richard, iobench, and the various Unladen Swallow
>>> benchmarks; not
>>> to mention other third-party stuff that can be found in e.g. the Computer
>>> Language Shootout).
>>>
>>
>> Already in the works :)
>>
>> As part of the common standard library and test suite that we agreed
>> on at the PyCon language summit last week, we're going to include a
>> common benchmark suite that all Python implementations can share. This
>> is still some months off, though, so there'll be plenty of time to
>> bikeshed^Wrationally discuss which benchmarks should go in there.
>>
>
> Where is the right place for us to discuss this common benchmark and test
> suite?

Dunno. Here, by default, but I'd subscribe to a tests-sig or
commonlibrary-sig or benchmark-sig if one were created.

> As the benchmark is developed I would like to ensure it can run on
> IronPython.

We want to ensure the same thing for the current unladen swallow
suite. If you find ways it currently doesn't, send us patches (until
we get it moved to the common library repository at which point you'll
be able to submit changes yourself). You should be able to check out
http://unladen-swallow.googlecode.com/svn/tests independently of the
rest of the repository. Follow the instructions at
http://code.google.com/p/unladen-swallow/wiki/Benchmarks to run
benchmarks though perf.py. You'll probably want to select benchmarks
individually rather than accepting the default of "all" because it's
currently not very resilient to tests that don't run on one of the
comparison pythons.

Personally, I'd be quite happy moving our performance tests into the
main python repository before the big library+tests move, but I don't
know what directory to put it in, and I don't know what Collin+Thomas
think of that.

> The test suite changes will need some discussion as well - Jython and
> IronPython (and probably PyPy) have almost identical changes to tests that
> currently rely on deterministic finalisation (reference counting) so it
> makes sense to test changes on both platforms and commit a single solution.

IMHO, any place in the test suite that relies on deterministic
finalization but isn't explicitly testing that CPython-specific
feature is a bug and should be fixed, even before we export it to the
new repository.

Jeffrey
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PyDict_SetItem hook

[Jeffrey Yasskin]

On Thu, Apr 2, 2009 at 5:57 PM, Guido van Rossum  wrote:
> On Thu, Apr 2, 2009 at 3:07 PM, Raymond Hettinger  wrote:
>>> Wow. Can you possibly be more negative?
>>
>> I think it's worse to give the poor guy the run around
>
> Mind your words please.
>
>> by making him run lots of random benchmarks.  In
>> the end, someone will run a timeit or have a specific
>> case that shows the full effect.  All of the respondents so far seem to have
>> a clear intuition that hook is right in the middle of a critical path.
>>  Their intuition matches
>> what I learned by spending a month trying to find ways
>> to optimize dictionaries.
>>
>> Am surprised that there has been no discussion of why this should be in the
>> default build (as opposed to a compile time option).  AFAICT, users have not
>> previously
>> requested a hook like this.
>
> I may be partially to blame for this. John and Stephan are requesting
> this because it would (mostly) fulfill one of the top wishes of the
> users of Wingware. So the use case is certainly real.
>
>> Also, there has been no discussion for an overall strategy
>> for monitoring containers in general.  Lists and tuples will
>> both defy this approach because there is so much code
>> that accesses the arrays directly.  Am not sure whether the
>> setitem hook would work for other implementations either.
>
> The primary use case is some kind of trap on assignment. While this
> cannot cover all cases, most non-local variables are stored in dicts.
> List mutations are not in the same league, as use case.
>
>> It seems weird to me that Collin's group can be working
>> so hard just to get a percent or two improvement in specific cases for
>> pickling while python-dev is readily entertaining a patch that slows down
>> the entire language.
>
> I don't actually believe that you can know whether this affects
> performance at all without serious benchmarking. The patch amounts to
> a single global flag check as long as the feature is disabled, and
> that flag could be read from the L1 cache.

When I was optimizing the tracing support in the eval loop, we started
with two memory loads and an if test. Removing the whole thing saved
about 3% of runtime, although I think that had been as high as 5% when
Neal measured it a year before. (That indicates that the exact
arrangement of the code can affect performance in subtle and annoying
ways.) Removing one of the two loads saved about 2% of runtime. I
don't remember exactly which benchmark that was; it may just have been
pybench.

Here, we're talking about introducing a load+if in dicts, which is
less critical than the eval loop, so I'd guess that the effect will be
less than 2% overall. I do think the real-life benchmarks are worth
getting for this, but they may not predict the effect after other code
changes. And I don't really have an opinion on what performance hit
for normal use is worth better debugging.

>> If my thoughts on the subject bug you, I'll happily
>> withdraw from the thread.  I don't aspire to be a
>> source of negativity.  I just happen to think this proposal isn't a good
>> idea.
>
> I think we need more proof either way.
>
>> Raymond
>>
>>
>>
>> - Original Message - From: "Guido van Rossum" 
>> To: "Raymond Hettinger" 
>> Cc: "Thomas Wouters" ; "John Ehresman"
>> ; 
>> Sent: Thursday, April 02, 2009 2:19 PM
>> Subject: Re: [Python-Dev] PyDict_SetItem hook
>>
>>
>> Wow. Can you possibly be more negative?
>>
>> 2009/4/2 Raymond Hettinger :
>>>
>>> The measurements are just a distractor. We all already know that the hook
>>> is being added to a critical path. Everyone will pay a cost for a feature
>>> that few people will use. This is a really bad idea. It is not part of a
>>> thorough, thought-out framework of container hooks (something that would
>>> need a PEP at the very least). The case for how it helps us is somewhat
>>> thin. The case for DTrace hooks was much stronger.
>>>
>>> If something does go in, it should be #ifdef'd out by default. But then, I
>>> don't think it should go in at all.
>>>
>>>
>>> Raymond
>>>
>>>
>>>
>>>
>>> On Thu, Apr 2, 2009 at 04:16, John Ehresman  wrote:

 Collin Winter wrote:
>
> Have you measured the impact on performance?

 I've tried to test using pystone, but am seeing more differences between
 runs than there is between python w/ the patch and w/o when there is no
 hook
 installed. The highest pystone is actually from the binary w/ the patch,
 which I don't really believe unless it's some low level code generation
 affect. The cost is one test of a global variable and then a switch to
 the
 branch that doesn't call the hooks.

 I'd be happy to try to come up with better numbers next week after I get
 home from pycon.
>>>
>>> Pystone is pretty much a useless benchmark. If it measures anything, it's
>>> the speed of the bytecode dispatcher (and it doesn't measure it
>>> particularly
>>> well.) PyBench isn't any better, in my 

Re: [Python-Dev] PEP 382: Namespace Packages

[glyph]

On 08:15 pm, mar...@v.loewis.de wrote:

Note that there is no such thing as a "defining namespace package" --
namespace package contents are symmetrical peers.


With the PEP, a "defining package" becomes possible - at most one
portion can define an __init__.py.


For what it's worth, this is a _super_ useful feature for Twisted.  We 
have one "defining package" for the "twisted" package (twisted core) and 
then a bunch of other things which want to put things into twisted.* 
(twisted.web, twisted.conch, et. al.).


For debian we already have separate packages, but such a definition of 
namespace packages would allow us to actually have things separated out 
on the cheeseshop as well.

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Should the io-c modules be put in their own directory?

[Benjamin Peterson]

2009/4/3 Antoine Pitrou :
> Alexandre Vassalotti  peadrop.com> writes:
>>
>> I just noticed that the new io-c modules were merged in the py3k
>> branch (I know, I am kind late on the news—blame school work). Anyway,
>> I am just wondering if it would be a good idea to put the io-c modules
>> in a sub-directory (like sqlite), instead of scattering them around in
>> the Modules/ directory.
>
> Welcome back!
>
> I have no particular opinion on this. I suggest waiting for Benjamin's advice
> and following it :-)

I'm +.2. This is the layout I would suggest:

Modules/
  _io/
 _io.c
 stringio.c
 textio.c
 etc

>
> (unless the FLUFL wants to chime in)
>
> Benjamin-makes-boring-decisions-easy'ly yrs,
>
> Antoine.


mad-with-power'ly yours,
Benjamin
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Should the io-c modules be put in their own directory?

[Benjamin Peterson]

2009/4/3 "Martin v. Löwis" :
>>> I just noticed that the new io-c modules were merged in the py3k
>>> branch (I know, I am kind late on the news—blame school work). Anyway,
>>> I am just wondering if it would be a good idea to put the io-c modules
>>> in a sub-directory (like sqlite), instead of scattering them around in
>>> the Modules/ directory.
>>
>> Welcome back!
>>
>> I have no particular opinion on this. I suggest waiting for Benjamin's advice
>> and following it :-)
>
> I would suggest to leave it as is:
> a) never change a running system
> b) flat is better than nested

It doesn't make sense, though, to have the 8 files that make up the
_io module scattered around in a directory with scores of other ones.



-- 
Regards,
Benjamin
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 382: Namespace Packages

[P.J. Eby]

At 10:15 PM 4/3/2009 +0200, Martin v. Löwis wrote:

I should make it clear that this is not the case. I envision it to work
this way: import zope
- searches sys.path, until finding either a directory zope, or a file
  zope.{py,pyc,pyd,...}
- if it is a directory, it checks for .pkg files. If it finds any,
  it processes them, extending __path__.
- it *then* checks for __init__.py, taking the first hit anywhere
  on __path__ (just like any module import would)
- if no .pkg was found, nor an __init__.py, it proceeds with the next
  sys.path item (skipping the directory entirely)


Ah, I missed that.  Maybe the above should be added to the PEP to clarify.

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] issue5578 - explanation

[Benjamin Peterson]

2009/4/3 Chris Withers :
> Guido van Rossum wrote:

 But anyways this is moot, the bug was only about exec in a class body
 *nested inside a function*.
>>>
>>> Indeed, I just hate seeing execs and it was an interesting mental
>>> exercise
>>> to try and get rid of the above one ;-)
>>>
>>> Assuming it breaks no tests, would there be objection to me committing
>>> the
>>> above change to the Python 3 trunk?
>>
>> That's up to Benjamin. Personally, I live by "if it ain't broke, don't
>> fix it." :-)
>
> Anything using an exec is broken by definition ;-)

"practicality beats purity"

>
> Benjamin?

+0


-- 
Regards,
Benjamin
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Should the io-c modules be put in their own directory?

[Guido van Rossum]

On Fri, Apr 3, 2009 at 2:15 PM, Benjamin Peterson  wrote:
> 2009/4/3 "Martin v. Löwis" :
 I just noticed that the new io-c modules were merged in the py3k
 branch (I know, I am kind late on the news—blame school work). Anyway,
 I am just wondering if it would be a good idea to put the io-c modules
 in a sub-directory (like sqlite), instead of scattering them around in
 the Modules/ directory.
>>>
>>> Welcome back!
>>>
>>> I have no particular opinion on this. I suggest waiting for Benjamin's 
>>> advice
>>> and following it :-)
>>
>> I would suggest to leave it as is:
>> a) never change a running system
>> b) flat is better than nested
>
> It doesn't make sense, though, to have the 8 files that make up the
> _io module scattered around in a directory with scores of other ones.

I think Benjamin is right. While most of the C source is indeed
exactly one level below the root, there's plenty of code that isn't,
e.g. _ctypes, cjkcodecs, expat, _multiprocessing, zlib. And even
Objects/stringlib.

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Integrate BeautifulSoup into stdlib?

[Jan Claeys]

Op dinsdag 24-03-2009 om 20:51 uur [tijdzone +0100], schreef "Martin v.
Löwis":
> The Windows story is indeed sad, as none of the Windows packaging
> formats provides support for dependencies

That's not entirely true; Cygwin comes with a package management tool
that probably could be used to set up a repository of python packages
for native Windows: 

This package manager is in no way dependent on Cygwin, supports (basic)
dependencies, etc.  Of course some people would have to take care of the
packaging work (just like happens for most open source OS distros and
for Mac OS X already).

It seems like XEmacs is already using a fork of that installer for the
same purpose.


-- 
Jan Claeys

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] issue5578 - explanation

[Alexandre Vassalotti]

On Tue, Mar 31, 2009 at 11:25 PM, Guido van Rossum  wrote:
> Well hold on for a minute, I remember we used to have an exec
> statement in a class body in the standard library, to define some file
> methods in socket.py IIRC.

FYI, collections.namedtuple is also implemented using exec.

- Alexandre
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Getting values stored inside sets

[Leif Walsh]

On Fri, Apr 3, 2009 at 8:07 AM, Hrvoje Niksic  wrote:
> But I can't seem to find a way to retrieve the element corresponding to
> 'foo', at least not without iterating over the entire set.  Is this an
> oversight or an intentional feature?  Or am I just missing an obvious way to
> do this?

>>> query_obj in s
True
>>> s_prime = s.copy()
>>> s_prime.discard(query_obj)
>>> x = s.difference(s_prime).pop()

Pretty ugly, but I think it only uses a shallow copy, and it might be
a bit better than iterating, if difference is intelligent.  I haven't
run any tests though.

-- 
Cheers,
Leif
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 382: Namespace Packages

[Brett Cannon]

On Fri, Apr 3, 2009 at 13:15, "Martin v. Löwis"  wrote:

> > Note that there is no such thing as a "defining namespace package" --
> > namespace package contents are symmetrical peers.
>
> With the PEP, a "defining package" becomes possible - at most one
> portion can define an __init__.py.
>
> I know that the current mechanisms don't support it, and it might
> not be useful in general, but now there is a clean way of doing it,
> so I wouldn't exclude it. Distribution-wise, all distributions
> relying on the defining package would need to require (or
> install_require, or depend on) it.
>
> > The above are also true for using only a '*' in .pkg files -- in that
> > event there are no sys.path changes.  (Frankly, I'm doubtful that
> > anybody is using extend_path and .pkg files to begin with, so I'd be
> > fine with a proposal that instead used something like '.nsp' files that
> > didn't even need to be opened and read -- which would let the directory
> > scan stop at the first .nsp file found.
>
> That would work for me as well. Nobody at PyCon could remember where
> .pkg files came from.
>
> > I believe the PEP does this as well, IIUC.
>
> Correct.
>
> >> * It's possible to have a defining package dir and add-one package
> >> dirs.
> >
> > Also possible in the PEP, although the __init__.py must be in the first
> > such directory on sys.path.
>
> I should make it clear that this is not the case. I envision it to work
> this way: import zope
> - searches sys.path, until finding either a directory zope, or a file
>  zope.{py,pyc,pyd,...}
> - if it is a directory, it checks for .pkg files. If it finds any,
>  it processes them, extending __path__.
> - it *then* checks for __init__.py, taking the first hit anywhere
>  on __path__ (just like any module import would)


Just so people know how this __init__ search could be done such that
__path__ is set from the .pkg is to treat it as a reload (assuming .pkg
files can only be found off of sys.path).


-Brett


> - if no .pkg was found, nor an __init__.py, it proceeds with the next
>  sys.path item (skipping the directory entirely)
>
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] And the winner is...

[Jan Claeys]

Op maandag 30-03-2009 om 21:54 uur [tijdzone -0500], schreef Guido van
Rossum:
> But is his humility enough to cancel out Linus's attitude?

I hope not, or the /.-crowd would become desperate...   ;-)


-- 
Jan Claeys

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Should the io-c modules be put in their own directory?

[Alexandre Vassalotti]

On Fri, Apr 3, 2009 at 5:12 PM, Benjamin Peterson  wrote:
> I'm +.2. This is the layout I would suggest:
>
> Modules/
>  _io/
>     _io.c
>     stringio.c
>     textio.c
>     etc
>

That seems good to me. I opened an issue on the tracker and included a patch.

http://bugs.python.org/issue5682

-- Alexandre
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

[Python-Dev] Going "offline" for three months

[Brett Cannon]

In order to hunker down and get my thesis proposal done by its due date, I
am disabling mail delivery for myself for all mail.python.org mailing lists
for three months (sans python-committers so I don't accidentally commit when
I shouldn't). If something comes up I should know about you can always email
or IM me directly.

See you all on July 1. Here is to hoping I don't suffer any withdrawal.

-Brett
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Getting values stored inside sets

[Nick Coghlan]

Paul Moore wrote:
> 2009/4/3 R. David Murray :
>> a == b
>>
>> So, python calls a.__eq__(b)
>>
>> Now, that function does:
>>
>> a.key == b
>>
>> Since b is an object with an __eq__ method, python calls
>> b.__eq__(a.key).
> 
> That's the bit I can't actually find documented anywhere.

It doesn't quite work the way RDM desribed it - he missed a step.

a == b

So, python calls a.__eq__(b)

Now, that function does:

a.key == b

which first calls a.key.__eq__(b) # This step was missing

Since str has no idea what an Element is, that returns NotImplemented.

Since __eq__ is defined as being commutative, the interpreter then tries
b.__eq__(a.key).

That function does:

b.key == a.key

which calls b.key.__eq__(a.key)

which is a well defined string comparison and returns the expected answer.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncogh...@gmail.com   |   Brisbane, Australia
---
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] issue5578 - explanation

[Martin v. Löwis]

Alexandre Vassalotti wrote:
> On Tue, Mar 31, 2009 at 11:25 PM, Guido van Rossum  wrote:
>> Well hold on for a minute, I remember we used to have an exec
>> statement in a class body in the standard library, to define some file
>> methods in socket.py IIRC.
> 
> FYI, collections.namedtuple is also implemented using exec.

Ah, but it uses "exec ... in ...". That is much safer than an
unqualified exec (where the issue is what namespace it executes in,
and, consequentially, what early binding is possible).

The patch bans only unqualified exec, IIUC.

Regards,
Martin
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Integrate BeautifulSoup into stdlib?

[Martin v. Löwis]

> That's not entirely true; Cygwin comes with a package management tool
> that probably could be used to set up a repository of python packages
> for native Windows: 

Ah, ok. It has the big disadvantage of not being Microsoft-endorsed,
though. In that sense, it feels very much like easy_install (which also
does dependencies).

Regards,
Martin
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Going "offline" for three months

[Ben Finney]

Brett Cannon  writes:

> See you all on July 1. Here is to hoping I don't suffer any
> withdrawal.

Ouch. Best of luck to you!

-- 
 \ “Giving every man a vote has no more made men wise and free |
  `\  than Christianity has made them good.” —Henry L. Mencken |
_o__)  |
Ben Finney

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Getting values stored inside sets

[Raymond Hettinger]

[Nick Coghlan]

It doesn't quite work the way RDM desribed it - he missed a step.


Thanks for the clarification.  We ought to write-out the process somewhere in a 
FAQ.

It may also be instructive to step through the recipe that answers the OP's
original request, http://code.activestate.com/recipes/499299/ 


The call "get_equivalent(set([1, 2, 3]), 2.0)" wraps the 2.0 in a new
object t and calls "t in set([1,2,3])".  The set.__contains__ method
hashes t using t.__hash__(self) and checks for an exact match 
using t.__eq__(other).  Both calls delegate to float objects but the 
latter also records the "other" that resulted in a successful equality

test (i.e. 2 is the member of the set that matched the 2.0).  The
get_equivalent call then returns the matching value, 2.0.

As far as I can tell, the technique is completely generic and lets
you reach inside any function or container to retrieve the "other"
value that is equivalent to "self".


Raymond
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Going "offline" for three months

[Michael Foord]

Brett Cannon wrote:
In order to hunker down and get my thesis proposal done by its due 
date, I am disabling mail delivery for myself for all mail.python.org 
 mailing lists for three months (sans 
python-committers so I don't accidentally commit when I shouldn't). If 
something comes up I should know about you can always email or IM me 
directly.


See you all on July 1. Here is to hoping I don't suffer any withdrawal.


We'll miss you. Hope you don't end up preferring Java. ;-)

Michael


-Brett


___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/fuzzyman%40voidspace.org.uk
  



--
http://www.ironpythoninaction.com/
http://www.voidspace.org.uk/blog


___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

[Python-Dev] core python tests (was: Re: PyDict_SetItem hook)

[C. Titus Brown]

On Fri, Apr 03, 2009 at 07:00:43PM +0100, Michael Foord wrote:
-> Collin Winter wrote:
-> >On Fri, Apr 3, 2009 at 10:28 AM, Michael Foord
-> > wrote:
-> >  
-> >>Collin Winter wrote:
-> >>
-> >>>As part of the common standard library and test suite that we agreed
-> >>>on at the PyCon language summit last week, we're going to include a
-> >>>common benchmark suite that all Python implementations can share. This
-> >>>is still some months off, though, so there'll be plenty of time to
-> >>>bikeshed^Wrationally discuss which benchmarks should go in there.
-> >>>
-> >>>  
-> >>Where is the right place for us to discuss this common benchmark and test
-> >>suite?
-> >>
-> >>As the benchmark is developed I would like to ensure it can run on
-> >>IronPython.
-> >>
-> >>The test suite changes will need some discussion as well - Jython and
-> >>IronPython (and probably PyPy) have almost identical changes to tests that
-> >>currently rely on deterministic finalisation (reference counting) so it
-> >>makes sense to test changes on both platforms and commit a single 
-> >>solution.
-> >>
-> >
-> >I believe Brett Cannon is the best person to talk to about this kind
-> >of thing. I don't know that any common mailing list has been set up,
-> >though there may be and Brett just hasn't told anyone yet :)
-> >
-> >Collin
-> >  
-> Which begs the question of whether we *should* have a separate mailing list.
-> 
-> I don't think we discussed this specific point in the language summit - 
-> although it makes sense. Should we have a list specifically for the test 
-> / benchmarking or would a more general implementations-sig be appropriate?
-> 
-> And is it really Brett who sets up mailing lists? My understanding is 
-> that he is pulling out of stuff for a while anyway, so that he can do 
-> Java / Phd type things... ;-)

'tis a sad loss for both Python-dev and the academic community...

I vote for a separate mailing list -- 'python-tests'? -- but I don't
know exactly how splintered to make the conversation.  It probably
belongs at python.org but if you want me to host it, I can.

N.B. There are a bunch of GSoC projects to work on or with the CPython
test framework (increase test coverage, write plugins to make it
runnable in nose or py.test, etc.).  I don't know that the students
should be active participants in such a list, but the mentors should at
least try to stay in the loop so that we don't completely waste our time.

cheers,
--titus
-- 
C. Titus Brown, c...@msu.edu
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Going "offline" for three months

[Brett Cannon]

On Fri, Apr 3, 2009 at 19:51, Michael Foord wrote:

> Brett Cannon wrote:
>
>> In order to hunker down and get my thesis proposal done by its due date, I
>> am disabling mail delivery for myself for all mail.python.org <
>> http://mail.python.org> mailing lists for three months (sans
>> python-committers so I don't accidentally commit when I shouldn't). If
>> something comes up I should know about you can always email or IM me
>> directly.
>>
>> See you all on July 1. Here is to hoping I don't suffer any withdrawal.
>>
>
> We'll miss you. Hope you don't end up preferring Java. ;-)


No, it would be more like JavaScript, but I don't see that happening either.

-Brett
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Should the io-c modules be put in their own directory?

[Martin v. Löwis]

> I think Benjamin is right. While most of the C source is indeed
> exactly one level below the root, there's plenty of code that isn't,
> e.g. _ctypes, cjkcodecs, expat, _multiprocessing, zlib. And even
> Objects/stringlib.

It's fine with me either way.

Martin

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] core python tests

[Nick Coghlan]

C. Titus Brown wrote:
> I vote for a separate mailing list -- 'python-tests'? -- but I don't
> know exactly how splintered to make the conversation.  It probably
> belongs at python.org but if you want me to host it, I can.

If too many things get moved off to SIGs there won't be anything left
for python-dev to talk about ;)

(Although in this case it makes sense, as I expect there will be
developers involved in alternate implementations that would like to be
part of the test suite discussion without having to sign up for the rest
of python-dev)

Cheers,
Nick.

-- 
Nick Coghlan   |   ncogh...@gmail.com   |   Brisbane, Australia
---
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

[Python-Dev] Documenting the Py3k coercion rules (was Re: Getting values stored inside sets)

[Nick Coghlan]

Raymond Hettinger wrote:
> 
> [Nick Coghlan]
>> It doesn't quite work the way RDM desribed it - he missed a step.
> 
> Thanks for the clarification.  We ought to write-out the process
> somewhere in a FAQ.

The closest we currently have to that is the write-up of the coercion
rules in 2.x:

http://docs.python.org/reference/datamodel.html#id5

Unfortunately, that mixes in a lot of CPython specific special cases
along with the old coerce() builtin that obscure the basic behaviour for
__op__ and __rop__ pairs.

Here's an initial stab at a write-up of the coercion rules for Py3k that
is accurate without getting too CPython specific:

"""
Given  "a OP b", the coercion sequence is:

1. Try "a.__op__(b)"
2. If "a.__op__" doesn't exist or the call returns NotImplemented, try
"b.__rop__(a)"
3. If "b.__rop__" doesn't exist or the call returns NotImplemented,
raise TypeError identifying "type(a)" and "type(b)" as unsupported
operands for OP
4. If step 1 or 2 is successful, then the result of the call is the
value of the expression

Given "a OP= b" the coercion sequence is:

1. Try "a = a.__iop__(b)"
2. If "a.__iop__" doesn't exist or the call returns not implemented, try
"a = a OP b" using the normal binary coercion rules above

Special cases:

- if "type(b)" is a strict subclass of "type(a)", then "b.__rop__" is
tried before "a.__op__". This allows subclasses to ensure an instance of
the subclass is returned when interacting with instances of the parent
class.

- rich comparisons are associated into __op__/__rop__ pairs as follows:
  __eq__/__eq__ (i.e. a == b is considered equivalent to b == a)
  __ne__/__ne__ (i.e. a != b is considered equivalent to b != a)
  __lt__/__gt__ (i.e. a < b is considered equivalent to b > a)
  __le__/__ge__ (i.e. a <= b is considered equivalent to b >= a)

- __rpow__ is never invoked for the 3 argument form of pow(), as the
coercion rules only apply to binary operations. In this case, a
NotImplemented return from the call to __pow__ is converted immediately
into a TypeError.
"""

Cheers,
Nick.

-- 
Nick Coghlan   |   ncogh...@gmail.com   |   Brisbane, Australia
---
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com