[Python-Dev] SQLite header scan order
Hi, The current version of setup.py looks for the sqlite header files in a number of sqlite-specific directories before looking into the default inc_dirs. I'd like to revert that order because that would make it possible to override the version of sqlite that gets picked up. Any objections to that? Ronald ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] SQLite header scan order
On May 26, 2006, at 8:35 AM, Ronald Oussoren wrote: The current version of setup.py looks for the sqlite header files in a number of sqlite-specific directories before looking into the default inc_dirs. I'd like to revert that order because that would make it possible to override the version of sqlite that gets picked up. Any objections to that? +1, the version that ships with Mac OS X 10.4 is pretty old. -bob ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Cost-Free Slice into FromString constructors--Long
[Tim] PyLong_FromString() only sees the starting address, and-- as it always does --parses until it hits a character that doesn't make sense for the input base. [Greg Ewing] This is the bug, then. long() shouldn't be using PyLong_FromString() to convert its argument, but something that takes an address and a size. (Is there a PyLong_FromStringAndSize()? If not, there should be.) Yes, that's what I said ;-): The internal parsing APIs don't currently support the buffer's offset length view of the world, so have no chance of working as hoped with any kind of buffer object now. Note that this isn't limited to long(): _no_ internal parsing routine has a sliceish API now, and none of the code on the way toward calling parsing routines is paying any attention to that it can't work when handed a buffer object. All of that must change. At least the complex() constructor gives up at once: b = buffer(123a, 0, 3) long(b) Traceback (most recent call last): File stdin, line 1, in module ValueError: invalid literal for long() with base 10: '123a' int(b) Traceback (most recent call last): File stdin, line 1, in module ValueError: invalid literal for int() with base 10: '123a' float(b) Traceback (most recent call last): File stdin, line 1, in module ValueError: invalid literal for float(): 123a complex(b) Traceback (most recent call last): File stdin, line 1, in module TypeError: complex() argument must be a string or a number ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] 2.5a2 try/except slow-down: Convert to type?
Neal Norwitz wrote: This is probably orthogonal to the problem, however, you may want to look into trying to speed up Python/errors.c. This link will probably not work due to sessions, but click on the latest run for python and Python/errors.c http://coverage.livinglogic.de/coverage/web/selectEntry.do?template=2850entryToSelect=547460 If you look at the coverage numbers, many functions are called millions of times. In many cases traceback is passed as NULL in other functions within errors.c. If you could get rid of that check and possibly others and do some fast pathing in there, it might speed up exceptions some (or normal cases where exceptions are raised internally, then discarded). If that helps, here is the list of the 1000 most executed lines of the last run of the test suite: http://styx.livinglogic.de/~walter/python/topcoverage.html Servus, Walter ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] A Horrible Inconsistency
Greg Ewing wrote: Steve Holden wrote: In actual fact the effbot has lately found itself so permeated with Windows that it has become constituionally incapable of using a forward slash. Don't know what's with the square brackets though ... I was thinking maybe that message had resulted from a Windows and a VMS port of the effbot that got mixed together somehow... That would be interesting: if we start to see semicolons in the sig strings we'll know which version of the bot is mailing. regards Steve -- Steve Holden +44 150 684 7255 +1 800 494 3119 Holden Web LLC/Ltd http://www.holdenweb.com Love me, love my blog http://holdenweb.blogspot.com Recent Ramblings http://del.icio.us/steve.holden ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Returning int instead of long from struct when possible for performance
On 5/26/06, Tim Peters [EMAIL PROTECTED] wrote: [Bob Ippolito] Given the interchangeability of int and long, I don't foresee any other complications with this change. Thoughts?+1, and for 2.5.Even int() doesn't always return an int anymore, and it's just stupid to bear the burden of an unbounded long when it's not really needed. Completely agreed. We've been unifying longs and integers for whole releases, I cannot imagine anyone not getting the hint. Ints and longs are the same thing, deal with it. Whether you get an int or a long for a particular value is a platform-dependant accident anyway (people seem to be ignoring 64-bit hardware all the time... I know Tim does :) -- Thomas Wouters [EMAIL PROTECTED]Hi! I'm a .signature virus! copy me into your .signature file to help me spread! ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] SQLite status?
We're coming down to the wire on _Python for Dummies_, and I'm trying to persuade the publisher to stick a blurb about SQLite on the cover, but my last understanding was that there was a small chance we might pull SQLite for insufficient docs. Is that still true? -- Aahz ([EMAIL PROTECTED]) * http://www.pythoncraft.com/ I saw `cout' being shifted Hello world times to the left and stopped right there. --Steve Gonedes ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] Request for patch review
I've worked on two patches for NeedForSpeed, and would like someone familiar with the areas they touch to review them before I check them in, breaking all the buildbots which aren't broken yet ;) They are: http://python.org/sf/1346214 Better dead code elimination for the AST compiler http://python.org/sf/921466 Reduce number of open calls on startup GB Thanks! Georg ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] partition() variants
I didn't find an answer in the str.partition() thread in the archives (it's enormous, so easy to miss the right message), so I have two questions: 1) Is str.rpartition() still wanted? 2) What about adding partition() to the re module? --amk ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] partition() variants
A.M. Kuchling wrote:
I didn't find an answer in the str.partition() thread in the archives
(it's enormous, so easy to miss the right message), so I have two
questions:
1) Is str.rpartition() still wanted?
2) What about adding partition() to the re module?
And what happens if the separator is an instance of a subclass?
class s2(str):
def __repr__(self):
return s2(%r) % str(self)
print foobar.partition(s2(o))
Currently this prints:
('f', s2('o'), 'obar')
Should this be
('f', 'o', 'obar')
or not?
And what about:
print s2(foobar).partition(x)
Currently this prints
(s2('foobar'), '', '')
Servus,
Walter
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] SQLite status?
On Friday 26 May 2006 21:12, Aahz wrote: We're coming down to the wire on _Python for Dummies_, and I'm trying to persuade the publisher to stick a blurb about SQLite on the cover, but my last understanding was that there was a small chance we might pull SQLite for insufficient docs. Is that still true? Not in my universe, we won't. As far as I know, the docs are checked in. I still need to review them. -- Anthony Baxter [EMAIL PROTECTED] It's never too late to have a happy childhood. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] A Horrible Inconsistency
2006/5/25, Fredrik Lundh [EMAIL PROTECTED]: -1 * (1, 2, 3) () -(1, 2, 3) Traceback (most recent call last): File stdin, line 1, in module TypeError: bad operand type for unary - We Really Need To Fix This! I don't see here an inconsistency. The operator * is not a multiplier as in math, it's more a repeater, so math multiplier attributes don't apply here. There's no concept like negative tuple or positive tuple, so the second example is clearly an error. Regarding the first line, in the docs expresely says Values of n less than 0 are treated as 0. I think that we can do one of the following, when we found -1 * (1, 2, 3): - Treat -1 as 0 and return an empty tuple (actual behavior). - Treat the negative as a reverser, so we get back (3, 2, 1). - Raise an error. Personally, +0 on the third. Regards, -- .Facundo Blog: http://www.taniquetil.com.ar/plog/ PyAr: http://www.python.org/ar/ ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] A Horrible Inconsistency
On Fri, May 26, 2006 at 12:37:02PM -0300, Facundo Batista wrote: - Treat the negative as a reverser, so we get back (3, 2, 1). Then we could get: print -123 321 Yay! Thanks, Sean -- Sometimes it pays to stay in bed on Monday, rather than spending the rest of the week debugging Monday's code. -- Christopher Thompson Sean Reifschneider, Member of Technical Staff [EMAIL PROTECTED] tummy.com, ltd. - Linux Consulting since 1995: Ask me about High Availability ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] A Horrible Inconsistency
2006/5/26, Sean Reifschneider [EMAIL PROTECTED]: On Fri, May 26, 2006 at 12:37:02PM -0300, Facundo Batista wrote: - Treat the negative as a reverser, so we get back (3, 2, 1). Then we could get: print -123 321 An integer is NOT a sequence. OTOH, that should be consistent to -1 * 123 321 And remember I voted for returning an error, ;) -- .Facundo Blog: http://www.taniquetil.com.ar/plog/ PyAr: http://www.python.org/ar/ ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] A Horrible Inconsistency
Facundo Batista wrote: 2006/5/26, Sean Reifschneider [EMAIL PROTECTED]: On Fri, May 26, 2006 at 12:37:02PM -0300, Facundo Batista wrote: - Treat the negative as a reverser, so we get back (3, 2, 1). Then we could get: print -123 321 An integer is NOT a sequence. OTOH, that should be consistent to -1 * 123 321 This is actually a nice idea, because it's even a more nonintuitive answer for Python newbies posting to c.l.py asking how to reverse a string wink Georg ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] A Horrible Inconsistency
Sean Reifschneider wrote: - Treat the negative as a reverser, so we get back (3, 2, 1). Then we could get: print -123 321 Yay! and while we're at it, let's fix this: 0.66 * (1, 2, 3) (1, 2) and maybe even this 0.5 * (1, 2, 3) (1, 1) but I guess the latter one might need a pronunciation. /F ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] A Horrible Inconsistency
On Friday 26 May 2006 11:50, Georg Brandl wrote: This is actually a nice idea, because it's even a more nonintuitive answer for Python newbies posting to c.l.py asking how to reverse a string wink Even better: 123*-1 We'd get to explain: - what the *- operator is all about, and - why we'd use it with a string and an int. I see possibilities here. :-) -Fred -- Fred L. Drake, Jr. fdrake at acm.org ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] A Horrible Inconsistency
Fredrik Lundh wrote: Sean Reifschneider wrote: - Treat the negative as a reverser, so we get back (3, 2, 1). Then we could get: print -123 321 Yay! and while we're at it, let's fix this: 0.66 * (1, 2, 3) (1, 2) and maybe even this 0.5 * (1, 2, 3) (1, 1) but I guess the latter one might need a pronunciation. No, no, no! Floating point is so inaccurate! It has to be Decimal(0.5) * (1, 2, 3) (1, Decimal(1)) Georg ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] A Horrible Inconsistency
Fred I see possibilities here. :-) Fred appears to be looking for more job security. ;-) Skip ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] A Horrible Inconsistency
Fred L. Drake, Jr. wrote: Even better: 123*-1 We'd get to explain: - what the *- operator is all about, and - why we'd use it with a string and an int. I see possibilities here. :-) the infamous *- clear operator? who snuck that one into python? /F ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] A Horrible Inconsistency
2006/5/26, Fred L. Drake, Jr. [EMAIL PROTECTED]: Even better: 123*-1 We'd get to explain: - what the *- operator is all about, and - why we'd use it with a string and an int. I see possibilities here. :-) All this different ways enforce my vote: we should get an error... Regards, -- .Facundo Blog: http://www.taniquetil.com.ar/plog/ PyAr: http://www.python.org/ar/ ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] Low-level exception invariants?
In various places we store triples of exception info, like a
PyFrameObject's f_exc_type, f_exc_value, and f_exc_traceback PyObject*
members.
No invariants are documented, and that's a shame. Patch 1145039 aims
to speed ceval a bit by relying on a weak guessed invariant, but I'd
like to make the stronger claim (which allows for stronger speedups)
that, at any moment,
all three are NULL
or
all three are not NULL
Now I know that's not true, although Python's test suite doesn't
provoke a violation. In particular, PyErr_SetNone() forces
tstate-curexc_value to NULL. Other code then special-cases the snot
out of XYZ_value, replacing NULL with Py_None on the fly, like
PyErr_Fetch(exc, val, tb);
if (val == NULL) {
val = Py_None;
Py_INCREF(val);
}
in the main eval loop. I'd much rather change PyErr_SetNone() to set
the value to Py_None to begin with -- that function is almost never
called, so special-casing in that is less damaging than special-casing
NULL everywhere else.
The docs are confused about this now too, claiming:
void PyErr_SetNone(PyObject *type)
This is a shorthand for PyErr_SetObject(type, Py_None).
It's actually a shorthand for PyErr_SetObject(type, NULL) now.
Does anyone see a real problem with _establishing_ an all NULL or
none NULL invariant? It makes outstanding ;-) _conceptual_ sense to
me, and would allow removing some masses of distributed test-branch
special-casing in code trying to use these guys.
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] A Horrible Inconsistency
On 5/26/06, Facundo Batista [EMAIL PROTECTED] wrote: All this different ways enforce my vote: we should get an error... Perhaps you missed Tim's post, so here's a few lines of my own code that I know would break: padding = [None] * (self.width - len(leaves)) left_padding = [None] * (self.left_width - len(left_window)) right_padding = [None] * (self.right_width - len(right_window)) Sure, I could write these as: padding = [None] * max(0, self.width - len(leaves)) left_padding = [None] * max(0, self.left_width - len(left_window)) right_padding = [None] * max(0, self.right_width - len(right_window)) But if this change goes in, I want a big we're breaking backwards incompatibility message somewhere. I say if you really want an exception raised for these cases, ask for the behavior change in Python 3000. All it would give us in Python 2.X is a bunch of broken code. STeVe -- Grammar am for people who can't think for myself. --- Bucky Katt, Get Fuzzy ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] epoll implementation
I wrote an epoll implementation which can be used as a drop-in replacement for parts of the select module (assuming the program is using only poll). The code can currently be used by doing: import epoll as select It was released under the Python license on sourceforge: http://sourceforge.net/projects/pyepoll Is there any interest in incorporating this into the standard python distribution? Ross ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] A Horrible Inconsistency
2006/5/26, Steven Bethard [EMAIL PROTECTED]: On 5/26/06, Facundo Batista [EMAIL PROTECTED] wrote: All this different ways enforce my vote: we should get an error... ... But if this change goes in, I want a big we're breaking backwards incompatibility message somewhere. I say if you really want an exception raised for these cases, ask for the behavior change in Python 3000. All it would give us in Python 2.X is a bunch of broken code. Of course that this change can not be made in 2.x. Regards, -- .Facundo Blog: http://www.taniquetil.com.ar/plog/ PyAr: http://www.python.org/ar/ ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] [Python-checkins] r46247 - in python/branches/sreifschneider-newnewexcept: Makefile.pre.in Objects/exceptions.c Python/exceptions.c
Since no Python code should be generating the warning (I would double-check that, obviously), I say take it out. At least from a PEP 352 perspective there is nothing about keeping it (or removing it).-Brett On 5/26/06, Richard Jones [EMAIL PROTECTED] wrote: On 25/05/2006, at 9:00 PM, Jim Jewett wrote: +/* + *OverflowWarning extends Warning + */ +SimpleExtendsException(PyExc_Warning, OverflowWarning, Base class for warnings about numeric overflow.Won't exist in Python 2.5.); Take it out now?What do people say? I implemented this one because it was implementedin the old code in 2.5. Happy to remove it. Richard___ Python-Dev mailing listPython-Dev@python.orghttp://mail.python.org/mailman/listinfo/python-devUnsubscribe: http://mail.python.org/mailman/options/python-dev/brett%40python.org ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] epoll implementation
Ross I wrote an epoll implementation which can be used as a drop-in Ross replacement for parts of the select module ... Ross Is there any interest in incorporating this into the standard Ross python distribution? Without going to the trouble of downloading epoll (always an adventure with SourceForget), can you explain what epoll does, how it's better than (parts of) select, how widely it's used and how stable it is? Thx, Skip ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] A Horrible Inconsistency
On 5/26/06, Facundo Batista [EMAIL PROTECTED] wrote: I think that we can do one of the following, when we found -1 * (1, 2, 3): - Treat -1 as 0 and return an empty tuple (actual behavior). - Treat the negative as a reverser, so we get back (3, 2, 1). - Raise an error. No, no, no. The important invariant is that n * seq is loop(seq)[:n*len(seq)] where loop(seq) is an endless loop of the elements of seq. So obviously, if n is negative, the result should be an infinite sequence that's == to loop(seq). -j ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] partition() variants
On 5/26/06, Walter Dörwald [EMAIL PROTECTED] wrote:
A.M. Kuchling wrote:
I didn't find an answer in the str.partition() thread in the archives
(it's enormous, so easy to miss the right message), so I have two
questions:
1) Is str.rpartition() still wanted?
Can't remember. Raymond?
2) What about adding partition() to the re module?
No.
And what happens if the separator is an instance of a subclass?
class s2(str):
def __repr__(self):
return s2(%r) % str(self)
print foobar.partition(s2(o))
Currently this prints:
('f', s2('o'), 'obar')
Should this be
('f', 'o', 'obar')
or not?
And what about:
print s2(foobar).partition(x)
Currently this prints
(s2('foobar'), '', '')
These are both fine with me.
--
--Guido van Rossum (home page: http://www.python.org/~guido/)
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Low-level exception invariants?
+1, if you can also prove that the traceback will never be null. I
failed at that myself last time I tried, but I didn't try very long or
hard.
--Guido
On 5/26/06, Tim Peters [EMAIL PROTECTED] wrote:
In various places we store triples of exception info, like a
PyFrameObject's f_exc_type, f_exc_value, and f_exc_traceback PyObject*
members.
No invariants are documented, and that's a shame. Patch 1145039 aims
to speed ceval a bit by relying on a weak guessed invariant, but I'd
like to make the stronger claim (which allows for stronger speedups)
that, at any moment,
all three are NULL
or
all three are not NULL
Now I know that's not true, although Python's test suite doesn't
provoke a violation. In particular, PyErr_SetNone() forces
tstate-curexc_value to NULL. Other code then special-cases the snot
out of XYZ_value, replacing NULL with Py_None on the fly, like
PyErr_Fetch(exc, val, tb);
if (val == NULL) {
val = Py_None;
Py_INCREF(val);
}
in the main eval loop. I'd much rather change PyErr_SetNone() to set
the value to Py_None to begin with -- that function is almost never
called, so special-casing in that is less damaging than special-casing
NULL everywhere else.
The docs are confused about this now too, claiming:
void PyErr_SetNone(PyObject *type)
This is a shorthand for PyErr_SetObject(type, Py_None).
It's actually a shorthand for PyErr_SetObject(type, NULL) now.
Does anyone see a real problem with _establishing_ an all NULL or
none NULL invariant? It makes outstanding ;-) _conceptual_ sense to
me, and would allow removing some masses of distributed test-branch
special-casing in code trying to use these guys.
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/guido%40python.org
--
--Guido van Rossum (home page: http://www.python.org/~guido/)
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] epoll implementation
I don't know what epoll is. On a related note, perhaps the needforspeed folks should look into supporting kqueue on systems where it's available? That's a really fast BSD-originated API to replace select/poll. (It's fast due to the way the API is designed.) --Guido On 5/26/06, Ross Cohen [EMAIL PROTECTED] wrote: I wrote an epoll implementation which can be used as a drop-in replacement for parts of the select module (assuming the program is using only poll). The code can currently be used by doing: import epoll as select It was released under the Python license on sourceforge: http://sourceforge.net/projects/pyepoll Is there any interest in incorporating this into the standard python distribution? Ross ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/guido%40python.org -- --Guido van Rossum (home page: http://www.python.org/~guido/) ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] epoll implementation
On Fri, May 26, 2006 at 11:47:43AM -0500, [EMAIL PROTECTED] wrote: Ross I wrote an epoll implementation which can be used as a drop-in Ross replacement for parts of the select module ... Ross Is there any interest in incorporating this into the standard Ross python distribution? Without going to the trouble of downloading epoll (always an adventure with SourceForget), can you explain what epoll does, how it's better than (parts of) select, how widely it's used and how stable it is? Sure, I can provide a short version of what's on this page: http://www.kegel.com/c10k.html epoll is a replacement for the standard poll() system call. It's available on Linux, introduced sometime during the 2.5 development series. Whereas poll requires that an fd list be maintained in userspace and handed in on every call, epoll maintains that list in the kernel and provides (de)registration functions for individual fds. This prevents the kernel from having to scan potentially thousands of file descriptors for each call to discover only a handful which had activity. Oddly enough, the interface to the poll call in the python select module much more closely resembles the design of epoll (kudos to whoever designed that). This code has undergone some testing with both codeville and BitTorrent, but I'm not going to claim it's seen as much testing as it should. The NeedForSpeed wiki lists /dev/epoll as a goal for the twisted folks. I assume that the naming here is either confusion with the Solaris /dev/poll or that it refers to the initial implementation of epoll, which I believe used a device file. Either way, this would give them that support for free, as well as everyone else. Ross ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] This
Facundo Batista [EMAIL PROTECTED] wrote in message news:[EMAIL PROTECTED] Just end user experience's two cents here (btw, this line is correct at English level?) Since you asked...your question would be better written is this line correct English? And the line before, while not formal English of the kind needed, say, for Decimal docs, works well enough for me as an expression of an informal conversational thought. Maybe should exist some quick measure method like, Maybe *there* should [This is one situation where English is wordier than, for instance, Spanish.] Terry Jan Reedy ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] epoll implementation
Guido van Rossum wrote: I don't know what epoll is. On a related note, perhaps the needforspeed folks should look into supporting kqueue on systems where it's available? That's a really fast BSD-originated API to replace select/poll. (It's fast due to the way the API is designed.) roughly speaking, epoll is kqueue for linux. /F ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] epoll implementation
On Fri, May 26, 2006 at 01:10:30PM -0400, Jean-Paul Calderone wrote: Including a wrapper for this functionality would be quite useful for many python network apps. However, I think with ctypes going into 2.5, it might be better to consider providing epoll support using a ctypes-based module rather than an extension module. I have to admit that I wrote this for python 2.4 and haven't yet educated myself on ctypes. If this is what's desired I can put some energy into it. However, the PyEpoll module is largely a modified version of the python 2.4 select module. Of course, if there is a volunteer to maintain and support an extension module, that's better than nothing. PyEpoll is missing a couple features I would like to see - the size of the epoll set is hard-coded to FD_SETSIZE, for example: while this makes little difference to Python 2.4.3 (since you cannot use sockets with fileno = FD_SETSIZE at all in that version of Python), it is a serious limitation for other versions of Python. Similarly, the number of events to retrieve when invoking epoll_wait() is also hardcoded to FD_SETSIZE. Real applications often want to tune this value to avoid being overwhelmed by io events. It is not true that this module limits the number of file descriptors to FD_SETSIZE. You were probable looking at the number of file descriptors returned by the epoll call, which is already limited by FD_SETSIZE because of the size of event array. In any case, this can be made tunable. Of course the other standard things should be added as well - documentation and test coverage, neither of which seem to be present at all in PyEpoll. Like I said, this code is a modified version of the python select module. It should (I hope, unless I accidentally removed some) include all the documentation from there, and any test cases for that module will apply here. The API provided to python programs has not changed, only the backend implementation of the API. My hope is that this can be included as a compile-time option for the select module instead of being its own module. Ross ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Low-level exception invariants?
[Guido]
+1, if you can also prove that the traceback will never be null. I
failed at that myself last time I tried, but I didn't try very long or
hard.
Thanks! I'm digging.
Stuck right now on this miserable problem that's apparently been here
forever: I changed PyErr_SetObject to start like so:
PyErr_SetObject(PyObject *exception, PyObject *value)
{
assert(exception != NULL);
This triggered instantly upon firing up Python, and it is was _called_ via
PyErr_Format(PyExc_AttributeError, ...
! I thought I had gone insane, until I realized this happens during
_PyExc_Init(): that can try to raise exceptions before the exception
pointers have been initialized. So long as we can get into exception
machinery while the exception objects point to trash, there aren't
many useful invariants ;-)
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] epoll implementation
Ross Cohen wrote: Is there any interest in incorporating this into the standard python distribution? I would like to see epoll support in Python, but not in the way PyEpoll is packaged. Instead, I think it should go into the select module, and be named epoll_*. Likewise, if kqueue was ever supported, I think it should also go into the select module. Regards, Martin ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] epoll implementation
On 05/26/2006-10:07AM, Guido van Rossum wrote: I don't know what epoll is. On a related note, perhaps the needforspeed folks should look into supporting kqueue on systems where it's available? That's a really fast BSD-originated API to replace select/poll. (It's fast due to the way the API is designed.) http://python-hpio.net/trac/wiki/LibEventPython libevent-python is a CPython wrapper extension for Niels Provos' libevent, a small, fast asynchronous IO framework that supports select(), poll(), kqueue(), /dev/poll, and epoll(). Libevent can be found here http://monkey.org/~provos/libevent/ and it explains the advantages of epoll/kqueue over select/poll. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] epoll implementation
On Fri, May 26, 2006 at 08:03:12PM +0200, Martin v. Löwis wrote: Ross Cohen wrote: Is there any interest in incorporating this into the standard python distribution? I would like to see epoll support in Python, but not in the way PyEpoll is packaged. Instead, I think it should go into the select module, and be named epoll_*. Likewise, if kqueue was ever supported, I think it should also go into the select module. I agree that it should go into the select module, but not as a seperate set of calls. epoll is strictly better than poll. kqueue is strictly better than poll. Windows has its own completion ports API. Why should an application developer have to detect what platform they are running on? Why not simply provide the best implementation for the platform through the python API everyone is already using? Ross ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] partition() variants
1) Is str.rpartition() still wanted? Yes. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] epoll implementation
Jonathan LaCour wrote: There is, in fact, an implementation of kqueue for Python already available. I have not used it myself, but it is available here: http://python-hpio.net/trac/ maybe the needforspeed people could take a look at this? take a look is really quite involved. We cannot just incorporate the code without permission of the author. So if the code is itself acceptable, then somebody would have to talk the author into contributing. Regards, Martin ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] partition() variants
Raymond Hettinger wrote: 1) Is str.rpartition() still wanted? Yes. I might have missed my earlier 30-minute deadline with one minute (not my fault! I was distracted! seriously!), but this time, I actually managed to get the code in there *before* I saw the pronouncement ;-) /F ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] epoll implementation
Ross Cohen wrote: I agree that it should go into the select module, but not as a seperate set of calls. epoll is strictly better than poll. kqueue is strictly better than poll. Windows has its own completion ports API. Why should an application developer have to detect what platform they are running on? Because the APIs have different semantics. Design some API for epoll, and make it replace select or poll (your choice), and I create you an application that breaks under your poll emulation. If a complete emulation was possible, the OS developers would not have invented a new API. Why not simply provide the best implementation for the platform through the python API everyone is already using? Well, what *is* the API everyone is already using? This is really something for a higher-level API that assumes a certain processing model (unless the OS API, which assumes no processing model). Python has a tradition of exposing system APIs *as is*, with no second-guessing either the OS developers, nor the application developers. Then, we put a unified API *on top* of that. For event processing, the standard library always provided asyncore/asynchat, although people don't like it. I'm sure the Twisted people would integrate epoll in no time, and with no API change for *their* users. Regards, Martin ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] epoll implementation
On Fri, 26 May 2006 14:31:33 -0400, Ross Cohen [EMAIL PROTECTED] wrote: On Fri, May 26, 2006 at 08:03:12PM +0200, Martin v. Löwis wrote: Ross Cohen wrote: Is there any interest in incorporating this into the standard python distribution? I would like to see epoll support in Python, but not in the way PyEpoll is packaged. Instead, I think it should go into the select module, and be named epoll_*. Likewise, if kqueue was ever supported, I think it should also go into the select module. I agree that it should go into the select module, but not as a seperate set of calls. How about not *only* as a separate set of calls? If poll(2) and epoll(4) are both available on the underlying platform, then they should both be exposed to Python as separate APIs. Then, on top of that, a relatively simple layer which selects the most efficient mechanism can be exposed, and developers can be encouraged to use that instead. Hiding the difference between poll and epoll would be detrimental to more advanced developers since it would hide the edge-triggered mode of epoll, which is still more efficient than the level-triggered mode, since only the level-triggered mode can be considered API-compatible with poll(2) without adding a much more complex layer on top. epoll is strictly better than poll. kqueue is strictly better than poll. AIUI, kqueue actually isn't implemented for PTYs on OS X, whereas poll(2) is. Given this, I don't think kqueue is actually strictly better. Although hopefully Apple will get their act together and fix this deficiency. Windows has its own completion ports API. Why should an application developer have to detect what platform they are running on? Why not simply provide the best implementation for the platform through the python API everyone is already using? So far as this is possible, I think it is a good idea. Jean-Paul ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] epoll implementation
On Fri, 26 May 2006 13:31:33 -0400, Ross Cohen [EMAIL PROTECTED] wrote: On Fri, May 26, 2006 at 01:10:30PM -0400, Jean-Paul Calderone wrote: Of course, if there is a volunteer to maintain and support an extension module, that's better than nothing. PyEpoll is missing a couple features I would like to see - the size of the epoll set is hard-coded to FD_SETSIZE, for example: while this makes little difference to Python 2.4.3 (since you cannot use sockets with fileno = FD_SETSIZE at all in that version of Python), it is a serious limitation for other versions of Python. Similarly, the number of events to retrieve when invoking epoll_wait() is also hardcoded to FD_SETSIZE. Real applications often want to tune this value to avoid being overwhelmed by io events. It is not true that this module limits the number of file descriptors to FD_SETSIZE. You were probable looking at the number of file descriptors returned by the epoll call, which is already limited by FD_SETSIZE because of the size of event array. In any case, this can be made tunable. Woops, you're right, of course. I forgot that the argument to epoll_create is only a hint, not a limit. Sorry about that. Jean-Paul ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] epoll implementation
Ross Cohen wrote: True, and as I mentioned before, the python API more closely matches epoll in this case. The level-triggered mode of epoll is an almost perfect match. Someone went to some lengths to hide the details of the system poll interface. Ah right, I missed that point. That makes it difficult to find an application that would break :-) One problem is that epoll allocates another file handle, when poll does not; the other problem could exist when the EPOLL constants differ in value from the POLL constants (which isn't the case on Linux), and then somebody uses numeric values instead of symbolic ones for register(). That said, I would be in favour of having select.poll silently use epoll where available. Of course, it would be good if a cheap run-time test could be made whether epoll is available at run-time (although just waiting for ENOSYS from epoll_create is probably cheap enough). Also, it would be good if the application could find out it is using epoll; for example, epollObject could expose a fileno member. Regards, Martin ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] epoll implementation
On Fri, May 26, 2006 at 02:49:44PM -0400, Jean-Paul Calderone wrote: On Fri, 26 May 2006 14:31:33 -0400, Ross Cohen [EMAIL PROTECTED] wrote: I agree that it should go into the select module, but not as a seperate set of calls. How about not *only* as a separate set of calls? If poll(2) and epoll(4) are both available on the underlying platform, then they should both be exposed to Python as separate APIs. Then, on top of that, a relatively simple layer which selects the most efficient mechanism can be exposed, and developers can be encouraged to use that instead. This is reasonable, though it requires some surgery to the select module. select.poll is a very reasonable layer over whatever mechanism is available. The system poll call would need to be properly wrapped and exposed in addition to epoll, because currently it is not. Hiding the difference between poll and epoll would be detrimental to more advanced developers since it would hide the edge-triggered mode of epoll, which is still more efficient than the level-triggered mode, since only the level-triggered mode can be considered API-compatible with poll(2) without adding a much more complex layer on top. I've never been fond of the edge-triggered mode. It's a pretty minor optimization, it saves scanning the set of previously returned fds to see if the events on them are still relevant. Given that there are added pitfalls to using that mode, it never seemed worth it. Any layer in python which tries to smooth over the differences will certainly kill the benefits. Of course, that's just my personal preference. If someone wants to use the edge-triggered mode, by all means let them. epoll is strictly better than poll. kqueue is strictly better than poll. AIUI, kqueue actually isn't implemented for PTYs on OS X, whereas poll(2) is. Given this, I don't think kqueue is actually strictly better. Although hopefully Apple will get their act together and fix this deficiency. Ok, I'm not familiar with intimate details of kqueue. However, if there were a select.poll implementation based on kqueue, it would still be an improvement, since there isn't *any* implementation on OS X right now. Ross ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] epoll implementation
How about not *only* as a separate set of calls? If poll(2) and epoll(4) are both available on the underlying platform, then they should both be exposed to Python as separate APIs. Then, on top of that, a relatively simple layer which selects the most efficient mechanism can be exposed, and developers can be encouraged to use that instead. Why does it have to be a separate API? Isn't the edge-triggered mode just a matter of passing EPOLLET into register? If so, I would advise against having two sets of functions, although having two sets of constants is propably fine. I would just provide a second constructor, select.epoll(), which guarantees that epoll is used as the implementation strategy (i.e. it would raise OSError if the kernel doesn't support epoll_create). Regards, Martin ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] epoll implementation
On Fri, May 26, 2006 at 10:12:15PM +0200, Martin v. Löwis wrote: That said, I would be in favour of having select.poll silently use epoll where available. Of course, it would be good if a cheap run-time test could be made whether epoll is available at run-time (although just waiting for ENOSYS from epoll_create is probably cheap enough). Also, it would be good if the application could find out it is using epoll; for example, epollObject could expose a fileno member. Woo! I'd be happy to code this up. First I'd like to get a feel for the prefered way of getting this integrated. Would people rather see: 1) Provide an epoll implementation which is used silently when the call is available. 2) Expose both poll(2) and epoll(4) in python and build select.poll on top of whatever is available. Ok, so 2 is only different in that it exposes the lower level APIs. I'd like to know if that's what people want. Ross ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] partition() variants
Guido van Rossum wrote:
On 5/26/06, Walter Dörwald [EMAIL PROTECTED] wrote:
[...]
And what happens if the separator is an instance of a subclass?
class s2(str):
def __repr__(self):
return s2(%r) % str(self)
print foobar.partition(s2(o))
Currently this prints:
('f', s2('o'), 'obar')
Should this be
('f', 'o', 'obar')
or not?
And what about:
print s2(foobar).partition(x)
Currently this prints
(s2('foobar'), '', '')
These are both fine with me.
split() doesn't behave that way:
s2(foobar).split(x)
['foo']
Servus,
Walter
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] partition() variants
I think you're getting to implementation details here. Whether a new
string is returned or a reference to the old one is an optimization
decision. I don't think it's worth legislating this behavior one way
or another (especially since it's mostly a theoretical issue).
--Guido
On 5/26/06, Walter Dörwald [EMAIL PROTECTED] wrote:
Guido van Rossum wrote:
On 5/26/06, Walter Dörwald [EMAIL PROTECTED] wrote:
[...]
And what happens if the separator is an instance of a subclass?
class s2(str):
def __repr__(self):
return s2(%r) % str(self)
print foobar.partition(s2(o))
Currently this prints:
('f', s2('o'), 'obar')
Should this be
('f', 'o', 'obar')
or not?
And what about:
print s2(foobar).partition(x)
Currently this prints
(s2('foobar'), '', '')
These are both fine with me.
split() doesn't behave that way:
s2(foobar).split(x)
['foo']
Servus,
Walter
--
--Guido van Rossum (home page: http://www.python.org/~guido/)
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] epoll implementation
Ross Cohen wrote: 1) Provide an epoll implementation which is used silently when the call is available. 2) Expose both poll(2) and epoll(4) in python and build select.poll on top of whatever is available. Ok, so 2 is only different in that it exposes the lower level APIs. I'd like to know if that's what people want. Supporting EPOLLET reliably looks like a valid use case, so there must be some way to specify I really want epoll. Whether this is through a second entry point, or through an optional flag to poll, I don't care (providing the flag might be actually more expressive, since it would also allow to specify I want poll, although I can't see a use case for that). Regards, Martin ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] epoll implementation
On 26-mei-2006, at 22:22, Ross Cohen wrote: AIUI, kqueue actually isn't implemented for PTYs on OS X, whereas poll(2) is. Given this, I don't think kqueue is actually strictly better. Although hopefully Apple will get their act together and fix this deficiency. Ok, I'm not familiar with intimate details of kqueue. However, if there were a select.poll implementation based on kqueue, it would still be an improvement, since there isn't *any* implementation on OS X right now. Huh? osx support poll just fine, at least in recent versions. 10.3's poll is slightly broken, the one on 10.4 works fine (AFAIK). Ronald ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] patch for mbcs codecs
Hello. This is my first post. I sent the patch for fixing mbcs codec bug to SourceForge, and I was adviced to find someone here with a multibyte version of Windows to look at the patch. http://sourceforge.net/tracker/index.php?func=detailaid=1455898group_id=5470atid=305470 Is there anyone? Any help will be appriciated. Thank you. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] epoll implementation
Ross Cohen wrote: I've never been fond of the edge-triggered mode. It's a pretty minor optimization, it saves scanning the set of previously returned fds to see if the events on them are still relevant. Given that there are added pitfalls to using that mode, it never seemed worth it. Any layer in python which tries to smooth over the differences will certainly kill the benefits. I thought about this (even though I never used it), and I think there are good uses for EPOLLET. Suppose you have a large set of descriptors where you possibly want to write to. So you poll for write, then write, then poll again. Eventually you fill the write buffer, and need to wait for the other side to read some data; you block in poll. Later, you have sent all data you wanted to send, yet the connection remains open. So you have to *remove* the fd from the poll list, because otherwise poll would return immediately each time. So you keep adding and removing file descriptors to the epoll set, essentially for each write operation. I imagine it would be more efficient to just leave the write file descriptors in the set, even if you have sent all data. With EPOLLET, the epoll_wait will just ignore those descriptors, and it will watch only those for which you have actually filled the transmit queue, and where you have pending data. This feature sounds reasonable to me. Ok, I'm not familiar with intimate details of kqueue. However, if there were a select.poll implementation based on kqueue, it would still be an improvement, since there isn't *any* implementation on OS X right now. That depends on the kernel version, no? I though 10.4 added poll. Also, kqueue support would be for the BSDs primarily. If somebody contributed a patch, I would accept it providing it was disabled on OSX (assuming kqueue really is not true poll replacement on this system). Regards, Martin ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] warnings about missing __init__.py in toplevel directories
Hi, Some time ago a warning was introduced for directories on sys.path that don't contain an __init__.py but have the same name as a package/ module that is being imported. Is it intentional that this triggers for toplevel imports? These warnings are triggered in the build process for PyObjC, which is in itself pretty harmless but very annoying. We have a source-deps directory that contains several external packages that are used during the build. Those external packages are included through the svn:external mechanism and are in directories named after those packages, with a .pth file to include them on sys.path. In most cases the package name is the same as the python package, which is triggering the warning. Our package structure: setup.py source-deps/ subprocess/ subprocess.pth # includes subproces on sys.path py2app/ py2app.pth # includes py2app on sys.path ... ... # the regular sources, documentation and stuff Ronald ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] warnings about missing __init__.py in toplevel directories
On 5/26/06, Ronald Oussoren [EMAIL PROTECTED] wrote: Some time ago a warning was introduced for directories on sys.path that don't contain an __init__.py but have the same name as a package/ module that is being imported. Is it intentional that this triggers for toplevel imports? Yes, since that's the major use case. -- --Guido van Rossum (home page: http://www.python.org/~guido/) ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] epoll implementation
On Fri, May 26, 2006 at 11:43:05PM +0200, Ronald Oussoren wrote: Ok, I'm not familiar with intimate details of kqueue. However, if there were a select.poll implementation based on kqueue, it would still be an improvement, since there isn't *any* implementation on OS X right now. Huh? osx support poll just fine, at least in recent versions. 10.3's poll is slightly broken, the one on 10.4 works fine (AFAIK). I'm probably not up to date. All I could find reference to was that poll() on 10.3 is emulated using select(). Ross ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] epoll implementation
On Fri, May 26, 2006 at 11:56:06PM +0200, Martin v. Löwis wrote: I thought about this (even though I never used it), and I think there are good uses for EPOLLET. There are, if the programmer wants to deal with it. Easy enough to add the flag and give them the choice. I'll put together a select module which adds this and code and uses it if the right stuff is present at compile and run time. epoll also allows 64 bits of data to be tucked away and returned when events happen. Could be useful for saving a dict lookup for every event. However, there are some refcounting issues. Dict lookup per event could be traded for one on deregistration. All it needs is a small forward-compatible extension to the current select.poll API. Ross ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] epoll implementation
Fredrik Lundh wrote: roughly speaking, epoll is kqueue for linux. There are many different select-like things around now (select, poll, epoll, kqueue -- are there others?) and random combinations of them seem to be available on any given platform. This makes writing platform-independent code that needs select-like functionality very awkward. Rather than adding yet another platform-dependent module, I'd like to see a unified Python interface in the stdlib that uses whichever is the best one available. -- Greg ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] epoll implementation
Greg Ewing wrote: Fredrik Lundh wrote: roughly speaking, epoll is kqueue for linux. There are many different select-like things around now (select, poll, epoll, kqueue -- are there others?) and random combinations of them seem to be available on any given platform. This makes writing platform-independent code that needs select-like functionality very awkward. Rather than adding yet another platform-dependent module, I'd like to see a unified Python interface in the stdlib that uses whichever is the best one available. Of course that would mean establishing which *was* the best available which, as we've seen this week, may not be easy. regards Steve -- Steve Holden +44 150 684 7255 +1 800 494 3119 Holden Web LLC/Ltd http://www.holdenweb.com Love me, love my blog http://holdenweb.blogspot.com Recent Ramblings http://del.icio.us/steve.holden ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] Proposal for a new itertools function: iwindow
Hi, in the last time, I've found myself reimplementing a generator that provides a sliding-window-view over a sequence, and I think this function is of a greater usefullness, so that it might be included in itertools. Basically, what the generator does it return all m consecutive elements from a sequence [i0, i1, ... in]. It then returns [i0, i1, i2], [i1, i2, i3], ... [in-2, in-1, in] (assuming that m = 3). In code, it looks like this: list(iwindow(range(0,5), 3)) [[0, 1, 2], [1, 2, 3], [2, 3, 4]] This list can be generated by using izip(a, a[1:], ..., a[n:]), but typing all the sequence arguments gets tedious. If a is not a sequence but a generator, tee(...) and islice has to be used. It might be possible that the windows should be padded, so that the sequence of windows starts with [pad, pad, ..., i0] and ends with [in, pad, pad, ...] list(iwindow(range(0,5), 3, pad=True)) [[None, None, 0], [None, 0, 1], [0, 1, 2], [1, 2, 3], [2, 3, 4], [3, 4, None], [4, None, None]] Additionally, the value used for padding can be specified. This makes the argument list of this function rather long, but the last two arguments are optional anyway: iwindow(iterable, window_size=3, pad = False, padding_value = None) Some open question remain: - should iwindow return lists or tuples? - what happens if the length of the iterable is smaller than the window size, and no padding is specified? Is this an error? Should the generator return no value at all or one window that is too small? I've attached a Python implementation of this function. If the function is deemed to be actually useful, I'd be happy to brush up my C and provide a C implementation along with docs and tests. best, Torsten PS: Please CC me, as I'm not subscribed to the list -- Torsten Marek [EMAIL PROTECTED] ID: A244C858 -- FP: 1902 0002 5DFC 856B F146 894C 7CC5 451E A244 C858 Keyserver: subkeys.pgp.net iwindow.py Description: application/python signature.asc Description: OpenPGP digital signature ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] epoll implementation
I wrote an epoll implementation which can be used as a drop-in replacement for parts of the select module (assuming the program is using only poll). The code can currently be used by doing: import epoll as select It was released under the Python license on sourceforge: http://sourceforge.net/projects/pyepoll Is there any interest in incorporating this into the standard python distribution? Ross ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] epoll implementation
On May 26, 2006, at 6:27 PM, Steve Holden wrote: Greg Ewing wrote: Fredrik Lundh wrote: roughly speaking, epoll is kqueue for linux. There are many different select-like things around now (select, poll, epoll, kqueue -- are there others?) and random combinations of them seem to be available on any given platform. This makes writing platform-independent code that needs select-like functionality very awkward. Rather than adding yet another platform-dependent module, I'd like to see a unified Python interface in the stdlib that uses whichever is the best one available. Of course that would mean establishing which *was* the best available which, as we've seen this week, may not be easy. I believe it's: kqueue on FreeBSD (for recent-enough versions thereof), otherwise epoll where available and nonbuggy, otherwise poll ditto, otherwise select -- that's roughly what Twisted uses for Reactor implementations, if memory serves me well. The platform- based heuristic should try to identify things this way but let the developer easily override if they know better. (One might add a Windows event loop as the best implementation available there -- Twisted does -- and GUI toolkit based event loops -- but in general that takes an abstraction level similar to Twisted's Reactor... maybe we should just repurpose that bit of Twisted?-) I don't think this is feasible for 2.5 (too late in the cycle to add major new stuff, IMHO), but it's well worth it for 2.6 (again IMHO). Alex ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] Need for Speed Sprint status
First off, good work to everyone involved. You did a tremendous job. I just hope to hell you're done, because I can't keep up! :-) It would help me enormously if someone could summarize the status and everything that went on. These are the things that would help me the most. * What are the speed diffs before/after the sprint * What was modified (summary) * What is left to do - doc - tests - code * Which branches are still planning to remain active * Lessons learned, how we can improve for the next time * Suggestions for further areas to look into improving It looks like there were a lot of additions to the string test suite, that's great. I'm not sure if the other areas touched got similar boosts to their tests. It would be good to upgrade all tests to verify corner cases of the implementation. These tests should also be documented that they are to test the implementation rather than the language spec. We don't want to write obscure tests that can't pass in other impls like Jython, IronPython, or PyPy. I will turn my amd64 box back on tomorrow and will also run Python through valgrind and pychecker when I get a chance. There are a couple of new Coverity complaints that need to be addressed. Cheers, n ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Need for Speed Sprint status
It would help me enormously if someone could summarize the status and everything that went on. These are the things that would help me the most. * What are the speed diffs before/after the sprint * What was modified (summary) * What is left to do - doc - tests - code * Which branches are still planning to remain active * Lessons learned, how we can improve for the next time * Suggestions for further areas to look into improving http://wiki.python.org/moin/NeedForSpeed/Successes http://wiki.python.org/moin/NeedForSpeed/Failures http://wiki.python.org/moin/NeedForSpeed/Deferred Raymond ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] epoll implementation
Steve Holden [EMAIL PROTECTED] writes: Rather than adding yet another platform-dependent module, I'd like to see a unified Python interface in the stdlib that uses whichever is the best one available. Of course that would mean establishing which *was* the best available which, as we've seen this week, may not be easy. There is a C aync event notification library called libevent which wraps kqueue, epoll, /dev/poll etc and provides a unified interface (See http://www.monkey.org/~provos/libevent/). Ganesan -- Ganesan Rajagopal ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
