Re: [Python-Dev] Hello, I'm the other new guy
2007/11/14, Christian Heimes <[EMAIL PROTECTED]>: > After Amaury introduced himself I've decided that I *have* to take some > time to introduce myself, too. Welcome you both to this journey. -- .Facundo Blog: http://www.taniquetil.com.ar/plog/ PyAr: http://www.python.org/ar/ ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] Summary of Tracker Issues
ACTIVITY SUMMARY (11/07/07 - 11/14/07) Tracker at http://bugs.python.org/ To view or respond to any of the issues listed below, click on the issue number. Do NOT respond to this message. 1325 open (+18) / 11600 closed (+25) / 12925 total (+43) Open issues with patches: 421 Average duration of open issues: 689 days. Median duration of open issues: 797 days. Open Issues Breakdown open 1320 (+18) pending 5 ( +0) Issues Created Or Reopened (44) ___ test_import breaks on Linux 11/09/07 http://bugs.python.org/issue1377reopened gvanrossum py3k Py3k's print() flushing problem 11/07/07 http://bugs.python.org/issue1400created wojtekwalczak py3k urllib2 302 POST 11/07/07 http://bugs.python.org/issue1401created andresriancho Interpreter cleanup: order of _PyGILState_Fini and PyInterpreter 11/07/07 http://bugs.python.org/issue1402created ronaldoussoren py_compile and compileall need unit tests11/08/07 http://bugs.python.org/issue1403created tiran py3k warnings module bug: BytesWarning: str() on a bytes instance 11/08/07 CLOSED http://bugs.python.org/issue1404created tiran patch Garbage collection not working correctly in Python 2.3 11/09/07 CLOSED http://bugs.python.org/issue1405reopened tiran Use widechar api for os.environ 11/08/07 CLOSED http://bugs.python.org/issue1406created theller py3k, patch [performance] Too many closed() checkings11/08/07 http://bugs.python.org/issue1407created wojtekwalczak py3k Inconsistence in multiply list 11/08/07 CLOSED http://bugs.python.org/issue1408created beco new keyword-only function parameters interact badly with nested 11/08/07 http://bugs.python.org/issue1409created _doublep py3k BaseHTTPServer cannot accept Unicode data11/08/07 CLOSED http://bugs.python.org/issue1410created isonno A typo in tutorial 11/09/07 CLOSED http://bugs.python.org/issue1411created falsetru test_subprocess fails on SuSE 10 11/09/07 http://bugs.python.org/issue1412created dvadasz int literal methods inaccessible 11/09/07 CLOSED http://bugs.python.org/issue1413created mykhal Fix for refleak tests11/09/07 http://bugs.python.org/issue1414created tiran py3k, patch py3k: pythonw.exe fails because std streams a missing11/10/07 CLOSED http://bugs.python.org/issue1415created tiran py3k @prop.setter decorators 11/10/07 CLOSED http://bugs.python.org/issue1416created gvanrossum patch Weakref not working properly 11/10/07 CLOSED http://bugs.python.org/issue1417created MHOOO Python/hypot.c is never used 11/10/07
[Python-Dev] Suggestions for Improvements to namedtuple
I was working on something very similar to namedtuple for a project of my own,
when it occurred to me that it's generally useful idea and maybe somebody else
was working on it too. So I searched, and found Raymond Hettinger's addition
to collections.py, and it does pretty much what I want. I have a few
suggestions which I hope are improvements. I will discuss each one, and then
at the bottom I will put my version of namedtuple. It is not based on the one
in Python SVN because it was mostly written before I found out about that one.
If my suggestions meet with approval, I could check out a copy of
collections.py and make a patch for further comment and eventual submission to
somebody with checkin privileges.
1. I think it is important that there be a way to create individual namedtuple
instances from an existing sequence that doesn't involve splitting the sequence
out into individual parameters and then re-assembling a tuple to pass to the
base tuple constructor. In my application, I'm taking database rows and
creating named tuples from them, with the named tuple type being automatically
created as appropriate. So there will be *lots* of named tuple instances
created, so for efficiency I would prefer to avoid using * to break up the
sequences generated directly by the database interface. I would like to pass
those sequences directly to the base tuple constructor.
To restore to my code the feature of being able to use individual parameters as
in collections.py, I added a classmethod to the generated classes called
__fromvalues__. This uses Signature, my other idea (next message) to convert a
call matching a procedure signature of (fieldname1, ...) into a dictionary, and
passes that dictionary into another classmethod __fromdict__ which creates a
named tuple instance from the dictionary contents.
The problem I see with this is that having to say
Point.__fromvalues__ (11, y=22)
instead of
Point (11, y=22)
is a little verbose. Perhaps there could be an __fromsequence__ instead for
the no-unpacking method of instance creation, as the most common use of
direct-from-sequence creation I think is in a more general circumstance.
2. It would be nice to be able to have default values for named tuple fields.
Using Signature it's easy to do this - I just specify a dictionary of defaults
at named tuple class creation time.
3. In my opinion __replace__ should be able to replace multiple fields. My
version takes either two parameters, same as collections.py, or a single
dictionary containing replacements.
4. I put as much of the implementation as possible of the named tuple classes
into a base class which I've called BaseNamedTuple. This defines the
classmethods __fromvalues__ and __fromdict__, as well as the regular methods
__repr__, __asdict__, and __replace__.
5. It didn't occur to me to use exec ... in so I just create the new type
using the type() function. To me, exec is a last resort, but I'm a Python
newbie so I'd be interested to hear what people have to say about this.
6. Not an improvement but a concern about my code: the generated classes and
instances have all the crucial stuff like __fields__ and __signature__ fully
read-write. It feels like those should be read-only properties. I think that
would require namedtuple to be a metaclass instead of just a function (in order
for the properties of the generated classes to be read-only). On the other
hand, I'm a recovering Java programmer, so maybe it's un-Pythonic to want stuff
to be read-only. Here I would especially appreciate any guidance more
experienced hands can offer.
And now, here is the code, together with a rudimentary example of how this
could be used to improve the "addr" functions in email.utils:
#!/usr/bin/env python
from operator import itemgetter
class BaseNamedTuple (tuple):
@classmethod
def __fromvalues__ (cls, *args, **keys):
return cls.__fromdict__ (cls.__signature__.expand_args (*args, **keys))
@classmethod
def __fromdict__ (cls, d):
return cls ([d[name] for name in cls.__fields__])
def __repr__ (self):
return self.__reprtemplate__ % self
def __asdict__ (self):
return dict (zip (self.__fields__, self))
def __replace__ (self, *args):
slist = list (self)
if len (args) == 1:
sdict = args[0]
elif len (args) == 2:
sdict = {args[0]: args[1]}
else:
raise TypeError
for key in sdict:
slist[self.__indices__[key]] = sdict[key]
return self.__class__ (slist)
def namedtuple (name, fields, defaults=None):
fields = tuple (fields)
result = type (name, (BaseNamedTuple,), {})
for i in range (len (fields)):
setattr (result, fields[i], property (itemgetter (i), None, result))
result.__fields__ = fields
result.__signature__ = Signature (fields, defaults=defaults)
result.__r
[Python-Dev] Python Library Addition: First-class Procedure Signatures
For another project (see my previous email on named tuples), I needed to
represent procedure signatures, and use them to expand arguments into the
dictionary of values that exists when execution of a procedure starts. To my
surprise, this capability didn't seem to be provided by the Python library,
even though it clearly is present within the Python system somewhere.
So I wrote a Signature class. Instances of the class represent all the
information present between the parentheses of a procedure definition.
Properties are provided to get the information out, and an expand_args method
can be called to expand arguments into a dictionary. This expand_args method
implements (if I've done it right) the argument conversion part of section
5.3.4 of the Python Reference Manual (http://docs.python.org/ref/calls.html).
I've put the code below, but I wonder if the real solution is just to create an
interface to already-existing capability? It occurs to me that the
implementation is likely to be in the interpreter itself and not written in
Python.
One possible improvement (and I'm not sure it's better, so I'm just putting it
out there): perhaps expand_args should be renamed to __call__. Then essentially
a Signature object would be a procedure whose body is just "return locals ()".
class Signature (object):
def __init__ (self, argnames,
excessargs=None, excesskeys=None, defaults=None):
self.__argnames = tuple (argnames)
self.__excessargs = excessargs
self.__excesskeys = excesskeys
if defaults is None:
defaults = {}
self.__defaults = dict (defaults)
@property
def argnames (self):
return self.__argnames
@property
def excessargs (self):
return self.__excessargs
@property
def excesskeys (self):
return self.__excesskeys
def defaults (self):
return dict (self.__defaults)
def expand_args (self, *args, **keys):
# Start with defaults
result = self.defaults ()
# Assign positional arguments
for i in range (min (len (args), len (self.argnames))):
result[self.argnames[i]] = args[i]
# Assign keyword arguments
for arg in self.argnames:
if arg in keys:
if arg in result:
raise TypeError
result[arg] = keys[arg]
del keys[arg]
# Check for missing arguments
for i in range (len (args), len (self.argnames)):
if not self.argnames[i] in result:
raise TypeError
# Excess positional arguments (*args parameter)
if self.excessargs is None:
if len (args) > len (self.argnames):
raise TypeError
else:
result[self.excessargs] = args[len (self.argnames):]
# Excess keyword arguments (**keys parameter)
if self.excesskeys is None:
if keys:
raise TypeError
else:
result[self.excesskeys] = keys
return result
Isaac Morland CSCF Web Guru
DC 2554C, x36650WWW Software Specialist
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Python Library Addition: First-class Procedure Signatures
On Nov 14, 2007 10:30 AM, Isaac Morland <[EMAIL PROTECTED]> wrote: > For another project (see my previous email on named tuples), I needed to > represent procedure signatures, and use them to expand arguments into the > dictionary of values that exists when execution of a procedure starts. To my > surprise, this capability didn't seem to be provided by the Python library, > even though it clearly is present within the Python system somewhere. > > So I wrote a Signature class. Instances of the class represent all the > information present between the parentheses of a procedure definition. > Properties are provided to get the information out, and an expand_args method > can be called to expand arguments into a dictionary. This expand_args method > implements (if I've done it right) the argument conversion part of section > 5.3.4 of the Python Reference Manual (http://docs.python.org/ref/calls.html). Have you seen http://www.python.org/dev/peps/pep-0362/? It sounds awfully similar to what you're proposing here. Collin Winter ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] CPython crash with '%o'
This is my first post, so here's a quick intro: I've recently joined the
IronPython team at Microsoft, after about 6 years as a developer on the the
.NET runtime (CLR).
I'm currently trying to make IronPython match CPython's behavior regarding some
% format string to a level of detail not documented by the spec.
(http://docs.python.org/lib/typesseq-strings.html)
While exploring to determine what CPython's behavior is, I'm hitting a
SystemError in CPython.
Given the following snippet:
class c(long):
def __oct__(self):
return '100'
x=c(5)
oct(x) # expected to print 100
'%o' % x # expected to use print '5'
It looks like in CPython, '%o' uses __int__ and so it should ignore __oct__ and
print 5.
However, I'm hitting an SystemError in CPython with this snippet:
Python 2.5.1 (r251:54863, Apr 18 2007, 08:51:08) [MSC v.1310 32 bit (Intel)]
onwin32
Type "help", "copyright", "credits" or "license" for more information.
>>> class c(long):
... def __oct__(self):
... return '100'
...
>>> x=c(5)
>>> oct(x) # expected to print 100
'100'
>>> '%o' % x # expected to use print '5'
Traceback (most recent call last):
File "", line 1, in
SystemError: \loewis\25\python\Objects\stringobject.c:4236: bad argument to
internal function
Note that if c derives from 'int' instead of 'long', everything works as
expected.
1. Can somebody confirm that '%o' should not use __oct__ and that is uses
__int__ instead, and that the correct output from ('%o' % x) is indeed 5?
2. Should I file a bug for the SystemError exception?
Thanks,
Mike
http://blogs.msdn.com/jmstall
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] CPython crash with '%o'
Mike Stall wrote:
> Traceback (most recent call last):
> File "", line 1, in
> SystemError: \loewis\25\python\Objects\stringobject.c:4236: bad argument to
> internal function
> Note that if c derives from 'int' instead of 'long', everything works as
> expected.
I'm able to reproduce the error with Python 2.5.1 and 2.5 svn on Linux.
It also happens when I overwrite __hex__ and do "%x" % c(5) for
subclasses of long but not for subclasses from int.
class c(long):
def __hex__(self):
return "100"
def __oct__(self):
return "100"
>>> x = c(5)
>>> hex(x)
'100'
>>> oct(x)
'100'
>>> "%o" % x
Traceback (most recent call last):
File "", line 1, in
SystemError: Objects/stringobject.c:4269: bad argument to internal function
>>> "%x" % x
Traceback (most recent call last):
File "", line 1, in
SystemError: Objects/stringobject.c:4269: bad argument to internal function
Objects/stringobject.c:_PyString_FormatLong(...)
...
/* To modify the string in-place, there can only be one
reference. */
if (result->ob_refcnt != 1) {
PyErr_BadInternalCall();
return NULL;
}
...
Christian
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] CPython crash with '%o'
Yes, and yes. There may be time still to fix the SystemError in 2.5.2!
On Nov 14, 2007 11:16 AM, Mike Stall <[EMAIL PROTECTED]> wrote:
>
>
>
>
> This is my first post, so here's a quick intro: I've recently joined the
> IronPython team at Microsoft, after about 6 years as a developer on the the
> .NET runtime (CLR).
>
> I'm currently trying to make IronPython match CPython's behavior regarding
> some % format string to a level of detail not documented by the spec.
> (http://docs.python.org/lib/typesseq-strings.html)
>
> While exploring to determine what CPython's behavior is, I'm hitting a
> SystemError in CPython.
>
> Given the following snippet:
>
> class c(long):
>def __oct__(self):
> return '100'
>
> x=c(5)
> oct(x) # expected to print 100
> '%o' % x # expected to use print '5'
>
> It looks like in CPython, '%o' uses __int__ and so it should ignore __oct__
> and print 5.
>
> However, I'm hitting an SystemError in CPython with this snippet:
>
> Python 2.5.1 (r251:54863, Apr 18 2007, 08:51:08) [MSC v.1310 32 bit (Intel)]
> onwin32
> Type "help", "copyright", "credits" or "license" for more information.
> >>> class c(long):
> ... def __oct__(self):
> ... return '100'
> ...
> >>> x=c(5)
> >>> oct(x) # expected to print 100
> '100'
> >>> '%o' % x # expected to use print '5'
> Traceback (most recent call last):
>File "", line 1, in
> SystemError: \loewis\25\python\Objects\stringobject.c:4236: bad argument to
> internal function
>
> Note that if c derives from 'int' instead of 'long', everything works as
> expected.
>
> 1. Can somebody confirm that '%o' should not use __oct__ and that is uses
> __int__ instead, and that the correct output from ('%o' % x) is indeed 5?
> 2. Should I file a bug for the SystemError exception?
>
> Thanks,
> Mike
> http://blogs.msdn.com/jmstall
>
>
> ___
> Python-Dev mailing list
> [email protected]
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> http://mail.python.org/mailman/options/python-dev/guido%40python.org
>
>
--
--Guido van Rossum (home page: http://www.python.org/~guido/)
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Hello, I'm the other new guy
On Nov 13, 2007 6:15 PM, Christian Heimes <[EMAIL PROTECTED]> wrote: > Hello Pythonistas and fellow core developers! > > After Amaury introduced himself I've decided that I *have* to take some > time to introduce myself, too. Obviously welcome, but you have been in the thick of things already, so I think you already knew you were welcome here. =) -Brett ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] CPython crash with '%o'
I take back the first "Yes" answer. Alas, the % operator is a mess,
especially for %d/%o/%x. For subclassed longs, it *is* attempting to
call the overridden __oct__, and this is causing the SystemError. It
looks like this code wasn't really revisited when subclassing int and
long was introduced. :-( For subclassed ints, it never looks at the
overridden __oct__. For objects that are neither, it attempts to
convert them to ints using __int__ first.
The trend in 3.0 is to completely kill __oct__ specialization and rely
only on __int__ instead. So maybe I should retract my retraction and
claim that '%o' % c(5) should indeed return '5'.
The bug stands though. There's a relatively simple fix.
--Guido
On Nov 14, 2007 11:41 AM, Guido van Rossum <[EMAIL PROTECTED]> wrote:
> Yes, and yes. There may be time still to fix the SystemError in 2.5.2!
>
>
> On Nov 14, 2007 11:16 AM, Mike Stall <[EMAIL PROTECTED]> wrote:
> >
> >
> >
> >
> > This is my first post, so here's a quick intro: I've recently joined the
> > IronPython team at Microsoft, after about 6 years as a developer on the the
> > .NET runtime (CLR).
> >
> > I'm currently trying to make IronPython match CPython's behavior regarding
> > some % format string to a level of detail not documented by the spec.
> > (http://docs.python.org/lib/typesseq-strings.html)
> >
> > While exploring to determine what CPython's behavior is, I'm hitting a
> > SystemError in CPython.
> >
> > Given the following snippet:
> >
> > class c(long):
> >def __oct__(self):
> > return '100'
> >
> > x=c(5)
> > oct(x) # expected to print 100
> > '%o' % x # expected to use print '5'
> >
> > It looks like in CPython, '%o' uses __int__ and so it should ignore __oct__
> > and print 5.
> >
> > However, I'm hitting an SystemError in CPython with this snippet:
> >
> > Python 2.5.1 (r251:54863, Apr 18 2007, 08:51:08) [MSC v.1310 32 bit (Intel)]
> > onwin32
> > Type "help", "copyright", "credits" or "license" for more information.
> > >>> class c(long):
> > ... def __oct__(self):
> > ... return '100'
> > ...
> > >>> x=c(5)
> > >>> oct(x) # expected to print 100
> > '100'
> > >>> '%o' % x # expected to use print '5'
> > Traceback (most recent call last):
> >File "", line 1, in
> > SystemError: \loewis\25\python\Objects\stringobject.c:4236: bad argument to
> > internal function
> >
> > Note that if c derives from 'int' instead of 'long', everything works as
> > expected.
> >
> > 1. Can somebody confirm that '%o' should not use __oct__ and that is uses
> > __int__ instead, and that the correct output from ('%o' % x) is indeed 5?
> > 2. Should I file a bug for the SystemError exception?
> >
> > Thanks,
> > Mike
> > http://blogs.msdn.com/jmstall
> >
> >
> > ___
> > Python-Dev mailing list
> > [email protected]
> > http://mail.python.org/mailman/listinfo/python-dev
> > Unsubscribe:
> > http://mail.python.org/mailman/options/python-dev/guido%40python.org
> >
> >
>
>
>
> --
> --Guido van Rossum (home page: http://www.python.org/~guido/)
>
--
--Guido van Rossum (home page: http://www.python.org/~guido/)
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Python Library Addition: First-class Procedure Signatures
On Nov 14, 2007 10:30 AM, Isaac Morland <[EMAIL PROTECTED]> wrote: > For another project (see my previous email on named tuples), I needed to > represent procedure signatures, and use them to expand arguments into the > dictionary of values that exists when execution of a procedure starts. To my > surprise, this capability didn't seem to be provided by the Python library, > even though it clearly is present within the Python system somewhere. > > So I wrote a Signature class. Instances of the class represent all the > information present between the parentheses of a procedure definition. > Properties are provided to get the information out, and an expand_args method > can be called to expand arguments into a dictionary. This expand_args method > implements (if I've done it right) the argument conversion part of section > 5.3.4 of the Python Reference Manual (http://docs.python.org/ref/calls.html). > As Collin already pointed out, it sounds like you want PEP 362 to get into the stdlib. I have not made a big push to try to get my existing implementation into Python 2.6/3.0, but I plan to at some point. > I've put the code below, but I wonder if the real solution is just to create > an > interface to already-existing capability? It occurs to me that the > implementation is likely to be in the interpreter itself and not written in > Python. > I don't see why a Python implementation is bad. If you make this information lazy then it is not such a big deal to have it take a little bit longer than if it was implemented in C. > One possible improvement (and I'm not sure it's better, so I'm just putting it > out there): perhaps expand_args should be renamed to __call__. Then > essentially > a Signature object would be a procedure whose body is just "return locals ()". __call__ is already used a method name for objects that can be called. -Brett ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Suggestions for Improvements to namedtuple
> for efficiency I would prefer to avoid using * to break > up the sequences generated directly by the database interface. There are some use cases that would be better served by an alternate design and others that are better served by the current design. For example, it would really suck to have to type-out an an enclosing set of parenthesis to build a named tuple explicitly: opt = StockOption((daysleft/360., strike/100.0, bool(putcall))). The alternate signature was discussed and rejected previously. See some of the details on previous python-dev posting and in the ASPN recipe discussion: at some length in the ASPN recipe: http://aspn.activestate.com/ASPN/Cookbook/Python/Recipe/500261 FWIW, the current design does work efficiently with your use case if you use itertools to apply the named tuple cast to large record sets. See the example in the docs: http://docs.python.org/dev/library/collections.html#named-tuple-factory-function-for-tuples-with-named-fields Also, fwiw, I've had good success using this starmap() approach in some performance critcal applications. Almost every class in Python could be subjected to the same discussion. Why doesn't every class and function accept a tuple of args instead of separate, serial arguments. The * operator and starmap() function were provided specifically to address the inter-conversion between the two approaches. > It would be nice to be able to have default values This came-up before, but I'll take another look at it this weekend. If it significantly complicates the API or impacts performance, then it is a non-starter; otherwise, I'll put together an implementation and float it on ASPN for comments. > In my opinion __replace__ should be able to replace multiple fields. This may be do-able. Will take a look at it. The current design is easily extended for this without impacting performance. The question is whether is makes sense in normal applications to do multiple replacements at once. FWIW, the str.replace function only has one target/replacement pair. > To me, exec is a last resort It was a last resort here also. Earlier versions of the recipe used type() but they were less flexible, slower, had crummy error reporting, and were much less clear. The exec version has the advantage of being able to print-out the template so you can see *exactly* what it is doing. I ended-up putting in a lot f field validation to prevent injection attacks (they are also they to provide more actionable error messages). > crucial stuff like __fields__ and __signature__ fully > read-write. It feels like those should be read-only properties. > ... On the other hand, I'm a recovering Java programmer, Welcome to the Python world of consenting adults. These fields could be made read-only but that is just a matter of taste. Looking through the standard library, you'll see making attributes read-only is not the typical practice. If users demand it, I don't see a problem with it, but until then, I don't see a reason to unnecessarily complicate and slow-down the code. FWIW, you're welcome to email me directly. I've been getting tons of user feedback and have already incorporated the better suggestions. Also, the recipe itself is a combination of the best of prior attempts. Unfortunately, each of several commenters revisit everything from scratch and propose their own complete rewrites that ignore the requests and commments of all those who preceded them. Fortunately, we're homing in on a darned good tool. Raymond ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] smlop - maintaned ? - what is the status of this module ?
Hi I have some scripts where this module is used - asyncore, xmlrpc, sgmlop. Some errors are arriving - memory errors reported by msvc2005 malloc/free. Where I should report bugs ? my env: py-2.5- partial cvs, sgmlop either from Frideric-s site or pyxml - same effect. tia, mak ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] smlop - maintaned ? - what is the status of this module ?
> I have some scripts where this module is used - asyncore, xmlrpc, sgmlop. > Some errors are arriving - memory errors reported by msvc2005 malloc/free. > > Where I should report bugs ? > > my env: py-2.5- partial cvs, sgmlop either from Frideric-s site or pyxml > - same effect. sgmlop is not part of the Python distribution, so you should report errors directly to Fredrik Lundh (sp!). Notice, however, that you should use msvc2003, not 2005, to compile it. Regards, Martin ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Suggestions for Improvements to namedtuple
> crucial stuff like __fields__ ... fully > read-write. On further thought, I think this is a good idea. Nothing good can come from writing to this class variable. Suggestion is checked-in in rev 58971. Curiously, it was already documented as read-only (I took the time machine out for a spin). Raymond ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] smlop - maintaned ? - what is the status of this module ?
On Nov 14, 2007 1:52 PM, Grzegorz Makarewicz <[EMAIL PROTECTED]> wrote: > Hi > > I have some scripts where this module is used - asyncore, xmlrpc, sgmlop. > Some errors are arriving - memory errors reported by msvc2005 malloc/free. > > Where I should report bugs ? http://bugs.python.org/ -Brett ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Suggestions for Improvements to namedtuple
> In my opinion __replace__ should be able to replace multiple fields. This suggestion is accepted and checked-in. See revision 58975. Surprisingly, the required signature change results in improved clarity. This was an all around win. Raymond ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] Alternative C implementation idea for collections.deque
Many years ago I implemented a deque type in C (for use in C programs)
using a single dynamically sized array as the underling type, along
with an extra integer to designate where the "start" of the list is.
Somehow, I had always imagined that Python's implementation worked the
same way, but was just looking at the code and found that it uses
doubly-linked lists of 62-item blocks.
I just wanted to toss my approach out there to see if it might offer
better performance and/or simpler code (my pure-C implementation is
only 80 lines of code). If there's interest, I'd be happy to work it
up into a patch. I could also do some performance tests if others
have specific use cases they'd like to see tested. I see the
advantages as follows:
- __getitem__ would take O(1) time instead of O(n)
- Dirt simple implementation
Compared to the existing code, memory allocation/deallocation is less
frequent but more expensive when necessary, working out to O(1)
amortized time (same as the current implementation).
The basic idea is to use a single dynamically sized array (much like
Python's list type). An extra integer points to the start of the
deque within the array, and the data implicitly wraps around the end
of the list back to the beginning. Here's the basic data structure:
struct deque
{
PyObject **obs;
int max_len;/* Number of objects we can store in the deque
before having to reallocate */
int len;/* Number of objects currently in the deque */
int start; /* Where in obs the objects starts */
}
When len == max_len and the user adds an element, we reallocate obs.
Much like Python's list type, this can be done in such a way that
memory allocation takes O(1) amortized time (e.g., by doubling
max_len)..
The start of the list is always obs[start] and the last element of the
list is at obs[(start + len - 1) % max_len]. The nth element of the
list is obs[(start + n) % max_len].
I think an example would be helpful. For illustrative purposes,
suppose we start with max_len=4. Each line is followed by a picture
of the data structure's contents:
>>> x = deque()
{obs = {NULL, NULL, NULL, NULL}, max_len = 4, len = 0, start = 0}
>>> x.extend((1,2,3,4))
{obs = {1, 2, 3, 4}, max_len = 4, len = 4, start = 0}
>>> x.popleft()
{obs = {NULL, 2, 3, 4}, max_len = 4, len = 3, start = 1}
>>> x.append(5)
{obs = {5, 2, 3, 4}, max_len = 4, len = 4, start = 1}
>>> x.popleft()
{obs = {5, NULL, 3, 4}, max_len = 4, len = 3, start = 2}
>>> x.pop()
{obs = {NULL, NULL, 3, 4}, max_len = 4, len = 2, start = 2}
Comments? Thoughts? Would a patch be welcome?
--
Daniel Stutzbach, Ph.D. President, Stutzbach Enterprises LLC
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Alternative C implementation idea for collections.deque
On Nov 15, 2007 12:57 AM, Raymond Hettinger <[EMAIL PROTECTED]> wrote: > FWIW, my development time is now somewhat limited > and I prefer to spend it on a todo list that has been around > for some time. I dread throwing that time away and > spending it on reviewing your patch, timing tons of test > cases, arguing the merits of alternate approaches, and ending-up > with substantially the same functionality that we already have. I respect that. I won't waste either of our time, then. Cheers, -- Daniel Stutzbach, Ph.D. President, Stutzbach Enterprises LLC ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Alternative C implementation idea for collections.deque
From: "Daniel Stutzbach" <[EMAIL PROTECTED]> > Many years ago I implemented a deque type in C (for use in C programs) > using a single dynamically sized array as the underling type, The approach was not used so we could avoid data movement associated with re-sizing. > I see the > advantages as follows: > > - __getitem__ would take O(1) time instead of O(n) While nice, that is somewhat at odds with how deques are used. The current implementation has O(1) access time with a tiny constant for access near the end-points. The only reason that __getitem__ was put in was to support fast access to the head and tail without actually popping the value. That it can access the middle was incidental and imho optimizing in the middle access would only encourage mis-use of the data structure. > Compared to the existing code, memory allocation/deallocation is less > frequent but more expensive when necessary, For most use cases, the current version rolls around with no memory allocations (reusable blocks are kept on a free list). > Comments? Thoughts? Would a patch be welcome? Might as well put it in the tracker for history/research purposes, but I think you're barking up the wrong tree. Deques are not about accessing the nth item in the middle. For their purpose, the current design works really well (the blocks are sized to fit in cache lines and no mass movements are necessary when the size changes -- the data never moves). If you're going to spend time, why not put it into developing a data structure we don't already have (red-black trees or some such). FWIW, my development time is now somewhat limited and I prefer to spend it on a todo list that has been around for some time. I dread throwing that time away and spending it on reviewing your patch, timing tons of test cases, arguing the merits of alternate approaches, and ending-up with substantially the same functionality that we already have. The big win for deques was getting the O(1) append/pop on each end. That gave a huge algorithmic speed boost to the Queue module and a number of other tools that were using lists for deque-style operations. Raymond ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
