Re: [Python-Dev] bytes.from_hex()
> "Martin" == Martin v Löwis <[EMAIL PROTECTED]> writes:
Martin> Stephen J. Turnbull wrote:
Bengt> The characters in b could be encoded in plain ascii, or
Bengt> utf16le, you have to know.
>> Which base64 are you thinking about? Both RFC 3548 and RFC
>> 2045 (MIME) specify subsets of US-ASCII explicitly.
Martin> Unfortunately, it is ambiguous as to whether they refer to
Martin> US-ASCII, the character set, or US-ASCII, the encoding.
True for RFC 3548, but the authors of RFC 2045 clearly had the
encoding in mind, since they depend on RFC 822.
Martin> It appears that RFC 3548 talks about the character set
Martin> only:
OK, although RFC 3548 cites RFC 20 (!) as its source for US-ASCII,
which clearly has bytes (though not necessarily octets) in mind, it
doesn't actually restrict base encoding to be a subset of US-ASCII.
On the other hand, RFC 3548 doesn't define "base64" (or any other base
encoding), it simply provides a set of requirements that a conforming
implementation must satisfy. Python can therefore choose to define
its base64 as a bytes->bytes codec, with the alphabet drawn from
US-ASCII interpreted as encoding.
I would definitely prefer that, as png_image = unicode.encode('base64')
violates MAL's intuitive schema for the method.
Martin> For an example where base64 is *not* necessarily
Martin> ASCII-encoded, see the "binary" data type in XML
Martin> Schema. There, base64 is embedded into an XML document,
Martin> and uses the encoding of the entire XML document. As a
Martin> result, you may get base64 data in utf16le.
I'll have to take a look. It depends on whether base64 is specified
as an octet-stream to Unicode stream transformation or as an embedding
of an intermediate representation into Unicode. Granted, defining the
base64 alphabet as a subset of Unicode seems like the logical way to
do it in the context of XML.
P.S. My apologies for munging your name in the To: header. I'm
having problems with my MUA.
--
School of Systems and Information Engineering http://turnbull.sk.tsukuba.ac.jp
University of TsukubaTennodai 1-1-1 Tsukuba 305-8573 JAPAN
Ask not how you can "do" free software business;
ask what your business can "do for" free software.
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Removing Non-Unicode Support?
Guido van Rossum wrote: > On 2/19/06, Jeff Rush <[EMAIL PROTECTED]> wrote: > [Quoting Neal Norwitz] > >>>I've heard of a bunch of people using --disable-unicode. I'm not sure >>>if it's curiosity or if there are really production builds without >>>unicode. Ask this on c.l.p too. >> > Do you know of any embedded platform that doesn't have unicode support > as a requirement? Python runs fine on Nokia phones running Symbian, > where *everything* is a Unicode string. 1. PalmOS, at least the last time I was involved with it. Python on a Palm is a very tight fit. 2. "GM862 Cellular Module with Python Interpreter" http://tinyurl.com/jgxz These may be dimishing markets as memory capacity increases and I wouldn't argue adding compile flags for such at this late date, but if the flags are already there, perhaps the slight inconvenience to Python-internal developers is worth it. Hey, perhaps dropping out Unicode support is not a big win - I just know it is useful at times to have a collection of flags to drop out floating point, complex arithmetic, language parsing and such for memory-constrained cases. -Jeff ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] s/bytes/octet/ [Was:Re: bytes.from_hex() [Was: PEP 332 revival in coordination with pep 349?]]
On Sat, 18 Feb 2006 09:59:38 +0100, =?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=
<[EMAIL PROTECTED]> wrote:
>Aahz wrote:
>> The problem is that they don't understand that "Martin v. L?wis" is not
>> Unicode -- once all strings are Unicode, this is guaranteed to work.
Well, after all the "string" literal escapes that were being used
to define byte values are all rewritten, yes, I'll believe the guarantee ;-)
(BTW, are there plans for migration tools?)
Ok, now back to the s/bytes/octet/ topic:
>
>This specific call, yes. I don't think the problem will go away as long
>as both encode and decode are available for both strings and byte
>arrays.
>
>> While it's not absolutely true, my experience of watching Unicode
>> confusion is that the simplest approach for newbies is: encode FROM
>> Unicode, decode TO Unicode.
>
>I think this is what should be in-grained into the library, also. It
>shouldn't try to give additional meaning to these terms.
>
Thinking about bytes recently, it occurs to me that bytes are really not
intrinsically
numeric in nature. They don't necessarily represent uint8's. E.g., a binary
file is
really a sequence of bit octets in its most primitive and abstract sense.
So I'm wondering if we shouldn't have an octet type analogous to unicode, and
instances of octet
would be vectors of octets as abstract 8-bit bit vectors, like instances of
unicode are vectors of abstract characters.
If you wanted integers you could map ord for integers guaranteed to be in
range(256).
The constructor would naturally take any suitable integer sequence so
octet([65,66,67]) would work.
In general, all encode methods would produce an octet instance, e.g.
unicode.encode.
octet.decode(octet_instance, 'src_encoding') or
octet_instance.decode('src_encoding') would do
all the familiar character code sequence decoding,
e.g., octet.decode(oseq, 'utf-8') or oseq.decode('utf-8') to make a unicode
instance.
Going from unicode, unicode.encode(uinst, 'utf-8') or uinst.encode('utf-8')
would produce an octet instance.
I think this is conceptually purer than the current bytes idea, since the
result really has no arithmetic significance.
Also, ord would work on a length-one octet instance, and produce the unsigned
integer value you'd expect, but would fail
if not length-one, like ord on unicode (or current str).
Thus octet would replace bytes as the binary info container, and would not have
any presumed aritmetic
significance, either as integer or as
character-of-current-source-encoding-inferred-from-integer-value-as-ord.
To get a text representation of octets, hex is natural, e.g., octet('6162
6380') # spaces ignored
so repr(octet('a deaf bee')) => "octet('adeafbee')" and
octet('616263').decode('ascii') => u'abc' and
back: u'abc.encode('ascii') => octet('616263'). The base64 codec looks
conceptually cleaner too, so long
as you keep in mind base64 as a character subset of unicode and the name of the
transformation function pair.
octet('616263').decode('base64') => u'YWJj\n' # octets -> characters
u'YWJj\n'.encode('base64') => octet('616263') # characters -> octets
If you wanted integer-nature bytes, you could have octet codecs for uint8 and
int8, e.g., octseq.decode('int8')
could produce a list of signed integers all in range(-128,128). Or maybe
map(dec_int8, octseq). The array
module could easily be a target for octet.decode, e.g.,
octseq.decode('array_B') or octet.decode(octseq, 'array_B'),
and octet(array_instance) the other way.
Likewise, other types could be destination for octet.decode.
E.g., if you had an abstraction for a display image one could have 'gif' and
'png' and 'bmp' etc
be like 'cp437', 'latin-1', and 'utf-8' etc are for decoding octest to unicode,
and write stuff like
o_seq = open('pic.gif','rb') # makes octet instance
img = o_seq.decode('gif89') # => img is abstract, internally represented
suitably but hidden, like unicode.
open('pic.png', 'wb').write(img.encode('png'))
UIAM PIL has this functionality, if not as encode/decode methods.
Similarly, there could be an abstract archive container, and you could have
arch = open('tree.tgz','rb').decode('tgz') # => might do lazy things
waiting for encode
egg_octets = arch.encode('python_egg') # convert to egg format?? (just
hand-waving ;-)
Probably all it would take is to wrap some things in abstract-container (AC)
types, to enforce the protocol.
Image(octet_seq, 'gif') might produce an AC that only saved a (octet_seq,
'gif') internally, or it might
do eager conversion per optional additional args. Certainly .bmp without rle
can be hugely wasteful.
For flexibility like eager vs not, or perhaps returning an iterator instead of
a byte sequence,
I guess the encode/decode signatures should be (enc, *args, **kw) and pass
those things on to
the worker functions? An abstract container could have a "pack" codec to do
serial composition/decomposition.
I'm sure Mal has all this stuff one way or another, but I w
Re: [Python-Dev] Proposal: defaultdict
On 2/19/06, Steve Holden <[EMAIL PROTECTED]> wrote: > > You are missing the rationale of the PEP process. The point is > > *not* documentation. The point of the PEP process is to channel > > and collect discussion, so that the BDFL can make a decision. > > The BDFL is not bound at all to the PEP process. > > > > To document things, we use (or should use) documentation. > > > > > One could wish this ideal had been the case for the import extensions > defined in PEP 302. (A bit off-topic, but that hit home, so I'll reply...) Agreed, and it's my fault they weren't, to some extent. I did try to find a suitable place, but the import docs are generally fairly scattered, and there wasn't a particularly good place to put the changes. Any suggestions would be gratefully accepted... Paul. ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Removing Non-Unicode Support?
Jeff Rush wrote: > Guido van Rossum wrote: >> On 2/19/06, Jeff Rush <[EMAIL PROTECTED]> wrote: >> [Quoting Neal Norwitz] >> I've heard of a bunch of people using --disable-unicode. I'm not sure if it's curiosity or if there are really production builds without unicode. Ask this on c.l.p too. >>> >> Do you know of any embedded platform that doesn't have unicode support >> as a requirement? Python runs fine on Nokia phones running Symbian, >> where *everything* is a Unicode string. > > 1. PalmOS, at least the last time I was involved with it. Python on a > Palm is a very tight fit. > > > 2. "GM862 Cellular Module with Python Interpreter" > http://tinyurl.com/jgxz > > These may be dimishing markets as memory capacity increases and I > wouldn't argue adding compile flags for such at this late date, but if > the flags are already there, perhaps the slight inconvenience to > Python-internal developers is worth it. > > Hey, perhaps dropping out Unicode support is not a big win - I just know > it is useful at times to have a collection of flags to drop out floating > point, complex arithmetic, language parsing and such for > memory-constrained cases. These switches make the code less maintainable. I'm not even talking about the testing overhead. I'd say that the parties interested in non-Unicode versions of Python should maintain these branches of Python. Dito for other stripped down versions. Note that this does not mean that we should forget about memory consumption issues. It's just that if there's only marginal interest in certain special builds of Python, I don't see the requirement for the Python core developers to maintain them. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Feb 20 2006) >>> Python/Zope Consulting and Support ...http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ...http://python.egenix.com/ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] defaultdict proposal round three
I'm withdrawing the last proposal. I'm not convinced by the argument that __contains__ should always return True (perhaps it should also insert the value?), nor by the complaint that a holy invariant would be violated (so what?). But the amount of discussion and the number of different viewpoints present makes it clear that the feature as I last proposed would be forever divisive. I see two alternatives. These will cause a different kind of philosophical discussion; so be it. I'll describe them relative to the last proposal; for those who wisely skipped the last thread, here's a link to the proposal: http://mail.python.org/pipermail/python-dev/2006-February/061261.html. Alternative A: add a new method to the dict type with the semantics of __getattr__ from the last proposal, using default_factory if not None (except on_missing is inlined). This avoids the discussion about broken invariants, but one could argue that it adds to an already overly broad API. Alternative B: provide a dict subclass that implements the __getattr__ semantics from the last proposal. It could be an unrelated type for all I care, but I do care about implementation inheritance since it should perform just as well as an unmodified dict object, and that's hard to do without sharing implementation (copying would be worse). Parting shots: - Even if the default_factory were passed to the constructor, it still ought to be a writable attribute so it can be introspected and modified. A defaultdict that can't change its default factory after its creation is less useful. - It would be unwise to have a default value that would be called if it was callable: what if I wanted the default to be a class instance that happens to have a __call__ method for unrelated reasons? Callability is an elusive propperty; APIs should not attempt to dynamically decide whether an argument is callable or not. - A third alternative would be to have a new method that takes an explicit defaut factory argument. This differs from setdefault() only in the type of the second argument. I'm not keen on this; the original use case came from an example where the readability of d.setdefault(key, []).append(value) was questioned, and I'm not sure that d.something(key, list).append(value) is any more readable. IOW I like (and I believe few have questioned) associating the default factory with the dict object instead of with the call site. Let the third round of the games begin! -- --Guido van Rossum (home page: http://www.python.org/~guido/) ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] bytes.from_hex() [Was: PEP 332 revival in coordination with pep 349?]
On Sat, 18 Feb 2006 23:33:15 +0100, Thomas Wouters <[EMAIL PROTECTED]> wrote:
>On Sat, Feb 18, 2006 at 01:21:18PM +0100, M.-A. Lemburg wrote:
>
[...]
>> > - The return value for the non-unicode encodings depends on the value of
>> >the encoding argument.
>
>> Not really: you'll always get a basestring instance.
>
But actually basestring is weird graft of semantic apples and empty bags IMO.
unicode is essentially an abstract character vector type,
and str is an abstract binary octet vector type having nothing to do with
characters
except by inferential association with an encoding.
>Which is not a particularly useful distinction, since in any real world
>application, you have to be careful not to mix unicode with (non-ascii)
>bytestrings. The only way to reliably deal with unicode is to have it
>well-contained (when migrating an application from using bytestrings to
>using unicode) or to use unicode everywhere, decoding/encoding at
>entrypoints. Containment is hard to achieve.
>
>> Still, I believe that this is an educational problem. There are
>> a couple of gotchas users will have to be aware of (and this is
>> unrelated to the methods in question):
>>
>> * "encoding" always refers to transforming original data into
>> a derived form
ISTM encoding separates type information from the source and sets it aside
as the identity of the encoding, and renders the data in a composite of
more primitive types, octets being the most primitive short of bits.
>>
>> * "decoding" always refers to transforming a derived form of
>> data back into its original form
Decoding of a composite of primitives requires additional separate information
(namely identification of the encoding) to create a higher composite type.
>>
>> * for Unicode codecs the original form is Unicode, the derived
>> form is, in most cases, a string
You mean a str instance, right? Where the original type as character vector
is gone. That's not a string in the sense of character string.
>>
>> As a result, if you want to use a Unicode codec such as utf-8,
>> you encode Unicode into a utf-8 string and decode a utf-8 string
>> into Unicode.
s/string/str instance/
>>
>> Encoding a string is only possible if the string itself is
>> original data, e.g. some data that is supposed to be transformed
>> into a base64 encoded form.
note what base64 really is for. It's essence is to create a _character_ sequence
which can succeed in being encoded as ascii. The concept of base64 going
str->str
is really a mental shortcut for s_str.decode('base64').encode('ascii'), where
3 octets are decoded as code for 4 characters modulo padding logic.
>>
>> Decoding Unicode is only possible if the Unicode string itself
>> represents a derived form, e.g. a sequence of hex literals.
Again, it's an abbreviation, e.g.
print u'4cf6776973'.encode('hex_chars_to_octets').decode('latin-1')
Should print Löwis
>
>Most of these gotchas would not have been gotchas had encode/decode only
>been usable for unicode encodings.
>
>> > That is why I disagree with the hypergeneralization of the encode/decode
>> > methods
>[..]
>> That's because you only look at one specific task.
>
>> Codecs also unify the various interfaces to common encodings
>> such as base64, uu or zip which are not Unicode related.
I think the trouble is that these view the transformations as octets->octets
whereas IMO decoding should always result in a container type that knows what
it is
semantically without association with external use-this-codec information. IOW,
octets.decode('zip') -> archive
archive.encode('bzip') -> octets
You could even subclass octet to make archive that knows it's an octet vector
representing a decoded zip, so it can have an encode method that could
(specifying 'zip' again) encode itself back to the original zip, or an alternate
method to encode itself as something else, which you couldn't do from plain
octets
without specifying both transformations at once. (hence the .recode idea, but I
don't
think that is as pure. The constructor for the container type could also be
used, like
Archive(octets, 'zip') analogous to unicode('abc', 'ascii')
IOW
octets + decoding info -> container type instance
container type instance + encoding info -> octets
>
>No, I think you misunderstand. I object to the hypergeneralization of the
>*encode/decode methods*, not the codec system. I would have been fine with
>another set of methods for non-unicode transformations. Although I would
>have been even more fine if they got their encoding not as a string, but as,
>say, a module object, or something imported from a module.
>
>Not that I think any of this matters; we have what we have and I'll have to
>live with it ;)
Probably.
BTW, you may notice I'm saying octet instead of bytes. I have another post on
that,
arguing that the basic binary information type should be octet, since binary
files
are made of octets that have no instrinsic numerical or character significance.
See other post i
Re: [Python-Dev] buildbot is all green
Guido van Rossum wrote: > FWIW, it looks like all the sample templates are still wasting a lot > of horizontal space in the first two columns the second is almost > always empty. Perhaps the author of the change could be placed *below* > the timestamp instead of next to it? Also for all practical purposes > we can probably get rid of the seconds in the timestamp. So far the cosmetic changes have been done purely in CSS, implementing the above would (AFAICT) require modifying the buildbot waterfall display HTML generation. Something that's been shied away from thus far. -- Benji York ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] (-1)**(1/2)==1?
Hello! My name is Jonathan, i´m new with Python. I try this in the command line: >>> (-1)**(1/2) 1 This is wrong, i think it must throw an exception. What do you think? Bye. Jonathan. ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Proposal: defaultdict
On Sun, Feb 19, 2006, Josiah Carlson wrote: > > I agree, there is nothing perfect. But at least in all of my use-cases, > and the majority of the ones I've seen 'in the wild', my previous post > provided an implementation that worked precisely like desired, and > precisely like a regular dictionary, except when accessing a > non-existant key via: value = dd[key] . __contains__, etc., all work > exactly like they do with a non-defaulting dictionary. Iteration via > popitem(), pop(key), items(), iteritems(), __iter__, etc., all work the > way you would expect them. This is the telling point, IMO. My company makes heavy use of a "default dict" (actually, it's a "default class" because using constants as the lookup keys is mostly what we do and the convenience of foo.bar is compelling over foo['bar']). Anyway, our semantics are as Josiah outlines, and I can't see much use case for the alternatives. Those of you arguing something different: do you have a real use case (that you've implemented in real code)? -- Aahz ([EMAIL PROTECTED]) <*> http://www.pythoncraft.com/ "19. A language that doesn't affect the way you think about programming, is not worth knowing." --Alan Perlis ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] (-1)**(1/2)==1?
Jonathan Barbero wrote: > Hello! > My name is Jonathan, i´m new with Python. > >I try this in the command line: > >>>> (-1)**(1/2) > 1 > >This is wrong, i think it must throw an exception. > What do you think? >>> 1/2 0 >>> (-1)**0 1 It's fine. If you want to get a floating point result from dividing, make one of the two numbers a float: >>> 1.0/2 0.5 >>> Georg ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] (-1)**(1/2)==1?
Georg, Please do not respond to off-topic posts on python-dev without redirecting them to comp.lang.python (or other suitable place). Thanks! On Mon, Feb 20, 2006, Georg Brandl wrote: > > Jonathan Barbero wrote: >> Hello! >> My name is Jonathan, i?m new with Python. >> >>I try this in the command line: >> >>>>> (-1)**(1/2) >> 1 >> >>This is wrong, i think it must throw an exception. >> What do you think? > 1/2 > 0 (-1)**0 > 1 > > It's fine. > > If you want to get a floating point result from dividing, > make one of the two numbers a float: > 1.0/2 > 0.5 -- Aahz ([EMAIL PROTECTED]) <*> http://www.pythoncraft.com/ "19. A language that doesn't affect the way you think about programming, is not worth knowing." --Alan Perlis ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] (-1)**(1/2)==1?
2006/2/20, Jonathan Barbero <[EMAIL PROTECTED]>: > Hello! > My name is Jonathan, i´m new with Python. Hello Jonathan. This list is only for developing Python itself, not for developing in Python. You should address this kind of question in comp.lang.python (available as a newsgroup and a mailing list), see here for instructions: http://www.python.org/community/lists.html >I try this in the command line: > >>>> (-1)**(1/2) > 1 > >This is wrong, i think it must throw an exception. > What do you think? It's OK, because (1/2) is zero, not 0.5. >>> 1/2 0 Regards, .Facundo Blog: http://www.taniquetil.com.ar/plog/ PyAr: http://www.python.org/ar/ ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] defaultdict proposal round three
On Feb 20, 2006, at 5:41 AM, Guido van Rossum wrote: ... > Alternative A: add a new method to the dict type with the semantics of > __getattr__ from the last proposal, using default_factory if not None > (except on_missing is inlined). This avoids the discussion about > broken invariants, but one could argue that it adds to an already > overly broad API. > > Alternative B: provide a dict subclass that implements the __getattr__ > semantics from the last proposal. It could be an unrelated type for > all I care, but I do care about implementation inheritance since it > should perform just as well as an unmodified dict object, and that's > hard to do without sharing implementation (copying would be worse). "Let's do both!"...;-). Add a method X to dict as per A _and_ provide in collections a subclass of dict that sets __getattr__ to X and also takes the value of default_dict as the first mandatory argument to __init__. Yes, mapping is a "fat interface", chock full of convenience methods, but that's what makes it OK to add another, when it's really convenient; and nearly nobody's been arguing against defaultdict, only about the details of its architecture, so the convenience of this X can be taken as established. As long as DictMixin changes accordingly, the downsides are small. Also having a collections.defaultdict as well as method X would be my preference, for even more convenience. From my POV, either or both of these additions would be an improvement wrt 2.4 (as would most of the other alternatives debated here), but I'm keen to have _some_ alternative get in, rather than all being blocked out of 2.5 by "analysis paralysis". Alex ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Does eval() leak?
On 2/16/06, John Marshall <[EMAIL PROTECTED]> wrote:
> Hi,
>
> Should I expect the virtual memory allocation
> to go up if I do the following?
> -
> raw = open("data").read()
> while True:
> d = eval(raw)
> -
>
> I would have expected the memory allocated to the
> object referenced by d to be deallocated, garbage
> collected, and reallocated for the new eval(raw)
> results, assigned to d.
>
> The file contains a large, SIMPLE (no self refs; all
> native python types/objects) dictionary (>300K).
You're probably running into the problem that the concrete parse tree
built up by the parser is rather large. While the memory used for that
tree is freed to Python's malloc pool, thus making it available for
other allocations by the same process, it is likely that the VM
allocation for the process will permanently go up.
When I try something like this (*) I see the virtual memory size go up
indefinitely with Python 2.3.5, but not with Python 2.4.1 or
2.5(head). Even so, the problem may be fragmentation instead of a
memory leak; fragmentation problems are even harded to debug than
leaks (since they depend on the heuristics applied by the platform's
malloc implementation).
You can file a bug for 2.3 but unless you also provide a patch it's
unlikely to be fixed; the memory allocation code was revamped
significantly for 2.4 so there's no simple backport of the fix
available.
(*)
d = {}
for i in range(10): d[repr(i)] = i
s = str(d)
while 1: x = eval(s); print 'x'
--
--Guido van Rossum (home page: http://www.python.org/~guido/)
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] defaultdict proposal round three
[GvR] > I'm not convinced by the argument > that __contains__ should always return True Me either. I cannot think of a more useless behavior or one more likely to have unexpected consequences. Besides, as Josiah pointed out, it is much easier for a subclass override to substitute always True return values than vice-versa. > Alternative A: add a new method to the dict type with the semantics of > __getattr__ from the last proposal Did you mean __getitem__? If not, then I'm missing what the current proposal is. >, using default_factory if not None > (except on_missing is inlined). This avoids the discussion about > broken invariants, but one could argue that it adds to an already > overly broad API. +1 I prefer this approach over subclassing. The mental load from an additional method is less than the load from a separate type (even a subclass). Also, avoidance of invariant issues is a big plus. Besides, if this allows setdefault() to be deprecated, it becomes an all-around win. > - Even if the default_factory were passed to the constructor, it still > ought to be a writable attribute so it can be introspected and > modified. A defaultdict that can't change its default factory after > its creation is less useful. Right! My preference is to have default_factory not passed to the constructor, so we are left with just one way to do it. But that is a nit. > - It would be unwise to have a default value that would be called if > it was callable: what if I wanted the default to be a class instance > that happens to have a __call__ method for unrelated reasons? > Callability is an elusive propperty; APIs should not attempt to > dynamically decide whether an argument is callable or not. That makes sense, though it seems over-the-top to need a zero-factory for a multiset. An alternative is to have two possible attributes: d.default_factory = list or d.default_value = 0 with an exception being raised when both are defined (the test is done when the attribute is created, not when the lookup is performed). Raymond ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] (-1)**(1/2)==1?
>>> 1/2 0 >>> (-1) ** (1./2) Traceback (most recent call last): File "", line 1, in ? ValueError: negative number cannot be raised to a fractional power Regards, Michael On 2/20/06, Jonathan Barbero <[EMAIL PROTECTED]> wrote: > Hello! > My name is Jonathan, i´m new with Python. > >I try this in the command line: > >>>> (-1)**(1/2) > 1 > >This is wrong, i think it must throw an exception. > What do you think? > > Bye. > Jonathan. > > > ___ > Python-Dev mailing list > [email protected] > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > http://mail.python.org/mailman/options/python-dev/michael.walter%40gmail.com > > > ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] defaultdict proposal round three
On Mon, 20 Feb 2006 05:41:43 -0800, "Guido van Rossum" <[EMAIL PROTECTED]>
wrote:
>I'm withdrawing the last proposal. I'm not convinced by the argument
>that __contains__ should always return True (perhaps it should also
>insert the value?), nor by the complaint that a holy invariant would
>be violated (so what?).
>
>But the amount of discussion and the number of different viewpoints
>present makes it clear that the feature as I last proposed would be
>forever divisive.
>
>I see two alternatives. These will cause a different kind of
>philosophical discussion; so be it. I'll describe them relative to the
>last proposal; for those who wisely skipped the last thread, here's a
>link to the proposal:
>http://mail.python.org/pipermail/python-dev/2006-February/061261.html.
>
>Alternative A: add a new method to the dict type with the semantics of
>__getattr__ from the last proposal, using default_factory if not None
>(except on_missing is inlined). This avoids the discussion about
>broken invariants, but one could argue that it adds to an already
>overly broad API.
>
>Alternative B: provide a dict subclass that implements the __getattr__
>semantics from the last proposal. It could be an unrelated type for
>all I care, but I do care about implementation inheritance since it
>should perform just as well as an unmodified dict object, and that's
>hard to do without sharing implementation (copying would be worse).
>
>Parting shots:
>
>- Even if the default_factory were passed to the constructor, it still
>ought to be a writable attribute so it can be introspected and
>modified. A defaultdict that can't change its default factory after
>its creation is less useful.
>
>- It would be unwise to have a default value that would be called if
>it was callable: what if I wanted the default to be a class instance
>that happens to have a __call__ method for unrelated reasons?
You'd have to put it in a lambda: thing_with_unrelated__call__method
>Callability is an elusive propperty; APIs should not attempt to
>dynamically decide whether an argument is callable or not.
>
>- A third alternative would be to have a new method that takes an
>explicit defaut factory argument. This differs from setdefault() only
>in the type of the second argument. I'm not keen on this; the original
>use case came from an example where the readability of
>
> d.setdefault(key, []).append(value)
>
>was questioned, and I'm not sure that
>
> d.something(key, list).append(value)
>
>is any more readable. IOW I like (and I believe few have questioned)
>associating the default factory with the dict object instead of with
>the call site.
>
>Let the third round of the games begin!
>
Sorry if I missed it, but is it established that defaulting lookup
will be spelled the same as traditional lookup, i.e. d[k] or d.__getitem__(k) ?
IOW, are default-enabled dicts really going to be be passed
into unknown contexts for use as a dict workalike? I can see using on_missing
for external side effects like logging etc., or _maybe_ modifying the dict with
a known separate set of keys that wouldn't be used for the normal purposes of
the dict.
ISTM a defaulting dict could only reasonably be passed into contexts that
expected it,
but that could still be useful also. How about d = dict() for a totally normal
dict,
and d.defaulting to get a view that uses d.default_factory if present? E.g.,
d = dict()
d.default_factory = list
for i,name in enumerate('Eeny Meeny Miny Moe'.split()): # prefix insert order
d.defaulting[name].append(i) # or hoist d.defaulting => dd[name].append(i)
Maybe d.defaulting could be a descriptor?
If the above were done, could d.on_missing be independent and always active if
present? E.g.,
d.on_missing = lambda self, key: self.__setitem__(key, 0) or 0
would be allowed to work on its own first, irrespective of whether
default_factory was set.
If it created d[key] it would effectively override default_factory if active,
and
if not active, it would still act, letting you instrument a "normal" dict with
special effects.
Of course, if you wanted to write an on_missing handler to use default_factory
like your original
example, you could. So on_missing would always trigger if present, for missing
keys, but
d.defaulting[k] would only call d.default_factory if the latter was set and the
key was missing
even after on_missing (if present) did something (e.g., it could be logging
passively).
Regards,
Bengt Richter
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Does eval() leak?
Martin v. Löwis wrote: > John Marshall wrote: > >>Should I expect the virtual memory allocation >>to go up if I do the following? > > > python-dev is a list for discussing development of Python, > not the development with Python. Please post this question > to [EMAIL PROTECTED] > > For python-dev, a message explaining where the memory leak > is and how to correct it would be more appropriate. Most > likely, there is no memory leak in eval. My question was not a "development with Python" question. However, I posted to python-list as you said. Only one person responded to a request to test the provided code (~10 lines) which demonstrates a problem with eval()--he confirmed my observations. As the problem _does exist_ for 2.3.5 which is the last 2.3 version still available at python.org, I would suggest people avoid using it if they do eval()s. Unfortunately I, myself, cannot check into it more. John ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] bytes.from_hex()
> "Josiah" == Josiah Carlson <[EMAIL PROTECTED]> writes:
Josiah> I try to internalize it by not thinking of strings as
Josiah> encoded data, but as binary data, and unicode as text. I
Josiah> then remind myself that unicode isn't native on-disk or
Josiah> cross-network (which stores and transports bytes, not
Josiah> characters), so one needs to encode it as binary data.
Josiah> It's a subtle difference, but it has worked so far for me.
Seems like a lot of work for something that for monolingual usage
should "Just Work" almost all of the time.
Josiah> I notice that you seem to be in Japan, so teaching unicode
Josiah> is a must.
Yes. Japan is more complicated than that, but in Python unicode is a
must.
Josiah> If you are using the "unicode is text" and "strings are
Josiah> data", and they aren't getting it; then I don't know.
Well, I can tell you that they don't get it. One problem is PEP 263.
It makes it very easy to write programs that do line-oriented I/O with
input() and print, and the students come to think it should always be
that easy. Since Japan has at least 6 common encodings that students
encounter on a daily basis while browsing the web, plus a couple more
that live inside of MSFT Word and Java, they're used to huge amounts
of magic. The normal response of novice programmers is to mandate
that users of their programs use the encoding of choice and put it in
ordinary strings so that it just works.
Ie, the average student just "eats" the F on the codecs assignment,
and writes the rest of her programs without them.
>> simple, and the exceptions for using a "nonexistent" method
>> mean I don't have to reinforce---the students will be able to
>> teach each other. The exceptions also directly help reinforce
>> the notion that text == Unicode.
Josiah> Are you sure that they would help? If .encode() and
Josiah> .decode() drop from strings and unicode (respectively),
Josiah> they get an AttributeError. That's almost useless.
Well, I'm not _sure_, but this is the kind of thing that you can learn
by rote. And it will happen on a sufficiently regular basis that a
large fraction of students will experience it. They'll ask each
other, and usually they'll find a classmate who knows what happened.
I haven't tried this with codecs, but that's been my experience with
statistical packages where some routines understand non-linear
equations but others insist on linear equations.[1] The error messages
("Equation is non-linear! Aaugh!") are not much more specific than
AttributeError.
Josiah> Raising a better exception (with more information) would
Josiah> be better in that case, but losing the functionality that
Josiah> either would offer seems unnecessary;
Well, the point is that for the "usual suspects" (ie, Unicode codecs)
there is no functionality that would be lost. As MAL pointed out, for
these codecs the "original" text is always Unicode; that's the role
Unicode is designed for, and by and large it fits the bill very well.
With few exceptions (such as rot13) the "derived" text will be bytes
that peripherals such as keyboards and terminals can generate and
display.
Josiah> "You are trying to encode/decode to/from incompatible
Josiah> types. expected: a->b got: x->y" is better. Some of those
Josiah> can be done *very soon*, given the capabilities of the
Josiah> encodings module,
That's probably the way to go.
If we can have a derived "Unicode codec" class that does this, that
would pretty much entirely serve the need I perceive. Beginning
students could learn to write iconv.py, more advanced students could
learn to create codec stacks to generate MIME bodies, which could
include base64 or quoted-printable bytes -> bytes codecs.
Footnotes:
[1] If you're not familiar with regression analysis, the problem is
that the equation "z = a*log(x) + b*log(y)" where a and b are to be
estimated is _linear_ in the sense that x, y, and z are data series,
and X = log(x) and Y = log(y) can be precomputed so that the equation
actually computed is "z = a*X + b*Y". On the other hand "z = a*(x +
b*y)" is _nonlinear_ because of the coefficient on y being a*b.
Students find this hard to grasp in the classroom, but they learn
quickly in the lab.
I believe the parameter/variable inversion that my students have
trouble with in statistics is similar to the "original"/"derived"
inversion that happens with "text you can see" (derived, string) and
"abstract text inside the program" (original, Unicode).
--
School of Systems and Information Engineering http://turnbull.sk.tsukuba.ac.jp
University of TsukubaTennodai 1-1-1 Tsukuba 305-8573 JAPAN
Ask not how you can "do" free software business;
ask what your business can "do for" free software.
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mail
[Python-Dev] documenting things [Was: Re: Proposal: defaultdict]
On Mon, 20 Feb 2006 12:01:02 +, "Paul Moore" <[EMAIL PROTECTED]> wrote: >On 2/19/06, Steve Holden <[EMAIL PROTECTED]> wrote: >> > You are missing the rationale of the PEP process. The point is >> > *not* documentation. The point of the PEP process is to channel >> > and collect discussion, so that the BDFL can make a decision. >> > The BDFL is not bound at all to the PEP process. >> > >> > To document things, we use (or should use) documentation. >> > >> > >> One could wish this ideal had been the case for the import extensions >> defined in PEP 302. > >(A bit off-topic, but that hit home, so I'll reply...) > >Agreed, and it's my fault they weren't, to some extent. I did try to >find a suitable place, but the import docs are generally fairly >scattered, and there wasn't a particularly good place to put the >changes. > >Any suggestions would be gratefully accepted... I've always thought we could leverage google to find good doc information if we would just tag it in some consistent way. E.g., if you wanted to post a partial draft of some pep doc, you could post it here and/or c.l.p with PEP 302 docs version 2 <<--ENDMARK-- text here = (use REST if ambitious) ... --ENDMARK-- If we had some standard tag lines, we could make an urllib tool to harvest the material and merge the most recent version paragraphs and auto-post it as html in one place for draft docs on python.org The same tagged section technique could be used re any documention, so long as update and/or addition text can be associated with where it should be tagged in as references. I think mouseover popup hints with clickable js popups for the additional material would be cool. It would mean automatically editing the doc to insert the hints though. Well, nothing there is rocket science, but neither is a wall of bricks so long and high you can't live long enough to complete it ;-) Regards, Bengt Richter ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Proposal: defaultdict
On 2/20/06, Aahz <[EMAIL PROTECTED]> wrote: > On Sun, Feb 19, 2006, Josiah Carlson wrote: > > > > I agree, there is nothing perfect. But at least in all of my use-cases, > > and the majority of the ones I've seen 'in the wild', my previous post > > provided an implementation that worked precisely like desired, and > > precisely like a regular dictionary, except when accessing a > > non-existant key via: value = dd[key] . __contains__, etc., all work > > exactly like they do with a non-defaulting dictionary. Iteration via > > popitem(), pop(key), items(), iteritems(), __iter__, etc., all work the > > way you would expect them. > > This is the telling point, IMO. My company makes heavy use of a "default > dict" (actually, it's a "default class" because using constants as the > lookup keys is mostly what we do and the convenience of foo.bar is > compelling over foo['bar']). Anyway, our semantics are as Josiah > outlines, and I can't see much use case for the alternatives. Can you say, for the record (since nobody else seems to care), if d.getorset(key, func) would work in your use cases? > Those of you arguing something different: do you have a real use case > (that you've implemented in real code)? (again, for the record) getorset provides the minimum needed functionality in a clean and intuitive way. Why go for a complicated solution when you simply don't need it? -- Adam Olsen, aka Rhamphoryncus ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] bdist_* to stdlib?
Op vr, 17-02-2006 te 23:22 +0100, schreef "Martin v. Löwis": > That, in turn, is because nobody is so short of disk space that > you really *have* to share /usr/share across architectures, I can see diskless thin clients that boot from flash memory doing things like that? (E.g. having documentation and header files and other less-important stuff on an nfs mount?) -- Jan Claeys ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] defaultdict proposal round three
On 2/20/06, Raymond Hettinger <[EMAIL PROTECTED]> wrote: > [GvR] > > Alternative A: add a new method to the dict type with the semantics of > > __getattr__ from the last proposal > > Did you mean __getitem__? Yes, sorry, I meant __getitem__. -- --Guido van Rossum (home page: http://www.python.org/~guido/) ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] Proposal: defaultdict
Adam Olsen asked: > ... d.getorset(key, func) would work in your use > cases? It is an improvement over setdefault, because it doesn't always evaluate the expensive func. (But why should every call have to pass in the function, when it is a property of the dictionary?) It doesn't actually *solve* the problem because it doesn't compose well. This makes it hard to use for configuration. (Use case from plucker web reader, where the config is arguably overdesigned, but ... the version here is simplified) There is a system-wide default config. Users have config files. A config file can be specified for this program run. In each of these, settings can be either general settings or channel-specific. The end result is that the value should be pulled from the first of about half a dozen dictionaries to have an answer. Because most settings are never used in most channels, and several channels are typically run at once, it feels wrong to pre-build the whole "anything they might ask" settings dictionary for each of them. On the other hand, I certainly don't want to write userchannelconfig.getorset(key, systemchannelconfig.getorset(key, ...) even once, let alone every time I get a config value. In other words, the program would work correctly if I passed in a normal but huge dictionary; I want to avoid that for reasons of efficiency. This isn't the only use for a mapping, but it is the only one I've seen where KeyError is "expected" by the program's normal flow. -jJ ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] defaultdict proposal round three
On Feb 20, 2006, at 8:35 AM, Raymond Hettinger wrote: > [GvR] >> I'm not convinced by the argument >> that __contains__ should always return True > > Me either. I cannot think of a more useless behavior or one more > likely to have > unexpected consequences. Besides, as Josiah pointed out, it is > much easier for > a subclass override to substitute always True return values than > vice-versa. Agreed on all counts. > I prefer this approach over subclassing. The mental load from an > additional > method is less than the load from a separate type (even a > subclass). Also, > avoidance of invariant issues is a big plus. Besides, if this allows > setdefault() to be deprecated, it becomes an all-around win. I'd love to remove setdefault in 3.0 -- but I don't think it can be done before that: default_factory won't cover the occasional use cases where setdefault is called with different defaults at different locations, and, rare as those cases may be, any 2.* should not break any existing code that uses that approach. >> - Even if the default_factory were passed to the constructor, it >> still >> ought to be a writable attribute so it can be introspected and >> modified. A defaultdict that can't change its default factory after >> its creation is less useful. > > Right! My preference is to have default_factory not passed to the > constructor, > so we are left with just one way to do it. But that is a nit. No big deal either way, but I see "passing the default factory to the ctor" as the "one obvious way to do it", so I'd rather have it (be it with a subclass or a classmethod-alternate constructor). I won't weep bitter tears if this drops out, though. >> - It would be unwise to have a default value that would be called if >> it was callable: what if I wanted the default to be a class instance >> that happens to have a __call__ method for unrelated reasons? >> Callability is an elusive propperty; APIs should not attempt to >> dynamically decide whether an argument is callable or not. > > That makes sense, though it seems over-the-top to need a zero- > factory for a > multiset. But int is a convenient zero-factory. > > An alternative is to have two possible attributes: > d.default_factory = list > or > d.default_value = 0 > with an exception being raised when both are defined (the test is > done when the > attribute is created, not when the lookup is performed). I see default_value as a way to get exactly the same beginner's error we already have with function defaults: a mutable object will not work as beginners expect, and we can confidently predict (based on the function defaults case) that python-list and python-help and python-tutor and a bazillion other venues will see an unending stream of confused beginners (in addition to those confused by mutable objects as default values for function arguments, but those can't be avoided). I presume you consider the "one obvious way" is to use default_value for immutables and default_factory for mutables, but based on a lot of experience teaching Python I feel certain that this won't be obvious to many, MANY users (and not just non-Dutch ones, either). Alex ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] defaultdict proposal round three
Guido van Rossum wrote:
> Alternative A: add a new method to the dict type with the semantics of
> __getattr__ from the last proposal, using default_factory if not None
> (except on_missing is inlined).
I'm not certain I understood this right but (after
s/__getattr__/__getitem__) this seems to suggest that for keeping a
dict of counts the code wouldn't really improve much:
dd = {}
dd.default_factory = int
for item in items:
# I want to do ``dd[item] += 1`` but with a regular method instead
# of __getitem__, this is not possible
dd[item] = dd.somenewmethod(item) + 1
I don't think that's much better than just calling ``dd.get(item,
0)``. Did I misunderstand Alternative A?
> Alternative B: provide a dict subclass that implements the __getattr__
> semantics from the last proposal.
If I didn't misinterpret Alternative A, I'd definitely prefer
Alternative B. A dict of counts is by far my most common use case...
STeVe
--
Grammar am for people who can't think for myself.
--- Bucky Katt, Get Fuzzy
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] Simple CPython stack overflow.
Just noticed this and wondered if it came under the Python should never
crash mantra. Should sys.getrecursionlimit() perhaps be taken into
account somewhere?
>>> D = {'a': None}
>>> for i in xrange(15):
... D = {'a': D}
...
>>> D
{'a': {'a': {'a': {'a': {'a': {'a': {'a': {'a': {'a': {'a': {'a':
{'a': {'a': {'a': {'a': {'a': {'a': {'a': ': {'a': {'a': {'a':
{'a': {'a': {'a': {'a': {[+]'a': {'a': {'a': {'a': {'a': {'a': {'a':
{'a': {'a' Bus error
bash$
Cheers,
David.
--
'tis better to be silent and be thought a fool,
than to speak and remove all doubt.
-- Lincoln
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Proposal: defaultdict
"Adam Olsen" <[EMAIL PROTECTED]> wrote: > Can you say, for the record (since nobody else seems to care), if > d.getorset(key, func) would work in your use cases? It doesn't work for the multiset/accumulation case: dd[key] += 1 - Josiah ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Simple CPython stack overflow.
Yes, this is the type of thing we've been struggling with for years.
There used to be way more of these. I can't guarantee it'll be fixed
with priority (it's mostly of the "then don't do that" type) but
please do file a bug so someone with inclination can fix it. The same
happens for deeply recursive tuples and lists BTW.
--Guido
On 2/20/06, David Wilson <[EMAIL PROTECTED]> wrote:
> Just noticed this and wondered if it came under the Python should never
> crash mantra. Should sys.getrecursionlimit() perhaps be taken into
> account somewhere?
>
> >>> D = {'a': None}
> >>> for i in xrange(15):
> ... D = {'a': D}
> ...
> >>> D
> {'a': {'a': {'a': {'a': {'a': {'a': {'a': {'a': {'a': {'a': {'a':
> {'a': {'a': {'a': {'a': {'a': {'a': {'a': ': {'a': {'a': {'a':
> {'a': {'a': {'a': {'a': {[+]'a': {'a': {'a': {'a': {'a': {'a': {'a':
> {'a': {'a' Bus error
> bash$
>
> Cheers,
>
>
> David.
>
> --
> 'tis better to be silent and be thought a fool,
> than to speak and remove all doubt.
> -- Lincoln
> ___
> Python-Dev mailing list
> [email protected]
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> http://mail.python.org/mailman/options/python-dev/guido%40python.org
>
--
--Guido van Rossum (home page: http://www.python.org/~guido/)
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] defaultdict proposal round three
On 2/20/06, Steven Bethard <[EMAIL PROTECTED]> wrote: > Guido van Rossum wrote: > > Alternative A: add a new method to the dict type with the semantics of > > [__getitem__] from the last proposal, using default_factory if not None > > (except on_missing is inlined). > > I'm not certain I understood this right but [...] > this seems to suggest that for keeping a > dict of counts the code wouldn't really improve much: You don't need a new feature for that use case; d[k] = d.get(k, 0) + 1 is perfectly fine there and hard to improve upon. It's the slightly more esoteric use case where the default is a list and you want to append to that list that we're trying to improve: currently the shortest version is d.setdefault(k, []).append(v) but that lacks legibility and creates an empty list that is thrown away most of the time. We're trying to obtain the minimal form d.foo(k).append(v) where the new list is created by implicitly calling d.default_factory if d[k] doesn't yet exist, and d.default_factory is set to the list constructor. -- --Guido van Rossum (home page: http://www.python.org/~guido/) ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] defaultdict proposal round three
Sorry to chime in so late, but why are we setting a value when the key isn't defined? It seems there are many situations where you want: a) default values, and b) the ability to determine if a value was defined. There are many times that I want d[key] to give me a value even when it isn't defined, but that doesn't always mean I want to _save_ that value in the dict. Sometimes I do, sometimes I don't. We should have some means of describing this in any defaultdict implementation On 2/20/06, Guido van Rossum <[EMAIL PROTECTED]> wrote: > I'm withdrawing the last proposal. I'm not convinced by the argument > that __contains__ should always return True (perhaps it should also > insert the value?), nor by the complaint that a holy invariant would > be violated (so what?). > > But the amount of discussion and the number of different viewpoints > present makes it clear that the feature as I last proposed would be > forever divisive. > > I see two alternatives. These will cause a different kind of > philosophical discussion; so be it. I'll describe them relative to the > last proposal; for those who wisely skipped the last thread, here's a > link to the proposal: > http://mail.python.org/pipermail/python-dev/2006-February/061261.html. > > Alternative A: add a new method to the dict type with the semantics of > __getattr__ from the last proposal, using default_factory if not None > (except on_missing is inlined). This avoids the discussion about > broken invariants, but one could argue that it adds to an already > overly broad API. > > Alternative B: provide a dict subclass that implements the __getattr__ > semantics from the last proposal. It could be an unrelated type for > all I care, but I do care about implementation inheritance since it > should perform just as well as an unmodified dict object, and that's > hard to do without sharing implementation (copying would be worse). > > Parting shots: > > - Even if the default_factory were passed to the constructor, it still > ought to be a writable attribute so it can be introspected and > modified. A defaultdict that can't change its default factory after > its creation is less useful. > > - It would be unwise to have a default value that would be called if > it was callable: what if I wanted the default to be a class instance > that happens to have a __call__ method for unrelated reasons? > Callability is an elusive propperty; APIs should not attempt to > dynamically decide whether an argument is callable or not. > > - A third alternative would be to have a new method that takes an > explicit defaut factory argument. This differs from setdefault() only > in the type of the second argument. I'm not keen on this; the original > use case came from an example where the readability of > > d.setdefault(key, []).append(value) > > was questioned, and I'm not sure that > > d.something(key, list).append(value) > > is any more readable. IOW I like (and I believe few have questioned) > associating the default factory with the dict object instead of with > the call site. > > Let the third round of the games begin! > > -- > --Guido van Rossum (home page: http://www.python.org/~guido/) > ___ > Python-Dev mailing list > [email protected] > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > http://mail.python.org/mailman/options/python-dev/crutcher%40gmail.com > -- Crutcher Dunnavant <[EMAIL PROTECTED]> littlelanguages.com monket.samedi-studios.com ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] buildbot is all green
""Martin v. Löwis"" <[EMAIL PROTECTED]> wrote in message news:[EMAIL PROTECTED] > Terry Reedy wrote: >> With a couple of more machines added, should there be two separate pages >> for trunk and 2.4 builds? Or do most checkins affect both? > > I'd like to avoid this, assuming that people only look at the "main" > page. An individual checkin affects either the trunk or 2.4, but never > both; many check-ins come in pairs. I mentioned the idea because I am not fond of horizontal scrolling, especially when there is a header column down the left and the prospect of several more columns being added, and thought others might feel the same. But this is, of course, for you, Guido, and whoever else is most concerned to decide. *If* someone revises the HTML generation, you might consider a main page that duplicates the column summary headers (but in two block of rows, one for trunk and one for maintenance) with links to the detail pages. If there are ever three simultaneous development tracks, say from 3.0 starting before the last 2.x is released, this might look even more attractive. Terry J. Reedy ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] defaultdict proposal round three
I'm thinking something mutch closer to this (note default_factory gets the key): def on_missing(self, key): if self.default_factory is not None: value = self.default_factory(key) if self.on_missing_define_key: self[key] = value return value raise KeyError(key) On 2/20/06, Crutcher Dunnavant <[EMAIL PROTECTED]> wrote: > Sorry to chime in so late, but why are we setting a value when the key > isn't defined? > > It seems there are many situations where you want: > a) default values, and > b) the ability to determine if a value was defined. > > There are many times that I want d[key] to give me a value even when > it isn't defined, but that doesn't always mean I want to _save_ that > value in the dict. Sometimes I do, sometimes I don't. We should have > some means of describing this in any defaultdict implementation > > On 2/20/06, Guido van Rossum <[EMAIL PROTECTED]> wrote: > > I'm withdrawing the last proposal. I'm not convinced by the argument > > that __contains__ should always return True (perhaps it should also > > insert the value?), nor by the complaint that a holy invariant would > > be violated (so what?). > > > > But the amount of discussion and the number of different viewpoints > > present makes it clear that the feature as I last proposed would be > > forever divisive. > > > > I see two alternatives. These will cause a different kind of > > philosophical discussion; so be it. I'll describe them relative to the > > last proposal; for those who wisely skipped the last thread, here's a > > link to the proposal: > > http://mail.python.org/pipermail/python-dev/2006-February/061261.html. > > > > Alternative A: add a new method to the dict type with the semantics of > > __getattr__ from the last proposal, using default_factory if not None > > (except on_missing is inlined). This avoids the discussion about > > broken invariants, but one could argue that it adds to an already > > overly broad API. > > > > Alternative B: provide a dict subclass that implements the __getattr__ > > semantics from the last proposal. It could be an unrelated type for > > all I care, but I do care about implementation inheritance since it > > should perform just as well as an unmodified dict object, and that's > > hard to do without sharing implementation (copying would be worse). > > > > Parting shots: > > > > - Even if the default_factory were passed to the constructor, it still > > ought to be a writable attribute so it can be introspected and > > modified. A defaultdict that can't change its default factory after > > its creation is less useful. > > > > - It would be unwise to have a default value that would be called if > > it was callable: what if I wanted the default to be a class instance > > that happens to have a __call__ method for unrelated reasons? > > Callability is an elusive propperty; APIs should not attempt to > > dynamically decide whether an argument is callable or not. > > > > - A third alternative would be to have a new method that takes an > > explicit defaut factory argument. This differs from setdefault() only > > in the type of the second argument. I'm not keen on this; the original > > use case came from an example where the readability of > > > > d.setdefault(key, []).append(value) > > > > was questioned, and I'm not sure that > > > > d.something(key, list).append(value) > > > > is any more readable. IOW I like (and I believe few have questioned) > > associating the default factory with the dict object instead of with > > the call site. > > > > Let the third round of the games begin! > > > > -- > > --Guido van Rossum (home page: http://www.python.org/~guido/) > > ___ > > Python-Dev mailing list > > [email protected] > > http://mail.python.org/mailman/listinfo/python-dev > > Unsubscribe: > > http://mail.python.org/mailman/options/python-dev/crutcher%40gmail.com > > > > > -- > Crutcher Dunnavant <[EMAIL PROTECTED]> > littlelanguages.com > monket.samedi-studios.com > -- Crutcher Dunnavant <[EMAIL PROTECTED]> littlelanguages.com monket.samedi-studios.com ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Proposal: defaultdict
On Mon, Feb 20, 2006, Adam Olsen wrote: > On 2/20/06, Aahz <[EMAIL PROTECTED]> wrote: >> On Sun, Feb 19, 2006, Josiah Carlson wrote: >>> >>> I agree, there is nothing perfect. But at least in all of my use-cases, >>> and the majority of the ones I've seen 'in the wild', my previous post >>> provided an implementation that worked precisely like desired, and >>> precisely like a regular dictionary, except when accessing a >>> non-existant key via: value = dd[key] . __contains__, etc., all work >>> exactly like they do with a non-defaulting dictionary. Iteration via >>> popitem(), pop(key), items(), iteritems(), __iter__, etc., all work the >>> way you would expect them. >> >> This is the telling point, IMO. My company makes heavy use of a "default >> dict" (actually, it's a "default class" because using constants as the >> lookup keys is mostly what we do and the convenience of foo.bar is >> compelling over foo['bar']). Anyway, our semantics are as Josiah >> outlines, and I can't see much use case for the alternatives. > > Can you say, for the record (since nobody else seems to care), if > d.getorset(key, func) would work in your use cases? Because I haven't been reading this thread all that closely, you'll have to remind me what this means. >> Those of you arguing something different: do you have a real use case >> (that you've implemented in real code)? > > (again, for the record) getorset provides the minimum needed > functionality in a clean and intuitive way. Why go for a complicated > solution when you simply don't need it? Ditto above. -- Aahz ([EMAIL PROTECTED]) <*> http://www.pythoncraft.com/ "19. A language that doesn't affect the way you think about programming, is not worth knowing." --Alan Perlis ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Proposal: defaultdict
On 2/20/06, Josiah Carlson <[EMAIL PROTECTED]> wrote: > "Adam Olsen" <[EMAIL PROTECTED]> wrote: > > Can you say, for the record (since nobody else seems to care), if > > d.getorset(key, func) would work in your use cases? > > It doesn't work for the multiset/accumulation case: > > dd[key] += 1 This is actually a fairly powerful argument for a subclass that redefines __getitem__ in favor of a new dict method. (Not to mention that it's much easier to pick a name for the subclass than for the method. :-) See the new thread I started. -- --Guido van Rossum (home page: http://www.python.org/~guido/) ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] buildbot is all green
Manuzhai wrote: >> No; nobody volunteered a machine yet (plus the hand-holding that >> is always necessary with Windows). >> > > What exactly is needed for this? Does it need to be a machine dedicated > to this stuff, or could I just run the tests once every day or so when I > feel like it and have them submitted to buildbot? > > Has a machine been volunteered ? I have a spare machine and an always on connection. Would the 'right' development tools be needed ? (In the case of Microsoft they are a touch expensive I believe.) All the best, Michael Foord > Regards, > > Manuzhai > > ___ > Python-Dev mailing list > [email protected] > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > http://mail.python.org/mailman/options/python-dev/fuzzyman%40voidspace.org.uk > > ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] defaultdict proposal round three
Alex Martelli wrote: >>I prefer this approach over subclassing. The mental load from an >>additional >>method is less than the load from a separate type (even a >>subclass). Also, >>avoidance of invariant issues is a big plus. Besides, if this allows >>setdefault() to be deprecated, it becomes an all-around win. > > > I'd love to remove setdefault in 3.0 -- but I don't think it can be > done before that: default_factory won't cover the occasional use > cases where setdefault is called with different defaults at different > locations, and, rare as those cases may be, any 2.* should not break > any existing code that uses that approach. Would it be deprecated in 2.*, or start deprecating in 3.0? Also, is default_factory=list threadsafe in the same way .setdefault is? That is, you can safely do this from multiple threads: d.setdefault(key, []).append(value) I believe this is safe with very few caveats -- setdefault itself is atomic (or else I'm writing some bad code ;). My impression is that default_factory will not generally be threadsafe in the way setdefault is. For instance: def make_list(): return [] d = dict d.default_factory = make_list # from multiple threads: d.getdef(key).append(value) This would not be correct (a value can be lost if two threads concurrently enter make_list for the same key). In the case of default_factory=list (using the list builtin) is the story different? Will this work on Jython, IronPython, or PyPy? Will this be a documented guarantee? Or alternately, are we just creating a new way to punish people who use threads? And if we push threadsafety up to user code, are we trading a very small speed issue (creating lists that are thrown away) for a much larger speed issue (acquiring a lock)? I tried to make a test for this threadsafety, actually -- using a technique besides setdefault which I knew was bad (try:except KeyError:). And (except using time.sleep(), which is cheating), I wasn't actually able to trigger the bug. Which is frustrating, because I know the bug is there. So apparently threadsafety is hard to test in this case. (If anyone is interested in trying it, I can email what I have.) Note that multidict -- among other possible concrete collection patterns (like Bag, OrderedDict, or others) -- can be readily implemented with threading guarantees. -- Ian Bicking / [EMAIL PROTECTED] / http://blog.ianbicking.org ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] defaultdict proposal round three
Steven Bethard wrote:
>>Alternative A: add a new method to the dict type with the semantics of
>>__getattr__ from the last proposal, using default_factory if not None
>>(except on_missing is inlined).
>
>
> I'm not certain I understood this right but (after
> s/__getattr__/__getitem__) this seems to suggest that for keeping a
> dict of counts the code wouldn't really improve much:
>
> dd = {}
> dd.default_factory = int
> for item in items:
> # I want to do ``dd[item] += 1`` but with a regular method instead
> # of __getitem__, this is not possible
> dd[item] = dd.somenewmethod(item) + 1
This would be better done with a bag (a set that can contain multiple
instances of the same item):
dd = collections.Bag()
for item in items:
dd.add(item)
Then to see how many there are of an item, perhaps something like:
dd.count(item)
No collections.Bag exists, but of course one should. It has nice
properties -- inclusion is done with __contains__ (with dicts it
probably has to be done with get), you can't accidentally go below zero,
the methods express intent, and presumably it will implement only a
meaningful set of methods.
--
Ian Bicking / [EMAIL PROTECTED] / http://blog.ianbicking.org
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] defaultdict proposal round three
On 2/20/06, Ian Bicking <[EMAIL PROTECTED]> wrote: > Would it be deprecated in 2.*, or start deprecating in 3.0? 3.0 will have no backwards compatibility allowances. Whenever someone says "remove this in 3.0" they mean exactly that. There will be too many incompatibilities in 3.0 to be bothered with deprecating them all; most likely we'll have to have some kind of (semi-)automatic conversion tool. Deprecation in 2.x is generally done to indicate that a feature will be removed in 2.y for y >= x+1. -- --Guido van Rossum (home page: http://www.python.org/~guido/) ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] defaultdict proposal round three
On Feb 20, 2006, at 12:33 PM, Guido van Rossum wrote: ... > You don't need a new feature for that use case; d[k] = d.get(k, 0) + 1 > is perfectly fine there and hard to improve upon. I see d[k]+=1 as a substantial improvement -- conceptually more direct, "I've now seen one more k than I had seen before". Alex ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Proposal: defaultdict
On Feb 20, 2006, at 12:38 PM, Aahz wrote: ... >> Can you say, for the record (since nobody else seems to care), if >> d.getorset(key, func) would work in your use cases? > > Because I haven't been reading this thread all that closely, you'll > have > to remind me what this means. Roughly the same (save for method/function difference) as: def getorset(d, key, func): if key not in d: d[key] = func() return d[key] Alex ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] defaultdict proposal round three
On 2/20/06, Ian Bicking <[EMAIL PROTECTED]> wrote:
> Also, is default_factory=list threadsafe in the same way .setdefault is?
> That is, you can safely do this from multiple threads:
>
>d.setdefault(key, []).append(value)
>
> I believe this is safe with very few caveats -- setdefault itself is
> atomic (or else I'm writing some bad code ;).
Only if the key is a string and all values in the dict are also
strings (or other builtins). And I don't think that Jython or
IronPython promise anything here.
Here's a sketch of a situation that isn't thread-safe:
class C:
def __eq__(self, other):
return False
def __hash__(self):
return hash("abc")
d = {C(): 42}
print d["abc"]
Because "abc" and C() have the same hash value, the lookup will
compare "abc" to C() which will invoke C.__eq__().
Why are you so keen on using a dictionary to share data between
threads that may both modify it? IMO this is asking for trouble --
the advice about sharing data between threads is always to use the
Queue module.
[...]
> Note that multidict -- among other possible concrete collection patterns
> (like Bag, OrderedDict, or others) -- can be readily implemented with
> threading guarantees.
I don't believe that this is as easy as you think.
--
--Guido van Rossum (home page: http://www.python.org/~guido/)
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] defaultdict proposal round three
On 2/20/06, Alex Martelli <[EMAIL PROTECTED]> wrote: > > On Feb 20, 2006, at 12:33 PM, Guido van Rossum wrote: > ... > > You don't need a new feature for that use case; d[k] = d.get(k, 0) + 1 > > is perfectly fine there and hard to improve upon. > > I see d[k]+=1 as a substantial improvement -- conceptually more > direct, "I've now seen one more k than I had seen before". Yes, I now agree. This means that I'm withdrawing proposal A (new method) and championing only B (a subclass that implements __getitem__() calling on_missing() and on_missing() defined in that subclass as before, calling default_factory unless it's None). I don't think this crisis is big enough to need *two* solutions, and this example shows B's superiority over A. -- --Guido van Rossum (home page: http://www.python.org/~guido/) ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] defaultdict proposal round three
[Crutcher Dunnavant ] >> There are many times that I want d[key] to give me a value even when >> it isn't defined, but that doesn't always mean I want to _save_ that >> value in the dict. How does that differ from the existing dict.get method? Raymond ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] bytes.from_hex()
Stephen J. Turnbull wrote: > Martin> For an example where base64 is *not* necessarily > Martin> ASCII-encoded, see the "binary" data type in XML > Martin> Schema. There, base64 is embedded into an XML document, > Martin> and uses the encoding of the entire XML document. As a > Martin> result, you may get base64 data in utf16le. > > I'll have to take a look. It depends on whether base64 is specified > as an octet-stream to Unicode stream transformation or as an embedding > of an intermediate representation into Unicode. Granted, defining the > base64 alphabet as a subset of Unicode seems like the logical way to > do it in the context of XML. Please do take a look. It is the only way: If you were to embed base64 *bytes* into character data content of an XML element, the resulting XML file might not be well-formed anymore (if the encoding of the XML file is not an ASCII superencoding). Regards, Martin ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] defaultdict proposal round three
Guido van Rossum wrote: > Why are you so keen on using a dictionary to share data between > threads that may both modify it? IMO this is asking for trouble -- > the advice about sharing data between threads is always to use the > Queue module. I use them often for a shared caches. But yeah, it's harder than I thought at first -- I think the actual cases I'm using work, since they use simple keys (ints, strings), but yeah, thread guarantees are too difficult to handle in general. Damn threads. -- Ian Bicking / [EMAIL PROTECTED] / http://blog.ianbicking.org ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Proposal: defaultdict
On Mon, Feb 20, 2006, Alex Martelli wrote:
> On Feb 20, 2006, at 12:38 PM, Aahz wrote:
> ...
>>> Can you say, for the record (since nobody else seems to care), if
>>> d.getorset(key, func) would work in your use cases?
>>
>> Because I haven't been reading this thread all that closely, you'll
>> have
>> to remind me what this means.
>
> Roughly the same (save for method/function difference) as:
>
> def getorset(d, key, func):
>if key not in d: d[key] = func()
>return d[key]
That has the problem of looking clumsy, and doubly so for our use case
where it's an attribute-based dict. Our style relies on the clean look
of code like this:
if order.street:
...
Even as a dict, that doesn't look horrible:
if order['street']:
...
OTOH, this starts looking ugly:
if order.get('street'):
...
And this is just plain bad:
if getattr(order, 'street'):
...
Note that because we have to deal with *both* the possibility that the
attribute/key may not be there *and* that it might be blank -- but both
are semantically equivalent for our application -- there's no other
clean coding style.
Now, I realize this is different from the "primary use case" for needing
mutable values, but any proposed default dict solution that doesn't
cleanly support my use case is less interesting to me.
--
Aahz ([EMAIL PROTECTED]) <*> http://www.pythoncraft.com/
"19. A language that doesn't affect the way you think about programming,
is not worth knowing." --Alan Perlis
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] buildbot is all green
Guido van Rossum wrote: > They don't; I think a separate page would be a fine idea. Ok, I have now split this into three pages. > FWIW, it looks like all the sample templates are still wasting a lot > of horizontal space in the first two columns the second is almost > always empty. Perhaps the author of the change could be placed *below* > the timestamp instead of next to it? Also for all practical purposes > we can probably get rid of the seconds in the timestamp. The latter was easy to do, so I did it. The former is tricky; contributions are welcome. Regards, Martin ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] buildbot is all green
Steve Holden wrote: > All formats would be improved of the headers could be made to float at > the top of the page as scrolling took place. Can this be done in CSS? If so, contributions are welcome. If not, can somebody prepare a modified page with the necessary changes (preferably only additional classes for the header or some such); I can then try to edit buildbot to add these changes into the page. Regards, Martin ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] buildbot is all green
Martin v. Löwis wrote: > Steve Holden wrote: >> All formats would be improved of the headers could be made to float at >> the top of the page as scrolling took place. > > Can this be done in CSS? If so, contributions are welcome. Not as it is. The big table would have to be split so that there is one table with the heading and one with the rest. But that would make the columns independent, so the header's column widths would differ from the content's. Even then, I don't know if there's a working solution for the headers to stay on top since * floats are only left or right aligned * the header's height is variable * position:absolute doesn't work in MSIE. regards, Georg ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Removing Non-Unicode Support?
M.-A. Lemburg wrote: > Note that this does not mean that we should forget about memory > consumption issues. It's just that if there's only marginal > interest in certain special builds of Python, I don't see the > requirement for the Python core developers to maintain them. Well, the cost of Unicode support is not so much in the algorithmic part, but in the tables that come along with it. AFAICT, everything but unicodectype is optional; that is 5KiB of code and 20KiB of data on x86. Actually, the size of the code *does* matter, at a second glance. Here are the largest object files in the Python code base on my system (not counting dynamic modules): textdata bss dec hex filename 4845 19968 0 2481360ed Objects/unicodectype.o 226332432 352 254176349 Objects/listobject.o 292591412 152 308237867 Objects/classobject.o 20696 11488 4 321887dbc Python/bltinmodule.o 33579 740 0 34319860f Objects/longobject.o 34119 16 288 344238677 Python/ceval.o 351792796 0 379759457 Modules/_sre.o 26539 15820 416 42775a717 Modules/posixmodule.o 3528388001056 45139b053 Objects/stringobject.o 50360 0 28 50388c4d4 Python/compile.o 684554624 440 73519 11f2f Objects/typeobject.o 6999393161196 80505 13a79 Objects/unicodeobject.o So it appears that dropping Unicode support can indeed provide some savings. For reference, we also have an option to drop complex numbers: 9654 692 4 10350286e Objects/complexobject.o Regards, Martin ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] bdist_* to stdlib?
Jan Claeys wrote: >>That, in turn, is because nobody is so short of disk space that >>you really *have* to share /usr/share across architectures, > > > I can see diskless thin clients that boot from flash memory doing things > like that? (E.g. having documentation and header files and other > less-important stuff on an nfs mount?) Having parts of the file system on NFS: sure, even have root on NFS: all the time. But if you have two classes of machines (say, diskless SPARC and diskless x86 PCs) for which you have to provide different sets of binaries on NFS: why do you have to share /usr/share across architectures? It will only save you a small percentage of disk space, and at additional hassles. Regards, Martin ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] buildbot is all green
Michael Foord wrote: > Has a machine been volunteered ? Not yet. > I have a spare machine and an always on connection. Would the 'right' > development tools be needed ? (In the case of Microsoft they are a touch > expensive I believe.) Any build process would do. I would prefer to see the official tools on the buildbot (i.e. VS.NET 2003), but anything else that can build Python and pass the test suite could do as well. One issue is that you also have to to work with me on defining the build steps: what sequence of commands to send in what order. For Unix, that is easy; for Windows, not so. Regards, Martin ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] defaultdict proposal round three
I wrote:
># I want to do ``dd[item] += 1``
Guido van Rossum wrote:
> You don't need a new feature for that use case; d[k] = d.get(k, 0) + 1
> is perfectly fine there and hard to improve upon.
Alex Martelli wrote:
> I see d[k]+=1 as a substantial improvement -- conceptually more
> direct, "I've now seen one more k than I had seen before".
Guido van Rossum wrote:
> Yes, I now agree. This means that I'm withdrawing proposal A (new
> method) and championing only B (a subclass that implements
> __getitem__() calling on_missing() and on_missing() defined in that
> subclass as before, calling default_factory unless it's None).
Probably already obvious from my previous post, but FWIW, +1.
Two unaddressed issues:
* What module should hold the type? I hope the collections module
isn't too controversial.
* Should default_factory be an argument to the constructor? The three
answers I see:
- "No." I'm not a big fan of this answer. Since the whole point of
creating a defaultdict type is to provide a default, requiring two
statements (the constructor call and the default_factory assignment)
to initialize such a dictionary seems a little inconvenient.
- "Yes and it should be followed by all the normal dict constructor
arguments." This is okay, but a few errors, like
``defaultdict({1:2})`` will pass silently (until you try to use the
dict, of course).
- "Yes and it should be the only constructor argument." This is my
favorite mainly because I think it's simple, and I couldn't think of
good examples where I really wanted to do ``defaultdict(list,
some_dict_or_iterable)`` or ``defaultdict(list,
**some_keyword_args)``. It's also forward compatible if we need to
add some of the dict constructor args in later.
STeVe
--
Grammar am for people who can't think for myself.
--- Bucky Katt, Get Fuzzy
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] Win64 AMD64 (aka x64) binaries available64
I have now produces a snapshot of a Win64 build for AMD64 processors (also known as EM64T or x64); this is different from IA-64 (which is also known as Itanium)... Anyway, the binaries are http://www.dcl.hpi.uni-potsdam.de/home/loewis/python-2.5.13199.amd64.msi This is from today's trunk. If you have general remarks/discussion, please post to python-dev. If you have specific bug reports, file them on SF. Bug fixes are particularly welcome. Known issues: - _ssl.pyd is not build (I get linker errors) - some of the tests fail (in some cases, due to bugs in the test suite) If you want to build extensions for this build using distutils, you need to 1. install the platform SDK (2003 SP1 should work) 2. open an AMD64 retail shell 3. run the included distutils It might be possible to drop 2) some day, but finding the SDK from the registry is really tricky. Regards, Martin ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] defaultdict proposal round three
On 2/20/06, Steven Bethard <[EMAIL PROTECTED]> wrote:
> I wrote:
> ># I want to do ``dd[item] += 1``
>
> Guido van Rossum wrote:
> > You don't need a new feature for that use case; d[k] = d.get(k, 0) + 1
> > is perfectly fine there and hard to improve upon.
>
> Alex Martelli wrote:
> > I see d[k]+=1 as a substantial improvement -- conceptually more
> > direct, "I've now seen one more k than I had seen before".
>
> Guido van Rossum wrote:
> > Yes, I now agree. This means that I'm withdrawing proposal A (new
> > method) and championing only B (a subclass that implements
> > __getitem__() calling on_missing() and on_missing() defined in that
> > subclass as before, calling default_factory unless it's None).
>
> Probably already obvious from my previous post, but FWIW, +1.
>
> Two unaddressed issues:
>
> * What module should hold the type? I hope the collections module
> isn't too controversial.
>
> * Should default_factory be an argument to the constructor? The three
> answers I see:
>
> - "No." I'm not a big fan of this answer. Since the whole point of
> creating a defaultdict type is to provide a default, requiring two
> statements (the constructor call and the default_factory assignment)
> to initialize such a dictionary seems a little inconvenient.
> - "Yes and it should be followed by all the normal dict constructor
> arguments." This is okay, but a few errors, like
> ``defaultdict({1:2})`` will pass silently (until you try to use the
> dict, of course).
> - "Yes and it should be the only constructor argument." This is my
> favorite mainly because I think it's simple, and I couldn't think of
> good examples where I really wanted to do ``defaultdict(list,
> some_dict_or_iterable)`` or ``defaultdict(list,
> **some_keyword_args)``. It's also forward compatible if we need to
> add some of the dict constructor args in later.
>
While #3 is my preferred solution as well, it does pose a Liskov
violation if this is a direct dict subclass instead of storing a dict
internally (can't remember the name of the design pattern that does
this). But I think it is good to have the constructor be different
since it does also help drive home the point that this is not a
standard dict.
-Brett
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] defaultdict proposal round three
On 2/20/06, Raymond Hettinger < [EMAIL PROTECTED]> wrote: An alternative is to have two possible attributes: d.default_factory = listor d.default_value = 0with an exception being raised when both are defined (the test is done when theattribute is created, not when the lookup is performed). Why not have the factory function take the key being looked up as an argument? Seems like there would be uses to customize the default based on the key. It also forces you to handle list factory functions and constant factory functions (amongst others) to be handled the same way: d.default_factory = lambda k : list() d.default_factory = lambda k : 0 Dan Gass ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] defaultdict proposal round three
[Steven Bethard]
> * Should default_factory be an argument to the constructor? The three
> answers I see:
>
> - "No." I'm not a big fan of this answer. Since the whole point of
> creating a defaultdict type is to provide a default, requiring two
> statements (the constructor call and the default_factory assignment)
> to initialize such a dictionary seems a little inconvenient.
You still have to allow assignments to the default_factory attribute to allow
the factory to be changed:
dd.default_factory = SomeFactory
If it's too much effort to do the initial setup in two lines, a classmethod
could serve as an alternate constructor (leaving the regular contructor fully
interchangeable with dicts):
dd = defaultdict.setup(list, {'k1':'v1', 'k2:v2'})
or when there are no initial values:
dd = defaultdict.setup(list)
Raymond
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] defaultdict proposal round three
On 2/20/06, Dan Gass <[EMAIL PROTECTED]> wrote: > Why not have the factory function take the key being looked up as an > argument? Seems like there would be uses to customize the default based on > the key. It also forces you to handle list factory functions and constant > factory functions (amongst others) to be handled the same way: > > d.default_factory = lambda k : list() > d.default_factory = lambda k : 0 Guido's currently backing "a subclass that implements __getitem__() calling on_missing() and on_missing() ... calling default_factory unless it's None". I think for 90% of the use-cases, you don't need a key argument. If you do, you should subclass defaultdict and override the on_missing() method. STeVe -- Grammar am for people who can't think for myself. --- Bucky Katt, Get Fuzzy ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] defaultdict proposal round three
On 2/20/06, Brett Cannon <[EMAIL PROTECTED]> wrote: > While #3 is my preferred solution as well, it does pose a Liskov > violation if this is a direct dict subclass instead of storing a dict > internally (can't remember the name of the design pattern that does > this). But I think it is good to have the constructor be different > since it does also help drive home the point that this is not a > standard dict. I've heard this argument a few times now from different folks and I'm tired of it. It's wrong. It's not true. It's a dead argument. It's pushing up the daisies, so to speak. Please stop abusing Barbara Liskov's name and remember that the constructor signature is *not* part of the interface to an instance! Changing the constructor signature in a subclass does *not* cause *any* "Liskov" violations because the constructor is not called by *users* of the object -- it is only called to *create* an object. As the *user* of an object you're not allowed to *create* another instance (unless the object provides an explicit API to do so, of course, in which case you deal with that API's signature, not with the constructor). -- --Guido van Rossum (home page: http://www.python.org/~guido/) ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] defaultdict proposal round three
On 2/20/06, Dan Gass <[EMAIL PROTECTED]> wrote: > Why not have the factory function take the key being looked up as an > argument? This was considered and rejected already. You can already customize based on the key by overriding on_missing() [*]. If the factory were to take a key argument, we couldn't use list or int as the factory function; we'd have to write lambda key: list(). There aren't that many use cases for having the factory function depend on the key anyway; it's mostly on_missing() that needs the key so it can insert the new value into the dict. [*] Earlier in this thread I wrote that on_missing() could be inlined. I take that back; I think it's better to have it be available explicitly so you can override it without having to override __getitem__(). This is faster, assuming most __getitem__() calls find the key already inserted, and reduces the amount of code you have to write to customize the behavior; it also reduces worries about how to call the superclass __getitem__ method (catching KeyError *might* catch an unrelated KeyError caused by a bug in the key's __hash__ or __eq__ method). -- --Guido van Rossum (home page: http://www.python.org/~guido/) ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] defaultdict proposal round three
Guido van Rossum wrote: > I see two alternatives. Have you considered the third alternative that's been mentioned -- a wrapper? The issue of __contains__ etc. could be sidestepped by not giving the wrapper a __contains__ method at all. If you want to do an 'in' test you do it on the underlying dict, and then the semantics are clear. Greg ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] buildbot is all green
Martin v. Löwis wrote: > Michael Foord wrote: > >> Has a machine been volunteered ? >> > > Not yet. > > >> I have a spare machine and an always on connection. Would the 'right' >> development tools be needed ? (In the case of Microsoft they are a touch >> expensive I believe.) >> > > Any build process would do. I would prefer to see the official tools on > the buildbot (i.e. VS.NET 2003), Man, that's a difficult (and expensive) piece of software to obtain, unless you're a student. I couldn't find a legal non-academic version for less than £100. I might hunt around though. Shame. I suspect that hacking the free compilers to work would require more knowledge than I possess. Sorry. > but anything else that can build Python > and pass the test suite could do as well. > > One issue is that you also have to to work with me on defining the build > steps: what sequence of commands to send in what order. For Unix, that > is easy; for Windows, not so. > > Working with you wouldn't be a problem. Looks like the idea is a currently a bit of a dead dog though. All the best, Michael Foord > Regards, > Martin > > ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] bytes.from_hex() [Was: PEP 332 revival in coordination with pep 349?]
Bengt Richter wrote:
> On Sat, 18 Feb 2006 23:33:15 +0100, Thomas Wouters <[EMAIL PROTECTED]> wrote:
> note what base64 really is for. It's essence is to create a _character_
> sequence
> which can succeed in being encoded as ascii. The concept of base64 going
> str->str
> is really a mental shortcut for s_str.decode('base64').encode('ascii'), where
> 3 octets are decoded as code for 4 characters modulo padding logic.
Wouldn't it be...
obj.encode('base64').encode('ascii')
This would probably also work...
obj.encode('base64').decode('ascii') -> ascii alphabet in unicode
Where the underlying sequence might be ...
obj -> bytes -> bytes:base64 -> base64 ascii character set
The point is to have the data in a safe to transmit form that can
survive being encoded and decoded into different forms along the
transmission path and still be restored at the final destination.
base64 ascii character set -> bytes:base64 -> original bytes -> obj
* a related note, derived from this and your other post in this thread.
If the str type constructor had an encode argument like the unicode type
does, along with a str.encoded_with attribute. Then it might be
possible to depreciate the .decode() and .encode() methods and remove
them form P3k entirely or use them as data coders/decoders instead of
char type encoders.
It could also create a clear separation between character encodings and
data coding. The following should give an exception.
str(str, 'rot13'))
Rot13 isn't a character encoding, but a data coding method.
data_str.encode('rot13') # could be ok
But this wouldn't...
new_str = data_str.encode('latin_1')# could cause an exception
We'd have to use...
new_str = str(data_str, 'latin_1') # New string sub type...
Cheers,
Ronald Adam
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] buildbot is all green
Michael Foord <[EMAIL PROTECTED]> wrote: > Martin v. Löwis wrote: > > Any build process would do. I would prefer to see the official tools on > > the buildbot (i.e. VS.NET 2003), I can get a free academic license for VS.NET 2003 professional with my university (MSDNAA), and I've also got a Windows machine sitting in my office with a few spare cycles. > > One issue is that you also have to to work with me on defining the build > > steps: what sequence of commands to send in what order. For Unix, that > > is easy; for Windows, not so. If you're up for it, I'm up for it. It'll take me a bit to get the software on the machine. Want me to ping you when I get the toolset installed? - Josiah ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] problem with genexp
Regarding this Grammar change; (last October)
from argument: [test '=' ] test [gen_for]
to argument: test [gen_for] | test '=' test ['(' gen_for ')']
- to raise error for "bar(a = i for i in range(10)) )"
I think we should change it to
argument: test [gen_for] | test '=' test
instead of
argument: test [gen_for] | test '=' test ['(' gen_for ')']
that is, without ['(' gen_for ')'] . We don't need that extra term,
because "test" itself includes generator expressions - with all those
parensises.
Actually with that extra ['(' gen_for ')'] , foo(a= 10 (for y in
'a')) is grammartically correct ; although that error seems to be
checked elsewhere.
I tested without ['(' gen_for ')'] , and worked fine passing
Lib/test/test_genexps.py
-Jiwon
On 10/20/05, Neal Norwitz <[EMAIL PROTECTED]> wrote:
> On 10/16/05, Neal Norwitz <[EMAIL PROTECTED]> wrote:
> > On 10/10/05, Neal Norwitz <[EMAIL PROTECTED]> wrote:
> > > There's a problem with genexp's that I think really needs to get
> > > fixed. See http://python.org/sf/1167751 the details are below. This
> > > code:
> > >
> > > >>> foo(a = i for i in range(10))
> > >
> > > I agree with the bug report that the code should either raise a
> > > SyntaxError or do the right thing.
> >
> > The change to Grammar/Grammar below seems to fix the problem and all
> > the tests pass. Can anyone comment on whether this fix is
> > correct/appropriate? Is there a better way to fix the problem?
>
> Since no one responded other than Jiwon, I checked in this change. I
> did *not* backport it since what was syntactically correct in 2.4.2
> would raise an error in 2.4.3. I'm not sure which is worse. I'll
> leave it up to Anthony whether this should be backported.
>
> BTW, the change was the same regardless of old code vs. new AST code.
>
> n
> ___
> Python-Dev mailing list
> [email protected]
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> http://mail.python.org/mailman/options/python-dev/seojiwon%40gmail.com
>
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] defaultdict proposal round three
Brett Cannon wrote: > While #3 is my preferred solution as well, it does pose a Liskov > violation if this is a direct dict subclass I'm not sure we should be too worried about that. Inheritance in Python has always been more about implementation than interface, so Liskov doesn't really apply in the same way it does in statically typed languages. In other words, just because A inherits from B in Python isn't meant to imply that an A is a drop-in replacement for a B. Greg ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Win64 AMD64 (aka x64) binaries available64
[Martin v. Loewis wrote] > If you want to build extensions for this build using distutils, you > need to > ... > 2. open an AMD64 retail shell > ... > > It might be possible to drop 2) some day, but finding the SDK from > the registry is really tricky. Look for: def find_platform_sdk_dir() here: http://cvs.sourceforge.net/viewcvs.py/pywin32/pywin32/setup.py?view=markup That is the best code I know for doing that. Trent -- Trent Mick [EMAIL PROTECTED] ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] defaultdict proposal round three
Greg Ewing wrote: > In other words, just because A inherits from B in > Python isn't meant to imply that an A is a drop-in > replacement for a B. Hmm - this is interesting. I'm not arguing Liskov violations or anything ... However, *because* Python uses duck typing, I tend to feel that subclasses in Python *should* be drop-in replacements. If it's not a drop-in replacement, then it should probably not subclass, but just use duck typing (probably by wrapping). Subclassing implies a stronger relationship to me. Which is why I think I prefer using a wrapper for a default dict, rather than a subclass. Tim Delaney ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] buildbot is all green
Josiah Carlson wrote: > If you're up for it, I'm up for it. It'll take me a bit to get the > software on the machine. Want me to ping you when I get the toolset > installed? Sure! That should work fine. It would be best if the buildbot would run with the environment variables all set up, so that both svn.exe and devenv.exe can be found in the path. Then I would need the sequence of commands that the buildbot master should issue (svn update, build, run tests, clean). Regards, Martin ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] defaultdict proposal round three
Delaney, Timothy (Tim) wrote: > However, *because* Python uses duck typing, I tend to feel that > subclasses in Python *should* be drop-in replacements. If it's not a > drop-in replacement, then it should probably not subclass, but just use > duck typing (probably by wrapping). Inheritance is more about code reuse than about polymorphism. Regards, Martin ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] s/bytes/octet/ [Was:Re: bytes.from_hex() [Was: PEP 332 revival in coordination with pep 349?]]
Bengt Richter wrote: > On Sat, 18 Feb 2006 09:59:38 +0100, > =?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?= <[EMAIL PROTECTED]> wrote: > Thinking about bytes recently, it occurs to me that bytes are really not > intrinsically > numeric in nature. They don't necessarily represent uint8's. E.g., a binary > file is > really a sequence of bit octets in its most primitive and abstract sense. In that you would want to do different types of operations on single byte (an octet) than you would on str, or integer, I agree. Storing byte information as 16 or 32 bits ints could take up a rather lot of memory in some cases. I don't think it's been clarified yet weather the bytes() type would be implemented in C where it could be a single object with access to it's individual bytes via indexing, or python list type object which stores integers, chars or some other byte length object like octets. My first impression is that it would be done in C with a way to access and change the actual bytes. So a Python octet type wouldn't be needed. But if it is implemented as a Python subclass of list or array, then an octet type would probably also be desired. > Bottom line thought: binary octets aren't numeric ;-) +1 Cheers, Ronald Adam ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Win64 AMD64 (aka x64) binaries available64
Trent Mick wrote: > Look for: > def find_platform_sdk_dir() > here: > http://cvs.sourceforge.net/viewcvs.py/pywin32/pywin32/setup.py?view=markup > > That is the best code I know for doing that. Right; I was planning something similar (although I would probably hard-code the 2003 SP1 registry key - it is not at all certain that future SDK releases will use the same registry scheme, and Microsoft has tricked users often enough in thinking they understood the scheme, just to change it with the next release entirely). Regards, Martin ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] defaultdict proposal round three
On Feb 20, 2006, at 3:04 PM, Brett Cannon wrote: ... >> - "Yes and it should be the only constructor argument." This is my ... > While #3 is my preferred solution as well, it does pose a Liskov > violation if this is a direct dict subclass instead of storing a dict How so? Liskov's principle is (in her own words): If for each object o1 of type S there is an object o2 of type T such that for all programs P defined in terms of T, the behavior of P is unchanged when o1 is substituted for o2 then S is a subtype of T. How can this ever be broken by the mere presence of incompatible signatures for T's and S's ctors? I believe the principle, as stated above, was imperfectly stated, btw (it WAS preceded by "something like the following substitution property", indicating that Liskov was groping towards a good formulation), but that's an aside -- the point is that the principle is about substitution of _objects_, i.e., _instances_ of the types S and T, not about substitution of the _types_ themselves for each other. Instances exist and are supposed to satisfy their invariants _after_ ctors are done executing; ctor's signatures don't matter. In Python, of course, you _could_ call type(o2)(...) and possibly get different behavior if that was changed into type(o1)(...) -- the curse of powerful introspection;-). But then, isn't it trivial to obtain cases in which the behavior is NOT unchanged? If it was always unchanged, what would be the point of ever subclassing?-) Say that o2 is an int and o1 is a bool -- just a "print o2" already breaks the principle as stated (it's harder to get a simpler P than this...). Unless you have explicitly documented invariants (such as "any 'print o' must emit 1+ digits followed by a newline" for integers), you cannot say that some alleged subclass is breaking Liskov's property, in general. Mere "change of behavior" in the most general case cannot qualify, if method overriding is to be any use; such change IS traditionally allowed as long as preconditions are looser and postconditions are stricter; and I believe than in any real-world subclassing, with sufficient introspection you'll always find a violation E.g., a subtype IS allowed to add methods, by Liskov's specific example; but then, len(dir(o1)) cannot fail to be a higher number than len(dir(o2)), from which you can easily construct a P which "changes behavior" for any definition you care to choose. E.g., pick constant N as the len(dir(...)) for instances of type T, and say that M>N is the len(dir(...)) for instances of S. Well, then, math.sqrt(N-len(dir(o2))) is well defined -- but change o2 into o1, and since N-M is <0, you'll get an exception. If you can give an introspection-free example showing how Liskov substitution would be broken by a mere change to incompatible signature in the ctor, I'll be grateful; but I don't think it can be done. Alex ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] defaultdict proposal round three
[Alex] >> I see d[k]+=1 as a substantial improvement -- conceptually more >> direct, "I've now seen one more k than I had seen before". [Guido] > Yes, I now agree. This means that I'm withdrawing proposal A (new > method) and championing only B (a subclass that implements > __getitem__() calling on_missing() and on_missing() defined in that > subclass as before, calling default_factory unless it's None). I don't > think this crisis is big enough to need *two* solutions, and this > example shows B's superiority over A. FWIW, I'm happy with the proposal and think it is a nice addition to Py2.5. Raymond ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] buildbot vs. Windows
If you're willing to commit to running a buildbot, and the only thing preventing you is shelling out $$$ to Microsoft, send me e-mail. I'll compile a list to send to the PSF and we'll either poke Microsoft to provide some more free licenses or pay for it ourselves. This is what the PSF is for! Note the emphasis on the word "commit", please. I'm setting an arbitrary deadline of Saturday Feb 25 so I don't have to monitor indefinitely. -- Aahz ([EMAIL PROTECTED]) <*> http://www.pythoncraft.com/ "19. A language that doesn't affect the way you think about programming, is not worth knowing." --Alan Perlis ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] defaultdict proposal round three
On Feb 20, 2006, at 5:05 PM, Raymond Hettinger wrote: > [Alex] >>> I see d[k]+=1 as a substantial improvement -- conceptually more >>> direct, "I've now seen one more k than I had seen before". > > [Guido] >> Yes, I now agree. This means that I'm withdrawing proposal A (new >> method) and championing only B (a subclass that implements >> __getitem__() calling on_missing() and on_missing() defined in that >> subclass as before, calling default_factory unless it's None). I >> don't >> think this crisis is big enough to need *two* solutions, and this >> example shows B's superiority over A. > > FWIW, I'm happy with the proposal and think it is a nice addition > to Py2.5. OK, sounds great to me. collections.defaultdict, then? Alex ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] defaultdict proposal round three
in two ways: 1) dict.get doesn't work for object dicts or in exec/eval contexts, and 2) dict.get requires me to generate the default value even if I'm not going to use it, a process which may be expensive. On 2/20/06, Raymond Hettinger <[EMAIL PROTECTED]> wrote: > [Crutcher Dunnavant ] > >> There are many times that I want d[key] to give me a value even when > >> it isn't defined, but that doesn't always mean I want to _save_ that > >> value in the dict. > > How does that differ from the existing dict.get method? > > > Raymond > -- Crutcher Dunnavant <[EMAIL PROTECTED]> littlelanguages.com monket.samedi-studios.com ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] defaultdict proposal round three
On 2/20/06, Alex Martelli <[EMAIL PROTECTED]> wrote:
> > [Alex]
> >>> I see d[k]+=1 as a substantial improvement -- conceptually more
> >>> direct, "I've now seen one more k than I had seen before".
> >
> > [Guido]
> >> Yes, I now agree. This means that I'm withdrawing proposal A (new
> >> method) and championing only B (a subclass that implements
> >> __getitem__() calling on_missing() and on_missing() defined in that
> >> subclass as before, calling default_factory unless it's None). I don't
> >> think this crisis is big enough to need *two* solutions, and this
> >> example shows B's superiority over A.
[Raymond]
> > FWIW, I'm happy with the proposal and think it is a nice addition
> > to Py2.5.
[Alex]
> OK, sounds great to me. collections.defaultdict, then?
I have a patch ready that implements this. I've assigned it to Raymond
for review. I'm just reusing the same SF patch as before:
python.org/sf/1433928.
One subtlety: for maximul flexibility and speed, the standard dict
type now defines an on_missing(key) method; however this version
*just* raises KeyError and the implementation actually doesn't call it
unless the class is a subtype (with the possibility of overriding
on_missing()).
collections.defaultdict overrides on_missing(key) to insert and return
self.fefault_factory() if it is not empty; otherwise it raises
KeyError. (It should really call the base class on_missing() but I
figured I'd just in-line it which is easier to code in C than a
super-call.)
The defaultdict signature takes an optional positional argument which
is the default_factory, defaulting to None. The remaining positional
and all keyword arguments are passed to the dict constructor. IOW:
d = defaultdict(list, [(1, 2)])
is equivalent to:
d = defaultdict()
d.default_factory = list
d.update([(1, 2)])
At this point, repr(d) will be:
defaultdict(, {1: 2})
Once Raymond approves the patch I'll check it in.
--
--Guido van Rossum (home page: http://www.python.org/~guido/)
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] defaultdict proposal round three
On 2/20/06, Guido van Rossum <[EMAIL PROTECTED]> wrote:
> [stuff with typos]
Here's the proofread version:
I have a patch ready that implements this. I've assigned it to Raymond
for review. I'm just reusing the same SF patch as before:
http://python.org/sf/1433928 .
One subtlety: for maximal flexibility and speed, the standard dict
type now defines an on_missing(key) method; however this version
*just* raises KeyError and the implementation actually doesn't call it
unless the class is a subtype (with the possibility of overriding
on_missing()).
collections.defaultdict overrides on_missing(key) to insert and return
self.default_factory() if it is not None; otherwise it raises
KeyError. (It should really call the base class on_missing() but I
figured I'd just in-line it which is easier to code in C than a
super-call.)
The defaultdict signature takes an optional positional argument which
is the default_factory, defaulting to None. The remaining positional
and all keyword arguments are passed to the dict constructor. IOW:
d = defaultdict(list, [(1, 2)])
is equivalent to:
d = defaultdict()
d.default_factory = list
d.update([(1, 2)])
At this point, repr(d) will be:
defaultdict(, {1: 2})
Once Raymond approves the patch I'll check it in.
--
--Guido van Rossum (home page: http://www.python.org/~guido/)
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] defaultdict proposal round three
"Martin v. Löwis" wrote: > Delaney, Timothy (Tim) wrote: >> However, *because* Python uses duck typing, I tend to feel that >> subclasses in Python *should* be drop-in replacements. If it's not a >> drop-in replacement, then it should probably not subclass, but just >> use duck typing (probably by wrapping). > > Inheritance is more about code reuse than about polymorphism. Oh - it's definitely no hard-and-fast rule. owever, I have found that *usually* people (including myself) only subclass when they want an is-a relationship, whereas duck typing is behaves-like. In any case, Guido has produced a patch, and the tone of his message sounded like a Pronouncement ... Tim Delaney ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] defaultdict proposal round three
On 2/20/06, Greg Ewing <[EMAIL PROTECTED]> wrote: > Have you considered the third alternative that's been > mentioned -- a wrapper? I don't like that at all. It's quite tricky to implement a fully transparent wrapper that supports all the special methods (__setitem__ etc.). It will be slower. And it will be more cumbersome to use. > The issue of __contains__ etc. could be sidestepped by > not giving the wrapper a __contains__ method at all. > If you want to do an 'in' test you do it on the > underlying dict, and then the semantics are clear. The semantics of defaultdict are crystal clear. __contains__(), keys() and friends represent the *actual*, *current* keys. Only __getitem__() calls on_missing() when the key is not present; being a "hook", on_missing() can do whatever it wants. What's the practical use case for not wanting __contains__() to function? All I hear is fear of theoretical bugs. -- --Guido van Rossum (home page: http://www.python.org/~guido/) ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] readline compilarion fails on OSX
On OSX (10.4.4) the readline module in the svn HEAD fails compilation as follows. This is particularly strange since the buildbot is green for OSX... What could be up with this? building 'readline' extension gcc -fno-strict-aliasing -Wno-long-double -no-cpp-precomp -mno-fused-madd -DNDEBUG -g -O3 -Wall -Wstrict-prototypes -I. -I/Users/guido/projects/python/trunk/./Include -I/Users/guido/projects/python/trunk/./Mac/Include -I../Include -I. -I/usr/local/include -I/Users/guido/projects/python/trunk/Include -I/Users/guido/projects/python/trunk/osx -c /Users/guido/projects/python/trunk/Modules/readline.c -o build/temp.darwin-8.4.0-Power_Macintosh-2.5/readline.o /Users/guido/projects/python/trunk/Modules/readline.c: In function 'write_history_file': /Users/guido/projects/python/trunk/Modules/readline.c:112: warning: implicit declaration of function 'history_truncate_file' /Users/guido/projects/python/trunk/Modules/readline.c: In function 'py_remove_history': /Users/guido/projects/python/trunk/Modules/readline.c:301: warning: implicit declaration of function 'remove_history' /Users/guido/projects/python/trunk/Modules/readline.c:301: warning: assignment makes pointer from integer without a cast /Users/guido/projects/python/trunk/Modules/readline.c:310: warning: passing argument 1 of 'free' discards qualifiers from pointer target type /Users/guido/projects/python/trunk/Modules/readline.c:312: warning: passing argument 1 of 'free' discards qualifiers from pointer target type /Users/guido/projects/python/trunk/Modules/readline.c: In function 'py_replace_history': /Users/guido/projects/python/trunk/Modules/readline.c:338: warning: implicit declaration of function 'replace_history_entry' /Users/guido/projects/python/trunk/Modules/readline.c:338: warning: assignment makes pointer from integer without a cast /Users/guido/projects/python/trunk/Modules/readline.c:347: warning: passing argument 1 of 'free' discards qualifiers from pointer target type /Users/guido/projects/python/trunk/Modules/readline.c:349: warning: passing argument 1 of 'free' discards qualifiers from pointer target type /Users/guido/projects/python/trunk/Modules/readline.c: In function 'get_current_history_length': /Users/guido/projects/python/trunk/Modules/readline.c:453: error: 'HISTORY_STATE' undeclared (first use in this function) /Users/guido/projects/python/trunk/Modules/readline.c:453: error: (Each undeclared identifier is reported only once /Users/guido/projects/python/trunk/Modules/readline.c:453: error: for each function it appears in.) /Users/guido/projects/python/trunk/Modules/readline.c:453: error: 'hist_st' undeclared (first use in this function) /Users/guido/projects/python/trunk/Modules/readline.c:455: warning: implicit declaration of function 'history_get_history_state' /Users/guido/projects/python/trunk/Modules/readline.c: In function 'insert_text': /Users/guido/projects/python/trunk/Modules/readline.c:503: warning: implicit declaration of function 'rl_insert_text' /Users/guido/projects/python/trunk/Modules/readline.c: In function 'on_completion': /Users/guido/projects/python/trunk/Modules/readline.c:637: error: 'rl_attempted_completion_over' undeclared (first use in this function) /Users/guido/projects/python/trunk/Modules/readline.c: In function 'flex_complete': /Users/guido/projects/python/trunk/Modules/readline.c:675: warning: passing argument 2 of 'completion_matches' from incompatible pointer type /Users/guido/projects/python/trunk/Modules/readline.c: In function 'setup_readline': /Users/guido/projects/python/trunk/Modules/readline.c:700: warning: passing argument 2 of 'rl_bind_key_in_map' from incompatible pointer type /Users/guido/projects/python/trunk/Modules/readline.c:701: warning: passing argument 2 of 'rl_bind_key_in_map' from incompatible pointer type /Users/guido/projects/python/trunk/Modules/readline.c: In function 'readline_until_enter_or_signal': /Users/guido/projects/python/trunk/Modules/readline.c:758: warning: passing argument 2 of 'rl_callback_handler_install' from incompatible pointer type /Users/guido/projects/python/trunk/Modules/readline.c:788: warning: implicit declaration of function 'rl_free_line_state' /Users/guido/projects/python/trunk/Modules/readline.c:789: warning: implicit declaration of function 'rl_cleanup_after_signal' /Users/guido/projects/python/trunk/Modules/readline.c: In function 'call_readline': /Users/guido/projects/python/trunk/Modules/readline.c:883: error: 'HISTORY_STATE' undeclared (first use in this function) /Users/guido/projects/python/trunk/Modules/readline.c:883: error: 'state' undeclared (first use in this function) /Users/guido/projects/python/trunk/Modules/readline.c:885: warning: assignment discards qualifiers from pointer target type (Yes, the keynote slides are coming along just fine... :-) -- --Guido van Rossum (home page: http://www.python.org/~guido/) ___ Python-Dev mailing list [email protected] http://mail.python.org/mailm
Re: [Python-Dev] readline compilarion fails on OSX
On Feb 20, 2006, at 6:48 PM, Guido van Rossum wrote: > On OSX (10.4.4) the readline module in the svn HEAD fails compilation > as follows. This is particularly strange since the buildbot is green > for OSX... What could be up with this? > > building 'readline' extension -lots of build junk- In Apple's quest to make our lives harder, they installed BSD libedit and symlinked it to readline. Python doesn't like that. The buildbot might have a real readline installation, or maybe the buildbot is skipping those tests. You'll need to install a real libreadline if you want it to work. I've also put together a little tarball that'll build readline.so statically, and there's pre-built eggs for OS X so the easy_install should be quick: http://python.org/pypi/readline -bob ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] readline compilarion fails on OSX
Thanks! That worked. But shouldn't we try to fix setup.py to detect this situation instead of making loud clattering noises? --Guido On 2/20/06, Bob Ippolito <[EMAIL PROTECTED]> wrote: > > On Feb 20, 2006, at 6:48 PM, Guido van Rossum wrote: > > > On OSX (10.4.4) the readline module in the svn HEAD fails compilation > > as follows. This is particularly strange since the buildbot is green > > for OSX... What could be up with this? > > > > building 'readline' extension > -lots of build junk- > > In Apple's quest to make our lives harder, they installed BSD libedit > and symlinked it to readline. Python doesn't like that. The > buildbot might have a real readline installation, or maybe the > buildbot is skipping those tests. > > You'll need to install a real libreadline if you want it to work. > > I've also put together a little tarball that'll build readline.so > statically, and there's pre-built eggs for OS X so the easy_install > should be quick: > http://python.org/pypi/readline > > -bob > > -- --Guido van Rossum (home page: http://www.python.org/~guido/) ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] buildbot vs. Windows
[Aahz] > If you're willing to commit to running a buildbot, and the only thing > preventing you is shelling out $$$ to Microsoft, send me e-mail. I'll > compile a list to send to the PSF and we'll either poke Microsoft to > provide some more free licenses or pay for it ourselves. This is what > the PSF is for! Speaking as a PSF director, I might not vote for that :-) Fact is I've been keeping the build & tests 100% healthy on WinXP Pro, and that requires more than just running the tests (it also requires repairing compiler warnings and Unixisms). Speaking of which, a number of test failures over the past few weeks were provoked here only under -r (run tests in random order) or under a debug build, and didn't look like those were specific to Windows. Adding -r to the buildbot test recipe is a decent idea. Getting _some_ debug-build test runs would also be good (or do we do that already?). Anyway, since XP Pro is effectively covered, I'd be keener to see a Windows buildbot running under a different flavor of Windows. I expect I'll eventually volunteer my home box to run an XP buildbot, but am in no hurry (and probably won't leave any machine here turned on 24/7 regardless). ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] bytes.from_hex()
> "Martin" == Martin v Löwis <[EMAIL PROTECTED]> writes: Martin> Please do take a look. It is the only way: If you were to Martin> embed base64 *bytes* into character data content of an XML Martin> element, the resulting XML file might not be well-formed Martin> anymore (if the encoding of the XML file is not an ASCII Martin> superencoding). Excuse me, I've been doing category theory recently. By "embedding" I mean a map from an intermediate object which is a stream of bytes to the corresponding stream of characters. In the case of UTF-16-coded characters, this would necessarily imply a representation change, as you say. What I advocate for Python is to require that the standard base64 codec be defined only on bytes, and always produce bytes. Any representation change should be done explicitly. This is surely conformant with RFC 2045's definition and with RFC 3548. -- School of Systems and Information Engineering http://turnbull.sk.tsukuba.ac.jp University of TsukubaTennodai 1-1-1 Tsukuba 305-8573 JAPAN Ask not how you can "do" free software business; ask what your business can "do for" free software. ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Proposal: defaultdict
On 2/20/06, Jim Jewett <[EMAIL PROTECTED]> wrote: > Adam Olsen asked: > > ... d.getorset(key, func) would work in your use cases? > > It is an improvement over setdefault, because it doesn't > always evaluate the expensive func. (But why should every > call have to pass in the function, when it is a property of > the dictionary?) Because usually it's a property of how you use it, not a property of the dictionary. The dictionary is just a generic storage mechanism. > [snip] > In other words, the program would work correctly if I passed > in a normal but huge dictionary; I want to avoid that for reasons > of efficiency. This isn't the only use for a mapping, but it is > the only one I've seen where KeyError is "expected" by the > program's normal flow. Looking at your explanation, I agree, getorset is useless for that use case. However, I'm beginning to think we shouldn't be comparing them. defaultdict is a powerful but heavyweight option, intended for complicated behavior. getorset and setdefault are intended to be very lightweight, even lighter than the "try/except KeyError" and "if key not in X: X[key] = default" memes we have right now. getorset's factory function is only appropriate for preexisting functions, not user defined ones. Essentially, I believe getorset should be discussed on its own merits, independent of the addition of a defaultdict class. Perhaps discussion of it (and the deprecation of setdefault) should wait until after defaultdict has been completed? -- Adam Olsen, aka Rhamphoryncus ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Proposal: defaultdict
On Sat, 2006-02-18 at 12:53 +0100, Pierre Barbier de Reuille wrote:
> > Guido> Over lunch with Alex Martelli, he proposed that a subclass of
> > Guido> dict with this behavior (but implemented in C) would be a good
> > Guido> addition to the language.
I agree that .setdefault() is a well-intentioned failure, although I'm
much less concerned about any potential performance impact than the fact
that it's completely unreadable. And while I like the basic idea, I
also agree that deriving from dict is problematic, both because of the
constructor signature is tough to forward, but also because dict is such
a fundamental type that APIs that return dicts may have to be changed to
allow passing in a factory type.
I'd rather like to see what Pierre proposes, with a few minor
differences.
> Well, first not ot break the current interface, and second because I think it
> reads better I would prefer :
>
> d = {'a': 1}'
> d['b'] # raises KeyError
> d.get('c') # evaluates to None
> d.default = 42
> d['b'] # evaluates to 42
> d.get('c') # evaluates to 42
So far so good.
> And to undo the default, you can simply do :
>
> del d.default
Although this I'm not crazy about. If you let .default be a callable,
you could also write this as
def keyerror(): raise KeyError
d.default = keyerror
or possibly just this as a shortcut:
d.default = KeyError
> > The only question in my mind is whether or not getting a non-existent value
> > under the influence of a given default value should stick that value in the
> > dictionary or not.
Agreed. I'm not sure whether .get(onearg) should return None
or .default. I /think/ I want the latter, but I'd have to play with
some real code to know for sure.
-Barry
signature.asc
Description: This is a digitally signed message part
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] bytes.from_hex()
On Feb 20, 2006, at 7:25 PM, Stephen J. Turnbull wrote: >> "Martin" == Martin v Löwis <[EMAIL PROTECTED]> writes: > > Martin> Please do take a look. It is the only way: If you were to > Martin> embed base64 *bytes* into character data content of an XML > Martin> element, the resulting XML file might not be well-formed > Martin> anymore (if the encoding of the XML file is not an ASCII > Martin> superencoding). > > Excuse me, I've been doing category theory recently. By "embedding" I > mean a map from an intermediate object which is a stream of bytes to > the corresponding stream of characters. In the case of UTF-16-coded > characters, this would necessarily imply a representation change, as > you say. > > What I advocate for Python is to require that the standard base64 > codec be defined only on bytes, and always produce bytes. Any > representation change should be done explicitly. This is surely > conformant with RFC 2045's definition and with RFC 3548. +1 -bob ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] PEP for Better Control of Nested Lexical Scopes
I am considering developing a PEP for enabling a mechanism to assign to free variables in a closure (nested function). My rationale is that with the advent of PEP 227 , Python has proper nested lexical scopes, but can have undesirable behavior (especially with new developers) when a user makes wants to make an assignment to a free variable within a nested function. Furthermore, after seeing numerous kludges to "solve" the problem with a mutable object, like a list, as the free variable do not seem "Pythonic." I have also seen mention that the use of classes can mitigate this, but that seems, IMHO, heavy handed in cases when an elegant solution using a closure would suffice and be more appropriate--especially when Python already has nested lexical scopes. I propose two possible approaches to solve this issue: 1. Adding a keyword such as "use" that would follow similar semantics as "global" does today. A nested scope could declare names with this keyword to enable assignment to such names to change the closest parent's binding. The semantic would be to keep the behavior we experience today but tell the compiler/interpreter that a name declared with the "use" keyword would explicitly use an enclosing scope. I personally like this approach the most since it would seem to be in keeping with the current way the language works and would probably be the most backwards compatible. The semantics for how this interacts with the global scope would also need to be defined (should "use" be equivalent to a global when no name exists all parent scopes, etc.) def incgen( inc = 1 ) : a = 6 def incrementer() : use a #use a, inc <-- list of names okay too a += inc return a return incrementer Of course, this approach suffers from a downside that every nested scope that wanted to assign to a parent scope's name would need to have the "use" keyword for those names--but one could argue that this is in keeping with one of Python's philosophies that "Explicit is better than implicit" (PEP 20). This approach also has to deal with a user declaring a name with " use" that is a named parameter--this would be a semantic error that could be handled like "global " does today with a SyntaxError. 2. Adding a keyword such as "scope" that would behave similarly to _javascript_'s " var" keyword. A name could be declared with such a keyword optionally and all nested scopes would use the declaring scope's binding when accessing or assigning to a particular name. This approach has similar benefits to my first approach, but is clearly more top-down than the first approach. Subsequent "scope" declarations would create a new binding at the declaring scope for the declaring and child scopes to use. This could potentially be a gotcha for users expecting the binding semantics in place today. Also the scope keyword would have to be allowed to be used on parameters to allow such parameter names to be used in a similar fashion in a child scope. def incgen( inc = 1 ) : #scope inc <-- allow scope declaration for bound parameters (not a big fan of this) scope a = 6 def incrementer() : a += inc return a return incrementer This approach would be similar to languages like _javascript_ that allow for explicit scope binding with the use of "var" or more static languages that allow re-declaring names at lower scopes. I am less in favor of this, because I don't think it feels very "Pythonic". As a point of reference, some languages such as Ruby will only bind a new name to a scope on assignment when an enclosing scope does not have the name bound. I do believe the Python name binding semantics have issues (for which the "global" keyword was born), but I feel that the "fixing" the Python semantic to a more "Ruby-like" one adds as many problems as it solves since the "Ruby-like" one is just as implicit in nature. Not to mention the backwards compatibility impact is probably much larger. I would like the community's opinion if there is enough out there that think this would be a worthwile endevour--or if there is already an initiative that I missed. Please let me know your questions, comments. Best Regards, Almann-- Almann T. Goo[EMAIL PROTECTED] ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Proposal: defaultdict
On Fri, 2006-02-17 at 11:09 -0800, Guido van Rossum wrote: > Thanks for all the constructive feedback. Here are some responses and > a new proposal. > > - Yes, I'd like to kill setdefault() in 3.0 if not sooner. A worthy goal, but not possible unless you want to break existing code. I don't think it's worth a DeprecationWarning either. Slating it for removal in 3.0 seems fine. Everything else about your proposal seems great. -Barry signature.asc Description: This is a digitally signed message part ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] readline compilarion fails on OSX
Guido> But shouldn't we try to fix setup.py to detect this situation Guido> instead of making loud clattering noises? Here's a first-cut try at a setup.py patch: http://python.org/sf/1435651 Unfortunately, I don't think distutils provides a clean way to detect symbols the way configure does, so it's a bit clumsy... Skip ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Proposal: defaultdict
"Adam Olsen" <[EMAIL PROTECTED]> wrote: > However, I'm beginning to think we shouldn't be comparing them. > defaultdict is a powerful but heavyweight option, intended for > complicated behavior. Check out Guido's patch. It's not that "heavyweight", and its intended behavior is to make some operations *more* intuitive, if not a bit faster in some cases. Whether or not getorset is introduced, I don't much care, as defaultdict will cover every use case I've been using setdefault for, as well as most of my use cases for get. - Josiah ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] defaultdict proposal round three
[Guido] > ... > What's the practical use case for not wanting __contains__() to > function? I don't know. I have practical use cases for wanting __contains__() to function, but there's been no call for those. For an example, think of any real use ;-) For example, I often use dicts to represent multisets, where a key maps to a strictly positive count of the number of times that key appears in the multiset. A default of 0 is the right thing to return for a key not in the multiset, so that M[k] += 1 works to add another k to multiset M regardless of whether k was already present. I sure hope I can implement multiset intersection as, e.g., def minter(a, b): if len(b) < len(a): # make `a` the smaller, and iterate over it a, b = b, a result = defaultdict defaulting to 0, however that's spelled for k in a: if k in b: result[k] = min(a[k], b[k]) return result Replacing the loop nest with: for k in a: result[k] = min(a[k], b[k]) would be semantically correct so far as it goes, but pragmatically wrong: I maintain my "strictly positive count" invariant because consuming RAM to hold elements "that aren't there" can be a pragmatic disaster. (When `k` is in `a` but not in `b`, I don't want `k` to be stored in `result`) I have other examples, but they come so easily it's better to leave that an exercise for the reader. ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP for Better Control of Nested Lexical Scopes
"Almann T. Goo" <[EMAIL PROTECTED]> wrote: > I would like the community's opinion if there is enough out there that think > this would be a worthwile endevour--or if there is already an initiative > that I missed. Please let me know your questions, comments. -1 Mechanisms which rely on manipulating variables within closures or nested scopes to function properly can be elegant, but I've not yet seen one that *really* is. You state that using classes can be "heavy handed", but one of the major uses of classes is as a *namespace*. Many desired uses of closures (including the various uses you have outlined) is to hide a *namespace*, and combining both closures with classes can offer that to you, without requiring a language change. Of course using classes directly with a bit of work can offer you everything you want from a closure, with all of the explcitness that you could ever want. As an aside, you mention both 'use' and 'scope' as possible keyword additions for various uses of nested scopes. In my experience, when one goes beyond 3 or so levels of nested scopes (methods of a class defined within a class namespace, or perhaps methods of a class defined within a method of a class), it starts getting to the point where the programmer is trying to be too clever. - Josiah ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP for Better Control of Nested Lexical Scopes
> Mechanisms which rely on manipulating variables within closures or > nested scopes to function properly can be elegant, but I've not yet seen > one that *really* is. This really isn't a case for or against what I'm proposing since we can already do this in today's Python with mutable variables in an enclosing scope (see below). I am proposing a language change to help make closures more orthogonal to the scoping constructs that are already in place for the global scope. > You state that using classes can be "heavy handed", > but one of the major uses of classes is as a *namespace*. Many desired > uses of closures (including the various uses you have outlined) > is to hide a *namespace*, and combining both closures with classes can offer > that to you, without requiring a language change. Closures are also used in more functional styles of programming for defining customized control structures (those Ruby folks like them for this purpose). Granted you can do this with classes/objects and defining interfaces the end result can be somewhat un-natural for some problems--but I don't want to get into an argument between closures vs. objects since that is not what my proposal is aimed at and Python already has both. > Of course using > classes directly with a bit of work can offer you everything you want > from a closure, with all of the explcitness that you could ever want. Really, the easiest way to emulate what I want in today's Python is to create a mutable object (like a dict or list) in the enclosing scope to work around the semantic that the first assignment in a local scope binds a new name. Doing this seems rather un-natural and forcing the use of classes doesn't seem more natural def incgen( inc = 1 ) : env = [ 6 ] def incrementor() : env[ 0 ] += inc return env[ 0 ] return incrementor This is a work around for something a developer cannot do more naturally today. I do not think using some combination of classes and closures makes things clearer--it is still working around what I would construe as the non-orthogonal nature of nested lexical scopes in Python since the language provides a construct to deal with the problem for global variables. a = 6 def incgen( inc = 1 ) : def incrementor() : global a a += inc return a return incrementor Granted this is a somewhat trivial example, but I think it demonstrates my point about how nested lexical scopes are second class (since the language has no equivalent construct for them) and don't behave like the global scope. > As an aside, you mention both 'use' and 'scope' as possible keyword > additions for various uses of nested scopes. In my experience, when one > goes beyond 3 or so levels of nested scopes (methods of a class defined > within a class namespace, or perhaps methods of a class defined within a > method of a class), it starts getting to the point where the programmer > is trying to be too clever. Even though I may agree with you on this, your argument is more of an argument against PEP 227 than what I am proposing. Again, today's Python already allows a developer to have deep nested scopes. -Almann -- Almann T. Goo [EMAIL PROTECTED] ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] defaultdict proposal round three
On Mon, 20 Feb 2006 11:09:48 -0800, Alex Martelli <[EMAIL PROTECTED]> wrote:
>
>On Feb 20, 2006, at 8:35 AM, Raymond Hettinger wrote:
>
>> [GvR]
>>> I'm not convinced by the argument
>>> that __contains__ should always return True
>>
>> Me either. I cannot think of a more useless behavior or one more
>> likely to have
>> unexpected consequences. Besides, as Josiah pointed out, it is
>> much easier for
>> a subclass override to substitute always True return values than
>> vice-versa.
>
>Agreed on all counts.
>
>> I prefer this approach over subclassing. The mental load from an
>> additional
>> method is less than the load from a separate type (even a
>> subclass). Also,
>> avoidance of invariant issues is a big plus. Besides, if this allows
>> setdefault() to be deprecated, it becomes an all-around win.
>
>I'd love to remove setdefault in 3.0 -- but I don't think it can be
>done before that: default_factory won't cover the occasional use
>cases where setdefault is called with different defaults at different
>locations, and, rare as those cases may be, any 2.* should not break
>any existing code that uses that approach.
>
>>> - Even if the default_factory were passed to the constructor, it
>>> still
>>> ought to be a writable attribute so it can be introspected and
>>> modified. A defaultdict that can't change its default factory after
>>> its creation is less useful.
>>
>> Right! My preference is to have default_factory not passed to the
>> constructor,
>> so we are left with just one way to do it. But that is a nit.
>
How about doing it as an expression, empowering ( ;-) the dict just afer
creation?
E.g., for
d = dict()
d.default_factory = list
you could write
d = dict()**list
I made a hack to illustrate functionality (code at end).
DD simulates the new dict without defaults.
>>> d = DD(a=1)
>>> d
{'a': 1}
So d is the plain dict with no default action enabled
>>> ddl = DD()**list
>>> ddl
DD({} <= list)
This is a new dict with list default factory
>>> ddl[42]
[]
Beats the heck out of ddl.setdefault(42, [])
>>> ddl[42].append(1)
>>> ddl[42].append(2)
>>> ddl
DD({42: [1, 2]} <= list)
Now take the non-default dict d and make an int default wrapper
>>> ddi = d**int
>>> ddi
DD({'a': 1} <= int)
Show there's no default on the orig:
>>> d['b']+=1
Traceback (most recent call last):
File "", line 1, in ?
KeyError: 'b'
But use the wrapper proxy:
>>> ddi['b']+=1
>>> ddi
DD({'a': 1, 'b': 1} <= int)
>>> ddi['b']+=1
>>> ddi
DD({'a': 1, 'b': 2} <= int)
Note that augassign works. And info is visible in d:
>>> d
{'a': 1, 'b': 2}
probably unusual use, but a one-off
d.setdefault('S', set()).add(42)
can be written
>>> (d**set)['S'].add(42)
>>> d
{'a': 1, 'S': set([42]), 'b': 2}
i.e., d**different_factory_value creates a temporary d-accessing proxy
with default_factory set to different_factory_value, without affecting
other bindings of d unless you rebind them with the expression result.
I haven't implemented a check for compatible types on mixed defaults.
e.g. the integer-default proxy will show 'S', but note:
>>> ddi['S']
set([42])
>>> ddi['S'] += 5
Traceback (most recent call last):
File "", line 1, in ?
TypeError: unsupported operand type(s) for +=: 'set' and 'int'
I guess the programmer deserves it ;-)
You can get a new defaulting proxy from an existing one, as it will
use the same base plain dict:
>>> ddd = ddi**dict
>>> ddd
DD({'a': 1, 'S': set([42]), 'b': 2, 'd': 0} <= dict)
>>> ddd['adict'].update(check=1, this=2)
>>> ddd
DD({'a': 1, 'S': set([42]), 'b': 2, 'adict': {'this': 2, 'check': 1}, 'd': 0}
<= dict)
>>> d
{'a': 1, 'S': set([42]), 'b': 2, 'adict': {'this': 2, 'check': 1}, 'd': 0}
Not sure what the C implementation ramifications would be, but it makes
setdefault easy to spell. And using both modes interchangeably is easy.
And stuff like
>>> d = DD()**int
>>> for c in open('dd.py').read(): d[c]+=1
...
>>> print sorted(d.items(), key=lambda t:t[1])[-5:]
[('f', 50), ('t', 52), ('_', 71), ('e', 74), (' ', 499)]
Is nice ;-)
>>> len(d)
Traceback (most recent call last):
File "", line 1, in ?
TypeError: len() of unsized object
Oops.
>>> len(d.keys())
40
>>> len(open('dd.py').read())
1185
>>> sum(d.values())
1185
>No big deal either way, but I see "passing the default factory to the
>ctor" as the "one obvious way to do it", so I'd rather have it (be it
>with a subclass or a classmethod-alternate constructor). I won't weep
>bitter tears if this drops out, though.
>
>
>>> - It would be unwise to have a default value that would be called if
>>> it was callable: what if I wanted the default to be a class instance
>>> that happens to have a __call__ method for unrelated reasons?
>>> Callability is an elusive propperty; APIs should not attempt to
>>> dynamically decide whether an argument is callable or not.
>>
>> That makes sense, though it seems over-the-top to need a zero-
[Python-Dev] Memory Error the right error for coding cookie promise violation?
Perhaps a more informative message would be nice.
Here's an easy way to trigger it:
>>> compile("#-*- coding: ascii -*-\nprint 'ab%c'\n"%0x80, '','exec')
Traceback (most recent call last):
File "", line 1, in ?
MemoryError
Regards,
Bengt Richter
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
