Re: [Python-Dev] [C++-sig] GCC version compatibility
On Sun, Aug 07, 2005 at 11:11:56PM +0200, "Martin v. Löwis" wrote: > I've looked at the patch, and it looks fairly safe, so I committed it. Thanks. I did not forget my promise to look into a more comprehensive approach to the C++ build issues. But I first need to better understand the potential impact on distutils. And, foremost, I need to finish my thesis whence my spare time projects go very slowly. Regards Christoph -- http://www.informatik.tu-darmstadt.de/TI/Mitarbeiter/cludwig.html LiDIA: http://www.informatik.tu-darmstadt.de/TI/LiDIA/Welcome.html ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Generalised String Coercion
Bob Ippolito wrote: > It's UTF-8 by default, I highly doubt many people bother to change it. I think your doubts are unfounded. Many Japanese people change it to EUC-JP (I believe), as UTF-8 support doesn't work well for them (or atleast didn't use to). Regards, Martin ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Generalised String Coercion
Guido van Rossum wrote:
> We might be able to get there halfway in Python 2.x: we could
> introduce the bytes type now, and provide separate APIs to read and
> write them. (In fact, the array module and the f.readinto() method
> make this possible today, but it's too klunky so nobody uses it.
> Perhaps a better API would be a new file-open mode ("B"?) to indicate
> that a file's read* operations should return bytes instead of strings.
> The bytes type could just be a very thin wrapper around array('b').
That answers an important question: so you want the bytes type to be
mutable (and, consequently, unsuitable as a dictionary key).
Regards,
Martin
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Generalised String Coercion
Phillip J. Eby wrote:
>>Hm. What would be the use case for using %s with binary, non-text data?
>
>
> Well, I could see using it to write things like netstrings,
> i.e. sock.send("%d:%s," % (len(data),data)) seems like the One Obvious Way
> to write a netstring in today's Python at least. But perhaps there's a
> subtlety I've missed here.
As written, this would stop working when strings become Unicode. It's
pretty clear what '%d' means (format the number in decimal numbers,
using "\N{DIGIT ZERO}" .. "\N{DIGIT NINE}" as the digits). It's not
all that clear what %s means: how do you get a sequence of characters
out of data, when data is a byte string?
Perhaps there could be byte string literals, so that you would write
sock.send(b"%d:%s," % (len(data),data))
but this would raise different questions:
- what does %d mean for a byte string formatting? str(len(data))
returns a character string, how do you get a byte string?
In the specific case of %d, encoding as ASCII would work, though.
- if byte strings are mutable, what about byte string literals?
I.e. if I do
x = b"%d:%s,"
x[1] = b'f'
and run through the code the second time, will the literal have
changed? Perhaps these would be displays, not literals (although
I never understood why Guido calls these displays)
Regards,
Martin
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Generalised String Coercion
> "Martin" == Martin v Löwis <[EMAIL PROTECTED]> writes: Martin> I think your doubts are unfounded. Many Japanese people Martin> change it to EUC-JP (I believe), as UTF-8 support doesn't Martin> work well for them (or atleast didn't use to). If you mean the UTF-8 support in Terminal, it's no better or worse than the EUC-JP support. The problem is that most Japanese Unix systems continue to default to EUC-JP, and many Windows hosts (including Samba file systems) default to Shift JIS. So people using Terminal tend to set it to match the default remote environment (few of them use shells on the Mac). All that is certainly true of my organization, for one example. -- School of Systems and Information Engineering http://turnbull.sk.tsukuba.ac.jp University of TsukubaTennodai 1-1-1 Tsukuba 305-8573 JAPAN Ask not how you can "do" free software business; ask what your business can "do for" free software. ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Generalised String Coercion
Stephen J. Turnbull wrote: > If you mean the UTF-8 support in Terminal, it's no better or worse > than the EUC-JP support. The problem is that most Japanese Unix > systems continue to default to EUC-JP, and many Windows hosts > (including Samba file systems) default to Shift JIS. So people using > Terminal tend to set it to match the default remote environment (few > of them use shells on the Mac). Right: that might be the biggest problem. ls(1) would not display the file names of the remote servers in any readable way. Thanks for the confirmation. Regards, Martin ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] __traceback__ and reference cycles
Hi all, There are various proposals to add an attribute on exception instances to store the traceback (see PEP 344). A detail not discussed, which I thought of historical interest only, is that today's exceptions try very hard to avoid reference cycles, in particular the cycle 'frame -> local variable -> traceback object -> frame' which was important for pre-GC versions of Python. A clause 'except Exception, e' would not create a local reference to the traceback, only to the exception instance. If the latter grows a __traceback__ attribute, it is no longer true, and every such except clause typically creates a cycle. Of course, we don't care, we have a GC -- do we? Well, there are cases where we do: see the attached program... In my opinion it should be considered a bug of today's Python that this program leaks memory very fast and takes longer and longer to run each loop (each loop takes half a second longer than the previous one!). (I don't know how this bug could be fixed, though.) Spoiling the fun of figuring out what is going on, the reason is that 'e_tb' creates a reference cycle involving the frame of __del__, which keeps a reference to 'self' alive. Python thinks 'self' was resurrected. The next time the GC runs, the cycle disappears, and the refcount of 'self' drops to zero again, calling __del__ again -- which gets resurrected again by a new cycle. Etc... Note that no cycle actually contains 'self'; they just point to 'self'. In summary, no X instance gets ever freed, but they all have their destructors called over and over again. Attaching a __traceback__ will only make this "bug" show up more often, as the 'except Exception, e' line in a __del__() method would be enough to trigger it. Not sure what to do about it. I just thought I should share these thoughts (I stumbled over almost this problem in PyPy). A bientot, Armin ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] __traceback__ and reference cycles
Hi, On Mon, Aug 08, 2005 at 10:31:06AM +0200, Armin Rigo wrote: > see the attached program... Oups. Here it is... Armin import sys, time def log(typ, val, tb): pass class X: def __del__(self): try: typo except Exception, e: e_type, e_value, e_tb = sys.exc_info() log(e_type, e_value, e_tb) t = time.time() while True: lst = [X() for i in range(1000)] t1 = time.time() print t1 - t t = t1 ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP: Migrating the Python CVS to Subversion
On Sun, Aug 07, 2005 at 11:51:49PM -0400, Barry Warsaw wrote: > Has anyone experienced svn corruptions with the fsfs backend? I > haven't, across quite a few repositories. I haven't. But I must admit that the repositories I'm working with aren't big. The bigest is at svn.colorstudy.com (I am working on SQLObject) and since the time Ian has switched from dbfs to fsfs I don't remember any problems with the repo. Speaking of merge. SVN relived much pain that CVS had gave me. With CVS I had a lot of conflicts - if the code to be merged is already there (had been merged from another branch) one got conflict. If the code contains CVS keywords (__version__ = "$Id$") cvs merge always produced conflicts. SVN fixed both problems so now I see only real conflicts. SVN just ignores the code to be merged if it has ben already merged; and SVN convert keywords internally to its default form ($Id$ instead of $Id: python.c 42 phd $) before merging. Oleg. -- Oleg Broytmannhttp://phd.pp.ru/[EMAIL PROTECTED] Programmers don't die, they just GOSUB without RETURN. ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Generalised String Coercion
"M.-A. Lemburg" <[EMAIL PROTECTED]> writes:
> Set the external encoding for stdin, stdout, stderr:
>
> (also an example for adding encoding support to an
> existing file object):
>
> def set_sys_std_encoding(encoding):
> # Load encoding support
> (encode, decode, streamreader, streamwriter) = codecs.lookup(encoding)
> # Wrap using stream writers and readers
> sys.stdin = streamreader(sys.stdin)
> sys.stdout = streamwriter(sys.stdout)
> sys.stderr = streamwriter(sys.stderr)
> # Add .encoding attribute for introspection
> sys.stdin.encoding = encoding
> sys.stdout.encoding = encoding
> sys.stderr.encoding = encoding
>
> set_sys_std_encoding('rot-13')
>
> Example session:
print 'hello'
> uryyb
raw_input()
> hello
> h'hello'
1/0
> Genpronpx (zbfg erprag pnyy ynfg):
> Svyr "", yvar 1, va ?
> MrebQvivfvbaReebe: vagrtre qvivfvba be zbqhyb ol mreb
>
> Note that the interactive session bypasses the sys.stdin
> redirection, which is why you can still enter Python
> commands in ASCII - not sure whether there's a reason
> for this, or whether it's just a missing feature.
Um, I'm not quite sure how this would be implemented. Interactive
input comes via PyOS_Readline which deals in FILE*s... this area of
the code always confuses me :(
Cheers,
mwh
--
As it seems to me, in Perl you have to be an expert to correctly make
a nested data structure like, say, a list of hashes of instances. In
Python, you have to be an idiot not to be able to do it, because you
just write it down. -- Peter Norvig, comp.lang.functional
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP: Migrating the Python CVS to Subversion
Barry Warsaw <[EMAIL PROTECTED]> writes: > Unfortunately, I don't think "we" (meaning specifically the collective > python.org admins) have much if any operational experience with > Perforce. Also (from someone who is on the fringes of the pydotorg admin set): I don't know that much about subversion administration. But, if it proves necessary, as it's an open source project and all, I'm willing to put some time into learning about it. I'm *much* less likely to do this for a closed source package unless someone is paying me to do it. Maybe I'm the only person who thinks this way, but if not, it's something to think about. Cheers, mwh -- 42. You can measure a programmer's perspective by noting his attitude on the continuing vitality of FORTRAN. -- Alan Perlis, http://www.cs.yale.edu/homes/perlis-alan/quotes.html ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Generalised String Coercion
Martin v. Löwis wrote:
> Guido van Rossum wrote:
>>The bytes type could just be a very thin wrapper around array('b').
>
> That answers an important question: so you want the bytes type to be
> mutable (and, consequently, unsuitable as a dictionary key).
I would suggest a bytes/frozenbytes pair, similar to set/frozenset and
list/tuple.
Cheers,
Nick.
--
Nick Coghlan | [EMAIL PROTECTED] | Brisbane, Australia
---
http://boredomandlaziness.blogspot.com
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Generalised String Coercion
Guido van Rossum wrote:
> [Guido]
>
>>>My first response to the PEP, however, is that instead of a new
>>>built-in function, I'd rather relax the requirement that str() return
>>>an 8-bit string -- after all, int() is allowed to return a long, so
>>>why couldn't str() be allowed to return a Unicode string?
>
>
> [MAL]
>
>>The problem here is that strings and Unicode are used in different
>>ways, whereas integers and longs are very similar. Strings are used
>>for both arbitrary data and text data, Unicode can only be used
>>for text data.
>
> Yes, that is the case in Python 2.x. In Python 3.x, I'd like to use a
> separate "bytes" array type for non-text and for encoded text data,
> just like Java; strings should always be considered text data.
>
> We might be able to get there halfway in Python 2.x: we could
> introduce the bytes type now, and provide separate APIs to read and
> write them.
>
> (In fact, the array module and the f.readinto() method
> make this possible today, but it's too klunky so nobody uses it.
> Perhaps a better API would be a new file-open mode ("B"?) to indicate
> that a file's read* operations should return bytes instead of strings.
> The bytes type could just be a very thin wrapper around array('b').
I'd prefer to keep such bytes type immutable (arrays are mutable),
otherwise, as Martin already mentioned, they wouldn't be usable
as dictionary keys and the transition from the current string
implementation would be made more difficult than necessary.
Since we won't have any use for the string type in Py3k,
why not simply strip it down to a plain bytes type ?
(I wouldn't want to lose or have to reinvent all the
optimizations that went into its implementation and which
are missing in the array implementation.)
About the file-type idea:
We already have text mode and binary mode - with their implementation
being platform dependent. I don't think that this is particularly
good area to add new functionality.
If you use codecs.open() to open a file, you could easily
write a codec which implements what you have in mind.
>>The new text() built-in would help make a clear distinction
>>between "convert this object to a string of bytes" and
>>"please convert this to a text representation". We need to
>>start making the separation somewhere and I think this is
>>a good non-invasive start.
>
>
> I agree with the latter, but I would prefer that any new APIs we use
> use a 'bytes' data type to represent non-text data, rather than having
> two different sets of APIs to differentiate between the use of 8-bit
> strings as text vs. data -- while we *currently* use 8-bit strings for
> both text and data, in Python 3.0 we won't, so then the interim APIs
> would have to change again. I'd rather intrduce a new data type and
> new APIs that work with it.
Well, let's put it this way: it all really depends on
what str() should mean in Py3k.
Given that str() is used for mixed content data strings,
simply aliasing str() to unicode() in Py3k would cause a
lot of breakage, due to changed semantics.
Aliasing str() to bytes() would also cause breakage, due
to the fact that bytes types wouldn't have string method
like e.g. .lower(), .upper(), etc.
Perhaps str() in Py3k should become a helper that
converts bytes() to Unicode, provided the content is
ASCII-only.
In any case, Py3k would only have unicode() for text
and bytes() for data, so there's no real need to continue
using str().
If we add the text() API in Py2k and with the above
meaning, then we could rename unicode() to text()
in Py3k - only a cosmetical change, but one that I would
find useful: text() and bytes() are more intuitive to
understand than unicode() and bytes().
>>Furthermore, the text() built-in could be used to only
>>allow 8-bit strings with ASCII content to pass through
>>and require that all non-ASCII content be returned as
>>Unicode.
>>
>>We wouldn't be able to enforce this in str().
>>
>>I'm +1 on adding text().
>
>
> I'm still -1.
>
>
>>I would also like to suggest a new formatting marker '%t'
>>to have the same semantics as text() - instead of changing
>>the semantics of %s as the Neil suggests in the PEP. Again,
>>the reason is to make the difference between text and
>>arbitrary data explicit and visible in the code.
>
>
> Hm. What would be the use case for using %s with binary, non-text data?
I guess we'd only keep it for backwards compatibility and
map it to the str() helper.
>>>The main problem for a smooth Unicode transition remains I/O, in my
>>>opinion; I'd like to see a PEP describing a way to attach an encoding
>>>to text files, and a way to decide on a default encoding for stdin,
>>>stdout, stderr.
>>
>>Hmm, not sure why you need PEPs for this:
>
>
> I'd forgotten how far we've come. I'm still unsure how the default
> encoding on stdin/stdout works.
Codecs in general work like this: they take an existing file-like
object and wrap it with new versions of .read(), .write(),
.readline(), etc. which filt
Re: [Python-Dev] Generalised String Coercion
Michael Hudson wrote:
> "M.-A. Lemburg" <[EMAIL PROTECTED]> writes:
>
>
>>Set the external encoding for stdin, stdout, stderr:
>>
>>(also an example for adding encoding support to an
>>existing file object):
>>
>>def set_sys_std_encoding(encoding):
>># Load encoding support
>>(encode, decode, streamreader, streamwriter) = codecs.lookup(encoding)
>># Wrap using stream writers and readers
>>sys.stdin = streamreader(sys.stdin)
>>sys.stdout = streamwriter(sys.stdout)
>>sys.stderr = streamwriter(sys.stderr)
>># Add .encoding attribute for introspection
>>sys.stdin.encoding = encoding
>>sys.stdout.encoding = encoding
>>sys.stderr.encoding = encoding
>>
>>set_sys_std_encoding('rot-13')
>>
>>Example session:
>>
>print 'hello'
>>
>>uryyb
>>
>raw_input()
>>
>>hello
>>h'hello'
>>
>1/0
>>
>>Genpronpx (zbfg erprag pnyy ynfg):
>> Svyr "", yvar 1, va ?
>>MrebQvivfvbaReebe: vagrtre qvivfvba be zbqhyb ol mreb
>>
>>Note that the interactive session bypasses the sys.stdin
>>redirection, which is why you can still enter Python
>>commands in ASCII - not sure whether there's a reason
>>for this, or whether it's just a missing feature.
>
>
> Um, I'm not quite sure how this would be implemented. Interactive
> input comes via PyOS_Readline which deals in FILE*s... this area of
> the code always confuses me :(
Me too.
It appears that this part of the Python code
has undergone so many iterations and patches, that the
structure has suffered a lot, e.g. the main() functions calls
PyRun_AnyFileFlags(stdin, "", &cf),
but the fp argument stdin is then subsequently
ignored if the tok_nextc() function finds that
a prompt is set.
Anyway, hacking along the same lines, I think
the above can be had by changing tok_stdin_decode()
to use a possibly available sys.stdin.decode()
method for the decoding of the data read by
PyOS_Readline(). This would then return Unicode
which tok_stdin_decode() could then encode to
UTF-8 which is the encoding that the tokenizer
can work on.
--
Marc-Andre Lemburg
eGenix.com
Professional Python Services directly from the Source (#1, Aug 08 2005)
>>> Python/Zope Consulting and Support ...http://www.egenix.com/
>>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...http://python.egenix.com/
::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free !
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Generalised String Coercion
At 10:07 AM 8/8/2005 +0200, Martin v. Löwis wrote:
>Phillip J. Eby wrote:
> >>Hm. What would be the use case for using %s with binary, non-text data?
> >
> >
> > Well, I could see using it to write things like netstrings,
> > i.e. sock.send("%d:%s," % (len(data),data)) seems like the One Obvious
> Way
> > to write a netstring in today's Python at least. But perhaps there's a
> > subtlety I've missed here.
>
>As written, this would stop working when strings become Unicode. It's
>pretty clear what '%d' means (format the number in decimal numbers,
>using "\N{DIGIT ZERO}" .. "\N{DIGIT NINE}" as the digits). It's not
>all that clear what %s means: how do you get a sequence of characters
>out of data, when data is a byte string?
>
>Perhaps there could be byte string literals, so that you would write
>
> sock.send(b"%d:%s," % (len(data),data))
Actually, thinking about it some more, it seems to me it's actually more
like this:
sock.send( ("%d:%s," %
(len(data),data.decode('latin1'))).encode('latin1') )
That is, if all we have is unicode and bytes, and 'data' is bytes, then
encoding and decoding from latin1 is the right way to do a netstring. It's
a bit more painful, but still doable.
>but this would raise different questions:
>- what does %d mean for a byte string formatting? str(len(data))
> returns a character string, how do you get a byte string?
> In the specific case of %d, encoding as ASCII would work, though.
>- if byte strings are mutable, what about byte string literals?
> I.e. if I do
>
> x = b"%d:%s,"
> x[1] = b'f'
>
> and run through the code the second time, will the literal have
> changed? Perhaps these would be displays, not literals (although
> I never understood why Guido calls these displays)
I'm thinking that bytes.decode and unicode.encode are the correct way to
convert between the two, and there's no such thing as a bytes literal. We
can always optimize "constant.encode(constant)" to a bytes display
internally if necessary, although it will be a pain for programs that have
lots of bytestring constants. OTOH, we've previously discussed having a
'bytes()' constructor, and perhaps it should use latin1 as its default
encoding.
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Generalised String Coercion
On Sun, Aug 07, 2005, Neil Schemenauer wrote: > On Sat, Aug 06, 2005 at 06:56:39PM -0700, Guido van Rossum wrote: >> >> My first response to the PEP, however, is that instead of a new >> built-in function, I'd rather relax the requirement that str() return >> an 8-bit string > > Do you have any thoughts on what the C API would be? It seems to me > that PyObject_Str cannot start returning a unicode object without a > lot of code breakage. I suppose we could introduce a function > called something like PyObject_String. OTOH, should Guido change his -1 on text(), that leads to the obvious PyObject_Text. -- Aahz ([EMAIL PROTECTED]) <*> http://www.pythoncraft.com/ The way to build large Python applications is to componentize and loosely-couple the hell out of everything. ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] pdb: should next command be extended?
On Sun, Aug 07, 2005, Ilya Sandler wrote: > > Solution: > > Should pdb's next command accept an optional numeric argument? It would > specify how many actual lines of code (not "line events") > should be skipped in the current frame before stopping, At OSCON, Anthony Baxter made the point that pdb is currently one of the more unPythonic modules. If you're feeling a lot of energy about this, rewriting pdb might be more productive. -- Aahz ([EMAIL PROTECTED]) <*> http://www.pythoncraft.com/ The way to build large Python applications is to componentize and loosely-couple the hell out of everything. ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Generalised String Coercion
Ouch. Too much discussion to respond to it all. Please remember that in Jythin and IronPython, str and unicode are already synonyms. That's how Python 3.0 will do it, except unicode will disappear as being redundant. I like the bytes/frozenbytes pair idea. Streams could grow a getpos()/setpos() API pair that can be used for stateful encodings (although it sounds like seek()/tell() would be okay to use in most cases as long as you read in units of whole lines). For sockets, send()/recv() would deal in bytes, and makefile() would get an encoding parameter. I'm not going to change my mind on text() unless someone explains what's so attractive about it. -- --Guido van Rossum (home page: http://www.python.org/~guido/) ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] __traceback__ and reference cycles
On 8/8/05, Armin Rigo <[EMAIL PROTECTED]> wrote: > Attaching a __traceback__ will only make this "bug" show up more often, > as the 'except Exception, e' line in a __del__() method would be enough > to trigger it. Hmm... We can blame this on __del__ as much as on __traceback__, of course... But it is definitely of concern. -- --Guido van Rossum (home page: http://www.python.org/~guido/) ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Generalised String Coercion
At 09:14 AM 8/8/2005 -0700, Guido van Rossum wrote: >I'm not going to change my mind on text() unless >someone explains what's so attractive about it. 1. It's obvious to non-programmers what it's for (str and unicode aren't) 2. It's more obvious to programmers that it's a *text* string rather than a string of bytes 3. It's easier to type than "unicode", but less opaque than "str" 4. Switching to 'text' and 'bytes' allows for a clean break from any mental baggage now associated with 'unicode' and 'str'. Of course, the flip side to these arguments is that in today's Python, one rarely has use for the string type names, except for coercion and some occasional type checking. On the other hand, if we end up with type declarations, then these issues become a bit more important. ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Generalised String Coercion
Guido van Rossum wrote: > Ouch. Too much discussion to respond to it all. Please remember that > in Jythin and IronPython, str and unicode are already synonyms. I know, but don't understand that argument: aren't we talking about Python in general, not some particular implementation ? Why should CPython applications break just to permit Jython and IronPython applications not to break ? > That's how Python 3.0 will do it, except unicode will disappear as being > redundant. I like the bytes/frozenbytes pair idea. Streams could grow > a getpos()/setpos() API pair that can be used for stateful encodings > (although it sounds like seek()/tell() would be okay to use in most > cases as long as you read in units of whole lines). Please don't confuse the raw bytes position in a file or stream with e.g. an index into the possibly decoded data. Those are two different pairs of shoes. Since the position into decoded data depends on what type of encoding your using and how you decode, the "position" would not be defined across streams, but depend on the features of a particular stream. > For sockets, send()/recv() would deal in bytes, and makefile() would get an > encoding parameter. I'm not going to change my mind on text() unless > someone explains what's so attractive about it. Please read my reply for some reasoning and also Phillips answer to your posting. Thanks, -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 08 2005) >>> Python/Zope Consulting and Support ...http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ...http://python.egenix.com/ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP: Migrating the Python CVS to Subversion
On Sun, Aug 07, 2005, Barry Warsaw wrote: > > We'd also have to teach the current crop of developers how to use the > client tools effectively. I think it's fairly simple to teach a CVS > user how to use Subversion, but have no idea if translating CVS > experience to Perforce is as straightforward. The impression I got from Alex Martelli is that it's not particularly straightforward. (Google apparently uses Perforce.) -- Aahz ([EMAIL PROTECTED]) <*> http://www.pythoncraft.com/ The way to build large Python applications is to componentize and loosely-couple the hell out of everything. ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 347: Migration to Subversion
On 8/7/05, "Martin v. Löwis" <[EMAIL PROTECTED]> wrote: > Brett Cannon wrote: > > What is going in under python/ ? If it is what is currently > > /dist/src/, then great and the renaming of the repository works. > > Just have a look yourself :-) Yes, this is dist/src. > Ah, OK. I didn't drill far enough down. Not enough experience with svn to realize that the directory was not just filled with default directories. > > But if that is what src/ is going to be used for > > This is nondist/src. Perhaps I should just move nondist/src/Compiler, > and drop nondist/src. > Wouldn't hurt. Since svn allows directory deletion there doesn't seem to be an huge need to worry about the projects/ directory getting to large. > > And I assume you are going to list the directory structure in the PEP > > at some point. > > Please take a look at the PEP. > OK, now I see it. I scanned the PEP initially but I didn't see it; guess I was expecting a more literal directory list than a paragraph on it. -Brett ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP: Migrating the Python CVS to Subversion
> > Since Python is Open Source are you looking at Per Force which you can > > use for free and seems to be a happy medium between something like CVS > > and something horrific like Clear Case? > > No. The PEP is only about Subversion. Why should we be looking at Per > Force? Only because Python is Open Source? Perforce offers free licensing to open source projects. > I think anything but Subversion is ruled out because: > - there is no offer to host that anywhere (for subversion, there is > already svn.python.org) > - there is no support for converting a CVS repository (for subversion, > there is cvs2svn) There *is* support for converting a CVS repository to Perforce [1]. Perforce is very good, stable, solid, reliable, good tools, etc. etc. but I'd tend to support SVN over Perforce for Python development. Perforce usage is quite different than CVS (would be a painful re-learning for old CVS-hands) and SVN tends to better support highly distributed development: most operations don't need to talk to the server, with Perforce (aka p4), almost *all* operations talk to the server. This can be somewhat mitigated with "p4proxy" (a tool that Perforce also provides) but people would be happier with SVN, I'd bet. [1] It is a project called VCP. Some details here (I didn't look too hard): http://www.cpan.org/modules/by-module/LWP/AUTRIJUS/VCP-autrijus-snapshot-0.9-20041020.readme Trent -- Trent Mick [EMAIL PROTECTED] ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP: Migrating the Python CVS to Subversion
[Aahz wrote] > On Sun, Aug 07, 2005, Barry Warsaw wrote: > > > > We'd also have to teach the current crop of developers how to use the > > client tools effectively. I think it's fairly simple to teach a CVS > > user how to use Subversion, but have no idea if translating CVS > > experience to Perforce is as straightforward. > > The impression I got from Alex Martelli is that it's not particularly > straightforward. Agreed. > (Google apparently uses Perforce.) We do at ActiveState as well. *The* Perl source code repository is a Perforce one (hosted separately here at ActiveState as well). Microsoft licenses the Perforce code and uses it (with some slight modifications I hear) internally. Trent -- Trent Mick [EMAIL PROTECTED] ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] PEP-343 - Context Managment variant
I know I came after the battle. And I have just another sight on context
managment.
Simple Context Managment may look in Python 2.4.1 like this:
Synhronized example:
def Synhronised(lock,func):
lock.acquire()
try:
func()
finally:
lock.release()
lock=Lock()
def Some():
local_var1=x
local_var2=y
local_var3=Z
def Work():
global local_var3
local_var3=Here_I_work(local_var1,local_var2,local_var3)
Synhronised(lock,Work)
return asd(local_var3)
FileOpenClose Example:
def FileOpenClose(name,mode,func):
f=file(name,mode)
try:
func(f)
finally:
f.close()
def Another(name):
local_var1=x
local_var2=y
local_var3=None
def Work(f):
global local_var3
local_var3=[[x,y(i)] for i in f]
FileOpenClose(name,'r',Work)
return local_var3
It was complicated because :
1. We must declare closure (local function) before using it
2. We must declare local vars, to which we wish assign in "global" statement
3. We cannot create variable local to outter function in closure, so we
must create it before
and declare in global
So one can say: "that is because there're no block lambda". (I can be wrong)
I think it is possible to introduce block-object in analogy to lambda-object
(and function-object)
It's difference from function:
it has no true self local variables, all global(free) and local variables it
steels from outter
scope. And all local vars, introduced in block are really introduced in
outter scope
(i think, during parse state). So its global dicts and local dicts are realy
corresponding dicts
of outter scope. (Excuse my english)
So, may be it would be faster than function call. I don't know.
Syntax can be following:
lock=Lock()
def Some():
local_var1=x
local_var2=y
local_var3=Z
Synhronised(lock,block)
local_var3=Here_I_work(local_var1,local_var2,local_var3)
return asd(local_var3)
def Another(name):
local_var1=x
local_var2=y
FileOpenClose(name,'r',block{f})
local_var3=[[x,y(i)] for i in f]
return local_var3
And here is sample of returning value:
def Foo(context,proc):
context.enter()
try:
res=proc(context.x,context.y)*2
except Exception,Err:
context.throw(Err)
finally:
context.exit()
return res
...
context=MyContext()
...
def Bar():
result=Foo(context,block{x,y})
continue x+y
return result
It's idea was token from Ruby. But I think, good idea is good whatever it came
from.
It can be not Pythonic.
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP: Migrating the Python CVS to Subversion
On Mon, Aug 08, 2005, Trent Mick wrote: >Martin: >> >> No. The PEP is only about Subversion. Why should we be looking at Per >> Force? Only because Python is Open Source? > > Perforce offers free licensing to open source projects. So did BitKeeper. Linux got bitten by that. We'd need a strong incentive to consider Perforce over Subversion just because of that issue. -- Aahz ([EMAIL PROTECTED]) <*> http://www.pythoncraft.com/ The way to build large Python applications is to componentize and loosely-couple the hell out of everything. ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP: Migrating the Python CVS to Subversion
Trent Mick wrote: >>No. The PEP is only about Subversion. Why should we be looking at Per >>Force? Only because Python is Open Source? > > > Perforce offers free licensing to open source projects. Ok, so I now got "it's mature", and "it would be without charges". Given that it is now running against Subversion, I would be still interested in advantages that it offers compared to svn. The biggest disadvantage, to me, is that few people know how to use it (myself included). I don't trust tools I've never used. So for me, as the author of this PEP, usage of the revsion control software is non-negotiable (selection of hoster, to a limited degree, is). If you want to see Perforce used for the Python development, you should write a counter-PEP, so we could let the BDFL decide. [This is a theoretical "you" here, since you then explain that you would still prefer to use subversion] Regards, Martin ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Generalised String Coercion
Phillip J. Eby wrote:
> Actually, thinking about it some more, it seems to me it's actually more
> like this:
>
>sock.send( ("%d:%s," %
> (len(data),data.decode('latin1'))).encode('latin1') )
While this would work, it would still feel wrong: the binary data
are *not* latin1 (most likely), so declaring them to be latin1 would
be confusing. Perhaps a synonym '8bit' for latin1 could be introduced.
Regards,
Martin
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP: Migrating the Python CVS to Subversion
One feature I like in Perforce (which Subversion doesn't have) is the ability to have pending changesets. A changeset is, as with subversion, something you check-in atomically. Pending changesets in Perforce allow you to (1) group related files in a source tree where you might be working on multiple things at once to ensure and (2) to build a change description as you go. In a large source tree this can be useful for separating chunks of work. There are other little things, like not being able to trim the check-in filelist when editing the check-in message. For example, say you have 10 files checked out scattered around the Python source tree and you want to check 9 of those in. Currently with svn you have to manually specify those 9 to be sure to not include the remaining one. With p4 you just say to check-in the whole tree and then remove that one from the list give you in your editor with entering the check-in message. Not that big of a deal. [Martin v. L?wis on Perforce] > The biggest disadvantage, to me, is that few people know how > to use it (myself included). Granted. For that reason and for a couple of others I mentioned (SVN will probably work better for offline and distributed developers) I think Subversion wins over Perforce. That is presuming, of course, that we find Subversion to be acceptibly stable/robust/manageble. Trent -- Trent Mick [EMAIL PROTECTED] ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Generalised String Coercion
On Sat, Aug 06, 2005 at 06:56:39PM -0700, Guido van Rossum wrote:
> My first response to the PEP, however, is that instead of a new
> built-in function, I'd rather relax the requirement that str() return
> an 8-bit string -- after all, int() is allowed to return a long, so
> why couldn't str() be allowed to return a Unicode string?
I've played with this idea a bit and it seems viable. I modified my
original patch to have string_new call PyObject_Text instead of
PyObject_Str. That change breaks only two tests, both in
test_email. The tracebacks are attached. Both problems seem
relatively shallow. Do you thing such a change could go into 2.5?
Neil
Traceback (most recent call last):
File "/home/nas/Python/py_cvs/Lib/email/test/test_email.py", line 2844, in
test_encoded_adjacent_nonencoded
h = make_header(decode_header(s))
File "/home/nas/Python/py_cvs/Lib/email/Header.py", line 123, in make_header
charset = Charset(charset)
File "/home/nas/Python/py_cvs/Lib/email/Charset.py", line 190, in __init__
input_charset = unicode(input_charset, 'ascii').lower()
TypeError: decoding Unicode is not supported
Traceback (most recent call last):
File "/home/nas/Python/py_cvs/Lib/email/test/test_email.py", line 2750, in
test_multilingual
eq(decode_header(enc),
File "/home/nas/Python/py_cvs/Lib/email/Header.py", line 85, in decode_header
dec = email.quopriMIME.header_decode(encoded)
File "/home/nas/Python/py_cvs/Lib/email/quopriMIME.py", line 319, in
header_decode
return re.sub(r'=\w{2}', _unquote_match, s)
File "/home/nas/Python/py_cvs/Lib/sre.py", line 142, in sub
return _compile(pattern, 0).sub(repl, string, count)
UnicodeDecodeError: 'ascii' codec can't decode byte 0xfc in position 0: ordinal
not in range(128)
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP: Migrating the Python CVS to Subversion
[Trent Mick] > ... > There are other little things, like not being able to trim the check-in > filelist when editing the check-in message. For example, say you have > 10 files checked out scattered around the Python source tree and you > want to check 9 of those in. This seems dubious, since you're not checking in the state you actually have locally, and you were careful to run the full Python test suite with your full local state ;-) > Currently with svn you have to manually specify those 9 to be sure to not > include the remaining one. With p4 you just say to check-in the whole tree > and then remove that one from the list give you in your editor with entering > the check-in message. Not that big of a deal. As a purely theoretical exercise , the last time I faced that under SVN, I opened the single file I didn't want to check-in in my editor, did "svn revert" on it from the cmdline, checked in the whole tree, and then hit the editor's "save" button. This doesn't scale well to skipping 25 of 50, but it's effective enough for 1 or 2. ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Generalised String Coercion
[Phillip J. Eby] > At 09:14 AM 8/8/2005 -0700, Guido van Rossum wrote: > > I'm not going to change my mind on text() unless someone explains > > what's so attractive about it. > 2. It's more obvious to programmers that it's a *text* string rather > than a string of bytes I've no opinion on the proposal on itself, except maybe that "text", that precise word or name, is a pretty bad choice. It is far too likely that people already use or want to use that precise identifier. There once was a suggestion for naming "text" the module now known as "textwrap", under the premise that it could be later extended for holding many other various text-related functions. Happily enough, this idea was not retained. "textwrap" is much more reasonable as a name. I found Python 1.5.2's "string" to be especially prone to clashing. I still find "socket" obtrusive in that respect. Consider "len" as an example of a clever choice, while "length" would not have been. "str" is also a good choice. "object" is a bit more annoying theoretically, yet we almost never need it in practice. "type" is annoying as a name (yet very nice as a concept), as if it was free to use, it would often serve to label our own things. The fact is we often need the built-in. Python should not choose common English words for its built-ins, without very careful thought, and be reluctant to any compulsion in this area. -- François Pinard http://pinard.progiciels-bpi.ca ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP: Migrating the Python CVS to Subversion
On Mon, 2005-08-08 at 15:49, Trent Mick wrote: > One feature I like in Perforce (which Subversion doesn't have) is the > ability to have pending changesets. A changeset is, as with subversion, > something you check-in atomically. Pending changesets in Perforce allow > you to (1) group related files in a source tree where you might be > working on multiple things at once to ensure and (2) to build a change > description as you go. In a large source tree this can be useful for > separating chunks of work. This seems like a poor workaround for crappy branch/merge support. I'm new to perforce, but the pending changesets seem dodgey to me... you are accumulating changes gradually without recording any history during the process... ie, no checkins until the end. Even worse, perforce seems to treat clients like "unversioned branches", allowing you to review and test pending changesets in other clients. This supposedly allows people to review/test each others changes before they are committed. The problem is, since these changes are not committed, there is no firm history of what what was reviewed/tested vs what gets committed... ie they could be different. Having multiple different pending changesets in one large source tree also feels like a workaround for high client overheads. Trying to develop and test a mixture of different changes in one source tree is asking for trouble... they can interact. Maybe I just haven't grokked perforce yet... which might be considered a black mark against it's learning curve :-) For me, the logical way to group a collection of changes is in a branch. This allows you to commit and track history of the collection of changes. You check out each branch into different directories and develop/test them independantly. The branch can then be reviewed and merged when it is complete. -- Donovan Baarda <[EMAIL PROTECTED]> ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP: Migrating the Python CVS to Subversion
[Tim Peters wrote] > [Trent Mick] > > ... > > There are other little things, like not being able to trim the check-in > > filelist when editing the check-in message. For example, say you have > > 10 files checked out scattered around the Python source tree and you > > want to check 9 of those in. > > This seems dubious, since you're not checking in the state you > actually have locally, Say that 10th file is a documentation fix for a module unrelated to the other 9 files. > and you were careful to run the full Python > test suite with your full local state ;-) Absolutely. Er. Always. :) Trent -- Trent Mick [EMAIL PROTECTED] ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP: Migrating the Python CVS to Subversion
Who made me the Perforce-bitch? Here I am screaming "Subversion! Subversion!" and y'all think I just using that as cover for a p4 lover affair. :) [Donovan Baarda wrote] > On Mon, 2005-08-08 at 15:49, Trent Mick wrote: > > One feature I like in Perforce (which Subversion doesn't have) is the > > ability to have pending changesets. A changeset is, as with subversion, > > something you check-in atomically. Pending changesets in Perforce allow > > you to (1) group related files in a source tree where you might be > > working on multiple things at once to ensure and (2) to build a change > > description as you go. In a large source tree this can be useful for > > separating chunks of work. > > This seems like a poor workaround for crappy branch/merge support. More like a pretty nice independent self-organizing feature that was necessitated as a workaround for a crappy solution (clientspecs) for huge data trees. > I'm new to perforce, but the pending changesets seem dodgey to me... you > are accumulating changes gradually without recording any history during > the process... ie, no checkins until the end. You want to do checkins of code in a consisten state. Some large changes take a couple of days to write. During which one may have to do a couple minor things in unrelated sections of a project. Having some mechanism to capture some thoughts and be able to say "these are the relevant source files for this work" is handy. Creating a branch for something that takes a couple of days is overkill. Perforce branching is pretty good in my experience. For very long projects one can easily create a branch. > Even worse, perforce seems to treat clients like "unversioned branches", > allowing you to review and test pending changesets in other clients. I'm not sure what you are talking about here. Yes, client information is stored on the server, but the *changes* (i.e. the diffs) on the client aren't so you must be talking about some other tool. Actually, if there *were* such a feature that would be quite handy. I'd love to be able to easily transfer my diffs developed on my Windows box to my Linux or Mac OS X box to quickly test changes there before checking in. > This supposedly allows people to review/test each others changes before > they are committed. The problem is, since these changes are not > committed, there is no firm history of what what was reviewed/tested vs > what gets committed... ie they could be different. The alternative being either that you have separate branches for everything (can be a pain) or just check-in for review (possibly breaking the build or functionality for other developers until the review is done). Actually the Perl guys working on PureMessage downstairs have two branches going in Perforce: one for checking into right away and then a cleaner tree to which only reviewed check-ins from the first are integrated. I'm not saying I am awash in pending changelists here. Nor that they should be used for what is better handled with branching. It is a tool (and a minor one). > Trying to develop and test a mixture of different changes in one > source tree is asking for trouble... they can interact. ...within reason. Trent -- Trent Mick [EMAIL PROTECTED] ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Generalised String Coercion
On Aug 8, 2005, at 12:14 PM, Guido van Rossum wrote: > Ouch. Too much discussion to respond to it all. Please remember that > in Jythin and IronPython, str and unicode are already synonyms. That's > how Python 3.0 will do it, except unicode will disappear as being > redundant. I like the bytes/frozenbytes pair idea. Streams could grow > a getpos()/setpos() API pair that can be used for stateful encodings > (although it sounds like seek()/tell() would be okay to use in most > cases as long as you read in units of whole lines). For sockets, > send()/recv() would deal in bytes, and makefile() would get an > encoding parameter. I'm not going to change my mind on text() unless > someone explains what's so attractive about it. Files no more have an encoding than sockets do. Reading/writing them should ideally (by default) result in bytes. codecs.open and codecs.StreamReaderWriter provide the character-converting wrapper around file-like objects. I agree that getpos/setpos may be a useful addition to the API, but only because it would allow StreamReaderWriter to override it to do something useful. For normal files it could simply be an alias for tell/seek. Of course, someone would have to actually implement the ability to save and restore state for every codec... Hum, actually, it somewhat makes sense for the "open" builtin to become what is now "codecs.open", for convenience's sake, although it does blur the distinction between a byte stream and a character stream somewhat. If that happens, I suppose it does actually make sense to give "makefile" the same signature. James ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] __traceback__ and reference cycles
[Armin Rigo] > There are various proposals to add an attribute on exception instances > to store the traceback (see PEP 344). A detail not discussed, which I > thought of historical interest only, is that today's exceptions try very > hard to avoid reference cycles, in particular the cycle > > 'frame -> local variable -> traceback object -> frame' > > which was important for pre-GC versions of Python. A clause 'except > Exception, e' would not create a local reference to the traceback, only > to the exception instance. If the latter grows a __traceback__ > attribute, it is no longer true, and every such except clause typically > creates a cycle. > > Of course, we don't care, we have a GC -- do we? Well, there are cases > where we do: see the attached program... In my opinion it should be > considered a bug of today's Python that this program leaks memory very > fast and takes longer and longer to run each loop (each loop takes half > a second longer than the previous one!). (I don't know how this bug > could be fixed, though.) > > Spoiling the fun of figuring out what is going on, the reason is that > 'e_tb' creates a reference cycle involving the frame of __del__, which > keeps a reference to 'self' alive. Python thinks 'self' was > resurrected. The next time the GC runs, the cycle disappears, and the > refcount of 'self' drops to zero again, calling __del__ again -- which > gets resurrected again by a new cycle. Etc... Note that no cycle > actually contains 'self'; they just point to 'self'. In summary, no X > instance gets ever freed, but they all have their destructors called > over and over again. > > Attaching a __traceback__ will only make this "bug" show up more often, > as the 'except Exception, e' line in a __del__() method would be enough > to trigger it. > > Not sure what to do about it. I just thought I should share these > thoughts (I stumbled over almost this problem in PyPy). I can't think of a Python feature with a higher aggregate braincell_burned / benefit ratio than __del__ methods. If P3K retains them-- or maybe even before --we should consider taking "the Java dodge" on this one. That is, decree that henceforth a __del__ method will get invoked by magic at most once on any given object O, no matter how often O is resurrected. It's been mentioned before, but it's at least theoretically backward-incompatible, so "it's scary". I can guarantee I don't have any code that would care, including all the ZODB code I watch over these days. For ZODB it's especially easy to be sure of this: the only __del__ method in the whole thing appears in the test suite, verifying that ZODB's object cache no longer gets into an infinite loop when a user-defined persistent object has a brain-dead __del__ method that reloads self from the database. (Interestingly enough, if Python guaranteed to call __del__ at most once, the infinite loop in ZODB's object cache never would have appeared in this case.) ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP: Migrating the Python CVS to Subversion
On Mon, 2005-08-08 at 17:51, Trent Mick wrote: [...] > [Donovan Baarda wrote] > > On Mon, 2005-08-08 at 15:49, Trent Mick wrote: [...] > You want to do checkins of code in a consisten state. Some large changes > take a couple of days to write. During which one may have to do a couple > minor things in unrelated sections of a project. Having some mechanism > to capture some thoughts and be able to say "these are the relevant I don't need to checkin in a consitent state if I'm working on a seperate branch. I can checkin any time I want to record a development checkpoint... I can capture the thoughts in the version history of the branch. > source files for this work" is handy. Creating a branch for something > that takes a couple of days is overkill. [...] > The alternative being either that you have separate branches for > everything (can be a pain) or just check-in for review (possibly It all comes down to how painless branch/merge is. Many esoteric "features" of version control systems feel like they are there to workaround the absence of proper branch/merge histories. Note: SVN doesn't have branch/merge histories either. -- Donovan Baarda <[EMAIL PROTECTED]> ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] __traceback__ and reference cycles
On 8/8/05, Tim Peters <[EMAIL PROTECTED]> wrote: > I can't think of a Python feature with a higher aggregate > braincell_burned / benefit ratio than __del__ methods. If P3K retains > them-- or maybe even before --we should consider taking "the Java > dodge" on this one. That is, decree that henceforth a __del__ method > will get invoked by magic at most once on any given object O, no > matter how often O is resurrected. I'm sympathetic to this one. Care to write a PEP? It could be really short and sweet as long as it provides enough information to implement the feature. -- --Guido van Rossum (home page: http://www.python.org/~guido/) ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] an alternative suggestion, Re: pdb: should next command be extended?
> > Should pdb's next command accept an optional numeric argument? It would > > specify how many actual lines of code (not "line events") > > should be skipped in the current frame before stopping, > That would differ from gdb's "next ", which does "next" n times. > It would be confusing if pdb accepted the same command, but it > meant something different. So, would implementing gdb's "until" command instead of "next N" be a better idea? In its simplest form (without arguments) "until" advances to the next (textually) source line... This would solve the original problem of getting over list comprehensions... There is a bit of a problem with abbreviation of "until": gdb abbreviates it as "u", while in pdb "u" means "up"...It might be a good idea to have the same abbreviations Ilya Problem: When the code contains list comprehensions (or for that matter any other looping construct), the only way to get quickly through this code in pdb is to set a temporary breakpoint on the line after the loop, which is inconvenient.. There is a SF bug report #1248119 about this behavior. On Sun, 7 Aug 2005, Ilya Sandler wrote: > > > On Sun, 7 Aug 2005, [ISO-8859-1] "Martin v. L?wis" wrote: > > > Ilya Sandler wrote: > > > Should pdb's next command accept an optional numeric argument? It would > > > specify how many actual lines of code (not "line events") > > > should be skipped in the current frame before stopping, > > [...] > > > What do you think? > > > > That would differ from gdb's "next ", which does "next" n times. > > It would be confusing if pdb accepted the same command, but it > > meant something different. > > But as far as I can tell, pdb's next is > already different from gdb's next! gdb's next seem to always go to the > different source line, while pdb's next may stay on the current line. > > The problem with "next " meaning "repeat next n times" is that it > seems to be less useful that the original suggestion. > > Any alternative suggestions to allow to step over list comprehensions and > such? (SF 1248119) > > > Plus, there is always a chance that > > +n is never reached, which would also be confusing. > > That should not be a problem, returning from the current frame should be > treated as a stopping condition (similarly to the current "next" > behaviour)... > > Ilya > > > > > So I'm -1 here. > > > > Regards, > > Martin > > > ___ > Python-Dev mailing list > [email protected] > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > http://mail.python.org/mailman/options/python-dev/ilya%40bluefir.net > ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] __traceback__ and reference cycles
At 09:12 PM 8/8/2005 -0400, Tim Peters wrote: >I can't think of a Python feature with a higher aggregate >braincell_burned / benefit ratio than __del__ methods. If P3K retains >them-- or maybe even before --we should consider taking "the Java >dodge" on this one. That is, decree that henceforth a __del__ method >will get invoked by magic at most once on any given object O, no >matter how often O is resurrected. How does that help? Doesn't it mean that we'll have to have some way of keeping track of which items' __del__ methods were called? ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] __traceback__ and reference cycles
On 8/8/05, Tim Peters <[EMAIL PROTECTED]> wrote: > I can't think of a Python feature with a higher aggregate > braincell_burned / benefit ratio than __del__ methods. If P3K retains > them-- or maybe even before --we should consider taking "the Java > dodge" on this one. That is, decree that henceforth a __del__ method > will get invoked by magic at most once on any given object O, no > matter how often O is resurrected. > Wasn't there talk of getting rid of __del__ a little while ago and instead use weakrefs to functions to handle cleaning up? Is that still feasible? And if so, would this alleviate the problem? -Brett ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] __traceback__ and reference cycles
[Tim Peters] >> If P3K retains them [__del__]-- or maybe even before --we should >> consider taking "the Java dodge" on this one. That is, decree that >> henceforth a __del__ method will get invoked by magic at most >> once on any given object O, no matter how often O is resurrected. [Phillip J. Eby] > How does that help? You have to dig into Armin's example (or read his explanation): every time __del__ is called on one of his X objects, it creates a cycle by binding sys.exec_info()[2] to the local vrbl `e_tb`. `self` is reachable from that cycle, so self's refcount does not fall to 0 when __del__ returns. The object is resurrected. When cyclic gc next runs, it determines that the cycle is trash, and runs around decref'ing the objects in the cycle. That eventually makes the refcount on the X object fall to 0 again too, but then its __del__ method also runs again, and creates an isomorphic cycle, resurrecting `self` again. Etc. Armin didn't point this out explicitly, but it's important to realize that gc.garbage remains empty the entire time you let his program run. The object with the __del__ method isn't _in_ a cycle here, it's hanging _off_ a cycle, which __del__ keeps recreating. Cyclic gc isn't inhibited by a __del__ on an object hanging off a trash cycle (but not in a trash cycle itself), but in this case it's ineffective anyway. If __del__ were invoked only the first time cyclic gc ran, the original cycle would go away during the next cyclic gc run, and a new cycle would not take its place. > Doesn't it mean that we'll have to have some way of keeping track of > which items' __del__ methods were called? Yes, by hook or by crook; and yup too, that may be unattractive. ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] __traceback__ and reference cycles
[Brett Cannon] > Wasn't there talk of getting rid of __del__ a little while ago and > instead use weakrefs to functions to handle cleaning up? There was from me, yes, with an eye toward P3K. > Is that still feasible? It never was, really. The combination of finalizers, cycles and resurrection is a freakin' mess, "even in theory". The way things are right now, Python's weakref gc endcase behavior is even more mystically implementation-driven than its __del__ gc endcase behavior, and nobody has had time to try to dream up a cleaner approach. > And if so, would this alleviate the problem? Absolutely . The underlying reason for optimism is that weakrefs in Python are designed to, at worst, let *other* objects learn that a given object has died, via a callback function. The weakly referenced object itself is not passed to the callback, and the presumption is that the weakly referenced object is unreachable trash at the time the callback is invoked. IOW, resurrection was "obviously" impossible, making endcase life very much simpler. That paragraph is from Modules/gc_weakref.txt, and you can read there all about why optimism hasn't work yet ;-) ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] pdb: should next command be extended?
> At OSCON, Anthony Baxter made the point that pdb is currently one of the > more unPythonic modules. What is unpythonic about pdb? Is this part of Anthony's presentation online? (Google found a summary and slides from presentation but they don't say anything about pdb's deficiencies) Ilya On Mon, 8 Aug 2005, Aahz wrote: > On Sun, Aug 07, 2005, Ilya Sandler wrote: > > > > Solution: > > > > Should pdb's next command accept an optional numeric argument? It would > > specify how many actual lines of code (not "line events") > > should be skipped in the current frame before stopping, > > At OSCON, Anthony Baxter made the point that pdb is currently one of the > more unPythonic modules. If you're feeling a lot of energy about this, > rewriting pdb might be more productive. > -- > Aahz ([EMAIL PROTECTED]) <*> http://www.pythoncraft.com/ > > The way to build large Python applications is to componentize and > loosely-couple the hell out of everything. > ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] Exception Reorg PEP revised yet again
version 1.7 scales the proposal back once more (http://www.python.org/peps/pep-0348.html). At this point the only changes to the hierarchy are the addition of BaseException and TerminatingException, and the change of inheritnace for KeyboardInterrupt, SystemExit, and NotImplementedError. At this point I don't think MAL or Raymond will have any major complaints. =) Assuming no one throws a fit over this version, discussing transition is the next step. I think the transition plan is fine, but if anyone has any specific input that would be great. I could probably stand to do a more specific timeline in terms of 2.x, 2.x+1, 3.0-1, etc., but that will have to wait for another day this week. And once that is settled I guess it is either time for pronouncement or it just sits there until Python 3.0 actually starts to come upon us. -Brett ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP: Migrating the Python CVS to Subversion
> "Donovan" == Donovan Baarda <[EMAIL PROTECTED]> writes: Donovan> It all comes down to how painless branch/merge is. Many Donovan> esoteric "features" of version control systems feel like Donovan> they are there to workaround the absence of proper Donovan> branch/merge histories. It's not that simple. I've followed both the Arch and the darcs lists---they handle a lot more branch/merge scenarios than Subversion does, but you still can't get away with zero discipline. On the other hand, for the purpose of the main repository for a well-factored project with disciplined workflow like Python, it's not obvious to me that the middle-complexity scenarios are that important. Furthermore, taking good advantage of the more complex branch/merge scenarios will require a change to Python workflow (for example, push- to-tracker will no longer be a convenient way to submit patches for most developers); that will be costly. More important, since none of the core Python people have spoken up strongly in favor of an advanced system, I would guess there's little experience to support planning a new workflow. (Cf. the Linux case, where Linus opted to roll his own.) I know many people in the Emacs communities who are successfully using CVS for the main repositories and various advanced systems (prcs, darcs, arch at least) for local branching and small group project communication. It seems fairly easy to automate that (much easier than extracting changeset information from CVS!) I think that as developers find they have need for such capabilities, the practice will grow in Python too, and then there may be a case to be built for moving the main repository to such a system. -- School of Systems and Information Engineering http://turnbull.sk.tsukuba.ac.jp University of TsukubaTennodai 1-1-1 Tsukuba 305-8573 JAPAN Ask not how you can "do" free software business; ask what your business can "do for" free software. ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Generalised String Coercion
> "Martin" == Martin v Löwis <[EMAIL PROTECTED]> writes: Martin> While this would work, it would still feel wrong: the Martin> binary data are *not* latin1 (most likely), so declaring Martin> them to be latin1 would be confusing. Perhaps a synonym Martin> '8bit' for latin1 could be introduced. Be careful. This alias has caused Emacs some amount of pain, as binary data escapes into contexts (such as Universal Newline processing) where it gets interpreted as character data. We've also had some problems in codec implementation, because latin1 and (eg) latin9 have some differences in semantics other than changing the coded character set for the GR register---controls are treated differently, for example, because they _are_ binary (alias latin1) octets, but not in the range of the latin9 code. I won't go so far as to say it won't work, but it will require careful design. -- School of Systems and Information Engineering http://turnbull.sk.tsukuba.ac.jp University of TsukubaTennodai 1-1-1 Tsukuba 305-8573 JAPAN Ask not how you can "do" free software business; ask what your business can "do for" free software. ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Exception Reorg PEP revised yet again
[Brett Cannon] > At this point the only > changes to the hierarchy are the addition of BaseException and > TerminatingException, and the change of inheritnace for > KeyboardInterrupt, SystemExit, and NotImplementedError. TerminatingException The rationale for adding TerminatingException needs to be developed or reconsidered. AFAICT, there hasn't been an exploration of existing code bases to determine that there is going to be even minimal use of "except TerminatingException". Are KeyboardInterrupt and SystemExit often caught together on the same line and handled in the same way? If so, isn't "except TerminatingException" less explicit, clear, and flexible than "except (KeyboardInterrupt, SystemExit)"? Do we need a second way to do it? Doesn't the new meaning of Exception already offer a better idiom: try: suite() except Exception: log_or_recover() except: handle_terminating_exceptions() else: Are there any benefits sufficient to warrant yet another new built-in? Does it also warrant violating FIBTN by introducing more structure? While I'm clear on why KeyboardInterrupt and SystemExit were moved from under Exception, it is not at all clear what problem is being solved by adding a new intermediate grouping. The PEP needs to address all of the above. Right now, it contains a definition rather than justification, research, and analysis. WindowsError This should be kept. Unlike module specific exceptions, this exception occurs in multiple places and diverse applications. It is appropriate to list as a builtin. "Too O/S specific" is not a reason for eliminating this. Looking at the codebase there does not appear to be a good substitute. Eliminating this one would break code, decrease clarity, and cause modules to grow competing variants. After the change, nothing would be better and many things would be worse. NotImplementedError --- Moving this is fine. Removing unnecessary nesting is a step forward. The PEP should list that as a justification. Bare excepts defaulting to Exception After further thought, I'm not as sure about this one and whether it is workable. The code fragment above highlights the issue. In a series of except clauses, each line only matches what was not caught by a previous clause. This is a useful and basic part of the syntax. It leaves a bare except to have the role of a final catchall (much like a default in C's switch-case). If one line uses "except Exception", then a subsequence bare except should probably catch KeyboardInterrupt and SystemExit. Otherwise, there is a risk of creating optical illusion errors (code that looks like it should work but is actually broken). I'm not certain on this one, but the PEP does need to fully explore the implications and think-out the consequent usability issues. > And once that is settled I guess it is either time for pronouncement > or it just sits there until Python 3.0 actually starts to come upon > us. What happened to "don't take this too seriously, I'm just trying to get the ball rolling"? This PEP or any Py3.0 PEP needs to sit a good while before pronouncement. Because 3.0 is not an active project, the PEP is unlikely to be a high priority review item by many of Python's best minds. It should not be stamped as accepted until they've had a chance to think it through. Because 3.0 is still somewhat ethereal, it is not reasonable to expect them to push aside their other work to look at this right now. The PEP needs to be kicked around on the newsgroup (naming and grouping discussions are easy and everyone will have an opinion). Also the folks with PyPy, BitTorrent, Zope, Twisted, IronPython, Jython, and such need to have a chance to have their say. Because of Py3.0's low visibility, these PEPs could easily slide through prematurely. Were the project imminent, it is likely that this PEP would have had significantly more discussion. Try not to get frustrated at these reviews. Because there was no research into existing code, working to solve known problems, evaluation of alternatives, or usability analysis, it is no surprise Sturgeon's Law would apply. Since Python has been around so long, it is also no surprise that what we have now is pretty good and that improvements won't be trivially easy to come by. Raymond ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Major revision of PEP 348 committed
Raymond Hettinger wrote: > If the PEP can't resist the urge to create new intermediate groupings, > then start by grepping through tons of Python code to find-out which > exceptions are typically caught on the same line. That would be a > worthwhile empirical study and may lead to useful insights. I was curious, so I did a little grepping (ok, os.walking and re.findalling) ;-) through the Python source. The only exceptions that were caught together more than 5 times were: AttributeError and TypeError (23 instances) in code.py doctest.py linecache.py mailbox.py idlelib/rpc.py lib-old/newdir.py lib-tk/Tkinter.py test/test_descr.py test/test_file.py test/test_funcattrs.py test/test_os.py Though these occur in a few different contexts, one relatively common one was when the code tried to set a possibly read-only attribute. ImportError and AttributeError (9 instances), in getpass.py locale.py pydoc.py tarfile.py xmlrpclib.py lib-tk/tkFileDialog.py test/test_largefile.py test/test_tarfile.py This seemed to be used when an incompatible module might be present. (Attributes were tested to make sure the module was the right one.) Also used when code tried to use "private" module attributes (e.g. _getdefaultlocale()). OverflowError and ValueError (9 instances), in csv.py ftplib.py mhlib.py mimify.py warnings.py test/test_resource.py These were generally around a call to int(x). I assume they're generally unnecessary now that int() silently converts to longs. IOError and OSError (6 instances), in pty.py tempfile.py whichdb.py distutils/dir_util.py idlelib/configHandler.py test/test_complex.py These were all around file/directory handling that I didn't study in too much detail. With the current hierarchy, there's no reason these couldn't just be catching EnvironmentError anyway. As you can see, even for the most common pairs of exceptions, the total number of times these pairs were caught was pretty small. Even ignoring the low counts, we see that the last two pairs or exceptions aren't really necessary, thanks to int/long unification and the existence of EnvironmentError, and the former two pairs argue *against* added nesting as it is unclear whether to group AttributeError with ImportError or TypeError. So it doesn't really look like the stdlib's going to provide much of a case for adding nesting to the exception hierarchy. Anyway, I know PEP 348's been scaled back at this point anyway, but I figured I might as well post my findings in case anyone was curious. STeVe -- You can wordify anything if you just verb it. --- Bucky Katt, Get Fuzzy ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP: Migrating the Python CVS to Subversion
Trent Mick wrote: > One feature I like in Perforce (which Subversion doesn't have) is the > ability to have pending changesets. That sounds useful. > Currently with svn you have to manually > specify those 9 to be sure to not include the remaining one. With p4 you > just say to check-in the whole tree and then remove that one from the > list give you in your editor with entering the check-in message. Not > that big of a deal. Depends on the client also. With Tortoise SVN, you do get a checkbox list where you can exclude files from the checkin. Regards, Martin ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] an alternative suggestion, Re: pdb: should next command be extended?
Ilya Sandler wrote: > So, would implementing gdb's "until" command instead of "next N" be a > better idea? In its simplest form (without arguments) "until" advances to > the next (textually) source line... This would solve the original problem of > getting over list comprehensions... I like that idea. > There is a bit of a problem with abbreviation of "until": gdb abbreviates > it as "u", while in pdb "u" means "up"...It might be a good idea to have the > same abbreviations Indeed. I don't know much about pdb internals, but I think "u" should become unbound, and "up" and "unt" should become the shortest abbreviations. Regards, Martin ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
