Re: [Python-Dev] Proposal for new 2to23 tool
On 12 Nov 2007, 03:24:34, Jan Claeys wrote: > > Op zondag 11-11-2007 om 17:19 uur [tijdzone -0800], schreef Brett > Cannon: > > On Nov 11, 2007 4:00 PM, Graham Horler <[EMAIL PROTECTED]> wrote: > > > I have been developing in Python since 1.5, and now have to support 2.1 > > > as a minimum version. I do like to keep my code runnable on newer > > > versions however, and am considering the feasability of forward > > > compatibility with Python 3.0. > > > > > > I also notice the Leo[1] project could use some assistance with forward > > > compatibility. > > > > > > So I was wondering if anyone else had a need for a 2to23.py tool to help > > > make code compatible with 3.0 but not break it for 2.x. > > > > What exactly are you proposing? We already have 2to3 > > (http://svn.python.org/view/sandbox/trunk/2to3/) for source-to-source > > translation from 2.x to 3.0. > > Graham wants to convert his code such that it works on both Python 2.x > (probably even early versions of it?) & Python 3.x. Not 2 instances of > code, but one source that works on both 2.x and 3.x... Absolutely ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] XML codec?
Martin v. Löwis wrote: >> I don't know. Is an XML document ill-formed if it doesn't contain an >> XML declaration, is not in UTF-8 or UTF-8, but there's external >> encoding info? > > If there is external encoding info, matching the actual encoding, > it would be well-formed. Of course, preserving that information would > be up to the application. OK. When the application passes an encoding to the decoder this is supposed to be the external encoding info, so for the decoder it makes sense to assume that the encoding passed to the encoder is the external encoding info and will be transmitted along with the encoded bytes. >> This looks good. Now we would have to extent the code to detect and >> replace the encoding in the XML declaration too. > > I'm still opposed to making this a codec. Right - for a pure Python > solution, the processing of the XML declaration would still need to > be implemented. > >>> I think there could be a much simpler routine to have the same >>> effect. - if it's less than 4 bytes, answer "need more data". >> Can there be an XML document that is less then 4 bytes? I guess not. > > No, the smallest document has exactly 4 characters (e.g. ""). > However, external entities may be smaller, such as "x". > >> But anyway: would a Python implementation of these two functions >> (detect_encoding()/fix_encoding()) be accepted? > > I could agree to a Python implementation of this algorithm as long > as it's not packaged as a codec. I still can't understand your objection to a codec. What's the difference between UTF-16 decoding and XML decoding? In fact PEP 263 IMHO does specify how to decode Python source, so in theory it could be a codec (in practice this probably wouldn't work because of bootstrapping problems). Servus, Walter ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] XML codec?
Martin v. Löwis wrote: >> > In case it isn't clear - this is exactly my view also. >> >> But is there an API to do it? As MAL points out that API would have >> to return not an encoding, but a pair of an encoding and the rewound >> stream. > > The API wouldn't operate on streams. Instead, you pass a string, and > it either returns the detected encoding, or an information telling that > it needs more data. No streams. But in many cases you read the data out of a stream and pass it to an incremental XML parser. So if you're transcoding the input (either because the XML parser can't handle the encoding in question or because there's an external encoding specified, but it's not possible to pass that to the parser), a codec makes the most sense. >> For non-seekable, non-peekable streams (if any), what you'd >> need would be a stream that consisted of a concatenation of the >> buffered data used for detection and the continuation of the stream. > > The application would read data out of the stream, and pass it to > the detection. It then can process it in whatever manner it meant to > process it in the first place. Servus, Walter ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] XML codec?
On 2007-11-11 23:22, Martin v. Löwis wrote: First, XML-RPC is not the only mechanism using XML over a network connection. Second, you don't want to do this if you're dealing with several 100 MB of data just because you want to figure out the encoding. >>> That's my original claim/question: what SPECIFIC application do >>> you have in mind that transfers XML over a network and where you >>> would want to have such a stream codec? >> XML-based web services used for business integration, e.g. based >> on ebXML. >> >> A common use case from our everyday consulting business is e.g. >> passing market and trading data to portfolio pricing web services. > > I still don't see the need for this feature from this example. > First, in ebXML messaging, the message are typically *not* large > (i.e. much smaller than 100 MB). Furthermore, the typical processing > of such a message would be to pass it directly to the XML parser, > no need for the functionality under discussion. I don't see the point in continuing this discussion. If you think you know better, that's fine. Just please don't generalize this to everyone else working with Python and XML. >>> Right. However, I' will remain opposed to adding this to the >>> standard library until I see why one would absolutely need to >>> have that. Not every piece of code that is useful in some >>> application should be added to the standard library. >> Agreed, but the application space of web services is large >> enough to warrant this. > > If that was the case, wouldn't the existing Python web service > libraries already include such a functionality? No. To finalize this: We have a -1 from Martin and a +1 from Walter, Guido and myself. Pretty clear vote if you ask me. I'd say we end the discussion here and move on. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Nov 12 2007) >>> Python/Zope Consulting and Support ...http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ...http://python.egenix.com/ Try mxODBC.Zope.DA for Windows,Linux,Solaris,MacOSX for free ! eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] XML codec?
On Nov 12, 2007, at 8:16 AM, M.-A. Lemburg wrote: > We have a -1 from Martin and a +1 from Walter, Guido and myself. > Pretty clear vote if you ask me. I'd say we end the discussion here > and move on. If we're counting, you've got a -1 on the codec from me as well. Martin's right: there's no value to embedding the logic of auto- detection into the codec. A function somewhere in the xml package is all that's warranted. -Fred -- Fred Drake ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] XML codec?
Fred Drake wrote: > On Nov 12, 2007, at 8:16 AM, M.-A. Lemburg wrote: >> We have a -1 from Martin and a +1 from Walter, Guido and myself. >> Pretty clear vote if you ask me. I'd say we end the discussion here >> and move on. > > If we're counting, you've got a -1 on the codec from me as well. > Martin's right: there's no value to embedding the logic of auto- > detection into the codec. It isn't "embedded". codecs.detect_xml_encoding() is callable without any problems (though not documented). > A function somewhere in the xml package is > all that's warranted. Who would use such a function for what? Servus, Walter ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] XML codec?
On Nov 12, 2007, at 8:56 AM, Walter Dörwald wrote: > It isn't "embedded". codecs.detect_xml_encoding() is callable without > any problems (though not documented). "Not documented" means not available, I think. > Who would use such a function for what? Being able to detect the encoding can be useful anytime you want information about a file, actually. In particular, presenting encoding information in a user interface (yes, you can call that contrived, but some people want to be able to see such things, and for them it's a requirement). If you want to parse the XML and re-encode, it's common to want to re-encode in the origin encoding; it's needed for that as well. If you just want to toss the text into an editor, the encoding is also needed. In that case, the codec approach *might* be acceptable (depending on the rest of the editor implementation), but the same re-encoding issue applies as well. Simply, it's sometimes desired to know the encoding for purposes that don't require immediate decoding. A function would be quite handing in these cases. -Fred -- Fred Drake ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] XML codec?
> Simply, it's sometimes desired to know the encoding for purposes that > don't require immediate decoding. A function would be quite handy > in these cases. In os.path? os.path.encoding(location)? Bill ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] XML codec?
On Nov 12, 2007, at 10:54 AM, Bill Janssen wrote: > In os.path? os.path.encoding(location)? I wasn't thinking it would be that general; determining the encoding for an arbitrary text file is a larger problem than it is for an XML file. An implementation based strictly on the rules from the XML specification should be in the xml package (somewhere). Determining that the file is an XML file is separate. I doubt this really makes sense in os.path. -Fred -- Fred Drake ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] Summary of Tracker Issues
ACTIVITY SUMMARY (11/05/07 - 11/12/07) Tracker at http://bugs.python.org/ To view or respond to any of the issues listed below, click on the issue number. Do NOT respond to this message. 1321 open (+17) / 11592 closed (+21) / 12913 total (+38) Open issues with patches: 418 Average duration of open issues: 689 days. Median duration of open issues: 791 days. Open Issues Breakdown open 1316 (+17) pending 5 ( +0) Issues Created Or Reopened (39) ___ test_import breaks on Linux 11/09/07 http://bugs.python.org/issue1377reopened gvanrossum py3k function comparing lacks NotImplemented error11/05/07 http://bugs.python.org/issue1393created _doublep simple patch, improving unreachable bytecode removing11/05/07 http://bugs.python.org/issue1394created _doublep patch py3k: duplicated line endings when using read(1) 11/06/07 http://bugs.python.org/issue1395created amaury.forgeotdarc py3k py3k-pep3137: patch for mailbox 11/06/07 CLOSED http://bugs.python.org/issue1396created tiran py3k, patch py3k-pep3137: failing unit test test_bsddb 11/06/07 http://bugs.python.org/issue1397created tiran py3k Can't pickle partial functions 11/07/07 CLOSED http://bugs.python.org/issue1398created danhs XML codec11/07/07 http://bugs.python.org/issue1399created doerwalter patch Py3k's print() flushing problem 11/07/07 http://bugs.python.org/issue1400created wojtekwalczak py3k urllib2 302 POST 11/07/07 http://bugs.python.org/issue1401created andresriancho Interpreter cleanup: order of _PyGILState_Fini and PyInterpreter 11/07/07 http://bugs.python.org/issue1402created ronaldoussoren py_compile and compileall need unit tests11/08/07 http://bugs.python.org/issue1403created tiran py3k warnings module bug: BytesWarning: str() on a bytes instance 11/08/07 CLOSED http://bugs.python.org/issue1404created tiran patch Garbage collection not working correctly in Python 2.3 11/09/07 CLOSED http://bugs.python.org/issue1405reopened tiran Use widechar api for os.environ 11/08/07 CLOSED http://bugs.python.org/issue1406created theller py3k, patch [performance] Too many closed() checkings11/08/07 http://bugs.python.org/issue1407created wojtekwalczak py3k Inconsistence in multiply list 11/08/07 CLOSED http://bugs.python.org/issue1408created beco new keyword-only function parameters interact badly with nested 11/08/07 http://bugs.python.org/issue1409created _doublep py3k BaseHTTPServer cannot accept Unicode data11/08/07 CLOSED http://bugs.python.org/issue1410created isonno A typo in tutorial 11/09/07
Re: [Python-Dev] Proposal for new 2to23 tool
On Nov 12, 2007 12:50 AM, Graham Horler <[EMAIL PROTECTED]> wrote: > On 12 Nov 2007, 03:24:34, Jan Claeys wrote: > > > > Op zondag 11-11-2007 om 17:19 uur [tijdzone -0800], schreef Brett > > Cannon: > > > On Nov 11, 2007 4:00 PM, Graham Horler <[EMAIL PROTECTED]> wrote: > > > > I have been developing in Python since 1.5, and now have to support 2.1 > > > > as a minimum version. I do like to keep my code runnable on newer > > > > versions however, and am considering the feasability of forward > > > > compatibility with Python 3.0. > > > > > > > > I also notice the Leo[1] project could use some assistance with forward > > > > compatibility. > > > > > > > > So I was wondering if anyone else had a need for a 2to23.py tool to help > > > > make code compatible with 3.0 but not break it for 2.x. > > > > > > What exactly are you proposing? We already have 2to3 > > > (http://svn.python.org/view/sandbox/trunk/2to3/) for source-to-source > > > translation from 2.x to 3.0. > > > > Graham wants to convert his code such that it works on both Python 2.x > > (probably even early versions of it?) & Python 3.x. Not 2 instances of > > code, but one source that works on both 2.x and 3.x... > > Absolutely > Well, we will do our best to make a common base between 2.6 and 3.0. But since things are still in flux who knows if everything can somehow be added through a __future__ statement in 2.6. As for supporting older versions, that won't happen. -Brett ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Proposal for new 2to23 tool
On Nov 12, 2007 12:50 AM, Graham Horler <[EMAIL PROTECTED]> wrote: > On 12 Nov 2007, 03:24:34, Jan Claeys wrote: > > > > Op zondag 11-11-2007 om 17:19 uur [tijdzone -0800], schreef Brett > > Cannon: > > > On Nov 11, 2007 4:00 PM, Graham Horler <[EMAIL PROTECTED]> wrote: > > > > I have been developing in Python since 1.5, and now have to support 2.1 > > > > as a minimum version. I do like to keep my code runnable on newer > > > > versions however, and am considering the feasability of forward > > > > compatibility with Python 3.0. > > > > > > > > I also notice the Leo[1] project could use some assistance with forward > > > > compatibility. > > > > > > > > So I was wondering if anyone else had a need for a 2to23.py tool to help > > > > make code compatible with 3.0 but not break it for 2.x. > > > > > > What exactly are you proposing? We already have 2to3 > > > (http://svn.python.org/view/sandbox/trunk/2to3/) for source-to-source > > > translation from 2.x to 3.0. > > > > Graham wants to convert his code such that it works on both Python 2.x > > (probably even early versions of it?) & Python 3.x. Not 2 instances of > > code, but one source that works on both 2.x and 3.x... > > Absolutely I don't believe that's possible. There are enough key differences between the two that having the same body of code work in both 2.x and 3.x would require an extensive, complicated runtime support system that no-one has all(the ability, the time, the motivation) to implement. And of course, even if you had such a system, the generated code wouldn't look anything like well-formed Python and would be a maintenance nightmare. Collin Winter ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Proposal for new 2to23 tool
Collin Winter wrote: > On Nov 12, 2007 12:50 AM, Graham Horler <[EMAIL PROTECTED]> wrote: >> On 12 Nov 2007, 03:24:34, Jan Claeys wrote: >>> Op zondag 11-11-2007 om 17:19 uur [tijdzone -0800], schreef Brett >>> Cannon: On Nov 11, 2007 4:00 PM, Graham Horler <[EMAIL PROTECTED]> wrote: > I have been developing in Python since 1.5, and now have to support 2.1 > as a minimum version. I do like to keep my code runnable on newer > versions however, and am considering the feasability of forward > compatibility with Python 3.0. > > I also notice the Leo[1] project could use some assistance with forward > compatibility. > > So I was wondering if anyone else had a need for a 2to23.py tool to help > make code compatible with 3.0 but not break it for 2.x. What exactly are you proposing? We already have 2to3 (http://svn.python.org/view/sandbox/trunk/2to3/) for source-to-source translation from 2.x to 3.0. >>> Graham wants to convert his code such that it works on both Python 2.x >>> (probably even early versions of it?) & Python 3.x. Not 2 instances of >>> code, but one source that works on both 2.x and 3.x... >> Absolutely > > I don't believe that's possible. There are enough key differences > between the two that having the same body of code work in both 2.x and > 3.x would require an extensive, complicated runtime support system > that no-one has all(the ability, the time, the motivation) to > implement. And of course, even if you had such a system, the generated > code wouldn't look anything like well-formed Python and would be a > maintenance nightmare. > Besides which, the migration path is already well-specified: write in 3.x-compatible 2.6 and use 2to3 to migrate the code with no further processing. This does eventually give you a single code base. Earlier versions need not apply - there is no way to accommodate them. regards Steve -- Steve Holden+1 571 484 6266 +1 800 494 3119 Holden Web LLC http://www.holdenweb.com/ ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Proposal for new 2to23 tool
On Nov 11, 2007 7:24 PM, Jan Claeys <[EMAIL PROTECTED]> wrote: > Graham wants to convert his code such that it works on both Python 2.x > (probably even early versions of it?) & Python 3.x. Not 2 instances of > code, but one source that works on both 2.x and 3.x... The transition strategy for 3.0 explicitly excludes this possibility. For example, you couldn't catch exceptions with an error variable, because the except clause syntax changed. You couldn't use print except to print a single string. You couldn't use unicode string literals; you couldn't distinguish between text and binary data in a meaningful way. -- --Guido van Rossum (home page: http://www.python.org/~guido/) ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] Fwd: volunteers for a test
Please indulge this off-topic post. I know there are a lot of Python developers here whose mother tongue is not English; would those folks mind participating in the experiment below? --Guido -- Forwarded message -- From: Jose Oriol Lopez Berengueres <[EMAIL PROTECTED]> Date: Nov 12, 2007 12:22 PM Subject: volunteers for a test To: [EMAIL PROTECTED] Dear Mr. Guido, A coleague and me at CERN are trying to measure the productivity differential of mothertonge and non-mothertonge programming... it just takes arround 3 minutes and we are looking. for all kinds of expert and non expert people... I was wondering since you are so famous if you know anybody willing to volunteer? http://cern.ch/test-volunteers Thanks a lot. J. oriol -- --Guido van Rossum (home page: http://www.python.org/~guido/) ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] XML codec?
>On Nov 12, 2007, at 8:16 AM, M.-A. Lemburg wrote: >> We have a -1 from Martin and a +1 from Walter, Guido and myself. >> Pretty clear vote if you ask me. I'd say we end the discussion here >> and move on. > >If we're counting, you've got a -1 on the codec from me as well. >Martin's right: there's no value to embedding the logic of auto- >detection into the codec. A function somewhere in the xml package is >all that's warranted. I agree with Fred here - it should be a function in the xml package, not a codec. -1 -- Andrew McNamara, Senior Developer, Object Craft http://www.object-craft.com.au/ ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
