Guido van Rossum wrote:
On 6/13/07, Ron Adam <[EMAIL PROTECTED]> wrote:
Attached both the str8 repr as s"..." and s'...', and the latest
no_raw_escape patch which I think is complete now and should apply
with no
problems.
I like the str8 repr patch enough to check it in.
I tracked the random fails I am having in test_tokenize.py down to it
doing
a round trip on random test_*.py files. If one of those files has a
problem it causes test_tokanize.py to fail also. So I added a line to
the
test to output the file name it does the round trip on so those can be
fixed as they are found.
Let me know it needs to be adjusted or something doesn't look right.
Well, I'm still philosophically uneasy with r'\' being a valid string
literal, for various reasons (one being that writing a string parser
becomes harder and harder). I definitely want r'\u1234' to be a
6-character string, however. Do you have a patch that does just that?
(We can argue over the rest later in a larger forum.)
The str8 patch caused tokanize.py to fail again also. ;-) Those s'' == ''
asserts of course.
I tracked it down to cStringIO.c only returning str8 types. Fixing that
*may* fix a number of other modules as well, but I'm not sure how, so I put
a str() around the returned value in tokanize.py with a note for now.
The attached path has various minor fix's. (But not the no raw escape
stuff yet.)
Cheers,
Ron
tokenize.py - Get rid of s'' errors by making returned value from
cStringIO.c to unicode with str().
test_tokanize.py - Added printing of roundtrip file names to test. This is
needed because files are a random sample and if they have errors, it causes
this module to fail. Without this there is no way to tell what is going on.
_fileio.c - Return unicode strings instead of str8 strings. (check this one.)
smtplib - Fixed strip() without args on bytes.
test_fileinput.py - Replaced bad writelines() call with a for loop with a
write() call. (buffer object doesn't have a writelines method, and
try-finally hid the error.)
Index: Lib/smtplib.py
===================================================================
--- Lib/smtplib.py (revision 56007)
+++ Lib/smtplib.py (working copy)
@@ -348,7 +348,7 @@
self.close()
raise SMTPServerDisconnected("Connection unexpectedly closed")
if self.debuglevel > 0: print('reply:', repr(line), file=stderr)
- resp.append(line[4:].strip())
+ resp.append(line[4:].strip(b' \t\n'))
code=line[:3]
# Check that the error code is syntactically correct.
# Don't attempt to read a continuation line if it is broken.
@@ -361,7 +361,7 @@
if line[3:4]!="-":
break
- errmsg = "\n".join(resp)
+ errmsg = b"\n".join(resp)
if self.debuglevel > 0:
print('reply: retcode (%s); Msg: %s' % (errcode,errmsg), file=stderr)
return errcode, errmsg
Index: Lib/tokenize.py
===================================================================
--- Lib/tokenize.py (revision 56007)
+++ Lib/tokenize.py (working copy)
@@ -262,6 +262,7 @@
while 1: # loop over lines in stream
try:
line = readline()
+ line = str(line) # XXX cStringIO.c returns str8 types.
except StopIteration:
line = ''
lnum = lnum + 1
Index: Lib/test/test_tokenize.py
===================================================================
--- Lib/test/test_tokenize.py (revision 56007)
+++ Lib/test/test_tokenize.py (working copy)
@@ -189,6 +189,7 @@
for f in testfiles:
# Print still working message since this test can be really slow
+ print(' round trip: ', f, file=sys.__stdout__)
if next_time <= time.time():
next_time = time.time() + _PRINT_WORKING_MSG_INTERVAL
print(' test_main still working, be patient...', file=sys.__stdout__)
Index: Lib/test/test_fileinput.py
===================================================================
--- Lib/test/test_fileinput.py (revision 56007)
+++ Lib/test/test_fileinput.py (working copy)
@@ -20,7 +20,8 @@
def writeTmp(i, lines, mode='w'): # opening in text mode is the default
name = TESTFN + str(i)
f = open(name, mode)
- f.writelines(lines)
+ for line in lines:
+ f.write(line)
f.close()
return name
@@ -155,6 +156,8 @@
remove_tempfiles(t1, t2)
def test_unicode_filenames(self):
+ # XXX A unicode string is always returned by writeTmp.
+ # So is this needed?
try:
t1 = writeTmp(1, ["A\nB"])
encoding = sys.getfilesystemencoding()
@@ -198,6 +201,8 @@
remove_tempfiles(t1)
def test_file_opening_hook(self):
+ # XXX The rot13 codec was removed.
+ # So this test needs to be changed to use something else.
try:
# cannot use openhook and inplace mode
fi = FileInput(inplace=1, openhook=lambda f, m: None)
Index: Modules/_fileio.c
===================================================================
--- Modules/_fileio.c (revision 56007)
+++ Modules/_fileio.c (working copy)
@@ -705,7 +705,7 @@
static PyObject *
get_mode(PyFileIOObject *self, void *closure)
{
- return PyString_FromString(mode_string(self));
+ return PyUnicode_FromString(mode_string(self));
}
static PyGetSetDef fileio_getsetlist[] = {
_______________________________________________
Python-3000 mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-3000
Unsubscribe:
http://mail.python.org/mailman/options/python-3000/archive%40mail-archive.com
