[issue11217] python-32 not linked in /usr/local/bin in framework builds

[Tom Loredo]

New submission from Tom Loredo :

When building a universal framework Python-2.7.1 with homebrew on 10.6.6, 
python-32 (and its target, python2.7-32) are built and installed in the 
framework executable path, but they are not linked in /usr/local/bin.  
msg101156 in Issue 8089 recognized this as a general MacPython problem in a 
2.6.5 release candidate but deferred a fix for 2.6.6.  Apparently the fix was 
never implemented, or perhaps it was decided the link was not appropriate.  It 
seems to me it *is* appropriate (framework users don't currently have easy 
command-line access to python-32), but I defer to the experts!

--
assignee: ronaldoussoren
components: Macintosh
messages: 128578
nosy: ronaldoussoren, tloredo
priority: normal
severity: normal
status: open
title: python-32 not linked in /usr/local/bin in framework builds
type: behavior
versions: Python 2.7

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue11188] test_time error on AIX

[STINNER Victor]

STINNER Victor  added the comment:

> http://code.google.com/p/y2038/wiki/AmazingDiscoveries

<<< AIX again

Merijn informs me that before year 0 AIX gets very, very slow. >>>

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue9960] test_structmembers fails on s390x (bigendian 64-bit): int/Py_ssize_t issue

[Dave Malcolm]

Dave Malcolm  added the comment:

FWIW, this regressed in 2.6.6 relative to 2.6.5, due to r79646 adding the 
T_STRING_INPLACE test code to the 2.6 branch.

Note to self: https://bugzilla.redhat.com/show_bug.cgi?id=677392

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue10181] Problems with Py_buffer management in memoryobject.c (and elsewhere?)

[Martin v . Löwis]

Changes by Martin v. Löwis :


--
nosy:  -loewis

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue11216] email.message.Message set_charset does not encode properly?

[R. David Murray]

R. David Murray  added the comment:

Well, the docs don't say that the headers will be removed or modified as 
needed, only added as needed ;/.  And in fact the set_charset code does "if 
'Content-Transfer-Encoding' not in self".  set_payload also says it is the 
caller's responsibility to "maintain the payload's invariants".  I'm not sure 
what that means, but I suspect it means making sure it is consistent with any 
existing Content-Transfer-Encoding header.

So it looks to me like this is (a) a doc bug and (b) an API bug(*).  The latter 
can only be corrected in 3.3.

I'm open to argument that this should be treated as a code bug, but that 
argument needs to include reasons why fixing it would not cause backward 
incompatibility problems.

Although I assigned this to myself so I don't lose it, a proposed documentation 
patch would be welcome.

(*) I say an API bug because my guess is that the API works the way it does on 
the theory that a program might be updating an existing Message object and 
should in that instance change only what it is directed to change (the payload 
content, and the charset attribute if specified), even if that would produce an 
invalid Message.  The way the email package evolved this would make sense, but 
I think if we were (when we are) doing it over now a better API would be to 
maintain a valid Message object through the standard API and provide a "lower 
level" API for those programs needing to produce RFC-invalid messages for 
whatever reason.

--
assignee:  -> r.david.murray
nosy: +r.david.murray
stage:  -> needs patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue11216] email.message.Message set_charset does not encode properly?

[Shay Rojansky]

Shay Rojansky  added the comment:

Sorry, the "print part" below doesn't work on Python3 (I'm back in 2.6), but I 
see the same trouble by using part.__str__()

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue11216] email.message.Message set_charset does not encode properly?

[Shay Rojansky]

New submission from Shay Rojansky :

This may be my misunderstanding of the correct behavior, thanks in advance for 
your patience...

The issue happens when calling set_charset (or set_payload charset) after a 
Message has already been created and contains a Content-Transfer-Encoding 
header. Here's an example:

>>> from email.mime.text import MIMEText
>>> 
>>> part = MIMEText('Some stuff aéàçça', 'plain', 'utf-8')
>>> print part
>From nobody Mon Feb 14 19:57:17 2011
Content-Type: text/plain; charset="utf-8"
MIME-Version: 1.0
Content-Transfer-Encoding: base64

U29tZSBzdHVmZiBhw6nDoMOnw6dh

>>> part.set_payload('Other stuff aéàçça', 'utf-8')
>>> print part
>From nobody Mon Feb 14 19:57:25 2011
Content-Type: text/plain; charset="utf-8"
MIME-Version: 1.0
Content-Transfer-Encoding: base64

Other stuff aéàçça
>>> 
>>> del part['Content-Transfer-Encoding']
>>> part.set_payload('Still some other stuff aéàçça', 'utf-8')
>>> print part
>From nobody Mon Feb 14 19:57:40 2011
Content-Type: text/plain; charset="utf-8"
MIME-Version: 1.0
Content-Transfer-Encoding: base64

U3RpbGwgc29tZSBvdGhlciBzdHVmZiBhw6nDoMOnw6dh

First the text part is created with charset UTF-8, a dump shows a 
properly-encoded base64 UTF-8 part.

Then an attempt is made to modify the payload. The set_charset documentation 
clearly states that the message will be properly encoded/converted, but we get 
a malformed part with Content-Transfer-Endogin=base64 but without a 
base64-encoded payload.

Finally, as a workaround, I delete the Content-Transfer-Encoding header and try 
again, at which point the new payload is properly encoded.

Again, I'm sure there are reasons for this behavior, which nevertheless seems 
like a bug to me (shouldn't set_charset perform base64 and change the 
Content-Transfer-Encoding if necessary regardless of previous headers?). Maybe 
a documentation update would help people with this.

Thank you very much!

--
components: Library (Lib)
messages: 128573
nosy: Shay.Rojansky
priority: normal
severity: normal
status: open
title: email.message.Message set_charset does not encode properly?
type: behavior
versions: Python 2.6, Python 3.1

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue11171] Python 2.7.1 does not start when "./configure" is used with "--prefix" != "--exec-prefix"

[Barry A. Warsaw]

Barry A. Warsaw  added the comment:

Fixed in release27-maint: r88422

I am unable to reproduce in Python 3.1 (which does not have the sysconfig 
module) or 3.2 (which uses a different build-flags specific location for its 
Makefile).

--
resolution:  -> fixed
status: open -> closed
versions:  -Python 3.1, Python 3.2

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue11213] distutils2 test issue

[Alexis Metaireau]

Alexis Metaireau  added the comment:

yeah, thanks for that !

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue11208] example misprint in documentation whatsnew/3.2

[Raymond Hettinger]

Raymond Hettinger  added the comment:

Fixed.
Thank for the report.

--
resolution:  -> fixed
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue11212] Python memory limit on AIX

[Antoine Pitrou]

Antoine Pitrou  added the comment:

> We could change the option to become:
> --with-aix-maxdata=value
> where value could be between 0 and 8, with a default at 0 (nothing
> passed to the compiler).

Doesn't LDFLAGS work for that? I don't think we want a lot of
platform-specific options like that.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue11212] Python memory limit on AIX

[Sébastien Sablé]

Sébastien Sablé  added the comment:

Yes, welcome into the strange world of AIX...

You can allow up to 8 segments, which represent 2GB:
http://publib.boulder.ibm.com/infocenter/tivihelp/v2r1/index.jsp?topic=/com.ibm.itame3.doc_5.1/am51_perftune113.htm

We could change the option to become:
--with-aix-maxdata=value
where value could be between 0 and 8, with a default at 0 (nothing passed to 
the compiler).

Would that be OK with you?

The good news is that thanks to this patch I was able to play the full python 
test suite on AIX:

313 tests OK.
8 tests failed:
test_asyncore test_cmath test_distutils test_fileio test_locale
test_resource test_time test_wait4
16 tests skipped:
test_ctypes test_curses test_dbm_gnu test_epoll test_gdb
test_kqueue test_largefile test_ossaudiodev test_pep277
test_startfile test_tk test_ttk_guionly test_unicode_file
test_winreg test_winsound test_zipfile64

Not too bad.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue11214] test_asyncore fails on AIX

[Giampaolo Rodola']

Giampaolo Rodola'  added the comment:

I'd say the test can be removed. It's not really important.

--
assignee:  -> giampaolo.rodola

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue11215] test_fileio error on AIX

[Antoine Pitrou]

Antoine Pitrou  added the comment:

This looks vaguely similar to http://bugs.python.org/issue6236#msg91007, 
although the latter has been fixed.

--
nosy: +pitrou

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue11188] test_time error on AIX

[Alexander Belopolsky]

Alexander Belopolsky  added the comment:

I have come across the following post:

http://code.google.com/p/y2038/wiki/AmazingDiscoveries

I think some of the findings there may be relevant as we push the
boundaries of supported time values.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue11214] test_asyncore fails on AIX

[Antoine Pitrou]

Changes by Antoine Pitrou :


--
nosy: +giampaolo.rodola

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue11215] test_fileio error on AIX

[Sébastien Sablé]

New submission from Sébastien Sablé :

I have the following error when running test_fileio on AIX:

==
FAIL: testAbles (test.test_fileio.OtherFileTests)
--
Traceback (most recent call last):
  File 
"/san_cis/home/cis/.buildbot/python-aix6/3.x.phenix.xlc/build/Lib/test/test_fileio.py",
 line 269, in testAbles
self.assertEqual(f.seekable(), False)
AssertionError: True != False

--
Ran 34 tests in 0.032s

FAILED (failures=1)

I haven't investigated yet.

--
messages: 128564
nosy: sable
priority: normal
severity: normal
status: open
title: test_fileio error on AIX
versions: Python 3.2

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue11214] test_asyncore fails on AIX

[Sébastien Sablé]

New submission from Sébastien Sablé :

When running test_asyncore on AIX, I get the following error:

[122/337] test_asyncore
test test_asyncore failed -- Traceback (most recent call last):
  File 
"/san_cis/home/cis/.buildbot/python-aix6/3.x.phenix.xlc/build/Lib/test/test_asyncore.py",
 line 332, in test_strerror
self.assertIn("unknown error", err.lower())
AssertionError: 'unknown error' not found in 'error -1 occurred.'

I haven't investigated yet.

--
messages: 128563
nosy: sable
priority: normal
severity: normal
status: open
title: test_asyncore fails on AIX
versions: Python 3.2

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue11188] test_time error on AIX

[STINNER Victor]

STINNER Victor  added the comment:

> but tm_wday = 42 did not solve the problem it seems.

Can you try with tm_yday=-1 or tm_isdst=-2?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue11188] test_time error on AIX

[STINNER Victor]

STINNER Victor  added the comment:

> So test_negative is now OK

Ok, I converted your comment to a patch: strftime_aix.patch.

--
keywords: +patch
Added file: http://bugs.python.org/file20762/strftime_aix.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue11213] distutils2 test issue

[Antoine Pitrou]

Antoine Pitrou  added the comment:

Ok, works fine!

--
resolution:  -> invalid
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue11213] distutils2 test issue

[Antoine Pitrou]

New submission from Antoine Pitrou :

Just to test that auto-nosy works.

--
assignee: tarek
components: Distutils2
messages: 128559
nosy: alexis, eric.araujo, pitrou, tarek
priority: normal
severity: normal
status: open
title: distutils2 test issue
versions: 3rd party

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue11212] Python memory limit on AIX

[Antoine Pitrou]

Antoine Pitrou  added the comment:

I don't see why we would provide an option to replace a magic number with 
another magic number. I find it strange that AIX doesn't simply enable 
unbounded virtual address spaces.
(by "unbounded" I mean "bounded by CPU limitations only")

--
nosy: +pitrou

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue9920] test_cmath on atan fails on AIX

[Sébastien Sablé]

Sébastien Sablé  added the comment:

Here is the result after applying that patch:

==
FAIL: test_specific_values (__main__.CMathTests)
--
Traceback (most recent call last):
  File "Lib/test/test_cmath.py", line 380, in test_specific_values
msg=error_message)
  File "Lib/test/test_cmath.py", line 128, in rAssertAlmostEqual
'got {!r}'.format(a, b))
AssertionError: atanh0225: atanh(complex(-0.0, 5.6067e-320))
Expected: complex(-0.0, 5.6067e-320)
Received: complex(0.0, 5.6067e-320)
Received value insufficiently close to expected value.

--
Ran 16 tests in 0.081s

FAILED (failures=1, skipped=2)

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue11188] test_time error on AIX

[Sébastien Sablé]

Sébastien Sablé  added the comment:

I tried the following patch (_AIX is defined on AIX platforms):

Index: Modules/timemodule.c
===
--- Modules/timemodule.c(révision 88420)
+++ Modules/timemodule.c(copie de travail)
@@ -474,7 +474,7 @@
 else if (!gettmarg(tup, &buf) || !checktm(&buf))
 return NULL;
 
-#if defined(_MSC_VER) || defined(sun)
+#if defined(_MSC_VER) || defined(sun) || defined(_AIX)
 if (buf.tm_year + 1900 < 1 ||  < buf.tm_year + 1900) {
 PyErr_Format(PyExc_ValueError,
  "strftime() requires year in [1; ]",
@@ -694,11 +694,12 @@
 time_t tt;
 if (!gettmarg(tup, &buf))
 return NULL;
-buf.tm_wday = -1;  /* sentinel; original value ignored */
+/* invalid value that will not be changed if there is an error. */
+buf.tm_wday = 42;
 tt = mktime(&buf);
 /* Return value of -1 does not necessarily mean an error, but tm_wday
  * cannot remain set to -1 if mktime succedded. */
-if (tt == (time_t)(-1) && buf.tm_wday == -1) {
+if ((tt == (time_t)(-1)) && (buf.tm_wday == 42)) {
 PyErr_SetString(PyExc_OverflowError,
 "mktime argument out of range");
 return NULL;

This resulted in the following:
==
ERROR: test_mktime (__main__.TestAsctime4dyear)
--
Traceback (most recent call last):
  File "Lib/test/test_time.py", line 351, in test_mktime
self.assertEqual(time.mktime(tt), t)
OverflowError: mktime argument out of range

==
ERROR: test_mktime (__main__.TestStrftime4dyear)
--
Traceback (most recent call last):
  File "Lib/test/test_time.py", line 351, in test_mktime
self.assertEqual(time.mktime(tt), t)
OverflowError: mktime argument out of range

==
ERROR: test_mktime (__main__.Test4dyearBool)
--
Traceback (most recent call last):
  File "Lib/test/test_time.py", line 351, in test_mktime
self.assertEqual(time.mktime(tt), t)
OverflowError: mktime argument out of range

--
Ran 42 tests in 1.395s

FAILED (errors=3)

So test_negative is now OK, but tm_wday = 42 did not solve the problem it seems.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue11193] test_subprocess error on AIX

[Sébastien Sablé]

Sébastien Sablé  added the comment:

$ LANG=C ./python -Wd -E -bb -c "import sys; print(ascii(sys.argv))" $(echo -ne 
"abc\xff")['-c', 'abc\xff']
['-c', 'abc\xff']
$ LANG=C ./python -Wd -E -bb -c "import locale; 
print(locale.nl_langinfo(locale.CODESET))"
ISO8859-1
$ LANG=C ./python -Wd -E -bb -c "import locale; 
print(locale.getpreferredencoding())"
ISO8859-1

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue11212] Python memory limit on AIX

[Sébastien Sablé]

Sébastien Sablé  added the comment:

Here is a patch that would provide an --enable-aix-maxdata option to configure, 
in order to support 512MB of mermory.

I have not tested it yet.

--
keywords: +patch
Added file: http://bugs.python.org/file20761/patch_maxdata.diff

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue11212] Python memory limit on AIX

[Sébastien Sablé]

New submission from Sébastien Sablé :

On AIX, executables are compiled by default so that they cannot allocate more 
than 256MB of memory.

This is not enough in some cases; for example this is not enough to get the 
Python test suite to run completely.

As explained in IBM documentation:

By default each program gets one segment register (see 2.24) for its
data segment. As each segment register covers 256 MB, any calls to
malloc more will fail. Also programs that declare large global or static
arrays may fail to load. To allocate more segment registers to your
program, use the linker option -bmaxdata to specify the number of bytes
you need in the data segment as follows:

cc -o myprog -bmaxdata:0x2000 myprog.c

The example above would allocate an additional segment register to allow
for 512MB of data.

export LDR_CNTRL=MAXDATA=0x2000
start_process
unset LDR_CNTRL

Marking An Executable For Large Page Use
The XCOFF header in an executable file contains a new flag to indicate that the 
program wants to use
large pages to back its data and heap segments. This flag can be set when the 
application is linked by
specifying the -blpdata option on the ld command. The flag can also be set or 
cleared using the ldedit
command. The "ldedit -blpdata filename" command sets the large page data/heap 
flag in the specified
file. The "ldedit -bnolpdata filename" clears the large page flag. The ldedit 
command may also be used
to set an executable's maxdata value

--
components: Build
messages: 128553
nosy: sable
priority: normal
severity: normal
status: open
title: Python memory limit on AIX
versions: Python 3.2

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue11211] gzip.open() fails for gzipped file

[STINNER Victor]

STINNER Victor  added the comment:

test.jml is a ZIP archive, not a gzip archive.

"gzip -d" is kind enough to unpack it even if it is ZIP file and not a gzip 
file.

So it's not a Python bug at all.

--
nosy: +haypo
resolution:  -> invalid
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue10181] Problems with Py_buffer management in memoryobject.c (and elsewhere?)

[Pauli Virtanen]

Pauli Virtanen  added the comment:

Nick's plan of action above seems mostly OK to me.

Though, it might be simpler to collapse the PyManagedBuffer object to 
PyMemoryView, even if this requires memoryviews to serve two purposes.

[Nick:]
> I'm still not comfortable with a convention that relies on *clients*
> of the PEP 3118 API not mucking with the internals of the Py_buffer
> struct.

Some points against: (i) Having to look up keys by memory address from an 
additional PyDict is more work for the exporter than just passing around some 
PyMem_Malloc'd information in `internal`. (ii) There is already an "obj" field 
in the structure that the consumers are supposed to not mess with. (iii) The 
exporter cannot use the `internal` field for anything if bf_releasebuffer 
cannot rely on it being intact.

If the recommended consumer API is changed so that Py_buffer mainly sits inside 
a PyObject, it becomes more clear that Py_buffer is read-only for the consumer 
(-- which is what I believe the PEP intended, but did not write down).

[Nick:]
> Altering release() to simply decrement the reference count of the 
> managed buffer would defeat the whole point of having that method, so
> it may be necessary to allow early release with outstanding references
> and then include a "still alive" check in the code that allows access
> to the buffer details (similar to the way weak references work).

Early release does not seem possible if the buffer does not come from the 
original object:

lst = []
with memoryview(a) as m:
b = numpy.array(m)
lst.append(b)

Now, m.__exit__ cannot release the buffer, since `b` holds a buffer-interface 
lock to `m`. `b` is 3rd party code, and does not know anything about 
MemoryViews.

Some alternatives: (i) require that bf_getbuffer always gives a new lock on all 
exported buffers, if there are multiple, (ii) allow memoryview.__exit__ to fail 
silently, (iii) drop the context management.

I guess (i) would be a sane choice -- I don't see many use cases for the same 
object exporting multiple different buffers. It's not needed by Numpy, and I 
suspect there is no existing 3rd party code that relies on this (because it 
doesn't work with the current implementation of memoryview :)

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue11211] gzip.open() fails for gzipped file

[Toni Mueller]

New submission from Toni Mueller :

I have files that I would like to read with Python, but can't:

$ python jmlreader.py woerter-allg.jml
Traceback (most recent call last):
  File "jmlreader.py", line 14, in 
readFile(sys.argv[1])
  File "jmlreader.py", line 10, in readFile
for line in f:
  File "/usr/lib/python2.6/gzip.py", line 438, in next
line = self.readline()
  File "/usr/lib/python2.6/gzip.py", line 393, in readline
c = self.read(readsize)
  File "/usr/lib/python2.6/gzip.py", line 219, in read
self._read(readsize)
  File "/usr/lib/python2.6/gzip.py", line 255, in _read
self._read_gzip_header()
  File "/usr/lib/python2.6/gzip.py", line 156, in _read_gzip_header
raise IOError, 'Not a gzipped file'
IOError: Not a gzipped file

The file itself is a gzipped file, though: 

$ gzip -cd woerter-allg.jml|head











Unfortunately, the file is a bit weird:

$ file woerter-allg.jml
woerter-allg.jml: Zip archive data, at least v2.0 to extract

Rewriting the program with zlib, instead of gzip, didn't help a bit:

$ python jmlreader.py woerter-allg.jml.gz 
Traceback (most recent call last):
  File "jmlreader.py", line 15, in 
readFile(sys.argv[1])
  File "jmlreader.py", line 11, in readFile
unc = zlib.decompress(s)
zlib.error: Error -3 while decompressing data: incorrect header check


IOW, the file was actually compressed with gzip, not zip.

It would be nice if the gzip module could read the file. I've attached a file 
that reproduces the problem.

--
components: Extension Modules
files: test.jml
messages: 128550
nosy: tonimueller
priority: normal
severity: normal
status: open
title: gzip.open() fails for gzipped file
type: behavior
versions: Python 2.6
Added file: http://bugs.python.org/file20760/test.jml

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue11193] test_subprocess error on AIX

[STINNER Victor]

STINNER Victor  added the comment:

>>> sys.getfilesystemencoding()
'iso8859-1'

Ok, I expected this result.

Can you also try:

$ LC_ALL=C ./python -c "import sys; print(ascii(sys.argv))" $(echo -ne 
"abc\xff")
['-c', 'abc\udcff']
$ LC_ALL=C python3.1 -c "import locale; 
print(locale.nl_langinfo(locale.CODESET))"
ANSI_X3.4-1968
$ LC_ALL=C python3.1 -c "import locale; print(locale.getpreferredencoding())"
ANSI_X3.4-1968

Anyway, test_undecodable_env() is not written to support ISO-8859-1 filesystem 
encoding. I should patch the test to check the filesystem encoding.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue11193] test_subprocess error on AIX

[Sébastien Sablé]

Sébastien Sablé  added the comment:

I am not sure this is what you want:

> LC_ALL=C ./python
Python 3.2rc3 (py3k:88417M, Feb 14 2011, 10:37:42) 
[GCC 4.2.0] on aix6
Type "help", "copyright", "credits" or "license" for more information.
>>> import sys
>>> sys.getfilesystemencoding()
'iso8859-1'
>>> 

What information can I provide to help solve this bug?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue7305] urllib2.urlopen() segfault using SSL on Solaris

[Fabian Groffen]

Fabian Groffen  added the comment:

Perhaps I should have been a bit more clear.  Python 2.6 works fine for me on 
Solaris 10/Sparc64 (64-bits).  Python 2.7.1 also works on Solaris 10/Sparc 
(32-bits), but not on Sparc64 (64-bits).

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue10181] Problems with Py_buffer management in memoryobject.c (and elsewhere?)

[Nick Coghlan]

Nick Coghlan  added the comment:

If it helps, one way to think of it is that post-redesign, the PyManagedBuffer 
would be the "real" API object, with Py_buffer merely a way for data exporters 
to provide the information needed to initialise the managed buffer properly. 
(That actually fairly accurately tracks the origin of the Py_buffer struct - to 
avoid huge numbers of output variables in the GetBuffer API calls)

Is such a solution ideal? No, but it would be significantly better than what we 
have now. If the PEP had been implemented as written, we would also have been 
much better off, but given that we failed to pick up the discrepancies between 
spec and implementation at the time, we're now stuck with trying to improve the 
situation without breaking backwards compatibility.

Since I now agree with your diagnosis that the heart of the problem is the fact 
that Py_buffer contains lots of pointers and thus is difficult to copy safely, 
wrapping it in a PyObject (without adding any additional behaviour) in order to 
minimise copying seems like the most obvious solution.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue11208] example misprint in documentation whatsnew/3.2

[R. David Murray]

Changes by R. David Murray :


--
nosy: +rhettinger

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue7305] urllib2.urlopen() segfault using SSL on Solaris

[Fabian Groffen]

Fabian Groffen  added the comment:

I can reproduce this on Solaris 10/Sparc64 only, using Python 2.7.1.  My core 
files aren't really useful though, nothing it can point to, everything is 
corrupt.

--
nosy: +grobian
versions: +Python 2.7

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue670664] HTMLParser.py - more robust SCRIPT tag parsing

[R. David Murray]

Changes by R. David Murray :


--
nosy: +r.david.murray

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue10181] Problems with Py_buffer management in memoryobject.c (and elsewhere?)

[Nick Coghlan]

Nick Coghlan  added the comment:

The managed buffer needs to be a separate object so that multiple memoryview 
objects can share it. Essentially, we would end up with two APIs - the raw 
Py_buffer API and the higher level PyManagedBuffer one.

C extensions would have the choice of using the low level API (most likely the 
case for existing extensions) or making life easier for themselves by using the 
higher level one. It is, at least from the consumer side, almost exactly what 
you suggested in the first place: a PEP 3118 style API where the main interface 
object is a true PyObject instance. Providers will still need to deal with 
correctly populating the passed in Py_buffer instances, but that can be handled 
easily enough by storing exported managed buffers in a Python list and storing 
indices in the Py_buffer internal field.

PyMemoryView itself isn't appropriate as that higher level API, since it does a 
lot more than just provide an easy way to share a Py_buffer struct 
(specifically, thanks to slicing, a memoryview instance may not expose the 
entire underlying buffer). Managed buffers would be much simpler, simply 
providing read-only access to the fields of the managed Py_buffer struct.

Depending on how we end up defining the required consumer semantics, 
PyMemoryView_FromBuffer may or may not be fully salvageable. If we find 
dup_buffer semantics that are universally correct, then we may be able to keep 
it in its current form by building those into an alternate ManagedBuffer 
constructor, otherwise we may need to restrict it to cases where the obj field 
is NULL.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue11193] test_subprocess error on AIX

[STINNER Victor]

STINNER Victor  added the comment:

> I don't understand why the test pass on FreeBSD, Solaris and
> Mac OS X, but not on AIX.

Oh, the command line and the filesystem encodings may be different. 
test_undecodable_env() of test_subprocess.py uses os.environ which uses 
sys.getfilesystemencoding(), whereas test_undecodable_code() of 
test_cmd_line.py uses _Py_char2wchar() which uses mbstowcs() functions (which 
use the locale encoding).

sys.getfilesystemencoding() is initialized from nl_langinfo(CODESET).

nl_langinfo(CODESET) gives maybe "ISO-8859-1" (or an alias to this encoding) 
even if LC_ALL=C.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue10181] Problems with Py_buffer management in memoryobject.c (and elsewhere?)

[Antoine Pitrou]

Antoine Pitrou  added the comment:

> >> I'm *much* happier with the rule based on malloc/free semantics where
> >> the *pointer* passed to PyObject_GetBuffer must match a single later
> >> call to PyObject_ReleaseBuffer.
> >
> > Agreed that Py_buffer should have been a PyObject from the start, but
> > the PEP chose differently.
> 
> malloc/free modelled semantics have *nothing* to do with Py_buffer
> being a full PyObject in its own right.

Well, the moment you're talking about having a struct of heterogeneous
fields whose address should remain identical, why not use the one
internal abstraction that is designed exactly for that, aka PyObject.
Sure you can devise an other ownership model, but what's the point?
I'm talking about avoiding NIH and an additional maintenance burden due
to avoidable cruft.

> > We now have backwards compatibility constraints. What do we do with
> > PyMemoryView_FromBuffer? Also, there's probably some code out there that
> > likes to copy Py_buffers around.
> 
> Such code is likely to be broken regardless of how we clarify the
> semantics,

Well, PyMemoryView_FromBuffer is used in the stdlib right now (in the IO
lib).

> I think Pauli's right, we need to make memoryview
> re-exporting significantly smarter in order to cope correctly with
> mutable objects.

Ok.

> 1. Add "PyManagedBuffer" as an even lower level wrapper around
> Py_buffer. The only thing this would support is doing GetBuffer on
> construction and ReleaseBuffer when destroyed (or when explicitly
> asked to do so), as well as access to the information in the Py_buffer
> struct.
> 
> 2. Adjust PyMemoryView to use that object to access the source object
> that implements the PEP 3118 API.

Ouch. Why not fold all that directly into memoryview? A third
abstraction for support of what is supposed to be a simple API sounds 
too much.
(see, if Py_buffer was a PyObject, we'd have only one abstraction)

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue11188] test_time error on AIX

[STINNER Victor]

STINNER Victor  added the comment:

>  File ".../Lib/test/test_time.py", line 351, in test_mktime
>self.assertEqual(time.mktime(tt), t)
> OverflowError: mktime argument out of range

I don't know which values are "out of range". But I guess that the test fails 
because of t=-1, which would mean that tm_wday field is not modified by 
mktime().

I don't know if mktime() does really not support t=-1, or if we should use 
another sentinel. The original patch to fix this issue (support t=-1, issue 
#1726687) used tm_wday=42 instead of tm_wday=-1:

+   /* invalid value that will not be changed if there is an error. */
+   buf.tm_wday = 42;
tt = mktime(&buf);
-   if (tt == (time_t)(-1)) {
+   if ((tt == (time_t)(-1)) && (buf.tm_wday == 42)) {
PyErr_SetString(PyExc_OverflowError,
"mktime argument out of range");
return NULL;

The current code uses:

buf.tm_wday = -1;  /* sentinel; original value ignored */   
tt = mktime(&buf);  
/* Return value of -1 does not necessarily mean an error, but tm_wday   
 * cannot remain set to -1 if mktime succedded. */  
if (tt == (time_t)(-1) && buf.tm_wday == -1) {  
PyErr_SetString(PyExc_OverflowError,
"mktime argument out of range");
return NULL;
} 


>  File ".../Lib/test/test_time.py", line 337, in test_negative
>self.assertIn(text, ('-1', '-001'))
> AssertionError: '000/' not found in ('-1', '-001')

AIX doesn't support negative tm_year value. It should be added to the 
timemodule blacklist (#ifdef):

#if defined(_MSC_VER) || defined(sun)
if (buf.tm_year + 1900 < 1 ||  < buf.tm_year + 1900) {
PyErr_Format(PyExc_ValueError,
 "strftime() requires year in [1; ]",
 buf.tm_year + 1900);
return NULL;
}
#endif

I don't know if there is a reliable C define to check if the current OS is AIX 
(something like #ifdef sun). Python configure script checks if "uname -s" 
output starts with "AIX". We should maybe add a Python define in pyconfig.h 
using the configure script.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue10181] Problems with Py_buffer management in memoryobject.c (and elsewhere?)

[Nick Coghlan]

Nick Coghlan  added the comment:

On further reflection, I realised point 4 in that suggestion isn't reliable in 
a formal sense. Consider:

with memoryview(a) as m:
m[:]

The slice won't reference the buffer again, but isn't guaranteed by the 
language definition to have been garbage collected at the time m.release() is 
called by the with statement.

Altering release() to simply decrement the reference count of the managed 
buffer would defeat the whole point of having that method, so it may be 
necessary to allow early release with outstanding references and then include a 
"still alive" check in the code that allows access to the buffer details 
(similar to the way weak references work).

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue10181] Problems with Py_buffer management in memoryobject.c (and elsewhere?)

[Nick Coghlan]

Nick Coghlan  added the comment:

OK, to summarise that revisit into a coherent proposal:

1. Add "PyManagedBuffer" as an even lower level wrapper around Py_buffer. The 
only thing this would support is doing GetBuffer on construction and 
ReleaseBuffer when destroyed (or when explicitly asked to do so), as well as 
access to the information in the Py_buffer struct.

2. Adjust PyMemoryView to use that object to access the source object that 
implements the PEP 3118 API.

3. When copying or slicing memoryview objects, the new PyMemoryView instance 
acquires a reference to the existing PyManagedBuffer instance instead of 
creating a new one.

4. If memoryview.release() attempts to explicitly release the buffer, but there 
are additional references to the PyManagedBuffer object, the request will fail 
with a BufferError.

5. dup_buffer is deleted (with extreme prejudice)

6. 3rd party code is encouraged in the C API documentation to never ever copy 
Py_buffer structs around and to use PyManagedBuffer instead.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue10181] Problems with Py_buffer management in memoryobject.c (and elsewhere?)

[Nick Coghlan]

Nick Coghlan  added the comment:

Another idea we may want to revisit is the PyManagedBuffer concept, which would 
be a true PyObject that existed solely to simplify sharing of Py_buffer structs.

If memoryview used such an object internally, then copying and slicing would be 
quite simple - just INCREF the managed buffer instance without letting the 
original source object know anything was going on.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue10181] Problems with Py_buffer management in memoryobject.c (and elsewhere?)

[Nick Coghlan]

Nick Coghlan  added the comment:

On Mon, Feb 14, 2011 at 8:33 AM, Antoine Pitrou  wrote:
>> I'm still not comfortable with a convention that relies on *clients*
>> of the PEP 3118 API not mucking with the internals of the Py_buffer
>> struct.
>
> Which clients? Those who export the buffer, or those who consume it?

Consumers. (I'll try to stick to provider/consumer terminology, as
that's clearer in this context

>> I'm *much* happier with the rule based on malloc/free semantics where
>> the *pointer* passed to PyObject_GetBuffer must match a single later
>> call to PyObject_ReleaseBuffer.
>
> Agreed that Py_buffer should have been a PyObject from the start, but
> the PEP chose differently.

malloc/free modelled semantics have *nothing* to do with Py_buffer
being a full PyObject in its own right. All I mean is that whatever
pointer you call ReleaseBuffer with should be the one you passed to
GetBuffer, and the only thing tp_releasebuffer implementations should
rely on is the address of that pointer rather than the struct
contents. However, from what Pauli has said, we may want to go with
the alternative approach of saying the struct address is irrelevant,
and only the content matter, using the "internal" field to
disambiguate different exported buffers. I believe either will work,
and either places additional constraints on buffer API consumers that
aren't currently clearly documented.

> We now have backwards compatibility constraints. What do we do with
> PyMemoryView_FromBuffer? Also, there's probably some code out there that
> likes to copy Py_buffers around.

Such code is likely to be broken regardless of how we clarify the
semantics, in the same way that our own dup_buffer is currently broken
under either set of semantics (i.e. calling ReleaseBuffer with a
different address in one case, clobbering the "internal" field in
other cases). We will probably need to expose an official Py_buffer
copying function that gets all the subtle details right so that
extension authors can more easily avoid making the same mistakes.

>> As far as the question of re-exporting the underlying view or not
>> goes, I agree having "memoryview(a)" potentially refer to different
>> underlying memory from "a" itself (because the source object has
>> changed since the first view was exported) is a recipe for confusion.
>
> If an object changes its buffer while it's exported somewhere, it will
> always result in confusion for the user, regardless of how the
> memoryview object is implemented. All normal uses of the buffer API
> assume that the buffer's memory doesn't change while it's being accessed
> by its consumer (what would it mean to SHA1-hash or zlib-compress a
> changing piece of memory?).
> So I don't know why the memoryview object *in particular* should care
> about this.

I'm not talking about an exported view changing its details (that's
explicitly disallowed by the PEP), I'm talking about the fact that
sequential calls to PyObject_GetBuffer are permitted to return
different answers. That's the point Pauli's PictureSet example
illustrated - even though the toy example uses a somewhat clumsy API,
it's perfectly legitimate according to the documentation, and it shows
that the current memoryview implementation may behave strangely when
you copy or slice a view of a mutable object, even though the view
itself is guaranteed to remain valid.

Consider the following:

Traceback (most recent call last):
  File "", line 1, in 
BufferError: Existing exports of data: object cannot be re-sized

Now suppose that instead of disallowing the resize, bytearray (or a
similar object) permitted it by allocating a new memory buffer, while
keeping a reference to the old buffer around until the memoryview
releases it (an approach that is perfectly legitimate according to the
PEP). In that case, our current "use the source object" approach to
memoryview copying and slicing will backfire badly, since copies and
slices will be working off the *new* (empty) state of the object,
while the original memoryview will still be looking at the old
populated state. I think Pauli's right, we need to make memoryview
re-exporting significantly smarter in order to cope correctly with
mutable objects.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue11210] PyErr_SetFromWindowsErrWithFilenameObject() doesn't exist: remove it from pyerrors.h

[STINNER Victor]

New submission from STINNER Victor :

PyAPI_FUNC(PyObject *) PyErr_SetFromWindowsErrWithFilenameObject(int, const 
char *); has a strange prototype: "FilenameObject" usually indicates a 
PyObject* argument, not a char* argument. The function doesn't exist in 
Python/errors.c, but there is a PyErr_SetExcFromWindowsErrWithFilenameObject() 
function.

PyErr_SetFromWindowsErrWithFilenameObject() was introduced in pyerrors.h by 
r28999 (8 years ago), but I am unable to find any concrete implementation.


Attached patch removes PyErr_SetFromWindowsErrWithFilenameObject() from 
pyerrors.h.

--
components: Interpreter Core, Unicode
files: pyerrors_h.patch
keywords: easy, patch
messages: 128536
nosy: haypo
priority: normal
severity: normal
status: open
title: PyErr_SetFromWindowsErrWithFilenameObject() doesn't exist: remove it 
from pyerrors.h
versions: Python 3.3
Added file: http://bugs.python.org/file20759/pyerrors_h.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com