[issue12291] file written using marshal in 3.2 can be read by 2.7, but not 3.2 or 3.3

Georg Brandl ge...@python.org added the comment: Can we get this committed for 3.2.1 then? -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue12291 ___

[issue12291] file written using marshal in 3.2 can be read by 2.7, but not 3.2 or 3.3

Roundup Robot devnull@devnull added the comment: New changeset edba722f3b02 by Vinay Sajip in branch '3.2': Closes #12291: Fixed bug which was found when doing multiple loads from one stream. http://hg.python.org/cpython/rev/edba722f3b02 -- nosy: +python-dev resolution: - fixed

[issue12291] file written using marshal in 3.2 can be read by 2.7, but not 3.2 or 3.3

Roundup Robot devnull@devnull added the comment: New changeset 42dd11028e94 by Vinay Sajip in branch 'default': Closes #12291 for 3.3 - merged fix from 3.2. http://hg.python.org/cpython/rev/42dd11028e94 -- ___ Python tracker rep...@bugs.python.org

[issue12291] file written using marshal in 3.2 can be read by 2.7, but not 3.2 or 3.3

Georg Brandl ge...@python.org added the comment: Thanks! -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue12291 ___ ___ Python-bugs-list mailing

[issue12291] file written using marshal in 3.2 can be read by 2.7, but not 3.2 or 3.3

Changes by STINNER Victor victor.stin...@haypocalc.com: -- nosy: +haypo ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue12291 ___ ___

[issue12291] file written using marshal in 3.2 can be read by 2.7, but not 3.2 or 3.3

Antoine Pitrou pit...@free.fr added the comment: Latest patch looks ok to me. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue12291 ___ ___

[issue12291] file written using marshal in 3.2 can be read by 2.7, but not 3.2 or 3.3

Antoine Pitrou pit...@free.fr added the comment: I think the question is: will the slowdown apply to module import, or only to explicit uses of the marshal module? If the latter, then I think we can live with it - we discourage using marshal as a general-purpose serialization scheme anyway.

[issue12291] file written using marshal in 3.2 can be read by 2.7, but not 3.2 or 3.3

Vinay Sajip vinay_sa...@yahoo.co.uk added the comment: Antoine Pitrou pit...@free.fr added the comment: I think the question is: will the slowdown apply to module import, or only to explicit uses of the marshal module? If the latter, then I think we can live with it - we discourage

[issue12291] file written using marshal in 3.2 can be read by 2.7, but not 3.2 or 3.3

Antoine Pitrou pit...@free.fr added the comment: - assertEqual = self.assertEqual is more of a nuisance than anything else; you're making the code more complicated to read just to save a few keystrokes; same for assertIsInstance = self.assertIsInstance I'm not sure how it's more

[issue12291] file written using marshal in 3.2 can be read by 2.7, but not 3.2 or 3.3

Changes by Vinay Sajip vinay_sa...@yahoo.co.uk: Added file: http://bugs.python.org/file22528/c3ba4f7b5db1.diff ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue12291 ___

[issue12291] file written using marshal in 3.2 can be read by 2.7, but not 3.2 or 3.3

Vinay Sajip vinay_sa...@yahoo.co.uk added the comment: Antoine Pitrou pit...@free.fr added the comment: It's not proscribed, but trying to remove the self. because it's supposed to be more readable is a bit of a strange thing to do. Also, people reading the test suite should be

[issue12291] file written using marshal in 3.2 can be read by 2.7, but not 3.2 or 3.3

Changes by Vinay Sajip vinay_sa...@yahoo.co.uk: Added file: http://bugs.python.org/file22530/9993567039c0.diff ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue12291 ___

[issue12291] file written using marshal in 3.2 can be read by 2.7, but not 3.2 or 3.3

Vinay Sajip vinay_sa...@yahoo.co.uk added the comment: As a result of the changes to marshal.c, test_importlib needs a small change: the test_bad_marshal raises an EOFError now, whereas it raised ValueError before. I think it's because the earlier code in marshal didn't properly check for EOF

[issue12291] file written using marshal in 3.2 can be read by 2.7, but not 3.2 or 3.3

Antoine Pitrou pit...@free.fr added the comment: Le Sun, 26 Jun 2011 19:49:03 +, Vinay Sajip rep...@bugs.python.org a écrit : Added file: http://bugs.python.org/file22487/0feab4e7b27f.diff Just a nit, could you give descriptive file names to your patches? Hex numbers quickly get

[issue12291] file written using marshal in 3.2 can be read by 2.7, but not 3.2 or 3.3

Changes by Dirkjan Ochtman dirk...@ochtman.nl: -- nosy: +djc ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue12291 ___ ___ Python-bugs-list mailing

[issue12291] file written using marshal in 3.2 can be read by 2.7, but not 3.2 or 3.3

Vinay Sajip vinay_sa...@yahoo.co.uk added the comment: Just a nit, could you give descriptive file names to your patches? Hex numbers quickly get confusing. Ok - I was under the impression that those names were generated automatically from the changeset hash, and that changing the name

[issue12291] file written using marshal in 3.2 can be read by 2.7, but not 3.2 or 3.3

engelbert gruber grub...@users.sourceforge.net added the comment: add interleaved writing to the same file (which might happen or not?) the tests pass when vinay's marshal.c is applied. sorry for another patch -- Added file:

[issue12291] file written using marshal in 3.2 can be read by 2.7, but not 3.2 or 3.3

Changes by Vinay Sajip vinay_sa...@yahoo.co.uk: Added file: http://bugs.python.org/file22487/0feab4e7b27f.diff ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue12291 ___

[issue12291] file written using marshal in 3.2 can be read by 2.7, but not 3.2 or 3.3

Vinay Sajip vinay_sa...@yahoo.co.uk added the comment: I've incorporated Engelbert's two patches into test_marshal.py in my sandbox fix branch. I coalesced them into a single additional test in the existing BugsTestCase. -- type: - behavior ___

[issue12291] file written using marshal in 3.2 can be read by 2.7, but not 3.2 or 3.3

engelbert gruber grub...@users.sourceforge.net added the comment: patch to test_marshal.py that obviously fails in current implementation. IMHO is file might not be seekable one can not cache so maybe do not do it. -- nosy: +grubert Added file:

[issue12291] file written using marshal in 3.2 can be read by 2.7, but not 3.2 or 3.3

Changes by Vinay Sajip vinay_sa...@yahoo.co.uk: Added file: http://bugs.python.org/file22440/ab1c38ffb8d4.diff ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue12291 ___

[issue12291] file written using marshal in 3.2 can be read by 2.7, but not 3.2 or 3.3

Changes by Terry J. Reedy tjre...@udel.edu: -- versions: -Python 3.4 ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue12291 ___ ___ Python-bugs-list

[issue12291] file written using marshal in 3.2 can be read by 2.7, but not 3.2 or 3.3

Changes by Vinay Sajip vinay_sa...@yahoo.co.uk: Added file: http://bugs.python.org/file22411/9e367c8fd949.diff ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue12291 ___

[issue12291] file written using marshal in 3.2 can be read by 2.7, but not 3.2 or 3.3

Changes by Vinay Sajip vinay_sa...@yahoo.co.uk: Removed file: http://bugs.python.org/file22290/marshal-patch.diff ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue12291 ___

[issue12291] file written using marshal in 3.2 can be read by 2.7, but not 3.2 or 3.3

Changes by Vinay Sajip vinay_sa...@yahoo.co.uk: Removed file: http://bugs.python.org/file22314/marshal-patch2.diff ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue12291 ___

[issue12291] file written using marshal in 3.2 can be read by 2.7, but not 3.2 or 3.3

Vinay Sajip vinay_sa...@yahoo.co.uk added the comment: The problem with calling fileno() and fdopen() is that you bypass the buffering information held in BufferedIOReader. The first call works, but the FILE * pointer is now positioned at 4K, rather than just past the end of the object just

[issue12291] file written using marshal in 3.2 can be read by 2.7, but not 3.2 or 3.3

Vinay Sajip vinay_sa...@yahoo.co.uk added the comment: This seems a bit hacky, and I'm not sure how reliable it is. I added this after the read_object call: if (is_file) { PyObject * newpos; int cp, np; cp = ftell(rf.fp); newpos = PyObject_CallMethod(f,

[issue12291] file written using marshal in 3.2 can be read by 2.7, but not 3.2 or 3.3

Benjamin Peterson benja...@python.org added the comment: 2011/6/19 Vinay Sajip rep...@bugs.python.org: Vinay Sajip vinay_sa...@yahoo.co.uk added the comment: This seems a bit hacky, and I'm not sure how reliable it is. I added this after the read_object call:    if (is_file) {        

[issue12291] file written using marshal in 3.2 can be read by 2.7, but not 3.2 or 3.3

Benjamin Peterson benja...@python.org added the comment: 2011/6/19 Vinay Sajip rep...@bugs.python.org: Vinay Sajip vinay_sa...@yahoo.co.uk added the comment: The problem with calling fileno() and fdopen() is that you bypass the buffering information held in BufferedIOReader. The first call

[issue12291] file written using marshal in 3.2 can be read by 2.7, but not 3.2 or 3.3

Vinay Sajip vinay_sa...@yahoo.co.uk added the comment: Benjamin Peterson benja...@python.org added the comment: assert(newpos != NULL) That's because the call is failing. Why? It's seemingly because the Python code did a seek (in Python) which was not communicated to the FILE

[issue12291] file written using marshal in 3.2 can be read by 2.7, but not 3.2 or 3.3

Vinay Sajip vinay_sa...@yahoo.co.uk added the comment: @Benjamin: I missed commenting on your Why not 0?, but here's the reasoning: one can't assume that the file only contains one object to be read, at the beginning of the file. It may be that some data is being written to file using

[issue12291] file written using marshal in 3.2 can be read by 2.7, but not 3.2 or 3.3

Benjamin Peterson benja...@python.org added the comment: I think you're right about playing with the bare fd being too fragile. I think a simpler solution is to read say 1024 bytes at a time and buffer it internally. -- ___ Python tracker

[issue12291] file written using marshal in 3.2 can be read by 2.7, but not 3.2 or 3.3

Vinay Sajip vinay_sa...@yahoo.co.uk added the comment: Benjamin Peterson benja...@python.org added the comment: I think you're right about playing with the bare fd being too fragile. I think a simpler solution is to read say 1024 bytes at a time and buffer it internally. Doesn't

[issue12291] file written using marshal in 3.2 can be read by 2.7, but not 3.2 or 3.3

Changes by R. David Murray rdmur...@bitdance.com: -- nosy: +pitrou ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue12291 ___ ___ Python-bugs-list

[issue12291] file written using marshal in 3.2 can be read by 2.7, but not 3.2 or 3.3

Georg Brandl ge...@python.org added the comment: Any reviewers? -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue12291 ___ ___ Python-bugs-list

[issue12291] file written using marshal in 3.2 can be read by 2.7, but not 3.2 or 3.3

Benjamin Peterson benja...@python.org added the comment: Why can't you just call fileno() on the file object? -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue12291 ___

[issue12291] file written using marshal in 3.2 can be read by 2.7, but not 3.2 or 3.3

Vinay Sajip vinay_sa...@yahoo.co.uk added the comment: Sorry I'm being dense, but which file object do you mean? -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue12291 ___

[issue12291] file written using marshal in 3.2 can be read by 2.7, but not 3.2 or 3.3

Benjamin Peterson benja...@python.org added the comment: 2011/6/18 Vinay Sajip rep...@bugs.python.org: Vinay Sajip vinay_sa...@yahoo.co.uk added the comment: Sorry I'm being dense, but which file object do you mean? The python file object. --

[issue12291] file written using marshal in 3.2 can be read by 2.7, but not 3.2 or 3.3

Vinay Sajip vinay_sa...@yahoo.co.uk added the comment: Benjamin Peterson benja...@python.org added the comment: Vinay Sajip vinay_sa...@yahoo.co.uk added the comment: Sorry I'm being dense, but which file object do you mean? The python file object. Do you mean special-case

[issue12291] file written using marshal in 3.2 can be read by 2.7, but not 3.2 or 3.3

Benjamin Peterson benja...@python.org added the comment: 2011/6/18 Vinay Sajip rep...@bugs.python.org: Vinay Sajip vinay_sa...@yahoo.co.uk added the comment: Benjamin Peterson benja...@python.org added the  comment:  Vinay Sajip vinay_sa...@yahoo.co.uk added the  comment: Sorry I'm

[issue12291] file written using marshal in 3.2 can be read by 2.7, but not 3.2 or 3.3

Vinay Sajip vinay_sa...@yahoo.co.uk added the comment: Patch is now in my public sandbox on hg.python.org. -- hgrepos: +26 ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue12291 ___

[issue12291] file written using marshal in 3.2 can be read by 2.7, but not 3.2 or 3.3

Vinay Sajip vinay_sa...@yahoo.co.uk added the comment: For some reason, Create Patch is failing with a [Errno 2] No such file or directory: '/home/roundup/trackers/tracker/cpython/Doc/Makefile' I've logged an issue on the meta tracker: http://psf.upfronthosting.co.za/roundup/meta/issue405

[issue12291] file written using marshal in 3.2 can be read by 2.7, but not 3.2 or 3.3

Vinay Sajip vinay_sa...@yahoo.co.uk added the comment: Attached is an improved patch: 1. It's against the 3.2 branch, rather than default. 2. It has an added multiple load test in test_marshal.py. 3. There's more error checking for an EOF condition. 4. I've removed tab chars and used /* C89

[issue12291] file written using marshal in 3.2 can be read by 2.7, but not 3.2 or 3.3

Changes by Vinay Sajip vinay_sa...@yahoo.co.uk: -- stage: - patch review ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue12291 ___ ___

[issue12291] file written using marshal in 3.2 can be read by 2.7, but not 3.2 or 3.3

New submission from Vinay Sajip vinay_sa...@yahoo.co.uk: The attached file 'data.bin' was written using Python 3.2. It can be read by Python 2.7, but in 3.2 and 3.3, after the first object is read, the file pointer is positioned at EOF, causing an error on subsequent reads. A simple test

[issue12291] file written using marshal in 3.2 can be read by 2.7, but not 3.2 or 3.3

Amaury Forgeot d'Arc amaur...@gmail.com added the comment: Sadly, marshal.load() looks broken: - The function starts with the comment /* XXX Quick hack -- need to do this differently */ - It starts by calling f.read() which consumes the whole file (and explains the issue reported here) -

[issue12291] file written using marshal in 3.2 can be read by 2.7, but not 3.2 or 3.3

Vinay Sajip vinay_sa...@yahoo.co.uk added the comment: I've added a patch which I think fixes marshal.load(), and would be grateful for a review - it's my first C patch for Python. The test_marshal test passes, though I haven't yet added a test for the failing scenario. -- keywords:

[issue12291] file written using marshal in 3.2 can be read by 2.7, but not 3.2 or 3.3

Amaury Forgeot d'Arc amaur...@gmail.com added the comment: - Please replace tabs characters by space - // comments are not accepted by some picky C89 compilers Also, calling f.read(1) for each character may have a large performance impact. I don't know how this can be solved though.

[issue12291] file written using marshal in 3.2 can be read by 2.7, but not 3.2 or 3.3

Benjamin Peterson benja...@python.org added the comment: It should probably be buffered at 512 bytes or some other reasonable number. -- nosy: +benjamin.peterson ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue12291

[issue12291] file written using marshal in 3.2 can be read by 2.7, but not 3.2 or 3.3

Changes by Barry A. Warsaw ba...@python.org: -- nosy: +georg.brandl priority: normal - release blocker ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue12291 ___

[issue12291] file written using marshal in 3.2 can be read by 2.7, but not 3.2 or 3.3

Changes by Arfrever Frehtes Taifersar Arahesis arfrever@gmail.com: -- nosy: +Arfrever ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue12291 ___

[issue12291] file written using marshal in 3.2 can be read by 2.7, but not 3.2 or 3.3

Georg Brandl ge...@python.org added the comment: Sounds blocker-ish enough to me. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue12291 ___ ___

[issue12291] file written using marshal in 3.2 can be read by 2.7, but not 3.2 or 3.3

Vinay Sajip vinay_sa...@yahoo.co.uk added the comment: @Amaury: Re formatting and comment style - I will address this. The diff I posted is to the default branch rather than 3.2, though it won't change the main substance of the patch. Re. performance and buffering: I agree this could have an

[issue12291] file written using marshal in 3.2 can be read by 2.7, but not 3.2 or 3.3

Vinay Sajip vinay_sa...@yahoo.co.uk added the comment: Another thought about buffering, which might be a bit of a show-stopper: if in a particular load() call we read ahead for buffering purposes, we've altered the underlying stream (which might not be seekable to allow position restoring).