[issue15642] Integrate pickle protocol version 4 GSoC work by Stefan Mihaila
Changes by Stefan Mihaila mstefa...@gmail.com: Added file: http://bugs.python.org/file30216/d0c3a8d4947a.diff ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue15642 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue15642] Integrate pickle protocol version 4 GSoC work by Stefan Mihaila
Stefan Mihaila added the comment: Hello. I apologize once again for not finalizing my work, but once I have started my final year of faculty and a job, I have been busy pretty much all the time. I would really like to finish this as I've really enjoyed working on it, and everything on PEP 3154 and some other stuff has already been implemented. The only remaining part was finalizing the code review and fixing some memory leaks that gave me some headaches at the time. I would really appreciate if you could give me a few more days before deciding to start a new implementation from scratch. I'll get to fixing those memory leaks in the next couple of days and then the code review can be finalized. Would this be acceptable to you? -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue15642 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue15642] Integrate pickle protocol version 4 GSoC work by Stefan Mihaila
Alexandre Vassalotti added the comment: I have started a new implementation of PEP 3154 since Stefan hasn't been active on his. Moving the discussion to Issue #17810. -- dependencies: -Unbinding of methods resolution: - out of date stage: patch review - committed/rejected status: open - closed superseder: - Implement PEP 3154 (pickle protocol 4) ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue15642 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue15642] Integrate pickle protocol version 4 GSoC work by Stefan Mihaila
Changes by Serhiy Storchaka storch...@gmail.com: -- nosy: +serhiy.storchaka ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue15642 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue15642] Integrate pickle protocol version 4 GSoC work by Stefan Mihaila
Changes by Jesús Cea Avión j...@jcea.es: -- nosy: +jcea ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue15642 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue15642] Integrate pickle protocol version 4 GSoC work by Stefan Mihaila
Antoine Pitrou added the comment: Are there also some known techniques on tracking down memory leaks? Nothing more than the usual debugging techniques. It is more of a matter of taste whether you like to add print() (or printf ;-)) calls, or set breakpoints in an actual debugger. i.e. pickle.dumps(3.0+1j, 4) leaks but pickle.dumps(3.0+1j, 3) does not. Well it looks like you've narrowed things down a bit here. However, there appears to be no difference in the code that gets executed in v3 to the one executed in v4. Even the differences in memoization? -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue15642 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue15642] Integrate pickle protocol version 4 GSoC work by Stefan Mihaila
Stefan Mihaila added the comment: Are there also some known techniques on tracking down memory leaks? I've played around with sys.gettotalrefcount to narrow down the place where the leaks occur, but they seem to only occur in v4, i.e. pickle.dumps(3.0+1j, 4) leaks but pickle.dumps(3.0+1j, 3) does not. However, there appears to be no difference in the code that gets executed in v3 to the one executed in v4. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue15642 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue15642] Integrate pickle protocol version 4 GSoC work by Stefan Mihaila
Stefan Mihaila added the comment: - I don't really like the idea of changing the semantics of the PUT and GET opcodes. I would prefer new opcodes if possible. Well, the semantics of PUT and GET haven't really changed. It's just that the PUT opcode is not generated anymore and memoization is done in agreement (i.e. both the pickler and the unpickler know when to memoize so their memo tables stay in sync). So, in fact, it's the semantics of the other opcodes that has slightly changed. - I would like to see benchmarks for this change. I've tried the following two snippets with timeit: ./python3.3 -m timeit \ -s 'from pickle import dumps' \ -s 'd=[a]*100' 'dumps(d,3)' # replace 3 with 4 for comparison ./python3.3 -m timeit \ -s 'from pickle import dumps' \ -s 'd=list(map(chr,range(0,256)))' \ 'dumps(d,3)' # replace 3 with 4 for comparison # you can also use loads(dumps(d,3)) here to benchmark both # operations at once The first one generates 99 BINGET opcodes. It generates 1 BINPUT opcode in pickle3 and no BINPUT opcodes in pickle4. There appears no noticeable speed difference. The second one generates no BINGET opcodes. It generates no BINPUT opcodes in v4, respectively 256 BINPUT opcodes in v3. It appears the v4 one is slightly faster, but I have a hard time comparing correctly, given that the measurements seem to have a very large standard deviation (v4 gets times somewhere between 32.3 and 44.2, whereas v3 gets times between 37.7 and 52.2). I'm not sure this is the best way to benchmark, so let me know what is usually used. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue15642 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue15642] Integrate pickle protocol version 4 GSoC work by Stefan Mihaila
Antoine Pitrou added the comment: Le mercredi 22 août 2012 à 15:49 +, Stefan Mihaila a écrit : I'm not sure this is the best way to benchmark, so let me know what is usually used. http://hg.python.org/benchmarks/ has pickling benchmarks, you may have to modify them to test different pickling protocols. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue15642 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue15642] Integrate pickle protocol version 4 GSoC work by Stefan Mihaila
Alexandre Vassalotti added the comment: Some quick thoughts about the new implicit memoization scheme in Stefan's implementation. - The new scheme will need to be documented in PEP 3154 before we can accept the change. - I don't really like the idea of changing the semantics of the PUT and GET opcodes. I would prefer new opcodes if possible. - I would like to see benchmarks for this change. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue15642 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue15642] Integrate pickle protocol version 4 GSoC work by Stefan Mihaila
Antoine Pitrou added the comment: Is this patch stable? Or are there further changes coming? -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue15642 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue15642] Integrate pickle protocol version 4 GSoC work by Stefan Mihaila
Stefan Mihaila added the comment: There are still some upcoming changes. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue15642 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue15642] Integrate pickle protocol version 4 GSoC work by Stefan Mihaila
Stefan Mihaila added the comment: Maybe you can set this issue as the superseder of issue9269, because the patches there have already been applied here. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue15642 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue15642] Integrate pickle protocol version 4 GSoC work by Stefan Mihaila
Changes by Andrew Svetlov andrew.svet...@gmail.com: -- nosy: +asvetlov ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue15642 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue15642] Integrate pickle protocol version 4 GSoC work by Stefan Mihaila
Changes by Alexandre Vassalotti alexan...@peadrop.com: Added file: http://bugs.python.org/file26881/pickle4-2.diff ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue15642 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue15642] Integrate pickle protocol version 4 GSoC work by Stefan Mihaila
Changes by Alexandre Vassalotti alexan...@peadrop.com: -- dependencies: +Unbinding of methods ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue15642 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue15642] Integrate pickle protocol version 4 GSoC work by Stefan Mihaila
Changes by Alexandre Vassalotti alexan...@peadrop.com: Removed file: http://bugs.python.org/file26881/pickle4-2.diff ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue15642 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue15642] Integrate pickle protocol version 4 GSoC work by Stefan Mihaila
Alexandre Vassalotti added the comment: Oops, wrong patch. Uploading the right one. -- Added file: http://bugs.python.org/file26882/pickle4-2.diff ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue15642 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue15642] Integrate pickle protocol version 4 GSoC work by Stefan Mihaila
Stefan Mihaila added the comment: Maybe we could postpone the review process for a few days until I fix some known issues -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue15642 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue15642] Integrate pickle protocol version 4 GSoC work by Stefan Mihaila
New submission from Alexandre Vassalotti: Stefan Mihaila has been working on the implementation of PEP 3154, plus some other enhancements. His work is pretty complete and ready to be reviewed. I will do my best to finish a thorough review of his changes by the end of next week. -- assignee: alexandre.vassalotti components: Library (Lib) hgrepos: 145 keywords: gsoc, patch messages: 168144 nosy: alexandre.vassalotti, mstefanro, pitrou priority: normal severity: normal stage: patch review status: open title: Integrate pickle protocol version 4 GSoC work by Stefan Mihaila type: enhancement versions: Python 3.4 ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue15642 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue15642] Integrate pickle protocol version 4 GSoC work by Stefan Mihaila
Changes by Alexandre Vassalotti alexan...@peadrop.com: Added file: http://bugs.python.org/file26794/pickle4.diff ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue15642 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue15642] Integrate pickle protocol version 4 GSoC work by Stefan Mihaila
Changes by Alexandre Vassalotti alexan...@peadrop.com: Removed file: http://bugs.python.org/file26794/pickle4.diff ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue15642 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue15642] Integrate pickle protocol version 4 GSoC work by Stefan Mihaila
Changes by Alexandre Vassalotti alexan...@peadrop.com: Added file: http://bugs.python.org/file26795/pickle4.diff ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue15642 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue15642] Integrate pickle protocol version 4 GSoC work by Stefan Mihaila
Alexandre Vassalotti added the comment: I get warnings when compiling with the patch: /home/avassalotti/pickle4/Modules/_pickle.c: In function ‘save_global_binary’: /home/avassalotti/pickle4/Modules/_pickle.c:2952: warning: pointer targets in passing argument 2 of ‘_Pickler_Write’ differ in signedness /home/avassalotti/pickle4/Modules/_pickle.c:872: note: expected ‘const char *’ but argument is of type ‘unsigned char *’ /home/avassalotti/pickle4/Modules/_pickle.c:2956: warning: pointer targets in passing argument 2 of ‘_Pickler_Write’ differ in signedness /home/avassalotti/pickle4/Modules/_pickle.c:872: note: expected ‘const char *’ but argument is of type ‘unsigned char *’ /home/avassalotti/pickle4/Modules/_pickle.c:2960: warning: pointer targets in passing argument 2 of ‘_Pickler_Write’ differ in signedness /home/avassalotti/pickle4/Modules/_pickle.c:872: note: expected ‘const char *’ but argument is of type ‘unsigned char *’ /home/avassalotti/pickle4/Modules/_pickle.c:2975: warning: pointer targets in passing argument 2 of ‘_Pickler_Write’ differ in signedness /home/avassalotti/pickle4/Modules/_pickle.c:872: note: expected ‘const char *’ but argument is of type ‘unsigned char *’ /home/avassalotti/pickle4/Modules/_pickle.c:2979: warning: pointer targets in passing argument 2 of ‘_Pickler_Write’ differ in signedness /home/avassalotti/pickle4/Modules/_pickle.c:872: note: expected ‘const char *’ but argument is of type ‘unsigned char *’ /home/avassalotti/pickle4/Modules/_pickle.c:2988: warning: pointer targets in passing argument 2 of ‘_Pickler_Write’ differ in signedness /home/avassalotti/pickle4/Modules/_pickle.c:872: note: expected ‘const char *’ but argument is of type ‘unsigned char *’ /home/avassalotti/pickle4/Modules/_pickle.c:3012: warning: pointer targets in passing argument 2 of ‘_Pickler_Write’ differ in signedness /home/avassalotti/pickle4/Modules/_pickle.c:872: note: expected ‘const char *’ but argument is of type ‘unsigned char *’ /home/avassalotti/pickle4/Modules/_pickle.c: In function ‘save_set’: /home/avassalotti/pickle4/Modules/_pickle.c:3368: warning: suggest parentheses around assignment used as truth value /home/avassalotti/pickle4/Modules/_pickle.c: In function ‘save_frozenset’: /home/avassalotti/pickle4/Modules/_pickle.c:3422: warning: suggest parentheses around assignment used as truth value These should be fixed. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue15642 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue15642] Integrate pickle protocol version 4 GSoC work by Stefan Mihaila
Alexandre Vassalotti added the comment: There are reference leaks in the _pickle.c part that will need to be fixed too. 22:36:29 [~/pickle4]$ ./python -m test.regrtest -R :: test_pickle [1/1] test_pickle beginning 9 repetitions 123456789 . test_pickle leaked [14780, 14780, 14780, 14780] references, sum=59120 1 test failed: test_pickle [319952 refs] -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue15642 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
