[issue2480] pickling of large recursive structures fails
Alexandre Vassalotti [EMAIL PROTECTED] added the comment: Bob Kline wrote: I just ran into this behavior with an attempt to pickle a dom tree for an XML document whose nesting level never got deeper than nine child nodes, and indeed it crashed the interpreter. Pickling recursive data-structure should not crash the interpreter. Please open a new issue and don't forget to provide an example case. the documentation for the pickle module [1] which claims (summarizing) that serializing recursive objects using marshal will fail but pickling recursive objects will not fail. The section of documentation, you are referring to, uses the term recursive object to means an object which contains a reference to itself. Anyway, the documentation [1] states clearly: Trying to pickle a highly recursive data structure may exceed the maximum recursion depth, a RuntimeError will be raised in this case. You can carefully raise this limit with sys.setrecursionlimit(). [1]: http://docs.python.org/lib/node317.html -- nosy: +alexandre.vassalotti type: crash - feature request __ Tracker [EMAIL PROTECTED] http://bugs.python.org/issue2480 __ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue2480] pickling of large recursive structures fails
Bob Kline [EMAIL PROTECTED] added the comment: I just ran into this behavior with an attempt to pickle a dom tree for an XML document whose nesting level never got deeper than nine child nodes, and indeed it crashed the interpreter. Throwing an exception would be preferable, of course, to silent blowing up Python, but even the exception seems to fly in the face of the documentation for the pickle module [1] which claims (summarizing) that serializing recursive objects using marshal will fail but pickling recursive objects will not fail. I can provide a repro for the XML/DOM pickling case if you think that would be helpful, but that seems redundant since essentially you've already illustrated the problem with your own repro case. Thanks for your work on the solution. [1] http://docs.python.org/lib/node314.html -- nosy: +bkline type: - crash __ Tracker [EMAIL PROTECTED] http://bugs.python.org/issue2480 __ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue2480] pickling of large recursive structures fails
Changes by Jesús Cea Avión [EMAIL PROTECTED]: -- nosy: +jcea __ Tracker [EMAIL PROTECTED] http://bugs.python.org/issue2480 __ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue2480] pickling of large recursive structures fails
Daniel Darabos [EMAIL PROTECTED] added the comment: So now I've learned that this is a result of the way Pickler is implemented. I think that it would make sense to create an implementation that is not that recursive and that would handle such structures better. I have now written one such implementation that is a subclass of the original pickler and handles recursive data structures. I presume that it is inferior in a couple of ways (performance and memory efficiency probably), but a complete rewrite could probably create an implementation that handles recursive data structures and is not inferior. -- title: pickling of recursive sets of objects fails - pickling of large recursive structures fails Added file: http://bugs.python.org/file9850/nonrecursivepickler.py __ Tracker [EMAIL PROTECTED] http://bugs.python.org/issue2480 __ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue2480] pickling of large recursive structures fails
Changes by Daniel Darabos [EMAIL PROTECTED]: Added file: http://bugs.python.org/file9851/picklertest.py __ Tracker [EMAIL PROTECTED] http://bugs.python.org/issue2480 __ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue2480] pickling of large recursive structures fails
Changes by Ralf Schmitt [EMAIL PROTECTED]: -- nosy: +schmir __ Tracker [EMAIL PROTECTED] http://bugs.python.org/issue2480 __ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue2480] pickling of large recursive structures fails
Daniel Darabos [EMAIL PROTECTED] added the comment: Sidenote: If I click edit for nonrecursivepickler.py, I get told that File has been classified as spam. Strange. Should I not use Viagra as a classname? :) (j/k I didn't do anything like that) However I have now fixed a mistake in my code. Added file: http://bugs.python.org/file9852/nonrecursivepickler-fixed.py __ Tracker [EMAIL PROTECTED] http://bugs.python.org/issue2480 __ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
