[issue11816] Refactor the dis module to provide better building blocks for bytecode analysis

Roundup Robot added the comment: New changeset bf997b22df06 by Victor Stinner in branch '3.5': Fix BytecodeTestCase.assertNotInBytecode() https://hg.python.org/cpython/rev/bf997b22df06 -- ___ Python tracker

[issue11816] Refactor the dis module to provide better building blocks for bytecode analysis

Nick Coghlan added the comment: I checked in the missing file after I woke up this morning. Maybe I'll learn to use hg import instead of patch some day... Sorry for the noise. -- status: open - closed ___ Python tracker rep...@bugs.python.org

[issue11816] Refactor the dis module to provide better building blocks for bytecode analysis

Thomas Kluyver added the comment: Ping - the latest patches (dis_api3 test_peepholer) are ready for review when someone's got a moment. Thanks! -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue11816

[issue11816] Refactor the dis module to provide better building blocks for bytecode analysis

Nick Coghlan added the comment: I created issue 17916 after realising that the new OO API doesn't yet provide an equivalent to dis.distb that returns an appropriate Bytecode object. (I don't think it makes sense to hold up this patch for that change) -- assignee: - ncoghlan

[issue11816] Refactor the dis module to provide better building blocks for bytecode analysis

Changes by Nick Coghlan ncogh...@gmail.com: -- stage: patch review - commit review ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue11816 ___ ___

[issue11816] Refactor the dis module to provide better building blocks for bytecode analysis

Nick Coghlan added the comment: Good thing test_peepholer was moved out to a separate patch - a failure of that picked up a bug in the new disassembly output (unifying the handling of name and constant dereferences had changed the way constant strings were reported in the disassembly, and the

[issue11816] Refactor the dis module to provide better building blocks for bytecode analysis

Roundup Robot added the comment: New changeset f65b867ce817 by Nick Coghlan in branch 'default': Issue #11816: multiple improvements to the dis module http://hg.python.org/cpython/rev/f65b867ce817 -- nosy: +python-dev ___ Python tracker

[issue11816] Refactor the dis module to provide better building blocks for bytecode analysis

Roundup Robot added the comment: New changeset d3fee4c64654 by Nick Coghlan in branch 'default': Issue #11816: switch test_peepholer to bytecode_helper http://hg.python.org/cpython/rev/d3fee4c64654 -- ___ Python tracker rep...@bugs.python.org

[issue11816] Refactor the dis module to provide better building blocks for bytecode analysis

Nick Coghlan added the comment: And two-and-a-bit years later, we're done - thanks all, any further feedback or problems can be filed as a new issue :) -- resolution: - fixed stage: commit review - committed/rejected status: open - closed ___

[issue11816] Refactor the dis module to provide better building blocks for bytecode analysis

Charles-François Natali added the comment: test_dis is failing on some buildbots: http://buildbot.python.org/all/builders/AMD64 Ubuntu LTS 3.x/builds/1674/steps/test/logs/stdio Re-running test 'test_dis' in verbose mode test test_dis crashed -- Traceback (most recent call last): File

[issue11816] Refactor the dis module to provide better building blocks for bytecode analysis

Antoine Pitrou added the comment: Yes, this is bytecode_helper hasn't been added to the repository. -- nosy: +pitrou ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue11816 ___

[issue11816] Refactor the dis module to provide better building blocks for bytecode analysis

Antoine Pitrou added the comment: (this is *because*, sorry) -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue11816 ___ ___ Python-bugs-list

[issue11816] Refactor the dis module to provide better building blocks for bytecode analysis

STINNER Victor added the comment: Ping! The test is still failing. -- nosy: +haypo ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue11816 ___ ___

[issue11816] Refactor the dis module to provide better building blocks for bytecode analysis

Thomas Kluyver added the comment: bytecode_helper is there in dis_api3.diff - anyone with commit rights should be able to add it to the repository. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue11816

[issue11816] Refactor the dis module to provide better building blocks for bytecode analysis

Roundup Robot added the comment: New changeset 84d1a0e32d3b by Nick Coghlan in branch 'default': Issue #11816: Add missing test helper http://hg.python.org/cpython/rev/84d1a0e32d3b -- ___ Python tracker rep...@bugs.python.org

[issue11816] Refactor the dis module to provide better building blocks for bytecode analysis

Thomas Kluyver added the comment: I've added docs and tests, and split the changes to test_peepholer into a separate patch. I haven't re-exposed details of the code object as attributes of Bytecode instances, because they're already available as e.g. bytecode.codeobj.co_names . I think it

[issue11816] Refactor the dis module to provide better building blocks for bytecode analysis

Changes by Thomas Kluyver tak...@gmail.com: Added file: http://bugs.python.org/file29695/test_peepholer.diff ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue11816 ___

[issue11816] Refactor the dis module to provide better building blocks for bytecode analysis

Thomas Kluyver added the comment: Updated version of the patch. Changed from review: - Included test.bytecode_helper module used by some tests - Updated docs to indicate that the changes are new in 3.4 - ByteCode - Bytecode - Added meaningful repr for Bytecode Still to do: - ? Re-expose

[issue11816] Refactor the dis module to provide better building blocks for bytecode analysis

Thomas Kluyver added the comment: I've updated Nick's patch so that test_dis and test_peephole pass again, and added a prototype ByteCode class (without any docs or tests for now, to allow for API discussion). The prototype ByteCode is instantiated with any of the objects that

[issue11816] Refactor the dis module to provide better building blocks for bytecode analysis

Nick Coghlan added the comment: Thanks Thomas! It's a promising start - a few more detailed comments in the patch review. I like the idea of creating the initial version as an object-oriented wrapper around the existing APIs, rather than completely refactoring the module to make everything

[issue11816] Refactor the dis module to provide better building blocks for bytecode analysis

Nick Coghlan added the comment: To clarify the vague allusion in my last comment, Ron's suggestion was along the lines of creating a dis.Bytecode object that encapsulated everything the dis module can figure out about a piece of compiled code. That would mean exposing the kind of info

[issue11816] Refactor the dis module to provide better building blocks for bytecode analysis

Changes by Yury Selivanov yselivanov...@gmail.com: -- nosy: +yselivanov ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue11816 ___ ___

[issue11816] Refactor the dis module to provide better building blocks for bytecode analysis

Nick Coghlan ncogh...@gmail.com added the comment: Given the imminent 3.3 beta 1 feature freeze and the fact I would like to explore Ron's suggestion of a higher level ByteCode object to encapsulate a sequence of instructions (along with additional information from the code object),

[issue11816] Refactor the dis module to provide better building blocks for bytecode analysis

Changes by Nick Coghlan ncogh...@gmail.com: -- hgrepos: -93 ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue11816 ___ ___ Python-bugs-list mailing

[issue11816] Refactor the dis module to provide better building blocks for bytecode analysis

Changes by Nick Coghlan ncogh...@gmail.com: -- hgrepos: +94 ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue11816 ___ ___ Python-bugs-list mailing

[issue11816] Refactor the dis module to provide better building blocks for bytecode analysis

Changes by Nick Coghlan ncogh...@gmail.com: Added file: http://bugs.python.org/file23853/5ce60675e572.diff ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue11816 ___

[issue11816] Refactor the dis module to provide better building blocks for bytecode analysis

Nick Coghlan ncogh...@gmail.com added the comment: MvL pointed out I hadn't updated the Hg repo reference when I moved my sandbox over to BitBucket - the diff it was generating was from the last time I updated my pydotorg sandbox in order to try something on the buildbots. --

[issue11816] Refactor the dis module to provide better building blocks for bytecode analysis

Ron Adam ron3...@gmail.com added the comment: Instead of a get_instructions() function, How about using a DisCode class that defines the API for accessing Opinfo tuples of a disassembled object. So instead of... for instr in dis.bytecode_instructions(thing): process(instr) You

[issue11816] Refactor the dis module to provide better building blocks for bytecode analysis

Changes by Nick Coghlan ncogh...@gmail.com: Removed file: http://bugs.python.org/file23569/issue11816_get_opinfo_branch_20111031.diff ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue11816 ___

[issue11816] Refactor the dis module to provide better building blocks for bytecode analysis

Changes by Nick Coghlan ncogh...@gmail.com: -- hgrepos: -17 ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue11816 ___ ___ Python-bugs-list mailing

[issue11816] Refactor the dis module to provide better building blocks for bytecode analysis

Changes by Nick Coghlan ncogh...@gmail.com: Removed file: http://bugs.python.org/file23773/9512712044a6.diff ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue11816 ___

[issue11816] Refactor the dis module to provide better building blocks for bytecode analysis

Changes by Nick Coghlan ncogh...@gmail.com: -- hgrepos: +93 ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue11816 ___ ___ Python-bugs-list mailing

[issue11816] Refactor the dis module to provide better building blocks for bytecode analysis

Changes by Nick Coghlan ncogh...@gmail.com: Added file: http://bugs.python.org/file23848/9512712044a6.diff ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue11816 ___

[issue11816] Refactor the dis module to provide better building blocks for bytecode analysis

Nick Coghlan ncogh...@gmail.com added the comment: OK, there's something crazy going on with Create Patch failing to pick up the latest changes. I've removed all the obsolete patches, and am doing another merge from default to see if I can get it to pick things up correctly. --

[issue11816] Refactor the dis module to provide better building blocks for bytecode analysis

Changes by Nick Coghlan ncogh...@gmail.com: Removed file: http://bugs.python.org/file23848/9512712044a6.diff ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue11816 ___

[issue11816] Refactor the dis module to provide better building blocks for bytecode analysis

Nick Coghlan ncogh...@gmail.com added the comment: Grr, Create Patch insists on trying to produce a patch based on https://bitbucket.org/ncoghlan/cpython_sandbox/changesets/9512712044a6. That checkin is from *September* and ignores all my recent changes :P Relevant meta-tracker issue:

[issue11816] Refactor the dis module to provide better building blocks for bytecode analysis

Nick Coghlan ncogh...@gmail.com added the comment: OK, manual up-to-date patch attached. -- Added file: http://bugs.python.org/file23849/issue11816_get_opinfo_branch_20111204.diff ___ Python tracker rep...@bugs.python.org

[issue11816] Refactor the dis module to provide better building blocks for bytecode analysis

Nick Coghlan ncogh...@gmail.com added the comment: @Ron: Now that it has a reasonably clear signature, I could see my way clear to making the Instruction._disassemble() method public, which makes it easy for people to compose their own disassembly output. For all the other display methods, I

[issue11816] Refactor the dis module to provide better building blocks for bytecode analysis

Changes by Nick Coghlan ncogh...@gmail.com: Removed file: http://bugs.python.org/file23773/9512712044a6.diff ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue11816 ___

[issue11816] Refactor the dis module to provide better building blocks for bytecode analysis

Changes by Nick Coghlan ncogh...@gmail.com: Added file: http://bugs.python.org/file23773/9512712044a6.diff ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue11816 ___

[issue11816] Refactor the dis module to provide better building blocks for bytecode analysis

Nick Coghlan ncogh...@gmail.com added the comment: I have updated my BitBucket repo with the following changes: - the main API is now dis.get_instructions() - the info class is now dis.Instruction - get_instructions() accepts a 'line_offset' argument that is added to any source code line

[issue11816] Refactor the dis module to provide better building blocks for bytecode analysis

Nick Coghlan ncogh...@gmail.com added the comment: Somewhat inevitably, the detailed disassembler tests broke when Antoine updated the code generation for function definitions (as part of PEP 3155). (At least, the tests broke, and PEP 3155 seems the most likely culprit). I subsequently

[issue11816] Refactor the dis module to provide better building blocks for bytecode analysis

Changes by Nick Coghlan ncogh...@gmail.com: Added file: http://bugs.python.org/file23773/9512712044a6.diff ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue11816 ___

[issue11816] Refactor the dis module to provide better building blocks for bytecode analysis

Nick Coghlan ncogh...@gmail.com added the comment: Meador's suggested name change has grown on me, so I plan to switch the name of the new API to get_instructions() and the new class to Instruction. -- assignee: rhettinger - ncoghlan ___ Python

[issue11816] Refactor the dis module to provide better building blocks for bytecode analysis

Nick Coghlan ncogh...@gmail.com added the comment: Bitbucket repo and attached patch updated relative to current tip. -- Added file: http://bugs.python.org/file23569/issue11816_get_opinfo_branch_20111031.diff ___ Python tracker

[issue11816] Refactor the dis module to provide better building blocks for bytecode analysis

Changes by Nick Coghlan ncogh...@gmail.com: Removed file: http://bugs.python.org/file23197/issue11816_get_opinfo_branch_20110920.diff ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue11816 ___

[issue11816] Refactor the dis module to provide better building blocks for bytecode analysis

Changes by Nick Coghlan ncogh...@gmail.com: Removed file: http://bugs.python.org/file23095/issue11816_get_opinfo_branch_20110904.diff ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue11816 ___

[issue11816] Refactor the dis module to provide better building blocks for bytecode analysis

Changes by Nick Coghlan ncogh...@gmail.com: Removed file: http://bugs.python.org/file23019/issue11816_get_opinfo_branch_20110824.diff ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue11816 ___

[issue11816] Refactor the dis module to provide better building blocks for bytecode analysis

Changes by Nick Coghlan ncogh...@gmail.com: Removed file: http://bugs.python.org/file21599/dis.patch ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue11816 ___

[issue11816] Refactor the dis module to provide better building blocks for bytecode analysis

Changes by Eric Snow ericsnowcurren...@gmail.com: -- nosy: +eric.snow ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue11816 ___ ___ Python-bugs-list

[issue11816] Refactor the dis module to provide better building blocks for bytecode analysis

Meador Inge mead...@gmail.com added the comment: I agree that 'bytecode_instructions' is a long-winded. FWIW, I have worked on or with a fair amount instruction level things and instruction or instr seem to be the established domain terminology. Here are a few examples: * Java ASM -

[issue11816] Refactor the dis module to provide better building blocks for bytecode analysis

Meador Inge mead...@gmail.com added the comment: I took a quick look over the final patch (I will do a more thorough review later). I like the general idea a lot. The first thing that popped out at me are the names 'OpInfo' and 'get_opinfo'. 'OpInfo' makes it sound like information concerning

[issue11816] Refactor the dis module to provide better building blocks for bytecode analysis

Nick Coghlan ncogh...@gmail.com added the comment: 'Op' is just an abbreviation of 'operation'. So 'operation code' becomes 'opcode' and 'operation information' becomes 'opinfo'. The fact that it comes for the 'dis' module gives the context that the *kind* of operation we're talking about is

[issue11816] Refactor the dis module to provide better building blocks for bytecode analysis

Nick Coghlan ncogh...@gmail.com added the comment: Attached patch should now be complete, including the documentation for the new keyword-only 'file' parameter on various dis module functions. -- Added file: http://bugs.python.org/file23197/issue11816_get_opinfo_branch_20110920.diff

[issue11816] Refactor the dis module to provide better building blocks for bytecode analysis

Changes by Meador Inge mead...@gmail.com: -- stage: - patch review ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue11816 ___ ___ Python-bugs-list

[issue11816] Refactor the dis module to provide better building blocks for bytecode analysis

Nick Coghlan ncogh...@gmail.com added the comment: Regenerated the get_opinfo patch against current 3.3 tip. Still haven't fixed the missing doc updates mentioned in my last message, though. -- Added file: http://bugs.python.org/file23095/issue11816_get_opinfo_branch_20110904.diff

[issue11816] Refactor the dis module to provide better building blocks for bytecode analysis

Raymond Hettinger raymond.hettin...@gmail.com added the comment: Nick, I still want to work on this one. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue11816 ___

[issue11816] Refactor the dis module to provide better building blocks for bytecode analysis

Changes by Eugene Toder elto...@gmail.com: -- nosy: -eltoder ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue11816 ___ ___ Python-bugs-list mailing

[issue11816] Refactor the dis module to provide better building blocks for bytecode analysis

Changes by Nick Coghlan ncogh...@gmail.com: Added file: http://bugs.python.org/file23018/7cead6bf5832.diff ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue11816 ___

[issue11816] Refactor the dis module to provide better building blocks for bytecode analysis

Changes by Nick Coghlan ncogh...@gmail.com: Removed file: http://bugs.python.org/file23018/7cead6bf5832.diff ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue11816 ___

[issue11816] Refactor the dis module to provide better building blocks for bytecode analysis

Nick Coghlan ncogh...@gmail.com added the comment: The diff generator didn't work - I've uploaded the current patch manually to make it easier to review than it is in my bitbucket repo. I just noticed there's a missing element in the docs patch at the moment - to make testing easier, Ryan

[issue11816] Refactor the dis module to provide better building blocks for bytecode analysis

Changes by Ryan Kelly r...@rfk.id.au: -- nosy: +rfk ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue11816 ___ ___ Python-bugs-list mailing list

[issue11816] Refactor the dis module to provide better building blocks for bytecode analysis

Changes by Nick Coghlan ncogh...@gmail.com: -- assignee: rhettinger - ncoghlan ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue11816 ___ ___

[issue11816] Refactor the dis module to provide better building blocks for bytecode analysis

Changes by Jesús Cea Avión j...@jcea.es: -- nosy: +jcea ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue11816 ___ ___ Python-bugs-list mailing list

[issue11816] Refactor the dis module to provide better building blocks for bytecode analysis

Jesús Cea Avión j...@jcea.es added the comment: Do not forget to update docs too. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue11816 ___ ___

[issue11816] Refactor the dis module to provide better building blocks for bytecode analysis

Changes by Alexander Belopolsky belopol...@users.sourceforge.net: -- nosy: +belopolsky ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue11816 ___ ___

[issue11816] Refactor the dis module to provide better building blocks for bytecode analysis

Nick Coghlan ncogh...@gmail.com added the comment: Changed issue title to cover ideas like get_opinfo(). -- title: Add functions to return disassembly as string - Refactor the dis module to provide better building blocks for bytecode analysis ___

[issue11816] Refactor the dis module to provide better building blocks for bytecode analysis

Nick Coghlan ncogh...@gmail.com added the comment: Oops, I forgot to edit my comment to match the OpInfo definition I used in the proof-of-concept: OpInfo = collections.namedtuple(OpInfo, opindex opcode opname oparg details starts_line is_jump_target) --

[issue11816] Refactor the dis module to provide better building blocks for bytecode analysis

Eugene Toder elto...@gmail.com added the comment: So in the near term, dis-based tests should continue to copy/paste sys.stdout redirection code? -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue11816

[issue11816] Refactor the dis module to provide better building blocks for bytecode analysis

Nick Coghlan ncogh...@gmail.com added the comment: If we decide our long term goal is the use of the opcode stream for programmatic access, then yes. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue11816

[issue11816] Refactor the dis module to provide better building blocks for bytecode analysis

Alex Gaynor alex.gay...@gmail.com added the comment: FWIW in PyPy we have https://bitbucket.org/pypy/pypy/src/default/lib_pypy/disassembler.py which we use for some of our tools. -- nosy: +alex ___ Python tracker rep...@bugs.python.org

[issue11816] Refactor the dis module to provide better building blocks for bytecode analysis

Changes by Raymond Hettinger raymond.hettin...@gmail.com: -- assignee: - rhettinger ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue11816 ___ ___