ANN: Pyasm version 0.3
Pyasm is a full-featured dynamic assembler written entirely in Python. By dynamic, I mean that it can be used to generate and execute machine code in python at runtime without requiring the generation of object files and linkage. It essentially allow 'inline' assembly in python modules on x86 platforms. In addition, pyasm can generate COFF format object files, which may be of interest to anyone writing a compiler in python. More details and downloads are available at: http://www.grant-olson.net/python/pyasm The 0.3 release is long overdue. It includes: + A binary installer for python 2.6. The previous binary installer was python 2.4. + Automatic mapping of python structure values in the assembler. For Example, assuming EAX is a pointer to a string, MOV [EAX+PyString_ob_sval],0x42424242 will change the first four letters of the string to B's + The assembler now supports implicit string creation: PUSH 'foo\n\0' + The ability to build and test with the mingw toolset on windows. This previously required an MSVC toolset. Thanks to Markus Läll for sorting this out. -Grant -- http://mail.python.org/mailman/listinfo/python-list
Re: pyasm 0.2 - dynamic x86 assembler for python
JanC wrote:
[an example of using decorators to control pyasm]
Another (perhaps wacky) approach would be to change the assembler source syntax
enough to make it legal Python - in particular, this means parenthesizing the
arguments - then it can just be stored in-line with other Python source. This
has the additional benefit that one could write support functions to enable the
source to be executed interactively in Python.
The following example uses the CPython opcodes, represented as Python functions.
Python control structures 'while' and 'if' are used as assembler directives
for flow.
Michael
import ackermann
Ackermann = assemble(ackermann.ackSrc)
[snip assembler output]
Ackermann
function ackSrc at 0x0185A570
Ackermann(3,8)
2045
# ackermann.py --
def ackSrc(m,n):
Compute Ackermann's function on a stack
# Can be assembled to Python bytecode, or (not implemented yet)
# executed in Python with suitable support functions
LOAD_CONST(Return)
LOAD_FAST(m)
LOAD_FAST(n)
while condition(ROT_TWO(), DUP_TOP(), LOAD_CONST(Return),
COMPARE_OP(!=)):
if condition(POP_TOP(), DUP_TOP(), LOAD_CONST(0), COMPARE_OP(==)):
POP_TOP()
POP_TOP()
LOAD_CONST(1)
BINARY_ADD()
else:
if condition(POP_TOP(), ROT_TWO(), DUP_TOP(), LOAD_CONST(0),
COMPARE_OP(==)):
POP_TOP()
POP_TOP()
LOAD_CONST(1)
BINARY_SUBTRACT()
LOAD_CONST(1)
else:
POP_TOP()
DUP_TOP()
LOAD_CONST(1)
BINARY_SUBTRACT()
ROT_THREE()
POP_TOP()
DUP_TOP()
LOAD_CONST(1)
BINARY_SUBTRACT()
ROT_THREE()
ROT_TWO()
POP_TOP()
POP_TOP()
return
# ByteCode.py --
Python ByteCode Assembler
Author: Michael Spencer
Version: 0 - Experiment
3/11/2005
Example usage:
import ackermann
Ackermann = assemble(ackermann.ackSrc)
[snip assembler output]
Ackermann
function ackSrc at 0x0185A570
Ackermann(3,8)
2045
import dis
import compiler
import compiler.ast as ast
opmap = dis.opmap
import new
import inspect
class AbstractVisitor(object):
Standard depth-first AST walker - dispatches to methods
based on Node class name
def __init__(self):
self._cache = {} # dispatch table
def visit(self, node,**kw):
#print Visiting: %s % node.__class__
if node is None: return None
cls = node.__class__
meth = self._cache.setdefault(cls,
getattr(self,'visit'+cls.__name__,self.default))
return meth(node, **kw)
def default(self, node, **kw):
for child in node.getChildNodes():
self.visit(child, **kw)
visitExpression = default
class Assembler(AbstractVisitor):
Python bytecode assembler
def __init__(self):
self._cache = {} # dispatch table
self.i = 0 # Bytecode instruction counter
self.co_varnames = []
self.co_consts = []
self.jumptable = {}
self.co_codelst = []
def emit(self, funcname, arg = None):
i = self.i
try:
opcode = opmap[funcname]
except KeyError:
raise SyntaxError, Unknown operation: %s % funcname
self.co_codelst.append(opcode)
if opcode dis.HAVE_ARGUMENT:
print %4s %4s %s %s % (i, opcode, funcname.ljust(20), arg)
self.co_codelst.extend(self._getlohi(arg))
self.i = i + 3
else:
print %4s %4s %s % (i, opcode, funcname.ljust(20))
self.i = i + 1
def getcodestring(self):
self._resolvejumps()
return .join(map(chr, self.co_codelst))
def getcode(self):
return new.code(self.co_argcount, # argcount
self.co_argcount, # nlocals
1,# stacksize
67, # flags
self.getcodestring(), # codestring
tuple(self.co_consts), # constants
tuple(self.co_varnames),# names
tuple(self.co_varnames),# varnames
assembly, # filename
self.co_name, # name
0, # firstlineno
# lnotab
)
def _getlohi(self, arg):
if isinstance(arg, int):
return arg % 256, arg / 256
else:
return None,None
def _resolvejumps(self):
for origin, dest in self.jumptable.iteritems():
self.co_codelst[origin+1:origin+3] = self._getlohi(dest - origin -
3)
def visitFunction(self, node, **kw):
self.co_name
Re: pyasm 0.2 - dynamic x86 assembler for python
[JanC] The code below makes it possible to write assembler code for different architectures (in case pyasm ever supports that ;) and also a Python code version to use when running on a system where no assembler code can be used. It prints: [Michael] Another (perhaps wacky) approach would be to change the assembler source syntax enough to make it legal Python - in particular, this means parenthesizing the arguments - then it can just be stored in-line with other Python source. This has the additional benefit that one could write support functions to enable the source to be executed interactively in Python. The following example uses the CPython opcodes, represented as Python functions. Python control structures 'while' and 'if' are used as assembler directives for flow. I've been trying to come up with responses to these two posts, but I'm just plain speechless. That should be taken as a complement. -- http://mail.python.org/mailman/listinfo/python-list
Re: PyAsm
Won't docstrings be removed in optimised bytecode ? that would stuff things up. Regards, Fuzzy http://www.voidspace.org.uk/python/index.shtml -- http://mail.python.org/mailman/listinfo/python-list
Re: PyAsm
I haven't checked PyPy out lately. I was under the impression the Pyrex/C backend was still doing static compilation. Guess I'll have to take a look. -- http://mail.python.org/mailman/listinfo/python-list
Re: pyasm 0.2 - dynamic x86 assembler for python
Hi! What about an interface like this: -- @pyasm def hello_world(*some_args): !CHARS hello_str 'Hello world!\n\0' !PROC hello_world PYTHON !ARG self !ARG args PUSH hello_str CALL PySys_WriteStdout ADD ESP, 0x4 MOV EAX,PyNone ADD [EAX],1 !ENDPROC hello_world(1,2,3) -- Meaning: Put the assembler into the doc-string of a function. Then use a decorator to run the assembler on the function's __doc__ string and build an assembly function that takes the same arguments to make the assembly function directly callable. Maybe the decorator line has to look like this: @pyasm(globals()) or something like that, I can't tell. I don't think it would be much work to implement this. Stefan -- http://mail.python.org/mailman/listinfo/python-list
Re: PyAsm
Stefan Behnel [EMAIL PROTECTED] wrote in message news:[EMAIL PROTECTED] Meaning: Put the assembler into the doc-string of a function. That has several issues. One is that you can't do string operations with it. Say you wanted some %d, %s etc in the string. If you use a documentation generator (eg epydoc) then the code becomes the API documentation for the function. Finally it bloats binary distributions. For example BitPim is 7-10MB binary distribution, full compressed with all doc strings removed. Including doc strings adds another 3MB to the compressed binary size! Instead I would suggest looking at the compile/eval/exec builtins in Python for inspiration. You can give a string to compile and it gives you something you can execute later in varying contexts. Roger -- http://mail.python.org/mailman/listinfo/python-list
Re: PyAsm
Hey Roger, I didn't realize that Stefan replied to the list and sent a private email reply. There seemed to be a lag in google groups today. I basically told him that I might be crazy enough to write an assembler in python, but I'm not crazy enough to start using those function decorators. I'm working more on the backend stuff now but I was considering adding the hook. I never realized that you couldn't use string interpolation on a docstring, so that's probably the showstopper. I don't want to take that functionality away. I was thinking that the decorator could cheat and just swallow the originating docstring when returning the assembly function. Introspection based tools (which I'm assuming epydoc is) would only see the new docstrings on the assembly function. Not that I have docstring functionality built in yet but it's on the todo list. Size also isn't an issue because I'm currently using a string anyway. But lack of string interpolation is an issue. -Grant P.S. Where'd you get that cool source code formatter for BitPim ;-) -- http://mail.python.org/mailman/listinfo/python-list
Re: PyAsm
On 10 Mar 2005 12:35:36 -0800, [EMAIL PROTECTED] [EMAIL PROTECTED] wrote: Hey Roger, I didn't realize that Stefan replied to the list and sent a private email reply. There seemed to be a lag in google groups today. I basically told him that I might be crazy enough to write an assembler in python, but I'm not crazy enough to start using those function decorators. I'm working more on the backend stuff now but I was considering adding the hook. I never realized that you couldn't use string interpolation on a docstring, so that's probably the showstopper. I don't want to take that functionality away. I was thinking that the decorator could cheat and just swallow the originating docstring when returning the assembly function. Introspection based tools (which I'm assuming epydoc is) would only see the new docstrings on the assembly function. Not that I have docstring functionality built in yet but it's on the todo list. Size also isn't an issue because I'm currently using a string anyway. But lack of string interpolation is an issue. Have you seen PyPy? They already have the ability to turn a native python function into pyrex and compile it on the fly. Stephen. -- http://mail.python.org/mailman/listinfo/python-list
[ANN] pyasm 0.2 - dynamic x86 assembler for python
PyASM by Grant Olson olsongt at verizon.net = PyASM is a dynamic x86 assembler for python. By dynamic, I mean that it can be used to generate inline assembly functions in python at runtime without requiring object file generation or linkage. New in version 0.2 -- + Linux Support. Will work in Linux environments as well as Windows. + Simplified Interface. You only need to use one function to generate code. + Preliminary Documentation. More information and downloads are available on my homepage: http://mysite.verizon.net/olsongt # ## helloWorld.py ## assembly hello world script # -- http://mail.python.org/mailman/listinfo/python-list
Re: pyasm 0.2 - dynamic x86 assembler for python
# ## helloWorld.py ## assembly hello world script # DOH! The example file got truncated. Here it is. # ## helloWorld.py ## assembly hello world script # from pyasm import pyasm pyasm(globals(),r !CHARS hello_str 'Hello world!\n\0' !PROC hello_world PYTHON !ARG self !ARG args PUSH hello_str CALL PySys_WriteStdout ADD ESP, 0x4 MOV EAX,PyNone ADD [EAX],1 !ENDPROC ) hello_world() -- http://mail.python.org/mailman/listinfo/python-list
[ANN] pyasm 0.1 - x86 assembler for Python
pyasm 0.1 - x86 assembler for Python This release is for early adopters only. It is not properly packaged and doesn't have very good documentation. It is however a functional assembler that should be of interest to some people. Current output targets include Windows-style COFF files that can be subsequently linked to produce executables, and more interestingly output can target memory in an existing Python process and binding within a Python namespace. That's right, you can now generate dynamic inline assembly straight from Python! A simple hello world function implementation is listed at the end of this message. The files test\test_object_creation.py and test\test_winmem.py in the distribution probably give the best examples of usage. Future plans include targeting ELF file formats and Linux memory at runtime, and of course real documentation. The package is available at: http://mysite.verizon.net/olsongt/pyasm-0.1.zip Enjoy, -Grant # # PYTHON HELLO WORLD IN ASSEMBLY # import pyasm.winmem from pyasm.x86asm import assembler, CDECL from pyasm.x86cpToMemory import CpToMemory nonePointer = id(None) a = assembler() a.ADStr(hello_world, Hello world!\n\0) a.AP(test_print, CDECL) a.AddLocal(self) a.AddLocal(args) #a.AI(INT 3) a.AI(PUSH hello_world) a.AI(CALL PySys_WriteStdout) #a.AI(INT 3) a.AI(MOV EAX,%s % id(None)) a.AI(ADD [EAX],0x1) #refcount manipulation a.EP() mem = CpToMemory(a.Compile(),pyasm.winmem) mem.MakeMemory() mem.BindPythonFunctions(globals()) test_print() # calls the assembly function -- http://mail.python.org/mailman/listinfo/python-announce-list Support the Python Software Foundation: http://www.python.org/psf/donations.html
[ANN] pyasm 0.1 - x86 assembler for Python
pyasm 0.1 - x86 assembler for Python This release is for early adopters only. It is not properly packaged and doesn't have very good documentation. It is however a functional assembler that should be of interest to some people. Current output targets include Windows-style COFF files that can be subsequently linked to produce executables, and more interestingly output can target memory in an existing Python process and binding within a Python namespace. That's right, you can now generate dynamic inline assembly straight from Python! A simple hello world function implementation is listed at the end of this message. The files test\test_object_creation.py and test\test_winmem.py in the distribution probably give the best examples of usage. Future plans include targeting ELF file formats and Linux memory at runtime, and of course real documentation. The package is available at: http://mysite.verizon.net/olsongt/pyasm-0.1.zip Enjoy, -Grant # # PYTHON HELLO WORLD IN ASSEMBLY # import pyasm.winmem from pyasm.x86asm import assembler, CDECL from pyasm.x86cpToMemory import CpToMemory nonePointer = id(None) a = assembler() a.ADStr(hello_world, Hello world!\n\0) a.AP(test_print, CDECL) a.AddLocal(self) a.AddLocal(args) #a.AI(INT 3) a.AI(PUSH hello_world) a.AI(CALL PySys_WriteStdout) #a.AI(INT 3) a.AI(MOV EAX,%s % id(None)) a.AI(ADD [EAX],0x1) #refcount manipulation a.EP() mem = CpToMemory(a.Compile(),pyasm.winmem) mem.MakeMemory() mem.BindPythonFunctions(globals()) test_print() # calls the assembly function -- http://mail.python.org/mailman/listinfo/python-list
