2010/8/10 NoiseEHC <[email protected]>: > >> We used to do that, the problem is that we don't control our platform >> as Google controls Android and you need to make sure that resources >> that need to be specific of each child process aren't shared (dbus and >> X connections, etc). >> >> I'm personally more interested in reducing the amount of resources we >> need to start activities (which is quite insane right now), but >> sharing more of those resources sounds like a good idea to me. >> > > Since I spent quite a bit of time analyzing the Python runtime here is my > conclusion, maybe it will make wasting all that time less painful. > > First, most of the memory consumed by an activity is the process heap. While > the Python runtime and native libraries are shared among processes, the heap > is not. Even if you fork processes it will not work because reference > counting will dirty the shared pages and my instinct tells me that just > loading a Python source file will reference almost all the shared pages > because they are mostly used to store already loaded modules. What I finally > did not do is to actually check the hypothesis that most of the heap is > filled by strings which are the identifiers in the loaded Python modules. My > goal was to somehow make identifiers constants and just readonly-mmap them > into the process (replace the pathetic .pyc loading mechanism). Note that my > plan was just a little subset of the .jar -> .dex converter. > > Second, Python has a special memory manager which works with buckets so > there are separate heaps for different object sizes. Somehow this special > memory manager is turned off in release mode and it uses the standard C heap > for some reason. Either it is a bug or just it turned out to be faster (more > work was put into glibc) I do not know, but handling the linked free list > can dirty shared pages as well or I am just mistaken... > > Third, the fact that Python is a dynamic language does not help any > prefetching or memory sharing. I am not too convicted either that this > dynamic nature is used at all in the Sugar codebase but you know I cannot > program in Python and frankly I do not feel the need to learn another > language. Just now, at my age of 34, I finally gave up and learned LISP > (more specifically clojure) and I hope that it will be the last programming > language I will have to learn (other than assembly languages of course)... > :) Now this point is interesting because if you thought that the Dalvik VM > could run the Sugar codebase via Jython then it just will not work. The > Dalvik VM just cannot load .jar files and Jython just generates them on the > fly because of the dynamic nature of Python. > > Fourth, moving Python (theoretically) to a GC environment (instead of > reference counting) also would not work if the GC is compacting because it > would also dirty the shared pages. So a mark and sweep nonmoving GC with > separately stored "visited" bits is the only feasible solution. Now guess > what the Dalvik VM does? > For more info (45 min + questions): > http://www.youtube.com/watch?v=ptjedOZEXPM > > So *my* conclusion is that solving this sharing problem is *very* hard. I am > not sure that simply translating all activities from Python to Java would > take more time. > > Another interesting thing is that meantime the unladen-swallow project > progresses (just more slowly than planned). Their plan is to make it the > default Python runtime so if it will happen (I cannot comment on that) then > the Python VM will use even more memory (though it will be faster) so Sugar > will be even less interesting on the myriad of low spec cheap ARM tablets > which will flood the market soon. > > I think that is all I can say about this subject so I just finish it here.
The PyPy project has a fully-featured python 2.5 interpreter that has much lower memory usage and a proper GC (so less dirty pages). They also have an x86 JIT which makes it much faster than CPython, at the cost of a bit of memory (still less than CPython). The only issue right now is extension support: ctypes is fully supported, but C/Python extension support is not complete by far. As for Jython on Android, Jython has a Java bytecode JIT. It should be entirely possible to write a dalvik backend to this JIT. So not only would rewriting everything to Java be a huge step backwards, but it would also be more work. _______________________________________________ Devel mailing list [email protected] http://lists.laptop.org/listinfo/devel
