On 11 Aug 2010, at 01:56, Lucian Branescu <lucian.brane...@gmail.com> wrote:
> 2010/8/10 NoiseEHC <noise...@freemail.hu>: >> >>> We used to do that, the problem is that we don't control our platform >>> as Google controls Android and you need to make sure that resources >>> that need to be specific of each child process aren't shared (dbus and >>> X connections, etc). >>> >>> I'm personally more interested in reducing the amount of resources we >>> need to start activities (which is quite insane right now), but >>> sharing more of those resources sounds like a good idea to me. >>> >> >> Since I spent quite a bit of time analyzing the Python runtime here is my >> conclusion, maybe it will make wasting all that time less painful. >> >> First, most of the memory consumed by an activity is the process heap. While >> the Python runtime and native libraries are shared among processes, the heap >> is not. Even if you fork processes it will not work because reference >> counting will dirty the shared pages and my instinct tells me that just >> loading a Python source file will reference almost all the shared pages >> because they are mostly used to store already loaded modules. What I finally >> did not do is to actually check the hypothesis that most of the heap is >> filled by strings which are the identifiers in the loaded Python modules. My >> goal was to somehow make identifiers constants and just readonly-mmap them >> into the process (replace the pathetic .pyc loading mechanism). Note that my >> plan was just a little subset of the .jar -> .dex converter. >> >> Second, Python has a special memory manager which works with buckets so >> there are separate heaps for different object sizes. Somehow this special >> memory manager is turned off in release mode and it uses the standard C heap >> for some reason. Either it is a bug or just it turned out to be faster (more >> work was put into glibc) I do not know, but handling the linked free list >> can dirty shared pages as well or I am just mistaken... >> >> Third, the fact that Python is a dynamic language does not help any >> prefetching or memory sharing. I am not too convicted either that this >> dynamic nature is used at all in the Sugar codebase but you know I cannot >> program in Python and frankly I do not feel the need to learn another >> language. Just now, at my age of 34, I finally gave up and learned LISP >> (more specifically clojure) and I hope that it will be the last programming >> language I will have to learn (other than assembly languages of course)... >> :) Now this point is interesting because if you thought that the Dalvik VM >> could run the Sugar codebase via Jython then it just will not work. The >> Dalvik VM just cannot load .jar files and Jython just generates them on the >> fly because of the dynamic nature of Python. >> >> Fourth, moving Python (theoretically) to a GC environment (instead of >> reference counting) also would not work if the GC is compacting because it >> would also dirty the shared pages. So a mark and sweep nonmoving GC with >> separately stored "visited" bits is the only feasible solution. Now guess >> what the Dalvik VM does? >> For more info (45 min + questions): >> http://www.youtube.com/watch?v=ptjedOZEXPM >> >> So *my* conclusion is that solving this sharing problem is *very* hard. I am >> not sure that simply translating all activities from Python to Java would >> take more time. >> >> Another interesting thing is that meantime the unladen-swallow project >> progresses (just more slowly than planned). Their plan is to make it the >> default Python runtime so if it will happen (I cannot comment on that) then >> the Python VM will use even more memory (though it will be faster) so Sugar >> will be even less interesting on the myriad of low spec cheap ARM tablets >> which will flood the market soon. >> >> I think that is all I can say about this subject so I just finish it here. > > The PyPy project has a fully-featured python 2.5 interpreter that has > much lower memory usage and a proper GC (so less dirty pages). They > also have an x86 JIT which makes it much faster than CPython, at the > cost of a bit of memory (still less than CPython). The only issue > right now is extension support: ctypes is fully supported, but > C/Python extension support is not complete by far. FWIW I experimented with PyPy a month or so back to see how fast my self organising map code would run. Needed to make a some code/module changes to get it to work, but it ran about twice as fast. I didn't check if memory usage had improved. --Gary > As for Jython on Android, Jython has a Java bytecode JIT. It should be > entirely possible to write a dalvik backend to this JIT. > > So not only would rewriting everything to Java be a huge step > backwards, but it would also be more work. > _______________________________________________ > Sugar-devel mailing list > Sugar-devel@lists.sugarlabs.org > http://lists.sugarlabs.org/listinfo/sugar-devel _______________________________________________ Sugar-devel mailing list Sugar-devel@lists.sugarlabs.org http://lists.sugarlabs.org/listinfo/sugar-devel