OK, now I can confirm the modified version of generate.py on my project run
using pypy without error.
Currently (still with the probable numerous call to the python interpreter) :
with python : [generate-build: 1:58.310 sec]
with pypy : [generate-build: 18:28.473 sec]
Resolving dependencies is the step that look like to take most of the time.
Anyway, it is definitively working using pypy.
On 26 août 2010, at 15:58, Jean-Baptiste BRIAUD -- Novlog wrote:
> Hi Andreas,
>
> On 26 août 2010, at 15:31, Andreas Ecker wrote:
>
>> Hi JBB,
>>
>> so for pypy you had to download 2.5 GB of code and it took you more than
>> 1 hour to compile it on a recent i7 machine!? Are you still complaining
>> about the qooxdoo generator speed, then?
>>
>
> Yes, there is worst situation than qooxdoo build, I was really shoked : more
> than 1 hour :-)
> Hopefully, I won't need recompile pypy as often as I launch generate build.
>
>> Just kidding ;-). Give it another try with using the "verbose" option,
>> i.e. call "pypy generate.py -v build". That should give you a more
>> detailed error message. Would be great you kept trying with the help of
>> the pypy people.
>>
>
> I did, finally a pypy guy asked my to remove the reload(sys) line.
> It failed to remove only that line so I finally removed the following block :
> reload(sys)
> sys.setdefaultencoding('utf-8')
> sys.setrecursionlimit(1500)
>
> now it start working ... but it take longer than without pypy !!!
>
> Finally it had failed on this error :
> 'ascii' codec can't encode character u'\u2019' in position 42: ordinal not in
> range(128)
>
> pypy acknowledge a bug for the reload(sys)
> https://codespeak.net/issue/pypy-dev/issue557
>
> I'll try this "hack" suggested by arigato that was on the pypy IRC (#pypy on
> freenode).
> He is really helpful.
>
> arigato:try to preserve sys.stdout around it, i.e.
> arigato:a = sys.stdout; reload(sys); sys.stdout = a
> arigato:it may help (but of course it's still a pypy bug to solve)
>
>> Since the hello world now seems to work fine with pypy (which to my
>> knowledge previously and without the very recent fixes to pypy never
>> worked), maybe the problems with your project are also due to the
>> qooxdoo default encoding UTF-8 (rather than ascii), particularly if you
>> have accented chars for French translations in your files. Just a
>> guess...
>>
>
> good guess, there is that block that failed in pypy but removing it make it
> failing because of some non ascii characters ...
> No idea what to do.
>
>
> Anyway, we already know that "as is" it will take longer than without pypy.
> Apparently it might be due to the numerous call to python itself.
> I had to replace "everywhere"
>
> CMD_PYTHON = 'python'
> by
> CMD_PYTHON = sys.executable
>
> The problem is not just to replace it but to understand why python
> interpreter is called apparently so often.
> I have to warn I don't speak py. It may not continue to be true ;-)
> I'm not sure if python is really called from generator.py except that
> CMD_PYTHON = 'python' look like it is called.
> The fact is that pypy rely on a JIT and relaunching it loose all the JIT
> benefit.
> Apparently, as a general purpose rule in py, it is not recommended to call
> python from python to execute a file.
> There is ways that I don't know to call python file without calling the
> interpreter each time.
> So, even without pypy context it might be a good idea to avoid launching
> python process between 2 py files.
>
> Hope this will help improving qx !
>
>> Please note that Thomas has been on vacation for the last two weeks and
>> is to return not before Monday.
>>
>> Bye,
>>
>> Andreas
>>
>>
>> On Thu, 2010-08-26 at 14:45 +0200, Jean-Baptiste BRIAUD -- Novlog wrote:
>>> Finally, I didn't gave up ... :-)
>>>
>>> I manage to get a proper C dev env in my Mac (by simply downloading 2.5 Go
>>> of xcode).
>>> Yes, 2.5 Go !
>>>
>>> Anyway, I was then able to recompile pypy from the trunk.
>>> I try to run generate.py using pypy.
>>> It had failed.
>>> I have no idea what was wring in the pypy code but the pypy guys downloaded
>>> qx sdk to have the the helloword application.
>>> They correct what had failed in pypy and I recompiled it (1hour each time,
>>> yes 1 hours of gcc ...)
>>>
>>> I created a qooxdoo hello world application and manage to launch a generate
>>> build using pypy rather than python.
>>>
>>> I then try to run the same way on my project.
>>> I apply the CMD_PYTHON = sys.executable modification on all my local
>>> generate.py and use pypy rather than python.
>>> It failed on a rather cryptic C error : [Errno 9] Bad file descriptor:
>>> '<fdopen>'
>>> There must be some new python code section called in the generate.py that
>>> is called for my project that is not called for the simple helloworld.
>>> Now, I give up cause I don't speak python and I used to speak C long time
>>> ago ...
>>>
>>> I think this experiment should encourage to try the pypy way to speedup all
>>> the python tool chain.
>>> Apparently this look like doable but I don't have the knowledge to help
>>> more and reach the goal : pypy generate.py build.
>>>
>>>
>>> On 25 août 2010, at 13:49, Jean-Baptiste BRIAUD -- Novlog wrote:
>>>
>>>> OK, pypy look like a very good thing to try.
>>>>
>>>> I had to replace
>>>> CMD_PYTHON = 'python'
>>>> by
>>>> CMD_PYTHON = sys.executable
>>>>
>>>> in the generate.py of the application folder to ensure pypy launch the
>>>> "real" generator.
>>>> Unfortunately, generate.py doesn't work with pypy.
>>>> It blow up in error.
>>>> I had an very interesting IRC session with the pypy guys.
>>>> I end up trying to build pypy from trunk.
>>>> There is version for several OS prebuilt but they asked me to try the
>>>> trunk because they already know that the latest prebuilt version will not
>>>> work with generate.py they kindly inspected.
>>>>
>>>> Anyway, after hours trying to use gcc on my Mac I gave up.
>>>>
>>>> conclusions :
>>>> 1. pypy is a real way to improve Python runtime but generate.py apparently
>>>> would need some hacks. Qooxdoo can you do something here ?
>>>> 2. The only thing I can think about now, is the precompiled version of
>>>> qooxdoo but I still have that merge question, see the other thread.
>>>>
>>>>
>>>> On 24 août 2010, at 23:35, <[email protected]> <[email protected]> wrote:
>>>>
>>>>> Hi.
>>>>>
>>>>> Maybe http://codespeak.net/pypy/dist/pypy/doc/ is a more viable direction
>>>>> of research?
>>>>>
>>>>> As far as I could understand psyco is not developed anymore because the
>>>>> effort goes now into developing pypy - an optimized python interpreter,
>>>>> which does the optimizations psyco did on all interpreted code by default.
>>>>>
>>>>> br,
>>>>>
>>>>> flj
>>>>>
>>>>>
>>
>
> ------------------------------------------------------------------------------
> Sell apps to millions through the Intel(R) Atom(Tm) Developer Program
> Be part of this innovative community and reach millions of netbook users
> worldwide. Take advantage of special opportunities to increase revenue and
> speed time-to-market. Join now, and jumpstart your future.
> http://p.sf.net/sfu/intel-atom-d2d_______________________________________________
> qooxdoo-devel mailing list
> [email protected]
> https://lists.sourceforge.net/lists/listinfo/qooxdoo-devel
------------------------------------------------------------------------------
Sell apps to millions through the Intel(R) Atom(Tm) Developer Program
Be part of this innovative community and reach millions of netbook users
worldwide. Take advantage of special opportunities to increase revenue and
speed time-to-market. Join now, and jumpstart your future.
http://p.sf.net/sfu/intel-atom-d2d
_______________________________________________
qooxdoo-devel mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/qooxdoo-devel
