I wonder how nuitka might do? http://nuitka.net/pages/overview.html
m On Wed, Feb 13, 2019 at 10:58:32AM -0500, Joseph Reagle wrote: > On 2/13/19 10:42 AM, Ren� Dudfield wrote: > > you can run it as a daemon/server(for example a little flask app). > > This optimization also works for cpython apps if you want to avoid > > the startup/import time. > > That would be a big change for a uncertain improvement, so I'm not willing to > go there yet. Performance is okay, but I want to see if I can improve it > further as a stand-alone. > > > Can the work be split up per xml file easily? Then perhaps > > multiprocessing will work nicely for you. > > > > Do you need to process all the files each time? Or can you avoid > > work? > > I've tried multiprocessing too, but it is slower. Parsing the XML can be done > in parallel but I suspect the overhead of multi-processing was the drag. I've > also thought about caching the XML parse trees, but serializing and reloading > pickles of unchanged parse trees seems slower than just parsing the XML anew. > Using lru_cache on text processing functions (e.g., removing accents) didn't > help either. > > I haven't been able to find good examples of people using multiprocessing or > pypy for XML processing, perhaps this is why. > > Thank you all for the suggestions! > > > > _______________________________________________ > pypy-dev mailing list > pypy-dev@python.org > https://mail.python.org/mailman/listinfo/pypy-dev -- Matt Billenstein m...@vazor.com http://www.vazor.com/ _______________________________________________ pypy-dev mailing list pypy-dev@python.org https://mail.python.org/mailman/listinfo/pypy-dev
