sbp commented on issue #111: URL: https://github.com/apache/tooling-trusted-release/issues/111#issuecomment-2910211171
The majority of the performance issue is [due to the asynchronous use of Jinja2](https://jinja.palletsprojects.com/en/stable/api/#async-support): > The compiled code uses await for functions and attributes, and uses `async for` loops. In order to support using both async and sync functions in this context, a small wrapper is placed around all calls and access, which adds overhead compared to purely async code. Using wrapped synchronous Jinja2 instead, i.e. a synchronous call in `asyncio.to_thread`, takes `/projects` from 110ms to 55ms. This is implemented in 2633c1c2348a0d5b91f8d05e57b3fde9a9aa827a. The speedup is more or less the same if using unwrapped, fully synchronous Jinja2. I also investigated whether database performance could be improved by using [apsw](https://github.com/rogerbinns/apsw), a package that glues together the C APIs of Python3 and Sqlite3. Presumably this has near minimal overhead, compared to SQLAlchemy. For the example query that I rewrote, performance was improved by 1.16x, which was about 2ms, over an average of 200 runs. Given the enormous interface benefits of using SQLAlchemy, it does not appear to be worth using apsw for its performance gains. We may be able to make performance gains for specific queries by tuning our indices, as we are yet to analyse our queries and adjust the indices for performance. Reusing sessions and caching results will also give some small gains. The performance gains in the aforementioned commit are, however, likely to be the most economical that we can make, and their basis probably accounts for the specific performance characteristics reported in this issue. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: dev-unsubscr...@tooling.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: dev-unsubscr...@tooling.apache.org For additional commands, e-mail: dev-h...@tooling.apache.org
