Hi Julian，
Great Job!
Looking forward to your later work.
BTW, happy new year for everybody！
Thanks!
Chao Wang
BONC ltd
[email protected]
On 1/1/2022 06:44，Julian Feinauer<[email protected]> wrote：
Hi all,
after a short discussion I had today with Jialin I took some minutes to see if
/ how code generation could help us to impove the performance on Queries.
The first thing I looked at was the Filterin in the SeriesReader.
Currently many Filters have if conditions / branches in their `satisfy` method.
As this method is called on every datapoint (that is not filtered out on
another level) there is lot of potential for improvement.
I pushed a branch with my code changes (which are not that much, actually), see
[1] for all technical details.
What did i do?
Basically I changed the existing filters to also return an AST of their "filter
operation". So I can visit all filters and dsynamically generate the Java Code
for the "perfect" Filter in that situation.
This is then dynamically compiled in a Java class and live loaded.
Of course this process takes some time (about 100 ms or a bit below, I think)
but then improves the speed on every data point.
What are my results so far?
I generated an example file with about 50 Mio. datapoints and three different
types of filters (simple, standard, complex).
And so far, my results show that the performance can be improved to up to 20%!!!
For simple queries its not beneficial and can make it even a bit slower (see
overhead above).
For standard queries I saw about 5% improvement in query times.
For complex queries i saw between 10% and 20% improvement in query time.
I will do some more benchmarks the next days and would love tow ork together
with other devs from the community to get this feature in the next release!
Best!
Julian
PS.: Happy new year for everybody
[1] Branch: https://github.com/apache/iotdb/tree/experimental/code-generation
[https://opengraph.githubassets.com/4a5319a07e99ead0a4adb97783f0ba1682e78fa6c074c1a9204c2ea82a2a347b/apache/iotdb]<https://github.com/apache/iotdb/tree/experimental/code-generation>
GitHub - apache/iotdb at
experimental/code-generation<https://github.com/apache/iotdb/tree/experimental/code-generation>
Apache IoTDB. Contribute to apache/iotdb development by creating an account on
GitHub.
github.com
