alamb commented on issue #19: URL: https://github.com/apache/arrow-datafusion-comet/issues/19#issuecomment-1953612647
Another perhaps interesting read from the literatire is: "Everything You Always Wanted to Know About Compiled and Vectorized Queries But Were Afraid to Ask" - https://www.vldb.org/pvldb/vol11/p2209-kersten.pdf The findings of the paper is that Vectorized evaluation (aka what we use in DataFusion) and compiled evaluation offer similar performance for most analytic style workloads (though each is better at certain cases than the otehr) The reason I think JIT / compiled engines are less common than vectorized engines is the software engineering challenges -- that they are often harder to maintain and debug when something goes wrong compared to "normal" vectoized code. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
