milenkovicm commented on issue #19: URL: https://github.com/apache/arrow-datafusion-comet/issues/19#issuecomment-1953823651
> Another perhaps interesting read from the literature is: > > "Everything You Always Wanted to Know About Compiled and Vectorized Queries But Were Afraid to Ask" - https://www.vldb.org/pvldb/vol11/p2209-kersten.pdf > > The findings of the paper is that Vectorized evaluation (aka what we use in DataFusion) and compiled evaluation offer similar performance for most analytic style workloads (though each is better at certain cases than the otehr) > > The reason I think JIT / compiled engines are less common than vectorized engines is the software engineering challenges -- that they are often harder to maintain and debug when something goes wrong compared to "normal" vectoized code. I believe authors of [_"Photon: A Fast Query Engine for Lakehouse Systems"_](https://people.eecs.berkeley.edu/~matei/papers/2022/sigmod_photon.pdf) came with similar conclusion. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
