kazuyukitanimura commented on PR #1055: URL: https://github.com/apache/datafusion-comet/pull/1055#issuecomment-2477138947
Thanks @viirya > It looks like less than 5%? > 27979 - 27281 = 698 ~= 2% > 24877 - 23534 = 1343 ~= 5% I used `Ave Time` (5 iterations) when I mentioned `10% ish` 100G q39a `(14413-13770)/14413 ~= 4.5%` 1T q39a `(33349-28714)/33349 ~= 14%` The performance gain can easily swing between 2%-15% due to the noise > So I believe that the diff is insignificant for longer query time. The performance gain is proportional to the input size by design because it saves the overhead for all input arrow vectors. If you mean `longer query time` by deeper query plan, then yes, the performance gain gets less effective as the benefit is one-time per data read and query procesing time takes more portions in the entire runtime. But there are many use cases I think for shallow query plans as well and we can still save $$$? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
