waynexia commented on PR #2648: URL: https://github.com/apache/arrow-datafusion/pull/2648#issuecomment-1147585555
Thank you all for the reviews :heart: >datafusion-contrib is probably the right place to experiment with plan-level JIT How should I get started in `datafusion-contrib`, I think I should ask someone permitted to create a repo? And should we move the entire `jit` sub-crate or just copy it. The row format will still need the JIT utils. >Something like https://github.com/apache/arrow-datafusion/issues/2427 might be a good starting point. Looks cool. I'll try to get involved. And I really like @alamb's reason 1! >The idea of JIT'ing multiple plan nodes together is a necessary part of this for sure, but one of the last ones. The first step is fully JIT'ing the expr evaluation. This makes sense. I shall revisit the task list. That paper also gives some interesting data, like compilation sometimes will miss more branches. And besides the conclusion, it shows (me) a concrete way to measure cost and improvement. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
