alamb opened a new issue, #13265: URL: https://github.com/apache/datafusion/issues/13265
## Introduction This ticket is a weekly summary of interesting things happening in DataFusion. Note this is not a complete list (it is what I remember / can find). Please feel free to leave comments on this ticket about things that I may have missed or you think should get wider attention by the community Loosely inspired by https://this-week-in-rust.org/ ## DataFusion Related Blogs - Not sure ## Ucoming Releases - [ ] https://github.com/apache/datafusion/issues/12470 (thanks @andygrove) - [ ] https://github.com/apache/datafusion-sqlparser-rs/issues/1423 (huge kudos to @iffyio for all the reviews) ## Recent Releases - [x] https://github.com/apache/datafusion/issues/13166 ## Highlights from last week(s): (I am sorry if I missed you -- please add a note to this ticket with anything you would like to highlight) ### FFI Bindings - @timsaucer added https://github.com/apache/datafusion/pull/12920 (FFI --> stable ABI for Table Providers) and a killer new example https://github.com/apache/datafusion/pull/13183 ### LogicalTypes are coming! - @notfilippo and @findepi have merged the first phase of logical types: https://github.com/apache/datafusion/pull/12853 ### Performance Highlights - @jayzhan @Dandandan @berkaysynnada and @2010YOUY01 improved repartition performance on multicpre https://github.com/apache/datafusion/pull/13133 - @Rachelint @jayzhan211 @2010YOUY01 and @Dandandan found another 10% performance improvement in many multi-column aggregate queries: https://github.com/apache/datafusion/pull/12996 - https://github.com/apache/datafusion/pull/13101 (finally!) ### Others - @goldmedal started using the new documentation API: https://github.com/apache/datafusion/pull/13255 - The work for hardening substrait continues with @akoshchiy, @vbarua, @Blizzara, @LatrecheYasser, @bvolpato, authoring [several](https://github.com/apache/datafusion/pull/13112) [PRs](https://github.com/apache/datafusion/pull/13114) [more](https://github.com/apache/datafusion/pull/12112) [more](https://github.com/apache/datafusion/pull/13127) [more ](https://github.com/apache/datafusion/pull/13173) - @Omega359 and @jonathanc-n almost wrapped up the new function documentation work: https://github.com/apache/datafusion/pull/13161 - @findepi has been on a tear cleaning up with [PR](https://github.com/apache/datafusion/pull/13185) after [PR](https://github.com/apache/datafusion/pull/13174) after [PR](https://github.com/apache/datafusion/pull/13245) - @jonahgao is nearing the final stages of support for the `EXECUTE` statement https://github.com/apache/datafusion/pull/13194 - @ngli-me started fixing a long standing rough edge with sort computations: https://github.com/apache/datafusion/pull/13146 - @eejbyfeldt continues bashing away at bugs / things that prevent complete TPC-DS run such as [this](https://github.com/apache/datafusion/pull/13184) and [this](https://github.com/apache/datafusion/pull/13248) - @LeslieKid added additional aggregate fuzzing test support https://github.com/apache/datafusion/pull/13226 - Thanks to @mnorfolk03 https://github.com/apache/datafusion/pull/13214 ## Major Projects / Discussions under way - https://github.com/apache/datafusion/issues/12821 -- show the world what you can do with focused engineering effort. Thanks to the epic work of @Rachelint, @goldmedal, @jayzhan211, @Dandandan @XiangpengHao and others, - https://github.com/apache/arrow-rs/issues/5523 - @XiangpengHao and @tustvold are working to make parquet *even better* - https://github.com/apache/datafusion/issues/12357 - Helping make DataFusion more visible: https://github.com/apache/datafusion/discussions/13049 @SamSynnada ## Looking to get more involved? Try code review! DataFusion has a long history of community members [contributing in all aspects of the project](https://datafusion.apache.org/contributor-guide/index.html). Reviewing PRs is an especially great way to get introduced to the project, help the community and grow your own knowledge -- researching and understanding the code enough to review PRs also often inspires additional ideas for improvements. We have [docs about reviews](https://datafusion.apache.org/contributor-guide/index.html#reviewing-pull-requests). TLDR is: look for test coverage, if the change is understandable and well documented, and if the code can be improved. When you think the PR looks good to merge, try `@` mentioning [one of the committers](https://projects.apache.org/committee.html?datafusion). ## Help wanted Please feel leave your own comments on this ticket if you are looking for help ## Community * [Weekly Call](https://docs.google.com/document/d/1NBpkIAuU7O9h8Br5CbFksDhX-L9TyO9wmGLPMe0Plc8/edit#heading=h.kpjkpncdmt1g) * Slack/Discord: [info links](https://datafusion.apache.org/contributor-guide/communication.html#slack-and-discord) ## Upcoming meetups: * [2024 Dec 18 Chicago](https://github.com/apache/datafusion/discussions/12894): https://lu.ma/eq5myc5i @adriangb @timsaucer * TBD: https://github.com/apache/datafusion/discussions/12988 * [2025 Jan 15 Boston](https://github.com/apache/datafusion/discussions/13165) ## Background: Previous update: - https://github.com/apache/datafusion/issues/13167 ## Andrew's Focus Areas: - https://github.com/apache/datafusion/issues/12821 (thanks to the epic work of @Rachelint, @goldmedal, @jayzhan211, @Dandandan @XiangpengHao and others, we are quite close) - https://github.com/apache/datafusion/issues/8709 (very close to finishing thanks @jcsherin @jatin510) - https://github.com/apache/datafusion/issues/12740 (also almost done thanks to @Omega359 and @jonathanc-n) - https://github.com/apache/datafusion/issues/12114 (thanks @LeslieKid for all your help so far) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@datafusion.apache.org.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: github-unsubscr...@datafusion.apache.org For additional commands, e-mail: github-h...@datafusion.apache.org
