alamb commented on PR #7250: URL: https://github.com/apache/arrow-datafusion/pull/7250#issuecomment-1693709988
An update here: 1. I think this code is now ready to propose adding to DataFusion and I plan to polish it up, write more tests / docs , and get a real PR up over the next few weeks Some schedule items: 1. I am out-ish for the next 1.5 weeks so I will likely not make much progress on this project until I return in September 2. For this implementation, avoiding memory explosions with DictionaryArrays (as we do in IOx) requires https://github.com/apache/arrow-rs/issues/506 (which @tustvold is working on, so I expect that will be ready in arrow `47.0.0`, in about 2 weeks time If anyone else needs this feature faster you are welcome to turn this PR into a real PR to DataFusion -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
