houqp commented on pull request #68: URL: https://github.com/apache/arrow-datafusion/pull/68#issuecomment-1007877042
Apologize for the late reply, still catching up to all the goodness happened during the break. I should have communicated this better to save @Igosuki some work :( But really impressive result, thank you for doing the benchmark! I have squashed our latest round of merge fixes into a single commit to keep things clean and pushed it to this PR. What is missing right now from my point of view is migrating the python crate and avro. @Igosuki do you mind sending your avro changes to my branch at https://github.com/houqp/arrow-datafusion/tree/arrow2_merge so I can merge it into this PR? After that I think this PR is ready for a formal review with another round of quick master merge to pick up the removal of the python binding. @alamb to answer your question, I think the arrow2 branch is ready to be tested more broadly now. I feel like we should merge this either to master or to an official arrow2 branch soon without waiting for arrow2 1.x release, which is probably going to take a while. The API in arrow2 is still evolving fast right now (in a good direction), so I don't see it getting stablized any time soon. For example, it just recently removed the RecoardBatch abstraction entirely, which also aligns with our goal to come up with our own recordbatch abstraction in datafusion: https://github.com/apache/arrow-datafusion/issues/1248 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@arrow.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
