amirgon commented on PR #43297: URL: https://github.com/apache/arrow/pull/43297#issuecomment-2235832712
We currently utilize the UCX ArrowFlight transport, which provides significant benefits, particularly in reducing CPU utilization. I understand that with Arrow 18, we'll need to implement this transport ourselves to maintain our current functionality. However, I have several concerns and questions regarding this decision: 1. **Rationale for Removal**: It's unclear why the UCX ArrowFlight transport is being removed and discouraged. As this feature was distributed as part of Arrow, we didn't anticipate it being discontinued. 2. **Recommended Alternative**: If Disassociated IPC is now the recommended approach, why isn't the UCX ArrowFlight transport being reimplemented using Disassociated IPC and distributed as part of Arrow? 3. **User Abstraction**: From a user perspective, the ability to choose a transport by simply prefixing the URL (e.g., `ucx:` or `grpc+tcp:`) provides a convenient abstraction. What's the motivation behind removing this abstraction and requiring users to implement protocol details themselves? 4. **Impact on Existing Users**: Given that this feature is already in use and providing benefits, have you considered the impact on existing users who rely on this functionality? 5. **Migration Path**: Could you provide guidance or a migration path for users who need to transition away from the UCX ArrowFlight transport? We would appreciate more clarity on these points and any considerations for maintaining similar functionality within the Arrow project. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@arrow.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
