elukey commented on issue #5659: [AIRFLOW-5033] Switched to snakebite-py3 URL: https://github.com/apache/airflow/pull/5659#issuecomment-614054132 @wittfabian I am curious about what problems have you encountered with snakebite (to understand if they are big ones or small that we can solve). It is true that I didn't have the time to add support for RPC encryption, but the task seems to be big and needs time/energy (any help is needed). More info on the current blockers are listed in https://github.com/internetarchive/snakebite-py3/issues/8 Switching to PyArrow was on the table even when I started, but it was abandoned for various reasons (might not hold anymore). The main problem of snakebite is that it doesn't rely on java libs (as pyarrow does IIRC) and Hadoop still sadly uses old ciphers for SASL. Anyway, happy if Airflow proceeds in any direction, or even more happy to collaborate with other people on snakebite-py3. Adding RPC encryption support by myself is not doable in a short amount of time.
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services
