gtwuser commented on issue #5701: URL: https://github.com/apache/hudi/issues/5701#issuecomment-1141217578
Thanks for getting back @xushiyan AWS glue supports Spark 3.1, but i suppose with `Hudi 0.11.0` bundle we get the parquet upgraded to `1.12`, unfortunately we are not able to upgrade to the latest version of Hudi. Issue #5636 and i havent yet raised the aws ticket. Also wondering how is it working for others if it is the case. From aws console: >Glue 3.0 – Supports Spark 3.1.1 and Python 3.7. Also includes new AWS Glue Spark runtime optimizations for performance and reliability. Upgraded several dependencies that were required for the new Spark version. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
