rangareddy commented on issue #11851: URL: https://github.com/apache/hudi/issues/11851#issuecomment-4989192934
Thanks for the report, @alberttwong. Here's the resolution. **Root cause:** the `The AWS SDK for Java 1.x entered maintenance mode…` line is a **deprecation notice, not an error** — the sync completes normally. In this run it's emitted by the S3A connector (`hadoop-aws`) you're using for `s3a://` (Hadoop 2.10.2 bundles AWS SDK v1: the stack trace shows `com.amazonaws.auth.InstanceProfileCredentialsProvider`). Hudi itself is already on AWS SDK v2 for the sync path, so this particular warning comes from Hadoop's dependency rather than from hudi-sync. Solution — silence the benign warning: ```sh export AWS_JAVA_V1_DISABLE_DEPRECATION_ANNOUNCEMENT=true # or pass -Daws.java.v1.disableDeprecationAnnouncement=true to the JVM ``` Re-run run_sync_tool.sh and the message will be gone. **Two fixes we're making to** run_sync_tool.sh so this is cleaner out of the box: 1. Add `-Daws.java.v1.disableDeprecationAnnouncement=true` to the launched JVM, so the notice is suppressed by default. 2. Fix the `ls: cannot access '.../packaging/hudi-hive-sync-bundle/target/hudi-hive-sync-*.jar'` error you also hit — the script assumed a source-build layout. It will now honor a HUDI_HIVE_UBER_JAR override, fall back to the bundle jar next to the script (packaged//opt/hudi installs), and fail with a clear message if it can't find it. Note on the underlying SDK: fully removing AWS SDK v1 from your S3 access path is a **Hadoop** upgrade — Hadoop 3.4.x's S3A moved to AWS SDK v2. Upgrading your Hadoop/hadoop-aws to 3.4+ removes the v1 usage at the source. (Hudi still has v1 only in the DynamoDB lock provider and Kinesis source, neither of which is used by hive-sync.) Closing as resolved with the workaround + script fix above. **Please feel free to reopen or comment if the warning persists after setting the flag, or if you run into anything else** — happy to help. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
