Aggarwal-Raghav commented on PR #5257: URL: https://github.com/apache/hive/pull/5257#issuecomment-2164640119
@zhangbutao, 1 question here, when I try to insert the data using tez, the query fails with NoAuthWithAWSException on staging dir and when I debugged it, I found that the conf is not passed to tez (on yarn) container because of https://github.com/apache/hive/blob/33cadc5b498b57f779cddc9bf4e3f8aef2d9f6dc/ql/src/java/org/apache/hadoop/hive/ql/exec/tez/DagUtils.java#L1478 As fs.s3a.secret.key and fs.s3a.access.key exist in default hidden list. I have tried it on hive3 as I don't have a setup of hive4 with S3. **_I understand that because of patch differences, issue might not occur with hive4 but in hive4 also, this DAGUtils is removing those s3 credentials from jobconf (tested on local setup with hdfs based table)_**. If you have any insights, can you help me understand if I am missing anything? The same happens in select query as well. In hive3 I have tested by removing the fs.s3a* configs from hidden list, then the insert/select query is passing. Also, do we want to add metaconf:fs.s3a.* configs to hidden list? As in the above attached screenshot, I am able to see the credential value. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: gitbox-unsubscr...@hive.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: gitbox-unsubscr...@hive.apache.org For additional commands, e-mail: gitbox-h...@hive.apache.org
