dwsmith1983 commented on PR #5365: URL: https://github.com/apache/datafusion-comet/pull/5365#issuecomment-5419684276
On the long bucket credential precedence: You're right, long wins. S3AUtils.getPassword keeps an already non-empty value, so the long form resolves first and the short form only fills in when it's unset. Flipped the comparison to long, short, global and added your exact conflicting fixture as a regression (distinct long and short pairs now decline). Two related hardenings went in alongside: the gate now declines when fs.s3a.aws.credentials.provider resolves to a class the native reader doesn't implement, so those scans fall back instead of failing mid query, and for non-credential options (endpoint etc.) we verified the long form is inert in Hadoop (propagateBucketOptions folds it into an unread key), so no gate is needed there; that's documented with a control test. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
