jarredhj0214 commented on code in PR #12121:
URL: https://github.com/apache/gravitino/pull/12121#discussion_r3642380719
##########
flink-connector/flink-common/src/main/java/org/apache/gravitino/flink/connector/paimon/GravitinoPaimonCatalog.java:
##########
@@ -139,6 +139,8 @@ public void open() throws CatalogException {
HadoopUtils.getHadoopConfiguration(Options.fromMap(withoutHadoopOptions(paimonOptions)));
moveHadoopOptionsToConf(paimonOptions, hadoopConf);
try {
+ // Keep vended credentials as Paimon options because Paimon native
S3/OSS FileIO loaders
+ // declare them as required options.
Review Comment:
> If the credentials are added back here, then this issue isn't actually
resolved, right?
You're right. I reworked this so storage credentials are not added back to
paimonOptions.
Now Hadoop-style options are moved into Hadoop Configuration, Paimon native
S3 options are translated to Hadoop S3A configuration and removed from
paimonOptions, and vended S3/OSS credentials are also written to Hadoop
Configuration. Only JDBC backend credentials remain in paimonOptions.
I also added/updated tests to verify that storage credentials are absent
from paimonOptions and still available from Hadoop Configuration. This should
address the logging leak without dropping the credentials from the filesystem
access path.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
