diqiu50 commented on code in PR #14352:
URL: https://github.com/apache/gravitino/pull/14352#discussion_r4409823258
##########
flink-connector/flink-common/src/main/java/org/apache/gravitino/flink/connector/paimon/GravitinoPaimonCatalog.java:
##########
@@ -197,6 +199,24 @@ private static void movePaimonStorageOptionsToConf(
paimonOptions, hadoopConf, PaimonConstants.OSS_SECRET_KEY,
PaimonConstants.OSS_SECRET_KEY);
}
+ /**
+ * Paimon's OSSFileIO (paimon-oss 1.x) builds its Hadoop configuration only
from the catalog
+ * options and never reads the Hadoop configuration, so after the move above
the OSS endpoint and
+ * keys would be invisible to it ("Aliyun OSS endpoint should not be null or
empty"). Hand those
+ * entries back to the inner catalog's options; they stay out of this
connector's own option map
+ * and other filesystems keep using the Hadoop configuration.
+ */
+ @VisibleForTesting
+ static void copyOssOptionsBackToPaimonOptions(
Review Comment:
This copies every `fs.oss.*` entry, including ones loaded from
`core-site.xml` under `HADOOP_CONF_DIR`, back into the Paimon options
regardless of the warehouse scheme. Could we only do this when the warehouse is
`oss://`? That keeps OSS secrets out of Paimon options for HDFS/S3 warehouses,
which is what #12051 was after. Also, `!paimonOptions.containsKey(key)` looks
always true here, since `extractHadoopConfiguration` and
`movePaimonStorageOptionsToConf` already removed all `fs.*` keys.
With this change the inner catalog options are no longer free of filesystem
credentials for OSS. Please also update the comments that still describe them
as sanitized:
- The `mutableOptions` field comment (L93-94)
- The `@param paimonOptions` Javadoc of `createInnerCatalog` (L167)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]