JingsongLi commented on code in PR #10350:
URL: https://github.com/apache/paimon/pull/10350#discussion_r4182807744
##########
paimon-filesystems/paimon-jindo/src/main/java/org/apache/paimon/jindo/JindoFileIO.java:
##########
@@ -256,9 +302,40 @@ private static synchronized PluginLoader getLoader() {
@Override
protected Pair<JindoHadoopSystem, String> createFileSystem(
org.apache.hadoop.fs.Path path, boolean enableCache) {
+ return createFileSystem(path, enableCache ? hadoopOptionsWithCache :
hadoopOptions);
+ }
+
+ Pair<JindoHadoopSystem, String> createFileSystem(
+ org.apache.hadoop.fs.Path path, CacheTarget target) {
+ return createFileSystem(path, cacheTargetOptions.get(target.name));
+ }
+
+ @Override
+ protected Pair<JindoHadoopSystem, String> getFileSystemPair(
+ org.apache.hadoop.fs.Path path, boolean enableCache) throws
IOException {
+ if (!enableCache || cacheRouting == null) {
+ return super.getFileSystemPair(path, enableCache);
+ }
+ CacheTarget target = cacheRouting.targetOf(new Path(path.toUri()));
+ if (target == null) {
+ return super.getFileSystemPair(path, false);
+ }
+ if (cacheTargetFsMap == null) {
+ synchronized (this) {
+ if (cacheTargetFsMap == null) {
+ cacheTargetFsMap = new ConcurrentHashMap<>();
+ }
+ }
+ }
+ String authority = path.toUri().getAuthority();
+ String key = target.name + "/" + (authority == null ? "DEFAULT" :
authority);
+ return cacheTargetFsMap.computeIfAbsent(key, k ->
createFileSystem(path, target));
Review Comment:
[P2] Preserve the checked initialization error on the cache-target path
createFileSystem wraps an IOException from fs.initialize in
UncheckedIOException. The parent getFileSystemPair catches and unwraps that
exception, but this new computeIfAbsent path does not. A transient first
initialization failure therefore bypasses IOException-based handling:
OrphanFilesClean.retryReadingFiles does not retry the first cached
manifest-list read, and ManifestSidecar.read can no longer fall back when a
sidecar is the first target read. That scenario is reachable after REST token
refresh creates a new Jindo instance while the manifest-list/main manifest
still hit the shared manifest cache and the separate sidecar cache misses:
selectBlocks reads the sidecar before the main cached manifest. The
fault-injection probe reproduces the exception/fallback difference with the
same SocketTimeoutException on both routes; a control which unwraps the target
exception restores the checked behavior. Please wrap this computeIfAbsent with
the same catch/throw-e.getCause() as the pa
rent and cover target initialization failure, not only errors from an
already-created filesystem.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
