sundapeng commented on code in PR #10350:
URL: https://github.com/apache/paimon/pull/10350#discussion_r4191903218
##########
paimon-filesystems/paimon-jindo/src/main/java/org/apache/paimon/jindo/JindoFileIO.java:
##########
@@ -256,9 +302,40 @@ private static synchronized PluginLoader getLoader() {
@Override
protected Pair<JindoHadoopSystem, String> createFileSystem(
org.apache.hadoop.fs.Path path, boolean enableCache) {
+ return createFileSystem(path, enableCache ? hadoopOptionsWithCache :
hadoopOptions);
+ }
+
+ Pair<JindoHadoopSystem, String> createFileSystem(
+ org.apache.hadoop.fs.Path path, CacheTarget target) {
+ return createFileSystem(path, cacheTargetOptions.get(target.name));
+ }
+
+ @Override
+ protected Pair<JindoHadoopSystem, String> getFileSystemPair(
+ org.apache.hadoop.fs.Path path, boolean enableCache) throws
IOException {
+ if (!enableCache || cacheRouting == null) {
+ return super.getFileSystemPair(path, enableCache);
+ }
+ CacheTarget target = cacheRouting.targetOf(new Path(path.toUri()));
+ if (target == null) {
+ return super.getFileSystemPair(path, false);
+ }
+ if (cacheTargetFsMap == null) {
+ synchronized (this) {
+ if (cacheTargetFsMap == null) {
+ cacheTargetFsMap = new ConcurrentHashMap<>();
+ }
+ }
+ }
+ String authority = path.toUri().getAuthority();
+ String key = target.name + "/" + (authority == null ? "DEFAULT" :
authority);
+ return cacheTargetFsMap.computeIfAbsent(key, k ->
createFileSystem(path, target));
Review Comment:
Thanks, fixed in db6dd66794. The cache-target `computeIfAbsent` now unwraps
`UncheckedIOException` the same way `HadoopCompliantFileIO.getFileSystemPair`
does, so a target that fails to initialize throws the original checked
`IOException`.
`JindoIoCacheRoutingTest#testCacheTargetInitializationErrorsStayChecked` covers
read, file status and exists on such a target.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
