JingsongLi commented on code in PR #10350:
URL: https://github.com/apache/paimon/pull/10350#discussion_r4245041255
##########
paimon-filesystems/paimon-jindo/src/main/java/org/apache/paimon/jindo/JindoFileIO.java:
##########
@@ -181,29 +225,40 @@ public Options hadoopOptions(Path path, String opType) {
shouldCache = writeCacheEnabled && shouldCache(path);
} else if (opType.equalsIgnoreCase("meta")) {
shouldCache = metaCacheEnabled && shouldCache(path);
+ } else if (opType.equalsIgnoreCase("exists")) {
+ shouldCache = existsCacheEnabled && shouldCache(path);
}
- if (shouldCache) {
- return hadoopOptionsWithCache;
- } else {
+ if (!shouldCache) {
return hadoopOptions;
+ } else if (cacheRouting != null) {
+ return cacheTargetOptions.get(cacheRouting.targetOf(path).name);
Review Comment:
[P2] Preserve the selected endpoint's scheme and addressing in native format
consumers
Returning these target options also routes Lance/Vortex reads and writes
through their existing option converters, but both converters hard-code an
HTTPS bucket-prefixed endpoint and virtual-hosted addressing. With
`io-cache.endpoint=http://127.0.0.1:8080`, `io-cache.policy=read,write`,
`io-cache.whitelist=data` and `io-cache.target.default.path-style-access=true`,
the actual Jindo selection is endpoint `127.0.0.1:8080`,
`fs.oss.https.enable=false`, and `fs.oss.second.level.domain.enable=true`.
Calling all four production Lance/Vortex reader/writer converters instead
produces `endpoint=https://bkt.127.0.0.1:8080` and
`virtual_hosted_style_request=true`. In particular, Vortex passes this map
unchanged to its native read/write APIs, so the documented HTTP/IP cache target
is contacted using the wrong host and protocol. Please adapt the native
consumers to the selected scheme/addressing (and add a Jindo-to-converter
regression test), or keep unsupported native target configurations on
the origin until that handoff is supported.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
