FrankChen021 commented on code in PR #19497:
URL: https://github.com/apache/druid/pull/19497#discussion_r3281507046
##########
extensions-core/orc-extensions/src/main/java/org/apache/druid/data/input/orc/OrcInputFormat.java:
##########
@@ -55,14 +57,15 @@ public OrcInputFormat(
this.conf = conf;
}
- private void initialize(Configuration conf)
+ // Ensure FileSystem class-level initialization happens with the correct
classloader (see #1714).
+ // Called once on first createReader() via AtomicBoolean guard to prevent a
race when multiple
+ // subtasks concurrently swap Thread.currentThread().setContextClassLoader()
(see #19491).
+ private void ensureFileSystemInitialized()
{
- //Initializing seperately since during eager initialization, resolving
- //namenode hostname throws an error if nodes are ephemeral
-
- // Ensure that FileSystem class level initialization happens with correct
CL
- // See https://github.com/apache/druid/issues/1714
- ClassLoader currCtxCl = Thread.currentThread().getContextClassLoader();
+ if (!fileSystemInitialized.compareAndSet(false, true)) {
Review Comment:
[P2] Reset initialization guard when FileSystem init fails
`compareAndSet(false, true)` marks the format initialized before
`FileSystem.get(conf)` has actually succeeded. If that call throws an
`IOException`, the flag remains true and every later `createReader` on the same
`OrcInputFormat` skips the classloader-protected Hadoop FileSystem
initialization instead of retrying, which is a regression from the previous
per-reader behavior. Set the flag only after `FileSystem.get(conf)` succeeds,
or reset it in the catch path before rethrowing.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
