deepujain opened a new pull request, #8303: URL: https://github.com/apache/hadoop/pull/8303
# PR description — copy into GitHub when opening the PR (do not commit this file) ## HADOOP-19833. Fix fs.s3a.classloader.isolation=false by using classloader from Configuration in getInstanceFromReflection ### Problem The configuration `fs.s3a.classloader.isolation=false` (introduced in HADOOP-18993) does not work as intended. The isolation flag is stored correctly in the Hadoop Configuration, but when creating instances via `S3AUtils.getInstanceFromReflection`, the classloader was always taken from `conf.getClassLoader()` without considering the isolation flag. When isolation is disabled, the Configuration may not have had its classloader set (e.g. `maybeIsolateClassloader` does not set it when isolation=false), so we must explicitly respect the flag and, when isolation is false, use the Configuration's classloader when set, otherwise the thread context classloader so that loading is not forced to use an isolated S3A classloader. ### Change In `S3AUtils.getInstanceFromReflection`: - When `conf != null` and `fs.s3a.classloader.isolation` is true (default): keep using `conf.getClassLoader()` (set by `maybeIsolateClassloader` to the S3A classloader). - When `conf != null` and `fs.s3a.classloader.isolation` is false: use the classloader from the Configuration (`conf.getClassLoader()`); if that is null, use `Thread.currentThread().getContextClassLoader()` so that classes are loaded from the application classpath. - If the resolved classloader is still null, fall back to `S3AUtils.class.getClassLoader()` for backward compatibility. This ensures that when users set `fs.s3a.classloader.isolation=false`, instance creation uses the Configuration's classloader (or the context classloader) instead of always using the S3A isolated classloader. ### Tests - Existing test `ITestS3AFileSystemIsolatedClassloader` covers isolation=true and isolation=false; the test expects that when isolation=false, `fs.getConf().getClassLoader()` is not the S3A classloader. This fix ensures the classloader used in `getInstanceFromReflection` is consistent with that expectation. ### JIRA Fixes HADOOP-19833 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
