yadavay-amzn opened a new pull request, #55869: URL: https://github.com/apache/spark/pull/55869
### What changes were proposed in this pull request? Add jars from `spark.jars` to `SharedState.jarClassLoader` during initialization. ### Why are the changes needed? When a persistent UDF is created via `CREATE FUNCTION ... AS 'com.test.HelloUDF'` (without `USING JAR`), the function metadata is stored in the catalog with empty `resources`. After ThriftServer restart, `loadFunctionResources` is a no-op and `makeFunctionBuilder` fails with `CANNOT_LOAD_FUNCTION_CLASS` because `Utils.classIsLoadable` returns false. The jar is on the classpath via `spark.jars`, but `SharedState.jarClassLoader` is created empty and relies on parent classloader delegation to reach it. This delegation is not reliable in ThriftServer's thread pool threads. The fix explicitly adds `spark.jars` to `jarClassLoader` so UDF classes are loadable regardless of parent delegation. ### Does this PR introduce _any_ user-facing change? Yes. Persistent UDFs registered without `USING JAR` will now resolve correctly after ThriftServer restart when the jar is specified via `spark.jars`. ### How was this patch tested? Added test in `SharedStateSuite` verifying that `spark.jars` appear in `jarClassLoader.getURLs`. ### Was this patch authored or co-authored using generative AI tooling? Yes -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
