Sitan Pang created FLINK-27104:
----------------------------------
Summary: ClassNotFound when submit jobs to session cluster with
external jars
Key: FLINK-27104
URL: https://issues.apache.org/jira/browse/FLINK-27104
Project: Flink
Issue Type: Improvement
Components: Client / Job Submission
Affects Versions: 1.11.0
Reporter: Sitan Pang
I'm trying to submit SQL jobs to session cluster on k8s, which need external
local udfs jars.
I meet ClassNotFound Exception because of the reasons below:
* 'pipeline.jars' will be overwritten by '-j' option which only accepts one
jar.
* 'pipeline.classpaths' will not be uploaded, so local files could not be
found in TM.
* 'pipelines.cached-files' will not be added to classpaths. In
Task#createUserCodeClassloader and ClientUtils#buildUserCodeClassLoader, only
'pipeline.jars' and 'pipeline.classpaths' are added to user class loader.
Only combining external jars into one could solve this, which means we need to
re-create a new jar every time.
Is there a better way to submit with external jars?
Or could we make 'pipeline.jars' directly used by users?
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
