[
https://issues.apache.org/jira/browse/FLINK-30072?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Zhu Zhu updated FLINK-30072:
----------------------------
Component/s: Runtime / Task
(was: Runtime / Coordination)
> Cannot assign instance of SerializedLambda to field
> KeyGroupStreamPartitioner.keySelector
> -----------------------------------------------------------------------------------------
>
> Key: FLINK-30072
> URL: https://issues.apache.org/jira/browse/FLINK-30072
> Project: Flink
> Issue Type: Bug
> Components: Runtime / Task
> Affects Versions: 1.16.0
> Reporter: Nico Kruber
> Priority: Major
>
> In application mode, if the {{usrlib}} directories of the JM and TM differ,
> e.g. same jars but different names, the job is failing and throws this
> cryptic exception on the JM:
> {code}
> 2022-11-17 09:55:12,968 INFO
> org.apache.flink.runtime.scheduler.adaptive.AdaptiveScheduler [] - Restarting
> job.
> org.apache.flink.streaming.runtime.tasks.StreamTaskException: Could not
> instantiate outputs in order.
> at
> org.apache.flink.streaming.api.graph.StreamConfig.getVertexNonChainedOutputs(StreamConfig.java:537)
> ~[flink-dist-1.16.0-ok.0.jar:1.16.0-ok.0]
> at
> org.apache.flink.streaming.runtime.tasks.StreamTask.createRecordWriters(StreamTask.java:1600)
> ~[flink-dist-1.16.0-ok.0.jar:1.16.0-ok.0]
> at
> org.apache.flink.streaming.runtime.tasks.StreamTask.createRecordWriterDelegate(StreamTask.java:1584)
> ~[flink-dist-1.16.0-ok.0.jar:1.16.0-ok.0]
> at
> org.apache.flink.streaming.runtime.tasks.StreamTask.<init>(StreamTask.java:408)
> ~[flink-dist-1.16.0-ok.0.jar:1.16.0-ok.0]
> at
> org.apache.flink.streaming.runtime.tasks.StreamTask.<init>(StreamTask.java:362)
> ~[flink-dist-1.16.0-ok.0.jar:1.16.0-ok.0]
> at
> org.apache.flink.streaming.runtime.tasks.StreamTask.<init>(StreamTask.java:335)
> ~[flink-dist-1.16.0-ok.0.jar:1.16.0-ok.0]
> at
> org.apache.flink.streaming.runtime.tasks.StreamTask.<init>(StreamTask.java:327)
> ~[flink-dist-1.16.0-ok.0.jar:1.16.0-ok.0]
> at
> org.apache.flink.streaming.runtime.tasks.StreamTask.<init>(StreamTask.java:317)
> ~[flink-dist-1.16.0-ok.0.jar:1.16.0-ok.0]
> at
> org.apache.flink.streaming.runtime.tasks.SourceOperatorStreamTask.<init>(SourceOperatorStreamTask.java:84)
> ~[flink-dist-1.16.0-ok.0.jar:1.16.0-ok.0]
> at
> jdk.internal.reflect.GeneratedConstructorAccessor38.newInstance(Unknown
> Source) ~[?:?]
> at
> jdk.internal.reflect.DelegatingConstructorAccessorImpl.newInstance(Unknown
> Source) ~[?:?]
> at java.lang.reflect.Constructor.newInstance(Unknown Source) ~[?:?]
> at
> org.apache.flink.runtime.taskmanager.Task.loadAndInstantiateInvokable(Task.java:1589)
> ~[flink-dist-1.16.0-ok.0.jar:1.16.0-ok.0]
> at org.apache.flink.runtime.taskmanager.Task.doRun(Task.java:714)
> ~[flink-dist-1.16.0-ok.0.jar:1.16.0-ok.0]
> at org.apache.flink.runtime.taskmanager.Task.run(Task.java:550)
> ~[flink-dist-1.16.0-ok.0.jar:1.16.0-ok.0]
> at java.lang.Thread.run(Unknown Source) ~[?:?]
> Caused by: java.lang.ClassCastException: cannot assign instance of
> java.lang.invoke.SerializedLambda to field
> org.apache.flink.streaming.runtime.partitioner.KeyGroupStreamPartitioner.keySelector
> of type org.apache.flink.api.java.functions.KeySelector in instance of
> org.apache.flink.streaming.runtime.partitioner.KeyGroupStreamPartitioner
> at java.io.ObjectStreamClass$FieldReflector.setObjFieldValues(Unknown
> Source) ~[?:?]
> at
> java.io.ObjectStreamClass$FieldReflector.checkObjectFieldValueTypes(Unknown
> Source) ~[?:?]
> at java.io.ObjectStreamClass.checkObjFieldValueTypes(Unknown Source)
> ~[?:?]
> at java.io.ObjectInputStream.defaultCheckFieldValues(Unknown Source)
> ~[?:?]
> at java.io.ObjectInputStream.readSerialData(Unknown Source) ~[?:?]
> at java.io.ObjectInputStream.readOrdinaryObject(Unknown Source) ~[?:?]
> at java.io.ObjectInputStream.readObject0(Unknown Source) ~[?:?]
> at java.io.ObjectInputStream.defaultReadFields(Unknown Source) ~[?:?]
> at java.io.ObjectInputStream.readSerialData(Unknown Source) ~[?:?]
> at java.io.ObjectInputStream.readOrdinaryObject(Unknown Source) ~[?:?]
> at java.io.ObjectInputStream.readObject0(Unknown Source) ~[?:?]
> at java.io.ObjectInputStream.readObject(Unknown Source) ~[?:?]
> at java.io.ObjectInputStream.readObject(Unknown Source) ~[?:?]
> at java.util.ArrayList.readObject(Unknown Source) ~[?:?]
> at jdk.internal.reflect.GeneratedMethodAccessor16.invoke(Unknown Source)
> ~[?:?]
> at jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown
> Source) ~[?:?]
> at java.lang.reflect.Method.invoke(Unknown Source) ~[?:?]
> at java.io.ObjectStreamClass.invokeReadObject(Unknown Source) ~[?:?]
> at java.io.ObjectInputStream.readSerialData(Unknown Source) ~[?:?]
> at java.io.ObjectInputStream.readOrdinaryObject(Unknown Source) ~[?:?]
> at java.io.ObjectInputStream.readObject0(Unknown Source) ~[?:?]
> at java.io.ObjectInputStream.readObject(Unknown Source) ~[?:?]
> at java.io.ObjectInputStream.readObject(Unknown Source) ~[?:?]
> at
> org.apache.flink.util.InstantiationUtil.deserializeObject(InstantiationUtil.java:617)
> ~[flink-dist-1.16.0-ok.0.jar:1.16.0-ok.0]
> at
> org.apache.flink.util.InstantiationUtil.deserializeObject(InstantiationUtil.java:602)
> ~[flink-dist-1.16.0-ok.0.jar:1.16.0-ok.0]
> at
> org.apache.flink.util.InstantiationUtil.deserializeObject(InstantiationUtil.java:589)
> ~[flink-dist-1.16.0-ok.0.jar:1.16.0-ok.0]
> at
> org.apache.flink.util.InstantiationUtil.readObjectFromConfig(InstantiationUtil.java:543)
> ~[flink-dist-1.16.0-ok.0.jar:1.16.0-ok.0]
> at
> org.apache.flink.streaming.api.graph.StreamConfig.getVertexNonChainedOutputs(StreamConfig.java:533)
> ~[flink-dist-1.16.0-ok.0.jar:1.16.0-ok.0]
> ... 15 more
> {code}
> It seems that our normal exception reporting is not really working (maybe
> also a problem for more common failure?). The TM logs do contain the actual
> exception though:
> {code}
> 2022-11-17 10:11:43,551 WARN org.apache.flink.runtime.taskmanager.Task
> [] - TumblingEventTimeWindows -> Sink: Print to Std. Out (1/1)#3
> (cc8e6b0246079230c5ac1bc335c70163_c27dcf7b54ef6bfd6cff02ca8870b681_0_3)
> switched from INITIALIZING to FAILED with failure cause:
> org.apache.flink.streaming.runtime.tasks.StreamTaskException: Cannot load
> user class:
> com.immerok.cloud.examples.WindowAggregation$MetadataEnrichingWindowFunction
> ClassLoader info: URL ClassLoader:
> file: 'usrlib/window-agg-2.jar' (missing)
> Class not resolvable through given classloader.
> at
> org.apache.flink.streaming.api.graph.StreamConfig.getStreamOperatorFactory(StreamConfig.java:397)
> at
> org.apache.flink.streaming.runtime.tasks.OperatorChain.<init>(OperatorChain.java:162)
> at
> org.apache.flink.streaming.runtime.tasks.RegularOperatorChain.<init>(RegularOperatorChain.java:60)
> at
> org.apache.flink.streaming.runtime.tasks.StreamTask.restoreInternal(StreamTask.java:681)
> at
> org.apache.flink.streaming.runtime.tasks.StreamTask.restore(StreamTask.java:669)
> at
> org.apache.flink.runtime.taskmanager.Task.runWithSystemExitMonitoring(Task.java:935)
> at
> org.apache.flink.runtime.taskmanager.Task.restoreAndInvoke(Task.java:904)
> at org.apache.flink.runtime.taskmanager.Task.doRun(Task.java:728)
> at org.apache.flink.runtime.taskmanager.Task.run(Task.java:550)
> at java.base/java.lang.Thread.run(Unknown Source)
> Caused by: java.lang.ClassNotFoundException:
> com.immerok.cloud.examples.WindowAggregation$MetadataEnrichingWindowFunction
> at java.base/java.net.URLClassLoader.findClass(Unknown Source)
> at java.base/java.lang.ClassLoader.loadClass(Unknown Source)
> at
> org.apache.flink.util.FlinkUserCodeClassLoader.loadClassWithoutExceptionHandling(FlinkUserCodeClassLoader.java:67)
> at
> org.apache.flink.util.ChildFirstClassLoader.loadClassWithoutExceptionHandling(ChildFirstClassLoader.java:74)
> at
> org.apache.flink.util.FlinkUserCodeClassLoader.loadClass(FlinkUserCodeClassLoader.java:51)
> at java.base/java.lang.ClassLoader.loadClass(Unknown Source)
> at
> org.apache.flink.util.FlinkUserCodeClassLoaders$SafetyNetWrapperClassLoader.loadClass(FlinkUserCodeClassLoaders.java:192)
> at java.base/java.lang.Class.forName0(Native Method)
> at java.base/java.lang.Class.forName(Unknown Source)
> at
> org.apache.flink.util.InstantiationUtil$ClassLoaderObjectInputStream.resolveClass(InstantiationUtil.java:78)
> at java.base/java.io.ObjectInputStream.readNonProxyDesc(Unknown Source)
> at java.base/java.io.ObjectInputStream.readClassDesc(Unknown Source)
> at java.base/java.io.ObjectInputStream.readOrdinaryObject(Unknown Source)
> at java.base/java.io.ObjectInputStream.readObject0(Unknown Source)
> at java.base/java.io.ObjectInputStream.defaultReadFields(Unknown Source)
> at java.base/java.io.ObjectInputStream.readSerialData(Unknown Source)
> at java.base/java.io.ObjectInputStream.readOrdinaryObject(Unknown Source)
> at java.base/java.io.ObjectInputStream.readObject0(Unknown Source)
> at java.base/java.io.ObjectInputStream.defaultReadFields(Unknown Source)
> at java.base/java.io.ObjectInputStream.readSerialData(Unknown Source)
> at java.base/java.io.ObjectInputStream.readOrdinaryObject(Unknown Source)
> at java.base/java.io.ObjectInputStream.readObject0(Unknown Source)
> at java.base/java.io.ObjectInputStream.defaultReadFields(Unknown Source)
> at java.base/java.io.ObjectInputStream.readSerialData(Unknown Source)
> at java.base/java.io.ObjectInputStream.readOrdinaryObject(Unknown Source)
> at java.base/java.io.ObjectInputStream.readObject0(Unknown Source)
> at java.base/java.io.ObjectInputStream.readObject(Unknown Source)
> at java.base/java.io.ObjectInputStream.readObject(Unknown Source)
> at
> org.apache.flink.util.InstantiationUtil.deserializeObject(InstantiationUtil.java:617)
> at
> org.apache.flink.util.InstantiationUtil.deserializeObject(InstantiationUtil.java:602)
> at
> org.apache.flink.util.InstantiationUtil.deserializeObject(InstantiationUtil.java:589)
> at
> org.apache.flink.util.InstantiationUtil.readObjectFromConfig(InstantiationUtil.java:543)
> at
> org.apache.flink.streaming.api.graph.StreamConfig.getStreamOperatorFactory(StreamConfig.java:383)
> ... 9 more
> {code}
> h2. How to reproduce:
> # on JM: {{<flink>/usrlib/window-agg-1.jar}}
> # on TM: {{<flink>/usrlib/window-agg-2.jar}}
> then start the job in application mode
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
