Good Morning Nathan,
The exception stack does not give enough information yet to come to a solution,
the way I would continue is this:
* Given that you run in a local environment probably means that you could
run your job in a debugger and
* Place an exception break point for java.lang.IllegalArgumentException
* Once after the 10^6 events you trap into the problem walk up the call
stack to
org.apache.flink.api.java.typeutils.runtime.PojoSerializer#initializeFields and
* Try to find out why this PojoSerializer instance has some int serializer
instead of a double serializer in the respective field-serializer index:
protected void initializeFields(T t) {
for (int i = 0; i < numFields; i++) {
if (fields[i] != null) {
try {
fields[i].set(t, fieldSerializers[i].createInstance()); //it
happens all here
} catch (IllegalAccessException e) {
throw new RuntimeException("Cannot initialize fields.", e);
}
}
}
}
I hope you come closer to a solution if you poke a little around.
Feel free to get back to me for further clarifications.
Thias
From: Nathan Yu <nuonathan...@twosigma.com>
Sent: Freitag, 13. August 2021 05:27
To: JING ZHANG <beyond1...@gmail.com>
Cc: user@flink.apache.org
Subject: RE: Bug with PojoSerializer? java.lang.IllegalArgumentException: Can
not set final double field Event.rating to java.lang.Integer
Yea, it goes through many events from this input before the exception is
thrown. I don’t know how the input schema can change though, as the input is
always producing objects from the same class.
From: JING ZHANG <beyond1...@gmail.com<mailto:beyond1...@gmail.com>>
Sent: Thursday, August 12, 2021 10:22 PM
To: Nathan Yu <nuonathan...@twosigma.com<mailto:nuonathan...@twosigma.com>>
Cc: user@flink.apache.org<mailto:user@flink.apache.org>
Subject: Re: Bug with PojoSerializer? java.lang.IllegalArgumentException: Can
not set final double field Event.rating to java.lang.Integer
Hi Yu,
The exception is thrown after processing some input data instead of at the
beginning of the input, right?
Is there any possible that input schema has updated?
Nathan Yu <nuonathan...@twosigma.com<mailto:nuonathan...@twosigma.com>>
于2021年8月13日周五 上午8:38写道：
• Using local environment:
StreamExecutionEnvironment.createLocalEnvironment()
• Event is a POJO class, with int, double, enum, and String fields
• Unfortunately it’s hard for me to reproduce in a small example, as it
seems to occur after 10e6+ events.
• Using flink-core-1.12.4
Stack:
Caused by: java.lang.IllegalArgumentException: Can not set final double field
com.twosigma.research.options.optticks.core.types.Event.askPrice to
java.lang.Integer
at
java.base/jdk.internal.reflect.UnsafeFieldAccessorImpl.throwSetIllegalArgumentException(UnsafeFieldAccessorImpl.java:167)
at
java.base/jdk.internal.reflect.UnsafeFieldAccessorImpl.throwSetIllegalArgumentException(UnsafeFieldAccessorImpl.java:171)
at
java.base/jdk.internal.reflect.UnsafeFieldAccessorImpl.ensureObj(UnsafeFieldAccessorImpl.java:58)
at
java.base/jdk.internal.reflect.UnsafeQualifiedDoubleFieldAccessorImpl.set(UnsafeQualifiedDoubleFieldAccessorImpl.java:77)
at java.base/java.lang.reflect.Field.set(Field.java:780)
at
org.apache.flink.api.java.typeutils.runtime.PojoSerializer.initializeFields(PojoSerializer.java:205)
at
org.apache.flink.api.java.typeutils.runtime.PojoSerializer.deserialize(PojoSerializer.java:388)
at
org.apache.flink.api.java.typeutils.runtime.PojoSerializer.deserialize(PojoSerializer.java:409)
at
org.apache.flink.streaming.runtime.streamrecord.StreamElementSerializer.deserialize(StreamElementSerializer.java:191)
at
org.apache.flink.streaming.runtime.streamrecord.StreamElementSerializer.deserialize(StreamElementSerializer.java:46)
at
org.apache.flink.runtime.plugable.NonReusingDeserializationDelegate.read(NonReusingDeserializationDelegate.java:53)
Diese Nachricht ist ausschliesslich für den Adressaten bestimmt und beinhaltet
unter Umständen vertrauliche Mitteilungen. Da die Vertraulichkeit von
e-Mail-Nachrichten nicht gewährleistet werden kann, übernehmen wir keine
Haftung für die Gewährung der Vertraulichkeit und Unversehrtheit dieser
Mitteilung. Bei irrtümlicher Zustellung bitten wir Sie um Benachrichtigung per
e-Mail und um Löschung dieser Nachricht sowie eventueller Anhänge. Jegliche
unberechtigte Verwendung oder Verbreitung dieser Informationen ist streng
verboten.
This message is intended only for the named recipient and may contain
confidential or privileged information. As the confidentiality of email
communication cannot be guaranteed, we do not accept any responsibility for the
confidentiality and the intactness of this message. If you have received it in
error, please advise the sender by return e-mail and delete this message and
any attachments. Any unauthorised use or dissemination of this information is
strictly prohibited.
