tartarus created FLINK-18332:
--------------------------------
Summary: Add error message to precondition in
KeyGroupPartitionedPriorityQueue
Key: FLINK-18332
URL: https://issues.apache.org/jira/browse/FLINK-18332
Project: Flink
Issue Type: Improvement
Components: Runtime / Queryable State
Affects Versions: 1.10.1, 1.10.0
Environment: CentOS 7.0
Flink 1.10.0
jdk-1.8
Reporter: tartarus
in my case, the user custom a KeySelector and use a static SimpleDateFormat to
format unix timestamp. sometimes job will throw an
ArrayIndexOutOfBoundsException
{code:java}
java.lang.ArrayIndexOutOfBoundsException: -49
at
org.apache.flink.runtime.state.heap.KeyGroupPartitionedPriorityQueue.getKeyGroupSubHeapForElement(KeyGroupPartitionedPriorityQueue.java:174)
at
org.apache.flink.runtime.state.heap.KeyGroupPartitionedPriorityQueue.add(KeyGroupPartitionedPriorityQueue.java:110)
at
org.apache.flink.streaming.api.operators.InternalTimerServiceImpl.registerProcessingTimeTimer(InternalTimerServiceImpl.java:203)
at
org.apache.flink.streaming.runtime.operators.windowing.WindowOperator$Context.registerProcessingTimeTimer(WindowOperator.java:901)
at
org.apache.flink.streaming.api.windowing.triggers.ProcessingTimeTrigger.onElement(ProcessingTimeTrigger.java:36)
at
org.apache.flink.streaming.api.windowing.triggers.ProcessingTimeTrigger.onElement(ProcessingTimeTrigger.java:28)
at
org.apache.flink.streaming.runtime.operators.windowing.WindowOperator$Context.onElement(WindowOperator.java:920)
at
org.apache.flink.streaming.runtime.operators.windowing.WindowOperator.processElement(WindowOperator.java:402)
at
org.apache.flink.streaming.runtime.tasks.OneInputStreamTask$StreamTaskNetworkOutput.processRecord(OneInputStreamTask.java:204)
at
org.apache.flink.streaming.runtime.tasks.OneInputStreamTask$StreamTaskNetworkOutput.emitRecord(OneInputStreamTask.java:196)
at
org.apache.flink.streaming.runtime.io.StreamTaskNetworkInput.processElement(StreamTaskNetworkInput.java:151)
at
org.apache.flink.streaming.runtime.io.StreamTaskNetworkInput.emitNext(StreamTaskNetworkInput.java:128)
at
org.apache.flink.streaming.runtime.io.StreamOneInputProcessor.processInput(StreamOneInputProcessor.java:69)
at
org.apache.flink.streaming.runtime.tasks.StreamTask.processInput(StreamTask.java:311)
at
org.apache.flink.streaming.runtime.tasks.mailbox.MailboxProcessor.runMailboxLoop(MailboxProcessor.java:187)
at
org.apache.flink.streaming.runtime.tasks.StreamTask.runMailboxLoop(StreamTask.java:487)
at
org.apache.flink.streaming.runtime.tasks.StreamTask.invoke(StreamTask.java:470)
at org.apache.flink.runtime.taskmanager.Task.doRun(Task.java:707)
at org.apache.flink.runtime.taskmanager.Task.run(Task.java:532)
at java.lang.Thread.run(Thread.java:745)
{code}
I reproduced this case.
Because keySelector.getKey() will be called twice on the same record, and
SimpleDateFormat is not thread safe, In the case of high concurrency and Cross
Days, the results returned by the two calls of keySelector.getKey() may be
different.
So the keygroup calculated in the second execution is different from the
result of the first calculation，then throw an ArrayIndexOutOfBoundsException.
I think the error message should be clearer, not just the
ArrayIndexOutOfBoundsException.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
