[
https://issues.apache.org/jira/browse/FLINK-32911?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17757866#comment-17757866
]
Matthias Pohl edited comment on FLINK-32911 at 8/23/23 8:27 AM:
----------------------------------------------------------------
I tried running the {{TopSpeedWindowing}} example with a checkpoint interval of
10s and the data input file that I [attached to this
Jira|https://issues.apache.org/jira/secure/attachment/13062368/data.txt.gz]. I
run into the following {{NullPointerException}}:
{code}
2023-08-23 09:56:03,111 ERROR
org.apache.flink.runtime.rpc.pekko.FencedPekkoRpcActor [] - Error while
executing remote procedure call public void
org.apache.flink.runtime.jobmaster.JobMaster.notifyEndOfData(org.apache.flink.runtime.executiongraph.ExecutionAttemptID).
java.lang.reflect.InvocationTargetException: null
at jdk.internal.reflect.GeneratedMethodAccessor20.invoke(Unknown
Source) ~[?:?]
at
jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
~[?:?]
at java.lang.reflect.Method.invoke(Method.java:568) ~[?:?]
at
org.apache.flink.runtime.rpc.pekko.PekkoRpcActor.lambda$handleRpcInvocation$0(PekkoRpcActor.java:301)
~[flink-rpc-akka4947976d-da31-483f-88c7-8f3ce057edfe.jar:1.18-SNAPSHOT]
at
org.apache.flink.runtime.concurrent.ClassLoadingUtils.runWithContextClassLoader(ClassLoadingUtils.java:83)
~[classes/:?]
at
org.apache.flink.runtime.rpc.pekko.PekkoRpcActor.handleRpcInvocation(PekkoRpcActor.java:300)
~[flink-rpc-akka4947976d-da31-483f-88c7-8f3ce057edfe.jar:1.18-SNAPSHOT]
at
org.apache.flink.runtime.rpc.pekko.PekkoRpcActor.handleRpcMessage(PekkoRpcActor.java:222)
~[flink-rpc-akka4947976d-da31-483f-88c7-8f3ce057edfe.jar:1.18-SNAPSHOT]
at
org.apache.flink.runtime.rpc.pekko.FencedPekkoRpcActor.handleRpcMessage(FencedPekkoRpcActor.java:85)
~[flink-rpc-akka4947976d-da31-483f-88c7-8f3ce057edfe.jar:1.18-SNAPSHOT]
at
org.apache.flink.runtime.rpc.pekko.PekkoRpcActor.handleMessage(PekkoRpcActor.java:168)
~[flink-rpc-akka4947976d-da31-483f-88c7-8f3ce057edfe.jar:1.18-SNAPSHOT]
at
org.apache.pekko.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:33)
[flink-rpc-akka4947976d-da31-483f-88c7-8f3ce057edfe.jar:1.18-SNAPSHOT]
at
org.apache.pekko.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:29)
[flink-rpc-akka4947976d-da31-483f-88c7-8f3ce057edfe.jar:1.18-SNAPSHOT]
at scala.PartialFunction.applyOrElse(PartialFunction.scala:127)
[flink-rpc-akka4947976d-da31-483f-88c7-8f3ce057edfe.jar:1.18-SNAPSHOT]
at scala.PartialFunction.applyOrElse$(PartialFunction.scala:126)
[flink-rpc-akka4947976d-da31-483f-88c7-8f3ce057edfe.jar:1.18-SNAPSHOT]
at
org.apache.pekko.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:29)
[flink-rpc-akka4947976d-da31-483f-88c7-8f3ce057edfe.jar:1.18-SNAPSHOT]
at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:175)
[flink-rpc-akka4947976d-da31-483f-88c7-8f3ce057edfe.jar:1.18-SNAPSHOT]
at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:176)
[flink-rpc-akka4947976d-da31-483f-88c7-8f3ce057edfe.jar:1.18-SNAPSHOT]
at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:176)
[flink-rpc-akka4947976d-da31-483f-88c7-8f3ce057edfe.jar:1.18-SNAPSHOT]
at org.apache.pekko.actor.Actor.aroundReceive(Actor.scala:547)
[flink-rpc-akka4947976d-da31-483f-88c7-8f3ce057edfe.jar:1.18-SNAPSHOT]
at org.apache.pekko.actor.Actor.aroundReceive$(Actor.scala:545)
[flink-rpc-akka4947976d-da31-483f-88c7-8f3ce057edfe.jar:1.18-SNAPSHOT]
at
org.apache.pekko.actor.AbstractActor.aroundReceive(AbstractActor.scala:229)
[flink-rpc-akka4947976d-da31-483f-88c7-8f3ce057edfe.jar:1.18-SNAPSHOT]
at org.apache.pekko.actor.ActorCell.receiveMessage(ActorCell.scala:590)
[flink-rpc-akka4947976d-da31-483f-88c7-8f3ce057edfe.jar:1.18-SNAPSHOT]
at org.apache.pekko.actor.ActorCell.invoke(ActorCell.scala:557)
[flink-rpc-akka4947976d-da31-483f-88c7-8f3ce057edfe.jar:1.18-SNAPSHOT]
at org.apache.pekko.dispatch.Mailbox.processMailbox(Mailbox.scala:280)
[flink-rpc-akka4947976d-da31-483f-88c7-8f3ce057edfe.jar:1.18-SNAPSHOT]
at org.apache.pekko.dispatch.Mailbox.run(Mailbox.scala:241)
[flink-rpc-akka4947976d-da31-483f-88c7-8f3ce057edfe.jar:1.18-SNAPSHOT]
at org.apache.pekko.dispatch.Mailbox.exec(Mailbox.scala:253)
[flink-rpc-akka4947976d-da31-483f-88c7-8f3ce057edfe.jar:1.18-SNAPSHOT]
at java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) [?:?]
at
java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182)
[?:?]
at java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) [?:?]
at java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622)
[?:?]
at
java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165)
[?:?]
Caused by: java.lang.NullPointerException: Cannot invoke
"java.util.BitSet.set(int)" because "subtaskStatus" is null
at
org.apache.flink.runtime.scheduler.VertexEndOfDataListener.recordTaskEndOfData(VertexEndOfDataListener.java:51)
~[classes/:?]
at
org.apache.flink.runtime.scheduler.SchedulerBase.notifyEndOfData(SchedulerBase.java:1079)
~[classes/:?]
at
org.apache.flink.runtime.jobmaster.JobMaster.notifyEndOfData(JobMaster.java:508)
~[classes/:?]
... 30 more
{code}
It looks like {{VertexEndOfDataListener}} is related to the change of
FLINK-28386. [~Jiang Xin] Shall I create a bug issue?
was (Author: mapohl):
I tried running the {{TopSpeedWindowing}} example with a checkpoint interval of
10s and the data input file that I attached to this Jira. I run into the
following {{NullPointerException}}:
{code}
2023-08-23 09:56:03,111 ERROR
org.apache.flink.runtime.rpc.pekko.FencedPekkoRpcActor [] - Error while
executing remote procedure call public void
org.apache.flink.runtime.jobmaster.JobMaster.notifyEndOfData(org.apache.flink.runtime.executiongraph.ExecutionAttemptID).
java.lang.reflect.InvocationTargetException: null
at jdk.internal.reflect.GeneratedMethodAccessor20.invoke(Unknown
Source) ~[?:?]
at
jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
~[?:?]
at java.lang.reflect.Method.invoke(Method.java:568) ~[?:?]
at
org.apache.flink.runtime.rpc.pekko.PekkoRpcActor.lambda$handleRpcInvocation$0(PekkoRpcActor.java:301)
~[flink-rpc-akka4947976d-da31-483f-88c7-8f3ce057edfe.jar:1.18-SNAPSHOT]
at
org.apache.flink.runtime.concurrent.ClassLoadingUtils.runWithContextClassLoader(ClassLoadingUtils.java:83)
~[classes/:?]
at
org.apache.flink.runtime.rpc.pekko.PekkoRpcActor.handleRpcInvocation(PekkoRpcActor.java:300)
~[flink-rpc-akka4947976d-da31-483f-88c7-8f3ce057edfe.jar:1.18-SNAPSHOT]
at
org.apache.flink.runtime.rpc.pekko.PekkoRpcActor.handleRpcMessage(PekkoRpcActor.java:222)
~[flink-rpc-akka4947976d-da31-483f-88c7-8f3ce057edfe.jar:1.18-SNAPSHOT]
at
org.apache.flink.runtime.rpc.pekko.FencedPekkoRpcActor.handleRpcMessage(FencedPekkoRpcActor.java:85)
~[flink-rpc-akka4947976d-da31-483f-88c7-8f3ce057edfe.jar:1.18-SNAPSHOT]
at
org.apache.flink.runtime.rpc.pekko.PekkoRpcActor.handleMessage(PekkoRpcActor.java:168)
~[flink-rpc-akka4947976d-da31-483f-88c7-8f3ce057edfe.jar:1.18-SNAPSHOT]
at
org.apache.pekko.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:33)
[flink-rpc-akka4947976d-da31-483f-88c7-8f3ce057edfe.jar:1.18-SNAPSHOT]
at
org.apache.pekko.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:29)
[flink-rpc-akka4947976d-da31-483f-88c7-8f3ce057edfe.jar:1.18-SNAPSHOT]
at scala.PartialFunction.applyOrElse(PartialFunction.scala:127)
[flink-rpc-akka4947976d-da31-483f-88c7-8f3ce057edfe.jar:1.18-SNAPSHOT]
at scala.PartialFunction.applyOrElse$(PartialFunction.scala:126)
[flink-rpc-akka4947976d-da31-483f-88c7-8f3ce057edfe.jar:1.18-SNAPSHOT]
at
org.apache.pekko.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:29)
[flink-rpc-akka4947976d-da31-483f-88c7-8f3ce057edfe.jar:1.18-SNAPSHOT]
at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:175)
[flink-rpc-akka4947976d-da31-483f-88c7-8f3ce057edfe.jar:1.18-SNAPSHOT]
at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:176)
[flink-rpc-akka4947976d-da31-483f-88c7-8f3ce057edfe.jar:1.18-SNAPSHOT]
at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:176)
[flink-rpc-akka4947976d-da31-483f-88c7-8f3ce057edfe.jar:1.18-SNAPSHOT]
at org.apache.pekko.actor.Actor.aroundReceive(Actor.scala:547)
[flink-rpc-akka4947976d-da31-483f-88c7-8f3ce057edfe.jar:1.18-SNAPSHOT]
at org.apache.pekko.actor.Actor.aroundReceive$(Actor.scala:545)
[flink-rpc-akka4947976d-da31-483f-88c7-8f3ce057edfe.jar:1.18-SNAPSHOT]
at
org.apache.pekko.actor.AbstractActor.aroundReceive(AbstractActor.scala:229)
[flink-rpc-akka4947976d-da31-483f-88c7-8f3ce057edfe.jar:1.18-SNAPSHOT]
at org.apache.pekko.actor.ActorCell.receiveMessage(ActorCell.scala:590)
[flink-rpc-akka4947976d-da31-483f-88c7-8f3ce057edfe.jar:1.18-SNAPSHOT]
at org.apache.pekko.actor.ActorCell.invoke(ActorCell.scala:557)
[flink-rpc-akka4947976d-da31-483f-88c7-8f3ce057edfe.jar:1.18-SNAPSHOT]
at org.apache.pekko.dispatch.Mailbox.processMailbox(Mailbox.scala:280)
[flink-rpc-akka4947976d-da31-483f-88c7-8f3ce057edfe.jar:1.18-SNAPSHOT]
at org.apache.pekko.dispatch.Mailbox.run(Mailbox.scala:241)
[flink-rpc-akka4947976d-da31-483f-88c7-8f3ce057edfe.jar:1.18-SNAPSHOT]
at org.apache.pekko.dispatch.Mailbox.exec(Mailbox.scala:253)
[flink-rpc-akka4947976d-da31-483f-88c7-8f3ce057edfe.jar:1.18-SNAPSHOT]
at java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) [?:?]
at
java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182)
[?:?]
at java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) [?:?]
at java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622)
[?:?]
at
java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165)
[?:?]
Caused by: java.lang.NullPointerException: Cannot invoke
"java.util.BitSet.set(int)" because "subtaskStatus" is null
at
org.apache.flink.runtime.scheduler.VertexEndOfDataListener.recordTaskEndOfData(VertexEndOfDataListener.java:51)
~[classes/:?]
at
org.apache.flink.runtime.scheduler.SchedulerBase.notifyEndOfData(SchedulerBase.java:1079)
~[classes/:?]
at
org.apache.flink.runtime.jobmaster.JobMaster.notifyEndOfData(JobMaster.java:508)
~[classes/:?]
... 30 more
{code}
It looks like {{VertexEndOfDataListener}} is related to the change of
FLINK-28386. [~Jiang Xin] Shall I create a bug issue?
> Release Testing: Verify FLINK-28386: Trigger an immediate checkpoint after
> all sinks finished
> ---------------------------------------------------------------------------------------------
>
> Key: FLINK-32911
> URL: https://issues.apache.org/jira/browse/FLINK-32911
> Project: Flink
> Issue Type: Sub-task
> Components: Tests
> Affects Versions: 1.18.0
> Reporter: Jiang Xin
> Assignee: Matthias Pohl
> Priority: Major
> Fix For: 1.18.0
>
> Attachments: data.txt.gz
>
>
> The feature is described in the
> [doc|https://nightlies.apache.org/flink/flink-docs-master/docs/dev/datastream/fault-tolerance/checkpointing/#waiting-for-the-final-checkpoint-before-task-exit]
> .
> To test the feature, we should run a Flink program on a bounded source and
> configure it with a large checkpoint interval. With this feature, the program
> should end up immediately when no more records need to be processed, instead
> of waiting for one more periodic checkpoint.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
