Hi Swapna, I was away from the keyboard for the last two weeks. Sorry for the late reply.
1. Dual-instantiation in Application Mode Yes, currently, the job creation event happens in client side, and any follow up status change events propagation happen in DefaultExecutionGraph. In this case, there are two instances of the same job listener created. We probably may consider moving the job creation event to dispatcher. In this case, a single job listener instance for each type in JVM needs to be created. I may work on a POC on this change. 2. If a single job listener instance will be created, then states within the instance will be shared within the whole job lifecycle. Would you please elaborate a little bit more on the question "And also handle cases like Job transition events coming before the JobCreationEvent?" Best Regards Peter Huang On Fri, Apr 3, 2026 at 10:48 AM Swapna Marru <[email protected]> wrote: > Hello Flink Devs, > > I am trying to understand the behavioral contract of > JobStatusChangedListener (FLIP-314) in Application Mode, to correctly fix > the OpenLineage Flink integration. > > In Application Mode, JobStatusChangedListenerFactory.createListener() is > invoked independently at two call sites, producing two distinct instances: > > - EmbeddedExecutor fires the lineage creation event on Instance A — > L138–142 > < > https://github.com/apache/flink/blob/master/flink-clients/src/main/java/org/apache/flink/client/deployment/application/executors/EmbeddedExecutor.java#L138-L142 > > > - DefaultExecutionGraphBuilder fires the job transition event on > Instance B — L147–149 > < > https://github.com/apache/flink/blob/master/flink-runtime/src/main/java/org/apache/flink/runtime/executiongraph/DefaultExecutionGraphBuilder.java#L147-L149 > > > > The OpenLineage integration assumes a single shared instance, where lineage > context captured during graph planning is referenced upon job transition — > L131–135 > < > https://github.com/OpenLineage/OpenLineage/blob/main/integration/flink/flink2/src/main/java/io/openlineage/flink/listener/OpenLineageJobStatusChangedListener.java#L131-L135 > >. > With two independent instances, Instance B has no JobId set on the context. > > Could you clarify: > > 1. In the FLIP, I see "In this FLIP the JobStatusChangedListener will be > in Client and JobMaster, which will report lineage information and job > status independently." So is dual-instantiation in Application Mode > intentional ? > 2. Are listener implementations expected to be stateless, externalizing > any shared state themselves? And also handle cases like Job transition > events coming before the JobCreationEvent ? > > -Thanks > > M.Swapna >
