Yi Pan (Data Infrastructure) commented on SAMZA-1385:

[~bharathkk], could you make the description of the issue a bit more clear? I 
assume you are saying that the leader election for stream creation and the 
leader election for job coordinator are using the same znode path. Hence, a 
single process can potentially acquire the leadership for stream creation and 
waiting for itself to become the leader JC, which won't happen since it is 
waiting for itself to release the leader position for stream creation, which is 
in the same znode?

So, at high-level, a straightforward solution is to separate the leader 
election nodes for stream creation and job model creation (i.e. job 
coordinator). Is that right?

> Coordination utils in LocalApplicationRunner uses same Zk path as 
> ZkJobCoordinatorFactory for leader election
> -------------------------------------------------------------------------------------------------------------
>                 Key: SAMZA-1385
>                 URL: https://issues.apache.org/jira/browse/SAMZA-1385
>             Project: Samza
>          Issue Type: Bug
>            Reporter: Bharath Kumarasubramanian
>            Assignee: Bharath Kumarasubramanian
>             Fix For: 0.13.1
> In case of applications which involves creating intermediate stream, we 
> perform the leader election twice. once for coordination for stream creation 
> and another for job coordination.
> We currently use the same zookeeper path for both of these and this results 
> in two nodes for the same processor in zookeeper. As a result, the process 
> goes into listener mode and stuck in job model generation phase.

This message was sent by Atlassian JIRA

Reply via email to