[jira] [Commented] (FLINK-12752) Add Option to Pass Seed for JobID Hash for StandaloneJobClusterEntrypoint
[
https://issues.apache.org/jira/browse/FLINK-12752?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17323256#comment-17323256
]
Flink Jira Bot commented on FLINK-12752:
This issue is assigned but has not received an update in 7 days so it has been
labeled "stale-assigned". If you are still working on the issue, please give an
update and remove the label. If you are no longer working on the issue, please
unassign so someone else may work on it. In 7 days the issue will be
automatically unassigned.
> Add Option to Pass Seed for JobID Hash for StandaloneJobClusterEntrypoint
> -
>
> Key: FLINK-12752
> URL: https://issues.apache.org/jira/browse/FLINK-12752
> Project: Flink
> Issue Type: Sub-task
> Components: flink-docker
>Reporter: Konstantin Knauf
>Assignee: Konstantin Knauf
>Priority: Major
> Labels: pull-request-available, stale-assigned
> Time Spent: 20m
> Remaining Estimate: 0h
>
> In case of a standalone job cluster setup, we would like to generate random a
> {{JobID}} for every job, but the {{JobID}} nevertheless needs to stay
> constant between JobManager process restarts.
> For this, I would like to add an additional command line options for the
> {{StandaloneJobClusterEntrypoint}} called {{jobId-seed}}. {{job-id-seed}} and
> {{job-id}} are mutually exclusive.
> * --jobId
> * --jobId-seed
> * (default)
> On Kubernetes, this new command line argument would need to be set to a
> property which is stable over Pod Restarts but changes for different K8s Jobs
> or K8s Deployments.
> *Open Questions*
> * what available information to use a seed for deployments on Kubernetes?
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (FLINK-12752) Add Option to Pass Seed for JobID Hash for StandaloneJobClusterEntrypoint
[
https://issues.apache.org/jira/browse/FLINK-12752?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16870878#comment-16870878
]
Konstantin Knauf commented on FLINK-12752:
--
[~1u0] Yes, originally I hoped there is some field in Kubernetes, which could
be used as the seed. After looking into it more, I now think, that no such
field exists. Nevertheless, I think, this is valuable because of the free-form
format and the provided templates. Additionally, this makes injecting the
{{JobID}} from outside more flexible for other environment, too.
> Add Option to Pass Seed for JobID Hash for StandaloneJobClusterEntrypoint
> -
>
> Key: FLINK-12752
> URL: https://issues.apache.org/jira/browse/FLINK-12752
> Project: Flink
> Issue Type: Sub-task
> Components: Deployment / Docker
>Reporter: Konstantin Knauf
>Assignee: Konstantin Knauf
>Priority: Major
> Labels: pull-request-available
> Time Spent: 20m
> Remaining Estimate: 0h
>
> In case of a standalone job cluster setup, we would like to generate random a
> {{JobID}} for every job, but the {{JobID}} nevertheless needs to stay
> constant between JobManager process restarts.
> For this, I would like to add an additional command line options for the
> {{StandaloneJobClusterEntrypoint}} called {{jobId-seed}}. A manually
> specified jobId would still take precedence (not breaking current behavior):
> * --jobId
> * --jobId-seed
> * (default)
> On Kubernetes, this new command line argument would need to be set to a
> property which is stable over Pod Restarts but changes for different K8s Jobs
> or K8s Deployments.
> *Open Questions*
> * what available information to use a seed for deployments on Kubernetes?
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (FLINK-12752) Add Option to Pass Seed for JobID Hash for StandaloneJobClusterEntrypoint
[
https://issues.apache.org/jira/browse/FLINK-12752?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16869378#comment-16869378
]
Alex commented on FLINK-12752:
--
If a user have to specify --{{jobId-seed}} manually, wouldn't it be equivalent
to ask the user to manually specify --{{jobId}} itself (aside of convenience,
that seed is in free-form format)?
_Offtopic:_ [~knaufk], would you be interested to extract the fixes of
end-to-end tests in your PR as a separate PR? That way, the fixes can be merged
faster and this should help running more nightly CI tests.
> Add Option to Pass Seed for JobID Hash for StandaloneJobClusterEntrypoint
> -
>
> Key: FLINK-12752
> URL: https://issues.apache.org/jira/browse/FLINK-12752
> Project: Flink
> Issue Type: Sub-task
> Components: Deployment / Docker
>Reporter: Konstantin Knauf
>Assignee: Konstantin Knauf
>Priority: Major
> Labels: pull-request-available
> Time Spent: 20m
> Remaining Estimate: 0h
>
> In case of a standalone job cluster setup, we would like to generate random a
> {{JobID}} for every job, but the {{JobID}} nevertheless needs to stay
> constant between JobManager process restarts.
> For this, I would like to add an additional command line options for the
> {{StandaloneJobClusterEntrypoint}} called {{jobId-seed}}. A manually
> specified jobId would still take precedence (not breaking current behavior):
> * --jobId
> * --jobId-seed
> * (default)
> On Kubernetes, this new command line argument would need to be set to a
> property which is stable over Pod Restarts but changes for different K8s Jobs
> or K8s Deployments.
> *Open Questions*
> * what available information to use a seed for deployments on Kubernetes?
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (FLINK-12752) Add Option to Pass Seed for JobID Hash for StandaloneJobClusterEntrypoint
[
https://issues.apache.org/jira/browse/FLINK-12752?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16857851#comment-16857851
]
Konstantin Knauf commented on FLINK-12752:
--
I had a quick chat with [~plucas] offline and it seems there is not really
anything in Kubernetes, which we could use. So, the plan would be for the time
being to specify the {{jobId-seed}} in the pod spec of the K8s Deployment/Job
explicitly.
> Add Option to Pass Seed for JobID Hash for StandaloneJobClusterEntrypoint
> -
>
> Key: FLINK-12752
> URL: https://issues.apache.org/jira/browse/FLINK-12752
> Project: Flink
> Issue Type: Sub-task
>Reporter: Konstantin Knauf
>Assignee: Konstantin Knauf
>Priority: Major
>
> In case of a standalone job cluster setup, we would like to generate random a
> {{JobID}} for every job, but the {{JobID}} nevertheless needs to stay
> constant between JobManager process restarts.
> For this, I would like to add an additional command line options for the
> {{StandaloneJobClusterEntrypoint}} called {{jobId-seed}}. A manually
> specified jobId would still take precedence (not breaking current behavior):
> * --jobId
> * --jobId-seed
> * (default)
> On Kubernetes, this new command line argument would need to be set to a
> property which is stable over Pod Restarts but changes for different K8s Jobs
> or K8s Deployments.
> *Open Questions*
> * what available information to use a seed for deployments on Kubernetes?
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
