[jira] [Updated] (FLINK-16548) Expose consistent environment variable to identify the component name and resource id of jm/tm
[
https://issues.apache.org/jira/browse/FLINK-16548?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
lincoln lee updated FLINK-16548:
Fix Version/s: (was: 1.19.0)
> Expose consistent environment variable to identify the component name and
> resource id of jm/tm
> --
>
> Key: FLINK-16548
> URL: https://issues.apache.org/jira/browse/FLINK-16548
> Project: Flink
> Issue Type: Improvement
> Components: Runtime / Configuration
>Reporter: hejianchao
>Priority: Minor
> Labels: auto-deprioritized-major
> Fix For: 1.20.0
>
>
> We proposed to expose environment variable to identify the component name and
> resource id of jm/tm.
> To be specified:
> - Expose {{FLINK_COMPONENT_NAME}}. For jm, it should be "jobmanager". For tm,
> it should be "taskexecutor".
> - Expose {{FLINK_COMPONENT_ID}}. For jm/tm, it should be the resource id.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (FLINK-16548) Expose consistent environment variable to identify the component name and resource id of jm/tm
[
https://issues.apache.org/jira/browse/FLINK-16548?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
lincoln lee updated FLINK-16548:
Fix Version/s: 1.20.0
> Expose consistent environment variable to identify the component name and
> resource id of jm/tm
> --
>
> Key: FLINK-16548
> URL: https://issues.apache.org/jira/browse/FLINK-16548
> Project: Flink
> Issue Type: Improvement
> Components: Runtime / Configuration
>Reporter: hejianchao
>Priority: Minor
> Labels: auto-deprioritized-major
> Fix For: 1.19.0, 1.20.0
>
>
> We proposed to expose environment variable to identify the component name and
> resource id of jm/tm.
> To be specified:
> - Expose {{FLINK_COMPONENT_NAME}}. For jm, it should be "jobmanager". For tm,
> it should be "taskexecutor".
> - Expose {{FLINK_COMPONENT_ID}}. For jm/tm, it should be the resource id.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (FLINK-16548) Expose consistent environment variable to identify the component name and resource id of jm/tm
[
https://issues.apache.org/jira/browse/FLINK-16548?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jing Ge updated FLINK-16548:
Fix Version/s: 1.19.0
(was: 1.18.0)
> Expose consistent environment variable to identify the component name and
> resource id of jm/tm
> --
>
> Key: FLINK-16548
> URL: https://issues.apache.org/jira/browse/FLINK-16548
> Project: Flink
> Issue Type: Improvement
> Components: Runtime / Configuration
>Reporter: hejianchao
>Priority: Minor
> Labels: auto-deprioritized-major
> Fix For: 1.19.0
>
>
> We proposed to expose environment variable to identify the component name and
> resource id of jm/tm.
> To be specified:
> - Expose {{FLINK_COMPONENT_NAME}}. For jm, it should be "jobmanager". For tm,
> it should be "taskexecutor".
> - Expose {{FLINK_COMPONENT_ID}}. For jm/tm, it should be the resource id.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (FLINK-16548) Expose consistent environment variable to identify the component name and resource id of jm/tm
[
https://issues.apache.org/jira/browse/FLINK-16548?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xintong Song updated FLINK-16548:
-
Fix Version/s: 1.18.0
(was: 1.17.0)
> Expose consistent environment variable to identify the component name and
> resource id of jm/tm
> --
>
> Key: FLINK-16548
> URL: https://issues.apache.org/jira/browse/FLINK-16548
> Project: Flink
> Issue Type: Improvement
> Components: Runtime / Configuration
>Reporter: hejianchao
>Priority: Minor
> Labels: auto-deprioritized-major
> Fix For: 1.18.0
>
>
> We proposed to expose environment variable to identify the component name and
> resource id of jm/tm.
> To be specified:
> - Expose {{FLINK_COMPONENT_NAME}}. For jm, it should be "jobmanager". For tm,
> it should be "taskexecutor".
> - Expose {{FLINK_COMPONENT_ID}}. For jm/tm, it should be the resource id.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (FLINK-16548) Expose consistent environment variable to identify the component name and resource id of jm/tm
[
https://issues.apache.org/jira/browse/FLINK-16548?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Godfrey He updated FLINK-16548:
---
Fix Version/s: 1.17.0
(was: 1.16.0)
> Expose consistent environment variable to identify the component name and
> resource id of jm/tm
> --
>
> Key: FLINK-16548
> URL: https://issues.apache.org/jira/browse/FLINK-16548
> Project: Flink
> Issue Type: Improvement
> Components: Runtime / Configuration
>Reporter: hejianchao
>Priority: Minor
> Labels: auto-deprioritized-major
> Fix For: 1.17.0
>
>
> We proposed to expose environment variable to identify the component name and
> resource id of jm/tm.
> To be specified:
> - Expose {{FLINK_COMPONENT_NAME}}. For jm, it should be "jobmanager". For tm,
> it should be "taskexecutor".
> - Expose {{FLINK_COMPONENT_ID}}. For jm/tm, it should be the resource id.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (FLINK-16548) Expose consistent environment variable to identify the component name and resource id of jm/tm
[
https://issues.apache.org/jira/browse/FLINK-16548?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Yun Gao updated FLINK-16548:
Fix Version/s: 1.16.0
> Expose consistent environment variable to identify the component name and
> resource id of jm/tm
> --
>
> Key: FLINK-16548
> URL: https://issues.apache.org/jira/browse/FLINK-16548
> Project: Flink
> Issue Type: Improvement
> Components: Runtime / Configuration
>Reporter: hejianchao
>Priority: Minor
> Labels: auto-deprioritized-major
> Fix For: 1.15.0, 1.16.0
>
>
> We proposed to expose environment variable to identify the component name and
> resource id of jm/tm.
> To be specified:
> - Expose {{FLINK_COMPONENT_NAME}}. For jm, it should be "jobmanager". For tm,
> it should be "taskexecutor".
> - Expose {{FLINK_COMPONENT_ID}}. For jm/tm, it should be the resource id.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[jira] [Updated] (FLINK-16548) Expose consistent environment variable to identify the component name and resource id of jm/tm
[
https://issues.apache.org/jira/browse/FLINK-16548?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xintong Song updated FLINK-16548:
-
Fix Version/s: (was: 1.14.0)
1.15.0
> Expose consistent environment variable to identify the component name and
> resource id of jm/tm
> --
>
> Key: FLINK-16548
> URL: https://issues.apache.org/jira/browse/FLINK-16548
> Project: Flink
> Issue Type: Improvement
> Components: Runtime / Configuration
>Reporter: hejianchao
>Priority: Minor
> Labels: auto-deprioritized-major
> Fix For: 1.15.0
>
>
> We proposed to expose environment variable to identify the component name and
> resource id of jm/tm.
> To be specified:
> - Expose {{FLINK_COMPONENT_NAME}}. For jm, it should be "jobmanager". For tm,
> it should be "taskexecutor".
> - Expose {{FLINK_COMPONENT_ID}}. For jm/tm, it should be the resource id.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (FLINK-16548) Expose consistent environment variable to identify the component name and resource id of jm/tm
[
https://issues.apache.org/jira/browse/FLINK-16548?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Flink Jira Bot updated FLINK-16548:
---
Labels: auto-deprioritized-major (was: stale-major)
Priority: Minor (was: Major)
This issue was labeled "stale-major" 7 ago and has not received any updates so
it is being deprioritized. If this ticket is actually Major, please raise the
priority and ask a committer to assign you the issue or revive the public
discussion.
> Expose consistent environment variable to identify the component name and
> resource id of jm/tm
> --
>
> Key: FLINK-16548
> URL: https://issues.apache.org/jira/browse/FLINK-16548
> Project: Flink
> Issue Type: Improvement
> Components: Runtime / Configuration
>Reporter: hejianchao
>Priority: Minor
> Labels: auto-deprioritized-major
> Fix For: 1.14.0
>
>
> We proposed to expose environment variable to identify the component name and
> resource id of jm/tm.
> To be specified:
> - Expose {{FLINK_COMPONENT_NAME}}. For jm, it should be "jobmanager". For tm,
> it should be "taskexecutor".
> - Expose {{FLINK_COMPONENT_ID}}. For jm/tm, it should be the resource id.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (FLINK-16548) Expose consistent environment variable to identify the component name and resource id of jm/tm
[
https://issues.apache.org/jira/browse/FLINK-16548?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Dawid Wysakowicz updated FLINK-16548:
-
Fix Version/s: (was: 1.13.0)
1.14.0
> Expose consistent environment variable to identify the component name and
> resource id of jm/tm
> --
>
> Key: FLINK-16548
> URL: https://issues.apache.org/jira/browse/FLINK-16548
> Project: Flink
> Issue Type: Improvement
> Components: Runtime / Configuration
>Reporter: hejianchao
>Priority: Major
> Labels: stale-major
> Fix For: 1.14.0
>
>
> We proposed to expose environment variable to identify the component name and
> resource id of jm/tm.
> To be specified:
> - Expose {{FLINK_COMPONENT_NAME}}. For jm, it should be "jobmanager". For tm,
> it should be "taskexecutor".
> - Expose {{FLINK_COMPONENT_ID}}. For jm/tm, it should be the resource id.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (FLINK-16548) Expose consistent environment variable to identify the component name and resource id of jm/tm
[
https://issues.apache.org/jira/browse/FLINK-16548?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Flink Jira Bot updated FLINK-16548:
---
Labels: stale-major (was: )
> Expose consistent environment variable to identify the component name and
> resource id of jm/tm
> --
>
> Key: FLINK-16548
> URL: https://issues.apache.org/jira/browse/FLINK-16548
> Project: Flink
> Issue Type: Improvement
> Components: Runtime / Configuration
>Reporter: hejianchao
>Priority: Major
> Labels: stale-major
> Fix For: 1.13.0
>
>
> We proposed to expose environment variable to identify the component name and
> resource id of jm/tm.
> To be specified:
> - Expose {{FLINK_COMPONENT_NAME}}. For jm, it should be "jobmanager". For tm,
> it should be "taskexecutor".
> - Expose {{FLINK_COMPONENT_ID}}. For jm/tm, it should be the resource id.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (FLINK-16548) Expose consistent environment variable to identify the component name and resource id of jm/tm
[
https://issues.apache.org/jira/browse/FLINK-16548?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Robert Metzger updated FLINK-16548:
---
Fix Version/s: (was: 1.12.0)
1.13.0
> Expose consistent environment variable to identify the component name and
> resource id of jm/tm
> --
>
> Key: FLINK-16548
> URL: https://issues.apache.org/jira/browse/FLINK-16548
> Project: Flink
> Issue Type: Improvement
> Components: Runtime / Configuration
>Reporter: hejianchao
>Priority: Major
> Fix For: 1.13.0
>
>
> We proposed to expose environment variable to identify the component name and
> resource id of jm/tm.
> To be specified:
> - Expose {{FLINK_COMPONENT_NAME}}. For jm, it should be "jobmanager". For tm,
> it should be "taskexecutor".
> - Expose {{FLINK_COMPONENT_ID}}. For jm/tm, it should be the resource id.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (FLINK-16548) Expose consistent environment variable to identify the component name and resource id of jm/tm
[
https://issues.apache.org/jira/browse/FLINK-16548?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Yangze Guo updated FLINK-16548:
---
Parent: (was: FLINK-15679)
Issue Type: Improvement (was: Sub-task)
> Expose consistent environment variable to identify the component name and
> resource id of jm/tm
> --
>
> Key: FLINK-16548
> URL: https://issues.apache.org/jira/browse/FLINK-16548
> Project: Flink
> Issue Type: Improvement
> Components: Runtime / Configuration
>Reporter: hejianchao
>Priority: Major
> Fix For: 1.12.0
>
>
> We proposed to expose environment variable to identify the component name and
> resource id of jm/tm.
> To be specified:
> - Expose {{FLINK_COMPONENT_NAME}}. For jm, it should be "jobmanager". For tm,
> it should be "taskexecutor".
> - Expose {{FLINK_COMPONENT_ID}}. For jm/tm, it should be the resource id.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (FLINK-16548) Expose consistent environment variable to identify the component name and resource id of jm/tm
[
https://issues.apache.org/jira/browse/FLINK-16548?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Yangze Guo updated FLINK-16548:
---
Fix Version/s: 1.12.0
> Expose consistent environment variable to identify the component name and
> resource id of jm/tm
> --
>
> Key: FLINK-16548
> URL: https://issues.apache.org/jira/browse/FLINK-16548
> Project: Flink
> Issue Type: Sub-task
> Components: Runtime / Configuration
>Reporter: hejianchao
>Priority: Major
> Fix For: 1.12.0
>
>
> We proposed to expose environment variable to identify the component name and
> resource id of jm/tm.
> To be specified:
> - Expose {{FLINK_COMPONENT_NAME}}. For jm, it should be "jobmanager". For tm,
> it should be "taskexecutor".
> - Expose {{FLINK_COMPONENT_ID}}. For jm/tm, it should be the resource id.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (FLINK-16548) Expose consistent environment variable to identify the component name and resource id of jm/tm
[
https://issues.apache.org/jira/browse/FLINK-16548?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Yangze Guo updated FLINK-16548:
---
Description:
We proposed to expose environment variable to identify the component name and
resource id of jm/tm.
To be specified:
- Expose {{FLINK_COMPONENT_NAME}}. For jm, it should be "jobmanager". For tm,
it should be "taskexecutor".
- Expose {{FLINK_COMPONENT_ID}}. For jm/tm, it should be the resource id.
was:We proposed to expose consistent e
> Expose consistent environment variable to identify the component name and
> resource id of jm/tm
> --
>
> Key: FLINK-16548
> URL: https://issues.apache.org/jira/browse/FLINK-16548
> Project: Flink
> Issue Type: Sub-task
> Components: Runtime / Configuration
>Reporter: hejianchao
>Priority: Major
>
> We proposed to expose environment variable to identify the component name and
> resource id of jm/tm.
> To be specified:
> - Expose {{FLINK_COMPONENT_NAME}}. For jm, it should be "jobmanager". For tm,
> it should be "taskexecutor".
> - Expose {{FLINK_COMPONENT_ID}}. For jm/tm, it should be the resource id.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (FLINK-16548) Expose consistent environment variable to identify the component name and resource id of jm/tm
[
https://issues.apache.org/jira/browse/FLINK-16548?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Yangze Guo updated FLINK-16548:
---
Description: We proposed to expose consistent e (was: Imagine that there
is now s3-log-appender.jar, and logs can be written to s3 with proper
configuration.
Then, in the same flink job, the exact same log4j configuration is used. How to
make the key written to s3 for different jm / tm carry the id that uniquely
identifies jm / tm? such as:
s3://bucket/flink-logs/jobId/${jm-id}/logfile
s3://bucket/flink-logs/jobId/${tm-1-id}/logfile
s3://bucket/flink-logs/jobId/${tm-2-id}/logfile
For the s3-log-appender here, it needs a way to sense the value of
${jm-id}/${tm-1-id}/${tm-2-id}, and it has the following claims :
1. This value can be used to locate the actual running jm, tm
2. The value is independent of cluster mode, whether standalone, yarn or
kubernetes)
> Expose consistent environment variable to identify the component name and
> resource id of jm/tm
> --
>
> Key: FLINK-16548
> URL: https://issues.apache.org/jira/browse/FLINK-16548
> Project: Flink
> Issue Type: Sub-task
> Components: Runtime / Configuration
>Reporter: hejianchao
>Priority: Major
>
> We proposed to expose consistent e
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (FLINK-16548) Expose consistent environment variable to identify the component name and resource id of jm/tm
[
https://issues.apache.org/jira/browse/FLINK-16548?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Yangze Guo updated FLINK-16548:
---
Summary: Expose consistent environment variable to identify the component
name and resource id of jm/tm (was: [Requirement] Consistent id to identify
jm/tm in different cluster modes)
> Expose consistent environment variable to identify the component name and
> resource id of jm/tm
> --
>
> Key: FLINK-16548
> URL: https://issues.apache.org/jira/browse/FLINK-16548
> Project: Flink
> Issue Type: Sub-task
> Components: Runtime / Configuration
>Reporter: hejianchao
>Priority: Major
>
> Imagine that there is now s3-log-appender.jar, and logs can be written to s3
> with proper configuration.
> Then, in the same flink job, the exact same log4j configuration is used. How
> to make the key written to s3 for different jm / tm carry the id that
> uniquely identifies jm / tm? such as:
> s3://bucket/flink-logs/jobId/${jm-id}/logfile
> s3://bucket/flink-logs/jobId/${tm-1-id}/logfile
> s3://bucket/flink-logs/jobId/${tm-2-id}/logfile
> For the s3-log-appender here, it needs a way to sense the value of
> ${jm-id}/${tm-1-id}/${tm-2-id}, and it has the following claims :
> 1. This value can be used to locate the actual running jm, tm
> 2. The value is independent of cluster mode, whether standalone, yarn or
> kubernetes
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
