[jira] [Commented] (FLINK-13548) Support priority of the Flink YARN application
[ https://issues.apache.org/jira/browse/FLINK-13548?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16901753#comment-16901753 ] thy commented on FLINK-13548: - ok, I currently have pushed a new commit according to the last discussion~ > Support priority of the Flink YARN application > -- > > Key: FLINK-13548 > URL: https://issues.apache.org/jira/browse/FLINK-13548 > Project: Flink > Issue Type: Improvement > Components: Deployment / YARN >Reporter: thy >Priority: Major > Labels: pull-request-available > Time Spent: 10m > Remaining Estimate: 0h > > Currently, Flink 1.9 does not support yarn priority submission. The default > priority of yarn submission jobs is based on YARN official doc. > Based on this, we can provide a ConfigOption in YarnConfigOptions. The > priority value is non-negative, the bigger the number, the higher the > priority. By default, we take -1. When the priority is negative, we use > default yarn queue priority. -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[jira] [Updated] (FLINK-13548) Support priority of the Flink YARN application
[ https://issues.apache.org/jira/browse/FLINK-13548?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] thy updated FLINK-13548: Description: Currently, Flink 1.9 does not support yarn priority submission. The default priority of yarn submission jobs is based on YARN official doc. Based on this, we can provide a ConfigOption in YarnConfigOptions. The priority value is non-negative, the bigger the number, the higher the priority. By default, we take -1. When the priority is negative, we use default yarn queue priority. was: Currently, Flink 1.9 does not support yarn priority submission. The default priority of yarn submission jobs is based on [YARN official doc|[https://hadoop.apache.org/docs/current/hadoop-yarn/hadoop-yarn-site/CapacityScheduler.html#Setup_for_application_priority]]. Based on this, we can provide a ConfigOption in YarnConfigOptions. The priority value is non-negative, the bigger the number, the higher the priority. By default, we take -1. When the priority is negative, we use default yarn queue priority. > Support priority of the Flink YARN application > -- > > Key: FLINK-13548 > URL: https://issues.apache.org/jira/browse/FLINK-13548 > Project: Flink > Issue Type: Improvement > Components: Deployment / YARN >Reporter: thy >Priority: Major > Labels: pull-request-available > Time Spent: 10m > Remaining Estimate: 0h > > Currently, Flink 1.9 does not support yarn priority submission. The default > priority of yarn submission jobs is based on YARN official doc. > Based on this, we can provide a ConfigOption in YarnConfigOptions. The > priority value is non-negative, the bigger the number, the higher the > priority. By default, we take -1. When the priority is negative, we use > default yarn queue priority. -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[jira] [Updated] (FLINK-13548) Support priority of the Flink YARN application
[ https://issues.apache.org/jira/browse/FLINK-13548?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] thy updated FLINK-13548: Description: Currently, Flink 1.9 does not support yarn priority submission. The default priority of yarn submission jobs is based on [YARN official doc|[https://hadoop.apache.org/docs/current/hadoop-yarn/hadoop-yarn-site/CapacityScheduler.html#Setup_for_application_priority]]. Based on this, we can provide a ConfigOption in YarnConfigOptions. The priority value is non-negative, the bigger the number, the higher the priority. By default, we take -1. When the priority is negative, we use default yarn queue priority. was: Currently, Flink 1.9 does not support yarn priority submission. The default priority of yarn submission jobs is based on yarn. Based on this, we can provide a ConfigOption in YarnConfigOptions. The priority is non negative, the bigger the number, the higher the priority. By default, we take -1. When the priority is negative, we use default yarn queue priority. > Support priority of the Flink YARN application > -- > > Key: FLINK-13548 > URL: https://issues.apache.org/jira/browse/FLINK-13548 > Project: Flink > Issue Type: Improvement > Components: Deployment / YARN >Reporter: thy >Priority: Major > Labels: pull-request-available > Time Spent: 10m > Remaining Estimate: 0h > > Currently, Flink 1.9 does not support yarn priority submission. The default > priority of yarn submission jobs is based on [YARN official > doc|[https://hadoop.apache.org/docs/current/hadoop-yarn/hadoop-yarn-site/CapacityScheduler.html#Setup_for_application_priority]]. > Based on this, we can provide a ConfigOption in YarnConfigOptions. The > priority value is non-negative, the bigger the number, the higher the > priority. By default, we take -1. When the priority is negative, we use > default yarn queue priority. -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[jira] [Updated] (FLINK-13548) Support priority of the Flink YARN application
[ https://issues.apache.org/jira/browse/FLINK-13548?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] thy updated FLINK-13548: Description: Currently, Flink 1.9 does not support yarn priority submission. The default priority of yarn submission jobs is based on yarn. Based on this, we can provide a ConfigOption in YarnConfigOptions. The priority is non negative, the bigger the number, the higher the priority. By default, we take -1. When the priority is negative, we use default yarn queue priority. was: Currently, Flink 1.9 does not support yarn priority submission. The default priority of yarn submission jobs is 1. Based on this, we can provide a ConfigOption in YarnConfigOptions. The submission of the priority range from 1 to 9(from low to high priority). When the user does not configure the parameters, the priority is still 1. > Support priority of the Flink YARN application > -- > > Key: FLINK-13548 > URL: https://issues.apache.org/jira/browse/FLINK-13548 > Project: Flink > Issue Type: Improvement > Components: Deployment / YARN >Reporter: thy >Priority: Major > Labels: pull-request-available > Time Spent: 10m > Remaining Estimate: 0h > > Currently, Flink 1.9 does not support yarn priority submission. The default > priority of yarn submission jobs is based on yarn. > Based on this, we can provide a ConfigOption in YarnConfigOptions. The > priority is non negative, the bigger the number, the higher the priority. By > default, we take -1. When the priority is negative, we use default yarn queue > priority. -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[jira] [Commented] (FLINK-13548) Support priority of the Flink YARN application
[ https://issues.apache.org/jira/browse/FLINK-13548?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16900977#comment-16900977 ] thy commented on FLINK-13548: - [~walterddr]: Yes, it is better to do a sanity check. Do you have any good idea to get the yarn priority info? Thanks~ > Support priority of the Flink YARN application > -- > > Key: FLINK-13548 > URL: https://issues.apache.org/jira/browse/FLINK-13548 > Project: Flink > Issue Type: Improvement > Components: Deployment / YARN >Reporter: thy >Priority: Major > Labels: pull-request-available > Time Spent: 10m > Remaining Estimate: 0h > > Currently, Flink 1.9 does not support yarn priority submission. The default > priority of yarn submission jobs is 1. > Based on this, we can provide a ConfigOption in YarnConfigOptions. The > submission of the priority range from 1 to 9(from low to high priority). When > the user does not configure the parameters, the priority is still 1. -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[jira] [Commented] (FLINK-13548) Support priority of the Flink YARN application
[ https://issues.apache.org/jira/browse/FLINK-13548?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16900099#comment-16900099 ] thy commented on FLINK-13548: - [~walterddr]: Thanks for your advice! First, I would say sorry that I made a mistake between our internal spark and community spark. The internal spark is implemented as what I talked above. Actually the community spark does not implement the yarn application priority. 1. As mentioned in YARN official doc about "Cluster-level priority": Any application submitted with a priority greater than the cluster-max priority will have its priority reset to the cluster-max priority. I think maybe it is not necessary to limit 1-9, just make sure that the priority is not negative. 2. The queue’s default priority will be used for any application submitted without a specified priority. An idea to envision the default value is that we set the default YARN PRIORITY OPTION to -1. Once we get -1 from the option, we do not set "appcontext.setPriority". Maybe there is a better choice? Thanks thy > Support priority of the Flink YARN application > -- > > Key: FLINK-13548 > URL: https://issues.apache.org/jira/browse/FLINK-13548 > Project: Flink > Issue Type: Improvement > Components: Deployment / YARN >Reporter: thy >Priority: Major > Labels: pull-request-available > Time Spent: 10m > Remaining Estimate: 0h > > Currently, Flink 1.9 does not support yarn priority submission. The default > priority of yarn submission jobs is 1. > Based on this, we can provide a ConfigOption in YarnConfigOptions. The > submission of the priority range from 1 to 9(from low to high priority). When > the user does not configure the parameters, the priority is still 1. -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[jira] [Created] (FLINK-13548) Support priority of the Flink YARN application
thy created FLINK-13548: --- Summary: Support priority of the Flink YARN application Key: FLINK-13548 URL: https://issues.apache.org/jira/browse/FLINK-13548 Project: Flink Issue Type: Improvement Components: Deployment / YARN Reporter: thy Currently, Flink 1.9 does not support yarn priority submission. The default priority of yarn submission jobs is 1. Based on this, we can provide a ConfigOption in YarnConfigOptions. The submission of the priority range from 1 to 9(from low to high priority). When the user does not configure the parameters, the priority is still 1. -- This message was sent by Atlassian JIRA (v7.6.14#76016)
