[jira] [Comment Edited] (YARN-7929) SLS supports setting container execution
[ https://issues.apache.org/jira/browse/YARN-7929?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16376667#comment-16376667 ] Jiandan Yang edited comment on YARN-7929 at 2/26/18 11:02 AM: --- Thanks very much for review, [~cheersyang] I totally agree with you about ContainerSimulator(1), NMSimulator(1,3), and Misc(1,2), and I will fix them latter. NMSimulator line 139 is not a valid check, it need to send nodeUtilization when resourceUtilizationRatio is between 0 and 1, and it does not need to send nodeUtilization when it less than 0. "safe cast from float to int" do you mean to use Math.around()? SynthTask's construction is only used in SynthJob. No one uses it if keeping the old constructor. was (Author: yangjiandan): Thanks very much for review, [~cheersyang] I totally agree with you about ContainerSimulator(1), NMSimulator(1,3), and Misc(2), and I will fix them latter. NMSimulator line 139 is not a valid check, it need to send nodeUtilization when resourceUtilizationRatio is between 0 and 1, and it does not need to send nodeUtilization when it less than 0. "safe cast from float to int" do you mean to use Math.around()? SynthTask's construction is only used in SynthJob. No one uses it if keeping the old constructor. the update of syn.json, syn_generic.json and syn_stream.json is to test adding execution type, setting opportunistic due to default execution type is guaranteed. > SLS supports setting container execution > > > Key: YARN-7929 > URL: https://issues.apache.org/jira/browse/YARN-7929 > Project: Hadoop YARN > Issue Type: Sub-task > Components: scheduler-load-simulator >Reporter: Jiandan Yang >Assignee: Jiandan Yang >Priority: Major > Attachments: YARN-7929.001.patch, YARN-7929.002.patch > > > SLS currently support three tracetype, SYNTH, SLS and RUMEN, but trace file > can not set execution type of container. > This jira will introduce execution type in SLS to help better simulation. > This will help the perf testing with regarding to the Opportunistic > Containers. > RUMEN has default execution type GUARANTEED > SYNTH set execution type by field map_execution_type and > reduce_execution_type > SLS set execution type by field container.execution_type > For compatibility set GUARANTEED as default value when not setting above > fields in trace file -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Comment Edited] (YARN-7929) SLS supports setting container execution
[ https://issues.apache.org/jira/browse/YARN-7929?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16374021#comment-16374021 ] Jiandan Yang edited comment on YARN-7929 at 2/24/18 1:50 AM: -- Hi [~youchen], thanks for your attention. I did encounter the issue of merging failed when I pull latest code in my local develop environment. I will upload a new patch based on latest code. "water level" to the NMSimulator simulates actual resource utilization, the scheduling of OPPORTUNISTIC containers through the central RM need actual node utilization according to design doc in YARN-1011. was (Author: yangjiandan): Hi [~yochen], thanks for your attention. I did encounter the issue of merging failed when I pull latest code in my local develop environment. I will upload a new patch based on latest code. "water level" to the NMSimulator simulates actual resource utilization, the scheduling of OPPORTUNISTIC containers through the central RM need actual node utilization according to design doc in YARN-1011. > SLS supports setting container execution > > > Key: YARN-7929 > URL: https://issues.apache.org/jira/browse/YARN-7929 > Project: Hadoop YARN > Issue Type: Sub-task > Components: scheduler-load-simulator >Reporter: Jiandan Yang >Assignee: Jiandan Yang >Priority: Major > Attachments: YARN-7929.001.patch, YARN-7929.002.patch > > > SLS currently support three tracetype, SYNTH, SLS and RUMEN, but trace file > can not set execution type of container. > This jira will introduce execution type in SLS to help better simulation. > This will help the perf testing with regarding to the Opportunistic > Containers. > RUMEN has default execution type GUARANTEED > SYNTH set execution type by field map_execution_type and > reduce_execution_type > SLS set execution type by field container.execution_type > For compatibility set GUARANTEED as default value when not setting above > fields in trace file -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Comment Edited] (YARN-7929) SLS supports setting container execution
[ https://issues.apache.org/jira/browse/YARN-7929?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16374021#comment-16374021 ] Jiandan Yang edited comment on YARN-7929 at 2/23/18 7:09 AM: -- Hi [~yochen], thanks for your attention. I did encounter the issue of merging failed when I pull latest code in my local develop environment. I will upload a new patch based on latest code. "water level" to the NMSimulator simulates actual resource utilization, the scheduling of OPPORTUNISTIC containers through the central RM need actual node utilization according to design doc in YARN-1011. was (Author: yangjiandan): Hi [~yochen], thanks for your attention. I did encounter the issue of merging failed when I pull latest code in my local develop environment. I will upload a new patch based latest code. "water level" to the NMSimulator simulates actual resource utilization, the scheduling of OPPORTUNISTIC containers through the central RM need actual node utilization according to design doc in YARN-1011. > SLS supports setting container execution > > > Key: YARN-7929 > URL: https://issues.apache.org/jira/browse/YARN-7929 > Project: Hadoop YARN > Issue Type: Sub-task > Components: scheduler-load-simulator >Reporter: Jiandan Yang >Assignee: Jiandan Yang >Priority: Major > Attachments: YARN-7929.001.patch > > > SLS currently support three tracetype, SYNTH, SLS and RUMEN, but trace file > can not set execution type of container. > This jira will introduce execution type in SLS to help better simulation. > This will help the perf testing with regarding to the Opportunistic > Containers. > RUMEN has default execution type GUARANTEED > SYNTH set execution type by field map_execution_type and > reduce_execution_type > SLS set execution type by field container.execution_type > For compatibility set GUARANTEED as default value when not setting above > fields in trace file -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
