[jira] [Created] (YARN-9706) [UI2] App Attempt state missing from Graph view
Yesha Vora created YARN-9706: Summary: [UI2] App Attempt state missing from Graph view Key: YARN-9706 URL: https://issues.apache.org/jira/browse/YARN-9706 Project: Hadoop YARN Issue Type: Bug Reporter: Yesha Vora 1) Launch a Dshell application or Yarn service application. 2) Go to app attempt page Grid view. State column shows N/A. 3) Go to app attempt Graph view. State data is not present on this page. Apparently, app attempt data is only shown in Grid view. Grid and Graph view should show the same details. -- This message was sent by Atlassian JIRA (v7.6.14#76016) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Created] (YARN-9707) [UI2] App Attempt state data is missing
Yesha Vora created YARN-9707:
Summary: [UI2] App Attempt state data is missing
Key: YARN-9707
URL: https://issues.apache.org/jira/browse/YARN-9707
Project: Hadoop YARN
Issue Type: Bug
Reporter: Yesha Vora
Steps:
1) Launch a Dshell application or Yarn service application.
2) Go to app attempt page Grid view. State column shows N/A.
Yarn UI1 shows app attempt state for Running and Finished application. This
ability is missing from UI2.
UI2 is using below rest call. This rest call does not show the app attempt
state details.
{code:title=ws/v1/cluster/apps/application_1563946396350_0002/appattempts?_=1564004553389}
115640045242901564004541852container_1563946396350_0002_01_01xx:yyxx:yyhttp://ixx:yy/node/containerlogs/container_1563946396350_0002_01_01/hrt_qaappattempt_1563946396350_0002_01{code}
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Created] (YARN-9705) [UI2] AM Node Web UI should not display full link
Yesha Vora created YARN-9705: Summary: [UI2] AM Node Web UI should not display full link Key: YARN-9705 URL: https://issues.apache.org/jira/browse/YARN-9705 Project: Hadoop YARN Issue Type: Bug Reporter: Yesha Vora App Attempt page shows AM Node Web UI. It displays the full link. It should not print out full link as display text. Rather, It should use display AM Node name which links to the node. -- This message was sent by Atlassian JIRA (v7.6.14#76016) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-9704) [UI2] Fix Pending, Allocated, Reserved Containers information for Fair Scheduler
[ https://issues.apache.org/jira/browse/YARN-9704?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yesha Vora updated YARN-9704: - Attachment: Screenshot 2019-07-25 at 10.51.08 PM.png > [UI2] Fix Pending, Allocated, Reserved Containers information for Fair > Scheduler > > > Key: YARN-9704 > URL: https://issues.apache.org/jira/browse/YARN-9704 > Project: Hadoop YARN > Issue Type: Bug >Reporter: Yesha Vora >Priority: Critical > Attachments: Screenshot 2019-07-25 at 10.51.08 PM.png > > > YARN UI2 shows "Pending, Allocated, Reserved Containers" information for fair > scheduler. In here, pending container information is not printed. UI2 shows > ",0,0" instead "0,0,0". > In UI1, This same information is displayed as Num of active Application & > Pending applications. > Num Active Applications: 0 > Num Pending Applications: 0 > It's not clear from UI2 what do we intend to show in "Pending, Allocated, > Reserved Containers"? Is it really containers or apps? -- This message was sent by Atlassian JIRA (v7.6.14#76016) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Created] (YARN-9704) [UI2] Fix Pending, Allocated, Reserved Containers information for Fair Scheduler
Yesha Vora created YARN-9704: Summary: [UI2] Fix Pending, Allocated, Reserved Containers information for Fair Scheduler Key: YARN-9704 URL: https://issues.apache.org/jira/browse/YARN-9704 Project: Hadoop YARN Issue Type: Bug Reporter: Yesha Vora YARN UI2 shows "Pending, Allocated, Reserved Containers" information for fair scheduler. In here, pending container information is not printed. UI2 shows ",0,0" instead "0,0,0". In UI1, This same information is displayed as Num of active Application & Pending applications. Num Active Applications:0 Num Pending Applications: 0 It's not clear from UI2 what do we intend to show in "Pending, Allocated, Reserved Containers"? Is it really containers or apps? -- This message was sent by Atlassian JIRA (v7.6.14#76016) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Created] (YARN-9609) Nodemanager Web Service should return logAggregationType for each file
Yesha Vora created YARN-9609: Summary: Nodemanager Web Service should return logAggregationType for each file Key: YARN-9609 URL: https://issues.apache.org/jira/browse/YARN-9609 Project: Hadoop YARN Issue Type: Bug Components: nodemanager Affects Versions: 3.1.1 Reporter: Yesha Vora Steps: 1) Launch sleeper yarn service 2) When sleeper component is in READY state, call NM web service to list the container files and its log aggregation status. http://NMHost:NMPort/ws/v1/node/containers/CONTAINERID/logs NM web service response shows a common log aggregation type response for all files. Instead, NM web service should return a log aggregation type for each file. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Created] (YARN-9570) pplication in pending-ordering-policy is not considered while container allocation
Yesha Vora created YARN-9570: Summary: pplication in pending-ordering-policy is not considered while container allocation Key: YARN-9570 URL: https://issues.apache.org/jira/browse/YARN-9570 Project: Hadoop YARN Issue Type: Bug Components: capacity scheduler Reporter: Yesha Vora This is 5 node cluster with total 15GB capacity. 1) Configure Capacity scheduler and set max cluster priority=10 2) launch app1 with no priority and wait for it to occupy full cluster application_1558135983180_0001 is launched with Priority=0 3) launch app2 with priority=2 and check its in ACCEPTED state application_1558135983180_0002 is launched with Priority=2 4) launch app3 with priority=3 and check its in ACCEPTED state application_1558135983180_0003 is launched with Priority=2 5) kill container from app1 6) Verify app3 with higher priority goes to RUNNING state. When max-application-master-percentage is set to 0.1, app2 goes to RUNNING state even though app3 has higher priority. Root cause: In CS LeafQueue, there's two ordering list: If the queue's total application master usage below maxAMResourcePerQueuePercent, the app will be added to the "ordering-policy" list. Otherwise, the app will be added to the "pending-ordering-policy" list. During allocation, only apps in "ordering-policy" are considered. If there's any app finish, or queue config changed, or node add/remove happen, "pending-ordering-policy" will be reconsidered, and some apps from "pending-ordering-policy" will be added to "ordering-policy". This behavior leads to the issue of this JIRA: The cluster has 15GB resource, the max-application-master-percentage is set to 0.1. So it can at most accept 2GB (rounded by 1GB) AM resource, which equals to 2 applications. When app2 submitted, it will be added to ordering-policy. When app3 submitted, it will be added to pending-ordering-policy. When we kill app1, it won't finish immediately. Instead, it will still be part of "odering-policy" until all containers of app1 released. (That makes app3 stays in pending-ordering-policy). So any resource released by app1, app3 cannot pick up. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-8551) Build Common module for MaWo application
[ https://issues.apache.org/jira/browse/YARN-8551?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yesha Vora updated YARN-8551: - Attachment: YARN-8551.0020.patch > Build Common module for MaWo application > > > Key: YARN-8551 > URL: https://issues.apache.org/jira/browse/YARN-8551 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Yesha Vora >Assignee: Yesha Vora >Priority: Major > Attachments: YARN-8551.001.patch, YARN-8551.0010.patch, > YARN-8551.0011.patch, YARN-8551.0012.patch, YARN-8551.0013.patch, > YARN-8551.0014.patch, YARN-8551.0015.patch, YARN-8551.0016.patch, > YARN-8551.0017.patch, YARN-8551.0018.patch, YARN-8551.0019.patch, > YARN-8551.002.patch, YARN-8551.0020.patch, YARN-8551.003.patch, > YARN-8551.004.patch, YARN-8551.005.patch, YARN-8551.006.patch, > YARN-8551.007.patch, YARN-8551.008.patch, YARN-8551.009.patch > > > Build Common module for MaWo application. > This module should include defination of Task. A Task should contain > * TaskID > * Task Command > * Task Environment > * Task Timeout > * Task Type > ** Simple Task > *** Its a single Task > ** Composite Task > *** Its a composition of multiple simple tasks > ** Teardown Task > *** Its a last task to be executed after a job is finished > ** Null Task > *** Its a null task -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-8551) Build Common module for MaWo application
[ https://issues.apache.org/jira/browse/YARN-8551?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yesha Vora updated YARN-8551: - Attachment: YARN-8551.0019.patch > Build Common module for MaWo application > > > Key: YARN-8551 > URL: https://issues.apache.org/jira/browse/YARN-8551 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Yesha Vora >Assignee: Yesha Vora >Priority: Major > Attachments: YARN-8551.001.patch, YARN-8551.0010.patch, > YARN-8551.0011.patch, YARN-8551.0012.patch, YARN-8551.0013.patch, > YARN-8551.0014.patch, YARN-8551.0015.patch, YARN-8551.0016.patch, > YARN-8551.0017.patch, YARN-8551.0018.patch, YARN-8551.0019.patch, > YARN-8551.002.patch, YARN-8551.003.patch, YARN-8551.004.patch, > YARN-8551.005.patch, YARN-8551.006.patch, YARN-8551.007.patch, > YARN-8551.008.patch, YARN-8551.009.patch > > > Build Common module for MaWo application. > This module should include defination of Task. A Task should contain > * TaskID > * Task Command > * Task Environment > * Task Timeout > * Task Type > ** Simple Task > *** Its a single Task > ** Composite Task > *** Its a composition of multiple simple tasks > ** Teardown Task > *** Its a last task to be executed after a job is finished > ** Null Task > *** Its a null task -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-8551) Build Common module for MaWo application
[ https://issues.apache.org/jira/browse/YARN-8551?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yesha Vora updated YARN-8551: - Attachment: YARN-8551.0018.patch > Build Common module for MaWo application > > > Key: YARN-8551 > URL: https://issues.apache.org/jira/browse/YARN-8551 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Yesha Vora >Assignee: Yesha Vora >Priority: Major > Attachments: YARN-8551.001.patch, YARN-8551.0010.patch, > YARN-8551.0011.patch, YARN-8551.0012.patch, YARN-8551.0013.patch, > YARN-8551.0014.patch, YARN-8551.0015.patch, YARN-8551.0016.patch, > YARN-8551.0017.patch, YARN-8551.0018.patch, YARN-8551.002.patch, > YARN-8551.003.patch, YARN-8551.004.patch, YARN-8551.005.patch, > YARN-8551.006.patch, YARN-8551.007.patch, YARN-8551.008.patch, > YARN-8551.009.patch > > > Build Common module for MaWo application. > This module should include defination of Task. A Task should contain > * TaskID > * Task Command > * Task Environment > * Task Timeout > * Task Type > ** Simple Task > *** Its a single Task > ** Composite Task > *** Its a composition of multiple simple tasks > ** Teardown Task > *** Its a last task to be executed after a job is finished > ** Null Task > *** Its a null task -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-8551) Build Common module for MaWo application
[ https://issues.apache.org/jira/browse/YARN-8551?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yesha Vora updated YARN-8551: - Attachment: YARN-8551.0017.patch > Build Common module for MaWo application > > > Key: YARN-8551 > URL: https://issues.apache.org/jira/browse/YARN-8551 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Yesha Vora >Assignee: Yesha Vora >Priority: Major > Attachments: YARN-8551.001.patch, YARN-8551.0010.patch, > YARN-8551.0011.patch, YARN-8551.0012.patch, YARN-8551.0013.patch, > YARN-8551.0014.patch, YARN-8551.0015.patch, YARN-8551.0016.patch, > YARN-8551.0017.patch, YARN-8551.002.patch, YARN-8551.003.patch, > YARN-8551.004.patch, YARN-8551.005.patch, YARN-8551.006.patch, > YARN-8551.007.patch, YARN-8551.008.patch, YARN-8551.009.patch > > > Build Common module for MaWo application. > This module should include defination of Task. A Task should contain > * TaskID > * Task Command > * Task Environment > * Task Timeout > * Task Type > ** Simple Task > *** Its a single Task > ** Composite Task > *** Its a composition of multiple simple tasks > ** Teardown Task > *** Its a last task to be executed after a job is finished > ** Null Task > *** Its a null task -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-8551) Build Common module for MaWo application
[ https://issues.apache.org/jira/browse/YARN-8551?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yesha Vora updated YARN-8551: - Attachment: YARN-8551.0016.patch > Build Common module for MaWo application > > > Key: YARN-8551 > URL: https://issues.apache.org/jira/browse/YARN-8551 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Yesha Vora >Assignee: Yesha Vora >Priority: Major > Attachments: YARN-8551.001.patch, YARN-8551.0010.patch, > YARN-8551.0011.patch, YARN-8551.0012.patch, YARN-8551.0013.patch, > YARN-8551.0014.patch, YARN-8551.0015.patch, YARN-8551.0016.patch, > YARN-8551.002.patch, YARN-8551.003.patch, YARN-8551.004.patch, > YARN-8551.005.patch, YARN-8551.006.patch, YARN-8551.007.patch, > YARN-8551.008.patch, YARN-8551.009.patch > > > Build Common module for MaWo application. > This module should include defination of Task. A Task should contain > * TaskID > * Task Command > * Task Environment > * Task Timeout > * Task Type > ** Simple Task > *** Its a single Task > ** Composite Task > *** Its a composition of multiple simple tasks > ** Teardown Task > *** Its a last task to be executed after a job is finished > ** Null Task > *** Its a null task -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Assigned] (YARN-8466) Add Chaos Monkey unit test framework for feature validation in scale
[ https://issues.apache.org/jira/browse/YARN-8466?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yesha Vora reassigned YARN-8466: Assignee: Yesha Vora > Add Chaos Monkey unit test framework for feature validation in scale > > > Key: YARN-8466 > URL: https://issues.apache.org/jira/browse/YARN-8466 > Project: Hadoop YARN > Issue Type: Task >Reporter: Wangda Tan >Assignee: Yesha Vora >Priority: Critical > Attachments: YARN-8466.poc.001.patch > > > Currently we don't have such framework for testing. > We need a framework to do this. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-8551) Build Common module for MaWo application
[ https://issues.apache.org/jira/browse/YARN-8551?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yesha Vora updated YARN-8551: - Attachment: YARN-8551.0015.patch > Build Common module for MaWo application > > > Key: YARN-8551 > URL: https://issues.apache.org/jira/browse/YARN-8551 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Yesha Vora >Assignee: Yesha Vora >Priority: Major > Attachments: YARN-8551.001.patch, YARN-8551.0010.patch, > YARN-8551.0011.patch, YARN-8551.0012.patch, YARN-8551.0013.patch, > YARN-8551.0014.patch, YARN-8551.0015.patch, YARN-8551.002.patch, > YARN-8551.003.patch, YARN-8551.004.patch, YARN-8551.005.patch, > YARN-8551.006.patch, YARN-8551.007.patch, YARN-8551.008.patch, > YARN-8551.009.patch > > > Build Common module for MaWo application. > This module should include defination of Task. A Task should contain > * TaskID > * Task Command > * Task Environment > * Task Timeout > * Task Type > ** Simple Task > *** Its a single Task > ** Composite Task > *** Its a composition of multiple simple tasks > ** Teardown Task > *** Its a last task to be executed after a job is finished > ** Null Task > *** Its a null task -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-8551) Build Common module for MaWo application
[ https://issues.apache.org/jira/browse/YARN-8551?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yesha Vora updated YARN-8551: - Attachment: YARN-8551.0014.patch > Build Common module for MaWo application > > > Key: YARN-8551 > URL: https://issues.apache.org/jira/browse/YARN-8551 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Yesha Vora >Assignee: Yesha Vora >Priority: Major > Attachments: YARN-8551.001.patch, YARN-8551.0010.patch, > YARN-8551.0011.patch, YARN-8551.0012.patch, YARN-8551.0013.patch, > YARN-8551.0014.patch, YARN-8551.002.patch, YARN-8551.003.patch, > YARN-8551.004.patch, YARN-8551.005.patch, YARN-8551.006.patch, > YARN-8551.007.patch, YARN-8551.008.patch, YARN-8551.009.patch > > > Build Common module for MaWo application. > This module should include defination of Task. A Task should contain > * TaskID > * Task Command > * Task Environment > * Task Timeout > * Task Type > ** Simple Task > *** Its a single Task > ** Composite Task > *** Its a composition of multiple simple tasks > ** Teardown Task > *** Its a last task to be executed after a job is finished > ** Null Task > *** Its a null task -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-9370) Better logging in recoverAssignedGpus in class GpuResourceAllocator
[
https://issues.apache.org/jira/browse/YARN-9370?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Yesha Vora updated YARN-9370:
-
Attachment: YARN-9370.006.patch
> Better logging in recoverAssignedGpus in class GpuResourceAllocator
> ---
>
> Key: YARN-9370
> URL: https://issues.apache.org/jira/browse/YARN-9370
> Project: Hadoop YARN
> Issue Type: Improvement
>Reporter: Szilard Nemeth
>Assignee: Yesha Vora
>Priority: Trivial
> Labels: newbie, newbie++
> Attachments: YARN-9370.001.patch, YARN-9370.002.patch,
> YARN-9370.003.patch, YARN-9370.004.patch, YARN-9370.005.patch,
> YARN-9370.006.patch
>
>
> The last line of
> org.apache.hadoop.yarn.server.nodemanager.containermanager.linux.resources.gpu.GpuResourceAllocator#recoverAssignedGpus
> is this:
> {code:java}
> usedDevices.put(gpuDevice, containerId);
> {code}
> We should have an info (or if not info, at least a debug) level log to
> indicate that a container is allocated to a GPU device during recovery.
> Please also check recovery related code, maybe there can be some room for
> improvement around logging.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-8551) Build Common module for MaWo application
[ https://issues.apache.org/jira/browse/YARN-8551?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yesha Vora updated YARN-8551: - Attachment: YARN-8551.0013.patch > Build Common module for MaWo application > > > Key: YARN-8551 > URL: https://issues.apache.org/jira/browse/YARN-8551 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Yesha Vora >Assignee: Yesha Vora >Priority: Major > Attachments: YARN-8551.001.patch, YARN-8551.0010.patch, > YARN-8551.0011.patch, YARN-8551.0012.patch, YARN-8551.0013.patch, > YARN-8551.002.patch, YARN-8551.003.patch, YARN-8551.004.patch, > YARN-8551.005.patch, YARN-8551.006.patch, YARN-8551.007.patch, > YARN-8551.008.patch, YARN-8551.009.patch > > > Build Common module for MaWo application. > This module should include defination of Task. A Task should contain > * TaskID > * Task Command > * Task Environment > * Task Timeout > * Task Type > ** Simple Task > *** Its a single Task > ** Composite Task > *** Its a composition of multiple simple tasks > ** Teardown Task > *** Its a last task to be executed after a job is finished > ** Null Task > *** Its a null task -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-8551) Build Common module for MaWo application
[ https://issues.apache.org/jira/browse/YARN-8551?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yesha Vora updated YARN-8551: - Attachment: YARN-8551.0012.patch > Build Common module for MaWo application > > > Key: YARN-8551 > URL: https://issues.apache.org/jira/browse/YARN-8551 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Yesha Vora >Assignee: Yesha Vora >Priority: Major > Attachments: YARN-8551.001.patch, YARN-8551.0010.patch, > YARN-8551.0011.patch, YARN-8551.0012.patch, YARN-8551.002.patch, > YARN-8551.003.patch, YARN-8551.004.patch, YARN-8551.005.patch, > YARN-8551.006.patch, YARN-8551.007.patch, YARN-8551.008.patch, > YARN-8551.009.patch > > > Build Common module for MaWo application. > This module should include defination of Task. A Task should contain > * TaskID > * Task Command > * Task Environment > * Task Timeout > * Task Type > ** Simple Task > *** Its a single Task > ** Composite Task > *** Its a composition of multiple simple tasks > ** Teardown Task > *** Its a last task to be executed after a job is finished > ** Null Task > *** Its a null task -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-8551) Build Common module for MaWo application
[ https://issues.apache.org/jira/browse/YARN-8551?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yesha Vora updated YARN-8551: - Attachment: YARN-8551.0011.patch > Build Common module for MaWo application > > > Key: YARN-8551 > URL: https://issues.apache.org/jira/browse/YARN-8551 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Yesha Vora >Assignee: Yesha Vora >Priority: Major > Attachments: YARN-8551.001.patch, YARN-8551.0010.patch, > YARN-8551.0011.patch, YARN-8551.002.patch, YARN-8551.003.patch, > YARN-8551.004.patch, YARN-8551.005.patch, YARN-8551.006.patch, > YARN-8551.007.patch, YARN-8551.008.patch, YARN-8551.009.patch > > > Build Common module for MaWo application. > This module should include defination of Task. A Task should contain > * TaskID > * Task Command > * Task Environment > * Task Timeout > * Task Type > ** Simple Task > *** Its a single Task > ** Composite Task > *** Its a composition of multiple simple tasks > ** Teardown Task > *** Its a last task to be executed after a job is finished > ** Null Task > *** Its a null task -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-9370) Better logging in recoverAssignedGpus in class GpuResourceAllocator
[
https://issues.apache.org/jira/browse/YARN-9370?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Yesha Vora updated YARN-9370:
-
Attachment: YARN-9370.005.patch
> Better logging in recoverAssignedGpus in class GpuResourceAllocator
> ---
>
> Key: YARN-9370
> URL: https://issues.apache.org/jira/browse/YARN-9370
> Project: Hadoop YARN
> Issue Type: Improvement
>Reporter: Szilard Nemeth
>Assignee: Yesha Vora
>Priority: Trivial
> Labels: newbie, newbie++
> Attachments: YARN-9370.001.patch, YARN-9370.002.patch,
> YARN-9370.003.patch, YARN-9370.004.patch, YARN-9370.005.patch
>
>
> The last line of
> org.apache.hadoop.yarn.server.nodemanager.containermanager.linux.resources.gpu.GpuResourceAllocator#recoverAssignedGpus
> is this:
> {code:java}
> usedDevices.put(gpuDevice, containerId);
> {code}
> We should have an info (or if not info, at least a debug) level log to
> indicate that a container is allocated to a GPU device during recovery.
> Please also check recovery related code, maybe there can be some room for
> improvement around logging.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-9370) Better logging in recoverAssignedGpus in class GpuResourceAllocator
[
https://issues.apache.org/jira/browse/YARN-9370?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16795388#comment-16795388
]
Yesha Vora commented on YARN-9370:
--
[~snemeth] / [~eyang]: Thank you for review. New patch with slf4j is updated.
> Better logging in recoverAssignedGpus in class GpuResourceAllocator
> ---
>
> Key: YARN-9370
> URL: https://issues.apache.org/jira/browse/YARN-9370
> Project: Hadoop YARN
> Issue Type: Improvement
>Reporter: Szilard Nemeth
>Assignee: Yesha Vora
>Priority: Trivial
> Labels: newbie, newbie++
> Attachments: YARN-9370.001.patch, YARN-9370.002.patch,
> YARN-9370.003.patch, YARN-9370.004.patch
>
>
> The last line of
> org.apache.hadoop.yarn.server.nodemanager.containermanager.linux.resources.gpu.GpuResourceAllocator#recoverAssignedGpus
> is this:
> {code:java}
> usedDevices.put(gpuDevice, containerId);
> {code}
> We should have an info (or if not info, at least a debug) level log to
> indicate that a container is allocated to a GPU device during recovery.
> Please also check recovery related code, maybe there can be some room for
> improvement around logging.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-9370) Better logging in recoverAssignedGpus in class GpuResourceAllocator
[
https://issues.apache.org/jira/browse/YARN-9370?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Yesha Vora updated YARN-9370:
-
Attachment: YARN-9370.004.patch
> Better logging in recoverAssignedGpus in class GpuResourceAllocator
> ---
>
> Key: YARN-9370
> URL: https://issues.apache.org/jira/browse/YARN-9370
> Project: Hadoop YARN
> Issue Type: Improvement
>Reporter: Szilard Nemeth
>Assignee: Yesha Vora
>Priority: Trivial
> Labels: newbie, newbie++
> Attachments: YARN-9370.001.patch, YARN-9370.002.patch,
> YARN-9370.003.patch, YARN-9370.004.patch
>
>
> The last line of
> org.apache.hadoop.yarn.server.nodemanager.containermanager.linux.resources.gpu.GpuResourceAllocator#recoverAssignedGpus
> is this:
> {code:java}
> usedDevices.put(gpuDevice, containerId);
> {code}
> We should have an info (or if not info, at least a debug) level log to
> indicate that a container is allocated to a GPU device during recovery.
> Please also check recovery related code, maybe there can be some room for
> improvement around logging.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-9370) Better logging in recoverAssignedGpus in class GpuResourceAllocator
[
https://issues.apache.org/jira/browse/YARN-9370?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Yesha Vora updated YARN-9370:
-
Attachment: YARN-9370.003.patch
> Better logging in recoverAssignedGpus in class GpuResourceAllocator
> ---
>
> Key: YARN-9370
> URL: https://issues.apache.org/jira/browse/YARN-9370
> Project: Hadoop YARN
> Issue Type: Improvement
>Reporter: Szilard Nemeth
>Assignee: Yesha Vora
>Priority: Trivial
> Labels: newbie, newbie++
> Attachments: YARN-9370.001.patch, YARN-9370.002.patch,
> YARN-9370.003.patch
>
>
> The last line of
> org.apache.hadoop.yarn.server.nodemanager.containermanager.linux.resources.gpu.GpuResourceAllocator#recoverAssignedGpus
> is this:
> {code:java}
> usedDevices.put(gpuDevice, containerId);
> {code}
> We should have an info (or if not info, at least a debug) level log to
> indicate that a container is allocated to a GPU device during recovery.
> Please also check recovery related code, maybe there can be some room for
> improvement around logging.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-9370) Better logging in recoverAssignedGpus in class GpuResourceAllocator
[
https://issues.apache.org/jira/browse/YARN-9370?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Yesha Vora updated YARN-9370:
-
Attachment: YARN-9370.002.patch
> Better logging in recoverAssignedGpus in class GpuResourceAllocator
> ---
>
> Key: YARN-9370
> URL: https://issues.apache.org/jira/browse/YARN-9370
> Project: Hadoop YARN
> Issue Type: Improvement
>Reporter: Szilard Nemeth
>Assignee: Yesha Vora
>Priority: Trivial
> Labels: newbie, newbie++
> Attachments: YARN-9370.001.patch, YARN-9370.002.patch
>
>
> The last line of
> org.apache.hadoop.yarn.server.nodemanager.containermanager.linux.resources.gpu.GpuResourceAllocator#recoverAssignedGpus
> is this:
> {code:java}
> usedDevices.put(gpuDevice, containerId);
> {code}
> We should have an info (or if not info, at least a debug) level log to
> indicate that a container is allocated to a GPU device during recovery.
> Please also check recovery related code, maybe there can be some room for
> improvement around logging.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-8551) Build Common module for MaWo application
[ https://issues.apache.org/jira/browse/YARN-8551?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yesha Vora updated YARN-8551: - Attachment: YARN-8551.0010.patch > Build Common module for MaWo application > > > Key: YARN-8551 > URL: https://issues.apache.org/jira/browse/YARN-8551 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Yesha Vora >Assignee: Yesha Vora >Priority: Major > Attachments: YARN-8551.001.patch, YARN-8551.0010.patch, > YARN-8551.002.patch, YARN-8551.003.patch, YARN-8551.004.patch, > YARN-8551.005.patch, YARN-8551.006.patch, YARN-8551.007.patch, > YARN-8551.008.patch, YARN-8551.009.patch > > > Build Common module for MaWo application. > This module should include defination of Task. A Task should contain > * TaskID > * Task Command > * Task Environment > * Task Timeout > * Task Type > ** Simple Task > *** Its a single Task > ** Composite Task > *** Its a composition of multiple simple tasks > ** Teardown Task > *** Its a last task to be executed after a job is finished > ** Null Task > *** Its a null task -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-8551) Build Common module for MaWo application
[ https://issues.apache.org/jira/browse/YARN-8551?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yesha Vora updated YARN-8551: - Attachment: YARN-8551.009.patch > Build Common module for MaWo application > > > Key: YARN-8551 > URL: https://issues.apache.org/jira/browse/YARN-8551 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Yesha Vora >Assignee: Yesha Vora >Priority: Major > Attachments: YARN-8551.001.patch, YARN-8551.002.patch, > YARN-8551.003.patch, YARN-8551.004.patch, YARN-8551.005.patch, > YARN-8551.006.patch, YARN-8551.007.patch, YARN-8551.008.patch, > YARN-8551.009.patch > > > Build Common module for MaWo application. > This module should include defination of Task. A Task should contain > * TaskID > * Task Command > * Task Environment > * Task Timeout > * Task Type > ** Simple Task > *** Its a single Task > ** Composite Task > *** Its a composition of multiple simple tasks > ** Teardown Task > *** Its a last task to be executed after a job is finished > ** Null Task > *** Its a null task -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-8551) Build Common module for MaWo application
[ https://issues.apache.org/jira/browse/YARN-8551?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yesha Vora updated YARN-8551: - Attachment: YARN-8551.008.patch > Build Common module for MaWo application > > > Key: YARN-8551 > URL: https://issues.apache.org/jira/browse/YARN-8551 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Yesha Vora >Assignee: Yesha Vora >Priority: Major > Attachments: YARN-8551.001.patch, YARN-8551.002.patch, > YARN-8551.003.patch, YARN-8551.004.patch, YARN-8551.005.patch, > YARN-8551.006.patch, YARN-8551.007.patch, YARN-8551.008.patch > > > Build Common module for MaWo application. > This module should include defination of Task. A Task should contain > * TaskID > * Task Command > * Task Environment > * Task Timeout > * Task Type > ** Simple Task > *** Its a single Task > ** Composite Task > *** Its a composition of multiple simple tasks > ** Teardown Task > *** Its a last task to be executed after a job is finished > ** Null Task > *** Its a null task -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-8551) Build Common module for MaWo application
[ https://issues.apache.org/jira/browse/YARN-8551?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yesha Vora updated YARN-8551: - Attachment: YARN-8551.007.patch > Build Common module for MaWo application > > > Key: YARN-8551 > URL: https://issues.apache.org/jira/browse/YARN-8551 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Yesha Vora >Assignee: Yesha Vora >Priority: Major > Attachments: YARN-8551.001.patch, YARN-8551.002.patch, > YARN-8551.003.patch, YARN-8551.004.patch, YARN-8551.005.patch, > YARN-8551.006.patch, YARN-8551.007.patch > > > Build Common module for MaWo application. > This module should include defination of Task. A Task should contain > * TaskID > * Task Command > * Task Environment > * Task Timeout > * Task Type > ** Simple Task > *** Its a single Task > ** Composite Task > *** Its a composition of multiple simple tasks > ** Teardown Task > *** Its a last task to be executed after a job is finished > ** Null Task > *** Its a null task -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Assigned] (YARN-9370) Better logging in recoverAssignedGpus in class GpuResourceAllocator
[
https://issues.apache.org/jira/browse/YARN-9370?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Yesha Vora reassigned YARN-9370:
Assignee: Yesha Vora
> Better logging in recoverAssignedGpus in class GpuResourceAllocator
> ---
>
> Key: YARN-9370
> URL: https://issues.apache.org/jira/browse/YARN-9370
> Project: Hadoop YARN
> Issue Type: Improvement
>Reporter: Szilard Nemeth
>Assignee: Yesha Vora
>Priority: Trivial
> Labels: newbie, newbie++
>
> The last line of
> org.apache.hadoop.yarn.server.nodemanager.containermanager.linux.resources.gpu.GpuResourceAllocator#recoverAssignedGpus
> is this:
> {code:java}
> usedDevices.put(gpuDevice, containerId);
> {code}
> We should have an info (or if not info, at least a debug) level log to
> indicate that a container is allocated to a GPU device during recovery.
> Please also check recovery related code, maybe there can be some room for
> improvement around logging.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-9370) Better logging in recoverAssignedGpus in class GpuResourceAllocator
[
https://issues.apache.org/jira/browse/YARN-9370?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Yesha Vora updated YARN-9370:
-
Attachment: YARN-9370.001.patch
> Better logging in recoverAssignedGpus in class GpuResourceAllocator
> ---
>
> Key: YARN-9370
> URL: https://issues.apache.org/jira/browse/YARN-9370
> Project: Hadoop YARN
> Issue Type: Improvement
>Reporter: Szilard Nemeth
>Assignee: Yesha Vora
>Priority: Trivial
> Labels: newbie, newbie++
> Attachments: YARN-9370.001.patch
>
>
> The last line of
> org.apache.hadoop.yarn.server.nodemanager.containermanager.linux.resources.gpu.GpuResourceAllocator#recoverAssignedGpus
> is this:
> {code:java}
> usedDevices.put(gpuDevice, containerId);
> {code}
> We should have an info (or if not info, at least a debug) level log to
> indicate that a container is allocated to a GPU device during recovery.
> Please also check recovery related code, maybe there can be some room for
> improvement around logging.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-8551) Build Common module for MaWo application
[ https://issues.apache.org/jira/browse/YARN-8551?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yesha Vora updated YARN-8551: - Attachment: YARN-8551.006.patch > Build Common module for MaWo application > > > Key: YARN-8551 > URL: https://issues.apache.org/jira/browse/YARN-8551 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Yesha Vora >Assignee: Yesha Vora >Priority: Major > Attachments: YARN-8551.001.patch, YARN-8551.002.patch, > YARN-8551.003.patch, YARN-8551.004.patch, YARN-8551.005.patch, > YARN-8551.006.patch > > > Build Common module for MaWo application. > This module should include defination of Task. A Task should contain > * TaskID > * Task Command > * Task Environment > * Task Timeout > * Task Type > ** Simple Task > *** Its a single Task > ** Composite Task > *** Its a composition of multiple simple tasks > ** Teardown Task > *** Its a last task to be executed after a job is finished > ** Null Task > *** Its a null task -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-8551) Build Common module for MaWo application
[ https://issues.apache.org/jira/browse/YARN-8551?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yesha Vora updated YARN-8551: - Attachment: YARN-8551.005.patch > Build Common module for MaWo application > > > Key: YARN-8551 > URL: https://issues.apache.org/jira/browse/YARN-8551 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Yesha Vora >Assignee: Yesha Vora >Priority: Major > Attachments: YARN-8551.001.patch, YARN-8551.002.patch, > YARN-8551.003.patch, YARN-8551.004.patch, YARN-8551.005.patch > > > Build Common module for MaWo application. > This module should include defination of Task. A Task should contain > * TaskID > * Task Command > * Task Environment > * Task Timeout > * Task Type > ** Simple Task > *** Its a single Task > ** Composite Task > *** Its a composition of multiple simple tasks > ** Teardown Task > *** Its a last task to be executed after a job is finished > ** Null Task > *** Its a null task -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-4404) Typo in comment in SchedulerUtils
[
https://issues.apache.org/jira/browse/YARN-4404?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Yesha Vora updated YARN-4404:
-
Attachment: YARN-4404.001.patch
> Typo in comment in SchedulerUtils
> -
>
> Key: YARN-4404
> URL: https://issues.apache.org/jira/browse/YARN-4404
> Project: Hadoop YARN
> Issue Type: Improvement
> Components: resourcemanager
>Affects Versions: 2.7.1
>Reporter: Daniel Templeton
>Assignee: Yesha Vora
>Priority: Trivial
> Labels: newbie
> Attachments: YARN-4404.001.patch
>
>
> The comment starting on line 254 says:
> {code}
> /**
>* Utility method to validate a resource request, by insuring that the
>* requested memory/vcore is non-negative and not greater than max
>*
>* @throws InvalidResourceRequestException when there is invalid request
>*/
> {code}
> "Insuring" should be "ensuring."
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Assigned] (YARN-4404) Typo in comment in SchedulerUtils
[
https://issues.apache.org/jira/browse/YARN-4404?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Yesha Vora reassigned YARN-4404:
Assignee: Yesha Vora (was: Devon Michaels)
> Typo in comment in SchedulerUtils
> -
>
> Key: YARN-4404
> URL: https://issues.apache.org/jira/browse/YARN-4404
> Project: Hadoop YARN
> Issue Type: Improvement
> Components: resourcemanager
>Affects Versions: 2.7.1
>Reporter: Daniel Templeton
>Assignee: Yesha Vora
>Priority: Trivial
> Labels: newbie
>
> The comment starting on line 254 says:
> {code}
> /**
>* Utility method to validate a resource request, by insuring that the
>* requested memory/vcore is non-negative and not greater than max
>*
>* @throws InvalidResourceRequestException when there is invalid request
>*/
> {code}
> "Insuring" should be "ensuring."
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-8913) Add helper scripts to launch MaWo App to run Hadoop unit tests on Hadoop Cluster
[ https://issues.apache.org/jira/browse/YARN-8913?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16667888#comment-16667888 ] Yesha Vora commented on YARN-8913: -- [~eyang], Thanks for feedback. Please find attached patch#3 addressing your suggestions. > Add helper scripts to launch MaWo App to run Hadoop unit tests on Hadoop > Cluster > > > Key: YARN-8913 > URL: https://issues.apache.org/jira/browse/YARN-8913 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Yesha Vora >Assignee: Yesha Vora >Priority: Blocker > Attachments: YARN-8913.001.patch, YARN-8913.002.patch, > YARN-8913.003.patch > > > MaWo application can be used to run Hadoop UT faster in a Hadoop cluster. > Develop helper scripts to orchestrate end-to-end workflow for running Hadoop > UT using MaWo app. > Pre-requisite: > * A Hadoop Cluster with HDFS and YARN installed > * Enable Docker on YARN feature > > Helper-scripts > * MaWo_Driver > ** create a docker image with latest hadoop source code > ** create payload to MaWo app (This is input to mawo app where Each MaWo > Task = UT execution of each Hadoop Module) > ** Upload payload file to HDFS > ** Update MaWo-Launch.json to resolve RM_HOST / Docker Image etc dynamically > ** Launch MaWo app in Hadoop cluster -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-8913) Add helper scripts to launch MaWo App to run Hadoop unit tests on Hadoop Cluster
[ https://issues.apache.org/jira/browse/YARN-8913?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yesha Vora updated YARN-8913: - Attachment: YARN-8913.003.patch > Add helper scripts to launch MaWo App to run Hadoop unit tests on Hadoop > Cluster > > > Key: YARN-8913 > URL: https://issues.apache.org/jira/browse/YARN-8913 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Yesha Vora >Assignee: Yesha Vora >Priority: Blocker > Attachments: YARN-8913.001.patch, YARN-8913.002.patch, > YARN-8913.003.patch > > > MaWo application can be used to run Hadoop UT faster in a Hadoop cluster. > Develop helper scripts to orchestrate end-to-end workflow for running Hadoop > UT using MaWo app. > Pre-requisite: > * A Hadoop Cluster with HDFS and YARN installed > * Enable Docker on YARN feature > > Helper-scripts > * MaWo_Driver > ** create a docker image with latest hadoop source code > ** create payload to MaWo app (This is input to mawo app where Each MaWo > Task = UT execution of each Hadoop Module) > ** Upload payload file to HDFS > ** Update MaWo-Launch.json to resolve RM_HOST / Docker Image etc dynamically > ** Launch MaWo app in Hadoop cluster -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-8551) Build Common module for MaWo application
[ https://issues.apache.org/jira/browse/YARN-8551?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yesha Vora updated YARN-8551: - Attachment: YARN-8551.004.patch > Build Common module for MaWo application > > > Key: YARN-8551 > URL: https://issues.apache.org/jira/browse/YARN-8551 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Yesha Vora >Assignee: Yesha Vora >Priority: Major > Attachments: YARN-8551.001.patch, YARN-8551.002.patch, > YARN-8551.003.patch, YARN-8551.004.patch > > > Build Common module for MaWo application. > This module should include defination of Task. A Task should contain > * TaskID > * Task Command > * Task Environment > * Task Timeout > * Task Type > ** Simple Task > *** Its a single Task > ** Composite Task > *** Its a composition of multiple simple tasks > ** Teardown Task > *** Its a last task to be executed after a job is finished > ** Null Task > *** Its a null task -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-8551) Build Common module for MaWo application
[ https://issues.apache.org/jira/browse/YARN-8551?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16661124#comment-16661124 ] Yesha Vora commented on YARN-8551: -- Sure, Removed Utils.py and updated Patch #4. > Build Common module for MaWo application > > > Key: YARN-8551 > URL: https://issues.apache.org/jira/browse/YARN-8551 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Yesha Vora >Assignee: Yesha Vora >Priority: Major > Attachments: YARN-8551.001.patch, YARN-8551.002.patch, > YARN-8551.003.patch, YARN-8551.004.patch > > > Build Common module for MaWo application. > This module should include defination of Task. A Task should contain > * TaskID > * Task Command > * Task Environment > * Task Timeout > * Task Type > ** Simple Task > *** Its a single Task > ** Composite Task > *** Its a composition of multiple simple tasks > ** Teardown Task > *** Its a last task to be executed after a job is finished > ** Null Task > *** Its a null task -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-8913) Add helper scripts to launch MaWo App to run Hadoop unit tests on Hadoop Cluster
[ https://issues.apache.org/jira/browse/YARN-8913?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yesha Vora updated YARN-8913: - Attachment: YARN-8913.002.patch > Add helper scripts to launch MaWo App to run Hadoop unit tests on Hadoop > Cluster > > > Key: YARN-8913 > URL: https://issues.apache.org/jira/browse/YARN-8913 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Yesha Vora >Assignee: Yesha Vora >Priority: Blocker > Attachments: YARN-8913.001.patch, YARN-8913.002.patch > > > MaWo application can be used to run Hadoop UT faster in a Hadoop cluster. > Develop helper scripts to orchestrate end-to-end workflow for running Hadoop > UT using MaWo app. > Pre-requisite: > * A Hadoop Cluster with HDFS and YARN installed > * Enable Docker on YARN feature > > Helper-scripts > * MaWo_Driver > ** create a docker image with latest hadoop source code > ** create payload to MaWo app (This is input to mawo app where Each MaWo > Task = UT execution of each Hadoop Module) > ** Upload payload file to HDFS > ** Update MaWo-Launch.json to resolve RM_HOST / Docker Image etc dynamically > ** Launch MaWo app in Hadoop cluster -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-8599) Build Master module for MaWo app
[ https://issues.apache.org/jira/browse/YARN-8599?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yesha Vora updated YARN-8599: - Attachment: YARN-8599.002.patch > Build Master module for MaWo app > > > Key: YARN-8599 > URL: https://issues.apache.org/jira/browse/YARN-8599 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Yesha Vora >Assignee: Yesha Vora >Priority: Major > Attachments: YARN-8599.001.patch, YARN-8599.002.patch > > > Master component for MaWo application is responsible for driving end-to-end > job execution. Its responsibility is > * Get Job definition and create a Queue of Tasks > * Assign Tasks to Worker > * Manage Workers lifecycle -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-8598) Build Master Job Module for MaWo Application
[ https://issues.apache.org/jira/browse/YARN-8598?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yesha Vora updated YARN-8598: - Attachment: YARN-8598.002.patch > Build Master Job Module for MaWo Application > > > Key: YARN-8598 > URL: https://issues.apache.org/jira/browse/YARN-8598 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Yesha Vora >Assignee: Yesha Vora >Priority: Major > Attachments: YARN-8598.001.patch, YARN-8598.002.patch > > > A job in MaWo application is a collection of Tasks. A Job consists of a setup > task, a list of tasks and a teardown task. > * JobBuilder > ** SimpleTaskJobBuilder : SimpleJobBuilder should be able to parse > simpleJobdescription file. In this file format, each line is considered as > Task > ** SimpleTaskJsonJobBuilder: Utility to parse json job description file. > -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-8597) Build Worker utility for MaWo Application
[ https://issues.apache.org/jira/browse/YARN-8597?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yesha Vora updated YARN-8597: - Attachment: YARN-8597.002.patch > Build Worker utility for MaWo Application > - > > Key: YARN-8597 > URL: https://issues.apache.org/jira/browse/YARN-8597 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Yesha Vora >Assignee: Yesha Vora >Priority: Major > Attachments: YARN-8597.001.patch, YARN-8597.002.patch > > > The worker is responsible for executing Tasks. > * Worker > ** Create a worker class which drives worker life cycle > ** Create WorkAssignment Protocol. It should be handle Register/deregister > worker, send heartbeat > ** Lifecycle: Register worker, Run Setup Task, Get Task from master and > execute it using TaskRunner, Run Teardown Task > * TaskRunner > ** Simple Task Runner : This runner should be able to execute a simple task > ** Composite Task Runner: This runner should be able to execute composite > task > * TaskWallTimeLimiter > ** Create a utility which can abort the task if the execution time exceeds > task timeout. > -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-8551) Build Common module for MaWo application
[ https://issues.apache.org/jira/browse/YARN-8551?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16660102#comment-16660102 ] Yesha Vora commented on YARN-8551: -- Thanks [~eyang]. Patch#3 is submitted with Apache license headers. > Build Common module for MaWo application > > > Key: YARN-8551 > URL: https://issues.apache.org/jira/browse/YARN-8551 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Yesha Vora >Assignee: Yesha Vora >Priority: Major > Attachments: YARN-8551.001.patch, YARN-8551.002.patch, > YARN-8551.003.patch > > > Build Common module for MaWo application. > This module should include defination of Task. A Task should contain > * TaskID > * Task Command > * Task Environment > * Task Timeout > * Task Type > ** Simple Task > *** Its a single Task > ** Composite Task > *** Its a composition of multiple simple tasks > ** Teardown Task > *** Its a last task to be executed after a job is finished > ** Null Task > *** Its a null task -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-8551) Build Common module for MaWo application
[ https://issues.apache.org/jira/browse/YARN-8551?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yesha Vora updated YARN-8551: - Attachment: YARN-8551.003.patch > Build Common module for MaWo application > > > Key: YARN-8551 > URL: https://issues.apache.org/jira/browse/YARN-8551 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Yesha Vora >Assignee: Yesha Vora >Priority: Major > Attachments: YARN-8551.001.patch, YARN-8551.002.patch, > YARN-8551.003.patch > > > Build Common module for MaWo application. > This module should include defination of Task. A Task should contain > * TaskID > * Task Command > * Task Environment > * Task Timeout > * Task Type > ** Simple Task > *** Its a single Task > ** Composite Task > *** Its a composition of multiple simple tasks > ** Teardown Task > *** Its a last task to be executed after a job is finished > ** Null Task > *** Its a null task -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-8913) Add helper scripts to launch MaWo App to run Hadoop unit tests on Hadoop Cluster
[ https://issues.apache.org/jira/browse/YARN-8913?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yesha Vora updated YARN-8913: - Attachment: YARN-8913.001.patch > Add helper scripts to launch MaWo App to run Hadoop unit tests on Hadoop > Cluster > > > Key: YARN-8913 > URL: https://issues.apache.org/jira/browse/YARN-8913 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Yesha Vora >Assignee: Yesha Vora >Priority: Blocker > Attachments: YARN-8913.001.patch > > > MaWo application can be used to run Hadoop UT faster in a Hadoop cluster. > Develop helper scripts to orchestrate end-to-end workflow for running Hadoop > UT using MaWo app. > Pre-requisite: > * A Hadoop Cluster with HDFS and YARN installed > * Enable Docker on YARN feature > > Helper-scripts > * MaWo_Driver > ** create a docker image with latest hadoop source code > ** create payload to MaWo app (This is input to mawo app where Each MaWo > Task = UT execution of each Hadoop Module) > ** Upload payload file to HDFS > ** Update MaWo-Launch.json to resolve RM_HOST / Docker Image etc dynamically > ** Launch MaWo app in Hadoop cluster -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Created] (YARN-8913) Add helper scripts to launch MaWo App to run Hadoop unit tests on Hadoop Cluster
Yesha Vora created YARN-8913: Summary: Add helper scripts to launch MaWo App to run Hadoop unit tests on Hadoop Cluster Key: YARN-8913 URL: https://issues.apache.org/jira/browse/YARN-8913 Project: Hadoop YARN Issue Type: Sub-task Reporter: Yesha Vora MaWo application can be used to run Hadoop UT faster in a Hadoop cluster. Develop helper scripts to orchestrate end-to-end workflow for running Hadoop UT using MaWo app. Pre-requisite: * A Hadoop Cluster with HDFS and YARN installed * Enable Docker on YARN feature Helper-scripts * MaWo_Driver ** create a docker image with latest hadoop source code ** create payload to MaWo app (This is input to mawo app where Each MaWo Task = UT execution of each Hadoop Module) ** Upload payload file to HDFS ** Update MaWo-Launch.json to resolve RM_HOST / Docker Image etc dynamically ** Launch MaWo app in Hadoop cluster -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Assigned] (YARN-8913) Add helper scripts to launch MaWo App to run Hadoop unit tests on Hadoop Cluster
[ https://issues.apache.org/jira/browse/YARN-8913?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yesha Vora reassigned YARN-8913: Assignee: Yesha Vora > Add helper scripts to launch MaWo App to run Hadoop unit tests on Hadoop > Cluster > > > Key: YARN-8913 > URL: https://issues.apache.org/jira/browse/YARN-8913 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Yesha Vora >Assignee: Yesha Vora >Priority: Blocker > > MaWo application can be used to run Hadoop UT faster in a Hadoop cluster. > Develop helper scripts to orchestrate end-to-end workflow for running Hadoop > UT using MaWo app. > Pre-requisite: > * A Hadoop Cluster with HDFS and YARN installed > * Enable Docker on YARN feature > > Helper-scripts > * MaWo_Driver > ** create a docker image with latest hadoop source code > ** create payload to MaWo app (This is input to mawo app where Each MaWo > Task = UT execution of each Hadoop Module) > ** Upload payload file to HDFS > ** Update MaWo-Launch.json to resolve RM_HOST / Docker Image etc dynamically > ** Launch MaWo app in Hadoop cluster -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-8912) Fix MaWo_Config to read WORKER_WORK_SPACE and MASTER_TASKS_STATUS_LOG_PATH from env
[ https://issues.apache.org/jira/browse/YARN-8912?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yesha Vora updated YARN-8912: - Attachment: YARN-8912.001.patch > Fix MaWo_Config to read WORKER_WORK_SPACE and MASTER_TASKS_STATUS_LOG_PATH > from env > --- > > Key: YARN-8912 > URL: https://issues.apache.org/jira/browse/YARN-8912 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Yesha Vora >Assignee: Yesha Vora >Priority: Critical > Attachments: YARN-8912.001.patch > > > Fix MaWo Configuration to read MASTER_TASKS_STATUS_LOG_PATH and > WORKER_WORK_SPACE from env. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Created] (YARN-8912) Fix MaWo_Config to read WORKER_WORK_SPACE and MASTER_TASKS_STATUS_LOG_PATH from env
Yesha Vora created YARN-8912: Summary: Fix MaWo_Config to read WORKER_WORK_SPACE and MASTER_TASKS_STATUS_LOG_PATH from env Key: YARN-8912 URL: https://issues.apache.org/jira/browse/YARN-8912 Project: Hadoop YARN Issue Type: Sub-task Reporter: Yesha Vora Fix MaWo Configuration to read MASTER_TASKS_STATUS_LOG_PATH and WORKER_WORK_SPACE from env. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-8912) Fix MaWo_Config to read WORKER_WORK_SPACE and MASTER_TASKS_STATUS_LOG_PATH from env
[ https://issues.apache.org/jira/browse/YARN-8912?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yesha Vora updated YARN-8912: - Priority: Critical (was: Major) > Fix MaWo_Config to read WORKER_WORK_SPACE and MASTER_TASKS_STATUS_LOG_PATH > from env > --- > > Key: YARN-8912 > URL: https://issues.apache.org/jira/browse/YARN-8912 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Yesha Vora >Assignee: Yesha Vora >Priority: Critical > > Fix MaWo Configuration to read MASTER_TASKS_STATUS_LOG_PATH and > WORKER_WORK_SPACE from env. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Assigned] (YARN-8912) Fix MaWo_Config to read WORKER_WORK_SPACE and MASTER_TASKS_STATUS_LOG_PATH from env
[ https://issues.apache.org/jira/browse/YARN-8912?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yesha Vora reassigned YARN-8912: Assignee: Yesha Vora > Fix MaWo_Config to read WORKER_WORK_SPACE and MASTER_TASKS_STATUS_LOG_PATH > from env > --- > > Key: YARN-8912 > URL: https://issues.apache.org/jira/browse/YARN-8912 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Yesha Vora >Assignee: Yesha Vora >Priority: Major > > Fix MaWo Configuration to read MASTER_TASKS_STATUS_LOG_PATH and > WORKER_WORK_SPACE from env. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-8901) Restart "NEVER" policy does not work with component dependency
[
https://issues.apache.org/jira/browse/YARN-8901?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Yesha Vora updated YARN-8901:
-
Affects Version/s: 3.1.1
> Restart "NEVER" policy does not work with component dependency
> --
>
> Key: YARN-8901
> URL: https://issues.apache.org/jira/browse/YARN-8901
> Project: Hadoop YARN
> Issue Type: Bug
>Affects Versions: 3.1.1
>Reporter: Yesha Vora
>Assignee: Suma Shivaprasad
>Priority: Critical
>
> Scenario:
> 1) Launch an application with two components. master and worker. Here, worker
> is dependent on master. ( Worker should be launched only after master is
> launched )
> 2) Set restart_policy = NEVER for both master and worker.
> {code:title=sample launch.json}
> {
> "name": "mawo-hadoop-ut",
> "artifact": {
> "type": "DOCKER",
> "id": "xxx"
> },
> "configuration": {
> "env": {
>"YARN_CONTAINER_RUNTIME_DOCKER_CONTAINER_NETWORK":
> "hadoop"
> },
> "properties": {
>"docker.network": "hadoop"
> }
> },
> "components": [{
> "dependencies": [],
> "resource": {
> "memory": "2048",
> "cpus": "1"
> },
> "name": "master",
> "run_privileged_container": true,
> "number_of_containers": 1,
> "launch_command": "start master",
> "restart_policy": "NEVER",
> }, {
> "dependencies": ["master"],
> "resource": {
> "memory": "8072",
> "cpus": "1"
> },
> "name": "worker",
> "run_privileged_container": true,
> "number_of_containers": 10,
> "launch_command": "start worker",
> "restart_policy": "NEVER",
> }],
> "lifetime": -1,
> "version": 1.0
> }{code}
> When restart policy is selected to NEVER, AM never launches Worker component.
> It get stuck with below message.
> {code}
> 2018-10-17 15:11:58,560 [Component dispatcher] INFO component.Component -
> [COMPONENT master] Transitioned from FLEXING to STABLE on CHECK_STABLE event.
> 2018-10-17 15:11:58,560 [pool-7-thread-1] INFO instance.ComponentInstance -
> [COMPINSTANCE master-0 : container_e41_1539027682947_0020_01_02]
> Transitioned from STARTED to READY on BECOME_READY event
> 2018-10-17 15:11:58,560 [pool-7-thread-1] INFO component.Component -
> [COMPONENT worker]: Dependency master not satisfied, only 1 of 1 instances
> are ready or the dependent component has not completed
> 2018-10-17 15:12:28,556 [pool-7-thread-1] INFO component.Component -
> [COMPONENT worker]: Dependency master not satisfied, only 1 of 1 instances
> are ready or the dependent component has not completed
> 2018-10-17 15:12:58,556 [pool-7-thread-1] INFO component.Component -
> [COMPONENT worker]: Dependency master not satisfied, only 1 of 1 instances
> are ready or the dependent component has not completed
> 2018-10-17 15:13:28,556 [pool-7-thread-1] INFO component.Component -
> [COMPONENT worker]: Dependency master not satisfied, only 1 of 1 instances
> are ready or the dependent component has not completed
> 2018-10-17 15:13:58,556 [pool-7-thread-1] INFO component.Component -
> [COMPONENT worker]: Dependency master not satisfied, only 1 of 1 instances
> are ready or the dependent component has not completed
> 2018-10-17 15:14:28,556 [pool-7-thread-1] INFO component.Component -
> [COMPONENT worker]: Dependency master not satisfied, only 1 of 1 instances
> are ready or the dependent component has not completed {code}
> 'NEVER' restart policy expects master component to be finished before
> starting workers. Master component can not finish the job without workers.
> Thus, it create a deadlock.
> The logic for 'NEVER' restart policy should be fixed to allow worker
> components to be launched as soon as master component is in READY state.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Created] (YARN-8901) Restart "NEVER" policy does not work with component dependency
Yesha Vora created YARN-8901:
Summary: Restart "NEVER" policy does not work with component
dependency
Key: YARN-8901
URL: https://issues.apache.org/jira/browse/YARN-8901
Project: Hadoop YARN
Issue Type: Bug
Reporter: Yesha Vora
Scenario:
1) Launch an application with two components. master and worker. Here, worker
is dependent on master. ( Worker should be launched only after master is
launched )
2) Set restart_policy = NEVER for both master and worker.
{code:title=sample launch.json}
{
"name": "mawo-hadoop-ut",
"artifact": {
"type": "DOCKER",
"id": "xxx"
},
"configuration": {
"env": {
"YARN_CONTAINER_RUNTIME_DOCKER_CONTAINER_NETWORK":
"hadoop"
},
"properties": {
"docker.network": "hadoop"
}
},
"components": [{
"dependencies": [],
"resource": {
"memory": "2048",
"cpus": "1"
},
"name": "master",
"run_privileged_container": true,
"number_of_containers": 1,
"launch_command": "start master",
"restart_policy": "NEVER",
}, {
"dependencies": ["master"],
"resource": {
"memory": "8072",
"cpus": "1"
},
"name": "worker",
"run_privileged_container": true,
"number_of_containers": 10,
"launch_command": "start worker",
"restart_policy": "NEVER",
}],
"lifetime": -1,
"version": 1.0
}{code}
When restart policy is selected to NEVER, AM never launches Worker component.
It get stuck with below message.
{code}
2018-10-17 15:11:58,560 [Component dispatcher] INFO component.Component -
[COMPONENT master] Transitioned from FLEXING to STABLE on CHECK_STABLE event.
2018-10-17 15:11:58,560 [pool-7-thread-1] INFO instance.ComponentInstance -
[COMPINSTANCE master-0 : container_e41_1539027682947_0020_01_02]
Transitioned from STARTED to READY on BECOME_READY event
2018-10-17 15:11:58,560 [pool-7-thread-1] INFO component.Component -
[COMPONENT worker]: Dependency master not satisfied, only 1 of 1 instances are
ready or the dependent component has not completed
2018-10-17 15:12:28,556 [pool-7-thread-1] INFO component.Component -
[COMPONENT worker]: Dependency master not satisfied, only 1 of 1 instances are
ready or the dependent component has not completed
2018-10-17 15:12:58,556 [pool-7-thread-1] INFO component.Component -
[COMPONENT worker]: Dependency master not satisfied, only 1 of 1 instances are
ready or the dependent component has not completed
2018-10-17 15:13:28,556 [pool-7-thread-1] INFO component.Component -
[COMPONENT worker]: Dependency master not satisfied, only 1 of 1 instances are
ready or the dependent component has not completed
2018-10-17 15:13:58,556 [pool-7-thread-1] INFO component.Component -
[COMPONENT worker]: Dependency master not satisfied, only 1 of 1 instances are
ready or the dependent component has not completed
2018-10-17 15:14:28,556 [pool-7-thread-1] INFO component.Component -
[COMPONENT worker]: Dependency master not satisfied, only 1 of 1 instances are
ready or the dependent component has not completed {code}
'NEVER' restart policy expects master component to be finished before starting
workers. Master component can not finish the job without workers. Thus, it
create a deadlock.
The logic for 'NEVER' restart policy should be fixed to allow worker components
to be launched as soon as master component is in READY state.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-8666) [UI2] Remove application tab from Yarn Queue Page
[ https://issues.apache.org/jira/browse/YARN-8666?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yesha Vora updated YARN-8666: - Attachment: YARN-8666.002.patch > [UI2] Remove application tab from Yarn Queue Page > - > > Key: YARN-8666 > URL: https://issues.apache.org/jira/browse/YARN-8666 > Project: Hadoop YARN > Issue Type: Bug > Components: yarn-ui-v2 >Affects Versions: 3.1.1 >Reporter: Yesha Vora >Assignee: Yesha Vora >Priority: Major > Attachments: Screen Shot 2018-08-14 at 3.43.18 PM.png, Screen Shot > 2018-09-06 at 12.50.14 PM.png, YARN-8666.001.patch, YARN-8666.002.patch > > > Yarn UI2 Queue page puts Application button. This button does not redirect to > any other page. In addition to that running application table is also > available on same page. > Thus, there is no need to have a button for application in Queue page. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-8753) [UI2] Lost nodes representation missing from Nodemanagers Chart
[ https://issues.apache.org/jira/browse/YARN-8753?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yesha Vora updated YARN-8753: - Attachment: YARN-8753.002.patch > [UI2] Lost nodes representation missing from Nodemanagers Chart > --- > > Key: YARN-8753 > URL: https://issues.apache.org/jira/browse/YARN-8753 > Project: Hadoop YARN > Issue Type: Bug > Components: yarn-ui-v2 >Affects Versions: 3.1.1 >Reporter: Yesha Vora >Assignee: Yesha Vora >Priority: Major > Attachments: Screen Shot 2018-09-06 at 6.16.02 PM.png, Screen Shot > 2018-09-06 at 6.16.14 PM.png, Screen Shot 2018-09-07 at 11.59.02 AM.png, > YARN-8753.001.patch, YARN-8753.002.patch > > > Nodemanagers Chart is present in Cluster overview and Nodes->Nodes Status > page. > This chart does not show nodemanagers if they are LOST. > Due to this issue, Node information page and Node status page shows different > node managers count. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-8754) [UI2] Improve terms on Component Instance page
[ https://issues.apache.org/jira/browse/YARN-8754?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yesha Vora updated YARN-8754: - Description: Component instance page has "node" and "host". These two fields are representing "bare_host" and "hostname" respectively. >From UI2 page thats not clear. Thus, table content need to be changed to "bare >host" from "node" . This page also has "Host URL" which is hard coded to N/A. Thus, removing this field from table. was: Component instance page has "node" and "host". These two fields are representing "bare_host" and "hostname" accordingly. >From UI2 page thats not clear. Thus, table content need to be changed to "bare >host" from "node" . This page also has "Host URL" which is hard coded to N/A. Thus, removing this field from table. > [UI2] Improve terms on Component Instance page > --- > > Key: YARN-8754 > URL: https://issues.apache.org/jira/browse/YARN-8754 > Project: Hadoop YARN > Issue Type: Bug > Components: yarn-ui-v2 >Affects Versions: 3.1.1 >Reporter: Yesha Vora >Assignee: Yesha Vora >Priority: Major > Attachments: Screen Shot 2018-09-07 at 4.12.54 PM.png, Screen Shot > 2018-09-07 at 4.30.11 PM.png, YARN-8754.001.patch > > > Component instance page has "node" and "host". These two fields are > representing "bare_host" and "hostname" respectively. > From UI2 page thats not clear. Thus, table content need to be changed to > "bare host" from "node" . > This page also has "Host URL" which is hard coded to N/A. Thus, removing this > field from table. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-8754) [UI2] Improve terms on Component Instance page
[ https://issues.apache.org/jira/browse/YARN-8754?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yesha Vora updated YARN-8754: - Attachment: Screen Shot 2018-09-07 at 4.30.11 PM.png > [UI2] Improve terms on Component Instance page > --- > > Key: YARN-8754 > URL: https://issues.apache.org/jira/browse/YARN-8754 > Project: Hadoop YARN > Issue Type: Bug > Components: yarn-ui-v2 >Affects Versions: 3.1.1 >Reporter: Yesha Vora >Assignee: Yesha Vora >Priority: Major > Attachments: Screen Shot 2018-09-07 at 4.12.54 PM.png, Screen Shot > 2018-09-07 at 4.30.11 PM.png, YARN-8754.001.patch > > > Component instance page has "node" and "host". These two fields are > representing "bare_host" and "hostname" accordingly. > From UI2 page thats not clear. Thus, table content need to be changed to > "bare host" from "node" . > This page also has "Host URL" which is hard coded to N/A. Thus, removing this > field from table. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-8754) [UI2] Improve terms on Component Instance page
[ https://issues.apache.org/jira/browse/YARN-8754?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=1660#comment-1660 ] Yesha Vora commented on YARN-8754: -- Find the screenshot of component instance page after fixing terms. !Screen Shot 2018-09-07 at 4.30.11 PM.png! > [UI2] Improve terms on Component Instance page > --- > > Key: YARN-8754 > URL: https://issues.apache.org/jira/browse/YARN-8754 > Project: Hadoop YARN > Issue Type: Bug > Components: yarn-ui-v2 >Affects Versions: 3.1.1 >Reporter: Yesha Vora >Assignee: Yesha Vora >Priority: Major > Attachments: Screen Shot 2018-09-07 at 4.12.54 PM.png, Screen Shot > 2018-09-07 at 4.30.11 PM.png, YARN-8754.001.patch > > > Component instance page has "node" and "host". These two fields are > representing "bare_host" and "hostname" accordingly. > From UI2 page thats not clear. Thus, table content need to be changed to > "bare host" from "node" . > This page also has "Host URL" which is hard coded to N/A. Thus, removing this > field from table. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-8754) [UI2] Improve terms on Component Instance page
[ https://issues.apache.org/jira/browse/YARN-8754?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yesha Vora updated YARN-8754: - Attachment: YARN-8754.001.patch > [UI2] Improve terms on Component Instance page > --- > > Key: YARN-8754 > URL: https://issues.apache.org/jira/browse/YARN-8754 > Project: Hadoop YARN > Issue Type: Bug > Components: yarn-ui-v2 >Affects Versions: 3.1.1 >Reporter: Yesha Vora >Assignee: Yesha Vora >Priority: Major > Attachments: Screen Shot 2018-09-07 at 4.12.54 PM.png, > YARN-8754.001.patch > > > Component instance page has "node" and "host". These two fields are > representing "bare_host" and "hostname" accordingly. > From UI2 page thats not clear. Thus, table content need to be changed to > "bare host" from "node" . > This page also has "Host URL" which is hard coded to N/A. Thus, removing this > field from table. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-8754) [UI2] Improve terms on Component Instance page
[ https://issues.apache.org/jira/browse/YARN-8754?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yesha Vora updated YARN-8754: - Attachment: Screen Shot 2018-09-07 at 4.12.54 PM.png > [UI2] Improve terms on Component Instance page > --- > > Key: YARN-8754 > URL: https://issues.apache.org/jira/browse/YARN-8754 > Project: Hadoop YARN > Issue Type: Bug > Components: yarn-ui-v2 >Affects Versions: 3.1.1 >Reporter: Yesha Vora >Assignee: Yesha Vora >Priority: Major > Attachments: Screen Shot 2018-09-07 at 4.12.54 PM.png > > > Component instance page has "node" and "host". These two fields are > representing "bare_host" and "hostname" accordingly. > From UI2 page thats not clear. Thus, table content need to be changed to > "bare host" from "node" . > This page also has "Host URL" which is hard coded to N/A. Thus, removing this > field from table. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-8754) [UI2] Improve terms on Component Instance page
[ https://issues.apache.org/jira/browse/YARN-8754?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yesha Vora updated YARN-8754: - Affects Version/s: 3.1.1 > [UI2] Improve terms on Component Instance page > --- > > Key: YARN-8754 > URL: https://issues.apache.org/jira/browse/YARN-8754 > Project: Hadoop YARN > Issue Type: Bug > Components: yarn-ui-v2 >Affects Versions: 3.1.1 >Reporter: Yesha Vora >Assignee: Yesha Vora >Priority: Major > > Component instance page has "node" and "host". These two fields are > representing "bare_host" and "hostname" accordingly. > From UI2 page thats not clear. Thus, table content need to be changed to > "bare host" from "node" . > This page also has "Host URL" which is hard coded to N/A. Thus, removing this > field from table. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Assigned] (YARN-8754) [UI2] Improve terms on Component Instance page
[ https://issues.apache.org/jira/browse/YARN-8754?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yesha Vora reassigned YARN-8754: Assignee: Yesha Vora > [UI2] Improve terms on Component Instance page > --- > > Key: YARN-8754 > URL: https://issues.apache.org/jira/browse/YARN-8754 > Project: Hadoop YARN > Issue Type: Bug > Components: yarn-ui-v2 >Affects Versions: 3.1.1 >Reporter: Yesha Vora >Assignee: Yesha Vora >Priority: Major > > Component instance page has "node" and "host". These two fields are > representing "bare_host" and "hostname" accordingly. > From UI2 page thats not clear. Thus, table content need to be changed to > "bare host" from "node" . > This page also has "Host URL" which is hard coded to N/A. Thus, removing this > field from table. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Created] (YARN-8754) [UI2] Improve terms on Component Instance page
Yesha Vora created YARN-8754: Summary: [UI2] Improve terms on Component Instance page Key: YARN-8754 URL: https://issues.apache.org/jira/browse/YARN-8754 Project: Hadoop YARN Issue Type: Bug Components: yarn-ui-v2 Reporter: Yesha Vora Component instance page has "node" and "host". These two fields are representing "bare_host" and "hostname" accordingly. >From UI2 page thats not clear. Thus, table content need to be changed to "bare >host" from "node" . This page also has "Host URL" which is hard coded to N/A. Thus, removing this field from table. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-8753) [UI2] Lost nodes representation missing from Nodemanagers Chart
[ https://issues.apache.org/jira/browse/YARN-8753?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16607650#comment-16607650 ] Yesha Vora commented on YARN-8753: -- Find the screenshot of Nodemanager chart after adding LOST. !Screen Shot 2018-09-07 at 11.59.02 AM.png! > [UI2] Lost nodes representation missing from Nodemanagers Chart > --- > > Key: YARN-8753 > URL: https://issues.apache.org/jira/browse/YARN-8753 > Project: Hadoop YARN > Issue Type: Bug > Components: yarn-ui-v2 >Affects Versions: 3.1.1 >Reporter: Yesha Vora >Assignee: Yesha Vora >Priority: Major > Attachments: Screen Shot 2018-09-06 at 6.16.02 PM.png, Screen Shot > 2018-09-06 at 6.16.14 PM.png, Screen Shot 2018-09-07 at 11.59.02 AM.png, > YARN-8753.001.patch > > > Nodemanagers Chart is present in Cluster overview and Nodes->Nodes Status > page. > This chart does not show nodemanagers if they are LOST. > Due to this issue, Node information page and Node status page shows different > node managers count. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-8753) [UI2] Lost nodes representation missing from Nodemanagers Chart
[ https://issues.apache.org/jira/browse/YARN-8753?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yesha Vora updated YARN-8753: - Attachment: Screen Shot 2018-09-07 at 11.59.02 AM.png > [UI2] Lost nodes representation missing from Nodemanagers Chart > --- > > Key: YARN-8753 > URL: https://issues.apache.org/jira/browse/YARN-8753 > Project: Hadoop YARN > Issue Type: Bug > Components: yarn-ui-v2 >Affects Versions: 3.1.1 >Reporter: Yesha Vora >Assignee: Yesha Vora >Priority: Major > Attachments: Screen Shot 2018-09-06 at 6.16.02 PM.png, Screen Shot > 2018-09-06 at 6.16.14 PM.png, Screen Shot 2018-09-07 at 11.59.02 AM.png, > YARN-8753.001.patch > > > Nodemanagers Chart is present in Cluster overview and Nodes->Nodes Status > page. > This chart does not show nodemanagers if they are LOST. > Due to this issue, Node information page and Node status page shows different > node managers count. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-8753) [UI2] Lost nodes representation missing from Nodemanagers Chart
[ https://issues.apache.org/jira/browse/YARN-8753?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yesha Vora updated YARN-8753: - Attachment: YARN-8753.001.patch > [UI2] Lost nodes representation missing from Nodemanagers Chart > --- > > Key: YARN-8753 > URL: https://issues.apache.org/jira/browse/YARN-8753 > Project: Hadoop YARN > Issue Type: Bug > Components: yarn-ui-v2 >Affects Versions: 3.1.1 >Reporter: Yesha Vora >Assignee: Yesha Vora >Priority: Major > Attachments: Screen Shot 2018-09-06 at 6.16.02 PM.png, Screen Shot > 2018-09-06 at 6.16.14 PM.png, YARN-8753.001.patch > > > Nodemanagers Chart is present in Cluster overview and Nodes->Nodes Status > page. > This chart does not show nodemanagers if they are LOST. > Due to this issue, Node information page and Node status page shows different > node managers count. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-8753) [UI2] Lost nodes representation missing from Nodemanagers Chart
[ https://issues.apache.org/jira/browse/YARN-8753?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yesha Vora updated YARN-8753: - Description: Nodemanagers Chart is present in Cluster overview and Nodes->Nodes Status page. This chart does not show nodemanagers if they are LOST. Due to this issue, Node information page and Node status page shows different node managers count. was: Nodemanagers Chart is present in Cluster overview and Nodes->Nodes Status page. This chart does not show nodemanagers if they are LOST. > [UI2] Lost nodes representation missing from Nodemanagers Chart > --- > > Key: YARN-8753 > URL: https://issues.apache.org/jira/browse/YARN-8753 > Project: Hadoop YARN > Issue Type: Bug > Components: yarn-ui-v2 >Affects Versions: 3.1.1 >Reporter: Yesha Vora >Assignee: Yesha Vora >Priority: Major > Attachments: Screen Shot 2018-09-06 at 6.16.02 PM.png, Screen Shot > 2018-09-06 at 6.16.14 PM.png > > > Nodemanagers Chart is present in Cluster overview and Nodes->Nodes Status > page. > This chart does not show nodemanagers if they are LOST. > Due to this issue, Node information page and Node status page shows different > node managers count. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-8753) [UI2] Lost nodes representation missing from Nodemanagers Chart
[ https://issues.apache.org/jira/browse/YARN-8753?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yesha Vora updated YARN-8753: - Attachment: Screen Shot 2018-09-06 at 6.16.02 PM.png > [UI2] Lost nodes representation missing from Nodemanagers Chart > --- > > Key: YARN-8753 > URL: https://issues.apache.org/jira/browse/YARN-8753 > Project: Hadoop YARN > Issue Type: Bug > Components: yarn-ui-v2 >Affects Versions: 3.1.1 >Reporter: Yesha Vora >Assignee: Yesha Vora >Priority: Major > Attachments: Screen Shot 2018-09-06 at 6.16.02 PM.png, Screen Shot > 2018-09-06 at 6.16.14 PM.png > > > Nodemanagers Chart is present in Cluster overview and Nodes->Nodes Status > page. > This chart does not show nodemanagers if they are LOST. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-8753) [UI2] Lost nodes representation missing from Nodemanagers Chart
[ https://issues.apache.org/jira/browse/YARN-8753?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yesha Vora updated YARN-8753: - Attachment: Screen Shot 2018-09-06 at 6.16.14 PM.png > [UI2] Lost nodes representation missing from Nodemanagers Chart > --- > > Key: YARN-8753 > URL: https://issues.apache.org/jira/browse/YARN-8753 > Project: Hadoop YARN > Issue Type: Bug > Components: yarn-ui-v2 >Affects Versions: 3.1.1 >Reporter: Yesha Vora >Assignee: Yesha Vora >Priority: Major > Attachments: Screen Shot 2018-09-06 at 6.16.02 PM.png, Screen Shot > 2018-09-06 at 6.16.14 PM.png > > > Nodemanagers Chart is present in Cluster overview and Nodes->Nodes Status > page. > This chart does not show nodemanagers if they are LOST. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Assigned] (YARN-8753) [UI2] Lost nodes representation missing from Nodemanagers Chart
[ https://issues.apache.org/jira/browse/YARN-8753?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yesha Vora reassigned YARN-8753: Assignee: Yesha Vora > [UI2] Lost nodes representation missing from Nodemanagers Chart > --- > > Key: YARN-8753 > URL: https://issues.apache.org/jira/browse/YARN-8753 > Project: Hadoop YARN > Issue Type: Bug > Components: yarn-ui-v2 >Affects Versions: 3.1.1 >Reporter: Yesha Vora >Assignee: Yesha Vora >Priority: Major > > Nodemanagers Chart is present in Cluster overview and Nodes->Nodes Status > page. > This chart does not show nodemanagers if they are LOST. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Created] (YARN-8753) [UI2] Lost nodes representation missing from Nodemanagers Chart
Yesha Vora created YARN-8753: Summary: [UI2] Lost nodes representation missing from Nodemanagers Chart Key: YARN-8753 URL: https://issues.apache.org/jira/browse/YARN-8753 Project: Hadoop YARN Issue Type: Bug Components: yarn-ui-v2 Affects Versions: 3.1.1 Reporter: Yesha Vora Nodemanagers Chart is present in Cluster overview and Nodes->Nodes Status page. This chart does not show nodemanagers if they are LOST. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-8666) [UI2] Remove application tab from Yarn Queue Page
[ https://issues.apache.org/jira/browse/YARN-8666?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16606329#comment-16606329 ] Yesha Vora commented on YARN-8666: -- Patch updated to remove "Applications" from Queue page. The screenshot after removing "Applications" is attached. > [UI2] Remove application tab from Yarn Queue Page > - > > Key: YARN-8666 > URL: https://issues.apache.org/jira/browse/YARN-8666 > Project: Hadoop YARN > Issue Type: Bug > Components: yarn-ui-v2 >Affects Versions: 3.1.1 >Reporter: Yesha Vora >Assignee: Yesha Vora >Priority: Major > Attachments: Screen Shot 2018-08-14 at 3.43.18 PM.png, Screen Shot > 2018-09-06 at 12.50.14 PM.png, YARN-8666.001.patch > > > Yarn UI2 Queue page puts Application button. This button does not redirect to > any other page. In addition to that running application table is also > available on same page. > Thus, there is no need to have a button for application in Queue page. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-8666) [UI2] Remove application tab from Yarn Queue Page
[ https://issues.apache.org/jira/browse/YARN-8666?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yesha Vora updated YARN-8666: - Attachment: YARN-8666.001.patch > [UI2] Remove application tab from Yarn Queue Page > - > > Key: YARN-8666 > URL: https://issues.apache.org/jira/browse/YARN-8666 > Project: Hadoop YARN > Issue Type: Bug > Components: yarn-ui-v2 >Affects Versions: 3.1.1 >Reporter: Yesha Vora >Assignee: Yesha Vora >Priority: Major > Attachments: Screen Shot 2018-08-14 at 3.43.18 PM.png, Screen Shot > 2018-09-06 at 12.50.14 PM.png, YARN-8666.001.patch > > > Yarn UI2 Queue page puts Application button. This button does not redirect to > any other page. In addition to that running application table is also > available on same page. > Thus, there is no need to have a button for application in Queue page. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-8666) [UI2] Remove application tab from Yarn Queue Page
[ https://issues.apache.org/jira/browse/YARN-8666?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yesha Vora updated YARN-8666: - Attachment: Screen Shot 2018-09-06 at 12.50.14 PM.png > [UI2] Remove application tab from Yarn Queue Page > - > > Key: YARN-8666 > URL: https://issues.apache.org/jira/browse/YARN-8666 > Project: Hadoop YARN > Issue Type: Bug > Components: yarn-ui-v2 >Affects Versions: 3.1.1 >Reporter: Yesha Vora >Assignee: Yesha Vora >Priority: Major > Attachments: Screen Shot 2018-08-14 at 3.43.18 PM.png, Screen Shot > 2018-09-06 at 12.50.14 PM.png, YARN-8666.001.patch > > > Yarn UI2 Queue page puts Application button. This button does not redirect to > any other page. In addition to that running application table is also > available on same page. > Thus, there is no need to have a button for application in Queue page. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-8599) Build Master module for MaWo app
[ https://issues.apache.org/jira/browse/YARN-8599?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yesha Vora updated YARN-8599: - Attachment: YARN-8599.001.patch > Build Master module for MaWo app > > > Key: YARN-8599 > URL: https://issues.apache.org/jira/browse/YARN-8599 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Yesha Vora >Assignee: Yesha Vora >Priority: Major > Attachments: YARN-8599.001.patch > > > Master component for MaWo application is responsible for driving end-to-end > job execution. Its responsibility is > * Get Job definition and create a Queue of Tasks > * Assign Tasks to Worker > * Manage Workers lifecycle -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-8598) Build Master Job Module for MaWo Application
[ https://issues.apache.org/jira/browse/YARN-8598?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yesha Vora updated YARN-8598: - Attachment: YARN-8598.001.patch > Build Master Job Module for MaWo Application > > > Key: YARN-8598 > URL: https://issues.apache.org/jira/browse/YARN-8598 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Yesha Vora >Assignee: Yesha Vora >Priority: Major > Attachments: YARN-8598.001.patch > > > A job in MaWo application is a collection of Tasks. A Job consists of a setup > task, a list of tasks and a teardown task. > * JobBuilder > ** SimpleTaskJobBuilder : SimpleJobBuilder should be able to parse > simpleJobdescription file. In this file format, each line is considered as > Task > ** SimpleTaskJsonJobBuilder: Utility to parse json job description file. > -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-8597) Build Worker utility for MaWo Application
[ https://issues.apache.org/jira/browse/YARN-8597?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yesha Vora updated YARN-8597: - Attachment: YARN-8597.001.patch > Build Worker utility for MaWo Application > - > > Key: YARN-8597 > URL: https://issues.apache.org/jira/browse/YARN-8597 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Yesha Vora >Assignee: Yesha Vora >Priority: Major > Attachments: YARN-8597.001.patch > > > The worker is responsible for executing Tasks. > * Worker > ** Create a worker class which drives worker life cycle > ** Create WorkAssignment Protocol. It should be handle Register/deregister > worker, send heartbeat > ** Lifecycle: Register worker, Run Setup Task, Get Task from master and > execute it using TaskRunner, Run Teardown Task > * TaskRunner > ** Simple Task Runner : This runner should be able to execute a simple task > ** Composite Task Runner: This runner should be able to execute composite > task > * TaskWallTimeLimiter > ** Create a utility which can abort the task if the execution time exceeds > task timeout. > -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-8597) Build Worker utility for MaWo Application
[ https://issues.apache.org/jira/browse/YARN-8597?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yesha Vora updated YARN-8597: - Attachment: (was: YARN-8597.001.patch) > Build Worker utility for MaWo Application > - > > Key: YARN-8597 > URL: https://issues.apache.org/jira/browse/YARN-8597 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Yesha Vora >Assignee: Yesha Vora >Priority: Major > Attachments: YARN-8597.001.patch > > > The worker is responsible for executing Tasks. > * Worker > ** Create a worker class which drives worker life cycle > ** Create WorkAssignment Protocol. It should be handle Register/deregister > worker, send heartbeat > ** Lifecycle: Register worker, Run Setup Task, Get Task from master and > execute it using TaskRunner, Run Teardown Task > * TaskRunner > ** Simple Task Runner : This runner should be able to execute a simple task > ** Composite Task Runner: This runner should be able to execute composite > task > * TaskWallTimeLimiter > ** Create a utility which can abort the task if the execution time exceeds > task timeout. > -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-8597) Build Worker utility for MaWo Application
[ https://issues.apache.org/jira/browse/YARN-8597?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yesha Vora updated YARN-8597: - Attachment: YARN-8597.001.patch > Build Worker utility for MaWo Application > - > > Key: YARN-8597 > URL: https://issues.apache.org/jira/browse/YARN-8597 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Yesha Vora >Assignee: Yesha Vora >Priority: Major > Attachments: YARN-8597.001.patch > > > The worker is responsible for executing Tasks. > * Worker > ** Create a worker class which drives worker life cycle > ** Create WorkAssignment Protocol. It should be handle Register/deregister > worker, send heartbeat > ** Lifecycle: Register worker, Run Setup Task, Get Task from master and > execute it using TaskRunner, Run Teardown Task > * TaskRunner > ** Simple Task Runner : This runner should be able to execute a simple task > ** Composite Task Runner: This runner should be able to execute composite > task > * TaskWallTimeLimiter > ** Create a utility which can abort the task if the execution time exceeds > task timeout. > -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-8551) Build Common module for MaWo application
[ https://issues.apache.org/jira/browse/YARN-8551?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16580521#comment-16580521 ] Yesha Vora commented on YARN-8551: -- Please find attached patch#2. > Build Common module for MaWo application > > > Key: YARN-8551 > URL: https://issues.apache.org/jira/browse/YARN-8551 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Yesha Vora >Assignee: Yesha Vora >Priority: Major > Attachments: YARN-8551.001.patch, YARN-8551.002.patch > > > Build Common module for MaWo application. > This module should include defination of Task. A Task should contain > * TaskID > * Task Command > * Task Environment > * Task Timeout > * Task Type > ** Simple Task > *** Its a single Task > ** Composite Task > *** Its a composition of multiple simple tasks > ** Teardown Task > *** Its a last task to be executed after a job is finished > ** Null Task > *** Its a null task -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Assigned] (YARN-8666) Remove application tab from Yarn Queue Page
[ https://issues.apache.org/jira/browse/YARN-8666?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yesha Vora reassigned YARN-8666: Assignee: Yesha Vora > Remove application tab from Yarn Queue Page > --- > > Key: YARN-8666 > URL: https://issues.apache.org/jira/browse/YARN-8666 > Project: Hadoop YARN > Issue Type: Bug > Components: yarn-ui-v2 >Affects Versions: 3.1.1 >Reporter: Yesha Vora >Assignee: Yesha Vora >Priority: Major > Attachments: Screen Shot 2018-08-14 at 3.43.18 PM.png > > > Yarn UI2 Queue page puts Application button. This button does not redirect to > any other page. In addition to that running application table is also > available on same page. > Thus, there is no need to have a button for application in Queue page. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-8666) Remove application tab from Yarn Queue Page
[ https://issues.apache.org/jira/browse/YARN-8666?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yesha Vora updated YARN-8666: - Affects Version/s: 3.1.1 > Remove application tab from Yarn Queue Page > --- > > Key: YARN-8666 > URL: https://issues.apache.org/jira/browse/YARN-8666 > Project: Hadoop YARN > Issue Type: Bug > Components: yarn-ui-v2 >Affects Versions: 3.1.1 >Reporter: Yesha Vora >Priority: Major > Attachments: Screen Shot 2018-08-14 at 3.43.18 PM.png > > > Yarn UI2 Queue page puts Application button. This button does not redirect to > any other page. In addition to that running application table is also > available on same page. > Thus, there is no need to have a button for application in Queue page. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-8666) Remove application tab from Yarn Queue Page
[ https://issues.apache.org/jira/browse/YARN-8666?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yesha Vora updated YARN-8666: - Attachment: Screen Shot 2018-08-14 at 3.43.18 PM.png > Remove application tab from Yarn Queue Page > --- > > Key: YARN-8666 > URL: https://issues.apache.org/jira/browse/YARN-8666 > Project: Hadoop YARN > Issue Type: Bug > Components: yarn-ui-v2 >Affects Versions: 3.1.1 >Reporter: Yesha Vora >Priority: Major > Attachments: Screen Shot 2018-08-14 at 3.43.18 PM.png > > > Yarn UI2 Queue page puts Application button. This button does not redirect to > any other page. In addition to that running application table is also > available on same page. > Thus, there is no need to have a button for application in Queue page. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Created] (YARN-8666) Remove application tab from Yarn Queue Page
Yesha Vora created YARN-8666: Summary: Remove application tab from Yarn Queue Page Key: YARN-8666 URL: https://issues.apache.org/jira/browse/YARN-8666 Project: Hadoop YARN Issue Type: Bug Components: yarn-ui-v2 Reporter: Yesha Vora Yarn UI2 Queue page puts Application button. This button does not redirect to any other page. In addition to that running application table is also available on same page. Thus, there is no need to have a button for application in Queue page. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-8551) Build Common module for MaWo application
[ https://issues.apache.org/jira/browse/YARN-8551?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yesha Vora updated YARN-8551: - Attachment: YARN-8551.002.patch > Build Common module for MaWo application > > > Key: YARN-8551 > URL: https://issues.apache.org/jira/browse/YARN-8551 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Yesha Vora >Assignee: Yesha Vora >Priority: Major > Attachments: YARN-8551.001.patch, YARN-8551.002.patch > > > Build Common module for MaWo application. > This module should include defination of Task. A Task should contain > * TaskID > * Task Command > * Task Environment > * Task Timeout > * Task Type > ** Simple Task > *** Its a single Task > ** Composite Task > *** Its a composition of multiple simple tasks > ** Teardown Task > *** Its a last task to be executed after a job is finished > ** Null Task > *** Its a null task -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-8551) Build Common module for MaWo application
[ https://issues.apache.org/jira/browse/YARN-8551?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yesha Vora updated YARN-8551: - Attachment: YARN-8551.001.patch > Build Common module for MaWo application > > > Key: YARN-8551 > URL: https://issues.apache.org/jira/browse/YARN-8551 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Yesha Vora >Assignee: Yesha Vora >Priority: Major > Attachments: YARN-8551.001.patch > > > Build Common module for MaWo application. > This module should include defination of Task. A Task should contain > * TaskID > * Task Command > * Task Environment > * Task Timeout > * Task Type > ** Simple Task > *** Its a single Task > ** Composite Task > *** Its a composition of multiple simple tasks > ** Teardown Task > *** Its a last task to be executed after a job is finished > ** Null Task > *** Its a null task -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Created] (YARN-8629) Container cleanup failed
Yesha Vora created YARN-8629:
Summary: Container cleanup failed
Key: YARN-8629
URL: https://issues.apache.org/jira/browse/YARN-8629
Project: Hadoop YARN
Issue Type: Bug
Reporter: Yesha Vora
When an application failed to launch container successfully, the cleanup of
container also failed with below message.
{code}
2018-08-06 03:28:20,351 WARN resources.CGroupsHandlerImpl
(CGroupsHandlerImpl.java:checkAndDeleteCgroup(523)) - Failed to read cgroup
tasks file.
java.io.FileNotFoundException:
/sys/fs/cgroup/cpu,cpuacct/hadoop-yarn-tmp-cxx/container_e02_156898541_0010_20_02/tasks
(No such file or directory)
at java.io.FileInputStream.open0(Native Method)
at java.io.FileInputStream.open(FileInputStream.java:195)
at java.io.FileInputStream.(FileInputStream.java:138)
at java.io.FileInputStream.(FileInputStream.java:93)
at
org.apache.hadoop.yarn.server.nodemanager.containermanager.linux.resources.CGroupsHandlerImpl.checkAndDeleteCgroup(CGroupsHandlerImpl.java:507)
at
org.apache.hadoop.yarn.server.nodemanager.containermanager.linux.resources.CGroupsHandlerImpl.deleteCGroup(CGroupsHandlerImpl.java:542)
at
org.apache.hadoop.yarn.server.nodemanager.containermanager.linux.resources.CGroupsCpuResourceHandlerImpl.postComplete(CGroupsCpuResourceHandlerImpl.java:238)
at
org.apache.hadoop.yarn.server.nodemanager.containermanager.linux.resources.ResourceHandlerChain.postComplete(ResourceHandlerChain.java:111)
at
org.apache.hadoop.yarn.server.nodemanager.LinuxContainerExecutor.postComplete(LinuxContainerExecutor.java:964)
at
org.apache.hadoop.yarn.server.nodemanager.LinuxContainerExecutor.reapContainer(LinuxContainerExecutor.java:787)
at
org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.cleanupContainer(ContainerLaunch.java:821)
at
org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainersLauncher.handle(ContainersLauncher.java:161)
at
org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainersLauncher.handle(ContainersLauncher.java:57)
at
org.apache.hadoop.yarn.event.AsyncDispatcher.dispatch(AsyncDispatcher.java:197)
at
org.apache.hadoop.yarn.event.AsyncDispatcher$1.run(AsyncDispatcher.java:126)
at java.lang.Thread.run(Thread.java:748)
2018-08-06 03:28:20,372 WARN resources.CGroupsHandlerImpl
(CGroupsHandlerImpl.java:checkAndDeleteCgroup(523)) - Failed to read cgroup
tasks file.{code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-8407) Container launch exception in AM log should be printed in ERROR level
[
https://issues.apache.org/jira/browse/YARN-8407?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Yesha Vora updated YARN-8407:
-
Attachment: YARN-8407.003.patch
> Container launch exception in AM log should be printed in ERROR level
> -
>
> Key: YARN-8407
> URL: https://issues.apache.org/jira/browse/YARN-8407
> Project: Hadoop YARN
> Issue Type: Bug
>Reporter: Yesha Vora
>Assignee: Yesha Vora
>Priority: Major
> Attachments: YARN-8407.001.patch, YARN-8407.002.patch,
> YARN-8407.003.patch
>
>
> when a container launch is failing due to docker image not available is
> logged as INFO level in AM log.
> Container launch failure should be logged as ERROR.
> Steps:
> launch httpd yarn-service application with invalid docker image
>
> {code:java}
> 2018-06-07 01:51:32,966 [Component dispatcher] INFO
> instance.ComponentInstance - [COMPINSTANCE httpd-0 :
> container_e05_1528335963594_0001_01_02]:
> container_e05_1528335963594_0001_01_02 completed. Reinsert back to
> pending list and requested a new container.
> exitStatus=-1, diagnostics=[2018-06-07 01:51:02.363]Exception from
> container-launch.
> Container id: container_e05_1528335963594_0001_01_02
> Exit code: 7
> Exception message: Launch container failed
> Shell error output: Unable to find image 'xxx/httpd:0.1' locally
> Trying to pull repository xxx/httpd ...
> /usr/bin/docker-current: Get https://xxx/v1/_ping: dial tcp: lookup xxx on
> yyy: no such host.
> See '/usr/bin/docker-current run --help'.
> Shell output: main : command provided 4
> main : run as user is hbase
> main : requested yarn user is hbase
> Creating script paths...
> Creating local dirs...
> Getting exit code file...
> Changing effective user to root...
> Wrote the exit code 7 to
> /grid/0/hadoop/yarn/local/nmPrivate/application_1528335963594_0001/container_e05_1528335963594_0001_01_02/container_e05_1528335963594_0001_01_02.pid.exitcode
> [2018-06-07 01:51:02.393]Diagnostic message from attempt :
> [2018-06-07 01:51:02.394]Container exited with a non-zero exit code 7. Last
> 4096 bytes of stderr.txt :
> [2018-06-07 01:51:32.428]Could not find
> nmPrivate/application_1528335963594_0001/container_e05_1528335963594_0001_01_02//container_e05_1528335963594_0001_01_02.pid
> in any of the directories
> 2018-06-07 01:51:32,966 [Component dispatcher] INFO
> instance.ComponentInstance - [COMPINSTANCE httpd-0 :
> container_e05_1528335963594_0001_01_02] Transitioned from STARTED to INIT
> on STOP event{code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Assigned] (YARN-8599) Build Master module for MaWo app
[ https://issues.apache.org/jira/browse/YARN-8599?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yesha Vora reassigned YARN-8599: Assignee: Yesha Vora > Build Master module for MaWo app > > > Key: YARN-8599 > URL: https://issues.apache.org/jira/browse/YARN-8599 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Yesha Vora >Assignee: Yesha Vora >Priority: Major > > Master component for MaWo application is responsible for driving end-to-end > job execution. Its responsibility is > * Get Job definition and create a Queue of Tasks > * Assign Tasks to Worker > * Manage Workers lifecycle -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Created] (YARN-8599) Build Master module for MaWo app
Yesha Vora created YARN-8599: Summary: Build Master module for MaWo app Key: YARN-8599 URL: https://issues.apache.org/jira/browse/YARN-8599 Project: Hadoop YARN Issue Type: Sub-task Reporter: Yesha Vora Master component for MaWo application is responsible for driving end-to-end job execution. Its responsibility is * Get Job definition and create a Queue of Tasks * Assign Tasks to Worker * Manage Workers lifecycle -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Created] (YARN-8598) Build Master Job Module for MaWo Application
Yesha Vora created YARN-8598: Summary: Build Master Job Module for MaWo Application Key: YARN-8598 URL: https://issues.apache.org/jira/browse/YARN-8598 Project: Hadoop YARN Issue Type: Sub-task Reporter: Yesha Vora A job in MaWo application is a collection of Tasks. A Job consists of a setup task, a list of tasks and a teardown task. * JobBuilder ** SimpleTaskJobBuilder : SimpleJobBuilder should be able to parse simpleJobdescription file. In this file format, each line is considered as Task ** SimpleTaskJsonJobBuilder: Utility to parse json job description file. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Assigned] (YARN-8598) Build Master Job Module for MaWo Application
[ https://issues.apache.org/jira/browse/YARN-8598?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yesha Vora reassigned YARN-8598: Assignee: Yesha Vora > Build Master Job Module for MaWo Application > > > Key: YARN-8598 > URL: https://issues.apache.org/jira/browse/YARN-8598 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Yesha Vora >Assignee: Yesha Vora >Priority: Major > > A job in MaWo application is a collection of Tasks. A Job consists of a setup > task, a list of tasks and a teardown task. > * JobBuilder > ** SimpleTaskJobBuilder : SimpleJobBuilder should be able to parse > simpleJobdescription file. In this file format, each line is considered as > Task > ** SimpleTaskJsonJobBuilder: Utility to parse json job description file. > -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Assigned] (YARN-8597) Build Worker utility for MaWo Application
[ https://issues.apache.org/jira/browse/YARN-8597?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yesha Vora reassigned YARN-8597: Assignee: Yesha Vora > Build Worker utility for MaWo Application > - > > Key: YARN-8597 > URL: https://issues.apache.org/jira/browse/YARN-8597 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Yesha Vora >Assignee: Yesha Vora >Priority: Major > > The worker is responsible for executing Tasks. > * Worker > ** Create a worker class which drives worker life cycle > ** Create WorkAssignment Protocol. It should be handle Register/deregister > worker, send heartbeat > ** Lifecycle: Register worker, Run Setup Task, Get Task from master and > execute it using TaskRunner, Run Teardown Task > * TaskRunner > ** Simple Task Runner : This runner should be able to execute a simple task > ** Composite Task Runner: This runner should be able to execute composite > task > * TaskWallTimeLimiter > ** Create a utility which can abort the task if the execution time exceeds > task timeout. > -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Created] (YARN-8597) Build Worker utility for MaWo Application
Yesha Vora created YARN-8597: Summary: Build Worker utility for MaWo Application Key: YARN-8597 URL: https://issues.apache.org/jira/browse/YARN-8597 Project: Hadoop YARN Issue Type: Sub-task Reporter: Yesha Vora The worker is responsible for executing Tasks. * Worker ** Create a worker class which drives worker life cycle ** Create WorkAssignment Protocol. It should be handle Register/deregister worker, send heartbeat ** Lifecycle: Register worker, Run Setup Task, Get Task from master and execute it using TaskRunner, Run Teardown Task * TaskRunner ** Simple Task Runner : This runner should be able to execute a simple task ** Composite Task Runner: This runner should be able to execute composite task * TaskWallTimeLimiter ** Create a utility which can abort the task if the execution time exceeds task timeout. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-8407) Container launch exception in AM log should be printed in ERROR level
[
https://issues.apache.org/jira/browse/YARN-8407?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16559063#comment-16559063
]
Yesha Vora commented on YARN-8407:
--
Thanks [~bibinchundatt] for review. Patch-2 is submitted.
> Container launch exception in AM log should be printed in ERROR level
> -
>
> Key: YARN-8407
> URL: https://issues.apache.org/jira/browse/YARN-8407
> Project: Hadoop YARN
> Issue Type: Bug
>Reporter: Yesha Vora
>Assignee: Yesha Vora
>Priority: Major
> Attachments: YARN-8407.001.patch, YARN-8407.002.patch
>
>
> when a container launch is failing due to docker image not available is
> logged as INFO level in AM log.
> Container launch failure should be logged as ERROR.
> Steps:
> launch httpd yarn-service application with invalid docker image
>
> {code:java}
> 2018-06-07 01:51:32,966 [Component dispatcher] INFO
> instance.ComponentInstance - [COMPINSTANCE httpd-0 :
> container_e05_1528335963594_0001_01_02]:
> container_e05_1528335963594_0001_01_02 completed. Reinsert back to
> pending list and requested a new container.
> exitStatus=-1, diagnostics=[2018-06-07 01:51:02.363]Exception from
> container-launch.
> Container id: container_e05_1528335963594_0001_01_02
> Exit code: 7
> Exception message: Launch container failed
> Shell error output: Unable to find image 'xxx/httpd:0.1' locally
> Trying to pull repository xxx/httpd ...
> /usr/bin/docker-current: Get https://xxx/v1/_ping: dial tcp: lookup xxx on
> yyy: no such host.
> See '/usr/bin/docker-current run --help'.
> Shell output: main : command provided 4
> main : run as user is hbase
> main : requested yarn user is hbase
> Creating script paths...
> Creating local dirs...
> Getting exit code file...
> Changing effective user to root...
> Wrote the exit code 7 to
> /grid/0/hadoop/yarn/local/nmPrivate/application_1528335963594_0001/container_e05_1528335963594_0001_01_02/container_e05_1528335963594_0001_01_02.pid.exitcode
> [2018-06-07 01:51:02.393]Diagnostic message from attempt :
> [2018-06-07 01:51:02.394]Container exited with a non-zero exit code 7. Last
> 4096 bytes of stderr.txt :
> [2018-06-07 01:51:32.428]Could not find
> nmPrivate/application_1528335963594_0001/container_e05_1528335963594_0001_01_02//container_e05_1528335963594_0001_01_02.pid
> in any of the directories
> 2018-06-07 01:51:32,966 [Component dispatcher] INFO
> instance.ComponentInstance - [COMPINSTANCE httpd-0 :
> container_e05_1528335963594_0001_01_02] Transitioned from STARTED to INIT
> on STOP event{code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-8407) Container launch exception in AM log should be printed in ERROR level
[
https://issues.apache.org/jira/browse/YARN-8407?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Yesha Vora updated YARN-8407:
-
Attachment: YARN-8407.002.patch
> Container launch exception in AM log should be printed in ERROR level
> -
>
> Key: YARN-8407
> URL: https://issues.apache.org/jira/browse/YARN-8407
> Project: Hadoop YARN
> Issue Type: Bug
>Reporter: Yesha Vora
>Assignee: Yesha Vora
>Priority: Major
> Attachments: YARN-8407.001.patch, YARN-8407.002.patch
>
>
> when a container launch is failing due to docker image not available is
> logged as INFO level in AM log.
> Container launch failure should be logged as ERROR.
> Steps:
> launch httpd yarn-service application with invalid docker image
>
> {code:java}
> 2018-06-07 01:51:32,966 [Component dispatcher] INFO
> instance.ComponentInstance - [COMPINSTANCE httpd-0 :
> container_e05_1528335963594_0001_01_02]:
> container_e05_1528335963594_0001_01_02 completed. Reinsert back to
> pending list and requested a new container.
> exitStatus=-1, diagnostics=[2018-06-07 01:51:02.363]Exception from
> container-launch.
> Container id: container_e05_1528335963594_0001_01_02
> Exit code: 7
> Exception message: Launch container failed
> Shell error output: Unable to find image 'xxx/httpd:0.1' locally
> Trying to pull repository xxx/httpd ...
> /usr/bin/docker-current: Get https://xxx/v1/_ping: dial tcp: lookup xxx on
> yyy: no such host.
> See '/usr/bin/docker-current run --help'.
> Shell output: main : command provided 4
> main : run as user is hbase
> main : requested yarn user is hbase
> Creating script paths...
> Creating local dirs...
> Getting exit code file...
> Changing effective user to root...
> Wrote the exit code 7 to
> /grid/0/hadoop/yarn/local/nmPrivate/application_1528335963594_0001/container_e05_1528335963594_0001_01_02/container_e05_1528335963594_0001_01_02.pid.exitcode
> [2018-06-07 01:51:02.393]Diagnostic message from attempt :
> [2018-06-07 01:51:02.394]Container exited with a non-zero exit code 7. Last
> 4096 bytes of stderr.txt :
> [2018-06-07 01:51:32.428]Could not find
> nmPrivate/application_1528335963594_0001/container_e05_1528335963594_0001_01_02//container_e05_1528335963594_0001_01_02.pid
> in any of the directories
> 2018-06-07 01:51:32,966 [Component dispatcher] INFO
> instance.ComponentInstance - [COMPINSTANCE httpd-0 :
> container_e05_1528335963594_0001_01_02] Transitioned from STARTED to INIT
> on STOP event{code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Created] (YARN-8587) Delays are noticed to launch docker container
Yesha Vora created YARN-8587:
Summary: Delays are noticed to launch docker container
Key: YARN-8587
URL: https://issues.apache.org/jira/browse/YARN-8587
Project: Hadoop YARN
Issue Type: Bug
Affects Versions: 3.1.1
Reporter: Yesha Vora
Launch dshell application. Wait for application to go in RUNNING state.
{code:java}
yarn jar /xx/hadoop-yarn-applications-distributedshell-*.jar -shell_command
"sleep 300" -num_containers 1 -shell_env YARN_CONTAINER_RUNTIME_TYPE=docker
-shell_env YARN_CONTAINER_RUNTIME_DOCKER_IMAGE=httpd:0.1 -shell_env
YARN_CONTAINER_RUNTIME_DOCKER_DELAYED_REMOVAL=true -jar
/usr/hdp/current/hadoop-yarn-client/hadoop-yarn-applications-distributedshell-xx.jar
{code}
Find out container allocation. Run docker inspect command for docker containers
launched by app.
Sometimes, the container is allocated to NM but docker PID is not up.
{code:java}
Command ssh -q -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null xxx
"sudo su - -c \"docker ps -a | grep
container_e02_1531189225093_0003_01_02\" root" failed after 0 retries
{code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-8407) Container launch exception in AM log should be printed in ERROR level
[
https://issues.apache.org/jira/browse/YARN-8407?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Yesha Vora updated YARN-8407:
-
Attachment: YARN-8407.001.patch
> Container launch exception in AM log should be printed in ERROR level
> -
>
> Key: YARN-8407
> URL: https://issues.apache.org/jira/browse/YARN-8407
> Project: Hadoop YARN
> Issue Type: Bug
>Reporter: Yesha Vora
>Assignee: Yesha Vora
>Priority: Major
> Attachments: YARN-8407.001.patch
>
>
> when a container launch is failing due to docker image not available is
> logged as INFO level in AM log.
> Container launch failure should be logged as ERROR.
> Steps:
> launch httpd yarn-service application with invalid docker image
>
> {code:java}
> 2018-06-07 01:51:32,966 [Component dispatcher] INFO
> instance.ComponentInstance - [COMPINSTANCE httpd-0 :
> container_e05_1528335963594_0001_01_02]:
> container_e05_1528335963594_0001_01_02 completed. Reinsert back to
> pending list and requested a new container.
> exitStatus=-1, diagnostics=[2018-06-07 01:51:02.363]Exception from
> container-launch.
> Container id: container_e05_1528335963594_0001_01_02
> Exit code: 7
> Exception message: Launch container failed
> Shell error output: Unable to find image 'xxx/httpd:0.1' locally
> Trying to pull repository xxx/httpd ...
> /usr/bin/docker-current: Get https://xxx/v1/_ping: dial tcp: lookup xxx on
> yyy: no such host.
> See '/usr/bin/docker-current run --help'.
> Shell output: main : command provided 4
> main : run as user is hbase
> main : requested yarn user is hbase
> Creating script paths...
> Creating local dirs...
> Getting exit code file...
> Changing effective user to root...
> Wrote the exit code 7 to
> /grid/0/hadoop/yarn/local/nmPrivate/application_1528335963594_0001/container_e05_1528335963594_0001_01_02/container_e05_1528335963594_0001_01_02.pid.exitcode
> [2018-06-07 01:51:02.393]Diagnostic message from attempt :
> [2018-06-07 01:51:02.394]Container exited with a non-zero exit code 7. Last
> 4096 bytes of stderr.txt :
> [2018-06-07 01:51:32.428]Could not find
> nmPrivate/application_1528335963594_0001/container_e05_1528335963594_0001_01_02//container_e05_1528335963594_0001_01_02.pid
> in any of the directories
> 2018-06-07 01:51:32,966 [Component dispatcher] INFO
> instance.ComponentInstance - [COMPINSTANCE httpd-0 :
> container_e05_1528335963594_0001_01_02] Transitioned from STARTED to INIT
> on STOP event{code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Created] (YARN-8580) yarn.resourcemanager.am.max-attempts is not respected for yarn services
Yesha Vora created YARN-8580:
Summary: yarn.resourcemanager.am.max-attempts is not respected for
yarn services
Key: YARN-8580
URL: https://issues.apache.org/jira/browse/YARN-8580
Project: Hadoop YARN
Issue Type: Bug
Components: yarn-native-services
Affects Versions: 3.1.1
Reporter: Yesha Vora
1) Max am attempt is set to 100 on all nodes. ( including gateway)
{code}
yarn.resourcemanager.am.max-attempts
100
{code}
2) Start a Yarn service ( Hbase tarball ) application
3) Kill AM 20 times
Here, App fails with below diagnostics.
{code}
bash-4.2$ /usr/hdp/current/hadoop-yarn-client/bin/yarn application -status
application_1532481557746_0001
18/07/25 18:43:34 INFO client.AHSProxy: Connecting to Application History
server at xxx/xxx:10200
18/07/25 18:43:34 INFO client.ConfiguredRMFailoverProxyProvider: Failing over
to rm2
18/07/25 18:43:34 INFO conf.Configuration: found resource resource-types.xml at
file:/etc/hadoop/3.0.0.0-1634/0/resource-types.xml
Application Report :
Application-Id : application_1532481557746_0001
Application-Name : hbase-tarball-lr
Application-Type : yarn-service
User : hbase
Queue : default
Application Priority : 0
Start-Time : 1532481864863
Finish-Time : 1532522943103
Progress : 100%
State : FAILED
Final-State : FAILED
Tracking-URL :
https://xxx:8090/cluster/app/application_1532481557746_0001
RPC Port : -1
AM Host : N/A
Aggregate Resource Allocation : 252150112 MB-seconds, 164141
vcore-seconds
Aggregate Resource Preempted : 0 MB-seconds, 0 vcore-seconds
Log Aggregation Status : SUCCEEDED
Diagnostics : Application application_1532481557746_0001 failed 20
times (global limit =100; local limit is =20) due to AM Container for
appattempt_1532481557746_0001_20 exited with exitCode: 137
Failing this attempt.Diagnostics: [2018-07-25 12:49:00.784]Container killed on
request. Exit code is 137
[2018-07-25 12:49:03.045]Container exited with a non-zero exit code 137.
[2018-07-25 12:49:03.045]Killed by external signal
For more detailed output, check the application tracking page:
https://xxx:8090/cluster/app/application_1532481557746_0001 Then click on links
to logs of each attempt.
. Failing the application.
Unmanaged Application : false
Application Node Label Expression :
AM container Node Label Expression :
TimeoutType : LIFETIME ExpiryTime : 2018-07-25T22:26:15.419+
RemainingTime : 0seconds
{code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Created] (YARN-8579) New AM attempt could not retrieve previous attempt component data
Yesha Vora created YARN-8579:
Summary: New AM attempt could not retrieve previous attempt
component data
Key: YARN-8579
URL: https://issues.apache.org/jira/browse/YARN-8579
Project: Hadoop YARN
Issue Type: Bug
Affects Versions: 3.1.1
Reporter: Yesha Vora
Steps:
1) Launch httpd-docker
2) Wait for app to be in STABLE state
3) Run validation for app (It takes around 3 mins)
4) Stop all Zks
5) Wait 60 sec
6) Kill AM
7) wait for 30 sec
8) Start all ZKs
9) Wait for application to finish
10) Validate expected containers of the app
Expected behavior:
New attempt of AM should start and docker containers launched by 1st attempt
should be recovered by new attempt.
Actual behavior:
New AM attempt starts. It can not recover 1st attempt docker containers. It can
not read component details from ZK.
Thus, it starts new attempt for all containers.
{code}
2018-07-19 22:42:47,595 [main] INFO service.ServiceScheduler - Registering
appattempt_1531977563978_0015_02, fault-test-zkrm-httpd-docker into registry
2018-07-19 22:42:47,611 [main] INFO service.ServiceScheduler - Received 1
containers from previous attempt.
2018-07-19 22:42:47,642 [main] INFO service.ServiceScheduler - Could not read
component paths:
`/users/hrt-qa/services/yarn-service/fault-test-zkrm-httpd-docker/components':
No such file or directory: KeeperErrorCode = NoNode for
/registry/users/hrt-qa/services/yarn-service/fault-test-zkrm-httpd-docker/components
2018-07-19 22:42:47,643 [main] INFO service.ServiceScheduler - Handling
container_e08_1531977563978_0015_01_03 from previous attempt
2018-07-19 22:42:47,643 [main] INFO service.ServiceScheduler - Record not
found in registry for container container_e08_1531977563978_0015_01_03 from
previous attempt, releasing
2018-07-19 22:42:47,649 [AMRM Callback Handler Thread] INFO
impl.TimelineV2ClientImpl - Updated timeline service address to xxx:33019
2018-07-19 22:42:47,651 [main] INFO service.ServiceScheduler - Triggering
initial evaluation of component httpd
2018-07-19 22:42:47,652 [main] INFO component.Component - [INIT COMPONENT
httpd]: 2 instances.
2018-07-19 22:42:47,652 [main] INFO component.Component - [COMPONENT httpd]
Requesting for 2 container(s){code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
