[jira] [Commented] (YARN-4011) Jobs fail since nm-local-dir not cleaned up when rogue job fills up disk

[ https://issues.apache.org/jira/browse/YARN-4011?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14904697#comment-14904697 ] Maysam Yabandeh commented on YARN-4011: --- Thanks [~jlowe]. I created MAPREDUCE-6489 for this. > Jobs

[jira] [Commented] (YARN-4011) Jobs fail since nm-local-dir not cleaned up when rogue job fills up disk

[ https://issues.apache.org/jira/browse/YARN-4011?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14901946#comment-14901946 ] Maysam Yabandeh commented on YARN-4011: --- We face this problem quite often in our ad hoc cluster and

[jira] [Resolved] (YARN-2007) AM expressing the estimated endtime to RM

[ https://issues.apache.org/jira/browse/YARN-2007?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maysam Yabandeh resolved YARN-2007. --- Resolution: Not A Problem AM expressing the estimated endtime to RM

[jira] [Resolved] (YARN-1969) Fair Scheduler: Add policy for Earliest Endtime First

[ https://issues.apache.org/jira/browse/YARN-1969?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maysam Yabandeh resolved YARN-1969. --- Resolution: Not A Problem After the fix in MAPREDUCE-5844 and a couple of other issues, I no

[jira] [Commented] (YARN-1067) AMRMClient heartbeat interval should not be static

[ https://issues.apache.org/jira/browse/YARN-1067?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14523922#comment-14523922 ] Maysam Yabandeh commented on YARN-1067: --- [~xgong] I do not currently have cycles to

[jira] [Commented] (YARN-1963) Support priorities across applications within the same queue

[ https://issues.apache.org/jira/browse/YARN-1963?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14148474#comment-14148474 ] Maysam Yabandeh commented on YARN-1963: --- Thanks [~sunilg] for the design doc. It

[jira] [Commented] (YARN-1963) Support priorities across applications within the same queue

[ https://issues.apache.org/jira/browse/YARN-1963?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14137496#comment-14137496 ] Maysam Yabandeh commented on YARN-1963: --- I was wondering what is the long-term plan

[jira] [Commented] (YARN-1530) [Umbrella] Store, manage and serve per-framework application-timeline data

[ https://issues.apache.org/jira/browse/YARN-1530?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14134024#comment-14134024 ] Maysam Yabandeh commented on YARN-1530: --- bq. YARN apps already depend on ZK/RM/HDFS

[jira] [Commented] (YARN-2405) NPE in FairSchedulerAppsBlock (scheduler page)

[ https://issues.apache.org/jira/browse/YARN-2405?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14112447#comment-14112447 ] Maysam Yabandeh commented on YARN-2405: --- LGTM. A minor comment: I was thinking that

[jira] [Commented] (YARN-2405) NPE in FairSchedulerAppsBlock (scheduler page)

[ https://issues.apache.org/jira/browse/YARN-2405?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14111597#comment-14111597 ] Maysam Yabandeh commented on YARN-2405: --- I am thinking of a simple patch that catches

[jira] [Updated] (YARN-2430) FairShareComparator: cache the results of getResourceUsage()

[ https://issues.apache.org/jira/browse/YARN-2430?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maysam Yabandeh updated YARN-2430: -- Assignee: Sandy Ryza (was: Maysam Yabandeh) FairShareComparator: cache the results of

[jira] [Commented] (YARN-2405) NPE in FairSchedulerAppsBlock (scheduler page)

[ https://issues.apache.org/jira/browse/YARN-2405?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14111725#comment-14111725 ] Maysam Yabandeh commented on YARN-2405: --- [~ozawa], we got the error in a fork of

[jira] [Commented] (YARN-2405) NPE in FairSchedulerAppsBlock (scheduler page)

[ https://issues.apache.org/jira/browse/YARN-2405?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14111758#comment-14111758 ] Maysam Yabandeh commented on YARN-2405: --- Sounds good to me. We also need to decide of

[jira] [Commented] (YARN-2405) NPE in FairSchedulerAppsBlock (scheduler page)

[ https://issues.apache.org/jira/browse/YARN-2405?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14111770#comment-14111770 ] Maysam Yabandeh commented on YARN-2405: --- [~ozawa], all right then. Looking forward

[jira] [Commented] (YARN-2430) FairShareComparator: cache the results of getResourceUsage()

[ https://issues.apache.org/jira/browse/YARN-2430?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14101686#comment-14101686 ] Maysam Yabandeh commented on YARN-2430: --- Here are the current alternative solutions:

[jira] [Created] (YARN-2430) FairShareComparator: cache the results of getResourceUsage()

Maysam Yabandeh created YARN-2430: - Summary: FairShareComparator: cache the results of getResourceUsage() Key: YARN-2430 URL: https://issues.apache.org/jira/browse/YARN-2430 Project: Hadoop YARN

[jira] [Commented] (YARN-2405) NPE in FairSchedulerAppsBlock (scheduler page)

[ https://issues.apache.org/jira/browse/YARN-2405?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14093111#comment-14093111 ] Maysam Yabandeh commented on YARN-2405: --- The problem seems to be that the two

[jira] [Created] (YARN-2405) NPE in FairSchedulerAppsBlock (scheduler page)

Maysam Yabandeh created YARN-2405: - Summary: NPE in FairSchedulerAppsBlock (scheduler page) Key: YARN-2405 URL: https://issues.apache.org/jira/browse/YARN-2405 Project: Hadoop YARN Issue

[jira] [Commented] (YARN-2104) Scheduler queue filter failed to work because index of queue column changed

[ https://issues.apache.org/jira/browse/YARN-2104?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14046425#comment-14046425 ] Maysam Yabandeh commented on YARN-2104: --- +1 Worked for us. And the failed unit test

[jira] [Commented] (YARN-2162) Fair Scheduler :ability to configure minResources and maxResources in terms of percentage

[ https://issues.apache.org/jira/browse/YARN-2162?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14033399#comment-14033399 ] Maysam Yabandeh commented on YARN-2162: --- If we add this feature, it should certainly

[jira] [Commented] (YARN-1969) Fair Scheduler: Add policy for Earliest Deadline First

[ https://issues.apache.org/jira/browse/YARN-1969?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13996444#comment-13996444 ] Maysam Yabandeh commented on YARN-1969: --- [~kkambatl], you are right. The title of the

[jira] [Commented] (YARN-1969) Fair Scheduler: Add policy for Earliest Deadline First

[ https://issues.apache.org/jira/browse/YARN-1969?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13995996#comment-13995996 ] Maysam Yabandeh commented on YARN-1969: --- Talked to [~jira.shegalov] offline. This

[jira] [Commented] (YARN-1969) Fair Scheduler: Add policy for Earliest Deadline First

[ https://issues.apache.org/jira/browse/YARN-1969?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13995993#comment-13995993 ] Maysam Yabandeh commented on YARN-1969: --- Thanks for the comment [~kasha]. I think

[jira] [Created] (YARN-2006) Estimate Job Endtime

Maysam Yabandeh created YARN-2006: - Summary: Estimate Job Endtime Key: YARN-2006 URL: https://issues.apache.org/jira/browse/YARN-2006 Project: Hadoop YARN Issue Type: Sub-task

[jira] [Created] (YARN-2007) AM expressing the estimated endtime to RM

Maysam Yabandeh created YARN-2007: - Summary: AM expressing the estimated endtime to RM Key: YARN-2007 URL: https://issues.apache.org/jira/browse/YARN-2007 Project: Hadoop YARN Issue Type:

[jira] [Commented] (YARN-2006) Estimate Job Endtime

[ https://issues.apache.org/jira/browse/YARN-2006?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13985582#comment-13985582 ] Maysam Yabandeh commented on YARN-2006: --- We built on top of the already existing

[jira] [Updated] (YARN-2006) Estimate Job Endtime

[ https://issues.apache.org/jira/browse/YARN-2006?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maysam Yabandeh updated YARN-2006: -- Attachment: YARN-1969.patch Estimate Job Endtime Key:

[jira] [Updated] (YARN-2006) Estimate Job Endtime

[ https://issues.apache.org/jira/browse/YARN-2006?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maysam Yabandeh updated YARN-2006: -- Issue Type: Improvement (was: Sub-task) Parent: (was: YARN-1969) Estimate Job

[jira] [Updated] (YARN-2007) AM expressing the estimated endtime to RM

[ https://issues.apache.org/jira/browse/YARN-2007?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maysam Yabandeh updated YARN-2007: -- Description: YARN-1969 adds a new earliest-endtime-first policy to the fair scheduler, which

[jira] [Commented] (YARN-1955) Reducer Preemption is too agressive

[ https://issues.apache.org/jira/browse/YARN-1955?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13973150#comment-13973150 ] Maysam Yabandeh commented on YARN-1955: --- I am thinking of a patch that postpones

[jira] [Created] (YARN-1955) Reducer Preemption is too agressive

Maysam Yabandeh created YARN-1955: - Summary: Reducer Preemption is too agressive Key: YARN-1955 URL: https://issues.apache.org/jira/browse/YARN-1955 Project: Hadoop YARN Issue Type: Bug

[jira] [Updated] (YARN-1955) Reducer Preemption is too aggressive

[ https://issues.apache.org/jira/browse/YARN-1955?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maysam Yabandeh updated YARN-1955: -- Summary: Reducer Preemption is too aggressive (was: Reducer Preemption is too agressive)

[jira] [Commented] (YARN-779) AMRMClient should clean up dangling unsatisfied request

[ https://issues.apache.org/jira/browse/YARN-779?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13774293#comment-13774293 ] Maysam Yabandeh commented on YARN-779: -- Sure [~acmurthy]. Note taken for other jiras.

[jira] [Commented] (YARN-1067) AMRMClient heartbeat interval should not be static

[ https://issues.apache.org/jira/browse/YARN-1067?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13760951#comment-13760951 ] Maysam Yabandeh commented on YARN-1067: --- Thanks for the comments [~sseth].

[jira] [Updated] (YARN-779) AMRMClient should clean up dangling unsatisfied request

[ https://issues.apache.org/jira/browse/YARN-779?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maysam Yabandeh updated YARN-779: - Attachment: YARN-779.patch Fixed the test case with correct setting for relaxedLocality.

[jira] [Commented] (YARN-779) AMRMClient should clean up dangling unsatisfied request

[ https://issues.apache.org/jira/browse/YARN-779?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13749341#comment-13749341 ] Maysam Yabandeh commented on YARN-779: -- I am thinking perhaps we can solve the problem

[jira] [Updated] (YARN-1067) AMRMClient heartbeat interval should not be static

[ https://issues.apache.org/jira/browse/YARN-1067?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maysam Yabandeh updated YARN-1067: -- Attachment: YARN-1067.patch The attached patch takes the first stab. Comments are appreciated.

[jira] [Updated] (YARN-1076) RM gets stuck with a reservation, ignoring new containers

[ https://issues.apache.org/jira/browse/YARN-1076?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maysam Yabandeh updated YARN-1076: -- Attachment: YARN-1076.patch I am attaching a unit test that shows the problem.

[jira] [Updated] (YARN-1076) RM gets stuck with a reservation, ignoring new containers

[ https://issues.apache.org/jira/browse/YARN-1076?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maysam Yabandeh updated YARN-1076: -- Description: LeafQueue#assignToQueue rejects newly available containers if

[jira] [Commented] (YARN-1076) RM gets stuck with a reservation, ignoring new containers

[ https://issues.apache.org/jira/browse/YARN-1076?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13745817#comment-13745817 ] Maysam Yabandeh commented on YARN-1076: --- [~ojoshi], I applied the latest patch of

[jira] [Commented] (YARN-1076) RM gets stuck with a reservation, ignoring new containers

[ https://issues.apache.org/jira/browse/YARN-1076?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13744683#comment-13744683 ] Maysam Yabandeh commented on YARN-1076: --- Hi [~ojoshi]. I am observing the problem

[jira] [Commented] (YARN-1069) Deterministic MiniYARNCluster

[ https://issues.apache.org/jira/browse/YARN-1069?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13742865#comment-13742865 ] Maysam Yabandeh commented on YARN-1069: --- Thanks for the pointer [~vinodkv]. The

[jira] [Created] (YARN-1076) RM gets stuck with a reservation, ignoring new containers

Maysam Yabandeh created YARN-1076: - Summary: RM gets stuck with a reservation, ignoring new containers Key: YARN-1076 URL: https://issues.apache.org/jira/browse/YARN-1076 Project: Hadoop YARN

[jira] [Commented] (YARN-1016) Define a HDFS based repository that allows YARN services to share resources

[ https://issues.apache.org/jira/browse/YARN-1016?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13741970#comment-13741970 ] Maysam Yabandeh commented on YARN-1016: --- [~kkasravi], your suggestion seems related

[jira] [Created] (YARN-1069) Deterministic MiniYARNCluster

Maysam Yabandeh created YARN-1069: - Summary: Deterministic MiniYARNCluster Key: YARN-1069 URL: https://issues.apache.org/jira/browse/YARN-1069 Project: Hadoop YARN Issue Type: Test

[jira] [Commented] (YARN-1069) Deterministic MiniYARNCluster

[ https://issues.apache.org/jira/browse/YARN-1069?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13742003#comment-13742003 ] Maysam Yabandeh commented on YARN-1069: --- One example I have in mind is sending NM

[jira] [Updated] (YARN-779) AMRMClient should clean up dangling unsatisfied request

[ https://issues.apache.org/jira/browse/YARN-779?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maysam Yabandeh updated YARN-779: - Attachment: YARN-779.patch The attached patch includes a unit test that shows how the bug

[jira] [Commented] (YARN-713) ResourceManager can exit unexpectedly if DNS is unavailable

[ https://issues.apache.org/jira/browse/YARN-713?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13713322#comment-13713322 ] Maysam Yabandeh commented on YARN-713: -- [~ojoshi], but the patch was ready since June

[jira] [Updated] (YARN-779) AMRMClient should clean up dangling unsatisfied request

[ https://issues.apache.org/jira/browse/YARN-779?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maysam Yabandeh updated YARN-779: - Assignee: (was: Maysam Yabandeh) AMRMClient should clean up dangling unsatisfied request

[jira] [Commented] (YARN-779) AMRMClient should clean up dangling unsatisfied request

[ https://issues.apache.org/jira/browse/YARN-779?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13684104#comment-13684104 ] Maysam Yabandeh commented on YARN-779: -- Thanks [~sandyr]. Let me run by you my

[jira] [Commented] (YARN-659) RMStateStore's removeApplication APIs should just take an applicationId

[ https://issues.apache.org/jira/browse/YARN-659?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13684136#comment-13684136 ] Maysam Yabandeh commented on YARN-659: -- But, it seems that FileSystemRMStateStore uses

[jira] [Commented] (YARN-779) AMRMClient should clean up dangling unsatisfied request

[ https://issues.apache.org/jira/browse/YARN-779?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13683407#comment-13683407 ] Maysam Yabandeh commented on YARN-779: -- Can you please explain the erroneous scenario a

[jira] [Assigned] (YARN-779) AMRMClient should clean up dangling unsatisfied request

[ https://issues.apache.org/jira/browse/YARN-779?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maysam Yabandeh reassigned YARN-779: Assignee: Maysam Yabandeh AMRMClient should clean up dangling unsatisfied request

[jira] [Updated] (YARN-713) ResourceManager can exit unexpectedly if DNS is unavailable

[ https://issues.apache.org/jira/browse/YARN-713?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maysam Yabandeh updated YARN-713: - Attachment: YARN-713.patch The attached patch applied the comments of [~vinodkv]. The unit test is

[jira] [Commented] (YARN-713) ResourceManager can exit unexpectedly if DNS is unavailable

[ https://issues.apache.org/jira/browse/YARN-713?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13678263#comment-13678263 ] Maysam Yabandeh commented on YARN-713: -- Thanks [~tgraves]. The changes of this patch is

[jira] [Updated] (YARN-713) ResourceManager can exit unexpectedly if DNS is unavailable

[ https://issues.apache.org/jira/browse/YARN-713?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maysam Yabandeh updated YARN-713: - Attachment: YARN-713.patch In the attached patch, the exception is handled in

[jira] [Updated] (YARN-512) Log aggregation root directory check is more expensive than it needs to be

[ https://issues.apache.org/jira/browse/YARN-512?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maysam Yabandeh updated YARN-512: - Attachment: YARN-512.patch The patch applies the changes requested by the jira, which are minor.