[jira] [Updated] (YARN-4053) Change the way metric values are stored in HBase Storage

2015-08-16 Thread Varun Saxena (JIRA)
[ https://issues.apache.org/jira/browse/YARN-4053?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Varun Saxena updated YARN-4053: --- Attachment: YARN-4053-YARN-2928.01.patch Change the way metric values are stored in HBase Storage

[jira] [Updated] (YARN-3224) Notify AM with containers (on decommissioning node) could be preempted after timeout.

2015-08-16 Thread Sunil G (JIRA)
[ https://issues.apache.org/jira/browse/YARN-3224?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sunil G updated YARN-3224: -- Attachment: 0001-YARN-3224.patch Attaching a work in progress patch. As this patch is directly dependent on

[jira] [Updated] (YARN-4039) New AM instances waste resource by waiting only for resource availability when all available resources are already used

2015-08-16 Thread Sadayuki Furuhashi (JIRA)
[ https://issues.apache.org/jira/browse/YARN-4039?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sadayuki Furuhashi updated YARN-4039: - Assignee: Tsuyoshi Ozawa (was: Sadayuki Furuhashi) New AM instances waste resource by

[jira] [Updated] (YARN-3224) Notify AM with containers (on decommissioning node) could be preempted after timeout.

2015-08-16 Thread Sunil G (JIRA)
[ https://issues.apache.org/jira/browse/YARN-3224?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sunil G updated YARN-3224: -- Release Note: (was: We should leverage YARN preemption framework to notify AM that some containers will be

[jira] [Updated] (YARN-3224) Notify AM with containers (on decommissioning node) could be preempted after timeout.

2015-08-16 Thread Sunil G (JIRA)
[ https://issues.apache.org/jira/browse/YARN-3224?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sunil G updated YARN-3224: -- Description: We should leverage YARN preemption framework to notify AM that some containers will be preempted

[jira] [Commented] (YARN-4053) Change the way metric values are stored in HBase Storage

2015-08-16 Thread Varun Saxena (JIRA)
[ https://issues.apache.org/jira/browse/YARN-4053?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14698602#comment-14698602 ] Varun Saxena commented on YARN-4053: This patch demonstrates the approach mentioned

[jira] [Commented] (YARN-4053) Change the way metric values are stored in HBase Storage

2015-08-16 Thread Hadoop QA (JIRA)
[ https://issues.apache.org/jira/browse/YARN-4053?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14698608#comment-14698608 ] Hadoop QA commented on YARN-4053: - \\ \\ | (x) *{color:red}-1 overall{color}* | \\ \\ ||

[jira] [Updated] (YARN-3997) An Application requesting multiple core containers can't preempt running application made of single core containers

2015-08-16 Thread Dan Shechter (JIRA)
[ https://issues.apache.org/jira/browse/YARN-3997?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dan Shechter updated YARN-3997: --- Description: When our cluster is configured with preemption, and is fully loaded with an application

[jira] [Commented] (YARN-3534) Collect memory/cpu usage on the node

2015-08-16 Thread Karthik Kambatla (JIRA)
[ https://issues.apache.org/jira/browse/YARN-3534?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14698663#comment-14698663 ] Karthik Kambatla commented on YARN-3534: +1 Collect memory/cpu usage on the node

[jira] [Commented] (YARN-3997) An Application requesting multiple core containers can't preempt running application made of single core containers

2015-08-16 Thread Chen Avnery (JIRA)
[ https://issues.apache.org/jira/browse/YARN-3997?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14698648#comment-14698648 ] Chen Avnery commented on YARN-3997: --- This has happened to me too and I would love for it

[jira] [Commented] (YARN-3534) Collect memory/cpu usage on the node

2015-08-16 Thread Hudson (JIRA)
[ https://issues.apache.org/jira/browse/YARN-3534?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14698674#comment-14698674 ] Hudson commented on YARN-3534: -- FAILURE: Integrated in Hadoop-Mapreduce-trunk-Java8 #286 (See

[jira] [Commented] (YARN-3997) An Application requesting multiple core containers can't preempt running application made of single core containers

2015-08-16 Thread Ilan Assayag (JIRA)
[ https://issues.apache.org/jira/browse/YARN-3997?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14698709#comment-14698709 ] Ilan Assayag commented on YARN-3997: Got exactly the same issue too. Very painful...

[jira] [Commented] (YARN-2467) Add SpanReceiverHost to ResourceManager

2015-08-16 Thread Masatake Iwasaki (JIRA)
[ https://issues.apache.org/jira/browse/YARN-2467?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14698730#comment-14698730 ] Masatake Iwasaki commented on YARN-2467: The failures in test seem not to be

[jira] [Updated] (YARN-2467) Add SpanReceiverHost to ResourceManager

2015-08-16 Thread Masatake Iwasaki (JIRA)
[ https://issues.apache.org/jira/browse/YARN-2467?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Masatake Iwasaki updated YARN-2467: --- Description: Per process SpanReceiverHost should be initialized in ResourceManager in the same

[jira] [Commented] (YARN-3534) Collect memory/cpu usage on the node

2015-08-16 Thread Hudson (JIRA)
[ https://issues.apache.org/jira/browse/YARN-3534?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14698667#comment-14698667 ] Hudson commented on YARN-3534: -- FAILURE: Integrated in Hadoop-trunk-Commit #8311 (See

[jira] [Commented] (YARN-3997) An Application requesting multiple core containers can't preempt running application made of single core containers

2015-08-16 Thread Uri Miron (JIRA)
[ https://issues.apache.org/jira/browse/YARN-3997?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14698734#comment-14698734 ] Uri Miron commented on YARN-3997: - I am getting the same issue. An Application requesting

[jira] [Commented] (YARN-2467) Add SpanReceiverHost to ResourceManager

2015-08-16 Thread Masatake Iwasaki (JIRA)
[ https://issues.apache.org/jira/browse/YARN-2467?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14698733#comment-14698733 ] Masatake Iwasaki commented on YARN-2467: The TraceAdmin added to AdminService is

[jira] [Updated] (YARN-4049) Add SpanReceiverHost to NodeManager

2015-08-16 Thread Masatake Iwasaki (JIRA)
[ https://issues.apache.org/jira/browse/YARN-4049?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Masatake Iwasaki updated YARN-4049: --- Attachment: YARN-4049.001.patch I'm attaching wip patch. The 001 can not be applied to trunk

[jira] [Commented] (YARN-3534) Collect memory/cpu usage on the node

2015-08-16 Thread Inigo Goiri (JIRA)
[ https://issues.apache.org/jira/browse/YARN-3534?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14698806#comment-14698806 ] Inigo Goiri commented on YARN-3534: --- Thank you [~kasha] for taking care of the review and

[jira] [Commented] (YARN-4024) YARN RM should avoid unnecessary resolving IP when NMs doing heartbeat

2015-08-16 Thread Hong Zhiguo (JIRA)
[ https://issues.apache.org/jira/browse/YARN-4024?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14698927#comment-14698927 ] Hong Zhiguo commented on YARN-4024: --- That's a good reason to have this cache.

[jira] [Commented] (YARN-4024) YARN RM should avoid unnecessary resolving IP when NMs doing heartbeat

2015-08-16 Thread Hong Zhiguo (JIRA)
[ https://issues.apache.org/jira/browse/YARN-4024?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14698929#comment-14698929 ] Hong Zhiguo commented on YARN-4024: --- Please ignore the last sentence a better way is to

[jira] [Created] (YARN-4056) Bundling: Searching for multiple containers in a single pass over {queues, applications, priorities}

2015-08-16 Thread Srikanth Kandula (JIRA)
Srikanth Kandula created YARN-4056: -- Summary: Bundling: Searching for multiple containers in a single pass over {queues, applications, priorities} Key: YARN-4056 URL:

[jira] [Updated] (YARN-3901) Populate flow run data in the flow_run table

2015-08-16 Thread Vrushali C (JIRA)
[ https://issues.apache.org/jira/browse/YARN-3901?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vrushali C updated YARN-3901: - Attachment: YARN-3901-YARN-2928.WIP.patch Uploading a work in progress patch. This patch is not yet

[jira] [Commented] (YARN-4056) Bundling: Searching for multiple containers in a single pass over {queues, applications, priorities}

2015-08-16 Thread Srikanth Kandula (JIRA)
[ https://issues.apache.org/jira/browse/YARN-4056?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14698925#comment-14698925 ] Srikanth Kandula commented on YARN-4056: Will look. Possibly. However, this arch

[jira] [Updated] (YARN-4056) Bundling: Searching for multiple containers in a single pass over {queues, applications, priorities}

2015-08-16 Thread Srikanth Kandula (JIRA)
[ https://issues.apache.org/jira/browse/YARN-4056?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Srikanth Kandula updated YARN-4056: --- Description: More than one container is allocated on many NM heartbeats. Yet, the current

[jira] [Updated] (YARN-4056) Bundling: Searching for multiple containers in a single pass over {queues, applications, priorities}

2015-08-16 Thread Srikanth Kandula (JIRA)
[ https://issues.apache.org/jira/browse/YARN-4056?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Srikanth Kandula updated YARN-4056: --- Attachment: bundling.docx Bundling: Searching for multiple containers in a single pass over

[jira] [Updated] (YARN-4056) Bundling: Searching for multiple containers in a single pass over {queues, applications, priorities}

2015-08-16 Thread Srikanth Kandula (JIRA)
[ https://issues.apache.org/jira/browse/YARN-4056?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Srikanth Kandula updated YARN-4056: --- Description: More than one container is allocated on many NM heartbeats. Yet, the current

[jira] [Assigned] (YARN-4056) Bundling: Searching for multiple containers in a single pass over {queues, applications, priorities}

2015-08-16 Thread Robert Grandl (JIRA)
[ https://issues.apache.org/jira/browse/YARN-4056?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Robert Grandl reassigned YARN-4056: --- Assignee: Robert Grandl Bundling: Searching for multiple containers in a single pass over

[jira] [Commented] (YARN-3893) Both RM in active state when Admin#transitionToActive failure from refeshAll()

2015-08-16 Thread Rohith Sharma K S (JIRA)
[ https://issues.apache.org/jira/browse/YARN-3893?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14698908#comment-14698908 ] Rohith Sharma K S commented on YARN-3893: - Sorry for coming very late.. This issue

[jira] [Commented] (YARN-4056) Bundling: Searching for multiple containers in a single pass over {queues, applications, priorities}

2015-08-16 Thread Karthik Kambatla (JIRA)
[ https://issues.apache.org/jira/browse/YARN-4056?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14698915#comment-14698915 ] Karthik Kambatla commented on YARN-4056: Is this similar to {{assignMultiple}} in

[jira] [Updated] (YARN-3980) Plumb resource-utilization info in node heartbeat through to the scheduler

2015-08-16 Thread Inigo Goiri (JIRA)
[ https://issues.apache.org/jira/browse/YARN-3980?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Inigo Goiri updated YARN-3980: -- Attachment: YARN-3980-v1.patch Fixing broken unit tests. Plumb resource-utilization info in node

[jira] [Updated] (YARN-3980) Plumb resource-utilization info in node heartbeat through to the scheduler

2015-08-16 Thread Inigo Goiri (JIRA)
[ https://issues.apache.org/jira/browse/YARN-3980?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Inigo Goiri updated YARN-3980: -- Attachment: YARN-3980-v1.patch Whitespace fix. Plumb resource-utilization info in node heartbeat

[jira] [Updated] (YARN-3980) Plumb resource-utilization info in node heartbeat through to the scheduler

2015-08-16 Thread Inigo Goiri (JIRA)
[ https://issues.apache.org/jira/browse/YARN-3980?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Inigo Goiri updated YARN-3980: -- Attachment: (was: YARN-3980-v1.patch) Plumb resource-utilization info in node heartbeat through to

[jira] [Commented] (YARN-3980) Plumb resource-utilization info in node heartbeat through to the scheduler

2015-08-16 Thread Hadoop QA (JIRA)
[ https://issues.apache.org/jira/browse/YARN-3980?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14698955#comment-14698955 ] Hadoop QA commented on YARN-3980: - \\ \\ | (x) *{color:red}-1 overall{color}* | \\ \\ ||

[jira] [Commented] (YARN-3980) Plumb resource-utilization info in node heartbeat through to the scheduler

2015-08-16 Thread Hadoop QA (JIRA)
[ https://issues.apache.org/jira/browse/YARN-3980?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14699003#comment-14699003 ] Hadoop QA commented on YARN-3980: - \\ \\ | (x) *{color:red}-1 overall{color}* | \\ \\ ||

[jira] [Updated] (YARN-3980) Plumb resource-utilization info in node heartbeat through to the scheduler

2015-08-16 Thread Inigo Goiri (JIRA)
[ https://issues.apache.org/jira/browse/YARN-3980?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Inigo Goiri updated YARN-3980: -- Attachment: (was: YARN-3980-v1.patch) Plumb resource-utilization info in node heartbeat through to

[jira] [Updated] (YARN-3980) Plumb resource-utilization info in node heartbeat through to the scheduler

2015-08-16 Thread Inigo Goiri (JIRA)
[ https://issues.apache.org/jira/browse/YARN-3980?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Inigo Goiri updated YARN-3980: -- Attachment: YARN-3980-v1.patch Fixed SLS. Plumb resource-utilization info in node heartbeat through to

[jira] [Commented] (YARN-3997) An Application requesting multiple core containers can't preempt running application made of single core containers

2015-08-16 Thread Dan Shechter (JIRA)
[ https://issues.apache.org/jira/browse/YARN-3997?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14698830#comment-14698830 ] Dan Shechter commented on YARN-3997: Hi, I was trying to find the existing unit-tests

[jira] [Updated] (YARN-4055) Report node resource utilization in heartbeat

2015-08-16 Thread Inigo Goiri (JIRA)
[ https://issues.apache.org/jira/browse/YARN-4055?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Inigo Goiri updated YARN-4055: -- Attachment: YARN-4055-v0.patch First version for sending the node resource utilization in the heartbeat.

[jira] [Commented] (YARN-4055) Report node resource utilization in heartbeat

2015-08-16 Thread Karthik Kambatla (JIRA)
[ https://issues.apache.org/jira/browse/YARN-4055?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14698863#comment-14698863 ] Karthik Kambatla commented on YARN-4055: +1 Report node resource utilization in

[jira] [Created] (YARN-4055) Report node resource utilization in heartbeat

2015-08-16 Thread Inigo Goiri (JIRA)
Inigo Goiri created YARN-4055: - Summary: Report node resource utilization in heartbeat Key: YARN-4055 URL: https://issues.apache.org/jira/browse/YARN-4055 Project: Hadoop YARN Issue Type:

[jira] [Updated] (YARN-4055) Report node resource utilization in heartbeat

2015-08-16 Thread Inigo Goiri (JIRA)
[ https://issues.apache.org/jira/browse/YARN-4055?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Inigo Goiri updated YARN-4055: -- Attachment: YARN-4055-v1.patch Changing type for node resource monitor in Node Manager. Report node

[jira] [Assigned] (YARN-3997) An Application requesting multiple core containers can't preempt running application made of single core containers

2015-08-16 Thread Karthik Kambatla (JIRA)
[ https://issues.apache.org/jira/browse/YARN-3997?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Karthik Kambatla reassigned YARN-3997: -- Assignee: Arun Suresh (was: Karthik Kambatla) Was discussing this with [~asuresh]

[jira] [Commented] (YARN-4055) Report node resource utilization in heartbeat

2015-08-16 Thread Karthik Kambatla (JIRA)
[ https://issues.apache.org/jira/browse/YARN-4055?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14698849#comment-14698849 ] Karthik Kambatla commented on YARN-4055: Thanks for filing and working on this,

[jira] [Commented] (YARN-4055) Report node resource utilization in heartbeat

2015-08-16 Thread Hadoop QA (JIRA)
[ https://issues.apache.org/jira/browse/YARN-4055?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14698860#comment-14698860 ] Hadoop QA commented on YARN-4055: - \\ \\ | (x) *{color:red}-1 overall{color}* | \\ \\ ||

[jira] [Commented] (YARN-4055) Report node resource utilization in heartbeat

2015-08-16 Thread Hudson (JIRA)
[ https://issues.apache.org/jira/browse/YARN-4055?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14698866#comment-14698866 ] Hudson commented on YARN-4055: -- FAILURE: Integrated in Hadoop-trunk-Commit #8312 (See

[jira] [Updated] (YARN-3980) Plumb resource-utilization info in node heartbeat through to the scheduler

2015-08-16 Thread Inigo Goiri (JIRA)
[ https://issues.apache.org/jira/browse/YARN-3980?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Inigo Goiri updated YARN-3980: -- Attachment: YARN-3980-v0.patch First version missing unit test based on MiniYARNCluster (WIP). Plumb

[jira] [Updated] (YARN-3980) Plumb resource-utilization info in node heartbeat through to the scheduler

2015-08-16 Thread Karthik Kambatla (JIRA)
[ https://issues.apache.org/jira/browse/YARN-3980?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Karthik Kambatla updated YARN-3980: --- Assignee: Inigo Goiri (was: Karthik Kambatla) Plumb resource-utilization info in node

[jira] [Created] (YARN-4057) If ContainersMonitor is not enabled, only print related log info one time

2015-08-16 Thread Jun Gong (JIRA)
Jun Gong created YARN-4057: -- Summary: If ContainersMonitor is not enabled, only print related log info one time Key: YARN-4057 URL: https://issues.apache.org/jira/browse/YARN-4057 Project: Hadoop YARN

[jira] [Commented] (YARN-3980) Plumb resource-utilization info in node heartbeat through to the scheduler

2015-08-16 Thread Karthik Kambatla (JIRA)
[ https://issues.apache.org/jira/browse/YARN-3980?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14699042#comment-14699042 ] Karthik Kambatla commented on YARN-3980: Barely skimmed through the patch. In

[jira] [Commented] (YARN-3980) Plumb resource-utilization info in node heartbeat through to the scheduler

2015-08-16 Thread Inigo Goiri (JIRA)
[ https://issues.apache.org/jira/browse/YARN-3980?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14699046#comment-14699046 ] Inigo Goiri commented on YARN-3980: --- It would change the previous code a lot but I think