[jira] [Commented] (YARN-3049) [Storage Implementation] Implement storage reader interface to fetch raw data from HBase backend

2015-08-06 Thread Zhijie Shen (JIRA)
[ https://issues.apache.org/jira/browse/YARN-3049?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14660853#comment-14660853 ] Zhijie Shen commented on YARN-3049: --- Here's a quick example: {code} @Test public void

[jira] [Commented] (YARN-3049) [Storage Implementation] Implement storage reader interface to fetch raw data from HBase backend

2015-08-06 Thread Zhijie Shen (JIRA)
[ https://issues.apache.org/jira/browse/YARN-3049?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14660952#comment-14660952 ] Zhijie Shen commented on YARN-3049: --- As the issue is not blocking the whole reader

[jira] [Updated] (YARN-3049) [Storage Implementation] Implement storage reader interface to fetch raw data from HBase backend

2015-08-05 Thread Zhijie Shen (JIRA)
[ https://issues.apache.org/jira/browse/YARN-3049?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhijie Shen updated YARN-3049: -- Attachment: YARN-3049-YARN-2928.7.patch Attach a new patch: 1. Rebase against YARN-3984. 2. Address

[jira] [Commented] (YARN-3984) Rethink event column key issue

2015-08-05 Thread Zhijie Shen (JIRA)
[ https://issues.apache.org/jira/browse/YARN-3984?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14659183#comment-14659183 ] Zhijie Shen commented on YARN-3984: --- bq. If the info map is not empty, this record would

[jira] [Updated] (YARN-3049) [Storage Implementation] Implement storage reader interface to fetch raw data from HBase backend

2015-08-04 Thread Zhijie Shen (JIRA)
[ https://issues.apache.org/jira/browse/YARN-3049?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhijie Shen updated YARN-3049: -- Attachment: YARN-3049-YARN-2928.6.patch Upload a new patch which makes HBase backend to make the

[jira] [Commented] (YARN-3049) [Storage Implementation] Implement storage reader interface to fetch raw data from HBase backend

2015-08-03 Thread Zhijie Shen (JIRA)
[ https://issues.apache.org/jira/browse/YARN-3049?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14652806#comment-14652806 ] Zhijie Shen commented on YARN-3049: --- Okay, what will the timestamp be used to do? If

[jira] [Updated] (YARN-3993) Change to use the AM flag in ContainerContext determine AM container

2015-08-03 Thread Zhijie Shen (JIRA)
[ https://issues.apache.org/jira/browse/YARN-3993?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhijie Shen updated YARN-3993: -- Attachment: YARN-3993-YARN-2928.0001.patch Rename the patch to make it for the branch. Change to use

[jira] [Created] (YARN-4013) Publisher V2 should write the unmanaged AM flag too

2015-08-03 Thread Zhijie Shen (JIRA)
Zhijie Shen created YARN-4013: - Summary: Publisher V2 should write the unmanaged AM flag too Key: YARN-4013 URL: https://issues.apache.org/jira/browse/YARN-4013 Project: Hadoop YARN Issue Type:

[jira] [Updated] (YARN-3049) [Storage Implementation] Implement storage reader interface to fetch raw data from HBase backend

2015-08-03 Thread Zhijie Shen (JIRA)
[ https://issues.apache.org/jira/browse/YARN-3049?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhijie Shen updated YARN-3049: -- Attachment: YARN-3049-YARN-2928.5.patch I made a change in the new patch to reflect my last proposal.

[jira] [Commented] (YARN-3993) Change to use the AM flag in ContainerContext determine AM container

2015-08-03 Thread Zhijie Shen (JIRA)
[ https://issues.apache.org/jira/browse/YARN-3993?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14652225#comment-14652225 ] Zhijie Shen commented on YARN-3993: --- +1 LGTM. Will commit after jenkins' comment.

[jira] [Commented] (YARN-3049) [Storage Implementation] Implement storage reader interface to fetch raw data from HBase backend

2015-08-03 Thread Zhijie Shen (JIRA)
[ https://issues.apache.org/jira/browse/YARN-3049?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14652206#comment-14652206 ] Zhijie Shen commented on YARN-3049: --- Hi Sangjin, Thanks for your comments. The proposed

[jira] [Commented] (YARN-3904) Refactor timelineservice.storage to add support to online and offline aggregation writers

2015-07-31 Thread Zhijie Shen (JIRA)
[ https://issues.apache.org/jira/browse/YARN-3904?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14650069#comment-14650069 ] Zhijie Shen commented on YARN-3904: --- bq. I'm not 100% sure if that's what we would like

[jira] [Commented] (YARN-4006) YARN ATS Alternate Kerberos HTTP Authentication Changes

2015-07-31 Thread Zhijie Shen (JIRA)
[ https://issues.apache.org/jira/browse/YARN-4006?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14650082#comment-14650082 ] Zhijie Shen commented on YARN-4006: --- Sure, take your time. I'll cancel the patch until

[jira] [Commented] (YARN-3049) [Storage Implementation] Implement storage reader interface to fetch raw data from HBase backend

2015-07-31 Thread Zhijie Shen (JIRA)
[ https://issues.apache.org/jira/browse/YARN-3049?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14650106#comment-14650106 ] Zhijie Shen commented on YARN-3049: --- What I meant before is that HBaseTimelineWriterImpl

[jira] [Updated] (YARN-4006) YARN ATS Alternate Kerberos HTTP Authentication Changes

2015-07-31 Thread Zhijie Shen (JIRA)
[ https://issues.apache.org/jira/browse/YARN-4006?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhijie Shen updated YARN-4006: -- Assignee: Greg Senia YARN ATS Alternate Kerberos HTTP Authentication Changes

[jira] [Commented] (YARN-3984) Rethink event column key issue

2015-07-31 Thread Zhijie Shen (JIRA)
[ https://issues.apache.org/jira/browse/YARN-3984?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14649598#comment-14649598 ] Zhijie Shen commented on YARN-3984: --- Sure, it's fine too. One question, do we make sure

[jira] [Commented] (YARN-4006) YARN ATS Alternate Kerberos HTTP Authentication Changes

2015-07-31 Thread Zhijie Shen (JIRA)
[ https://issues.apache.org/jira/browse/YARN-4006?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14650015#comment-14650015 ] Zhijie Shen commented on YARN-4006: --- [~gss2002], it seems that the patch doesn't come

[jira] [Commented] (YARN-3049) [Storage Implementation] Implement storage reader interface to fetch raw data from HBase backend

2015-07-30 Thread Zhijie Shen (JIRA)
[ https://issues.apache.org/jira/browse/YARN-3049?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14648218#comment-14648218 ] Zhijie Shen commented on YARN-3049: --- [~gtCarrera9], thanks for review. I've addressed

[jira] [Updated] (YARN-3049) [Storage Implementation] Implement storage reader interface to fetch raw data from HBase backend

2015-07-30 Thread Zhijie Shen (JIRA)
[ https://issues.apache.org/jira/browse/YARN-3049?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhijie Shen updated YARN-3049: -- Attachment: YARN-3049-YARN-2928.3.patch [Storage Implementation] Implement storage reader interface to

[jira] [Commented] (YARN-3904) Refactor timelineservice.storage to add support to online and offline aggregation writers

2015-07-30 Thread Zhijie Shen (JIRA)
[ https://issues.apache.org/jira/browse/YARN-3904?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14648457#comment-14648457 ] Zhijie Shen commented on YARN-3904: --- [~gtCarrera9], thanks for the patch. Bellow are my

[jira] [Commented] (YARN-3942) Timeline store to read events from HDFS

2015-07-30 Thread Zhijie Shen (JIRA)
[ https://issues.apache.org/jira/browse/YARN-3942?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14648048#comment-14648048 ] Zhijie Shen commented on YARN-3942: --- Yeah, I prefer creating a TimelineEntityFileClient

[jira] [Commented] (YARN-3049) [Storage Implementation] Implement storage reader interface to fetch raw data from HBase backend

2015-07-29 Thread Zhijie Shen (JIRA)
[ https://issues.apache.org/jira/browse/YARN-3049?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14646339#comment-14646339 ] Zhijie Shen commented on YARN-3049: ---

[jira] [Commented] (YARN-3992) TestApplicationPriority.testApplicationPriorityAllocation fails intermittently

2015-07-29 Thread Zhijie Shen (JIRA)
[ https://issues.apache.org/jira/browse/YARN-3992?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14646443#comment-14646443 ] Zhijie Shen commented on YARN-3992: --- The problem was found with jenkins build on

[jira] [Created] (YARN-3992) TestApplicationPriority.testApplicationPriorityAllocation fails intermittently

2015-07-29 Thread Zhijie Shen (JIRA)
Zhijie Shen created YARN-3992: - Summary: TestApplicationPriority.testApplicationPriorityAllocation fails intermittently Key: YARN-3992 URL: https://issues.apache.org/jira/browse/YARN-3992 Project: Hadoop

[jira] [Created] (YARN-3993) Change to use the AM flag in ContainerContext determine AM container

2015-07-29 Thread Zhijie Shen (JIRA)
Zhijie Shen created YARN-3993: - Summary: Change to use the AM flag in ContainerContext determine AM container Key: YARN-3993 URL: https://issues.apache.org/jira/browse/YARN-3993 Project: Hadoop YARN

[jira] [Updated] (YARN-3993) Change to use the AM flag in ContainerContext determine AM container

2015-07-29 Thread Zhijie Shen (JIRA)
[ https://issues.apache.org/jira/browse/YARN-3993?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhijie Shen updated YARN-3993: -- Labels: newbie (was: ) Change to use the AM flag in ContainerContext determine AM container

[jira] [Comment Edited] (YARN-3993) Change to use the AM flag in ContainerContext determine AM container

2015-07-29 Thread Zhijie Shen (JIRA)
[ https://issues.apache.org/jira/browse/YARN-3993?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14646558#comment-14646558 ] Zhijie Shen edited comment on YARN-3993 at 7/29/15 6:26 PM:

[jira] [Commented] (YARN-3993) Change to use the AM flag in ContainerContext determine AM container

2015-07-29 Thread Zhijie Shen (JIRA)
[ https://issues.apache.org/jira/browse/YARN-3993?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14646558#comment-14646558 ] Zhijie Shen commented on YARN-3993: --- [~sunilg], thanks for your interest. It's not

[jira] [Commented] (YARN-3984) Rethink event column key issue

2015-07-29 Thread Zhijie Shen (JIRA)
[ https://issues.apache.org/jira/browse/YARN-3984?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14646924#comment-14646924 ] Zhijie Shen commented on YARN-3984: --- [~vrushalic], thanks for picking it up. The

[jira] [Commented] (YARN-3984) Rethink event column key issue

2015-07-29 Thread Zhijie Shen (JIRA)
[ https://issues.apache.org/jira/browse/YARN-3984?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14646935#comment-14646935 ] Zhijie Shen commented on YARN-3984: --- In fact, metric has the same problem, but it may be

[jira] [Commented] (YARN-3942) Timeline store to read events from HDFS

2015-07-29 Thread Zhijie Shen (JIRA)
[ https://issues.apache.org/jira/browse/YARN-3942?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14647217#comment-14647217 ] Zhijie Shen commented on YARN-3942: --- [~jlowe], thanks for sharing more information about

[jira] [Updated] (YARN-3814) REST API implementation for getting raw entities in TimelineReader

2015-07-29 Thread Zhijie Shen (JIRA)
[ https://issues.apache.org/jira/browse/YARN-3814?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhijie Shen updated YARN-3814: -- Attachment: YARN-3814.reference.patch I attached the patch which contains the web services only from the

[jira] [Updated] (YARN-3049) [Storage Implementation] Implement storage reader interface to fetch raw data from HBase backend

2015-07-28 Thread Zhijie Shen (JIRA)
[ https://issues.apache.org/jira/browse/YARN-3049?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhijie Shen updated YARN-3049: -- Attachment: YARN-3049-YARN-2928.2.patch After YARN-3908, I updated the patch according to the HBase

[jira] [Assigned] (YARN-3981) support timeline clients not associated with an application

2015-07-27 Thread Zhijie Shen (JIRA)
[ https://issues.apache.org/jira/browse/YARN-3981?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhijie Shen reassigned YARN-3981: - Assignee: Zhijie Shen support timeline clients not associated with an application

[jira] [Commented] (YARN-3981) support timeline clients not associated with an application

2015-07-27 Thread Zhijie Shen (JIRA)
[ https://issues.apache.org/jira/browse/YARN-3981?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14643056#comment-14643056 ] Zhijie Shen commented on YARN-3981: --- Thanks for filing the jira. I'm going to pick this

[jira] [Commented] (YARN-3908) Bugs in HBaseTimelineWriterImpl

2015-07-27 Thread Zhijie Shen (JIRA)
[ https://issues.apache.org/jira/browse/YARN-3908?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14643113#comment-14643113 ] Zhijie Shen commented on YARN-3908: --- [~vrushalic], thanks for fixing the problem. W.R.T

[jira] [Commented] (YARN-3908) Bugs in HBaseTimelineWriterImpl

2015-07-27 Thread Zhijie Shen (JIRA)
[ https://issues.apache.org/jira/browse/YARN-3908?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14643267#comment-14643267 ] Zhijie Shen commented on YARN-3908: --- Okay, it's fair point. It seems that the key design

[jira] [Commented] (YARN-3908) Bugs in HBaseTimelineWriterImpl

2015-07-27 Thread Zhijie Shen (JIRA)
[ https://issues.apache.org/jira/browse/YARN-3908?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14643476#comment-14643476 ] Zhijie Shen commented on YARN-3908: --- Sure, as most folks are comfortable with the latest

[jira] [Resolved] (YARN-3908) Bugs in HBaseTimelineWriterImpl

2015-07-27 Thread Zhijie Shen (JIRA)
[ https://issues.apache.org/jira/browse/YARN-3908?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhijie Shen resolved YARN-3908. --- Resolution: Fixed Hadoop Flags: Reviewed Fix Version/s: YARN-2928 Committed the patch to

[jira] [Created] (YARN-3984) Rethink event column key issue

2015-07-27 Thread Zhijie Shen (JIRA)
Zhijie Shen created YARN-3984: - Summary: Rethink event column key issue Key: YARN-3984 URL: https://issues.apache.org/jira/browse/YARN-3984 Project: Hadoop YARN Issue Type: Sub-task

[jira] [Commented] (YARN-3942) Timeline store to read events from HDFS

2015-07-27 Thread Zhijie Shen (JIRA)
[ https://issues.apache.org/jira/browse/YARN-3942?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14643790#comment-14643790 ] Zhijie Shen commented on YARN-3942: --- Thanks for this work. Agree it's a good interim step

[jira] [Commented] (YARN-3949) ensure timely flush of timeline writes

2015-07-24 Thread Zhijie Shen (JIRA)
[ https://issues.apache.org/jira/browse/YARN-3949?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14640910#comment-14640910 ] Zhijie Shen commented on YARN-3949: --- IMHO, given write + flush, it's not necessary to

[jira] [Commented] (YARN-3908) Bugs in HBaseTimelineWriterImpl

2015-07-22 Thread Zhijie Shen (JIRA)
[ https://issues.apache.org/jira/browse/YARN-3908?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14637300#comment-14637300 ] Zhijie Shen commented on YARN-3908: --- bq. Is it the event id + timestamp? How about the

[jira] [Commented] (YARN-3949) ensure timely flush of timeline writes

2015-07-22 Thread Zhijie Shen (JIRA)
[ https://issues.apache.org/jira/browse/YARN-3949?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14638190#comment-14638190 ] Zhijie Shen commented on YARN-3949: --- The proposal looks good to me for now. We may need

[jira] [Commented] (YARN-3908) Bugs in HBaseTimelineWriterImpl

2015-07-20 Thread Zhijie Shen (JIRA)
[ https://issues.apache.org/jira/browse/YARN-3908?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14633857#comment-14633857 ] Zhijie Shen commented on YARN-3908: --- I found two more issues upon debugging the reader

[jira] [Commented] (YARN-3049) [Storage Implementation] Implement storage reader interface to fetch raw data from HBase backend

2015-07-20 Thread Zhijie Shen (JIRA)
[ https://issues.apache.org/jira/browse/YARN-3049?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14634352#comment-14634352 ] Zhijie Shen commented on YARN-3049: --- [~sjlee0], yeah, for POC purpose, I temporally do

[jira] [Commented] (YARN-3908) Bugs in HBaseTimelineWriterImpl

2015-07-20 Thread Zhijie Shen (JIRA)
[ https://issues.apache.org/jira/browse/YARN-3908?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14634200#comment-14634200 ] Zhijie Shen commented on YARN-3908: --- I set up hbase-1.0.1.1 as a single node cluster on

[jira] [Updated] (YARN-3049) [Storage Implementation] Implement storage reader interface to fetch raw data from HBase backend

2015-07-20 Thread Zhijie Shen (JIRA)
[ https://issues.apache.org/jira/browse/YARN-3049?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhijie Shen updated YARN-3049: -- Attachment: YARN-3049-WIP.3.patch Upload a new WIP patch with some bug fixes, including the the two

[jira] [Updated] (YARN-3049) [Storage Implementation] Implement storage reader interface to fetch raw data from HBase backend

2015-07-16 Thread Zhijie Shen (JIRA)
[ https://issues.apache.org/jira/browse/YARN-3049?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhijie Shen updated YARN-3049: -- Attachment: YARN-3049-WIP.2.patch [~sjlee0] and [~gtCarrera9], thanks for review the patch. I'm

[jira] [Commented] (YARN-3814) REST API implementation for getting raw entities in TimelineReader

2015-07-15 Thread Zhijie Shen (JIRA)
[ https://issues.apache.org/jira/browse/YARN-3814?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14628647#comment-14628647 ] Zhijie Shen commented on YARN-3814: --- [~varun_saxena], thanks for putting the patch. It

[jira] [Commented] (YARN-3814) REST API implementation for getting raw entities in TimelineReader

2015-07-15 Thread Zhijie Shen (JIRA)
[ https://issues.apache.org/jira/browse/YARN-3814?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14628708#comment-14628708 ] Zhijie Shen commented on YARN-3814: --- I didn't go beyond the current reader interface.

[jira] [Updated] (YARN-3049) [Storage Implementation] Implement storage reader interface to fetch raw data from HBase backend

2015-07-13 Thread Zhijie Shen (JIRA)
[ https://issues.apache.org/jira/browse/YARN-3049?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhijie Shen updated YARN-3049: -- Attachment: YARN-3049-WIP.1.patch Attache a WIP patch so that the community can take a look while I

[jira] [Commented] (YARN-3908) Bugs in HBaseTimelineWriterImpl

2015-07-13 Thread Zhijie Shen (JIRA)
[ https://issues.apache.org/jira/browse/YARN-3908?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14625369#comment-14625369 ] Zhijie Shen commented on YARN-3908: --- [~vrushalic] and [~sjlee0], thanks for helping fix

[jira] [Commented] (YARN-3908) Bugs in HBaseTimelineWriterImpl

2015-07-13 Thread Zhijie Shen (JIRA)
[ https://issues.apache.org/jira/browse/YARN-3908?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14625397#comment-14625397 ] Zhijie Shen commented on YARN-3908: --- Yeah, but the method based on metric value number is

[jira] [Commented] (YARN-3116) [Collector wireup] We need an assured way to determine if a container is an AM container on NM

2015-07-13 Thread Zhijie Shen (JIRA)
[ https://issues.apache.org/jira/browse/YARN-3116?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14625663#comment-14625663 ] Zhijie Shen commented on YARN-3116: --- Congrats on your first patch, [~giovanni.fumarola]!

[jira] [Created] (YARN-3914) Entity created time should be part of the row key of entity table

2015-07-10 Thread Zhijie Shen (JIRA)
Zhijie Shen created YARN-3914: - Summary: Entity created time should be part of the row key of entity table Key: YARN-3914 URL: https://issues.apache.org/jira/browse/YARN-3914 Project: Hadoop YARN

[jira] [Commented] (YARN-3914) Entity created time should be part of the row key of entity table

2015-07-10 Thread Zhijie Shen (JIRA)
[ https://issues.apache.org/jira/browse/YARN-3914?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14623170#comment-14623170 ] Zhijie Shen commented on YARN-3914: --- This will not block the implementation of

[jira] [Commented] (YARN-3116) [Collector wireup] We need an assured way to determine if a container is an AM container on NM

2015-07-10 Thread Zhijie Shen (JIRA)
[ https://issues.apache.org/jira/browse/YARN-3116?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14623176#comment-14623176 ] Zhijie Shen commented on YARN-3116: --- +1 for the last patch. Will commit it after jenkins

[jira] [Commented] (YARN-3116) [Collector wireup] We need an assured way to determine if a container is an AM container on NM

2015-07-10 Thread Zhijie Shen (JIRA)
[ https://issues.apache.org/jira/browse/YARN-3116?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14622998#comment-14622998 ] Zhijie Shen commented on YARN-3116: --- one nit: can we move ContainerType to server/api?

[jira] [Commented] (YARN-3116) [Collector wireup] We need an assured way to determine if a container is an AM container on NM

2015-07-09 Thread Zhijie Shen (JIRA)
[ https://issues.apache.org/jira/browse/YARN-3116?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14621034#comment-14621034 ] Zhijie Shen commented on YARN-3116: --- [~kkaranasos], thanks for notifying us of YARN-2882.

[jira] [Commented] (YARN-3901) Populate flow run data in the flow_run table

2015-07-09 Thread Zhijie Shen (JIRA)
[ https://issues.apache.org/jira/browse/YARN-3901?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14621051#comment-14621051 ] Zhijie Shen commented on YARN-3901: --- Yeah, I have dependency on this table for reader. If

[jira] [Commented] (YARN-3836) add equals and hashCode to TimelineEntity and other classes in the data model

2015-07-09 Thread Zhijie Shen (JIRA)
[ https://issues.apache.org/jira/browse/YARN-3836?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14621545#comment-14621545 ] Zhijie Shen commented on YARN-3836: --- +1 LGTM add equals and hashCode to TimelineEntity

[jira] [Commented] (YARN-3908) Bugs in HBaseTimelineWriterImpl

2015-07-09 Thread Zhijie Shen (JIRA)
[ https://issues.apache.org/jira/browse/YARN-3908?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14621592#comment-14621592 ] Zhijie Shen commented on YARN-3908: --- 1. TimelineEvent has a timestamp associated with it.

[jira] [Commented] (YARN-3116) [Collector wireup] We need an assured way to determine if a container is an AM container on NM

2015-07-09 Thread Zhijie Shen (JIRA)
[ https://issues.apache.org/jira/browse/YARN-3116?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14621581#comment-14621581 ] Zhijie Shen commented on YARN-3116: --- [~kkaranasos], I didn't touch the detail on

[jira] [Commented] (YARN-3836) add equals and hashCode to TimelineEntity and other classes in the data model

2015-07-09 Thread Zhijie Shen (JIRA)
[ https://issues.apache.org/jira/browse/YARN-3836?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14621209#comment-14621209 ] Zhijie Shen commented on YARN-3836: --- bq. l.550: It sounds like now the type takes

[jira] [Commented] (YARN-3908) Bugs in HBaseTimelineWriterImpl

2015-07-09 Thread Zhijie Shen (JIRA)
[ https://issues.apache.org/jira/browse/YARN-3908?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14621480#comment-14621480 ] Zhijie Shen commented on YARN-3908: --- Assign it to [~vrushalic] by default. Please feel

[jira] [Created] (YARN-3908) Bugs in HBaseTimelineWriterImpl

2015-07-09 Thread Zhijie Shen (JIRA)
Zhijie Shen created YARN-3908: - Summary: Bugs in HBaseTimelineWriterImpl Key: YARN-3908 URL: https://issues.apache.org/jira/browse/YARN-3908 Project: Hadoop YARN Issue Type: Bug

[jira] [Updated] (YARN-3908) Bugs in HBaseTimelineWriterImpl

2015-07-09 Thread Zhijie Shen (JIRA)
[ https://issues.apache.org/jira/browse/YARN-3908?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhijie Shen updated YARN-3908: -- Issue Type: Sub-task (was: Bug) Parent: YARN-2928 Bugs in HBaseTimelineWriterImpl

[jira] [Comment Edited] (YARN-3908) Bugs in HBaseTimelineWriterImpl

2015-07-09 Thread Zhijie Shen (JIRA)
[ https://issues.apache.org/jira/browse/YARN-3908?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14621480#comment-14621480 ] Zhijie Shen edited comment on YARN-3908 at 7/10/15 12:23 AM: -

[jira] [Commented] (YARN-3049) [Storage Implementation] Implement storage reader interface to fetch raw data from HBase backend

2015-07-08 Thread Zhijie Shen (JIRA)
[ https://issues.apache.org/jira/browse/YARN-3049?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14619347#comment-14619347 ] Zhijie Shen commented on YARN-3049: --- Updated the title accordingly to describe the scope

[jira] [Updated] (YARN-3049) [Storage Implementation] Implement storage reader interface to fetch raw data from HBase backend

2015-07-08 Thread Zhijie Shen (JIRA)
[ https://issues.apache.org/jira/browse/YARN-3049?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhijie Shen updated YARN-3049: -- Summary: [Storage Implementation] Implement storage reader interface to fetch raw data from HBase

[jira] [Updated] (YARN-3049) [Storage Implementation] Implement the storage reader interface to fetch raw data

2015-07-08 Thread Zhijie Shen (JIRA)
[ https://issues.apache.org/jira/browse/YARN-3049?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhijie Shen updated YARN-3049: -- Summary: [Storage Implementation] Implement the storage reader interface to fetch raw data (was:

[jira] [Commented] (YARN-3047) [Data Serving] Set up ATS reader with basic request serving structure and lifecycle

2015-07-08 Thread Zhijie Shen (JIRA)
[ https://issues.apache.org/jira/browse/YARN-3047?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14619440#comment-14619440 ] Zhijie Shen commented on YARN-3047: --- Thanks for kicking another jenkins build. IAC, the

[jira] [Commented] (YARN-3901) Populate flow run data in the flow_run table

2015-07-08 Thread Zhijie Shen (JIRA)
[ https://issues.apache.org/jira/browse/YARN-3901?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14619759#comment-14619759 ] Zhijie Shen commented on YARN-3901: --- [~vrushalic], just want to confirm with you that the

[jira] [Commented] (YARN-3116) [Collector wireup] We need an assured way to determine if a container is an AM container on NM

2015-07-08 Thread Zhijie Shen (JIRA)
[ https://issues.apache.org/jira/browse/YARN-3116?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14619564#comment-14619564 ] Zhijie Shen commented on YARN-3116: --- Xuan, thanks for your comment. I think this is a

[jira] [Commented] (YARN-3116) [Collector wireup] We need an assured way to determine if a container is an AM container on NM

2015-07-08 Thread Zhijie Shen (JIRA)
[ https://issues.apache.org/jira/browse/YARN-3116?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14619518#comment-14619518 ] Zhijie Shen commented on YARN-3116: --- Is TestAppRunnability failure related to this patch?

[jira] [Updated] (YARN-3116) [Collector wireup] We need an assured way to determine if a container is an AM container on NM

2015-07-08 Thread Zhijie Shen (JIRA)
[ https://issues.apache.org/jira/browse/YARN-3116?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhijie Shen updated YARN-3116: -- Attachment: YARN-3116.v8.patch Fixed TestAppRunnability as well in the new patch. [Collector wireup]

[jira] [Commented] (YARN-3836) add equals and hashCode to TimelineEntity and other classes in the data model

2015-07-08 Thread Zhijie Shen (JIRA)
[ https://issues.apache.org/jira/browse/YARN-3836?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14619729#comment-14619729 ] Zhijie Shen commented on YARN-3836: --- bq. I see that we're implementing the Comparable

[jira] [Commented] (YARN-3047) [Data Serving] Set up ATS reader with basic request serving structure and lifecycle

2015-07-07 Thread Zhijie Shen (JIRA)
[ https://issues.apache.org/jira/browse/YARN-3047?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14617307#comment-14617307 ] Zhijie Shen commented on YARN-3047: --- Would you please hold a while? I plan to take a look

[jira] [Commented] (YARN-3047) [Data Serving] Set up ATS reader with basic request serving structure and lifecycle

2015-07-07 Thread Zhijie Shen (JIRA)
[ https://issues.apache.org/jira/browse/YARN-3047?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14617553#comment-14617553 ] Zhijie Shen commented on YARN-3047: --- It looks good to me overall, exception the config.

[jira] [Updated] (YARN-3116) [Collector wireup] We need an assured way to determine if a container is an AM container on NM

2015-07-07 Thread Zhijie Shen (JIRA)
[ https://issues.apache.org/jira/browse/YARN-3116?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhijie Shen updated YARN-3116: -- Attachment: YARN-3116.v6.patch Fixed the test failure in the new patch. Otherwise, the previous patch

[jira] [Commented] (YARN-3051) [Storage abstraction] Create backing storage read interface for ATS readers

2015-07-06 Thread Zhijie Shen (JIRA)
[ https://issues.apache.org/jira/browse/YARN-3051?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14615387#comment-14615387 ] Zhijie Shen commented on YARN-3051: --- How about we using common csv lib to handle the

[jira] [Commented] (YARN-3051) [Storage abstraction] Create backing storage read interface for ATS readers

2015-07-06 Thread Zhijie Shen (JIRA)
[ https://issues.apache.org/jira/browse/YARN-3051?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14615839#comment-14615839 ] Zhijie Shen commented on YARN-3051: --- Okay, then it seems to be fine. I didn't notice it's

[jira] [Commented] (YARN-3051) [Storage abstraction] Create backing storage read interface for ATS readers

2015-07-06 Thread Zhijie Shen (JIRA)
[ https://issues.apache.org/jira/browse/YARN-3051?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14615930#comment-14615930 ] Zhijie Shen commented on YARN-3051: --- Will commit the patch late today if no more

[jira] [Commented] (YARN-3051) [Storage abstraction] Create backing storage read interface for ATS readers

2015-07-06 Thread Zhijie Shen (JIRA)
[ https://issues.apache.org/jira/browse/YARN-3051?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14615792#comment-14615792 ] Zhijie Shen commented on YARN-3051: --- bq. The current FS implementation had cluster as

[jira] [Commented] (YARN-3051) [Storage abstraction] Create backing storage read interface for ATS readers

2015-07-06 Thread Zhijie Shen (JIRA)
[ https://issues.apache.org/jira/browse/YARN-3051?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14615747#comment-14615747 ] Zhijie Shen commented on YARN-3051: --- Hi Varun, thanks for updating the patch. I have only

[jira] [Updated] (YARN-3047) [Data Serving] Set up ATS reader with basic request serving structure and lifecycle

2015-07-06 Thread Zhijie Shen (JIRA)
[ https://issues.apache.org/jira/browse/YARN-3047?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhijie Shen updated YARN-3047: -- Labels: (was: BB2015-05-TBR) [Data Serving] Set up ATS reader with basic request serving structure

[jira] [Commented] (YARN-3047) [Data Serving] Set up ATS reader with basic request serving structure and lifecycle

2015-07-06 Thread Zhijie Shen (JIRA)
[ https://issues.apache.org/jira/browse/YARN-3047?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14616031#comment-14616031 ] Zhijie Shen commented on YARN-3047: --- YARN-3051 has been committed. Would you please

[jira] [Commented] (YARN-3881) Writing RM cluster-level metrics

2015-07-02 Thread Zhijie Shen (JIRA)
[ https://issues.apache.org/jira/browse/YARN-3881?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14612638#comment-14612638 ] Zhijie Shen commented on YARN-3881: --- Once the metrics are ready, we can build

[jira] [Created] (YARN-3880) Writing more RM side app-level metrics

2015-07-02 Thread Zhijie Shen (JIRA)
Zhijie Shen created YARN-3880: - Summary: Writing more RM side app-level metrics Key: YARN-3880 URL: https://issues.apache.org/jira/browse/YARN-3880 Project: Hadoop YARN Issue Type: Sub-task

[jira] [Commented] (YARN-3051) [Storage abstraction] Create backing storage read interface for ATS readers

2015-07-02 Thread Zhijie Shen (JIRA)
[ https://issues.apache.org/jira/browse/YARN-3051?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14612113#comment-14612113 ] Zhijie Shen commented on YARN-3051: --- 2. I meant we store appId, user, flowId, flowRunId

[jira] [Updated] (YARN-3881) Writing RM cluster-level metrics

2015-07-02 Thread Zhijie Shen (JIRA)
[ https://issues.apache.org/jira/browse/YARN-3881?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhijie Shen updated YARN-3881: -- Attachment: metrics.json Writing RM cluster-level metrics

[jira] [Created] (YARN-3881) Writing RM cluster-level metrics

2015-07-02 Thread Zhijie Shen (JIRA)
Zhijie Shen created YARN-3881: - Summary: Writing RM cluster-level metrics Key: YARN-3881 URL: https://issues.apache.org/jira/browse/YARN-3881 Project: Hadoop YARN Issue Type: Sub-task

[jira] [Commented] (YARN-3881) Writing RM cluster-level metrics

2015-07-02 Thread Zhijie Shen (JIRA)
[ https://issues.apache.org/jira/browse/YARN-3881?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14612429#comment-14612429 ] Zhijie Shen commented on YARN-3881: --- IMHO, we need to add an addition API to direct write

[jira] [Commented] (YARN-3116) [Collector wireup] We need an assured way to determine if a container is an AM container on NM

2015-07-01 Thread Zhijie Shen (JIRA)
[ https://issues.apache.org/jira/browse/YARN-3116?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14611445#comment-14611445 ] Zhijie Shen commented on YARN-3116: --- 1. I think normal execution won't have null attempt,

[jira] [Commented] (YARN-3116) [Collector wireup] We need an assured way to determine if a container is an AM container on NM

2015-07-01 Thread Zhijie Shen (JIRA)
[ https://issues.apache.org/jira/browse/YARN-3116?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14611081#comment-14611081 ] Zhijie Shen commented on YARN-3116: --- Looks good to me overall. Two nits: 1. I thought it

[jira] [Commented] (YARN-3051) [Storage abstraction] Create backing storage read interface for ATS readers

2015-07-01 Thread Zhijie Shen (JIRA)
[ https://issues.apache.org/jira/browse/YARN-3051?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14611156#comment-14611156 ] Zhijie Shen commented on YARN-3051: --- [~varun_saxena], thanks for being patient about the

[jira] [Commented] (YARN-3116) [Collector wireup] We need an assured way to determine if a container is an AM container on NM

2015-06-30 Thread Zhijie Shen (JIRA)
[ https://issues.apache.org/jira/browse/YARN-3116?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14609377#comment-14609377 ] Zhijie Shen commented on YARN-3116: --- In fact jenkins complains much fewer checkstyle

[jira] [Commented] (YARN-3823) Fix mismatch in default values for yarn.scheduler.maximum-allocation-vcores property

2015-06-30 Thread Zhijie Shen (JIRA)
[ https://issues.apache.org/jira/browse/YARN-3823?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14608811#comment-14608811 ] Zhijie Shen commented on YARN-3823: --- Hm... I think we forgot changing the default in the

[jira] [Commented] (YARN-3051) [Storage abstraction] Create backing storage read interface for ATS readers

2015-06-30 Thread Zhijie Shen (JIRA)
[ https://issues.apache.org/jira/browse/YARN-3051?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14608774#comment-14608774 ] Zhijie Shen commented on YARN-3051: --- bq. One more thing I have changed is limit of

  1   2   3   4   5   6   7   8   9   10   >