[jira] [Updated] (YARN-7598) Document how to use classpath isolation for aux-services in YARN

2018-04-18 Thread Xuan Gong (JIRA)
[ https://issues.apache.org/jira/browse/YARN-7598?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xuan Gong updated YARN-7598: Attachment: YARN-7598.5.patch > Document how to use classpath isolation for aux-services in YARN >

[jira] [Commented] (YARN-7598) Document how to use classpath isolation for aux-services in YARN

2018-04-10 Thread Xuan Gong (JIRA)
[ https://issues.apache.org/jira/browse/YARN-7598?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16432601#comment-16432601 ] Xuan Gong commented on YARN-7598: - Thanks for the review. [~djp] Uploaded a new patch to address all your

[jira] [Updated] (YARN-7598) Document how to use classpath isolation for aux-services in YARN

2018-04-10 Thread Xuan Gong (JIRA)
[ https://issues.apache.org/jira/browse/YARN-7598?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xuan Gong updated YARN-7598: Attachment: YARN-7598.4.patch > Document how to use classpath isolation for aux-services in YARN >

[jira] [Updated] (YARN-8128) Document better the per-node per-app file limit in YARN log aggregation

2018-04-08 Thread Xuan Gong (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8128?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xuan Gong updated YARN-8128: Attachment: YARN-8128.1.patch > Document better the per-node per-app file limit in YARN log aggregation >

[jira] [Created] (YARN-8128) Document better the per-node per-app file limit in YARN log aggregation

2018-04-08 Thread Xuan Gong (JIRA)
Xuan Gong created YARN-8128: --- Summary: Document better the per-node per-app file limit in YARN log aggregation Key: YARN-8128 URL: https://issues.apache.org/jira/browse/YARN-8128 Project: Hadoop YARN

[jira] [Updated] (YARN-7598) Document how to use classpath isolation for aux-services in YARN

2018-04-08 Thread Xuan Gong (JIRA)
[ https://issues.apache.org/jira/browse/YARN-7598?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xuan Gong updated YARN-7598: Attachment: YARN-7598.3.patch > Document how to use classpath isolation for aux-services in YARN >

[jira] [Resolved] (YARN-5193) For long running services, aggregate logs when a container completes instead of when the app completes

2018-04-08 Thread Xuan Gong (JIRA)
[ https://issues.apache.org/jira/browse/YARN-5193?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xuan Gong resolved YARN-5193. - Resolution: Won't Fix > For long running services, aggregate logs when a container completes instead > of

[jira] [Commented] (YARN-5193) For long running services, aggregate logs when a container completes instead of when the app completes

2018-04-08 Thread Xuan Gong (JIRA)
[ https://issues.apache.org/jira/browse/YARN-5193?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16429882#comment-16429882 ] Xuan Gong commented on YARN-5193: - Using the new log format can solve this issue.  > For long running

[jira] [Commented] (YARN-1151) Ability to configure auxiliary services from HDFS-based JAR files

2018-04-04 Thread Xuan Gong (JIRA)
[ https://issues.apache.org/jira/browse/YARN-1151?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16426068#comment-16426068 ] Xuan Gong commented on YARN-1151: - [~leftnoteasy] bq. I'm not sure if I understand correct: assume we have

[jira] [Updated] (YARN-1151) Ability to configure auxiliary services from HDFS-based JAR files

2018-04-04 Thread Xuan Gong (JIRA)
[ https://issues.apache.org/jira/browse/YARN-1151?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xuan Gong updated YARN-1151: Attachment: YARN-1151.6.patch > Ability to configure auxiliary services from HDFS-based JAR files >

[jira] [Updated] (YARN-1151) Ability to configure auxiliary services from HDFS-based JAR files

2018-04-03 Thread Xuan Gong (JIRA)
[ https://issues.apache.org/jira/browse/YARN-1151?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xuan Gong updated YARN-1151: Attachment: YARN-1151.5.patch > Ability to configure auxiliary services from HDFS-based JAR files >

[jira] [Commented] (YARN-1151) Ability to configure auxiliary services from HDFS-based JAR files

2018-04-02 Thread Xuan Gong (JIRA)
[ https://issues.apache.org/jira/browse/YARN-1151?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16422956#comment-16422956 ] Xuan Gong commented on YARN-1151: - Thanks for the review. [~rkanter] bq. for the classpath. If it's a

[jira] [Updated] (YARN-1151) Ability to configure auxiliary services from HDFS-based JAR files

2018-04-02 Thread Xuan Gong (JIRA)
[ https://issues.apache.org/jira/browse/YARN-1151?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xuan Gong updated YARN-1151: Attachment: YARN-1151.4.patch > Ability to configure auxiliary services from HDFS-based JAR files >

[jira] [Updated] (YARN-1151) Ability to configure auxiliary services from HDFS-based JAR files

2018-03-29 Thread Xuan Gong (JIRA)
[ https://issues.apache.org/jira/browse/YARN-1151?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xuan Gong updated YARN-1151: Attachment: YARN-1151.3.patch > Ability to configure auxiliary services from HDFS-based JAR files >

[jira] [Commented] (YARN-1151) Ability to configure auxiliary services from HDFS-based JAR files

2018-03-29 Thread Xuan Gong (JIRA)
[ https://issues.apache.org/jira/browse/YARN-1151?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16419825#comment-16419825 ] Xuan Gong commented on YARN-1151: - Testcase failure and findbug issue are not related. Uploaded a new

[jira] [Commented] (YARN-1151) Ability to configure auxiliary services from HDFS-based JAR files

2018-03-28 Thread Xuan Gong (JIRA)
[ https://issues.apache.org/jira/browse/YARN-1151?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16417819#comment-16417819 ] Xuan Gong commented on YARN-1151: - [~rkanter]   Could you review the latest patch, please? > Ability to

[jira] [Updated] (YARN-1151) Ability to configure auxiliary services from HDFS-based JAR files

2018-03-28 Thread Xuan Gong (JIRA)
[ https://issues.apache.org/jira/browse/YARN-1151?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xuan Gong updated YARN-1151: Attachment: YARN-1151.2.patch > Ability to configure auxiliary services from HDFS-based JAR files >

[jira] [Commented] (YARN-1151) Ability to configure auxiliary services from HDFS-based JAR files

2018-03-26 Thread Xuan Gong (JIRA)
[ https://issues.apache.org/jira/browse/YARN-1151?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16414411#comment-16414411 ] Xuan Gong commented on YARN-1151: - Thanks for the comments. [~vinodkv] All of your comments make sense.

[jira] [Updated] (YARN-1151) Ability to configure auxiliary services from HDFS-based JAR files

2018-03-25 Thread Xuan Gong (JIRA)
[ https://issues.apache.org/jira/browse/YARN-1151?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xuan Gong updated YARN-1151: Attachment: YARN-1151.branch-2.poc.3.patch > Ability to configure auxiliary services from HDFS-based JAR

[jira] [Updated] (YARN-1151) Ability to configure auxiliary services from HDFS-based JAR files

2018-03-25 Thread Xuan Gong (JIRA)
[ https://issues.apache.org/jira/browse/YARN-1151?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xuan Gong updated YARN-1151: Attachment: YARN-1151.branch-2.poc.2.patch > Ability to configure auxiliary services from HDFS-based JAR

[jira] [Updated] (YARN-5523) Yarn running container log fetching causes OutOfMemoryError

2018-03-22 Thread Xuan Gong (JIRA)
[ https://issues.apache.org/jira/browse/YARN-5523?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xuan Gong updated YARN-5523: Description: Fetching a 256MB log from container caused OOM on client {code:java} $ yarn logs -applicationId

[jira] [Commented] (YARN-1151) Ability to configure auxiliary services from HDFS-based JAR files

2018-03-21 Thread Xuan Gong (JIRA)
[ https://issues.apache.org/jira/browse/YARN-1151?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16409013#comment-16409013 ] Xuan Gong commented on YARN-1151: - Uploaded a poc patch for the fix. In the fix, we created a new

[jira] [Updated] (YARN-1151) Ability to configure auxiliary services from HDFS-based JAR files

2018-03-21 Thread Xuan Gong (JIRA)
[ https://issues.apache.org/jira/browse/YARN-1151?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xuan Gong updated YARN-1151: Attachment: YARN-1151.branch-2.poc.patch > Ability to configure auxiliary services from HDFS-based JAR files

[jira] [Updated] (YARN-1151) Ability to configure auxiliary services from HDFS-based JAR files

2018-03-16 Thread Xuan Gong (JIRA)
[ https://issues.apache.org/jira/browse/YARN-1151?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xuan Gong updated YARN-1151: Attachment: [YARN-1151] [Design] Configure auxiliary services from HDFS-based JAR files.pdf > Ability to

[jira] [Commented] (YARN-7952) RM should be able to recover log aggregation status after restart/fail-over

2018-03-15 Thread Xuan Gong (JIRA)
[ https://issues.apache.org/jira/browse/YARN-7952?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16399973#comment-16399973 ] Xuan Gong commented on YARN-7952: - findbug warning and testcase failure are not related > RM should be

[jira] [Commented] (YARN-7952) RM should be able to recover log aggregation status after restart/fail-over

2018-03-14 Thread Xuan Gong (JIRA)
[ https://issues.apache.org/jira/browse/YARN-7952?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16399634#comment-16399634 ] Xuan Gong commented on YARN-7952: - Uploaded a new patch to address all the comments > RM should be able to

[jira] [Updated] (YARN-7952) RM should be able to recover log aggregation status after restart/fail-over

2018-03-14 Thread Xuan Gong (JIRA)
[ https://issues.apache.org/jira/browse/YARN-7952?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xuan Gong updated YARN-7952: Attachment: YARN-7952.9.patch > RM should be able to recover log aggregation status after restart/fail-over

[jira] [Updated] (YARN-7952) RM should be able to recover log aggregation status after restart/fail-over

2018-03-14 Thread Xuan Gong (JIRA)
[ https://issues.apache.org/jira/browse/YARN-7952?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xuan Gong updated YARN-7952: Attachment: YARN-7952.8.patch > RM should be able to recover log aggregation status after restart/fail-over

[jira] [Commented] (YARN-7952) RM should be able to recover log aggregation status after restart/fail-over

2018-03-14 Thread Xuan Gong (JIRA)
[ https://issues.apache.org/jira/browse/YARN-7952?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16398185#comment-16398185 ] Xuan Gong commented on YARN-7952: - Thanks for the comments. [~rkanter] Attached a new patch to address all

[jira] [Commented] (YARN-7952) RM should be able to recover log aggregation status after restart/fail-over

2018-03-08 Thread Xuan Gong (JIRA)
[ https://issues.apache.org/jira/browse/YARN-7952?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16392307#comment-16392307 ] Xuan Gong commented on YARN-7952: - [~leftnoteasy] The unit test failure is not related. I can pass it

[jira] [Updated] (YARN-7952) RM should be able to recover log aggregation status after restart/fail-over

2018-03-08 Thread Xuan Gong (JIRA)
[ https://issues.apache.org/jira/browse/YARN-7952?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xuan Gong updated YARN-7952: Attachment: YARN-7952.7.patch > RM should be able to recover log aggregation status after restart/fail-over

[jira] [Commented] (YARN-7952) RM should be able to recover log aggregation status after restart/fail-over

2018-03-08 Thread Xuan Gong (JIRA)
[ https://issues.apache.org/jira/browse/YARN-7952?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16391008#comment-16391008 ] Xuan Gong commented on YARN-7952: - Thanks, [~leftnoteasy] Addressed all of your comments. Also fix the

[jira] [Updated] (YARN-7952) RM should be able to recover log aggregation status after restart/fail-over

2018-03-08 Thread Xuan Gong (JIRA)
[ https://issues.apache.org/jira/browse/YARN-7952?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xuan Gong updated YARN-7952: Attachment: YARN-7952.6.patch > RM should be able to recover log aggregation status after restart/fail-over

[jira] [Commented] (YARN-7952) RM should be able to recover log aggregation status after restart/fail-over

2018-03-07 Thread Xuan Gong (JIRA)
[ https://issues.apache.org/jira/browse/YARN-7952?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16390501#comment-16390501 ] Xuan Gong commented on YARN-7952: - rebase the patch, and kill off the jenkins > RM should be able to

[jira] [Updated] (YARN-7952) RM should be able to recover log aggregation status after restart/fail-over

2018-03-07 Thread Xuan Gong (JIRA)
[ https://issues.apache.org/jira/browse/YARN-7952?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xuan Gong updated YARN-7952: Attachment: YARN-7952.5.patch > RM should be able to recover log aggregation status after restart/fail-over

[jira] [Updated] (YARN-7952) Find a way to persist the log aggregation status

2018-03-06 Thread Xuan Gong (JIRA)
[ https://issues.apache.org/jira/browse/YARN-7952?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xuan Gong updated YARN-7952: Attachment: YARN-7952.3.patch > Find a way to persist the log aggregation status >

[jira] [Updated] (YARN-7952) Find a way to persist the log aggregation status

2018-03-06 Thread Xuan Gong (JIRA)
[ https://issues.apache.org/jira/browse/YARN-7952?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xuan Gong updated YARN-7952: Attachment: YARN-7952.3.patch > Find a way to persist the log aggregation status >

[jira] [Commented] (YARN-7952) Find a way to persist the log aggregation status

2018-03-06 Thread Xuan Gong (JIRA)
[ https://issues.apache.org/jira/browse/YARN-7952?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16387707#comment-16387707 ] Xuan Gong commented on YARN-7952: - Thanks for the review. [~leftnoteasy] bq. It looks like pullLock should

[jira] [Comment Edited] (YARN-7952) Find a way to persist the log aggregation status

2018-03-06 Thread Xuan Gong (JIRA)
[ https://issues.apache.org/jira/browse/YARN-7952?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16387707#comment-16387707 ] Xuan Gong edited comment on YARN-7952 at 3/6/18 12:52 PM: -- Thanks for the review.

[jira] [Commented] (YARN-7891) LogAggregationIndexedFileController should support HAR file

2018-03-05 Thread Xuan Gong (JIRA)
[ https://issues.apache.org/jira/browse/YARN-7891?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16387117#comment-16387117 ] Xuan Gong commented on YARN-7891: - [~leftnoteasy] Yes, that is correct. As I mentioned previously, we do

[jira] [Commented] (YARN-7952) Find a way to persist the log aggregation status

2018-03-04 Thread Xuan Gong (JIRA)
[ https://issues.apache.org/jira/browse/YARN-7952?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16385623#comment-16385623 ] Xuan Gong commented on YARN-7952: - Yes, re-use previous configuration makes sense. But we probably need to

[jira] [Updated] (YARN-7952) Find a way to persist the log aggregation status

2018-03-04 Thread Xuan Gong (JIRA)
[ https://issues.apache.org/jira/browse/YARN-7952?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xuan Gong updated YARN-7952: Attachment: YARN-7952.2.patch > Find a way to persist the log aggregation status >

[jira] [Commented] (YARN-7891) LogAggregationIndexedFileController should support HAR file

2018-03-02 Thread Xuan Gong (JIRA)
[ https://issues.apache.org/jira/browse/YARN-7891?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16383542#comment-16383542 ] Xuan Gong commented on YARN-7891: - Thanks for the review. Previous code does not work. It does not handle

[jira] [Commented] (YARN-7952) Find a way to persist the log aggregation status

2018-03-01 Thread Xuan Gong (JIRA)
[ https://issues.apache.org/jira/browse/YARN-7952?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16382151#comment-16382151 ] Xuan Gong commented on YARN-7952: - Did the manual test, and verify after RM restart/failover, we can

[jira] [Updated] (YARN-7952) Find a way to persist the log aggregation status

2018-03-01 Thread Xuan Gong (JIRA)
[ https://issues.apache.org/jira/browse/YARN-7952?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xuan Gong updated YARN-7952: Attachment: YARN-7952.1.patch > Find a way to persist the log aggregation status >

[jira] [Commented] (YARN-7952) Find a way to persist the log aggregation status

2018-02-28 Thread Xuan Gong (JIRA)
[ https://issues.apache.org/jira/browse/YARN-7952?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16381348#comment-16381348 ] Xuan Gong commented on YARN-7952: - create a poc patch based on the option#2. will do the test and add more

[jira] [Updated] (YARN-7952) Find a way to persist the log aggregation status

2018-02-28 Thread Xuan Gong (JIRA)
[ https://issues.apache.org/jira/browse/YARN-7952?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xuan Gong updated YARN-7952: Attachment: YARN-7952-poc.patch > Find a way to persist the log aggregation status >

[jira] [Commented] (YARN-7952) Find a way to persist the log aggregation status

2018-02-20 Thread Xuan Gong (JIRA)
[ https://issues.apache.org/jira/browse/YARN-7952?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16370900#comment-16370900 ] Xuan Gong commented on YARN-7952: - [~rkanter] [~leftnoteasy] [~vinodkv] What do you think? > Find a way

[jira] [Commented] (YARN-7952) Find a way to persist the log aggregation status

2018-02-20 Thread Xuan Gong (JIRA)
[ https://issues.apache.org/jira/browse/YARN-7952?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16370898#comment-16370898 ] Xuan Gong commented on YARN-7952: - Right now, the NM would send its own log aggregation status to RM

[jira] [Created] (YARN-7952) Find a way to persist the log aggregation status

2018-02-20 Thread Xuan Gong (JIRA)
Xuan Gong created YARN-7952: --- Summary: Find a way to persist the log aggregation status Key: YARN-7952 URL: https://issues.apache.org/jira/browse/YARN-7952 Project: Hadoop YARN Issue Type: Bug

[jira] [Created] (YARN-7951) Find a way to persist the log aggregation status

2018-02-20 Thread Xuan Gong (JIRA)
Xuan Gong created YARN-7951: --- Summary: Find a way to persist the log aggregation status Key: YARN-7951 URL: https://issues.apache.org/jira/browse/YARN-7951 Project: Hadoop YARN Issue Type: Bug

[jira] [Updated] (YARN-7891) LogAggregationIndexedFileController should support HAR file

2018-02-04 Thread Xuan Gong (JIRA)
[ https://issues.apache.org/jira/browse/YARN-7891?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xuan Gong updated YARN-7891: Attachment: YARN-7891.1.patch > LogAggregationIndexedFileController should support HAR file >

[jira] [Commented] (YARN-7697) NM goes down with OOM due to leak in log-aggregation

2018-02-03 Thread Xuan Gong (JIRA)
[ https://issues.apache.org/jira/browse/YARN-7697?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16351657#comment-16351657 ] Xuan Gong commented on YARN-7697: - Fix the findBug issue > NM goes down with OOM due to leak in

[jira] [Updated] (YARN-7697) NM goes down with OOM due to leak in log-aggregation

2018-02-03 Thread Xuan Gong (JIRA)
[ https://issues.apache.org/jira/browse/YARN-7697?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xuan Gong updated YARN-7697: Attachment: YARN-7697.3.patch > NM goes down with OOM due to leak in log-aggregation >

[jira] [Assigned] (YARN-7891) LogAggregationIndexedFileController should support HAR file

2018-02-03 Thread Xuan Gong (JIRA)
[ https://issues.apache.org/jira/browse/YARN-7891?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xuan Gong reassigned YARN-7891: --- Assignee: Xuan Gong > LogAggregationIndexedFileController should support HAR file >

[jira] [Created] (YARN-7891) LogAggregationIndexedFileController should support HAR file

2018-02-03 Thread Xuan Gong (JIRA)
Xuan Gong created YARN-7891: --- Summary: LogAggregationIndexedFileController should support HAR file Key: YARN-7891 URL: https://issues.apache.org/jira/browse/YARN-7891 Project: Hadoop YARN Issue

[jira] [Updated] (YARN-7697) NM goes down with OOM due to leak in log-aggregation

2018-02-03 Thread Xuan Gong (JIRA)
[ https://issues.apache.org/jira/browse/YARN-7697?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xuan Gong updated YARN-7697: Affects Version/s: 3.1.0 > NM goes down with OOM due to leak in log-aggregation >

[jira] [Commented] (YARN-7883) Make HAR tool support IndexedLogAggregtionController

2018-02-02 Thread Xuan Gong (JIRA)
[ https://issues.apache.org/jira/browse/YARN-7883?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16351050#comment-16351050 ] Xuan Gong commented on YARN-7883: - Created a MapReduce ticket to track the work progress. Close this one

[jira] [Resolved] (YARN-7883) Make HAR tool support IndexedLogAggregtionController

2018-02-02 Thread Xuan Gong (JIRA)
[ https://issues.apache.org/jira/browse/YARN-7883?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xuan Gong resolved YARN-7883. - Resolution: Duplicate > Make HAR tool support IndexedLogAggregtionController >

[jira] [Created] (YARN-7883) Make HAR tool support IndexedLogAggregtionController

2018-02-02 Thread Xuan Gong (JIRA)
Xuan Gong created YARN-7883: --- Summary: Make HAR tool support IndexedLogAggregtionController Key: YARN-7883 URL: https://issues.apache.org/jira/browse/YARN-7883 Project: Hadoop YARN Issue Type:

[jira] [Commented] (YARN-7598) Document how to use classpath isolation for aux-services in YARN

2018-01-29 Thread Xuan Gong (JIRA)
[ https://issues.apache.org/jira/browse/YARN-7598?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16344130#comment-16344130 ] Xuan Gong commented on YARN-7598: - [~djp] Thanks for the comment. Attached a new patch for this > Document

[jira] [Updated] (YARN-7598) Document how to use classpath isolation for aux-services in YARN

2018-01-29 Thread Xuan Gong (JIRA)
[ https://issues.apache.org/jira/browse/YARN-7598?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xuan Gong updated YARN-7598: Attachment: YARN-7598.2.patch > Document how to use classpath isolation for aux-services in YARN >

[jira] [Commented] (YARN-7468) Provide means for container network policy control

2018-01-11 Thread Xuan Gong (JIRA)
[ https://issues.apache.org/jira/browse/YARN-7468?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16322796#comment-16322796 ] Xuan Gong commented on YARN-7468: - [~leftnoteasy] Updated. Thanks > Provide means for container network

[jira] [Updated] (YARN-7468) Provide means for container network policy control

2018-01-11 Thread Xuan Gong (JIRA)
[ https://issues.apache.org/jira/browse/YARN-7468?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xuan Gong updated YARN-7468: Attachment: YARN-7468.trunk.5.patch > Provide means for container network policy control >

[jira] [Updated] (YARN-7468) Provide means for container network policy control

2018-01-10 Thread Xuan Gong (JIRA)
[ https://issues.apache.org/jira/browse/YARN-7468?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xuan Gong updated YARN-7468: Attachment: YARN-7468.trunk.4.patch > Provide means for container network policy control >

[jira] [Updated] (YARN-7468) Provide means for container network policy control

2018-01-09 Thread Xuan Gong (JIRA)
[ https://issues.apache.org/jira/browse/YARN-7468?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xuan Gong updated YARN-7468: Attachment: YARN-7468.trunk.3.patch > Provide means for container network policy control >

[jira] [Commented] (YARN-7468) Provide means for container network policy control

2018-01-09 Thread Xuan Gong (JIRA)
[ https://issues.apache.org/jira/browse/YARN-7468?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16319394#comment-16319394 ] Xuan Gong commented on YARN-7468: - Thanks for the review. Attached a new patch for all the comments >

[jira] [Updated] (YARN-7697) NM goes down with OOM due to leak in log-aggregation

2018-01-04 Thread Xuan Gong (JIRA)
[ https://issues.apache.org/jira/browse/YARN-7697?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xuan Gong updated YARN-7697: Attachment: YARN-7697.2.patch > NM goes down with OOM due to leak in log-aggregation >

[jira] [Updated] (YARN-7697) NM goes down with OOM due to leak in log-aggregation

2018-01-03 Thread Xuan Gong (JIRA)
[ https://issues.apache.org/jira/browse/YARN-7697?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xuan Gong updated YARN-7697: Attachment: YARN-7697.1.patch > NM goes down with OOM due to leak in log-aggregation >

[jira] [Commented] (YARN-7697) NM goes down with OOM due to leak in log-aggregation

2018-01-03 Thread Xuan Gong (JIRA)
[ https://issues.apache.org/jira/browse/YARN-7697?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16310159#comment-16310159 ] Xuan Gong commented on YARN-7697: - Upload a fix for this issue which includes two parts: 1) not use

[jira] [Commented] (YARN-7697) NM goes down with OOM due to leak in log-aggregation

2018-01-03 Thread Xuan Gong (JIRA)
[ https://issues.apache.org/jira/browse/YARN-7697?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16310156#comment-16310156 ] Xuan Gong commented on YARN-7697: - The issue happens after file truncate process. Looks like the truncate

[jira] [Updated] (YARN-7697) NM goes down with OOM due to leak in log-aggregation

2018-01-03 Thread Xuan Gong (JIRA)
[ https://issues.apache.org/jira/browse/YARN-7697?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xuan Gong updated YARN-7697: Description: 2017-12-29 01:43:50,601 FATAL yarn.YarnUncaughtExceptionHandler

[jira] [Created] (YARN-7697) NM goes down with OOM due to leak in log-aggregation

2018-01-03 Thread Xuan Gong (JIRA)
Xuan Gong created YARN-7697: --- Summary: NM goes down with OOM due to leak in log-aggregation Key: YARN-7697 URL: https://issues.apache.org/jira/browse/YARN-7697 Project: Hadoop YARN Issue Type: Bug

[jira] [Updated] (YARN-7598) Document how to use classpath isolation for aux-services in YARN

2017-12-18 Thread Xuan Gong (JIRA)
[ https://issues.apache.org/jira/browse/YARN-7598?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xuan Gong updated YARN-7598: Attachment: YARN-7598.trunk.1.patch > Document how to use classpath isolation for aux-services in YARN >

[jira] [Updated] (YARN-7468) Provide means for container network policy control

2017-12-15 Thread Xuan Gong (JIRA)
[ https://issues.apache.org/jira/browse/YARN-7468?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xuan Gong updated YARN-7468: Attachment: YARN-7468.trunk.2.patch > Provide means for container network policy control >

[jira] [Updated] (YARN-7468) Provide means for container network policy control

2017-12-15 Thread Xuan Gong (JIRA)
[ https://issues.apache.org/jira/browse/YARN-7468?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xuan Gong updated YARN-7468: Attachment: YARN-7468.trunk.2.patch > Provide means for container network policy control >

[jira] [Updated] (YARN-7468) Provide means for container network policy control

2017-12-15 Thread Xuan Gong (JIRA)
[ https://issues.apache.org/jira/browse/YARN-7468?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xuan Gong updated YARN-7468: Attachment: YARN-7468.trunk.1.patch > Provide means for container network policy control >

[jira] [Updated] (YARN-7468) Provide means for container network policy control

2017-12-15 Thread Xuan Gong (JIRA)
[ https://issues.apache.org/jira/browse/YARN-7468?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xuan Gong updated YARN-7468: Attachment: YARN-7468.trunk.1.patch uploaded an initial patch for the fix > Provide means for container

[jira] [Assigned] (YARN-7468) Provide means for container network policy control

2017-12-15 Thread Xuan Gong (JIRA)
[ https://issues.apache.org/jira/browse/YARN-7468?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xuan Gong reassigned YARN-7468: --- Assignee: Xuan Gong > Provide means for container network policy control >

[jira] [Updated] (YARN-5418) When partial log aggregation is enabled, display the list of aggregated files on the container log page

2017-12-14 Thread Xuan Gong (JIRA)
[ https://issues.apache.org/jira/browse/YARN-5418?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xuan Gong updated YARN-5418: Attachment: YARN-5418.trunk.5.patch > When partial log aggregation is enabled, display the list of

[jira] [Commented] (YARN-5418) When partial log aggregation is enabled, display the list of aggregated files on the container log page

2017-12-14 Thread Xuan Gong (JIRA)
[ https://issues.apache.org/jira/browse/YARN-5418?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16291293#comment-16291293 ] Xuan Gong commented on YARN-5418: - Thanks for the review. Uploaded a new patch to address all the comments

[jira] [Updated] (YARN-5418) When partial log aggregation is enabled, display the list of aggregated files on the container log page

2017-12-14 Thread Xuan Gong (JIRA)
[ https://issues.apache.org/jira/browse/YARN-5418?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xuan Gong updated YARN-5418: Attachment: YARN-5418.5.branch-2.patch > When partial log aggregation is enabled, display the list of

[jira] [Assigned] (YARN-7598) Document how to use classpath isolation for aux-services in YARN

2017-12-12 Thread Xuan Gong (JIRA)
[ https://issues.apache.org/jira/browse/YARN-7598?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xuan Gong reassigned YARN-7598: --- Assignee: Xuan Gong > Document how to use classpath isolation for aux-services in YARN >

[jira] [Commented] (YARN-7468) Provide means for container network policy control

2017-12-08 Thread Xuan Gong (JIRA)
[ https://issues.apache.org/jira/browse/YARN-7468?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16283834#comment-16283834 ] Xuan Gong commented on YARN-7468: - uploaded a new doc. Please take a look. > Provide means for container

[jira] [Updated] (YARN-7468) Provide means for container network policy control

2017-12-08 Thread Xuan Gong (JIRA)
[ https://issues.apache.org/jira/browse/YARN-7468?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xuan Gong updated YARN-7468: Attachment: [YARN-7468] [Design] Provide means for container network policy control.pdf > Provide means for

[jira] [Commented] (YARN-7468) Provide means for container network policy control

2017-12-07 Thread Xuan Gong (JIRA)
[ https://issues.apache.org/jira/browse/YARN-7468?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16282899#comment-16282899 ] Xuan Gong commented on YARN-7468: - I am working on a design doc. Will upload it tomorrow. > Provide means

[jira] [Updated] (YARN-7468) Provide means for container network policy control

2017-12-07 Thread Xuan Gong (JIRA)
[ https://issues.apache.org/jira/browse/YARN-7468?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xuan Gong updated YARN-7468: Issue Type: Task (was: Improvement) > Provide means for container network policy control >

[jira] [Updated] (YARN-5418) When partial log aggregation is enabled, display the list of aggregated files on the container log page

2017-12-06 Thread Xuan Gong (JIRA)
[ https://issues.apache.org/jira/browse/YARN-5418?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xuan Gong updated YARN-5418: Attachment: YARN-5418.4.branch-2.patch > When partial log aggregation is enabled, display the list of

[jira] [Commented] (YARN-5418) When partial log aggregation is enabled, display the list of aggregated files on the container log page

2017-12-06 Thread Xuan Gong (JIRA)
[ https://issues.apache.org/jira/browse/YARN-5418?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16281104#comment-16281104 ] Xuan Gong commented on YARN-5418: - create a new patch for this. [~leftnoteasy] Please review. > When

[jira] [Updated] (YARN-5418) When partial log aggregation is enabled, display the list of aggregated files on the container log page

2017-12-06 Thread Xuan Gong (JIRA)
[ https://issues.apache.org/jira/browse/YARN-5418?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xuan Gong updated YARN-5418: Attachment: YARN-5418.branch-2.4.patch YARN-5418.trunk.4.patch > When partial log

[jira] [Commented] (YARN-7468) Provide means for container network policy control

2017-12-03 Thread Xuan Gong (JIRA)
[ https://issues.apache.org/jira/browse/YARN-7468?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16276159#comment-16276159 ] Xuan Gong commented on YARN-7468: - Thanks, [~clayb] for creating the Jira. In general, we are trying to

[jira] [Commented] (YARN-7381) Enable the configuration: yarn.nodemanager.log-container-debug-info.enabled

2017-12-01 Thread Xuan Gong (JIRA)
[ https://issues.apache.org/jira/browse/YARN-7381?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16275214#comment-16275214 ] Xuan Gong commented on YARN-7381: - bq. I believe we currently aggregate all local files to a single HDFS

[jira] [Created] (YARN-7598) Document how to use classpath isolation for aux-services in YARN

2017-12-01 Thread Xuan Gong (JIRA)
Xuan Gong created YARN-7598: --- Summary: Document how to use classpath isolation for aux-services in YARN Key: YARN-7598 URL: https://issues.apache.org/jira/browse/YARN-7598 Project: Hadoop YARN

[jira] [Commented] (YARN-7558) "yarn logs" command fails to get logs for running containers if UI authentication is enabled.

2017-11-29 Thread Xuan Gong (JIRA)
[ https://issues.apache.org/jira/browse/YARN-7558?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16271283#comment-16271283 ] Xuan Gong commented on YARN-7558: - Thanks,[~djp] for the review. bq. is it possible to add a UT to cover

[jira] [Commented] (YARN-7558) "yarn logs" command fails to get logs for running containers if UI authentication is enabled.

2017-11-28 Thread Xuan Gong (JIRA)
[ https://issues.apache.org/jira/browse/YARN-7558?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16269143#comment-16269143 ] Xuan Gong commented on YARN-7558: - Testcase failure is not related > "yarn logs" command fails to get logs

[jira] [Created] (YARN-7571) YARN has multiple Queue Configuration Providers

2017-11-27 Thread Xuan Gong (JIRA)
Xuan Gong created YARN-7571: --- Summary: YARN has multiple Queue Configuration Providers Key: YARN-7571 URL: https://issues.apache.org/jira/browse/YARN-7571 Project: Hadoop YARN Issue Type: Sub-task

[jira] [Created] (YARN-7570) Merge Scheduler configuration Store and ZK RM State Store

2017-11-27 Thread Xuan Gong (JIRA)
Xuan Gong created YARN-7570: --- Summary: Merge Scheduler configuration Store and ZK RM State Store Key: YARN-7570 URL: https://issues.apache.org/jira/browse/YARN-7570 Project: Hadoop YARN Issue

[jira] [Created] (YARN-7569) Add ability to dump and upload scheduler configuration XML

2017-11-27 Thread Xuan Gong (JIRA)
Xuan Gong created YARN-7569: --- Summary: Add ability to dump and upload scheduler configuration XML Key: YARN-7569 URL: https://issues.apache.org/jira/browse/YARN-7569 Project: Hadoop YARN Issue

[jira] [Created] (YARN-7568) Add audit log for add/remove queues

2017-11-27 Thread Xuan Gong (JIRA)
Xuan Gong created YARN-7568: --- Summary: Add audit log for add/remove queues Key: YARN-7568 URL: https://issues.apache.org/jira/browse/YARN-7568 Project: Hadoop YARN Issue Type: Sub-task

[jira] [Updated] (YARN-7567) Make sure that OrgQueue works in secure ZK environment

2017-11-27 Thread Xuan Gong (JIRA)
[ https://issues.apache.org/jira/browse/YARN-7567?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xuan Gong updated YARN-7567: Description: In YARN-5734, we introduced OrgQueue for easy CapacityScheduler queue management. And we are

  1   2   3   4   5   6   7   8   9   10   >