[jira] [Commented] (YARN-6929) yarn.nodemanager.remote-app-log-dir structure is not scalable

2019-04-30 Thread Prabhu Joseph (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6929?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16830815#comment-16830815 ] Prabhu Joseph commented on YARN-6929: - Thanks [~eyang]. > yarn.nodemanager.remote-app-log-dir

[jira] [Commented] (YARN-6929) yarn.nodemanager.remote-app-log-dir structure is not scalable

2019-04-30 Thread Hudson (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6929?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16830714#comment-16830714 ] Hudson commented on YARN-6929: -- SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #16482 (See

[jira] [Commented] (YARN-6929) yarn.nodemanager.remote-app-log-dir structure is not scalable

2019-04-30 Thread Eric Yang (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6929?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16830687#comment-16830687 ] Eric Yang commented on YARN-6929: - +1 Patch 11 looks good to me. > yarn.nodemanager.remote-app-log-dir

[jira] [Commented] (YARN-6929) yarn.nodemanager.remote-app-log-dir structure is not scalable

2019-04-27 Thread Hadoop QA (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6929?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16827667#comment-16827667 ] Hadoop QA commented on YARN-6929: - | (/) *{color:green}+1 overall{color}* | \\ \\ || Vote || Subsystem ||

[jira] [Commented] (YARN-6929) yarn.nodemanager.remote-app-log-dir structure is not scalable

2019-04-27 Thread Prabhu Joseph (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6929?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16827641#comment-16827641 ] Prabhu Joseph commented on YARN-6929: - Thanks [~eyang] for testing the patch. Have compared two

[jira] [Commented] (YARN-6929) yarn.nodemanager.remote-app-log-dir structure is not scalable

2019-04-26 Thread Eric Yang (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6929?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16827343#comment-16827343 ] Eric Yang commented on YARN-6929: - [~Prabhu Joseph] If the {aggregation_log_root}/{user} does not exist.

[jira] [Commented] (YARN-6929) yarn.nodemanager.remote-app-log-dir structure is not scalable

2019-04-26 Thread Prabhu Joseph (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6929?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16827030#comment-16827030 ] Prabhu Joseph commented on YARN-6929: - Thanks [~adam.antal] for checking. Yes,

[jira] [Commented] (YARN-6929) yarn.nodemanager.remote-app-log-dir structure is not scalable

2019-04-26 Thread Adam Antal (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6929?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16826983#comment-16826983 ] Adam Antal commented on YARN-6929: -- This patch has a very good side-effect. For not-posix compliant

[jira] [Commented] (YARN-6929) yarn.nodemanager.remote-app-log-dir structure is not scalable

2019-04-26 Thread Prabhu Joseph (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6929?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16826801#comment-16826801 ] Prabhu Joseph commented on YARN-6929: - Thanks [~pbacsko] for the review. [~eyang] Could you please

[jira] [Commented] (YARN-6929) yarn.nodemanager.remote-app-log-dir structure is not scalable

2019-04-16 Thread Peter Bacsko (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6929?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16818918#comment-16818918 ] Peter Bacsko commented on YARN-6929: Latest patch LGTM +1 (non-binding) >

[jira] [Commented] (YARN-6929) yarn.nodemanager.remote-app-log-dir structure is not scalable

2019-04-16 Thread Prabhu Joseph (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6929?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16818738#comment-16818738 ] Prabhu Joseph commented on YARN-6929: - [~pbacsko] [~eyang] Have changed the app log dir structure to

[jira] [Commented] (YARN-6929) yarn.nodemanager.remote-app-log-dir structure is not scalable

2019-04-12 Thread Hadoop QA (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6929?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16816671#comment-16816671 ] Hadoop QA commented on YARN-6929: - | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem ||

[jira] [Commented] (YARN-6929) yarn.nodemanager.remote-app-log-dir structure is not scalable

2019-04-11 Thread Prabhu Joseph (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6929?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16815996#comment-16815996 ] Prabhu Joseph commented on YARN-6929: - This looks good to me. Will change the code accordingly.

[jira] [Commented] (YARN-6929) yarn.nodemanager.remote-app-log-dir structure is not scalable

2019-04-11 Thread Eric Yang (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6929?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16815991#comment-16815991 ] Eric Yang commented on YARN-6929: - {quote}Looks cluster_timestamp is missed, it is also required as the

[jira] [Commented] (YARN-6929) yarn.nodemanager.remote-app-log-dir structure is not scalable

2019-04-11 Thread Prabhu Joseph (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6929?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16815939#comment-16815939 ] Prabhu Joseph commented on YARN-6929: - [~eyang] Thanks for reviewing again. Have few doubts with new

[jira] [Commented] (YARN-6929) yarn.nodemanager.remote-app-log-dir structure is not scalable

2019-04-11 Thread Eric Yang (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6929?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16815815#comment-16815815 ] Eric Yang commented on YARN-6929: - [~Prabhu Joseph] {code}{aggregation_log_root} / / bucket_{suffix} /

[jira] [Commented] (YARN-6929) yarn.nodemanager.remote-app-log-dir structure is not scalable

2019-04-10 Thread Prabhu Joseph (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6929?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16815068#comment-16815068 ] Prabhu Joseph commented on YARN-6929: - [~pbacsko] [~eyang] Can you review the 009 patch when you get

[jira] [Commented] (YARN-6929) yarn.nodemanager.remote-app-log-dir structure is not scalable

2019-04-10 Thread Hadoop QA (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6929?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16814114#comment-16814114 ] Hadoop QA commented on YARN-6929: - | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem ||

[jira] [Commented] (YARN-6929) yarn.nodemanager.remote-app-log-dir structure is not scalable

2019-04-09 Thread Prabhu Joseph (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6929?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16814033#comment-16814033 ] Prabhu Joseph commented on YARN-6929: - Build Failure is due to YARN-999 and fixed in the

[jira] [Commented] (YARN-6929) yarn.nodemanager.remote-app-log-dir structure is not scalable

2019-04-09 Thread Hadoop QA (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6929?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16813822#comment-16813822 ] Hadoop QA commented on YARN-6929: - | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem ||

[jira] [Commented] (YARN-6929) yarn.nodemanager.remote-app-log-dir structure is not scalable

2019-04-09 Thread Prabhu Joseph (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6929?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16813661#comment-16813661 ] Prabhu Joseph commented on YARN-6929: - [~pbacsko] Thanks for reviewing. Have attached 008 patch

[jira] [Commented] (YARN-6929) yarn.nodemanager.remote-app-log-dir structure is not scalable

2019-04-08 Thread Peter Bacsko (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6929?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16812294#comment-16812294 ] Peter Bacsko commented on YARN-6929: [~Prabhu Joseph] Here's my quick take on the latest patch. This

[jira] [Commented] (YARN-6929) yarn.nodemanager.remote-app-log-dir structure is not scalable

2019-04-07 Thread Prabhu Joseph (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6929?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16811866#comment-16811866 ] Prabhu Joseph commented on YARN-6929: - [~eyang] Have changed the App Log Dir Structure to below

[jira] [Commented] (YARN-6929) yarn.nodemanager.remote-app-log-dir structure is not scalable

2019-04-05 Thread Prabhu Joseph (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6929?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16811143#comment-16811143 ] Prabhu Joseph commented on YARN-6929: - Thanks [~eyang] for detailed review. Working on it, will update

[jira] [Commented] (YARN-6929) yarn.nodemanager.remote-app-log-dir structure is not scalable

2019-04-05 Thread Eric Yang (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6929?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16810986#comment-16810986 ] Eric Yang commented on YARN-6929: - If I understand the patch correctly,

[jira] [Commented] (YARN-6929) yarn.nodemanager.remote-app-log-dir structure is not scalable

2019-04-05 Thread Hadoop QA (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6929?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16810725#comment-16810725 ] Hadoop QA commented on YARN-6929: - | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem ||

[jira] [Commented] (YARN-6929) yarn.nodemanager.remote-app-log-dir structure is not scalable

2019-04-05 Thread Prabhu Joseph (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6929?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16810657#comment-16810657 ] Prabhu Joseph commented on YARN-6929: - [~snemeth] [~pbacsko] [~eyang] Can you review this jira when

[jira] [Commented] (YARN-6929) yarn.nodemanager.remote-app-log-dir structure is not scalable

2019-02-19 Thread Prabhu Joseph (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6929?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16772691#comment-16772691 ] Prabhu Joseph commented on YARN-6929: - [~jlowe] Can you review this jira when you get time. Thanks. >

[jira] [Commented] (YARN-6929) yarn.nodemanager.remote-app-log-dir structure is not scalable

2019-01-23 Thread Prabhu Joseph (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6929?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16749748#comment-16749748 ] Prabhu Joseph commented on YARN-6929: - [~jlowe] [~rohithsharma] Can you review this jira when you get

[jira] [Commented] (YARN-6929) yarn.nodemanager.remote-app-log-dir structure is not scalable

2019-01-21 Thread Hadoop QA (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6929?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16747815#comment-16747815 ] Hadoop QA commented on YARN-6929: - | (/) *{color:green}+1 overall{color}* | \\ \\ || Vote || Subsystem ||

[jira] [Commented] (YARN-6929) yarn.nodemanager.remote-app-log-dir structure is not scalable

2019-01-20 Thread Hadoop QA (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6929?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16747720#comment-16747720 ] Hadoop QA commented on YARN-6929: - | (/) *{color:green}+1 overall{color}* | \\ \\ || Vote || Subsystem ||

[jira] [Commented] (YARN-6929) yarn.nodemanager.remote-app-log-dir structure is not scalable

2019-01-20 Thread Hadoop QA (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6929?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16747530#comment-16747530 ] Hadoop QA commented on YARN-6929: - | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem ||

[jira] [Commented] (YARN-6929) yarn.nodemanager.remote-app-log-dir structure is not scalable

2019-01-20 Thread Prabhu Joseph (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6929?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16747508#comment-16747508 ] Prabhu Joseph commented on YARN-6929: - [~jlowe] Thanks for the detailed review. Apologies for late

[jira] [Commented] (YARN-6929) yarn.nodemanager.remote-app-log-dir structure is not scalable

2018-01-08 Thread Jason Lowe (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6929?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16317251#comment-16317251 ] Jason Lowe commented on YARN-6929: -- Sorry for the delay. Looks like the patch needs to be rebased. How

[jira] [Commented] (YARN-6929) yarn.nodemanager.remote-app-log-dir structure is not scalable

2018-01-07 Thread genericqa (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6929?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16315808#comment-16315808 ] genericqa commented on YARN-6929: - | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem ||

[jira] [Commented] (YARN-6929) yarn.nodemanager.remote-app-log-dir structure is not scalable

2018-01-07 Thread Prabhu Joseph (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6929?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16315795#comment-16315795 ] Prabhu Joseph commented on YARN-6929: - [~jlowe] Can you review this when you get some time. Thanks. >

[jira] [Commented] (YARN-6929) yarn.nodemanager.remote-app-log-dir structure is not scalable

2017-10-27 Thread Prabhu Joseph (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6929?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16222056#comment-16222056 ] Prabhu Joseph commented on YARN-6929: - [~jlowe] [~rohithsharma] Need your help in reviewing this patch.

[jira] [Commented] (YARN-6929) yarn.nodemanager.remote-app-log-dir structure is not scalable

2017-10-27 Thread Hadoop QA (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6929?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16222020#comment-16222020 ] Hadoop QA commented on YARN-6929: - | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem ||

[jira] [Commented] (YARN-6929) yarn.nodemanager.remote-app-log-dir structure is not scalable

2017-10-26 Thread Hadoop QA (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6929?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16221665#comment-16221665 ] Hadoop QA commented on YARN-6929: - | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem ||

[jira] [Commented] (YARN-6929) yarn.nodemanager.remote-app-log-dir structure is not scalable

2017-10-25 Thread Hadoop QA (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6929?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16218650#comment-16218650 ] Hadoop QA commented on YARN-6929: - | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem ||

[jira] [Commented] (YARN-6929) yarn.nodemanager.remote-app-log-dir structure is not scalable

2017-10-24 Thread Hadoop QA (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6929?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16217623#comment-16217623 ] Hadoop QA commented on YARN-6929: - | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem ||

[jira] [Commented] (YARN-6929) yarn.nodemanager.remote-app-log-dir structure is not scalable

2017-08-03 Thread Prabhu Joseph (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6929?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16113858#comment-16113858 ] Prabhu Joseph commented on YARN-6929: - Yes clear now. {code} aggregation_log_root / user /

[jira] [Commented] (YARN-6929) yarn.nodemanager.remote-app-log-dir structure is not scalable

2017-08-03 Thread Jason Lowe (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6929?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16113417#comment-16113417 ] Jason Lowe commented on YARN-6929: -- bq. I think Max Bucket Size can be derived from

[jira] [Commented] (YARN-6929) yarn.nodemanager.remote-app-log-dir structure is not scalable

2017-08-03 Thread Prabhu Joseph (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6929?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16113232#comment-16113232 ] Prabhu Joseph commented on YARN-6929: - Yes got it. I think Max Bucket Size can be derived from

[jira] [Commented] (YARN-6929) yarn.nodemanager.remote-app-log-dir structure is not scalable

2017-08-03 Thread Jason Lowe (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6929?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16113198#comment-16113198 ] Jason Lowe commented on YARN-6929: -- bq. Hoping 1 hour will have less than millions of apps I think the

[jira] [Commented] (YARN-6929) yarn.nodemanager.remote-app-log-dir structure is not scalable

2017-08-03 Thread Prabhu Joseph (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6929?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16113176#comment-16113176 ] Prabhu Joseph commented on YARN-6929: - Thanks, missed it. Hash can be generated from

[jira] [Commented] (YARN-6929) yarn.nodemanager.remote-app-log-dir structure is not scalable

2017-08-03 Thread Jason Lowe (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6929?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16112846#comment-16112846 ] Jason Lowe commented on YARN-6929: -- The timestamp in the application ID is the cluster start timestamp

[jira] [Commented] (YARN-6929) yarn.nodemanager.remote-app-log-dir structure is not scalable

2017-08-02 Thread Prabhu Joseph (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6929?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16112071#comment-16112071 ] Prabhu Joseph commented on YARN-6929: - Date can be retrieved from the timestamp present in the

[jira] [Commented] (YARN-6929) yarn.nodemanager.remote-app-log-dir structure is not scalable

2017-08-02 Thread Jason Lowe (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6929?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16111623#comment-16111623 ] Jason Lowe commented on YARN-6929: -- bq. This can be improved with adding date as a subdirectory When a