[jira] [Commented] (HADOOP-17445) Update the year to 2021
[ https://issues.apache.org/jira/browse/HADOOP-17445?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17254483#comment-17254483 ] Sunil G commented on HADOOP-17445: -- +1 I tried to update all target versions for active branches. [~hexiaoqiao] pls check > Update the year to 2021 > --- > > Key: HADOOP-17445 > URL: https://issues.apache.org/jira/browse/HADOOP-17445 > Project: Hadoop Common > Issue Type: Task >Affects Versions: 3.2.2, 3.3.1, 3.4.0, 3.1.5, 3.2.3 >Reporter: Xiaoqiao He >Assignee: Xiaoqiao He >Priority: Major > Attachments: HADOOP-17445.patch > > > Update the year to 2021. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-17445) Update the year to 2021
[ https://issues.apache.org/jira/browse/HADOOP-17445?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sunil G updated HADOOP-17445: - Affects Version/s: 3.2.3 3.1.5 3.4.0 3.3.1 3.2.2 > Update the year to 2021 > --- > > Key: HADOOP-17445 > URL: https://issues.apache.org/jira/browse/HADOOP-17445 > Project: Hadoop Common > Issue Type: Task >Affects Versions: 3.2.2, 3.3.1, 3.4.0, 3.1.5, 3.2.3 >Reporter: Xiaoqiao He >Assignee: Xiaoqiao He >Priority: Major > Attachments: HADOOP-17445.patch > > > Update the year to 2021. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-17329) mvn site commands fails due to MetricsSystemImpl changes
[
https://issues.apache.org/jira/browse/HADOOP-17329?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17222655#comment-17222655
]
Sunil G commented on HADOOP-17329:
--
All changes are pushed to branch-3.2.2(new branch), branch-3.2, branch-3.3,
trunk
> mvn site commands fails due to MetricsSystemImpl changes
>
>
> Key: HADOOP-17329
> URL: https://issues.apache.org/jira/browse/HADOOP-17329
> Project: Hadoop Common
> Issue Type: Bug
>Reporter: Xiaoqiao He
>Assignee: Xiaoqiao He
>Priority: Major
> Fix For: 3.2.2, 3.3.1, 3.4.0
>
> Attachments: HADOOP-17329.001.patch, HADOOP-17329.002.patch
>
>
> When prepare for branch-3.2.2 release, i found there is one issue while
> create release. And it also exist in trunk.
> command line: mvn install site site:stage -DskipTests -DskipShade -Pdist,src
> -Preleasedocs,docs
> failed log show as the following:
> {quote}[ERROR] Failed to execute goal
> org.apache.maven.plugins:maven-site-plugin:3.6:site (default-site) on project
> hadoop-common: failed to get report for
> org.apache.maven.plugins:maven-dependency-plugin: Failed to execute goal
> org.apache.maven.plugins:maven-compiler-plugin:3.1:compile (default-compile)
> on project hadoop-common: Compilation failure
> [ERROR]
> /Users/hexiaoqiao/Source/hadoop-common/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/metrics2/impl/MetricsSystemImpl.java:[298,5]
> method register(java.lang.String,java.lang.String,T) is already defined
> in class org.apache.hadoop.metrics2.impl.MetricsSystemImpl{quote}
> I am not sure why source code of class MetricsSystemImpl will be changed
> while building, I try to revert HADOOP-17081 everything seems OK.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-17329) mvn site commands fails due to MetricsSystemImpl changes
[
https://issues.apache.org/jira/browse/HADOOP-17329?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sunil G updated HADOOP-17329:
-
Fix Version/s: 3.4.0
3.3.1
3.2.2
Hadoop Flags: Reviewed
Resolution: Fixed
Status: Resolved (was: Patch Available)
> mvn site commands fails due to MetricsSystemImpl changes
>
>
> Key: HADOOP-17329
> URL: https://issues.apache.org/jira/browse/HADOOP-17329
> Project: Hadoop Common
> Issue Type: Bug
>Reporter: Xiaoqiao He
>Assignee: Xiaoqiao He
>Priority: Major
> Fix For: 3.2.2, 3.3.1, 3.4.0
>
> Attachments: HADOOP-17329.001.patch, HADOOP-17329.002.patch
>
>
> When prepare for branch-3.2.2 release, i found there is one issue while
> create release. And it also exist in trunk.
> command line: mvn install site site:stage -DskipTests -DskipShade -Pdist,src
> -Preleasedocs,docs
> failed log show as the following:
> {quote}[ERROR] Failed to execute goal
> org.apache.maven.plugins:maven-site-plugin:3.6:site (default-site) on project
> hadoop-common: failed to get report for
> org.apache.maven.plugins:maven-dependency-plugin: Failed to execute goal
> org.apache.maven.plugins:maven-compiler-plugin:3.1:compile (default-compile)
> on project hadoop-common: Compilation failure
> [ERROR]
> /Users/hexiaoqiao/Source/hadoop-common/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/metrics2/impl/MetricsSystemImpl.java:[298,5]
> method register(java.lang.String,java.lang.String,T) is already defined
> in class org.apache.hadoop.metrics2.impl.MetricsSystemImpl{quote}
> I am not sure why source code of class MetricsSystemImpl will be changed
> while building, I try to revert HADOOP-17081 everything seems OK.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-17329) mvn site commands fails due to MetricsSystemImpl changes
[
https://issues.apache.org/jira/browse/HADOOP-17329?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sunil G updated HADOOP-17329:
-
Summary: mvn site commands fails due to MetricsSystemImpl changes (was:
mvn site fails due to MetricsSystemImpl class)
> mvn site commands fails due to MetricsSystemImpl changes
>
>
> Key: HADOOP-17329
> URL: https://issues.apache.org/jira/browse/HADOOP-17329
> Project: Hadoop Common
> Issue Type: Bug
>Reporter: Xiaoqiao He
>Assignee: Xiaoqiao He
>Priority: Major
> Attachments: HADOOP-17329.001.patch, HADOOP-17329.002.patch
>
>
> When prepare for branch-3.2.2 release, i found there is one issue while
> create release. And it also exist in trunk.
> command line: mvn install site site:stage -DskipTests -DskipShade -Pdist,src
> -Preleasedocs,docs
> failed log show as the following:
> {quote}[ERROR] Failed to execute goal
> org.apache.maven.plugins:maven-site-plugin:3.6:site (default-site) on project
> hadoop-common: failed to get report for
> org.apache.maven.plugins:maven-dependency-plugin: Failed to execute goal
> org.apache.maven.plugins:maven-compiler-plugin:3.1:compile (default-compile)
> on project hadoop-common: Compilation failure
> [ERROR]
> /Users/hexiaoqiao/Source/hadoop-common/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/metrics2/impl/MetricsSystemImpl.java:[298,5]
> method register(java.lang.String,java.lang.String,T) is already defined
> in class org.apache.hadoop.metrics2.impl.MetricsSystemImpl{quote}
> I am not sure why source code of class MetricsSystemImpl will be changed
> while building, I try to revert HADOOP-17081 everything seems OK.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-17329) mvn site fails due to MetricsSystemImpl class
[
https://issues.apache.org/jira/browse/HADOOP-17329?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17221541#comment-17221541
]
Sunil G commented on HADOOP-17329:
--
+1
Committing shortly. Thanks [~hexiaoqiao]
> mvn site fails due to MetricsSystemImpl class
> -
>
> Key: HADOOP-17329
> URL: https://issues.apache.org/jira/browse/HADOOP-17329
> Project: Hadoop Common
> Issue Type: Bug
>Reporter: Xiaoqiao He
>Assignee: Xiaoqiao He
>Priority: Major
> Attachments: HADOOP-17329.001.patch, HADOOP-17329.002.patch
>
>
> When prepare for branch-3.2.2 release, i found there is one issue while
> create release. And it also exist in trunk.
> command line: mvn install site site:stage -DskipTests -DskipShade -Pdist,src
> -Preleasedocs,docs
> failed log show as the following:
> {quote}[ERROR] Failed to execute goal
> org.apache.maven.plugins:maven-site-plugin:3.6:site (default-site) on project
> hadoop-common: failed to get report for
> org.apache.maven.plugins:maven-dependency-plugin: Failed to execute goal
> org.apache.maven.plugins:maven-compiler-plugin:3.1:compile (default-compile)
> on project hadoop-common: Compilation failure
> [ERROR]
> /Users/hexiaoqiao/Source/hadoop-common/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/metrics2/impl/MetricsSystemImpl.java:[298,5]
> method register(java.lang.String,java.lang.String,T) is already defined
> in class org.apache.hadoop.metrics2.impl.MetricsSystemImpl{quote}
> I am not sure why source code of class MetricsSystemImpl will be changed
> while building, I try to revert HADOOP-17081 everything seems OK.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-17329) mvn site fails due to MetricsSystemImpl class
[
https://issues.apache.org/jira/browse/HADOOP-17329?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17221166#comment-17221166
]
Sunil G commented on HADOOP-17329:
--
Also you have removed a code snippet. is that safe to remove and needed to be
removed for this issue?
> mvn site fails due to MetricsSystemImpl class
> -
>
> Key: HADOOP-17329
> URL: https://issues.apache.org/jira/browse/HADOOP-17329
> Project: Hadoop Common
> Issue Type: Bug
>Reporter: Xiaoqiao He
>Assignee: Xiaoqiao He
>Priority: Major
> Attachments: HADOOP-17329.001.patch
>
>
> When prepare for branch-3.2.2 release, i found there is one issue while
> create release. And it also exist in trunk.
> command line: mvn install site site:stage -DskipTests -DskipShade -Pdist,src
> -Preleasedocs,docs
> failed log show as the following:
> {quote}[ERROR] Failed to execute goal
> org.apache.maven.plugins:maven-site-plugin:3.6:site (default-site) on project
> hadoop-common: failed to get report for
> org.apache.maven.plugins:maven-dependency-plugin: Failed to execute goal
> org.apache.maven.plugins:maven-compiler-plugin:3.1:compile (default-compile)
> on project hadoop-common: Compilation failure
> [ERROR]
> /Users/hexiaoqiao/Source/hadoop-common/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/metrics2/impl/MetricsSystemImpl.java:[298,5]
> method register(java.lang.String,java.lang.String,T) is already defined
> in class org.apache.hadoop.metrics2.impl.MetricsSystemImpl{quote}
> I am not sure why source code of class MetricsSystemImpl will be changed
> while building, I try to revert HADOOP-17081 everything seems OK.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-17329) mvn site fails due to MetricsSystemImpl class
[
https://issues.apache.org/jira/browse/HADOOP-17329?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17221164#comment-17221164
]
Sunil G commented on HADOOP-17329:
--
[~hexiaoqiao] cud u pls attach a new path after doing "The patch has 6 line(s)
that end in blanks. Use git apply --blanks=fix <>. "
> mvn site fails due to MetricsSystemImpl class
> -
>
> Key: HADOOP-17329
> URL: https://issues.apache.org/jira/browse/HADOOP-17329
> Project: Hadoop Common
> Issue Type: Bug
>Reporter: Xiaoqiao He
>Assignee: Xiaoqiao He
>Priority: Major
> Attachments: HADOOP-17329.001.patch
>
>
> When prepare for branch-3.2.2 release, i found there is one issue while
> create release. And it also exist in trunk.
> command line: mvn install site site:stage -DskipTests -DskipShade -Pdist,src
> -Preleasedocs,docs
> failed log show as the following:
> {quote}[ERROR] Failed to execute goal
> org.apache.maven.plugins:maven-site-plugin:3.6:site (default-site) on project
> hadoop-common: failed to get report for
> org.apache.maven.plugins:maven-dependency-plugin: Failed to execute goal
> org.apache.maven.plugins:maven-compiler-plugin:3.1:compile (default-compile)
> on project hadoop-common: Compilation failure
> [ERROR]
> /Users/hexiaoqiao/Source/hadoop-common/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/metrics2/impl/MetricsSystemImpl.java:[298,5]
> method register(java.lang.String,java.lang.String,T) is already defined
> in class org.apache.hadoop.metrics2.impl.MetricsSystemImpl{quote}
> I am not sure why source code of class MetricsSystemImpl will be changed
> while building, I try to revert HADOOP-17081 everything seems OK.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-16982) Update Netty to 4.1.48.Final
[ https://issues.apache.org/jira/browse/HADOOP-16982?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sunil G updated HADOOP-16982: - Priority: Blocker (was: Major) > Update Netty to 4.1.48.Final > > > Key: HADOOP-16982 > URL: https://issues.apache.org/jira/browse/HADOOP-16982 > Project: Hadoop Common > Issue Type: Task >Affects Versions: 3.3.0 >Reporter: Wei-Chiu Chuang >Priority: Blocker > > We are currently on Netty 4.1.45.Final. We should update to the latest > 4.1.48.Final -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-16982) Update Netty to 4.1.48.Final
[ https://issues.apache.org/jira/browse/HADOOP-16982?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sunil G updated HADOOP-16982: - Target Version/s: 3.3.0 > Update Netty to 4.1.48.Final > > > Key: HADOOP-16982 > URL: https://issues.apache.org/jira/browse/HADOOP-16982 > Project: Hadoop Common > Issue Type: Task >Affects Versions: 3.3.0 >Reporter: Wei-Chiu Chuang >Priority: Blocker > Fix For: 3.3.0 > > > We are currently on Netty 4.1.45.Final. We should update to the latest > 4.1.48.Final -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-16982) Update Netty to 4.1.48.Final
[ https://issues.apache.org/jira/browse/HADOOP-16982?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sunil G updated HADOOP-16982: - Fix Version/s: 3.3.0 > Update Netty to 4.1.48.Final > > > Key: HADOOP-16982 > URL: https://issues.apache.org/jira/browse/HADOOP-16982 > Project: Hadoop Common > Issue Type: Task >Affects Versions: 3.3.0 >Reporter: Wei-Chiu Chuang >Priority: Blocker > Fix For: 3.3.0 > > > We are currently on Netty 4.1.45.Final. We should update to the latest > 4.1.48.Final -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-16818) ABFS: Combine append+flush calls for blockblob & appendblob
[ https://issues.apache.org/jira/browse/HADOOP-16818?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17065907#comment-17065907 ] Sunil G commented on HADOOP-16818: -- [~ayushtkn] [~ishaniahuja]build is broken. We may need to revert this. [~brahma] a > ABFS: Combine append+flush calls for blockblob & appendblob > > > Key: HADOOP-16818 > URL: https://issues.apache.org/jira/browse/HADOOP-16818 > Project: Hadoop Common > Issue Type: Sub-task > Components: fs/azure >Affects Versions: 3.3.0 >Reporter: Bilahari T H >Assignee: Ishani >Priority: Minor > Fix For: 3.4.0 > > > Combine append+flush calls for blockblob & appendblob -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-15270) yarn rmadmin -getGroups returns group from which the user has been removed
[
https://issues.apache.org/jira/browse/HADOOP-15270?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16409366#comment-16409366
]
Sunil G commented on HADOOP-15270:
--
Changes to this issue is is YARN. converting as a YARN ticket.
> yarn rmadmin -getGroups returns group from which the user has been removed
> --
>
> Key: HADOOP-15270
> URL: https://issues.apache.org/jira/browse/HADOOP-15270
> Project: Hadoop Common
> Issue Type: Bug
>Reporter: Sumana Sathish
>Assignee: Sunil G
>Priority: Critical
>
> {code:title= adding group hrt_yarn_rmadmin_test}
> sudo su - -c "groupadd hrt_yarn_rmadmin_test" root
> {code}
> {Code:title=adding user hrt_yarn_rmadmin_test to group hrt_yarn_rmadmin_test}
> sudo su - -c "useradd hrt_yarn_rmadmin_test -g hrt_yarn_rmadmin_test" root
> {Code}
> {Code:title= adding group hrt_yarn_rmadmin_test_group2 }
> sudo su - -c "groupadd hrt_yarn_rmadmin_test_group2" root
> {Code}
> {Code:title=adding user hrt_yarn_rmadmin_test to group
> hrt_yarn_rmadmin_test_group2}
> sudo su - -c "usermod -a -G hrt_yarn_rmadmin_test_group2
> hrt_yarn_rmadmin_test" root
> {Code}
> Refresh and getGroups
> {code}
> yarn rmadmin -refreshUserToGroupsMappings
> /usr/hdp/current/hadoop-yarn-client/bin/yarn rmadmin -getGroups
> hrt_yarn_rmadmin_test
> hrt_yarn_rmadmin_test : hrt_yarn_rmadmin_test hrt_yarn_rmadmin_test_group2
> {code}
> Delete group hrt_yarn_rmadmin_test_group2 from user hrt_yarn_rmadmin_test
> and refresh and do getGroups.
> We can still see group hrt_yarn_rmadmin_test_group2
> {code}
> sudo su - -c "gpasswd -d hrt_yarn_rmadmin_test hrt_yarn_rmadmin_test_group2"
> root
> {code}
> Removing user hrt_yarn_rmadmin_test from group hrt_yarn_rmadmin_test_group2
> {code}
> bash-4.2$ /usr/hdp/current/hadoop-yarn-client/bin/yarn rmadmin
> -refreshUserToGroupsMappings
> /usr/hdp/current/hadoop-yarn-client/bin/yarn rmadmin -getGroups
> hrt_yarn_rmadmin_test
> hrt_yarn_rmadmin_test : hrt_yarn_rmadmin_test hrt_yarn_rmadmin_test_group2
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Assigned] (HADOOP-15270) yarn rmadmin -getGroups returns group from which the user has been removed
[
https://issues.apache.org/jira/browse/HADOOP-15270?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sunil G reassigned HADOOP-15270:
Assignee: Sunil G
> yarn rmadmin -getGroups returns group from which the user has been removed
> --
>
> Key: HADOOP-15270
> URL: https://issues.apache.org/jira/browse/HADOOP-15270
> Project: Hadoop Common
> Issue Type: Bug
>Reporter: Sumana Sathish
>Assignee: Sunil G
>Priority: Critical
>
> {code:title= adding group hrt_yarn_rmadmin_test}
> sudo su - -c "groupadd hrt_yarn_rmadmin_test" root
> {code}
> {Code:title=adding user hrt_yarn_rmadmin_test to group hrt_yarn_rmadmin_test}
> sudo su - -c "useradd hrt_yarn_rmadmin_test -g hrt_yarn_rmadmin_test" root
> {Code}
> {Code:title= adding group hrt_yarn_rmadmin_test_group2 }
> sudo su - -c "groupadd hrt_yarn_rmadmin_test_group2" root
> {Code}
> {Code:title=adding user hrt_yarn_rmadmin_test to group
> hrt_yarn_rmadmin_test_group2}
> sudo su - -c "usermod -a -G hrt_yarn_rmadmin_test_group2
> hrt_yarn_rmadmin_test" root
> {Code}
> Refresh and getGroups
> {code}
> yarn rmadmin -refreshUserToGroupsMappings
> /usr/hdp/current/hadoop-yarn-client/bin/yarn rmadmin -getGroups
> hrt_yarn_rmadmin_test
> hrt_yarn_rmadmin_test : hrt_yarn_rmadmin_test hrt_yarn_rmadmin_test_group2
> {code}
> Delete group hrt_yarn_rmadmin_test_group2 from user hrt_yarn_rmadmin_test
> and refresh and do getGroups.
> We can still see group hrt_yarn_rmadmin_test_group2
> {code}
> sudo su - -c "gpasswd -d hrt_yarn_rmadmin_test hrt_yarn_rmadmin_test_group2"
> root
> {code}
> Removing user hrt_yarn_rmadmin_test from group hrt_yarn_rmadmin_test_group2
> {code}
> bash-4.2$ /usr/hdp/current/hadoop-yarn-client/bin/yarn rmadmin
> -refreshUserToGroupsMappings
> /usr/hdp/current/hadoop-yarn-client/bin/yarn rmadmin -getGroups
> hrt_yarn_rmadmin_test
> hrt_yarn_rmadmin_test : hrt_yarn_rmadmin_test hrt_yarn_rmadmin_test_group2
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Created] (HADOOP-15116) NPE in ResourceManager when ZooKeeper goes down temporary (HA Mode)
Sunil G created HADOOP-15116:
Summary: NPE in ResourceManager when ZooKeeper goes down temporary
(HA Mode)
Key: HADOOP-15116
URL: https://issues.apache.org/jira/browse/HADOOP-15116
Project: Hadoop Common
Issue Type: Bug
Components: ha
Affects Versions: 3.0.0-beta1
Reporter: Sunil G
In an HA enabled cluster (3.0), we found that RM is failing to start with an
NPE from ActiveStandbyElector. Zookeeper was down at this time, hence client
retries were coming for a while
{code}
2017-12-13 18:21:22,460 INFO org.apache.zookeeper.ClientCnxn: Opening socket
connection to server 127.0.0.1/127.0.0.1:2181. Will not attempt to authenticate
using SASL (unknown error)
2017-12-13 18:21:22,544 INFO org.apache.hadoop.service.AbstractService: Service
org.apache.hadoop.yarn.server.resourcemanager.ActiveStandbyElectorBasedElectorService
failed in state INITED; cause: java.lang.NullPointerException
java.lang.NullPointerException
at
org.apache.hadoop.ha.ActiveStandbyElector$3.run(ActiveStandbyElector.java:1039)
at
org.apache.hadoop.ha.ActiveStandbyElector$3.run(ActiveStandbyElector.java:1036)
at
org.apache.hadoop.ha.ActiveStandbyElector.zkDoWithRetries(ActiveStandbyElector.java:1101)
at
org.apache.hadoop.ha.ActiveStandbyElector.zkDoWithRetries(ActiveStandbyElector.java:1093)
at
org.apache.hadoop.ha.ActiveStandbyElector.createWithRetries(ActiveStandbyElector.java:1036)
at
org.apache.hadoop.ha.ActiveStandbyElector.ensureParentZNode(ActiveStandbyElector.java:347)
at
org.apache.hadoop.yarn.server.resourcemanager.ActiveStandbyElectorBasedElectorService.serviceInit(ActiveStandbyElectorBasedElectorService.java:110)
at
org.apache.hadoop.service.AbstractService.init(AbstractService.java:164)
at
org.apache.hadoop.service.CompositeService.serviceInit(CompositeService.java:108)
at
org.apache.hadoop.yarn.server.resourcemanager.ResourceManager.serviceInit(ResourceManager.java:326)
at
org.apache.hadoop.service.AbstractService.init(AbstractService.java:164)
at
org.apache.hadoop.yarn.server.resourcemanager.ResourceManager.main(ResourceManager.java:1420)
2017-12-13 18:21:22,545 INFO org.apache.hadoop.ha.ActiveStandbyElector:
Yielding from election
2017-12-13 18:21:22,545 INFO org.apache.hadoop.service.AbstractService: Service
ResourceManager failed in state INITED; cause: java.lang.NullPointerException
{code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-14966) Handle JDK-8071638 for hadoop-common
[ https://issues.apache.org/jira/browse/HADOOP-14966?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16214688#comment-16214688 ] Sunil G commented on HADOOP-14966: -- Yes. 2.9 release stuffs are also ongoing. Please mark it to correct versions > Handle JDK-8071638 for hadoop-common > > > Key: HADOOP-14966 > URL: https://issues.apache.org/jira/browse/HADOOP-14966 > Project: Hadoop Common > Issue Type: Bug >Affects Versions: 2.8.0, 3.0.0-alpha1 >Reporter: Bibin A Chundatt >Assignee: Bibin A Chundatt >Priority: Blocker > Attachments: HADOOP-14966.001.patch > > > Impact modules > -- YARN nodemanger cache clean up > -- Mapreduce Log/History cleaner > Will add jira in YARN & MAPREDUCE to track the same -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-14966) Handle JDK-8071638 for hadoop-common
[ https://issues.apache.org/jira/browse/HADOOP-14966?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16214665#comment-16214665 ] Sunil G commented on HADOOP-14966: -- Thanks [~bibinchundatt]. Fix seems straight forward as per JDK-8071638. For 2.8.2, RC is done and voting is in progress. I can see that this issue is marked for 2.8.0. Will this be blocker for 2.8.2 ? cc/ [~djp] and [~rohithsharma] > Handle JDK-8071638 for hadoop-common > > > Key: HADOOP-14966 > URL: https://issues.apache.org/jira/browse/HADOOP-14966 > Project: Hadoop Common > Issue Type: Bug >Affects Versions: 2.8.0, 3.0.0-alpha1 >Reporter: Bibin A Chundatt >Assignee: Bibin A Chundatt >Priority: Blocker > Attachments: HADOOP-14966.001.patch > > > Impact modules > -- YARN nodemanger cache clean up > -- Mapreduce Log/History cleaner > Will add jira in YARN & MAPREDUCE to track the same -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-14658) branch-2 compilation is broken in hadoop-azure
[
https://issues.apache.org/jira/browse/HADOOP-14658?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16086742#comment-16086742
]
Sunil G commented on HADOOP-14658:
--
Thank you very much [~ste...@apache.org] for review and commit.
Yes, I will ensure a full test suite run in such scenarios from next time.
Thank you for pointing out.
> branch-2 compilation is broken in hadoop-azure
> --
>
> Key: HADOOP-14658
> URL: https://issues.apache.org/jira/browse/HADOOP-14658
> Project: Hadoop Common
> Issue Type: Bug
> Components: build, fs/azure
>Affects Versions: 2.9.0
> Environment: java 7
>Reporter: Sunil G
>Assignee: Sunil G
>Priority: Blocker
> Fix For: 2.9.0
>
> Attachments: HADOOP-14658-branch-2.001.patch,
> YARN-6280-branch-2.001.patch
>
>
> Compilation failure.
> [link|https://builds.apache.org/job/PreCommit-YARN-Build/16414/artifact/patchprocess/branch-mvninstall-root.txt]
> {noformat}
> [ERROR]
> /home/sunilg/hadoop/repo/branch-2/hadoop/hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azure/WasbRemoteCallHelper.java:[194,19]
> cannot find symbol
> [ERROR] symbol: method join(java.lang.String,java.lang.String[])
> [ERROR] location: class java.lang.String
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-14658) branch-2 compilation is broken in hadoop-azure
[
https://issues.apache.org/jira/browse/HADOOP-14658?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16085813#comment-16085813
]
Sunil G commented on HADOOP-14658:
--
Thank you.
I ran against WASB/Japan. Here is my local test results.
{noformat}
Running org.apache.hadoop.fs.azure.TestWasbRemoteCallHelper
Tests run: 10, Failures: 0, Errors: 0, Skipped: 10, Time elapsed: 3.128 sec -
in org.apache.hadoop.fs.azure.TestWasbRemoteCallHelper
Results :
Tests run: 10, Failures: 0, Errors: 0, Skipped: 10
{noformat}
> branch-2 compilation is broken in hadoop-azure
> --
>
> Key: HADOOP-14658
> URL: https://issues.apache.org/jira/browse/HADOOP-14658
> Project: Hadoop Common
> Issue Type: Bug
> Components: build, fs/azure
>Affects Versions: 2.9.0
> Environment: java 7
>Reporter: Sunil G
>Assignee: Sunil G
>Priority: Blocker
> Attachments: HADOOP-14658-branch-2.001.patch,
> YARN-6280-branch-2.001.patch
>
>
> Compilation failure.
> [link|https://builds.apache.org/job/PreCommit-YARN-Build/16414/artifact/patchprocess/branch-mvninstall-root.txt]
> {noformat}
> [ERROR]
> /home/sunilg/hadoop/repo/branch-2/hadoop/hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azure/WasbRemoteCallHelper.java:[194,19]
> cannot find symbol
> [ERROR] symbol: method join(java.lang.String,java.lang.String[])
> [ERROR] location: class java.lang.String
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-14658) branch-2 compilation is broken in hadoop-azure
[
https://issues.apache.org/jira/browse/HADOOP-14658?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sunil G updated HADOOP-14658:
-
Attachment: HADOOP-14658-branch-2.001.patch
Thats my bad :( Extremely sorry.
Accidentally shared a wrong patch. Attaching correct one.
> branch-2 compilation is broken in hadoop-azure
> --
>
> Key: HADOOP-14658
> URL: https://issues.apache.org/jira/browse/HADOOP-14658
> Project: Hadoop Common
> Issue Type: Bug
> Components: build, fs/azure
>Affects Versions: 2.9.0
> Environment: java 7
>Reporter: Sunil G
>Assignee: Sunil G
>Priority: Blocker
> Attachments: HADOOP-14658-branch-2.001.patch,
> YARN-6280-branch-2.001.patch
>
>
> Compilation failure.
> [link|https://builds.apache.org/job/PreCommit-YARN-Build/16414/artifact/patchprocess/branch-mvninstall-root.txt]
> {noformat}
> [ERROR]
> /home/sunilg/hadoop/repo/branch-2/hadoop/hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azure/WasbRemoteCallHelper.java:[194,19]
> cannot find symbol
> [ERROR] symbol: method join(java.lang.String,java.lang.String[])
> [ERROR] location: class java.lang.String
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-14658) branch-2 compilation is broken in hadoop-azure
[
https://issues.apache.org/jira/browse/HADOOP-14658?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sunil G updated HADOOP-14658:
-
Status: Patch Available (was: Open)
> branch-2 compilation is broken in hadoop-azure
> --
>
> Key: HADOOP-14658
> URL: https://issues.apache.org/jira/browse/HADOOP-14658
> Project: Hadoop Common
> Issue Type: Bug
> Components: build, fs/azure
>Affects Versions: 2.9.0
> Environment: java 7
>Reporter: Sunil G
>Priority: Blocker
> Attachments: YARN-6280-branch-2.001.patch
>
>
> Compilation failure.
> [link|https://builds.apache.org/job/PreCommit-YARN-Build/16414/artifact/patchprocess/branch-mvninstall-root.txt]
> {noformat}
> [ERROR]
> /home/sunilg/hadoop/repo/branch-2/hadoop/hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azure/WasbRemoteCallHelper.java:[194,19]
> cannot find symbol
> [ERROR] symbol: method join(java.lang.String,java.lang.String[])
> [ERROR] location: class java.lang.String
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-14658) branch-2 compilation is broken in hadoop-azure
[
https://issues.apache.org/jira/browse/HADOOP-14658?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sunil G updated HADOOP-14658:
-
Attachment: YARN-6280-branch-2.001.patch
Attaching branch-2 patch. I tried to use StringUtils.join from haddop common
util package.
> branch-2 compilation is broken in hadoop-azure
> --
>
> Key: HADOOP-14658
> URL: https://issues.apache.org/jira/browse/HADOOP-14658
> Project: Hadoop Common
> Issue Type: Bug
> Components: build, fs/azure
>Affects Versions: 2.9.0
> Environment: java 7
>Reporter: Sunil G
>Priority: Blocker
> Attachments: YARN-6280-branch-2.001.patch
>
>
> Compilation failure.
> [link|https://builds.apache.org/job/PreCommit-YARN-Build/16414/artifact/patchprocess/branch-mvninstall-root.txt]
> {noformat}
> [ERROR]
> /home/sunilg/hadoop/repo/branch-2/hadoop/hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azure/WasbRemoteCallHelper.java:[194,19]
> cannot find symbol
> [ERROR] symbol: method join(java.lang.String,java.lang.String[])
> [ERROR] location: class java.lang.String
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-14658) branch-2 compilation is broken in hadoop-azure
[
https://issues.apache.org/jira/browse/HADOOP-14658?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16085619#comment-16085619
]
Sunil G commented on HADOOP-14658:
--
Yes. Java 8 seems fine.
> branch-2 compilation is broken in hadoop-azure
> --
>
> Key: HADOOP-14658
> URL: https://issues.apache.org/jira/browse/HADOOP-14658
> Project: Hadoop Common
> Issue Type: Bug
> Components: build, fs/azure
>Affects Versions: 2.9.0
> Environment: java 7
>Reporter: Sunil G
>Priority: Blocker
>
> Compilation failure.
> [link|https://builds.apache.org/job/PreCommit-YARN-Build/16414/artifact/patchprocess/branch-mvninstall-root.txt]
> {noformat}
> [ERROR]
> /home/sunilg/hadoop/repo/branch-2/hadoop/hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azure/WasbRemoteCallHelper.java:[194,19]
> cannot find symbol
> [ERROR] symbol: method join(java.lang.String,java.lang.String[])
> [ERROR] location: class java.lang.String
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-14658) branch-2 compilation is broken in hadoop-azure
[
https://issues.apache.org/jira/browse/HADOOP-14658?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sunil G updated HADOOP-14658:
-
Priority: Blocker (was: Major)
> branch-2 compilation is broken in hadoop-azure
> --
>
> Key: HADOOP-14658
> URL: https://issues.apache.org/jira/browse/HADOOP-14658
> Project: Hadoop Common
> Issue Type: Bug
>Affects Versions: 2.9.0
>Reporter: Sunil G
>Priority: Blocker
>
> Compilation failure.
> [link|https://builds.apache.org/job/PreCommit-YARN-Build/16414/artifact/patchprocess/branch-mvninstall-root.txt]
> {noformat}
> [ERROR]
> /home/sunilg/hadoop/repo/branch-2/hadoop/hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azure/WasbRemoteCallHelper.java:[194,19]
> cannot find symbol
> [ERROR] symbol: method join(java.lang.String,java.lang.String[])
> [ERROR] location: class java.lang.String
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Created] (HADOOP-14658) branch-2 compilation is broken in hadoop-azure
Sunil G created HADOOP-14658:
Summary: branch-2 compilation is broken in hadoop-azure
Key: HADOOP-14658
URL: https://issues.apache.org/jira/browse/HADOOP-14658
Project: Hadoop Common
Issue Type: Bug
Affects Versions: 2.9.0
Reporter: Sunil G
Compilation failure.
[link|https://builds.apache.org/job/PreCommit-YARN-Build/16414/artifact/patchprocess/branch-mvninstall-root.txt]
{noformat}
[ERROR]
/home/sunilg/hadoop/repo/branch-2/hadoop/hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azure/WasbRemoteCallHelper.java:[194,19]
cannot find symbol
[ERROR] symbol: method join(java.lang.String,java.lang.String[])
[ERROR] location: class java.lang.String
{noformat}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-14285) Update minimum version of Maven from 3.0 to 3.3
[ https://issues.apache.org/jira/browse/HADOOP-14285?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15960204#comment-15960204 ] Sunil G commented on HADOOP-14285: -- +1 from me as well. I could commit this if there are no objections later today. > Update minimum version of Maven from 3.0 to 3.3 > --- > > Key: HADOOP-14285 > URL: https://issues.apache.org/jira/browse/HADOOP-14285 > Project: Hadoop Common > Issue Type: Improvement >Reporter: Akira Ajisaka >Assignee: Akira Ajisaka > Attachments: HADOOP-14285.01.patch > > > YARN-6421 requires Apache Maven 3.1+ -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-12321) Make JvmPauseMonitor an AbstractService
[
https://issues.apache.org/jira/browse/HADOOP-12321?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15522851#comment-15522851
]
Sunil G commented on HADOOP-12321:
--
HI [~brahmareddy]
As discussed
[here|https://issues.apache.org/jira/browse/MAPREDUCE-6462?focusedCommentId=14743205=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-14743205]
, there were no added advantage to count the number of children for a service.
hence it was removed. Any issues?
> Make JvmPauseMonitor an AbstractService
> ---
>
> Key: HADOOP-12321
> URL: https://issues.apache.org/jira/browse/HADOOP-12321
> Project: Hadoop Common
> Issue Type: New Feature
>Affects Versions: 2.8.0
>Reporter: Steve Loughran
>Assignee: Sunil G
> Fix For: 2.9.0, 3.0.0-alpha1
>
> Attachments: 0001-HADOOP-12321.patch, 0002-HADOOP-12321.patch,
> 0004-HADOOP-12321.patch, HADOOP-12321-003.patch,
> HADOOP-12321-005-aggregated.patch
>
> Original Estimate: 1h
> Remaining Estimate: 1h
>
> The new JVM pause monitor has been written with its own start/stop lifecycle
> which has already proven brittle to both ordering of operations and, even
> after HADOOP-12313, is not thread safe (both start and stop are potentially
> re-entrant).
> It also requires every class which supports the monitor to add another field
> and perform the lifecycle operations in its own lifecycle, which, for all
> Yarn services, is the YARN app lifecycle (as implemented in Hadoop common)
> Making the monitor a subclass of {{AbstractService}} and moving the
> init/start & stop operations in {{serviceInit()}}, {{serviceStart()}} &
> {{serviceStop()}} methods will fix the concurrency and state model issues,
> and make it trivial to add as a child to any YARN service which subclasses
> {{CompositeService}} (most the NM and RM apps) will be able to hook up the
> monitor simply by creating one in the ctor and adding it as a child.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-9613) [JDK8] Update jersey version to latest 1.x release
[
https://issues.apache.org/jira/browse/HADOOP-9613?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15291543#comment-15291543
]
Sunil G commented on HADOOP-9613:
-
Adding to the point mentioned by [~leftnoteasy]. "the only issue I can see is,
when no application submitted/running on the cluster, applications page cannot
be loaded. ", I am also seeing issues when empty tables or entries are present
in few pages of yarn-ui (empty application page, empty container/app page for a
node etc). Once some app or containers are available, we can navigate and
access those pages. Attaching error what I am getting from debug console.
This needs to be investigated (JERSEY-1168). I ll also try to analyze the
possible reasons, if some one already knows, pls share.
{noformat}
vendor-08d8702….js:11 TypeError: Cannot read property 'map' of undefined
at e.default.t.default.JSONAPISerializer.extend.normalizeArrayResponse
(yarn-ui-09e4e3e….js:3)
vendor-08d8702….js:11 TypeError: Cannot read property '0' of undefined
at e.default.t.default.Route.extend.actions.error (yarn-ui-09e4e3e….js:2)
{noformat}
> [JDK8] Update jersey version to latest 1.x release
> --
>
> Key: HADOOP-9613
> URL: https://issues.apache.org/jira/browse/HADOOP-9613
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: build
>Affects Versions: 2.4.0, 3.0.0-alpha1
>Reporter: Timothy St. Clair
>Assignee: Tsuyoshi Ozawa
> Labels: UpgradeKeyLibrary, maven
> Attachments: HADOOP-2.2.0-9613.patch,
> HADOOP-9613.004.incompatible.patch, HADOOP-9613.005.incompatible.patch,
> HADOOP-9613.006.incompatible.patch, HADOOP-9613.007.incompatible.patch,
> HADOOP-9613.008.incompatible.patch, HADOOP-9613.009.incompatible.patch,
> HADOOP-9613.010.incompatible.patch, HADOOP-9613.011.incompatible.patch,
> HADOOP-9613.012.incompatible.patch, HADOOP-9613.013.incompatible.patch,
> HADOOP-9613.014.incompatible.patch, HADOOP-9613.014.incompatible.patch,
> HADOOP-9613.015.incompatible.patch, HADOOP-9613.016.incompatible.patch,
> HADOOP-9613.017.incompatible.patch, HADOOP-9613.1.patch, HADOOP-9613.2.patch,
> HADOOP-9613.3.patch, HADOOP-9613.patch
>
>
> Update pom.xml dependencies exposed when running a mvn-rpmbuild against
> system dependencies on Fedora 18.
> The existing version is 1.8 which is quite old.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-12687) SecureUtil#getByName should also try to resolve direct hostname, incase multiple loopback addresses are present in /etc/hosts
[
https://issues.apache.org/jira/browse/HADOOP-12687?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15211322#comment-15211322
]
Sunil G commented on HADOOP-12687:
--
Yes [~jiajia]. These failures are still present. INFRA-11150 is raised to
change YARN precommit build machine hostname so that this issue can be
permanently fixed. Till then, unfortunately we will have this error from YARN
pre-commit build. Committers are aware of this, and will consider the same
accordingly.
> SecureUtil#getByName should also try to resolve direct hostname, incase
> multiple loopback addresses are present in /etc/hosts
> -
>
> Key: HADOOP-12687
> URL: https://issues.apache.org/jira/browse/HADOOP-12687
> Project: Hadoop Common
> Issue Type: Bug
>Reporter: Junping Du
>Assignee: Sunil G
> Labels: security
> Attachments: 0001-YARN-4352.patch, 0002-YARN-4352.patch,
> 0003-HADOOP-12687.patch, 0004-HADOOP-12687.patch
>
>
> From
> https://builds.apache.org/job/PreCommit-YARN-Build/9661/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-client-jdk1.7.0_79.txt,
> we can see the tests in TestYarnClient, TestAMRMClient and TestNMClient get
> timeout which can be reproduced locally.
> When {{/etc/hosts}} has multiple loopback entries,
> {{InetAddress.getByName(null)}} will be returning the first entry present in
> etc/hosts. Hence its possible that machine hostname can be second in list and
> cause {{UnKnownHostException}}.
> Suggesting a direct resolve for such hostname scenarios.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HADOOP-12687) SecureUtil#getByName should also try to resolve direct hostname incase multiple loopback addresses are present in etc/hosts
[
https://issues.apache.org/jira/browse/HADOOP-12687?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sunil G updated HADOOP-12687:
-
Description:
>From
>https://builds.apache.org/job/PreCommit-YARN-Build/9661/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-client-jdk1.7.0_79.txt,
> we can see the tests in TestYarnClient, TestAMRMClient and TestNMClient get
>timeout which can be reproduced locally.
When {{/etc/hosts}} has multiple loopback entries,
was:
>From
>https://builds.apache.org/job/PreCommit-YARN-Build/9661/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-client-jdk1.7.0_79.txt,
> we can see the tests in TestYarnClient, TestAMRMClient and TestNMClient get
>timeout which can be reproduced locally.
> SecureUtil#getByName should also try to resolve direct hostname incase
> multiple loopback addresses are present in etc/hosts
> ---
>
> Key: HADOOP-12687
> URL: https://issues.apache.org/jira/browse/HADOOP-12687
> Project: Hadoop Common
> Issue Type: Bug
>Reporter: Junping Du
>Assignee: Sunil G
> Labels: security
> Attachments: 0001-YARN-4352.patch, 0002-YARN-4352.patch,
> 0003-HADOOP-12687.patch
>
>
> From
> https://builds.apache.org/job/PreCommit-YARN-Build/9661/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-client-jdk1.7.0_79.txt,
> we can see the tests in TestYarnClient, TestAMRMClient and TestNMClient get
> timeout which can be reproduced locally.
> When {{/etc/hosts}} has multiple loopback entries,
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HADOOP-12687) SecureUtil#getByName should also try to resolve direct hostname incase multiple loopback addresses are present in etc/hosts
[ https://issues.apache.org/jira/browse/HADOOP-12687?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sunil G updated HADOOP-12687: - Description: >From >https://builds.apache.org/job/PreCommit-YARN-Build/9661/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-client-jdk1.7.0_79.txt, > we can see the tests in TestYarnClient, TestAMRMClient and TestNMClient get >timeout which can be reproduced locally. was:From https://builds.apache.org/job/PreCommit-YARN-Build/9661/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-client-jdk1.7.0_79.txt, we can see the tests in TestYarnClient, TestAMRMClient and TestNMClient get timeout which can be reproduced locally. > SecureUtil#getByName should also try to resolve direct hostname incase > multiple loopback addresses are present in etc/hosts > --- > > Key: HADOOP-12687 > URL: https://issues.apache.org/jira/browse/HADOOP-12687 > Project: Hadoop Common > Issue Type: Bug >Reporter: Junping Du >Assignee: Sunil G > Labels: security > Attachments: 0001-YARN-4352.patch, 0002-YARN-4352.patch, > 0003-HADOOP-12687.patch > > > From > https://builds.apache.org/job/PreCommit-YARN-Build/9661/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-client-jdk1.7.0_79.txt, > we can see the tests in TestYarnClient, TestAMRMClient and TestNMClient get > timeout which can be reproduced locally. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HADOOP-12687) Timeout for tests in TestYarnClient, TestAMRMClient and TestNMClient
[ https://issues.apache.org/jira/browse/HADOOP-12687?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sunil G updated HADOOP-12687: - Attachment: 0003-HADOOP-12687.patch Yes [~rohithsharma] That change looks fine for me. Updating by changing as per comment. > Timeout for tests in TestYarnClient, TestAMRMClient and TestNMClient > > > Key: HADOOP-12687 > URL: https://issues.apache.org/jira/browse/HADOOP-12687 > Project: Hadoop Common > Issue Type: Bug >Reporter: Junping Du >Assignee: Sunil G > Labels: security > Attachments: 0001-YARN-4352.patch, 0002-YARN-4352.patch, > 0003-HADOOP-12687.patch > > > From > https://builds.apache.org/job/PreCommit-YARN-Build/9661/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-client-jdk1.7.0_79.txt, > we can see the tests in TestYarnClient, TestAMRMClient and TestNMClient get > timeout which can be reproduced locally. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HADOOP-12687) SecureUtil#getByName should also try to resolve direct hostname incase multiple loopback addresses are present in etc/hosts
[ https://issues.apache.org/jira/browse/HADOOP-12687?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sunil G updated HADOOP-12687: - Summary: SecureUtil#getByName should also try to resolve direct hostname incase multiple loopback addresses are present in etc/hosts (was: Timeout for tests in TestYarnClient, TestAMRMClient and TestNMClient) > SecureUtil#getByName should also try to resolve direct hostname incase > multiple loopback addresses are present in etc/hosts > --- > > Key: HADOOP-12687 > URL: https://issues.apache.org/jira/browse/HADOOP-12687 > Project: Hadoop Common > Issue Type: Bug >Reporter: Junping Du >Assignee: Sunil G > Labels: security > Attachments: 0001-YARN-4352.patch, 0002-YARN-4352.patch, > 0003-HADOOP-12687.patch > > > From > https://builds.apache.org/job/PreCommit-YARN-Build/9661/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-client-jdk1.7.0_79.txt, > we can see the tests in TestYarnClient, TestAMRMClient and TestNMClient get > timeout which can be reproduced locally. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HADOOP-12687) SecureUtil#getByName should also try to resolve direct hostname incase multiple loopback addresses are present in etc/hosts
[
https://issues.apache.org/jira/browse/HADOOP-12687?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sunil G updated HADOOP-12687:
-
Description:
>From
>https://builds.apache.org/job/PreCommit-YARN-Build/9661/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-client-jdk1.7.0_79.txt,
> we can see the tests in TestYarnClient, TestAMRMClient and TestNMClient get
>timeout which can be reproduced locally.
When {{/etc/hosts}} has multiple loopback entries,
{{InetAddress.getByName(null)}} will be returning the first entry present in
etc/hosts. Hence its possible that machine hostname can be second in list and
cause
was:
>From
>https://builds.apache.org/job/PreCommit-YARN-Build/9661/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-client-jdk1.7.0_79.txt,
> we can see the tests in TestYarnClient, TestAMRMClient and TestNMClient get
>timeout which can be reproduced locally.
When {{/etc/hosts}} has multiple loopback entries,
> SecureUtil#getByName should also try to resolve direct hostname incase
> multiple loopback addresses are present in etc/hosts
> ---
>
> Key: HADOOP-12687
> URL: https://issues.apache.org/jira/browse/HADOOP-12687
> Project: Hadoop Common
> Issue Type: Bug
>Reporter: Junping Du
>Assignee: Sunil G
> Labels: security
> Attachments: 0001-YARN-4352.patch, 0002-YARN-4352.patch,
> 0003-HADOOP-12687.patch
>
>
> From
> https://builds.apache.org/job/PreCommit-YARN-Build/9661/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-client-jdk1.7.0_79.txt,
> we can see the tests in TestYarnClient, TestAMRMClient and TestNMClient get
> timeout which can be reproduced locally.
> When {{/etc/hosts}} has multiple loopback entries,
> {{InetAddress.getByName(null)}} will be returning the first entry present in
> etc/hosts. Hence its possible that machine hostname can be second in list and
> cause
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HADOOP-12687) SecureUtil#getByName should also try to resolve direct hostname incase multiple loopback addresses are present in etc/hosts
[
https://issues.apache.org/jira/browse/HADOOP-12687?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sunil G updated HADOOP-12687:
-
Description:
>From
>https://builds.apache.org/job/PreCommit-YARN-Build/9661/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-client-jdk1.7.0_79.txt,
> we can see the tests in TestYarnClient, TestAMRMClient and TestNMClient get
>timeout which can be reproduced locally.
When {{/etc/hosts}} has multiple loopback entries,
{{InetAddress.getByName(null)}} will be returning the first entry present in
etc/hosts. Hence its possible that machine hostname can be second in list and
cause {{UnKnownHostException}}.
Suggesting a direct resolve for such hostname scenarios.
was:
>From
>https://builds.apache.org/job/PreCommit-YARN-Build/9661/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-client-jdk1.7.0_79.txt,
> we can see the tests in TestYarnClient, TestAMRMClient and TestNMClient get
>timeout which can be reproduced locally.
When {{/etc/hosts}} has multiple loopback entries,
{{InetAddress.getByName(null)}} will be returning the first entry present in
etc/hosts. Hence its possible that machine hostname can be second in list and
cause
> SecureUtil#getByName should also try to resolve direct hostname incase
> multiple loopback addresses are present in etc/hosts
> ---
>
> Key: HADOOP-12687
> URL: https://issues.apache.org/jira/browse/HADOOP-12687
> Project: Hadoop Common
> Issue Type: Bug
>Reporter: Junping Du
>Assignee: Sunil G
> Labels: security
> Attachments: 0001-YARN-4352.patch, 0002-YARN-4352.patch,
> 0003-HADOOP-12687.patch
>
>
> From
> https://builds.apache.org/job/PreCommit-YARN-Build/9661/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-client-jdk1.7.0_79.txt,
> we can see the tests in TestYarnClient, TestAMRMClient and TestNMClient get
> timeout which can be reproduced locally.
> When {{/etc/hosts}} has multiple loopback entries,
> {{InetAddress.getByName(null)}} will be returning the first entry present in
> etc/hosts. Hence its possible that machine hostname can be second in list and
> cause {{UnKnownHostException}}.
> Suggesting a direct resolve for such hostname scenarios.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HADOOP-12687) SecureUtil#getByName should also try to resolve direct hostname incase multiple loopback addresses are present in etc/hosts
[
https://issues.apache.org/jira/browse/HADOOP-12687?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sunil G updated HADOOP-12687:
-
Attachment: 0004-HADOOP-12687.patch
Reattaching patch to trigger jenkins again.
> SecureUtil#getByName should also try to resolve direct hostname incase
> multiple loopback addresses are present in etc/hosts
> ---
>
> Key: HADOOP-12687
> URL: https://issues.apache.org/jira/browse/HADOOP-12687
> Project: Hadoop Common
> Issue Type: Bug
>Reporter: Junping Du
>Assignee: Sunil G
> Labels: security
> Attachments: 0001-YARN-4352.patch, 0002-YARN-4352.patch,
> 0003-HADOOP-12687.patch, 0004-HADOOP-12687.patch
>
>
> From
> https://builds.apache.org/job/PreCommit-YARN-Build/9661/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-client-jdk1.7.0_79.txt,
> we can see the tests in TestYarnClient, TestAMRMClient and TestNMClient get
> timeout which can be reproduced locally.
> When {{/etc/hosts}} has multiple loopback entries,
> {{InetAddress.getByName(null)}} will be returning the first entry present in
> etc/hosts. Hence its possible that machine hostname can be second in list and
> cause {{UnKnownHostException}}.
> Suggesting a direct resolve for such hostname scenarios.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HADOOP-12687) SecureUtil#getByName should also try to resolve direct hostname, incase multiple loopback addresses are present in /etc/hosts
[
https://issues.apache.org/jira/browse/HADOOP-12687?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15086841#comment-15086841
]
Sunil G commented on HADOOP-12687:
--
Thanks [~rohithsharma] for the review and commit. And thanks [~vinayrpet] for
the review!
> SecureUtil#getByName should also try to resolve direct hostname, incase
> multiple loopback addresses are present in /etc/hosts
> -
>
> Key: HADOOP-12687
> URL: https://issues.apache.org/jira/browse/HADOOP-12687
> Project: Hadoop Common
> Issue Type: Bug
>Reporter: Junping Du
>Assignee: Sunil G
> Labels: security
> Fix For: 2.9.0
>
> Attachments: 0001-YARN-4352.patch, 0002-YARN-4352.patch,
> 0003-HADOOP-12687.patch, 0004-HADOOP-12687.patch
>
>
> From
> https://builds.apache.org/job/PreCommit-YARN-Build/9661/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-client-jdk1.7.0_79.txt,
> we can see the tests in TestYarnClient, TestAMRMClient and TestNMClient get
> timeout which can be reproduced locally.
> When {{/etc/hosts}} has multiple loopback entries,
> {{InetAddress.getByName(null)}} will be returning the first entry present in
> etc/hosts. Hence its possible that machine hostname can be second in list and
> cause {{UnKnownHostException}}.
> Suggesting a direct resolve for such hostname scenarios.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HADOOP-12687) SecureUtil#getByName should also try to resolve direct hostname, incase multiple loopback addresses are present in /etc/hosts
[
https://issues.apache.org/jira/browse/HADOOP-12687?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sunil G updated HADOOP-12687:
-
Summary: SecureUtil#getByName should also try to resolve direct hostname,
incase multiple loopback addresses are present in /etc/hosts (was:
SecureUtil#getByName should also try to resolve direct hostname incase multiple
loopback addresses are present in etc/hosts)
> SecureUtil#getByName should also try to resolve direct hostname, incase
> multiple loopback addresses are present in /etc/hosts
> -
>
> Key: HADOOP-12687
> URL: https://issues.apache.org/jira/browse/HADOOP-12687
> Project: Hadoop Common
> Issue Type: Bug
>Reporter: Junping Du
>Assignee: Sunil G
> Labels: security
> Attachments: 0001-YARN-4352.patch, 0002-YARN-4352.patch,
> 0003-HADOOP-12687.patch, 0004-HADOOP-12687.patch
>
>
> From
> https://builds.apache.org/job/PreCommit-YARN-Build/9661/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-client-jdk1.7.0_79.txt,
> we can see the tests in TestYarnClient, TestAMRMClient and TestNMClient get
> timeout which can be reproduced locally.
> When {{/etc/hosts}} has multiple loopback entries,
> {{InetAddress.getByName(null)}} will be returning the first entry present in
> etc/hosts. Hence its possible that machine hostname can be second in list and
> cause {{UnKnownHostException}}.
> Suggesting a direct resolve for such hostname scenarios.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Moved] (HADOOP-12687) Timeout for tests in TestYarnClient, TestAMRMClient and TestNMClient
[ https://issues.apache.org/jira/browse/HADOOP-12687?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sunil G moved YARN-4352 to HADOOP-12687: Target Version/s: (was: 2.8.0) Key: HADOOP-12687 (was: YARN-4352) Project: Hadoop Common (was: Hadoop YARN) > Timeout for tests in TestYarnClient, TestAMRMClient and TestNMClient > > > Key: HADOOP-12687 > URL: https://issues.apache.org/jira/browse/HADOOP-12687 > Project: Hadoop Common > Issue Type: Bug >Reporter: Junping Du >Assignee: Sunil G > Labels: security > Attachments: 0001-YARN-4352.patch, 0002-YARN-4352.patch > > > From > https://builds.apache.org/job/PreCommit-YARN-Build/9661/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-client-jdk1.7.0_79.txt, > we can see the tests in TestYarnClient, TestAMRMClient and TestNMClient get > timeout which can be reproduced locally. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HADOOP-12687) Timeout for tests in TestYarnClient, TestAMRMClient and TestNMClient
[ https://issues.apache.org/jira/browse/HADOOP-12687?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15081030#comment-15081030 ] Sunil G commented on HADOOP-12687: -- Thanks [~vinayrpet]. I have moved to HADOOP-COMMON. > Timeout for tests in TestYarnClient, TestAMRMClient and TestNMClient > > > Key: HADOOP-12687 > URL: https://issues.apache.org/jira/browse/HADOOP-12687 > Project: Hadoop Common > Issue Type: Bug >Reporter: Junping Du >Assignee: Sunil G > Labels: security > Attachments: 0001-YARN-4352.patch, 0002-YARN-4352.patch > > > From > https://builds.apache.org/job/PreCommit-YARN-Build/9661/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-client-jdk1.7.0_79.txt, > we can see the tests in TestYarnClient, TestAMRMClient and TestNMClient get > timeout which can be reproduced locally. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HADOOP-12321) Make JvmPauseMonitor an AbstractService
[
https://issues.apache.org/jira/browse/HADOOP-12321?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15044895#comment-15044895
]
Sunil G commented on HADOOP-12321:
--
Yes, the idea sounds perfect. A single revert will really help in case of any
major issues.
> Make JvmPauseMonitor an AbstractService
> ---
>
> Key: HADOOP-12321
> URL: https://issues.apache.org/jira/browse/HADOOP-12321
> Project: Hadoop Common
> Issue Type: New Feature
>Affects Versions: 2.8.0
>Reporter: Steve Loughran
>Assignee: Sunil G
> Fix For: 2.9.0
>
> Attachments: 0001-HADOOP-12321.patch, 0002-HADOOP-12321.patch,
> 0004-HADOOP-12321.patch, HADOOP-12321-003.patch,
> HADOOP-12321-005-aggregated.patch
>
> Original Estimate: 1h
> Remaining Estimate: 1h
>
> The new JVM pause monitor has been written with its own start/stop lifecycle
> which has already proven brittle to both ordering of operations and, even
> after HADOOP-12313, is not thread safe (both start and stop are potentially
> re-entrant).
> It also requires every class which supports the monitor to add another field
> and perform the lifecycle operations in its own lifecycle, which, for all
> Yarn services, is the YARN app lifecycle (as implemented in Hadoop common)
> Making the monitor a subclass of {{AbstractService}} and moving the
> init/start & stop operations in {{serviceInit()}}, {{serviceStart()}} &
> {{serviceStop()}} methods will fix the concurrency and state model issues,
> and make it trivial to add as a child to any YARN service which subclasses
> {{CompositeService}} (most the NM and RM apps) will be able to hook up the
> monitor simply by creating one in the ctor and adding it as a child.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HADOOP-12321) Make JvmPauseMonitor an AbstractService
[
https://issues.apache.org/jira/browse/HADOOP-12321?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15044309#comment-15044309
]
Sunil G commented on HADOOP-12321:
--
Thanks Steve, I just noticed that you halve already handled that jiras as
well. Thank you.
> Make JvmPauseMonitor an AbstractService
> ---
>
> Key: HADOOP-12321
> URL: https://issues.apache.org/jira/browse/HADOOP-12321
> Project: Hadoop Common
> Issue Type: New Feature
>Affects Versions: 2.8.0
>Reporter: Steve Loughran
>Assignee: Sunil G
> Fix For: 2.9.0
>
> Attachments: 0001-HADOOP-12321.patch, 0002-HADOOP-12321.patch,
> 0004-HADOOP-12321.patch, HADOOP-12321-003.patch,
> HADOOP-12321-005-aggregated.patch
>
> Original Estimate: 1h
> Remaining Estimate: 1h
>
> The new JVM pause monitor has been written with its own start/stop lifecycle
> which has already proven brittle to both ordering of operations and, even
> after HADOOP-12313, is not thread safe (both start and stop are potentially
> re-entrant).
> It also requires every class which supports the monitor to add another field
> and perform the lifecycle operations in its own lifecycle, which, for all
> Yarn services, is the YARN app lifecycle (as implemented in Hadoop common)
> Making the monitor a subclass of {{AbstractService}} and moving the
> init/start & stop operations in {{serviceInit()}}, {{serviceStart()}} &
> {{serviceStop()}} methods will fix the concurrency and state model issues,
> and make it trivial to add as a child to any YARN service which subclasses
> {{CompositeService}} (most the NM and RM apps) will be able to hook up the
> monitor simply by creating one in the ctor and adding it as a child.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HADOOP-12321) Make JvmPauseMonitor an AbstractService
[
https://issues.apache.org/jira/browse/HADOOP-12321?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15044293#comment-15044293
]
Sunil G commented on HADOOP-12321:
--
Thank you [~steve_l] for the review, support and commit.!!
I ll mark other jiras for YARN, MR, and HDFS as done since we handled all
here. Thank you.
> Make JvmPauseMonitor an AbstractService
> ---
>
> Key: HADOOP-12321
> URL: https://issues.apache.org/jira/browse/HADOOP-12321
> Project: Hadoop Common
> Issue Type: New Feature
>Affects Versions: 2.8.0
>Reporter: Steve Loughran
>Assignee: Sunil G
> Fix For: 2.9.0
>
> Attachments: 0001-HADOOP-12321.patch, 0002-HADOOP-12321.patch,
> 0004-HADOOP-12321.patch, HADOOP-12321-003.patch,
> HADOOP-12321-005-aggregated.patch
>
> Original Estimate: 1h
> Remaining Estimate: 1h
>
> The new JVM pause monitor has been written with its own start/stop lifecycle
> which has already proven brittle to both ordering of operations and, even
> after HADOOP-12313, is not thread safe (both start and stop are potentially
> re-entrant).
> It also requires every class which supports the monitor to add another field
> and perform the lifecycle operations in its own lifecycle, which, for all
> Yarn services, is the YARN app lifecycle (as implemented in Hadoop common)
> Making the monitor a subclass of {{AbstractService}} and moving the
> init/start & stop operations in {{serviceInit()}}, {{serviceStart()}} &
> {{serviceStop()}} methods will fix the concurrency and state model issues,
> and make it trivial to add as a child to any YARN service which subclasses
> {{CompositeService}} (most the NM and RM apps) will be able to hook up the
> monitor simply by creating one in the ctor and adding it as a child.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HADOOP-12321) Make JvmPauseMonitor to AbstractService
[
https://issues.apache.org/jira/browse/HADOOP-12321?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15041661#comment-15041661
]
Sunil G commented on HADOOP-12321:
--
Hi [~steve_l]
Still didnt get a clean jenkins. However apart from know test case failures,
other are fine locally.
But one unused import is present in {{JvmPauseMonitor}}. Could you pls see the
results, and as needed I can give an incremental patch here.
> Make JvmPauseMonitor to AbstractService
> ---
>
> Key: HADOOP-12321
> URL: https://issues.apache.org/jira/browse/HADOOP-12321
> Project: Hadoop Common
> Issue Type: New Feature
>Affects Versions: 2.8.0
>Reporter: Steve Loughran
>Assignee: Sunil G
> Attachments: 0001-HADOOP-12321.patch, 0002-HADOOP-12321.patch,
> 0004-HADOOP-12321.patch, HADOOP-12321-003.patch,
> HADOOP-12321-005-aggregated.patch
>
> Original Estimate: 1h
> Remaining Estimate: 1h
>
> The new JVM pause monitor has been written with its own start/stop lifecycle
> which has already proven brittle to both ordering of operations and, even
> after HADOOP-12313, is not thread safe (both start and stop are potentially
> re-entrant).
> It also requires every class which supports the monitor to add another field
> and perform the lifecycle operations in its own lifecycle, which, for all
> Yarn services, is the YARN app lifecycle (as implemented in Hadoop common)
> Making the monitor a subclass of {{AbstractService}} and moving the
> init/start & stop operations in {{serviceInit()}}, {{serviceStart()}} &
> {{serviceStop()}} methods will fix the concurrency and state model issues,
> and make it trivial to add as a child to any YARN service which subclasses
> {{CompositeService}} (most the NM and RM apps) will be able to hook up the
> monitor simply by creating one in the ctor and adding it as a child.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HADOOP-12321) Make JvmPauseMonitor to AbstractService
[
https://issues.apache.org/jira/browse/HADOOP-12321?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15037214#comment-15037214
]
Sunil G commented on HADOOP-12321:
--
I ran failed test cases other than the know test cases. Passing locally.
YARN-4306 tracks TestClientRMTokens failures.
> Make JvmPauseMonitor to AbstractService
> ---
>
> Key: HADOOP-12321
> URL: https://issues.apache.org/jira/browse/HADOOP-12321
> Project: Hadoop Common
> Issue Type: New Feature
>Affects Versions: 2.8.0
>Reporter: Steve Loughran
>Assignee: Sunil G
> Attachments: 0001-HADOOP-12321.patch, 0002-HADOOP-12321.patch,
> 0004-HADOOP-12321.patch, HADOOP-12321-003.patch,
> HADOOP-12321-005-aggregated.patch
>
> Original Estimate: 1h
> Remaining Estimate: 1h
>
> The new JVM pause monitor has been written with its own start/stop lifecycle
> which has already proven brittle to both ordering of operations and, even
> after HADOOP-12313, is not thread safe (both start and stop are potentially
> re-entrant).
> It also requires every class which supports the monitor to add another field
> and perform the lifecycle operations in its own lifecycle, which, for all
> Yarn services, is the YARN app lifecycle (as implemented in Hadoop common)
> Making the monitor a subclass of {{AbstractService}} and moving the
> init/start & stop operations in {{serviceInit()}}, {{serviceStart()}} &
> {{serviceStop()}} methods will fix the concurrency and state model issues,
> and make it trivial to add as a child to any YARN service which subclasses
> {{CompositeService}} (most the NM and RM apps) will be able to hook up the
> monitor simply by creating one in the ctor and adding it as a child.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HADOOP-12321) Make JvmPauseMonitor to AbstractService
[
https://issues.apache.org/jira/browse/HADOOP-12321?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15035749#comment-15035749
]
Sunil G commented on HADOOP-12321:
--
Hi [~steve_l]
It looks like jenkins is not triggered yet.
I will cancel the patch and try to update patch again. Then will submit it
again.
> Make JvmPauseMonitor to AbstractService
> ---
>
> Key: HADOOP-12321
> URL: https://issues.apache.org/jira/browse/HADOOP-12321
> Project: Hadoop Common
> Issue Type: New Feature
>Affects Versions: 2.8.0
>Reporter: Steve Loughran
>Assignee: Sunil G
> Attachments: 0001-HADOOP-12321.patch, 0002-HADOOP-12321.patch,
> 0004-HADOOP-12321.patch, HADOOP-12321-003.patch,
> HADOOP-12321-005-aggregated.patch
>
> Original Estimate: 1h
> Remaining Estimate: 1h
>
> The new JVM pause monitor has been written with its own start/stop lifecycle
> which has already proven brittle to both ordering of operations and, even
> after HADOOP-12313, is not thread safe (both start and stop are potentially
> re-entrant).
> It also requires every class which supports the monitor to add another field
> and perform the lifecycle operations in its own lifecycle, which, for all
> Yarn services, is the YARN app lifecycle (as implemented in Hadoop common)
> Making the monitor a subclass of {{AbstractService}} and moving the
> init/start & stop operations in {{serviceInit()}}, {{serviceStart()}} &
> {{serviceStop()}} methods will fix the concurrency and state model issues,
> and make it trivial to add as a child to any YARN service which subclasses
> {{CompositeService}} (most the NM and RM apps) will be able to hook up the
> monitor simply by creating one in the ctor and adding it as a child.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HADOOP-12321) Make JvmPauseMonitor to AbstractService
[
https://issues.apache.org/jira/browse/HADOOP-12321?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15032068#comment-15032068
]
Sunil G commented on HADOOP-12321:
--
Hi [~steve_l]
I was able to share the pull request above. It seems like jenkins didnt get
kicked in with this. I am not very sure whether I need to cancel and submit the
patch again, cud u pls help to check the same.
> Make JvmPauseMonitor to AbstractService
> ---
>
> Key: HADOOP-12321
> URL: https://issues.apache.org/jira/browse/HADOOP-12321
> Project: Hadoop Common
> Issue Type: New Feature
>Affects Versions: 2.8.0
>Reporter: Steve Loughran
>Assignee: Sunil G
> Attachments: 0001-HADOOP-12321.patch, 0002-HADOOP-12321.patch,
> 0004-HADOOP-12321.patch, HADOOP-12321-003.patch,
> HADOOP-12321-005-aggregated.patch
>
> Original Estimate: 1h
> Remaining Estimate: 1h
>
> The new JVM pause monitor has been written with its own start/stop lifecycle
> which has already proven brittle to both ordering of operations and, even
> after HADOOP-12313, is not thread safe (both start and stop are potentially
> re-entrant).
> It also requires every class which supports the monitor to add another field
> and perform the lifecycle operations in its own lifecycle, which, for all
> Yarn services, is the YARN app lifecycle (as implemented in Hadoop common)
> Making the monitor a subclass of {{AbstractService}} and moving the
> init/start & stop operations in {{serviceInit()}}, {{serviceStart()}} &
> {{serviceStop()}} methods will fix the concurrency and state model issues,
> and make it trivial to add as a child to any YARN service which subclasses
> {{CompositeService}} (most the NM and RM apps) will be able to hook up the
> monitor simply by creating one in the ctor and adding it as a child.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HADOOP-12321) Make JvmPauseMonitor to AbstractService
[
https://issues.apache.org/jira/browse/HADOOP-12321?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15026399#comment-15026399
]
Sunil G commented on HADOOP-12321:
--
Hi [~steve_l]
Could I add a clean rebased patch here directly, or via pull request.
> Make JvmPauseMonitor to AbstractService
> ---
>
> Key: HADOOP-12321
> URL: https://issues.apache.org/jira/browse/HADOOP-12321
> Project: Hadoop Common
> Issue Type: New Feature
>Affects Versions: 2.8.0
>Reporter: Steve Loughran
>Assignee: Sunil G
> Attachments: 0001-HADOOP-12321.patch, 0002-HADOOP-12321.patch,
> 0004-HADOOP-12321.patch, HADOOP-12321-003.patch,
> HADOOP-12321-005-aggregated.patch
>
> Original Estimate: 1h
> Remaining Estimate: 1h
>
> The new JVM pause monitor has been written with its own start/stop lifecycle
> which has already proven brittle to both ordering of operations and, even
> after HADOOP-12313, is not thread safe (both start and stop are potentially
> re-entrant).
> It also requires every class which supports the monitor to add another field
> and perform the lifecycle operations in its own lifecycle, which, for all
> Yarn services, is the YARN app lifecycle (as implemented in Hadoop common)
> Making the monitor a subclass of {{AbstractService}} and moving the
> init/start & stop operations in {{serviceInit()}}, {{serviceStart()}} &
> {{serviceStop()}} methods will fix the concurrency and state model issues,
> and make it trivial to add as a child to any YARN service which subclasses
> {{CompositeService}} (most the NM and RM apps) will be able to hook up the
> monitor simply by creating one in the ctor and adding it as a child.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HADOOP-12321) Make JvmPauseMonitor to AbstractService
[
https://issues.apache.org/jira/browse/HADOOP-12321?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15026831#comment-15026831
]
Sunil G commented on HADOOP-12321:
--
Yes, that's fine. I will do the same.
> Make JvmPauseMonitor to AbstractService
> ---
>
> Key: HADOOP-12321
> URL: https://issues.apache.org/jira/browse/HADOOP-12321
> Project: Hadoop Common
> Issue Type: New Feature
>Affects Versions: 2.8.0
>Reporter: Steve Loughran
>Assignee: Sunil G
> Attachments: 0001-HADOOP-12321.patch, 0002-HADOOP-12321.patch,
> 0004-HADOOP-12321.patch, HADOOP-12321-003.patch,
> HADOOP-12321-005-aggregated.patch
>
> Original Estimate: 1h
> Remaining Estimate: 1h
>
> The new JVM pause monitor has been written with its own start/stop lifecycle
> which has already proven brittle to both ordering of operations and, even
> after HADOOP-12313, is not thread safe (both start and stop are potentially
> re-entrant).
> It also requires every class which supports the monitor to add another field
> and perform the lifecycle operations in its own lifecycle, which, for all
> Yarn services, is the YARN app lifecycle (as implemented in Hadoop common)
> Making the monitor a subclass of {{AbstractService}} and moving the
> init/start & stop operations in {{serviceInit()}}, {{serviceStart()}} &
> {{serviceStop()}} methods will fix the concurrency and state model issues,
> and make it trivial to add as a child to any YARN service which subclasses
> {{CompositeService}} (most the NM and RM apps) will be able to hook up the
> monitor simply by creating one in the ctor and adding it as a child.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HADOOP-12321) Make JvmPauseMonitor to AbstractService
[
https://issues.apache.org/jira/browse/HADOOP-12321?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15021859#comment-15021859
]
Sunil G commented on HADOOP-12321:
--
Hi [~steve_l], looks like the new pull request doesn't apply clean.
> Make JvmPauseMonitor to AbstractService
> ---
>
> Key: HADOOP-12321
> URL: https://issues.apache.org/jira/browse/HADOOP-12321
> Project: Hadoop Common
> Issue Type: New Feature
>Affects Versions: 2.8.0
>Reporter: Steve Loughran
>Assignee: Sunil G
> Attachments: 0001-HADOOP-12321.patch, 0002-HADOOP-12321.patch,
> 0004-HADOOP-12321.patch, HADOOP-12321-003.patch,
> HADOOP-12321-005-aggregated.patch
>
> Original Estimate: 1h
> Remaining Estimate: 1h
>
> The new JVM pause monitor has been written with its own start/stop lifecycle
> which has already proven brittle to both ordering of operations and, even
> after HADOOP-12313, is not thread safe (both start and stop are potentially
> re-entrant).
> It also requires every class which supports the monitor to add another field
> and perform the lifecycle operations in its own lifecycle, which, for all
> Yarn services, is the YARN app lifecycle (as implemented in Hadoop common)
> Making the monitor a subclass of {{AbstractService}} and moving the
> init/start & stop operations in {{serviceInit()}}, {{serviceStart()}} &
> {{serviceStop()}} methods will fix the concurrency and state model issues,
> and make it trivial to add as a child to any YARN service which subclasses
> {{CompositeService}} (most the NM and RM apps) will be able to hook up the
> monitor simply by creating one in the ctor and adding it as a child.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HADOOP-12321) Make JvmPauseMonitor to AbstractService
[
https://issues.apache.org/jira/browse/HADOOP-12321?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14978198#comment-14978198
]
Sunil G commented on HADOOP-12321:
--
Hi [~steve_l],
Test case failures are not related. Raised YARN-4306 to track RM test failures
in DelegationToken.
> Make JvmPauseMonitor to AbstractService
> ---
>
> Key: HADOOP-12321
> URL: https://issues.apache.org/jira/browse/HADOOP-12321
> Project: Hadoop Common
> Issue Type: New Feature
>Affects Versions: 2.8.0
>Reporter: Steve Loughran
>Assignee: Sunil G
> Attachments: 0001-HADOOP-12321.patch, 0002-HADOOP-12321.patch,
> 0004-HADOOP-12321.patch, HADOOP-12321-003.patch,
> HADOOP-12321-005-aggregated.patch
>
> Original Estimate: 1h
> Remaining Estimate: 1h
>
> The new JVM pause monitor has been written with its own start/stop lifecycle
> which has already proven brittle to both ordering of operations and, even
> after HADOOP-12313, is not thread safe (both start and stop are potentially
> re-entrant).
> It also requires every class which supports the monitor to add another field
> and perform the lifecycle operations in its own lifecycle, which, for all
> Yarn services, is the YARN app lifecycle (as implemented in Hadoop common)
> Making the monitor a subclass of {{AbstractService}} and moving the
> init/start & stop operations in {{serviceInit()}}, {{serviceStart()}} &
> {{serviceStop()}} methods will fix the concurrency and state model issues,
> and make it trivial to add as a child to any YARN service which subclasses
> {{CompositeService}} (most the NM and RM apps) will be able to hook up the
> monitor simply by creating one in the ctor and adding it as a child.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HADOOP-12321) Make JvmPauseMonitor to AbstractService
[
https://issues.apache.org/jira/browse/HADOOP-12321?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sunil G updated HADOOP-12321:
-
Attachment: HADOOP-12321-005-aggregated.patch
Reattaching patch to see jenkins report again.
> Make JvmPauseMonitor to AbstractService
> ---
>
> Key: HADOOP-12321
> URL: https://issues.apache.org/jira/browse/HADOOP-12321
> Project: Hadoop Common
> Issue Type: New Feature
>Affects Versions: 2.8.0
>Reporter: Steve Loughran
>Assignee: Sunil G
> Attachments: 0001-HADOOP-12321.patch, 0002-HADOOP-12321.patch,
> 0004-HADOOP-12321.patch, HADOOP-12321-003.patch,
> HADOOP-12321-005-aggregated.patch
>
> Original Estimate: 1h
> Remaining Estimate: 1h
>
> The new JVM pause monitor has been written with its own start/stop lifecycle
> which has already proven brittle to both ordering of operations and, even
> after HADOOP-12313, is not thread safe (both start and stop are potentially
> re-entrant).
> It also requires every class which supports the monitor to add another field
> and perform the lifecycle operations in its own lifecycle, which, for all
> Yarn services, is the YARN app lifecycle (as implemented in Hadoop common)
> Making the monitor a subclass of {{AbstractService}} and moving the
> init/start & stop operations in {{serviceInit()}}, {{serviceStart()}} &
> {{serviceStop()}} methods will fix the concurrency and state model issues,
> and make it trivial to add as a child to any YARN service which subclasses
> {{CompositeService}} (most the NM and RM apps) will be able to hook up the
> monitor simply by creating one in the ctor and adding it as a child.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HADOOP-12321) Make JvmPauseMonitor to AbstractService
[
https://issues.apache.org/jira/browse/HADOOP-12321?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sunil G updated HADOOP-12321:
-
Attachment: (was: HADOOP-12321-005-aggregated.patch)
> Make JvmPauseMonitor to AbstractService
> ---
>
> Key: HADOOP-12321
> URL: https://issues.apache.org/jira/browse/HADOOP-12321
> Project: Hadoop Common
> Issue Type: New Feature
>Affects Versions: 2.8.0
>Reporter: Steve Loughran
>Assignee: Sunil G
> Attachments: 0001-HADOOP-12321.patch, 0002-HADOOP-12321.patch,
> 0004-HADOOP-12321.patch, HADOOP-12321-003.patch
>
> Original Estimate: 1h
> Remaining Estimate: 1h
>
> The new JVM pause monitor has been written with its own start/stop lifecycle
> which has already proven brittle to both ordering of operations and, even
> after HADOOP-12313, is not thread safe (both start and stop are potentially
> re-entrant).
> It also requires every class which supports the monitor to add another field
> and perform the lifecycle operations in its own lifecycle, which, for all
> Yarn services, is the YARN app lifecycle (as implemented in Hadoop common)
> Making the monitor a subclass of {{AbstractService}} and moving the
> init/start & stop operations in {{serviceInit()}}, {{serviceStart()}} &
> {{serviceStop()}} methods will fix the concurrency and state model issues,
> and make it trivial to add as a child to any YARN service which subclasses
> {{CompositeService}} (most the NM and RM apps) will be able to hook up the
> monitor simply by creating one in the ctor and adding it as a child.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HADOOP-12321) Make JvmPauseMonitor to AbstractService
[
https://issues.apache.org/jira/browse/HADOOP-12321?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14963172#comment-14963172
]
Sunil G commented on HADOOP-12321:
--
Hi [~steve_l]
I feel we are still yet to get clean jenkins :(
- Findbugs error pages are not opening (404 error)
- Javadoc errors are for webapp and node labels , not related to this patch.
- checksyle error is for file length
- test cases are mostly not related, still i will run all these locally and
will update the info.
> Make JvmPauseMonitor to AbstractService
> ---
>
> Key: HADOOP-12321
> URL: https://issues.apache.org/jira/browse/HADOOP-12321
> Project: Hadoop Common
> Issue Type: New Feature
>Affects Versions: 2.8.0
>Reporter: Steve Loughran
>Assignee: Sunil G
> Attachments: 0001-HADOOP-12321.patch, 0002-HADOOP-12321.patch,
> 0004-HADOOP-12321.patch, HADOOP-12321-003.patch,
> HADOOP-12321-005-aggregated.patch
>
> Original Estimate: 1h
> Remaining Estimate: 1h
>
> The new JVM pause monitor has been written with its own start/stop lifecycle
> which has already proven brittle to both ordering of operations and, even
> after HADOOP-12313, is not thread safe (both start and stop are potentially
> re-entrant).
> It also requires every class which supports the monitor to add another field
> and perform the lifecycle operations in its own lifecycle, which, for all
> Yarn services, is the YARN app lifecycle (as implemented in Hadoop common)
> Making the monitor a subclass of {{AbstractService}} and moving the
> init/start & stop operations in {{serviceInit()}}, {{serviceStart()}} &
> {{serviceStop()}} methods will fix the concurrency and state model issues,
> and make it trivial to add as a child to any YARN service which subclasses
> {{CompositeService}} (most the NM and RM apps) will be able to hook up the
> monitor simply by creating one in the ctor and adding it as a child.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HADOOP-12321) Make JvmPauseMonitor to AbstractService
[
https://issues.apache.org/jira/browse/HADOOP-12321?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14957373#comment-14957373
]
Sunil G commented on HADOOP-12321:
--
Hi [~steve_l]
In other dependent JIRAS (YARN, HDFS etc), I couldnt get any clean run. Here in
main ticket, jenkins results looks fine. Is this fine? or do i need to trigger
and try running jenkins in all sub tickets. Kindly advise.
> Make JvmPauseMonitor to AbstractService
> ---
>
> Key: HADOOP-12321
> URL: https://issues.apache.org/jira/browse/HADOOP-12321
> Project: Hadoop Common
> Issue Type: New Feature
>Affects Versions: 2.8.0
>Reporter: Steve Loughran
>Assignee: Sunil G
> Attachments: 0001-HADOOP-12321.patch, 0002-HADOOP-12321.patch,
> 0004-HADOOP-12321.patch, HADOOP-12321-003.patch,
> HADOOP-12321-005-aggregated.patch
>
> Original Estimate: 1h
> Remaining Estimate: 1h
>
> The new JVM pause monitor has been written with its own start/stop lifecycle
> which has already proven brittle to both ordering of operations and, even
> after HADOOP-12313, is not thread safe (both start and stop are potentially
> re-entrant).
> It also requires every class which supports the monitor to add another field
> and perform the lifecycle operations in its own lifecycle, which, for all
> Yarn services, is the YARN app lifecycle (as implemented in Hadoop common)
> Making the monitor a subclass of {{AbstractService}} and moving the
> init/start & stop operations in {{serviceInit()}}, {{serviceStart()}} &
> {{serviceStop()}} methods will fix the concurrency and state model issues,
> and make it trivial to add as a child to any YARN service which subclasses
> {{CompositeService}} (most the NM and RM apps) will be able to hook up the
> monitor simply by creating one in the ctor and adding it as a child.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HADOOP-12321) Make JvmPauseMonitor to AbstractService
[
https://issues.apache.org/jira/browse/HADOOP-12321?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14900784#comment-14900784
]
Sunil G commented on HADOOP-12321:
--
Hi [~steve_l]
2 failures are present, which looks like not related to this patch. Kindly help
to check the same.
> Make JvmPauseMonitor to AbstractService
> ---
>
> Key: HADOOP-12321
> URL: https://issues.apache.org/jira/browse/HADOOP-12321
> Project: Hadoop Common
> Issue Type: New Feature
>Affects Versions: 2.8.0
>Reporter: Steve Loughran
>Assignee: Sunil G
> Attachments: 0001-HADOOP-12321.patch, 0002-HADOOP-12321.patch,
> 0004-HADOOP-12321.patch, HADOOP-12321-003.patch,
> HADOOP-12321-005-aggregated.patch
>
> Original Estimate: 1h
> Remaining Estimate: 1h
>
> The new JVM pause monitor has been written with its own start/stop lifecycle
> which has already proven brittle to both ordering of operations and, even
> after HADOOP-12313, is not thread safe (both start and stop are potentially
> re-entrant).
> It also requires every class which supports the monitor to add another field
> and perform the lifecycle operations in its own lifecycle, which, for all
> Yarn services, is the YARN app lifecycle (as implemented in Hadoop common)
> Making the monitor a subclass of {{AbstractService}} and moving the
> init/start & stop operations in {{serviceInit()}}, {{serviceStart()}} &
> {{serviceStop()}} methods will fix the concurrency and state model issues,
> and make it trivial to add as a child to any YARN service which subclasses
> {{CompositeService}} (most the NM and RM apps) will be able to hook up the
> monitor simply by creating one in the ctor and adding it as a child.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HADOOP-12321) Make JvmPauseMonitor to AbstractService
[
https://issues.apache.org/jira/browse/HADOOP-12321?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14875875#comment-14875875
]
Sunil G commented on HADOOP-12321:
--
I feel here also hadoop-common jar is not latest which is picked. Hence
JvmPauseMonitor.init() failed for HDFS test cases. A clean build is needed,
will kick jenkins later.
> Make JvmPauseMonitor to AbstractService
> ---
>
> Key: HADOOP-12321
> URL: https://issues.apache.org/jira/browse/HADOOP-12321
> Project: Hadoop Common
> Issue Type: New Feature
>Affects Versions: 2.8.0
>Reporter: Steve Loughran
>Assignee: Sunil G
> Attachments: 0001-HADOOP-12321.patch, 0002-HADOOP-12321.patch,
> 0004-HADOOP-12321.patch, HADOOP-12321-003.patch,
> HADOOP-12321-005-aggregated.patch
>
> Original Estimate: 1h
> Remaining Estimate: 1h
>
> The new JVM pause monitor has been written with its own start/stop lifecycle
> which has already proven brittle to both ordering of operations and, even
> after HADOOP-12313, is not thread safe (both start and stop are potentially
> re-entrant).
> It also requires every class which supports the monitor to add another field
> and perform the lifecycle operations in its own lifecycle, which, for all
> Yarn services, is the YARN app lifecycle (as implemented in Hadoop common)
> Making the monitor a subclass of {{AbstractService}} and moving the
> init/start & stop operations in {{serviceInit()}}, {{serviceStart()}} &
> {{serviceStop()}} methods will fix the concurrency and state model issues,
> and make it trivial to add as a child to any YARN service which subclasses
> {{CompositeService}} (most the NM and RM apps) will be able to hook up the
> monitor simply by creating one in the ctor and adding it as a child.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HADOOP-12321) Make JvmPauseMonitor to AbstractService
[
https://issues.apache.org/jira/browse/HADOOP-12321?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14745508#comment-14745508
]
Sunil G commented on HADOOP-12321:
--
There are few more test failures here, but a much cleaner report is got in
MAPREDUCE-5462 and those failed cases are know pblms which are getting handled
in other JIRAs.
> Make JvmPauseMonitor to AbstractService
> ---
>
> Key: HADOOP-12321
> URL: https://issues.apache.org/jira/browse/HADOOP-12321
> Project: Hadoop Common
> Issue Type: New Feature
>Affects Versions: 2.8.0
>Reporter: Steve Loughran
>Assignee: Sunil G
> Attachments: 0001-HADOOP-12321.patch, 0002-HADOOP-12321.patch,
> 0004-HADOOP-12321.patch, HADOOP-12321-003.patch,
> HADOOP-12321-005-aggregated.patch
>
> Original Estimate: 1h
> Remaining Estimate: 1h
>
> The new JVM pause monitor has been written with its own start/stop lifecycle
> which has already proven brittle to both ordering of operations and, even
> after HADOOP-12313, is not thread safe (both start and stop are potentially
> re-entrant).
> It also requires every class which supports the monitor to add another field
> and perform the lifecycle operations in its own lifecycle, which, for all
> Yarn services, is the YARN app lifecycle (as implemented in Hadoop common)
> Making the monitor a subclass of {{AbstractService}} and moving the
> init/start & stop operations in {{serviceInit()}}, {{serviceStart()}} &
> {{serviceStop()}} methods will fix the concurrency and state model issues,
> and make it trivial to add as a child to any YARN service which subclasses
> {{CompositeService}} (most the NM and RM apps) will be able to hook up the
> monitor simply by creating one in the ctor and adding it as a child.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HADOOP-12321) Make JvmPauseMonitor to AbstractService
[
https://issues.apache.org/jira/browse/HADOOP-12321?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14745509#comment-14745509
]
Sunil G commented on HADOOP-12321:
--
Sorry. I meant MAPREDUCE-6462
> Make JvmPauseMonitor to AbstractService
> ---
>
> Key: HADOOP-12321
> URL: https://issues.apache.org/jira/browse/HADOOP-12321
> Project: Hadoop Common
> Issue Type: New Feature
>Affects Versions: 2.8.0
>Reporter: Steve Loughran
>Assignee: Sunil G
> Attachments: 0001-HADOOP-12321.patch, 0002-HADOOP-12321.patch,
> 0004-HADOOP-12321.patch, HADOOP-12321-003.patch,
> HADOOP-12321-005-aggregated.patch
>
> Original Estimate: 1h
> Remaining Estimate: 1h
>
> The new JVM pause monitor has been written with its own start/stop lifecycle
> which has already proven brittle to both ordering of operations and, even
> after HADOOP-12313, is not thread safe (both start and stop are potentially
> re-entrant).
> It also requires every class which supports the monitor to add another field
> and perform the lifecycle operations in its own lifecycle, which, for all
> Yarn services, is the YARN app lifecycle (as implemented in Hadoop common)
> Making the monitor a subclass of {{AbstractService}} and moving the
> init/start & stop operations in {{serviceInit()}}, {{serviceStart()}} &
> {{serviceStop()}} methods will fix the concurrency and state model issues,
> and make it trivial to add as a child to any YARN service which subclasses
> {{CompositeService}} (most the NM and RM apps) will be able to hook up the
> monitor simply by creating one in the ctor and adding it as a child.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HADOOP-12321) Make JvmPauseMonitor to AbstractService
[
https://issues.apache.org/jira/browse/HADOOP-12321?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sunil G updated HADOOP-12321:
-
Attachment: HADOOP-12321-005-aggregated.patch
Attaching an aggregated patch.
For test failures in MAPREDUCE-6462, I also feel we can remove that service
check. It looks like doesnt add much of a value. Kicking Jenkins.
> Make JvmPauseMonitor to AbstractService
> ---
>
> Key: HADOOP-12321
> URL: https://issues.apache.org/jira/browse/HADOOP-12321
> Project: Hadoop Common
> Issue Type: New Feature
>Affects Versions: 2.8.0
>Reporter: Steve Loughran
>Assignee: Sunil G
> Attachments: 0001-HADOOP-12321.patch, 0002-HADOOP-12321.patch,
> 0004-HADOOP-12321.patch, HADOOP-12321-003.patch,
> HADOOP-12321-005-aggregated.patch
>
> Original Estimate: 1h
> Remaining Estimate: 1h
>
> The new JVM pause monitor has been written with its own start/stop lifecycle
> which has already proven brittle to both ordering of operations and, even
> after HADOOP-12313, is not thread safe (both start and stop are potentially
> re-entrant).
> It also requires every class which supports the monitor to add another field
> and perform the lifecycle operations in its own lifecycle, which, for all
> Yarn services, is the YARN app lifecycle (as implemented in Hadoop common)
> Making the monitor a subclass of {{AbstractService}} and moving the
> init/start & stop operations in {{serviceInit()}}, {{serviceStart()}} &
> {{serviceStop()}} methods will fix the concurrency and state model issues,
> and make it trivial to add as a child to any YARN service which subclasses
> {{CompositeService}} (most the NM and RM apps) will be able to hook up the
> monitor simply by creating one in the ctor and adding it as a child.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HADOOP-12321) Make JvmPauseMonitor to AbstractService
[
https://issues.apache.org/jira/browse/HADOOP-12321?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sunil G updated HADOOP-12321:
-
Status: Open (was: Patch Available)
Cancelling patch to upload HADOOP specific and aggregated patches.
> Make JvmPauseMonitor to AbstractService
> ---
>
> Key: HADOOP-12321
> URL: https://issues.apache.org/jira/browse/HADOOP-12321
> Project: Hadoop Common
> Issue Type: New Feature
>Affects Versions: 2.8.0
>Reporter: Steve Loughran
>Assignee: Sunil G
> Attachments: 0001-HADOOP-12321.patch, 0002-HADOOP-12321.patch,
> 0004-HADOOP-12321.patch, HADOOP-12321-003.patch
>
> Original Estimate: 1h
> Remaining Estimate: 1h
>
> The new JVM pause monitor has been written with its own start/stop lifecycle
> which has already proven brittle to both ordering of operations and, even
> after HADOOP-12313, is not thread safe (both start and stop are potentially
> re-entrant).
> It also requires every class which supports the monitor to add another field
> and perform the lifecycle operations in its own lifecycle, which, for all
> Yarn services, is the YARN app lifecycle (as implemented in Hadoop common)
> Making the monitor a subclass of {{AbstractService}} and moving the
> init/start & stop operations in {{serviceInit()}}, {{serviceStart()}} &
> {{serviceStop()}} methods will fix the concurrency and state model issues,
> and make it trivial to add as a child to any YARN service which subclasses
> {{CompositeService}} (most the NM and RM apps) will be able to hook up the
> monitor simply by creating one in the ctor and adding it as a child.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HADOOP-12321) Make JvmPauseMonitor to AbstractService
[
https://issues.apache.org/jira/browse/HADOOP-12321?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sunil G updated HADOOP-12321:
-
Attachment: 0004-HADOOP-12321.patch
> Make JvmPauseMonitor to AbstractService
> ---
>
> Key: HADOOP-12321
> URL: https://issues.apache.org/jira/browse/HADOOP-12321
> Project: Hadoop Common
> Issue Type: New Feature
>Affects Versions: 2.8.0
>Reporter: Steve Loughran
>Assignee: Sunil G
> Attachments: 0001-HADOOP-12321.patch, 0002-HADOOP-12321.patch,
> 0004-HADOOP-12321.patch, HADOOP-12321-003.patch
>
> Original Estimate: 1h
> Remaining Estimate: 1h
>
> The new JVM pause monitor has been written with its own start/stop lifecycle
> which has already proven brittle to both ordering of operations and, even
> after HADOOP-12313, is not thread safe (both start and stop are potentially
> re-entrant).
> It also requires every class which supports the monitor to add another field
> and perform the lifecycle operations in its own lifecycle, which, for all
> Yarn services, is the YARN app lifecycle (as implemented in Hadoop common)
> Making the monitor a subclass of {{AbstractService}} and moving the
> init/start & stop operations in {{serviceInit()}}, {{serviceStart()}} &
> {{serviceStop()}} methods will fix the concurrency and state model issues,
> and make it trivial to add as a child to any YARN service which subclasses
> {{CompositeService}} (most the NM and RM apps) will be able to hook up the
> monitor simply by creating one in the ctor and adding it as a child.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HADOOP-12321) Make JvmPauseMonitor to AbstractService
[
https://issues.apache.org/jira/browse/HADOOP-12321?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sunil G updated HADOOP-12321:
-
Status: Patch Available (was: Open)
> Make JvmPauseMonitor to AbstractService
> ---
>
> Key: HADOOP-12321
> URL: https://issues.apache.org/jira/browse/HADOOP-12321
> Project: Hadoop Common
> Issue Type: New Feature
>Affects Versions: 2.8.0
>Reporter: Steve Loughran
>Assignee: Sunil G
> Attachments: 0001-HADOOP-12321.patch, 0002-HADOOP-12321.patch,
> 0004-HADOOP-12321.patch, HADOOP-12321-003.patch,
> HADOOP-12321-005-aggregated.patch
>
> Original Estimate: 1h
> Remaining Estimate: 1h
>
> The new JVM pause monitor has been written with its own start/stop lifecycle
> which has already proven brittle to both ordering of operations and, even
> after HADOOP-12313, is not thread safe (both start and stop are potentially
> re-entrant).
> It also requires every class which supports the monitor to add another field
> and perform the lifecycle operations in its own lifecycle, which, for all
> Yarn services, is the YARN app lifecycle (as implemented in Hadoop common)
> Making the monitor a subclass of {{AbstractService}} and moving the
> init/start & stop operations in {{serviceInit()}}, {{serviceStart()}} &
> {{serviceStop()}} methods will fix the concurrency and state model issues,
> and make it trivial to add as a child to any YARN service which subclasses
> {{CompositeService}} (most the NM and RM apps) will be able to hook up the
> monitor simply by creating one in the ctor and adding it as a child.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HADOOP-12386) RetryPolicies.RETRY_FOREVER should be able to specify a retry interval
[ https://issues.apache.org/jira/browse/HADOOP-12386?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14743603#comment-14743603 ] Sunil G commented on HADOOP-12386: -- Test case passed locally. [~leftnoteasy] could you please take a look. > RetryPolicies.RETRY_FOREVER should be able to specify a retry interval > -- > > Key: HADOOP-12386 > URL: https://issues.apache.org/jira/browse/HADOOP-12386 > Project: Hadoop Common > Issue Type: Bug >Reporter: Wangda Tan >Assignee: Sunil G > Attachments: 0001-HADOOP-12386.patch, 0002-HADOOP-12386.patch > > > Problems mentioned in YARN-4113, We should be able to specify retry interval > in RetryPolicies.RETRY_FOREVER. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HADOOP-12321) Make JvmPauseMonitor to AbstractService
[
https://issues.apache.org/jira/browse/HADOOP-12321?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14743215#comment-14743215
]
Sunil G commented on HADOOP-12321:
--
Thank you [~steve_l]
I think I overlooked those failures. Will make the changes and will upload a
common patch under all 3 sub jiras.
> Make JvmPauseMonitor to AbstractService
> ---
>
> Key: HADOOP-12321
> URL: https://issues.apache.org/jira/browse/HADOOP-12321
> Project: Hadoop Common
> Issue Type: New Feature
>Affects Versions: 2.8.0
>Reporter: Steve Loughran
>Assignee: Sunil G
> Attachments: 0001-HADOOP-12321.patch, 0002-HADOOP-12321.patch,
> HADOOP-12321-003.patch
>
> Original Estimate: 1h
> Remaining Estimate: 1h
>
> The new JVM pause monitor has been written with its own start/stop lifecycle
> which has already proven brittle to both ordering of operations and, even
> after HADOOP-12313, is not thread safe (both start and stop are potentially
> re-entrant).
> It also requires every class which supports the monitor to add another field
> and perform the lifecycle operations in its own lifecycle, which, for all
> Yarn services, is the YARN app lifecycle (as implemented in Hadoop common)
> Making the monitor a subclass of {{AbstractService}} and moving the
> init/start & stop operations in {{serviceInit()}}, {{serviceStart()}} &
> {{serviceStop()}} methods will fix the concurrency and state model issues,
> and make it trivial to add as a child to any YARN service which subclasses
> {{CompositeService}} (most the NM and RM apps) will be able to hook up the
> monitor simply by creating one in the ctor and adding it as a child.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HADOOP-12321) Make JvmPauseMonitor to AbstractService
[
https://issues.apache.org/jira/browse/HADOOP-12321?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14742749#comment-14742749
]
Sunil G commented on HADOOP-12321:
--
Thank you [~steve_l] for aggregating. By seeing jenkins result, compilation
broke for nm,rm,nn,dn,hs as its still expecting JvmPauseMonitor ctor to accept
config. It seems other projects are not considering the changed JvmPauseMonitor.
With same patch in MAPREDUCE-6462, compilation is fine and there are test case
failures. But look like those are not related to this.
> Make JvmPauseMonitor to AbstractService
> ---
>
> Key: HADOOP-12321
> URL: https://issues.apache.org/jira/browse/HADOOP-12321
> Project: Hadoop Common
> Issue Type: New Feature
>Affects Versions: 2.8.0
>Reporter: Steve Loughran
>Assignee: Sunil G
> Attachments: 0001-HADOOP-12321.patch, 0002-HADOOP-12321.patch,
> HADOOP-12321-003.patch
>
> Original Estimate: 1h
> Remaining Estimate: 1h
>
> The new JVM pause monitor has been written with its own start/stop lifecycle
> which has already proven brittle to both ordering of operations and, even
> after HADOOP-12313, is not thread safe (both start and stop are potentially
> re-entrant).
> It also requires every class which supports the monitor to add another field
> and perform the lifecycle operations in its own lifecycle, which, for all
> Yarn services, is the YARN app lifecycle (as implemented in Hadoop common)
> Making the monitor a subclass of {{AbstractService}} and moving the
> init/start & stop operations in {{serviceInit()}}, {{serviceStart()}} &
> {{serviceStop()}} methods will fix the concurrency and state model issues,
> and make it trivial to add as a child to any YARN service which subclasses
> {{CompositeService}} (most the NM and RM apps) will be able to hook up the
> monitor simply by creating one in the ctor and adding it as a child.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HADOOP-12386) RetryPolicies.RETRY_FOREVER should be able to specify a retry interval
[ https://issues.apache.org/jira/browse/HADOOP-12386?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sunil G updated HADOOP-12386: - Attachment: 0002-HADOOP-12386.patch Thank you [~leftnoteasy] I think that makes sense to me. It will help to avoid duplicate handling. Updating a patch for same. > RetryPolicies.RETRY_FOREVER should be able to specify a retry interval > -- > > Key: HADOOP-12386 > URL: https://issues.apache.org/jira/browse/HADOOP-12386 > Project: Hadoop Common > Issue Type: Bug >Reporter: Wangda Tan >Assignee: Sunil G > Attachments: 0001-HADOOP-12386.patch, 0002-HADOOP-12386.patch > > > Problems mentioned in YARN-4113, We should be able to specify retry interval > in RetryPolicies.RETRY_FOREVER. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HADOOP-12386) RetryPolicies.RETRY_FOREVER should be able to specify a retry interval
[ https://issues.apache.org/jira/browse/HADOOP-12386?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sunil G updated HADOOP-12386: - Attachment: 0001-HADOOP-12386.patch Attaching an initial version. Kindly help to check. > RetryPolicies.RETRY_FOREVER should be able to specify a retry interval > -- > > Key: HADOOP-12386 > URL: https://issues.apache.org/jira/browse/HADOOP-12386 > Project: Hadoop Common > Issue Type: Bug >Reporter: Wangda Tan >Assignee: Sunil G > Attachments: 0001-HADOOP-12386.patch > > > Problems mentioned in YARN-4113, We should be able to specify retry interval > in RetryPolicies.RETRY_FOREVER. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HADOOP-12386) RetryPolicies.RETRY_FOREVER should be able to specify a retry interval
[ https://issues.apache.org/jira/browse/HADOOP-12386?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sunil G updated HADOOP-12386: - Status: Patch Available (was: Open) > RetryPolicies.RETRY_FOREVER should be able to specify a retry interval > -- > > Key: HADOOP-12386 > URL: https://issues.apache.org/jira/browse/HADOOP-12386 > Project: Hadoop Common > Issue Type: Bug >Reporter: Wangda Tan >Assignee: Sunil G > Attachments: 0001-HADOOP-12386.patch > > > Problems mentioned in YARN-4113, We should be able to specify retry interval > in RetryPolicies.RETRY_FOREVER. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Assigned] (HADOOP-12386) RetryPolicies.RETRY_FOREVER should be able to specify a retry interval
[ https://issues.apache.org/jira/browse/HADOOP-12386?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sunil G reassigned HADOOP-12386: Assignee: Sunil G > RetryPolicies.RETRY_FOREVER should be able to specify a retry interval > -- > > Key: HADOOP-12386 > URL: https://issues.apache.org/jira/browse/HADOOP-12386 > Project: Hadoop Common > Issue Type: Bug >Reporter: Wangda Tan >Assignee: Sunil G > > Problems mentioned in YARN-4113, We should be able to specify retry interval > in RetryPolicies.RETRY_FOREVER. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HADOOP-12386) RetryPolicies.RETRY_FOREVER should be able to specify a retry interval
[ https://issues.apache.org/jira/browse/HADOOP-12386?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14735246#comment-14735246 ] Sunil G commented on HADOOP-12386: -- Thanks [~leftnoteasy] for filing this. I have done some initial work in this in YARN-4113, pls reassign otherwise. > RetryPolicies.RETRY_FOREVER should be able to specify a retry interval > -- > > Key: HADOOP-12386 > URL: https://issues.apache.org/jira/browse/HADOOP-12386 > Project: Hadoop Common > Issue Type: Bug >Reporter: Wangda Tan >Assignee: Sunil G > > Problems mentioned in YARN-4113, We should be able to specify retry interval > in RetryPolicies.RETRY_FOREVER. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HADOOP-12321) Make JvmPauseMonitor to AbstractService
[
https://issues.apache.org/jira/browse/HADOOP-12321?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14727834#comment-14727834
]
Sunil G commented on HADOOP-12321:
--
Hi [~steve_l]
HDFS-8947, MAPREDUCE-6462 and YARN-4072 are the dependent JIRAS. With all this
patches, we can compile the changed patch. Could you please help to check the
same.
> Make JvmPauseMonitor to AbstractService
> ---
>
> Key: HADOOP-12321
> URL: https://issues.apache.org/jira/browse/HADOOP-12321
> Project: Hadoop Common
> Issue Type: New Feature
>Affects Versions: 2.8.0
>Reporter: Steve Loughran
>Assignee: Sunil G
> Attachments: 0001-HADOOP-12321.patch, 0002-HADOOP-12321.patch
>
> Original Estimate: 1h
> Remaining Estimate: 1h
>
> The new JVM pause monitor has been written with its own start/stop lifecycle
> which has already proven brittle to both ordering of operations and, even
> after HADOOP-12313, is not thread safe (both start and stop are potentially
> re-entrant).
> It also requires every class which supports the monitor to add another field
> and perform the lifecycle operations in its own lifecycle, which, for all
> Yarn services, is the YARN app lifecycle (as implemented in Hadoop common)
> Making the monitor a subclass of {{AbstractService}} and moving the
> init/start & stop operations in {{serviceInit()}}, {{serviceStart()}} &
> {{serviceStop()}} methods will fix the concurrency and state model issues,
> and make it trivial to add as a child to any YARN service which subclasses
> {{CompositeService}} (most the NM and RM apps) will be able to hook up the
> monitor simply by creating one in the ctor and adding it as a child.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HADOOP-12321) Make JvmPauseMonitor to AbstractService
[
https://issues.apache.org/jira/browse/HADOOP-12321?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sunil G updated HADOOP-12321:
-
Status: Open (was: Patch Available)
Make JvmPauseMonitor to AbstractService
---
Key: HADOOP-12321
URL: https://issues.apache.org/jira/browse/HADOOP-12321
Project: Hadoop Common
Issue Type: New Feature
Affects Versions: 2.8.0
Reporter: Steve Loughran
Assignee: Sunil G
Attachments: 0001-HADOOP-12321.patch
Original Estimate: 1h
Remaining Estimate: 1h
The new JVM pause monitor has been written with its own start/stop lifecycle
which has already proven brittle to both ordering of operations and, even
after HADOOP-12313, is not thread safe (both start and stop are potentially
re-entrant).
It also requires every class which supports the monitor to add another field
and perform the lifecycle operations in its own lifecycle, which, for all
Yarn services, is the YARN app lifecycle (as implemented in Hadoop common)
Making the monitor a subclass of {{AbstractService}} and moving the
init/start stop operations in {{serviceInit()}}, {{serviceStart()}}
{{serviceStop()}} methods will fix the concurrency and state model issues,
and make it trivial to add as a child to any YARN service which subclasses
{{CompositeService}} (most the NM and RM apps) will be able to hook up the
monitor simply by creating one in the ctor and adding it as a child.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HADOOP-12321) Make JvmPauseMonitor to AbstractService
[
https://issues.apache.org/jira/browse/HADOOP-12321?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sunil G updated HADOOP-12321:
-
Attachment: 0002-HADOOP-12321.patch
Thank you [~steve_l]
I have updated the patch by changing test case.
Also created 3 subjiras in HDFS, YARN and MapReduce separately and linked to
this main Jira. If all are applied together, it will compile successfully.
Make JvmPauseMonitor to AbstractService
---
Key: HADOOP-12321
URL: https://issues.apache.org/jira/browse/HADOOP-12321
Project: Hadoop Common
Issue Type: New Feature
Affects Versions: 2.8.0
Reporter: Steve Loughran
Assignee: Sunil G
Attachments: 0001-HADOOP-12321.patch, 0002-HADOOP-12321.patch
Original Estimate: 1h
Remaining Estimate: 1h
The new JVM pause monitor has been written with its own start/stop lifecycle
which has already proven brittle to both ordering of operations and, even
after HADOOP-12313, is not thread safe (both start and stop are potentially
re-entrant).
It also requires every class which supports the monitor to add another field
and perform the lifecycle operations in its own lifecycle, which, for all
Yarn services, is the YARN app lifecycle (as implemented in Hadoop common)
Making the monitor a subclass of {{AbstractService}} and moving the
init/start stop operations in {{serviceInit()}}, {{serviceStart()}}
{{serviceStop()}} methods will fix the concurrency and state model issues,
and make it trivial to add as a child to any YARN service which subclasses
{{CompositeService}} (most the NM and RM apps) will be able to hook up the
monitor simply by creating one in the ctor and adding it as a child.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HADOOP-12321) Make JvmPauseMonitor to AbstractService
[
https://issues.apache.org/jira/browse/HADOOP-12321?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sunil G updated HADOOP-12321:
-
Status: Patch Available (was: Open)
Make JvmPauseMonitor to AbstractService
---
Key: HADOOP-12321
URL: https://issues.apache.org/jira/browse/HADOOP-12321
Project: Hadoop Common
Issue Type: New Feature
Affects Versions: 2.8.0
Reporter: Steve Loughran
Assignee: Sunil G
Attachments: 0001-HADOOP-12321.patch
Original Estimate: 1h
Remaining Estimate: 1h
The new JVM pause monitor has been written with its own start/stop lifecycle
which has already proven brittle to both ordering of operations and, even
after HADOOP-12313, is not thread safe (both start and stop are potentially
re-entrant).
It also requires every class which supports the monitor to add another field
and perform the lifecycle operations in its own lifecycle, which, for all
Yarn services, is the YARN app lifecycle (as implemented in Hadoop common)
Making the monitor a subclass of {{AbstractService}} and moving the
init/start stop operations in {{serviceInit()}}, {{serviceStart()}}
{{serviceStop()}} methods will fix the concurrency and state model issues,
and make it trivial to add as a child to any YARN service which subclasses
{{CompositeService}} (most the NM and RM apps) will be able to hook up the
monitor simply by creating one in the ctor and adding it as a child.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HADOOP-12321) Make JvmPauseMonitor to AbstractService
[
https://issues.apache.org/jira/browse/HADOOP-12321?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14704303#comment-14704303
]
Sunil G commented on HADOOP-12321:
--
Thank you [~steve_l]
Test case also needs to call start(). I will add the same.
Patch wont apply for now as NameNode/DataNode instantiate
{{JvmPauseMonitor()}}. I need to keep a dummy ctor for now to make the
compilation successful.
And later this ctor can be removed once other sub tickets are in. Not very
clean though, How do u feel? Else all patches need to be there as single patch.
Make JvmPauseMonitor to AbstractService
---
Key: HADOOP-12321
URL: https://issues.apache.org/jira/browse/HADOOP-12321
Project: Hadoop Common
Issue Type: New Feature
Affects Versions: 2.8.0
Reporter: Steve Loughran
Assignee: Sunil G
Attachments: 0001-HADOOP-12321.patch
Original Estimate: 1h
Remaining Estimate: 1h
The new JVM pause monitor has been written with its own start/stop lifecycle
which has already proven brittle to both ordering of operations and, even
after HADOOP-12313, is not thread safe (both start and stop are potentially
re-entrant).
It also requires every class which supports the monitor to add another field
and perform the lifecycle operations in its own lifecycle, which, for all
Yarn services, is the YARN app lifecycle (as implemented in Hadoop common)
Making the monitor a subclass of {{AbstractService}} and moving the
init/start stop operations in {{serviceInit()}}, {{serviceStart()}}
{{serviceStop()}} methods will fix the concurrency and state model issues,
and make it trivial to add as a child to any YARN service which subclasses
{{CompositeService}} (most the NM and RM apps) will be able to hook up the
monitor simply by creating one in the ctor and adding it as a child.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HADOOP-12321) Make JvmPauseMonitor to AbstractService
[
https://issues.apache.org/jira/browse/HADOOP-12321?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sunil G updated HADOOP-12321:
-
Attachment: 0001-HADOOP-12321.patch
Uploading an initial version. I will raise other dependent jiras for hdfs,yarn
and jhs if this patch is fine.
Make JvmPauseMonitor to AbstractService
---
Key: HADOOP-12321
URL: https://issues.apache.org/jira/browse/HADOOP-12321
Project: Hadoop Common
Issue Type: New Feature
Affects Versions: 2.8.0
Reporter: Steve Loughran
Assignee: Sunil G
Attachments: 0001-HADOOP-12321.patch
Original Estimate: 1h
Remaining Estimate: 1h
The new JVM pause monitor has been written with its own start/stop lifecycle
which has already proven brittle to both ordering of operations and, even
after HADOOP-12313, is not thread safe (both start and stop are potentially
re-entrant).
It also requires every class which supports the monitor to add another field
and perform the lifecycle operations in its own lifecycle, which, for all
Yarn services, is the YARN app lifecycle (as implemented in Hadoop common)
Making the monitor a subclass of {{AbstractService}} and moving the
init/start stop operations in {{serviceInit()}}, {{serviceStart()}}
{{serviceStop()}} methods will fix the concurrency and state model issues,
and make it trivial to add as a child to any YARN service which subclasses
{{CompositeService}} (most the NM and RM apps) will be able to hook up the
monitor simply by creating one in the ctor and adding it as a child.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HADOOP-12321) Make JvmPauseMonitor to AbstractService
[
https://issues.apache.org/jira/browse/HADOOP-12321?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14695570#comment-14695570
]
Sunil G commented on HADOOP-12321:
--
I think this change definitely helps in resolving the current complexities in
the use cases of JvmMonitor. I would like to give a try if its fine. :)
Make JvmPauseMonitor to AbstractService
---
Key: HADOOP-12321
URL: https://issues.apache.org/jira/browse/HADOOP-12321
Project: Hadoop Common
Issue Type: New Feature
Affects Versions: 2.8.0
Reporter: Steve Loughran
Original Estimate: 1h
Remaining Estimate: 1h
The new JVM pause monitor has been written with its own start/stop lifecycle
which has already proven brittle to both ordering of operations and, even
after HADOOP-12313, is not thread safe (both start and stop are potentially
re-entrant).
It also requires every class which supports the monitor to add another field
and perform the lifecycle operations in its own lifecycle, which, for all
Yarn services, is the YARN app lifecycle (as implemented in Hadoop common)
Making the monitor a subclass of {{AbstractService}} and moving the
init/start stop operations in {{serviceInit()}}, {{serviceStart()}}
{{serviceStop()}} methods will fix the concurrency and state model issues,
and make it trivial to add as a child to any YARN service which subclasses
{{CompositeService}} (most the NM and RM apps) will be able to hook up the
monitor simply by creating one in the ctor and adding it as a child.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HADOOP-12321) Make JvmPauseMonitor to AbstractService
[
https://issues.apache.org/jira/browse/HADOOP-12321?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14696514#comment-14696514
]
Sunil G commented on HADOOP-12321:
--
Thank you [~rkanter], I will make the changes in JvmPauseMonitor and will
update the invocation for various modules as per new service way.
Make JvmPauseMonitor to AbstractService
---
Key: HADOOP-12321
URL: https://issues.apache.org/jira/browse/HADOOP-12321
Project: Hadoop Common
Issue Type: New Feature
Affects Versions: 2.8.0
Reporter: Steve Loughran
Assignee: Sunil G
Original Estimate: 1h
Remaining Estimate: 1h
The new JVM pause monitor has been written with its own start/stop lifecycle
which has already proven brittle to both ordering of operations and, even
after HADOOP-12313, is not thread safe (both start and stop are potentially
re-entrant).
It also requires every class which supports the monitor to add another field
and perform the lifecycle operations in its own lifecycle, which, for all
Yarn services, is the YARN app lifecycle (as implemented in Hadoop common)
Making the monitor a subclass of {{AbstractService}} and moving the
init/start stop operations in {{serviceInit()}}, {{serviceStart()}}
{{serviceStop()}} methods will fix the concurrency and state model issues,
and make it trivial to add as a child to any YARN service which subclasses
{{CompositeService}} (most the NM and RM apps) will be able to hook up the
monitor simply by creating one in the ctor and adding it as a child.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
