[jira] [Updated] (HADOOP-18088) Replace log4j 1.x with reload4j
[ https://issues.apache.org/jira/browse/HADOOP-18088?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Masatake Iwasaki updated HADOOP-18088: -- Fix Version/s: (was: 3.5.0) > Replace log4j 1.x with reload4j > --- > > Key: HADOOP-18088 > URL: https://issues.apache.org/jira/browse/HADOOP-18088 > Project: Hadoop Common > Issue Type: Improvement >Reporter: Wei-Chiu Chuang >Assignee: Wei-Chiu Chuang >Priority: Major > Labels: pull-request-available > Fix For: 3.4.0, 2.10.2, 3.2.4, 3.3.3 > > Time Spent: 8h > Remaining Estimate: 0h > > As proposed in the dev mailing list > (https://lists.apache.org/thread/fdzkv80mzkf3w74z9120l0k0rc3v7kqk) let's > replace log4j 1 with reload4j in the maintenance releases (i.e. 3.3.x, 3.2.x > and 2.10.x) -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-18088) Replace log4j 1.x with reload4j
[ https://issues.apache.org/jira/browse/HADOOP-18088?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Masatake Iwasaki updated HADOOP-18088: -- Fix Version/s: (was: 3.4.1) > Replace log4j 1.x with reload4j > --- > > Key: HADOOP-18088 > URL: https://issues.apache.org/jira/browse/HADOOP-18088 > Project: Hadoop Common > Issue Type: Improvement >Reporter: Wei-Chiu Chuang >Assignee: Wei-Chiu Chuang >Priority: Major > Labels: pull-request-available > Fix For: 3.4.0, 2.10.2, 3.2.4, 3.3.3, 3.5.0 > > Time Spent: 8h > Remaining Estimate: 0h > > As proposed in the dev mailing list > (https://lists.apache.org/thread/fdzkv80mzkf3w74z9120l0k0rc3v7kqk) let's > replace log4j 1 with reload4j in the maintenance releases (i.e. 3.3.x, 3.2.x > and 2.10.x) -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-18088) Replace log4j 1.x with reload4j
[ https://issues.apache.org/jira/browse/HADOOP-18088?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Masatake Iwasaki updated HADOOP-18088: -- Fix Version/s: 3.5.0 3.4.1 > Replace log4j 1.x with reload4j > --- > > Key: HADOOP-18088 > URL: https://issues.apache.org/jira/browse/HADOOP-18088 > Project: Hadoop Common > Issue Type: Improvement >Reporter: Wei-Chiu Chuang >Assignee: Wei-Chiu Chuang >Priority: Major > Labels: pull-request-available > Fix For: 2.10.2, 3.2.4, 3.3.3, 3.5.0, 3.4.1 > > Time Spent: 8h > Remaining Estimate: 0h > > As proposed in the dev mailing list > (https://lists.apache.org/thread/fdzkv80mzkf3w74z9120l0k0rc3v7kqk) let's > replace log4j 1 with reload4j in the maintenance releases (i.e. 3.3.x, 3.2.x > and 2.10.x) -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Comment Edited] (HADOOP-15984) Update jersey from 1.19 to 2.x
[
https://issues.apache.org/jira/browse/HADOOP-15984?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17816869#comment-17816869
]
Masatake Iwasaki edited comment on HADOOP-15984 at 2/13/24 6:48 AM:
{quote}The reason this JIRA is particularly challenging is that the Hadoop
project cannot simultaneously support Jersey 1 and Jersey 2. I am considering
introducing Jersey 2 into Hadoop third-party dependencies and shading the
related paths to ensure that Jersey 1 and Jersey 2 can be introduced into the
Hadoop project simultaneously.
{quote}
[~slfan1989] IIUC, we need some hacks in class loading mechanism to make JAX-RS
1.1 and 2.1 co-exist as [~aajisaka] mentioned. Since Jersey 1.x and 2.x have
different package name and groupId, we usually don't need relocation for them.
*
[https://stackoverflow.com/questions/29632455/getting-jersey-1-x-and-2-x-to-coexist]
*
[https://download.oracle.com/javaee-archive/jersey.java.net/users/2014/04/18550.html]
was (Author: iwasakims):
{quote}
The reason this JIRA is particularly challenging is that the Hadoop project
cannot simultaneously support Jersey 1 and Jersey 2. I am considering
introducing Jersey 2 into Hadoop third-party dependencies and shading the
related paths to ensure that Jersey 1 and Jersey 2 can be introduced into the
Hadoop project simultaneously.
{quote}
[~slfan1989] IIUC, we need some hacks in class loading mechanism to make JAX-RS
1.1 and 2.0 co-exist as [~aajisaka] mentioned. Since Jersey 1.x and 2.x have
different package name and groupId, we usually don't need relocation for them.
*
https://stackoverflow.com/questions/29632455/getting-jersey-1-x-and-2-x-to-coexist
*
https://download.oracle.com/javaee-archive/jersey.java.net/users/2014/04/18550.html
> Update jersey from 1.19 to 2.x
> --
>
> Key: HADOOP-15984
> URL: https://issues.apache.org/jira/browse/HADOOP-15984
> Project: Hadoop Common
> Issue Type: Improvement
>Reporter: Akira Ajisaka
>Priority: Major
> Labels: pull-request-available
> Time Spent: 2h 10m
> Remaining Estimate: 0h
>
> jersey-json 1.19 depends on Jackson 1.9.2. Let's upgrade.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-15984) Update jersey from 1.19 to 2.x
[
https://issues.apache.org/jira/browse/HADOOP-15984?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17816869#comment-17816869
]
Masatake Iwasaki commented on HADOOP-15984:
---
{quote}
The reason this JIRA is particularly challenging is that the Hadoop project
cannot simultaneously support Jersey 1 and Jersey 2. I am considering
introducing Jersey 2 into Hadoop third-party dependencies and shading the
related paths to ensure that Jersey 1 and Jersey 2 can be introduced into the
Hadoop project simultaneously.
{quote}
[~slfan1989] IIUC, we need some hacks in class loading mechanism to make JAX-RS
1.1 and 2.0 co-exist as [~aajisaka] mentioned. Since Jersey 1.x and 2.x have
different package name and groupId, we usually don't need relocation for them.
*
https://stackoverflow.com/questions/29632455/getting-jersey-1-x-and-2-x-to-coexist
*
https://download.oracle.com/javaee-archive/jersey.java.net/users/2014/04/18550.html
> Update jersey from 1.19 to 2.x
> --
>
> Key: HADOOP-15984
> URL: https://issues.apache.org/jira/browse/HADOOP-15984
> Project: Hadoop Common
> Issue Type: Improvement
>Reporter: Akira Ajisaka
>Priority: Major
> Labels: pull-request-available
> Time Spent: 2h 10m
> Remaining Estimate: 0h
>
> jersey-json 1.19 depends on Jackson 1.9.2. Let's upgrade.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-17919) Fix command line example in Hadoop Cluster Setup documentation
[
https://issues.apache.org/jira/browse/HADOOP-17919?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17809208#comment-17809208
]
Masatake Iwasaki commented on HADOOP-17919:
---
[~slfan1989] I think we should keep 3.4.0 in the "Fix Version/s" as we usually
do for clarity. Since both branch-3.3 and branch-3.4 are maintained, we can not
assume that 3.4.0 contains all fixes of 3.3.x (and previous patch releases).
> Fix command line example in Hadoop Cluster Setup documentation
> --
>
> Key: HADOOP-17919
> URL: https://issues.apache.org/jira/browse/HADOOP-17919
> Project: Hadoop Common
> Issue Type: Bug
> Components: documentation
>Affects Versions: 3.3.1, 3.4.0
>Reporter: Rintaro Ikeda
>Assignee: Rintaro Ikeda
>Priority: Minor
> Labels: docuentation, pull-request-available
> Fix For: 3.3.2, 3.2.4
>
> Time Spent: 40m
> Remaining Estimate: 0h
>
> About Hadoop cluster setup documentation
> ([https://hadoop.apache.org/docs/stable/hadoop-project-dist/hadoop-common/ClusterSetup.html])
> The option is specified in the following example, but HDFS
> command ignores it.
> {noformat}
> `[hdfs]$ $HADOOP_HOME/bin/hdfs namenode -format `
> {noformat}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-13816) Ambiguous plugin version warning from maven build.
[
https://issues.apache.org/jira/browse/HADOOP-13816?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Masatake Iwasaki updated HADOOP-13816:
--
Fix Version/s: 2.10.3
> Ambiguous plugin version warning from maven build.
> --
>
> Key: HADOOP-13816
> URL: https://issues.apache.org/jira/browse/HADOOP-13816
> Project: Hadoop Common
> Issue Type: Bug
>Affects Versions: 3.0.0-alpha2
>Reporter: Kai
>Assignee: Kai
>Priority: Minor
> Fix For: 3.0.0-alpha2, 2.10.3
>
> Attachments: HADOOP-13816.01.patch
>
>
> When we try to build Hadoop with maven, the below warning is shown.
> {code}
> [WARNING]
> [WARNING] Some problems were encountered while building the effective model
> for org.apache.hadoop:hadoop-rumen:jar:3.0.0-alpha2-SNAPSHOT
> [WARNING] 'dependencies.dependency.(groupId:artifactId:type:classifier)' must
> be unique: com.fasterxml.jackson.core:jackson-databind:jar -> duplicate
> declaration of version (?) @ line 102, column 17
> [WARNING]
> [WARNING] Some problems were encountered while building the effective model
> for org.apache.hadoop:hadoop-build-tools:jar:3.0.0-alpha2-SNAPSHOT
> [WARNING] 'build.plugins.plugin.version' for
> org.apache.maven.plugins:maven-remote-resources-plugin is missing. @
> org.apache.hadoop:hadoop-build-tools:[unknown-version],
> /Users/sasakikai/dev/hadoop/hadoop-build-tools/pom.xml, line 80, column 15
> [WARNING] 'build.plugins.plugin.version' for
> org.apache.maven.plugins:maven-resources-plugin is missing. @
> org.apache.hadoop:hadoop-build-tools:[unknown-version],
> /Users/sasakikai/dev/hadoop/hadoop-build-tools/pom.xml, line 54, column 15
> {code}
> It is required to
> - remove duplicated definition {{jackson-databind}}
> - specify the version {{maven-resource-plugin}} and
> {{maven-remote-resources-plugin}}.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-13816) Ambiguous plugin version warning from maven build.
[
https://issues.apache.org/jira/browse/HADOOP-13816?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17793758#comment-17793758
]
Masatake Iwasaki commented on HADOOP-13816:
---
cherry-picked this to branch-2.10 in order to fix the error on running
dev-support/bin/create-release script.
{noformat}
$ less patchprocess/mvn_install_maven_plugins.log
...
[ESC[1;33mWARNINGESC[m] Some problems were encountered while building the
effective model for org.apache.hadoop:hadoop-build-tools:jar:2.10.3-SNAPSHOT
[ESC[1;33mWARNINGESC[m] 'build.plugins.plugin.version' for
org.apache.maven.plugins:maven-resources-plugin is missing. @
org.apache.hadoop:hadoop-build-tools:[unknown-version],
/build/source/hadoop-build-tools/pom.xml, line 54, column 15
[ESC[1;33mWARNINGESC[m] 'build.plugins.plugin.version' for
org.apache.maven.plugins:maven-remote-resources-plugin is missing. @
org.apache.hadoop:hadoop-build-tools:[unknown-version],
/build/source/hadoop-build-tools/pom.xml, line 80, column 15
...
[ESC[1;31mERRORESC[m] Failed to execute goal
ESC[32morg.apache.maven.plugins:maven-remote-resources-plugin:3.1.0:bundleESC[m
ESC[1m(default)ESC[m on project ESC[36mhadoop-build-toolsESC[m:
ESC[1;31mExecution default of goal
org.apache.maven.plugins:maven-remote-resources-plugin:3.1.0:bundle failed:\
Unable to load the mojo 'bundle' in the plugin
'org.apache.maven.plugins:maven-remote-resources-plugin:3.1.0' due to an API
incompatibility:
org.codehaus.plexus.component.repository.exception.ComponentLookupException:
org/apache/maven/plugin/resources/remote/BundleRemoteResourcesMojo :
Unsupported\
major.minor version 52.0ESC[m
{noformat}
> Ambiguous plugin version warning from maven build.
> --
>
> Key: HADOOP-13816
> URL: https://issues.apache.org/jira/browse/HADOOP-13816
> Project: Hadoop Common
> Issue Type: Bug
>Affects Versions: 3.0.0-alpha2
>Reporter: Kai
>Assignee: Kai
>Priority: Minor
> Fix For: 3.0.0-alpha2
>
> Attachments: HADOOP-13816.01.patch
>
>
> When we try to build Hadoop with maven, the below warning is shown.
> {code}
> [WARNING]
> [WARNING] Some problems were encountered while building the effective model
> for org.apache.hadoop:hadoop-rumen:jar:3.0.0-alpha2-SNAPSHOT
> [WARNING] 'dependencies.dependency.(groupId:artifactId:type:classifier)' must
> be unique: com.fasterxml.jackson.core:jackson-databind:jar -> duplicate
> declaration of version (?) @ line 102, column 17
> [WARNING]
> [WARNING] Some problems were encountered while building the effective model
> for org.apache.hadoop:hadoop-build-tools:jar:3.0.0-alpha2-SNAPSHOT
> [WARNING] 'build.plugins.plugin.version' for
> org.apache.maven.plugins:maven-remote-resources-plugin is missing. @
> org.apache.hadoop:hadoop-build-tools:[unknown-version],
> /Users/sasakikai/dev/hadoop/hadoop-build-tools/pom.xml, line 80, column 15
> [WARNING] 'build.plugins.plugin.version' for
> org.apache.maven.plugins:maven-resources-plugin is missing. @
> org.apache.hadoop:hadoop-build-tools:[unknown-version],
> /Users/sasakikai/dev/hadoop/hadoop-build-tools/pom.xml, line 54, column 15
> {code}
> It is required to
> - remove duplicated definition {{jackson-databind}}
> - specify the version {{maven-resource-plugin}} and
> {{maven-remote-resources-plugin}}.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Resolved] (HADOOP-18941) Modify HBase version in BUILDING.txt
[ https://issues.apache.org/jira/browse/HADOOP-18941?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Masatake Iwasaki resolved HADOOP-18941. --- Fix Version/s: 3.4.0 Hadoop Flags: Reviewed Resolution: Fixed > Modify HBase version in BUILDING.txt > > > Key: HADOOP-18941 > URL: https://issues.apache.org/jira/browse/HADOOP-18941 > Project: Hadoop Common > Issue Type: Bug >Reporter: Zepeng Zhang >Assignee: Zepeng Zhang >Priority: Minor > Labels: pull-request-available > Fix For: 3.4.0 > > > In current BUILDING.txt document, the version of HBase, which is used by YARN > Timeline Service V2 is a bit older than the actual one. Hence, I hereby > request to modify this uncertain description in the document. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-18942) Upgrade ZooKeeper to 3.7.2
[ https://issues.apache.org/jira/browse/HADOOP-18942?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Masatake Iwasaki updated HADOOP-18942: -- Fix Version/s: 3.3.7 > Upgrade ZooKeeper to 3.7.2 > -- > > Key: HADOOP-18942 > URL: https://issues.apache.org/jira/browse/HADOOP-18942 > Project: Hadoop Common > Issue Type: Improvement >Reporter: Masatake Iwasaki >Assignee: Masatake Iwasaki >Priority: Major > Labels: pull-request-available > Fix For: 3.4.0, 3.3.7 > > > While HADOOP-18613 is proposing to upgrade ZooKeeper to 3.8, it will bring > dependency conflicts. Upgrading to ZooKeeper 3.7 could be alternative > short-term fix for addressing CVEs. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Resolved] (HADOOP-18942) Upgrade ZooKeeper to 3.7.2
[ https://issues.apache.org/jira/browse/HADOOP-18942?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Masatake Iwasaki resolved HADOOP-18942. --- Fix Version/s: 3.4.0 Hadoop Flags: Reviewed Resolution: Fixed > Upgrade ZooKeeper to 3.7.2 > -- > > Key: HADOOP-18942 > URL: https://issues.apache.org/jira/browse/HADOOP-18942 > Project: Hadoop Common > Issue Type: Improvement >Reporter: Masatake Iwasaki >Assignee: Masatake Iwasaki >Priority: Major > Labels: pull-request-available > Fix For: 3.4.0 > > > While HADOOP-18613 is proposing to upgrade ZooKeeper to 3.8, it will bring > dependency conflicts. Upgrading to ZooKeeper 3.7 could be alternative > short-term fix for addressing CVEs. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-18942) Upgrade ZooKeeper to 3.7.2
[ https://issues.apache.org/jira/browse/HADOOP-18942?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17776491#comment-17776491 ] Masatake Iwasaki commented on HADOOP-18942: --- Thanks, [~groot]. I've already worked on it and submitted a PR. > Upgrade ZooKeeper to 3.7.2 > -- > > Key: HADOOP-18942 > URL: https://issues.apache.org/jira/browse/HADOOP-18942 > Project: Hadoop Common > Issue Type: Improvement >Reporter: Masatake Iwasaki >Assignee: Masatake Iwasaki >Priority: Major > Labels: pull-request-available > > While HADOOP-18613 is proposing to upgrade ZooKeeper to 3.8, it will bring > dependency conflicts. Upgrading to ZooKeeper 3.7 could be alternative > short-term fix for addressing CVEs. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-18867) Upgrade ZooKeeper to 3.6.4
[
https://issues.apache.org/jira/browse/HADOOP-18867?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Masatake Iwasaki updated HADOOP-18867:
--
Fix Version/s: 3.3.7
> Upgrade ZooKeeper to 3.6.4
> --
>
> Key: HADOOP-18867
> URL: https://issues.apache.org/jira/browse/HADOOP-18867
> Project: Hadoop Common
> Issue Type: Improvement
> Components: build
>Affects Versions: 3.3.6
>Reporter: Masatake Iwasaki
>Assignee: Masatake Iwasaki
>Priority: Minor
> Labels: pull-request-available
> Fix For: 3.4.0, 3.3.7
>
>
> While ZooKeeper 3.6 is already EOL, we can upgrade to the final release of
> the ZooKeeper 3.6 as short-term fix until bumping to ZooKeeper 3.7 or later.
> Dependency convergence error must be addressed on {{-Dhbase.profile=2.0}}.
> {noformat}
> $ mvn clean install -Dzookeeper.version=3.6.4 -Dhbase.profile=2.0 -DskipTests
> clean install
> Dependency convergence error for
> org.apache.yetus:audience-annotations:jar:0.13.0:compile paths to dependency
> are:
> +-org.apache.hadoop:hadoop-yarn-server-timelineservice-hbase-common:jar:3.4.0-SNAPSHOT
> +-org.apache.hadoop:hadoop-common:test-jar:tests:3.4.0-SNAPSHOT:test
> +-org.apache.zookeeper:zookeeper:jar:3.6.4:compile
> +-org.apache.zookeeper:zookeeper-jute:jar:3.6.4:compile
> +-org.apache.yetus:audience-annotations:jar:0.13.0:compile
> and
> +-org.apache.hadoop:hadoop-yarn-server-timelineservice-hbase-common:jar:3.4.0-SNAPSHOT
> +-org.apache.hadoop:hadoop-common:test-jar:tests:3.4.0-SNAPSHOT:test
> +-org.apache.zookeeper:zookeeper:jar:3.6.4:compile
> +-org.apache.yetus:audience-annotations:jar:0.13.0:compile
> and
> +-org.apache.hadoop:hadoop-yarn-server-timelineservice-hbase-common:jar:3.4.0-SNAPSHOT
> +-org.apache.hbase:hbase-common:jar:2.2.4:compile
> +-org.apache.yetus:audience-annotations:jar:0.5.0:compile
> {noformat}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Resolved] (HADOOP-18867) Upgrade ZooKeeper to 3.6.4
[
https://issues.apache.org/jira/browse/HADOOP-18867?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Masatake Iwasaki resolved HADOOP-18867.
---
Fix Version/s: 3.4.0
Hadoop Flags: Reviewed
Resolution: Fixed
> Upgrade ZooKeeper to 3.6.4
> --
>
> Key: HADOOP-18867
> URL: https://issues.apache.org/jira/browse/HADOOP-18867
> Project: Hadoop Common
> Issue Type: Improvement
> Components: build
>Affects Versions: 3.3.6
>Reporter: Masatake Iwasaki
>Assignee: Masatake Iwasaki
>Priority: Minor
> Labels: pull-request-available
> Fix For: 3.4.0
>
>
> While ZooKeeper 3.6 is already EOL, we can upgrade to the final release of
> the ZooKeeper 3.6 as short-term fix until bumping to ZooKeeper 3.7 or later.
> Dependency convergence error must be addressed on {{-Dhbase.profile=2.0}}.
> {noformat}
> $ mvn clean install -Dzookeeper.version=3.6.4 -Dhbase.profile=2.0 -DskipTests
> clean install
> Dependency convergence error for
> org.apache.yetus:audience-annotations:jar:0.13.0:compile paths to dependency
> are:
> +-org.apache.hadoop:hadoop-yarn-server-timelineservice-hbase-common:jar:3.4.0-SNAPSHOT
> +-org.apache.hadoop:hadoop-common:test-jar:tests:3.4.0-SNAPSHOT:test
> +-org.apache.zookeeper:zookeeper:jar:3.6.4:compile
> +-org.apache.zookeeper:zookeeper-jute:jar:3.6.4:compile
> +-org.apache.yetus:audience-annotations:jar:0.13.0:compile
> and
> +-org.apache.hadoop:hadoop-yarn-server-timelineservice-hbase-common:jar:3.4.0-SNAPSHOT
> +-org.apache.hadoop:hadoop-common:test-jar:tests:3.4.0-SNAPSHOT:test
> +-org.apache.zookeeper:zookeeper:jar:3.6.4:compile
> +-org.apache.yetus:audience-annotations:jar:0.13.0:compile
> and
> +-org.apache.hadoop:hadoop-yarn-server-timelineservice-hbase-common:jar:3.4.0-SNAPSHOT
> +-org.apache.hbase:hbase-common:jar:2.2.4:compile
> +-org.apache.yetus:audience-annotations:jar:0.5.0:compile
> {noformat}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Created] (HADOOP-18942) Upgrade ZooKeeper to 3.7.2
Masatake Iwasaki created HADOOP-18942: - Summary: Upgrade ZooKeeper to 3.7.2 Key: HADOOP-18942 URL: https://issues.apache.org/jira/browse/HADOOP-18942 Project: Hadoop Common Issue Type: Improvement Reporter: Masatake Iwasaki Assignee: Masatake Iwasaki While HADOOP-18613 is proposing to upgrade ZooKeeper to 3.8, it will bring dependency conflicts. Upgrading to ZooKeeper 3.7 could be alternative short-term fix for addressing CVEs. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-18867) Upgrade ZooKeeper to 3.6.4
[
https://issues.apache.org/jira/browse/HADOOP-18867?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17761668#comment-17761668
]
Masatake Iwasaki commented on HADOOP-18867:
---
YARN-11558 should be fixed first for successful build in trunk.
> Upgrade ZooKeeper to 3.6.4
> --
>
> Key: HADOOP-18867
> URL: https://issues.apache.org/jira/browse/HADOOP-18867
> Project: Hadoop Common
> Issue Type: Improvement
> Components: build
>Affects Versions: 3.3.6
>Reporter: Masatake Iwasaki
>Assignee: Masatake Iwasaki
>Priority: Minor
> Labels: pull-request-available
>
> While ZooKeeper 3.6 is already EOL, we can upgrade to the final release of
> the ZooKeeper 3.6 as short-term fix until bumping to ZooKeeper 3.7 or later.
> Dependency convergence error must be addressed on {{-Dhbase.profile=2.0}}.
> {noformat}
> $ mvn clean install -Dzookeeper.version=3.6.4 -Dhbase.profile=2.0 -DskipTests
> clean install
> Dependency convergence error for
> org.apache.yetus:audience-annotations:jar:0.13.0:compile paths to dependency
> are:
> +-org.apache.hadoop:hadoop-yarn-server-timelineservice-hbase-common:jar:3.4.0-SNAPSHOT
> +-org.apache.hadoop:hadoop-common:test-jar:tests:3.4.0-SNAPSHOT:test
> +-org.apache.zookeeper:zookeeper:jar:3.6.4:compile
> +-org.apache.zookeeper:zookeeper-jute:jar:3.6.4:compile
> +-org.apache.yetus:audience-annotations:jar:0.13.0:compile
> and
> +-org.apache.hadoop:hadoop-yarn-server-timelineservice-hbase-common:jar:3.4.0-SNAPSHOT
> +-org.apache.hadoop:hadoop-common:test-jar:tests:3.4.0-SNAPSHOT:test
> +-org.apache.zookeeper:zookeeper:jar:3.6.4:compile
> +-org.apache.yetus:audience-annotations:jar:0.13.0:compile
> and
> +-org.apache.hadoop:hadoop-yarn-server-timelineservice-hbase-common:jar:3.4.0-SNAPSHOT
> +-org.apache.hbase:hbase-common:jar:2.2.4:compile
> +-org.apache.yetus:audience-annotations:jar:0.5.0:compile
> {noformat}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Created] (HADOOP-18867) Upgrade ZooKeeper to 3.6.4
Masatake Iwasaki created HADOOP-18867:
-
Summary: Upgrade ZooKeeper to 3.6.4
Key: HADOOP-18867
URL: https://issues.apache.org/jira/browse/HADOOP-18867
Project: Hadoop Common
Issue Type: Improvement
Reporter: Masatake Iwasaki
Assignee: Masatake Iwasaki
While ZooKeeper 3.6 is already EOL, we can upgrade to the final release of the
ZooKeeper 3.6 as short-term fix until bumping to ZooKeeper 3.7 or later.
Dependency convergence error must be addressed on {{-Dhbase.profile=2.0}}.
{noformat}
$ mvn clean install -Dzookeeper.version=3.6.4 -Dhbase.profile=2.0 -DskipTests
clean install
Dependency convergence error for
org.apache.yetus:audience-annotations:jar:0.13.0:compile paths to dependency
are:
+-org.apache.hadoop:hadoop-yarn-server-timelineservice-hbase-common:jar:3.4.0-SNAPSHOT
+-org.apache.hadoop:hadoop-common:test-jar:tests:3.4.0-SNAPSHOT:test
+-org.apache.zookeeper:zookeeper:jar:3.6.4:compile
+-org.apache.zookeeper:zookeeper-jute:jar:3.6.4:compile
+-org.apache.yetus:audience-annotations:jar:0.13.0:compile
and
+-org.apache.hadoop:hadoop-yarn-server-timelineservice-hbase-common:jar:3.4.0-SNAPSHOT
+-org.apache.hadoop:hadoop-common:test-jar:tests:3.4.0-SNAPSHOT:test
+-org.apache.zookeeper:zookeeper:jar:3.6.4:compile
+-org.apache.yetus:audience-annotations:jar:0.13.0:compile
and
+-org.apache.hadoop:hadoop-yarn-server-timelineservice-hbase-common:jar:3.4.0-SNAPSHOT
+-org.apache.hbase:hbase-common:jar:2.2.4:compile
+-org.apache.yetus:audience-annotations:jar:0.5.0:compile
{noformat}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-17719) DistCp with snapshot diff should support file systems supporting getSnapshotDiffReport
[
https://issues.apache.org/jira/browse/HADOOP-17719?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Masatake Iwasaki updated HADOOP-17719:
--
Resolution: Duplicate
Status: Resolved (was: Patch Available)
closing as duplicate of HDFS-16911
> DistCp with snapshot diff should support file systems supporting
> getSnapshotDiffReport
> ---
>
> Key: HADOOP-17719
> URL: https://issues.apache.org/jira/browse/HADOOP-17719
> Project: Hadoop Common
> Issue Type: Improvement
>Reporter: Masatake Iwasaki
>Assignee: Masatake Iwasaki
>Priority: Major
> Labels: pull-request-available
> Time Spent: 1h 10m
> Remaining Estimate: 0h
>
> {{distcp -diff}} currently supports only DistributedFileSystem and
> WebHdfsFileSystem. Other file system could be used if getSnapshotDiffReport
> is supported.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-18796) Lock triple-beam to the version compatible with node.js 12 to avoid compilation error
[
https://issues.apache.org/jira/browse/HADOOP-18796?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Masatake Iwasaki updated HADOOP-18796:
--
Summary: Lock triple-beam to the version compatible with node.js 12 to
avoid compilation error (was: Compilation fails with triple-beam@1.4.0: The
engine "node" is incompatible with this module. Expected version ">= 16.0.0".
Got "12.22.1")
> Lock triple-beam to the version compatible with node.js 12 to avoid
> compilation error
> -
>
> Key: HADOOP-18796
> URL: https://issues.apache.org/jira/browse/HADOOP-18796
> Project: Hadoop Common
> Issue Type: Bug
> Components: build
>Affects Versions: 3.4.0
>Reporter: Ayush Saxena
>Assignee: Masatake Iwasaki
>Priority: Major
>
> {code:java}
> 2023-07-10T07:32:20.1069856Z [INFO] yarn install v1.22.5
> 2023-07-10T07:32:20.1435049Z [INFO] info No lockfile found.
> 2023-07-10T07:32:20.1489685Z [INFO] [1/4] Resolving packages...
> 2023-07-10T07:32:20.9138642Z [INFO] warning angular-route@1.6.10: For the
> actively supported Angular, see https://www.npmjs.com/package/@angular/core.
> AngularJS support has officially ended. For extended AngularJS support
> options, see https://goo.gle/angularjs-path-forward.
> 2023-07-10T07:32:20.9433934Z [INFO] warning angular@1.6.10: For the actively
> supported Angular, see https://www.npmjs.com/package/@angular/core. AngularJS
> support has officially ended. For extended AngularJS support options, see
> https://goo.gle/angularjs-path-forward.
> 2023-07-10T07:32:21.5051267Z [INFO] [2/4] Fetching packages...
> 2023-07-10T07:32:23.2554282Z [INFO] error triple-beam@1.4.0: The engine
> "node" is incompatible with this module. Expected version ">= 16.0.0". Got
> "12.22.1"
> 2023-07-10T07:32:23.2651719Z [INFO] error Found incompatible module.
> 2023-07-10T07:32:23.2664307Z [INFO] info Visit
> https://yarnpkg.com/en/docs/cli/install for documentation about this command.
> {code}
> Ref:
> https://pipelines.actions.githubusercontent.com/serviceHosts/a3203f99-c5b3-4def-b81d-c5a8ebd0356d/_apis/pipelines/1/runs/294/signedlogcontent/2?urlExpires=2023-07-10T09%3A08%3A57.5185178Z=HMACV1=OHImbS%2FdsTBFKRxwRnV6jFPnumE0GxbB%2BalrVpAgmyw%3D
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-18796) Compilation fails with triple-beam@1.4.0: The engine "node" is incompatible with this module. Expected version ">= 16.0.0". Got "12.22.1"
[
https://issues.apache.org/jira/browse/HADOOP-18796?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17741834#comment-17741834
]
Masatake Iwasaki commented on HADOOP-18796:
---
{noformat}
[INFO] error triple-beam@1.4.1: The engine "node" is incompatible with this
module. Expected version ">= 14.0.0". Got "12.22.1"
{noformat}
Recently released triple-beam v1.4.1 relaxed node engines requirement to `">=
14.0.0"`.
https://github.com/winstonjs/triple-beam/compare/v1.4.0...v1.4.1
While upgrading node.js to 14 fixed the issue, it breaks yarn-ui. It looks not
easy to fix yarn-ui side. Pinning the triple-beam or using node.js 14 in only
hadoop-yarn-applications-catalog-webapp could be an option.
> Compilation fails with triple-beam@1.4.0: The engine "node" is incompatible
> with this module. Expected version ">= 16.0.0". Got "12.22.1"
> -
>
> Key: HADOOP-18796
> URL: https://issues.apache.org/jira/browse/HADOOP-18796
> Project: Hadoop Common
> Issue Type: Bug
> Components: build
>Affects Versions: 3.4.0
>Reporter: Ayush Saxena
>Assignee: Masatake Iwasaki
>Priority: Major
>
> {code:java}
> 2023-07-10T07:32:20.1069856Z [INFO] yarn install v1.22.5
> 2023-07-10T07:32:20.1435049Z [INFO] info No lockfile found.
> 2023-07-10T07:32:20.1489685Z [INFO] [1/4] Resolving packages...
> 2023-07-10T07:32:20.9138642Z [INFO] warning angular-route@1.6.10: For the
> actively supported Angular, see https://www.npmjs.com/package/@angular/core.
> AngularJS support has officially ended. For extended AngularJS support
> options, see https://goo.gle/angularjs-path-forward.
> 2023-07-10T07:32:20.9433934Z [INFO] warning angular@1.6.10: For the actively
> supported Angular, see https://www.npmjs.com/package/@angular/core. AngularJS
> support has officially ended. For extended AngularJS support options, see
> https://goo.gle/angularjs-path-forward.
> 2023-07-10T07:32:21.5051267Z [INFO] [2/4] Fetching packages...
> 2023-07-10T07:32:23.2554282Z [INFO] error triple-beam@1.4.0: The engine
> "node" is incompatible with this module. Expected version ">= 16.0.0". Got
> "12.22.1"
> 2023-07-10T07:32:23.2651719Z [INFO] error Found incompatible module.
> 2023-07-10T07:32:23.2664307Z [INFO] info Visit
> https://yarnpkg.com/en/docs/cli/install for documentation about this command.
> {code}
> Ref:
> https://pipelines.actions.githubusercontent.com/serviceHosts/a3203f99-c5b3-4def-b81d-c5a8ebd0356d/_apis/pipelines/1/runs/294/signedlogcontent/2?urlExpires=2023-07-10T09%3A08%3A57.5185178Z=HMACV1=OHImbS%2FdsTBFKRxwRnV6jFPnumE0GxbB%2BalrVpAgmyw%3D
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Assigned] (HADOOP-18796) Compilation fails with triple-beam@1.4.0: The engine "node" is incompatible with this module. Expected version ">= 16.0.0". Got "12.22.1"
[
https://issues.apache.org/jira/browse/HADOOP-18796?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Masatake Iwasaki reassigned HADOOP-18796:
-
Assignee: Masatake Iwasaki
> Compilation fails with triple-beam@1.4.0: The engine "node" is incompatible
> with this module. Expected version ">= 16.0.0". Got "12.22.1"
> -
>
> Key: HADOOP-18796
> URL: https://issues.apache.org/jira/browse/HADOOP-18796
> Project: Hadoop Common
> Issue Type: Bug
> Components: build
>Affects Versions: 3.4.0
>Reporter: Ayush Saxena
>Assignee: Masatake Iwasaki
>Priority: Major
>
> {code:java}
> 2023-07-10T07:32:20.1069856Z [INFO] yarn install v1.22.5
> 2023-07-10T07:32:20.1435049Z [INFO] info No lockfile found.
> 2023-07-10T07:32:20.1489685Z [INFO] [1/4] Resolving packages...
> 2023-07-10T07:32:20.9138642Z [INFO] warning angular-route@1.6.10: For the
> actively supported Angular, see https://www.npmjs.com/package/@angular/core.
> AngularJS support has officially ended. For extended AngularJS support
> options, see https://goo.gle/angularjs-path-forward.
> 2023-07-10T07:32:20.9433934Z [INFO] warning angular@1.6.10: For the actively
> supported Angular, see https://www.npmjs.com/package/@angular/core. AngularJS
> support has officially ended. For extended AngularJS support options, see
> https://goo.gle/angularjs-path-forward.
> 2023-07-10T07:32:21.5051267Z [INFO] [2/4] Fetching packages...
> 2023-07-10T07:32:23.2554282Z [INFO] error triple-beam@1.4.0: The engine
> "node" is incompatible with this module. Expected version ">= 16.0.0". Got
> "12.22.1"
> 2023-07-10T07:32:23.2651719Z [INFO] error Found incompatible module.
> 2023-07-10T07:32:23.2664307Z [INFO] info Visit
> https://yarnpkg.com/en/docs/cli/install for documentation about this command.
> {code}
> Ref:
> https://pipelines.actions.githubusercontent.com/serviceHosts/a3203f99-c5b3-4def-b81d-c5a8ebd0356d/_apis/pipelines/1/runs/294/signedlogcontent/2?urlExpires=2023-07-10T09%3A08%3A57.5185178Z=HMACV1=OHImbS%2FdsTBFKRxwRnV6jFPnumE0GxbB%2BalrVpAgmyw%3D
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-18796) Compilation fails with triple-beam@1.4.0: The engine "node" is incompatible with this module. Expected version ">= 16.0.0". Got "12.22.1"
[
https://issues.apache.org/jira/browse/HADOOP-18796?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17741817#comment-17741817
]
Masatake Iwasaki commented on HADOOP-18796:
---
[~ayushtkn] I can reproduce this on my local. Maybe cache or locally installed
node.js affects on your local environment. Let me look into the cause and fix.
> Compilation fails with triple-beam@1.4.0: The engine "node" is incompatible
> with this module. Expected version ">= 16.0.0". Got "12.22.1"
> -
>
> Key: HADOOP-18796
> URL: https://issues.apache.org/jira/browse/HADOOP-18796
> Project: Hadoop Common
> Issue Type: Bug
> Components: build
>Affects Versions: 3.4.0
>Reporter: Ayush Saxena
>Priority: Major
>
> {code:java}
> 2023-07-10T07:32:20.1069856Z [INFO] yarn install v1.22.5
> 2023-07-10T07:32:20.1435049Z [INFO] info No lockfile found.
> 2023-07-10T07:32:20.1489685Z [INFO] [1/4] Resolving packages...
> 2023-07-10T07:32:20.9138642Z [INFO] warning angular-route@1.6.10: For the
> actively supported Angular, see https://www.npmjs.com/package/@angular/core.
> AngularJS support has officially ended. For extended AngularJS support
> options, see https://goo.gle/angularjs-path-forward.
> 2023-07-10T07:32:20.9433934Z [INFO] warning angular@1.6.10: For the actively
> supported Angular, see https://www.npmjs.com/package/@angular/core. AngularJS
> support has officially ended. For extended AngularJS support options, see
> https://goo.gle/angularjs-path-forward.
> 2023-07-10T07:32:21.5051267Z [INFO] [2/4] Fetching packages...
> 2023-07-10T07:32:23.2554282Z [INFO] error triple-beam@1.4.0: The engine
> "node" is incompatible with this module. Expected version ">= 16.0.0". Got
> "12.22.1"
> 2023-07-10T07:32:23.2651719Z [INFO] error Found incompatible module.
> 2023-07-10T07:32:23.2664307Z [INFO] info Visit
> https://yarnpkg.com/en/docs/cli/install for documentation about this command.
> {code}
> Ref:
> https://pipelines.actions.githubusercontent.com/serviceHosts/a3203f99-c5b3-4def-b81d-c5a8ebd0356d/_apis/pipelines/1/runs/294/signedlogcontent/2?urlExpires=2023-07-10T09%3A08%3A57.5185178Z=HMACV1=OHImbS%2FdsTBFKRxwRnV6jFPnumE0GxbB%2BalrVpAgmyw%3D
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Comment Edited] (HADOOP-18609) Fix source code modification after building with docs profile
[
https://issues.apache.org/jira/browse/HADOOP-18609?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17682502#comment-17682502
]
Masatake Iwasaki edited comment on HADOOP-18609 at 1/31/23 11:33 AM:
-
I found this on HADOOP-18601 but turned out to be not related to the missing
Javadoc of HADOOP-18598.
This issue causes error on invocation of {{mvn site}} after {{mvn install}} like
{noformat}
$ mvn clean install -DskipTests -DskipShade -Pdocs
$ mvn site site:stage -DstagingDirectory=/var/www/html/hadoop-site/trunk
{noformat}
was (Author: iwasakims):
I found this on HADOOP-18601 but turned out to be not related to the missing
Javadoc.
This issue causes error on invocation of {{mvn site}} after {{mvn install}} like
{noformat}
$ mvn clean install -DskipTests -DskipShade -Pdocs
$ mvn site site:stage -DstagingDirectory=/var/www/html/hadoop-site/trunk
{noformat}
> Fix source code modification after building with docs profile
> -
>
> Key: HADOOP-18609
> URL: https://issues.apache.org/jira/browse/HADOOP-18609
> Project: Hadoop Common
> Issue Type: Bug
>Reporter: Masatake Iwasaki
>Assignee: Masatake Iwasaki
>Priority: Major
>
> MetricsSystem.java and MetricsSystemImpl.java are automatically modified by
> building with {{-Pdocs}}.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Resolved] (HADOOP-18598) maven site generation doesn't include javadocs
[ https://issues.apache.org/jira/browse/HADOOP-18598?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Masatake Iwasaki resolved HADOOP-18598. --- Fix Version/s: 3.4.0 3.3.5 Hadoop Flags: Reviewed Resolution: Fixed > maven site generation doesn't include javadocs > -- > > Key: HADOOP-18598 > URL: https://issues.apache.org/jira/browse/HADOOP-18598 > Project: Hadoop Common > Issue Type: Bug > Components: site >Affects Versions: 3.3.5 >Reporter: Steve Loughran >Assignee: Steve Loughran >Priority: Blocker > Labels: pull-request-available > Fix For: 3.4.0, 3.3.5 > > > the rc0 excluded all the site docs. running mvn site on trunk throws up site > plugin issues, which may be related, so start by updating that. > rc validation scripts to include checks for the api/index.html -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-18609) Fix source code modification after building with docs profile
[
https://issues.apache.org/jira/browse/HADOOP-18609?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17682502#comment-17682502
]
Masatake Iwasaki commented on HADOOP-18609:
---
I found this on HADOOP-18601 but turned out to be not related to the missing
Javadoc.
This issue causes error on invocation of {{mvn site}} after {{mvn install}} like
{noformat}
$ mvn clean install -DskipTests -DskipShade -Pdocs
$ mvn site site:stage -DstagingDirectory=/var/www/html/hadoop-site/trunk
{noformat}
> Fix source code modification after building with docs profile
> -
>
> Key: HADOOP-18609
> URL: https://issues.apache.org/jira/browse/HADOOP-18609
> Project: Hadoop Common
> Issue Type: Bug
>Reporter: Masatake Iwasaki
>Assignee: Masatake Iwasaki
>Priority: Major
>
> MetricsSystem.java and MetricsSystemImpl.java are automatically modified by
> building with {{-Pdocs}}.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-18609) Fix source code modification after building with docs profile
[
https://issues.apache.org/jira/browse/HADOOP-18609?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17682499#comment-17682499
]
Masatake Iwasaki commented on HADOOP-18609:
---
{noformat}
$ mvn clean install -DskipTests -DskipShade -Pdocs
$ git diff
diff --git
a/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/metrics2/MetricsSystem.java
b/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/metrics2/MetricsSystem.java
index fef8c4b7e4b..7b21cc302d1 100644
---
a/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/metrics2/MetricsSystem.java
+++
b/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/metrics2/MetricsSystem.java
@@ -42,6 +42,18 @@
@InterfaceAudience.Private
public abstract MetricsSystem init(String prefix);
+ /**
+ * Register a metrics source
+ * @paramthe actual type of the source object
+ * @param source object to register
+ * @param name of the source. Must be unique or null (then extracted from
+ * the annotations of the source object.)
+ * @param desc the description of the source (or null. See above.)
+ * @return the source object
+ * @exception MetricsException
+ */
+ public abstract T register(String name, String desc, T source);
+
/**
* Register a metrics source
* @paramthe actual type of the source object
diff --git
a/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/metrics2/impl/MetricsSystemImpl.java
b/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/metrics2/impl/MetricsSystemImpl.java
index 6c5a71a708f..beb72e42723 100644
---
a/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/metrics2/impl/MetricsSystemImpl.java
+++
b/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/metrics2/impl/MetricsSystemImpl.java
@@ -293,6 +293,31 @@ T register(final String name, final String description,
final T sink) {
return sink;
}
+ @Override public synchronized
+ T register(final String name, final String description, final T sink) {
+LOG.debug(name +", "+ description);
+if (allSinks.containsKey(name)) {
+ if(sinks.get(name) == null) {
+registerSink(name, description, sink);
+ } else {
+LOG.warn("Sink "+ name +" already exists!");
+ }
+ return sink;
+}
+allSinks.put(name, sink);
+if (config != null) {
+ registerSink(name, description, sink);
+}
+// We want to re-register the sink to pick up new config
+// when the metrics system restarts.
+register(name, new AbstractCallback() {
+ @Override public void postStart() {
+register(name, description, sink);
+ }
+});
+return sink;
+ }
+
synchronized void registerSink(String name, String desc, MetricsSink sink) {
checkNotNull(config, "config");
MetricsConfig conf = sinkConfigs.get(name);
{noformat}
> Fix source code modification after building with docs profile
> -
>
> Key: HADOOP-18609
> URL: https://issues.apache.org/jira/browse/HADOOP-18609
> Project: Hadoop Common
> Issue Type: Bug
>Reporter: Masatake Iwasaki
>Assignee: Masatake Iwasaki
>Priority: Major
>
> MetricsSystem.java and MetricsSystemImpl.java are automatically modified by
> building with {{-Pdocs}}.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Created] (HADOOP-18609) Fix source code modification after building with docs profile
Masatake Iwasaki created HADOOP-18609:
-
Summary: Fix source code modification after building with docs
profile
Key: HADOOP-18609
URL: https://issues.apache.org/jira/browse/HADOOP-18609
Project: Hadoop Common
Issue Type: Bug
Reporter: Masatake Iwasaki
Assignee: Masatake Iwasaki
MetricsSystem.java and MetricsSystemImpl.java are automatically modified by
building with {{-Pdocs}}.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-18598) maven site generation doesn't include javadocs
[
https://issues.apache.org/jira/browse/HADOOP-18598?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17682497#comment-17682497
]
Masatake Iwasaki commented on HADOOP-18598:
---
{quote}
rc validation scripts to include checks for the api/index.html
{quote}
[~ste...@apache.org] I found your mention about adding validation after I
merged PR#5319 as a short-term fix. I filed HADOOP-18608 as a follow-up for the
validation.
> maven site generation doesn't include javadocs
> --
>
> Key: HADOOP-18598
> URL: https://issues.apache.org/jira/browse/HADOOP-18598
> Project: Hadoop Common
> Issue Type: Bug
> Components: site
>Affects Versions: 3.3.5
>Reporter: Steve Loughran
>Assignee: Steve Loughran
>Priority: Blocker
> Labels: pull-request-available
>
> the rc0 excluded all the site docs. running mvn site on trunk throws up site
> plugin issues, which may be related, so start by updating that.
> rc validation scripts to include checks for the api/index.html
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Created] (HADOOP-18608) Add validation to check Javadocs are included in site documentation
Masatake Iwasaki created HADOOP-18608: - Summary: Add validation to check Javadocs are included in site documentation Key: HADOOP-18608 URL: https://issues.apache.org/jira/browse/HADOOP-18608 Project: Hadoop Common Issue Type: Improvement Reporter: Masatake Iwasaki We should add validation to avoid an issue like HADOOP-18598 found in release process. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-18601) Fix build failure with docs profile
[
https://issues.apache.org/jira/browse/HADOOP-18601?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Masatake Iwasaki updated HADOOP-18601:
--
Fix Version/s: 3.4.0
Hadoop Flags: Reviewed
Resolution: Fixed
Status: Resolved (was: Patch Available)
> Fix build failure with docs profile
> ---
>
> Key: HADOOP-18601
> URL: https://issues.apache.org/jira/browse/HADOOP-18601
> Project: Hadoop Common
> Issue Type: Bug
>Reporter: Masatake Iwasaki
>Assignee: Masatake Iwasaki
>Priority: Major
> Labels: pull-request-available
> Fix For: 3.4.0
>
>
> Build with docs profile failed due to depcheck error.
> {noformat}
> $ mvn clean install -DskipTests -DskipShade -Pdocs
> ...
> [INFO] --- maven-enforcer-plugin:3.0.0:enforce (depcheck) @ hadoop-hdfs ---
> [WARNING]
> Dependency convergence error for xerces:xercesImpl:jar:2.12.2:provided paths
> to dependency are:
> +-org.apache.hadoop:hadoop-hdfs:jar:3.4.0-SNAPSHOT
> +-org.apache.hadoop:hadoop-common:jar:3.4.0-SNAPSHOT:provided
> +-xerces:xercesImpl:jar:2.12.2:provided
> and
> +-org.apache.hadoop:hadoop-hdfs:jar:3.4.0-SNAPSHOT
> +-org.apache.hadoop:hadoop-common:test-jar:tests:3.4.0-SNAPSHOT:test
> +-xerces:xercesImpl:jar:2.12.2:test
> and
> +-org.apache.hadoop:hadoop-hdfs:jar:3.4.0-SNAPSHOT
> +-org.apache.hadoop:hadoop-hdfs-client:jar:3.4.0-SNAPSHOT:provided
> +-xerces:xercesImpl:jar:2.12.2:provided
> and
> +-org.apache.hadoop:hadoop-hdfs:jar:3.4.0-SNAPSHOT
> +-com.google.code.findbugs:findbugs:jar:3.0.1:provided
> +-jaxen:jaxen:jar:1.1.6:provided
> +-xerces:xercesImpl:jar:2.6.2:provided
> and
> +-org.apache.hadoop:hadoop-hdfs:jar:3.4.0-SNAPSHOT
> +-xerces:xercesImpl:jar:2.12.2:compile
> [WARNING] Rule 0: org.apache.maven.plugins.enforcer.DependencyConvergence
> failed with message:
> Failed while enforcing releasability. See above detailed error message.
> [INFO]
>
> {noformat}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-18601) Fix build failure with docs profile
[
https://issues.apache.org/jira/browse/HADOOP-18601?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Masatake Iwasaki updated HADOOP-18601:
--
Status: Patch Available (was: Open)
> Fix build failure with docs profile
> ---
>
> Key: HADOOP-18601
> URL: https://issues.apache.org/jira/browse/HADOOP-18601
> Project: Hadoop Common
> Issue Type: Bug
>Reporter: Masatake Iwasaki
>Assignee: Masatake Iwasaki
>Priority: Major
> Labels: pull-request-available
>
> Build with docs profile failed due to depcheck error.
> {noformat}
> $ mvn clean install -DskipTests -DskipShade -Pdocs
> ...
> [INFO] --- maven-enforcer-plugin:3.0.0:enforce (depcheck) @ hadoop-hdfs ---
> [WARNING]
> Dependency convergence error for xerces:xercesImpl:jar:2.12.2:provided paths
> to dependency are:
> +-org.apache.hadoop:hadoop-hdfs:jar:3.4.0-SNAPSHOT
> +-org.apache.hadoop:hadoop-common:jar:3.4.0-SNAPSHOT:provided
> +-xerces:xercesImpl:jar:2.12.2:provided
> and
> +-org.apache.hadoop:hadoop-hdfs:jar:3.4.0-SNAPSHOT
> +-org.apache.hadoop:hadoop-common:test-jar:tests:3.4.0-SNAPSHOT:test
> +-xerces:xercesImpl:jar:2.12.2:test
> and
> +-org.apache.hadoop:hadoop-hdfs:jar:3.4.0-SNAPSHOT
> +-org.apache.hadoop:hadoop-hdfs-client:jar:3.4.0-SNAPSHOT:provided
> +-xerces:xercesImpl:jar:2.12.2:provided
> and
> +-org.apache.hadoop:hadoop-hdfs:jar:3.4.0-SNAPSHOT
> +-com.google.code.findbugs:findbugs:jar:3.0.1:provided
> +-jaxen:jaxen:jar:1.1.6:provided
> +-xerces:xercesImpl:jar:2.6.2:provided
> and
> +-org.apache.hadoop:hadoop-hdfs:jar:3.4.0-SNAPSHOT
> +-xerces:xercesImpl:jar:2.12.2:compile
> [WARNING] Rule 0: org.apache.maven.plugins.enforcer.DependencyConvergence
> failed with message:
> Failed while enforcing releasability. See above detailed error message.
> [INFO]
>
> {noformat}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-18601) Fix build failure with docs profile
[
https://issues.apache.org/jira/browse/HADOOP-18601?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17681899#comment-17681899
]
Masatake Iwasaki commented on HADOOP-18601:
---
another error on hadoop-yarn-server-timelineservice-documentstore.
{noformat}
[WARNING]
Dependency convergence error for xml-apis:xml-apis:jar:1.0.b2:provided paths to
dependency are:
+-org.apache.hadoop:hadoop-yarn-server-timelineservice-documentstore:jar:3.4.0-SNAPSHOT
+-com.microsoft.azure:azure-cosmosdb:jar:2.4.5:compile
+-com.microsoft.azure:azure-cosmosdb-commons:jar:2.4.5:compile
+-commons-validator:commons-validator:jar:1.6:compile
+-commons-digester:commons-digester:jar:1.8.1:compile
+-xml-apis:xml-apis:jar:1.0.b2:provided
and
+-org.apache.hadoop:hadoop-yarn-server-timelineservice-documentstore:jar:3.4.0-SNAPSHOT
+-xerces:xercesImpl:jar:2.12.2:compile
+-xml-apis:xml-apis:jar:1.4.01:compile
{noformat}
> Fix build failure with docs profile
> ---
>
> Key: HADOOP-18601
> URL: https://issues.apache.org/jira/browse/HADOOP-18601
> Project: Hadoop Common
> Issue Type: Bug
>Reporter: Masatake Iwasaki
>Assignee: Masatake Iwasaki
>Priority: Major
>
> Build with docs profile failed due to depcheck error.
> {noformat}
> $ mvn clean install -DskipTests -DskipShade -Pdocs
> ...
> [INFO] --- maven-enforcer-plugin:3.0.0:enforce (depcheck) @ hadoop-hdfs ---
> [WARNING]
> Dependency convergence error for xerces:xercesImpl:jar:2.12.2:provided paths
> to dependency are:
> +-org.apache.hadoop:hadoop-hdfs:jar:3.4.0-SNAPSHOT
> +-org.apache.hadoop:hadoop-common:jar:3.4.0-SNAPSHOT:provided
> +-xerces:xercesImpl:jar:2.12.2:provided
> and
> +-org.apache.hadoop:hadoop-hdfs:jar:3.4.0-SNAPSHOT
> +-org.apache.hadoop:hadoop-common:test-jar:tests:3.4.0-SNAPSHOT:test
> +-xerces:xercesImpl:jar:2.12.2:test
> and
> +-org.apache.hadoop:hadoop-hdfs:jar:3.4.0-SNAPSHOT
> +-org.apache.hadoop:hadoop-hdfs-client:jar:3.4.0-SNAPSHOT:provided
> +-xerces:xercesImpl:jar:2.12.2:provided
> and
> +-org.apache.hadoop:hadoop-hdfs:jar:3.4.0-SNAPSHOT
> +-com.google.code.findbugs:findbugs:jar:3.0.1:provided
> +-jaxen:jaxen:jar:1.1.6:provided
> +-xerces:xercesImpl:jar:2.6.2:provided
> and
> +-org.apache.hadoop:hadoop-hdfs:jar:3.4.0-SNAPSHOT
> +-xerces:xercesImpl:jar:2.12.2:compile
> [WARNING] Rule 0: org.apache.maven.plugins.enforcer.DependencyConvergence
> failed with message:
> Failed while enforcing releasability. See above detailed error message.
> [INFO]
>
> {noformat}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-18601) Fix build failure with docs profile
[
https://issues.apache.org/jira/browse/HADOOP-18601?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17679051#comment-17679051
]
Masatake Iwasaki commented on HADOOP-18601:
---
MetricsSystem.java and MetricsSystemImpl.java seems to be automatically
modified by building with {{-Pdocs}}. I'm not sure this is the cause. If this
is unrelated, I will file another JIRA.
{noformat}
$ mvn clean install -DskipTests -DskipShade -Pdocs
$ git diff
diff --git
a/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/metrics2/MetricsSystem.java
b/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/metrics2/MetricsSystem.java
index fef8c4b7e4b..7b21cc302d1 100644
---
a/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/metrics2/MetricsSystem.java
+++
b/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/metrics2/MetricsSystem.java
@@ -42,6 +42,18 @@
@InterfaceAudience.Private
public abstract MetricsSystem init(String prefix);
+ /**
+ * Register a metrics source
+ * @paramthe actual type of the source object
+ * @param source object to register
+ * @param name of the source. Must be unique or null (then extracted from
+ * the annotations of the source object.)
+ * @param desc the description of the source (or null. See above.)
+ * @return the source object
+ * @exception MetricsException
+ */
+ public abstract T register(String name, String desc, T source);
+
/**
* Register a metrics source
* @paramthe actual type of the source object
diff --git
a/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/metrics2/impl/MetricsSystemImpl.java
b/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/metrics2/impl/MetricsSystemImpl.java
index 6c5a71a708f..beb72e42723 100644
---
a/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/metrics2/impl/MetricsSystemImpl.java
+++
b/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/metrics2/impl/MetricsSystemImpl.java
@@ -293,6 +293,31 @@ T register(final String name, final String description,
final T sink) {
return sink;
}
+ @Override public synchronized
+ T register(final String name, final String description, final T sink) {
+LOG.debug(name +", "+ description);
+if (allSinks.containsKey(name)) {
+ if(sinks.get(name) == null) {
+registerSink(name, description, sink);
+ } else {
+LOG.warn("Sink "+ name +" already exists!");
+ }
+ return sink;
+}
+allSinks.put(name, sink);
+if (config != null) {
+ registerSink(name, description, sink);
+}
+// We want to re-register the sink to pick up new config
+// when the metrics system restarts.
+register(name, new AbstractCallback() {
+ @Override public void postStart() {
+register(name, description, sink);
+ }
+});
+return sink;
+ }
+
synchronized void registerSink(String name, String desc, MetricsSink sink) {
checkNotNull(config, "config");
MetricsConfig conf = sinkConfigs.get(name);
{noformat}
> Fix build failure with docs profile
> ---
>
> Key: HADOOP-18601
> URL: https://issues.apache.org/jira/browse/HADOOP-18601
> Project: Hadoop Common
> Issue Type: Bug
>Reporter: Masatake Iwasaki
>Assignee: Masatake Iwasaki
>Priority: Major
>
> Build with docs profile failed due to depcheck error.
> {noformat}
> $ mvn clean install -DskipTests -DskipShade -Pdocs
> ...
> [INFO] --- maven-enforcer-plugin:3.0.0:enforce (depcheck) @ hadoop-hdfs ---
> [WARNING]
> Dependency convergence error for xerces:xercesImpl:jar:2.12.2:provided paths
> to dependency are:
> +-org.apache.hadoop:hadoop-hdfs:jar:3.4.0-SNAPSHOT
> +-org.apache.hadoop:hadoop-common:jar:3.4.0-SNAPSHOT:provided
> +-xerces:xercesImpl:jar:2.12.2:provided
> and
> +-org.apache.hadoop:hadoop-hdfs:jar:3.4.0-SNAPSHOT
> +-org.apache.hadoop:hadoop-common:test-jar:tests:3.4.0-SNAPSHOT:test
> +-xerces:xercesImpl:jar:2.12.2:test
> and
> +-org.apache.hadoop:hadoop-hdfs:jar:3.4.0-SNAPSHOT
> +-org.apache.hadoop:hadoop-hdfs-client:jar:3.4.0-SNAPSHOT:provided
> +-xerces:xercesImpl:jar:2.12.2:provided
> and
> +-org.apache.hadoop:hadoop-hdfs:jar:3.4.0-SNAPSHOT
> +-com.google.code.findbugs:findbugs:jar:3.0.1:provided
> +-jaxen:jaxen:jar:1.1.6:provided
> +-xerces:xercesImpl:jar:2.6.2:provided
> and
> +-org.apache.hadoop:hadoop-hdfs:jar:3.4.0-SNAPSHOT
> +-xerces:xercesImpl:jar:2.12.2:compile
> [WARNING] Rule 0: org.apache.maven.plugins.enforcer.DependencyConvergence
> failed with message:
> Failed while enforcing releasability. See above detailed error message.
> [INFO]
>
> {noformat}
--
This message
[jira] [Commented] (HADOOP-18601) Fix build failure with docs profile
[
https://issues.apache.org/jira/browse/HADOOP-18601?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17679048#comment-17679048
]
Masatake Iwasaki commented on HADOOP-18601:
---
I can not reproduce this on branch-3.3. While the error surfaced on building
hadoop-hdfs project, I filed this under HADOOP since the cause looks dependency
conflicts between hadoop-common and hadoop-hdfs.
> Fix build failure with docs profile
> ---
>
> Key: HADOOP-18601
> URL: https://issues.apache.org/jira/browse/HADOOP-18601
> Project: Hadoop Common
> Issue Type: Bug
>Reporter: Masatake Iwasaki
>Assignee: Masatake Iwasaki
>Priority: Major
>
> Build with docs profile failed due to depcheck error.
> {noformat}
> $ mvn clean install -DskipTests -DskipShade -Pdocs
> ...
> [INFO] --- maven-enforcer-plugin:3.0.0:enforce (depcheck) @ hadoop-hdfs ---
> [WARNING]
> Dependency convergence error for xerces:xercesImpl:jar:2.12.2:provided paths
> to dependency are:
> +-org.apache.hadoop:hadoop-hdfs:jar:3.4.0-SNAPSHOT
> +-org.apache.hadoop:hadoop-common:jar:3.4.0-SNAPSHOT:provided
> +-xerces:xercesImpl:jar:2.12.2:provided
> and
> +-org.apache.hadoop:hadoop-hdfs:jar:3.4.0-SNAPSHOT
> +-org.apache.hadoop:hadoop-common:test-jar:tests:3.4.0-SNAPSHOT:test
> +-xerces:xercesImpl:jar:2.12.2:test
> and
> +-org.apache.hadoop:hadoop-hdfs:jar:3.4.0-SNAPSHOT
> +-org.apache.hadoop:hadoop-hdfs-client:jar:3.4.0-SNAPSHOT:provided
> +-xerces:xercesImpl:jar:2.12.2:provided
> and
> +-org.apache.hadoop:hadoop-hdfs:jar:3.4.0-SNAPSHOT
> +-com.google.code.findbugs:findbugs:jar:3.0.1:provided
> +-jaxen:jaxen:jar:1.1.6:provided
> +-xerces:xercesImpl:jar:2.6.2:provided
> and
> +-org.apache.hadoop:hadoop-hdfs:jar:3.4.0-SNAPSHOT
> +-xerces:xercesImpl:jar:2.12.2:compile
> [WARNING] Rule 0: org.apache.maven.plugins.enforcer.DependencyConvergence
> failed with message:
> Failed while enforcing releasability. See above detailed error message.
> [INFO]
>
> {noformat}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Created] (HADOOP-18601) Fix build failure with docs profile
Masatake Iwasaki created HADOOP-18601:
-
Summary: Fix build failure with docs profile
Key: HADOOP-18601
URL: https://issues.apache.org/jira/browse/HADOOP-18601
Project: Hadoop Common
Issue Type: Bug
Reporter: Masatake Iwasaki
Assignee: Masatake Iwasaki
Build with docs profile failed due to depcheck error.
{noformat}
$ mvn clean install -DskipTests -DskipShade -Pdocs
...
[INFO] --- maven-enforcer-plugin:3.0.0:enforce (depcheck) @ hadoop-hdfs ---
[WARNING]
Dependency convergence error for xerces:xercesImpl:jar:2.12.2:provided paths to
dependency are:
+-org.apache.hadoop:hadoop-hdfs:jar:3.4.0-SNAPSHOT
+-org.apache.hadoop:hadoop-common:jar:3.4.0-SNAPSHOT:provided
+-xerces:xercesImpl:jar:2.12.2:provided
and
+-org.apache.hadoop:hadoop-hdfs:jar:3.4.0-SNAPSHOT
+-org.apache.hadoop:hadoop-common:test-jar:tests:3.4.0-SNAPSHOT:test
+-xerces:xercesImpl:jar:2.12.2:test
and
+-org.apache.hadoop:hadoop-hdfs:jar:3.4.0-SNAPSHOT
+-org.apache.hadoop:hadoop-hdfs-client:jar:3.4.0-SNAPSHOT:provided
+-xerces:xercesImpl:jar:2.12.2:provided
and
+-org.apache.hadoop:hadoop-hdfs:jar:3.4.0-SNAPSHOT
+-com.google.code.findbugs:findbugs:jar:3.0.1:provided
+-jaxen:jaxen:jar:1.1.6:provided
+-xerces:xercesImpl:jar:2.6.2:provided
and
+-org.apache.hadoop:hadoop-hdfs:jar:3.4.0-SNAPSHOT
+-xerces:xercesImpl:jar:2.12.2:compile
[WARNING] Rule 0: org.apache.maven.plugins.enforcer.DependencyConvergence
failed with message:
Failed while enforcing releasability. See above detailed error message.
[INFO]
{noformat}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-18598) maven site generation doesn't include javadocs
[ https://issues.apache.org/jira/browse/HADOOP-18598?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17679013#comment-17679013 ] Masatake Iwasaki commented on HADOOP-18598: --- [~ste...@apache.org] [Downgrading maven-site-plugin to 3.9.1|https://github.com/apache/hadoop/pull/5319] (assuming maven-javadoc-plugin-3.0.1) could be a quick fix. maven-site-plugin-3.11.0 assumes recent maven-javadoc-plugin (3.3.2). https://github.com/apache/maven-site-plugin/blob/maven-site-plugin-3.11.0/pom.xml#L210 Bumping the version of maven-javadoc-plugin to 3.3.2 in Hadoop breaks build due to newly surfacing javadoc warnings. > maven site generation doesn't include javadocs > -- > > Key: HADOOP-18598 > URL: https://issues.apache.org/jira/browse/HADOOP-18598 > Project: Hadoop Common > Issue Type: Bug > Components: site >Affects Versions: 3.3.5 >Reporter: Steve Loughran >Assignee: Steve Loughran >Priority: Blocker > Labels: pull-request-available > > the rc0 excluded all the site docs. running mvn site on trunk throws up site > plugin issues, which may be related, so start by updating that. > rc validation scripts to include checks for the api/index.html -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-18375) Fix failure of shelltest for hadoop_add_ldlibpath
[ https://issues.apache.org/jira/browse/HADOOP-18375?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Masatake Iwasaki updated HADOOP-18375: -- Fix Version/s: 3.2.5 > Fix failure of shelltest for hadoop_add_ldlibpath > - > > Key: HADOOP-18375 > URL: https://issues.apache.org/jira/browse/HADOOP-18375 > Project: Hadoop Common > Issue Type: Bug > Components: test >Reporter: Masatake Iwasaki >Assignee: Masatake Iwasaki >Priority: Minor > Labels: pull-request-available > Fix For: 3.4.0, 3.3.9, 3.2.5 > > Time Spent: 40m > Remaining Estimate: 0h > > Test cases in hadoop_add_ldlibpath.bats failed. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-18375) Fix failure of shelltest for hadoop_add_ldlibpath
[ https://issues.apache.org/jira/browse/HADOOP-18375?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Masatake Iwasaki updated HADOOP-18375: -- Fix Version/s: 3.3.9 > Fix failure of shelltest for hadoop_add_ldlibpath > - > > Key: HADOOP-18375 > URL: https://issues.apache.org/jira/browse/HADOOP-18375 > Project: Hadoop Common > Issue Type: Bug > Components: test >Reporter: Masatake Iwasaki >Assignee: Masatake Iwasaki >Priority: Minor > Labels: pull-request-available > Fix For: 3.4.0, 3.3.9 > > Time Spent: 40m > Remaining Estimate: 0h > > Test cases in hadoop_add_ldlibpath.bats failed. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-18375) Fix failure of shelltest for hadoop_add_ldlibpath
[ https://issues.apache.org/jira/browse/HADOOP-18375?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Masatake Iwasaki updated HADOOP-18375: -- Fix Version/s: 3.4.0 Hadoop Flags: Reviewed Resolution: Fixed Status: Resolved (was: Patch Available) > Fix failure of shelltest for hadoop_add_ldlibpath > - > > Key: HADOOP-18375 > URL: https://issues.apache.org/jira/browse/HADOOP-18375 > Project: Hadoop Common > Issue Type: Bug > Components: test >Reporter: Masatake Iwasaki >Assignee: Masatake Iwasaki >Priority: Minor > Labels: pull-request-available > Fix For: 3.4.0 > > Time Spent: 40m > Remaining Estimate: 0h > > Test cases in hadoop_add_ldlibpath.bats failed. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-18418) Upgrade bundled Tomcat to 8.5.82
[ https://issues.apache.org/jira/browse/HADOOP-18418?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Masatake Iwasaki updated HADOOP-18418: -- Fix Version/s: 2.10.3 Hadoop Flags: Reviewed Resolution: Fixed Status: Resolved (was: Patch Available) > Upgrade bundled Tomcat to 8.5.82 > > > Key: HADOOP-18418 > URL: https://issues.apache.org/jira/browse/HADOOP-18418 > Project: Hadoop Common > Issue Type: Improvement > Components: httpfs, kms >Affects Versions: 2.10.2 >Reporter: groot >Assignee: groot >Priority: Major > Labels: pull-request-available > Fix For: 2.10.3 > > > h4. > Currently we are using 8.5.81 which is affected by CVE-2022-34305 > More Details - [https://github.com/advisories/GHSA-6j88-6whg-x687] > Lets upgrade to 8.5.82 > -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-18069) CVE-2021-0341 in okhttp@2.7.5 detected in hdfs-client
[ https://issues.apache.org/jira/browse/HADOOP-18069?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Masatake Iwasaki updated HADOOP-18069: -- Fix Version/s: (was: 3.3.4) > CVE-2021-0341 in okhttp@2.7.5 detected in hdfs-client > --- > > Key: HADOOP-18069 > URL: https://issues.apache.org/jira/browse/HADOOP-18069 > Project: Hadoop Common > Issue Type: Bug > Components: hdfs-client >Affects Versions: 3.3.1 >Reporter: Eugene Shinn (Truveta) >Assignee: groot >Priority: Major > Labels: pull-request-available > Time Spent: 6h 10m > Remaining Estimate: 0h > > Our static vulnerability scanner (Fortify On Demand) detected [NVD - > CVE-2021-0341 > (nist.gov)|https://nvd.nist.gov/vuln/detail/CVE-2021-0341#VulnChangeHistorySection] > in our application. We traced the vulnerability to a transitive dependency > coming from hadoop-hdfs-client, which depends on okhttp@2.7.5 > ([hadoop/pom.xml at trunk · apache/hadoop > (github.com)|https://github.com/apache/hadoop/blob/trunk/hadoop-project/pom.xml#L137]). > To resolve this issue, okhttp should be upgraded to 4.9.2+ (ref: > [CVE-2021-0341 · Issue #6724 · square/okhttp > (github.com)|https://github.com/square/okhttp/issues/6724]). -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-18393) Hadoop 3.3.2 has CVEs coming from dependencies
[ https://issues.apache.org/jira/browse/HADOOP-18393?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Masatake Iwasaki updated HADOOP-18393: -- Fix Version/s: (was: 3.3.4) > Hadoop 3.3.2 has CVEs coming from dependencies > -- > > Key: HADOOP-18393 > URL: https://issues.apache.org/jira/browse/HADOOP-18393 > Project: Hadoop Common > Issue Type: Improvement > Components: build >Affects Versions: 3.3.2 >Reporter: suman agrawal >Priority: Major > > Hi Team, > > Hadoop version 3.3.1 which is compatible for our application have > Vulnerebilities: > Is there any plan to fix this > CVE-2021-37404 hadoop versions < 3.3.2 Apache Hadoop potential heap buffer > overflow in libhdfs. > CVE-2020-10650 jackson < 2.9.10.4 > CVE-2021-33036 hadoop < 3.3.2 > CVE-2022-31159 aws xfer manager download < 1.12.262 -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-18136) Verify FileUtils.unTar() handling of missing .tar files
[ https://issues.apache.org/jira/browse/HADOOP-18136?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Masatake Iwasaki updated HADOOP-18136: -- Fix Version/s: 3.3.3 (was: 3.3.4) > Verify FileUtils.unTar() handling of missing .tar files > --- > > Key: HADOOP-18136 > URL: https://issues.apache.org/jira/browse/HADOOP-18136 > Project: Hadoop Common > Issue Type: Improvement > Components: test, util >Affects Versions: 3.1.4, 2.10.1, 3.3.1, 3.2.3 >Reporter: Steve Loughran >Assignee: Steve Loughran >Priority: Minor > Fix For: 2.10.2, 3.2.4, 3.3.3 > > > add a test to verify FileUtils.unTar() of a non .gz fails meaningfully if > file isn't present; fix if not. > test both the unix and windows paths. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-18376) Fix failure of instanciation of s3gatewayrequestlog appender due to lack of HttpRequestLogAppender
[
https://issues.apache.org/jira/browse/HADOOP-18376?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17572301#comment-17572301
]
Masatake Iwasaki commented on HADOOP-18376:
---
HADOOP-17526 removed the org.apache.hadoop.http.HttpRequestLogAppender. It is
already marked as "incompatible change".
> Fix failure of instanciation of s3gatewayrequestlog appender due to lack of
> HttpRequestLogAppender
> --
>
> Key: HADOOP-18376
> URL: https://issues.apache.org/jira/browse/HADOOP-18376
> Project: Hadoop Common
> Issue Type: Bug
>Reporter: Masatake Iwasaki
>Assignee: Masatake Iwasaki
>Priority: Major
>
> {noformat}
> $ bin/hadoop version
> log4j:ERROR Could not instantiate class
> [org.apache.hadoop.http.HttpRequestLogAppender].
> java.lang.ClassNotFoundException:
> org.apache.hadoop.http.HttpRequestLogAppender
> at java.net.URLClassLoader.findClass(URLClassLoader.java:387)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:418)
> at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:352)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:351)
> at java.lang.Class.forName0(Native Method)
> at java.lang.Class.forName(Class.java:264)
> at org.apache.log4j.helpers.Loader.loadClass(Loader.java:198)
> at
> org.apache.log4j.helpers.OptionConverter.instantiateByClassName(OptionConverter.java:327)
> at
> org.apache.log4j.helpers.OptionConverter.instantiateByKey(OptionConverter.java:124)
> at
> org.apache.log4j.PropertyConfigurator.parseAppender(PropertyConfigurator.java:785)
> at
> org.apache.log4j.PropertyConfigurator.parseCategory(PropertyConfigurator.java:768)
> at
> org.apache.log4j.PropertyConfigurator.parseCatsAndRenderers(PropertyConfigurator.java:672)
> at
> org.apache.log4j.PropertyConfigurator.doConfigure(PropertyConfigurator.java:516)
> at
> org.apache.log4j.PropertyConfigurator.doConfigure(PropertyConfigurator.java:580)
> at
> org.apache.log4j.helpers.OptionConverter.selectAndConfigure(OptionConverter.java:526)
> at org.apache.log4j.LogManager.(LogManager.java:127)
> at
> org.slf4j.impl.Log4jLoggerFactory.(Log4jLoggerFactory.java:66)
> at
> org.slf4j.impl.StaticLoggerBinder.(StaticLoggerBinder.java:72)
> at
> org.slf4j.impl.StaticLoggerBinder.(StaticLoggerBinder.java:45)
> at org.slf4j.LoggerFactory.bind(LoggerFactory.java:150)
> at
> org.slf4j.LoggerFactory.performInitialization(LoggerFactory.java:124)
> at org.slf4j.LoggerFactory.getILoggerFactory(LoggerFactory.java:417)
> at org.slf4j.LoggerFactory.getLogger(LoggerFactory.java:362)
> at org.slf4j.LoggerFactory.getLogger(LoggerFactory.java:388)
> at org.apache.hadoop.util.VersionInfo.(VersionInfo.java:37)
> log4j:ERROR Could not instantiate appender named "s3gatewayrequestlog".
> Hadoop 3.4.0-SNAPSHOT
> Source code repository Unknown -r c92ff0b4f1135e84527e25087d9cc65250bd0222
> Compiled by rocky on 2022-07-27T23:17Z
> Compiled with protoc 3.7.1
> From source with checksum 486fac66d9b4349cc0fd3d5e8bc38e40
> This command was run using
> /home/rocky/dist/hadoop-3.4.0-SNAPSHOT/share/hadoop/common/hadoop-common-3.4.0-SNAPSHOT.jar
> {noformat}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-18376) Fix failure of instanciation of s3gatewayrequestlog appender due to lack of HttpRequestLogAppender
[
https://issues.apache.org/jira/browse/HADOOP-18376?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17572298#comment-17572298
]
Masatake Iwasaki commented on HADOOP-18376:
---
HADOOP-17055 removed related code from log4j.properties.
> Fix failure of instanciation of s3gatewayrequestlog appender due to lack of
> HttpRequestLogAppender
> --
>
> Key: HADOOP-18376
> URL: https://issues.apache.org/jira/browse/HADOOP-18376
> Project: Hadoop Common
> Issue Type: Bug
>Reporter: Masatake Iwasaki
>Assignee: Masatake Iwasaki
>Priority: Major
>
> {noformat}
> $ bin/hadoop version
> log4j:ERROR Could not instantiate class
> [org.apache.hadoop.http.HttpRequestLogAppender].
> java.lang.ClassNotFoundException:
> org.apache.hadoop.http.HttpRequestLogAppender
> at java.net.URLClassLoader.findClass(URLClassLoader.java:387)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:418)
> at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:352)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:351)
> at java.lang.Class.forName0(Native Method)
> at java.lang.Class.forName(Class.java:264)
> at org.apache.log4j.helpers.Loader.loadClass(Loader.java:198)
> at
> org.apache.log4j.helpers.OptionConverter.instantiateByClassName(OptionConverter.java:327)
> at
> org.apache.log4j.helpers.OptionConverter.instantiateByKey(OptionConverter.java:124)
> at
> org.apache.log4j.PropertyConfigurator.parseAppender(PropertyConfigurator.java:785)
> at
> org.apache.log4j.PropertyConfigurator.parseCategory(PropertyConfigurator.java:768)
> at
> org.apache.log4j.PropertyConfigurator.parseCatsAndRenderers(PropertyConfigurator.java:672)
> at
> org.apache.log4j.PropertyConfigurator.doConfigure(PropertyConfigurator.java:516)
> at
> org.apache.log4j.PropertyConfigurator.doConfigure(PropertyConfigurator.java:580)
> at
> org.apache.log4j.helpers.OptionConverter.selectAndConfigure(OptionConverter.java:526)
> at org.apache.log4j.LogManager.(LogManager.java:127)
> at
> org.slf4j.impl.Log4jLoggerFactory.(Log4jLoggerFactory.java:66)
> at
> org.slf4j.impl.StaticLoggerBinder.(StaticLoggerBinder.java:72)
> at
> org.slf4j.impl.StaticLoggerBinder.(StaticLoggerBinder.java:45)
> at org.slf4j.LoggerFactory.bind(LoggerFactory.java:150)
> at
> org.slf4j.LoggerFactory.performInitialization(LoggerFactory.java:124)
> at org.slf4j.LoggerFactory.getILoggerFactory(LoggerFactory.java:417)
> at org.slf4j.LoggerFactory.getLogger(LoggerFactory.java:362)
> at org.slf4j.LoggerFactory.getLogger(LoggerFactory.java:388)
> at org.apache.hadoop.util.VersionInfo.(VersionInfo.java:37)
> log4j:ERROR Could not instantiate appender named "s3gatewayrequestlog".
> Hadoop 3.4.0-SNAPSHOT
> Source code repository Unknown -r c92ff0b4f1135e84527e25087d9cc65250bd0222
> Compiled by rocky on 2022-07-27T23:17Z
> Compiled with protoc 3.7.1
> From source with checksum 486fac66d9b4349cc0fd3d5e8bc38e40
> This command was run using
> /home/rocky/dist/hadoop-3.4.0-SNAPSHOT/share/hadoop/common/hadoop-common-3.4.0-SNAPSHOT.jar
> {noformat}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Resolved] (HADOOP-18376) Fix failure of instanciation of s3gatewayrequestlog appender due to lack of HttpRequestLogAppender
[
https://issues.apache.org/jira/browse/HADOOP-18376?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Masatake Iwasaki resolved HADOOP-18376.
---
Resolution: Invalid
The cause was old log4j.properties in my testing environment.
> Fix failure of instanciation of s3gatewayrequestlog appender due to lack of
> HttpRequestLogAppender
> --
>
> Key: HADOOP-18376
> URL: https://issues.apache.org/jira/browse/HADOOP-18376
> Project: Hadoop Common
> Issue Type: Bug
>Reporter: Masatake Iwasaki
>Assignee: Masatake Iwasaki
>Priority: Major
>
> {noformat}
> $ bin/hadoop version
> log4j:ERROR Could not instantiate class
> [org.apache.hadoop.http.HttpRequestLogAppender].
> java.lang.ClassNotFoundException:
> org.apache.hadoop.http.HttpRequestLogAppender
> at java.net.URLClassLoader.findClass(URLClassLoader.java:387)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:418)
> at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:352)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:351)
> at java.lang.Class.forName0(Native Method)
> at java.lang.Class.forName(Class.java:264)
> at org.apache.log4j.helpers.Loader.loadClass(Loader.java:198)
> at
> org.apache.log4j.helpers.OptionConverter.instantiateByClassName(OptionConverter.java:327)
> at
> org.apache.log4j.helpers.OptionConverter.instantiateByKey(OptionConverter.java:124)
> at
> org.apache.log4j.PropertyConfigurator.parseAppender(PropertyConfigurator.java:785)
> at
> org.apache.log4j.PropertyConfigurator.parseCategory(PropertyConfigurator.java:768)
> at
> org.apache.log4j.PropertyConfigurator.parseCatsAndRenderers(PropertyConfigurator.java:672)
> at
> org.apache.log4j.PropertyConfigurator.doConfigure(PropertyConfigurator.java:516)
> at
> org.apache.log4j.PropertyConfigurator.doConfigure(PropertyConfigurator.java:580)
> at
> org.apache.log4j.helpers.OptionConverter.selectAndConfigure(OptionConverter.java:526)
> at org.apache.log4j.LogManager.(LogManager.java:127)
> at
> org.slf4j.impl.Log4jLoggerFactory.(Log4jLoggerFactory.java:66)
> at
> org.slf4j.impl.StaticLoggerBinder.(StaticLoggerBinder.java:72)
> at
> org.slf4j.impl.StaticLoggerBinder.(StaticLoggerBinder.java:45)
> at org.slf4j.LoggerFactory.bind(LoggerFactory.java:150)
> at
> org.slf4j.LoggerFactory.performInitialization(LoggerFactory.java:124)
> at org.slf4j.LoggerFactory.getILoggerFactory(LoggerFactory.java:417)
> at org.slf4j.LoggerFactory.getLogger(LoggerFactory.java:362)
> at org.slf4j.LoggerFactory.getLogger(LoggerFactory.java:388)
> at org.apache.hadoop.util.VersionInfo.(VersionInfo.java:37)
> log4j:ERROR Could not instantiate appender named "s3gatewayrequestlog".
> Hadoop 3.4.0-SNAPSHOT
> Source code repository Unknown -r c92ff0b4f1135e84527e25087d9cc65250bd0222
> Compiled by rocky on 2022-07-27T23:17Z
> Compiled with protoc 3.7.1
> From source with checksum 486fac66d9b4349cc0fd3d5e8bc38e40
> This command was run using
> /home/rocky/dist/hadoop-3.4.0-SNAPSHOT/share/hadoop/common/hadoop-common-3.4.0-SNAPSHOT.jar
> {noformat}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-18376) Fix failure of instanciation of s3gatewayrequestlog appender due to lack of HttpRequestLogAppender
[
https://issues.apache.org/jira/browse/HADOOP-18376?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Masatake Iwasaki updated HADOOP-18376:
--
Summary: Fix failure of instanciation of s3gatewayrequestlog appender due
to lack of HttpRequestLogAppender (was: Fix failure of instanciation of
s3gatewayrequestlog due to lack of HttpRequestLogAppender)
> Fix failure of instanciation of s3gatewayrequestlog appender due to lack of
> HttpRequestLogAppender
> --
>
> Key: HADOOP-18376
> URL: https://issues.apache.org/jira/browse/HADOOP-18376
> Project: Hadoop Common
> Issue Type: Bug
>Reporter: Masatake Iwasaki
>Assignee: Masatake Iwasaki
>Priority: Major
>
> {noformat}
> $ bin/hadoop version
> log4j:ERROR Could not instantiate class
> [org.apache.hadoop.http.HttpRequestLogAppender].
> java.lang.ClassNotFoundException:
> org.apache.hadoop.http.HttpRequestLogAppender
> at java.net.URLClassLoader.findClass(URLClassLoader.java:387)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:418)
> at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:352)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:351)
> at java.lang.Class.forName0(Native Method)
> at java.lang.Class.forName(Class.java:264)
> at org.apache.log4j.helpers.Loader.loadClass(Loader.java:198)
> at
> org.apache.log4j.helpers.OptionConverter.instantiateByClassName(OptionConverter.java:327)
> at
> org.apache.log4j.helpers.OptionConverter.instantiateByKey(OptionConverter.java:124)
> at
> org.apache.log4j.PropertyConfigurator.parseAppender(PropertyConfigurator.java:785)
> at
> org.apache.log4j.PropertyConfigurator.parseCategory(PropertyConfigurator.java:768)
> at
> org.apache.log4j.PropertyConfigurator.parseCatsAndRenderers(PropertyConfigurator.java:672)
> at
> org.apache.log4j.PropertyConfigurator.doConfigure(PropertyConfigurator.java:516)
> at
> org.apache.log4j.PropertyConfigurator.doConfigure(PropertyConfigurator.java:580)
> at
> org.apache.log4j.helpers.OptionConverter.selectAndConfigure(OptionConverter.java:526)
> at org.apache.log4j.LogManager.(LogManager.java:127)
> at
> org.slf4j.impl.Log4jLoggerFactory.(Log4jLoggerFactory.java:66)
> at
> org.slf4j.impl.StaticLoggerBinder.(StaticLoggerBinder.java:72)
> at
> org.slf4j.impl.StaticLoggerBinder.(StaticLoggerBinder.java:45)
> at org.slf4j.LoggerFactory.bind(LoggerFactory.java:150)
> at
> org.slf4j.LoggerFactory.performInitialization(LoggerFactory.java:124)
> at org.slf4j.LoggerFactory.getILoggerFactory(LoggerFactory.java:417)
> at org.slf4j.LoggerFactory.getLogger(LoggerFactory.java:362)
> at org.slf4j.LoggerFactory.getLogger(LoggerFactory.java:388)
> at org.apache.hadoop.util.VersionInfo.(VersionInfo.java:37)
> log4j:ERROR Could not instantiate appender named "s3gatewayrequestlog".
> Hadoop 3.4.0-SNAPSHOT
> Source code repository Unknown -r c92ff0b4f1135e84527e25087d9cc65250bd0222
> Compiled by rocky on 2022-07-27T23:17Z
> Compiled with protoc 3.7.1
> From source with checksum 486fac66d9b4349cc0fd3d5e8bc38e40
> This command was run using
> /home/rocky/dist/hadoop-3.4.0-SNAPSHOT/share/hadoop/common/hadoop-common-3.4.0-SNAPSHOT.jar
> {noformat}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Created] (HADOOP-18376) Fix failure of instanciation of s3gatewayrequestlog due to lack of HttpRequestLogAppender
Masatake Iwasaki created HADOOP-18376:
-
Summary: Fix failure of instanciation of s3gatewayrequestlog due
to lack of HttpRequestLogAppender
Key: HADOOP-18376
URL: https://issues.apache.org/jira/browse/HADOOP-18376
Project: Hadoop Common
Issue Type: Bug
Reporter: Masatake Iwasaki
Assignee: Masatake Iwasaki
{noformat}
$ bin/hadoop version
log4j:ERROR Could not instantiate class
[org.apache.hadoop.http.HttpRequestLogAppender].
java.lang.ClassNotFoundException: org.apache.hadoop.http.HttpRequestLogAppender
at java.net.URLClassLoader.findClass(URLClassLoader.java:387)
at java.lang.ClassLoader.loadClass(ClassLoader.java:418)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:352)
at java.lang.ClassLoader.loadClass(ClassLoader.java:351)
at java.lang.Class.forName0(Native Method)
at java.lang.Class.forName(Class.java:264)
at org.apache.log4j.helpers.Loader.loadClass(Loader.java:198)
at
org.apache.log4j.helpers.OptionConverter.instantiateByClassName(OptionConverter.java:327)
at
org.apache.log4j.helpers.OptionConverter.instantiateByKey(OptionConverter.java:124)
at
org.apache.log4j.PropertyConfigurator.parseAppender(PropertyConfigurator.java:785)
at
org.apache.log4j.PropertyConfigurator.parseCategory(PropertyConfigurator.java:768)
at
org.apache.log4j.PropertyConfigurator.parseCatsAndRenderers(PropertyConfigurator.java:672)
at
org.apache.log4j.PropertyConfigurator.doConfigure(PropertyConfigurator.java:516)
at
org.apache.log4j.PropertyConfigurator.doConfigure(PropertyConfigurator.java:580)
at
org.apache.log4j.helpers.OptionConverter.selectAndConfigure(OptionConverter.java:526)
at org.apache.log4j.LogManager.(LogManager.java:127)
at org.slf4j.impl.Log4jLoggerFactory.(Log4jLoggerFactory.java:66)
at org.slf4j.impl.StaticLoggerBinder.(StaticLoggerBinder.java:72)
at
org.slf4j.impl.StaticLoggerBinder.(StaticLoggerBinder.java:45)
at org.slf4j.LoggerFactory.bind(LoggerFactory.java:150)
at org.slf4j.LoggerFactory.performInitialization(LoggerFactory.java:124)
at org.slf4j.LoggerFactory.getILoggerFactory(LoggerFactory.java:417)
at org.slf4j.LoggerFactory.getLogger(LoggerFactory.java:362)
at org.slf4j.LoggerFactory.getLogger(LoggerFactory.java:388)
at org.apache.hadoop.util.VersionInfo.(VersionInfo.java:37)
log4j:ERROR Could not instantiate appender named "s3gatewayrequestlog".
Hadoop 3.4.0-SNAPSHOT
Source code repository Unknown -r c92ff0b4f1135e84527e25087d9cc65250bd0222
Compiled by rocky on 2022-07-27T23:17Z
Compiled with protoc 3.7.1
>From source with checksum 486fac66d9b4349cc0fd3d5e8bc38e40
This command was run using
/home/rocky/dist/hadoop-3.4.0-SNAPSHOT/share/hadoop/common/hadoop-common-3.4.0-SNAPSHOT.jar
{noformat}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-18375) Fix failure of shelltest for hadoop_add_ldlibpath
[ https://issues.apache.org/jira/browse/HADOOP-18375?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Masatake Iwasaki updated HADOOP-18375: -- Status: Patch Available (was: Open) > Fix failure of shelltest for hadoop_add_ldlibpath > - > > Key: HADOOP-18375 > URL: https://issues.apache.org/jira/browse/HADOOP-18375 > Project: Hadoop Common > Issue Type: Bug > Components: test >Reporter: Masatake Iwasaki >Assignee: Masatake Iwasaki >Priority: Minor > Labels: pull-request-available > Time Spent: 0.5h > Remaining Estimate: 0h > > Test cases in hadoop_add_ldlibpath.bats failed. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Created] (HADOOP-18375) Fix failure of shelltest for hadoop_add_ldlibpath
Masatake Iwasaki created HADOOP-18375: - Summary: Fix failure of shelltest for hadoop_add_ldlibpath Key: HADOOP-18375 URL: https://issues.apache.org/jira/browse/HADOOP-18375 Project: Hadoop Common Issue Type: Bug Components: test Reporter: Masatake Iwasaki Assignee: Masatake Iwasaki Test cases in hadoop_add_ldlibpath.bats failed. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-18375) Fix failure of shelltest for hadoop_add_ldlibpath
[
https://issues.apache.org/jira/browse/HADOOP-18375?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17572214#comment-17572214
]
Masatake Iwasaki commented on HADOOP-18375:
---
{noformat}
[exec] not ok 3 hadoop_add_ldlibpath (simple dupecheck)
[exec] # (from function `hadoop_add_colonpath' in file
../../main/bin/hadoop-functions.sh, line 1251,
[exec] # from function `hadoop_add_ldlibpath' in file
../../main/bin/hadoop-functions.sh, line 1286,
[exec] # in test file hadoop_add_ldlibpath.bats, line 32)
[exec] # `hadoop_add_ldlibpath "${TMP}"' failed
[exec] # bindir:
/home/rocky/srcs/hadoop/hadoop-common-project/hadoop-common/src/test/scripts
[exec] # DEBUG: Append colonpath(LD_LIBRARY_PATH):
/home/rocky/srcs/hadoop/hadoop-common-project/hadoop-common/target/test-dir/bats.86220.1498
[exec] # DEBUG: Rejected colonpath(LD_LIBRARY_PATH):
/home/rocky/srcs/hadoop/hadoop-common-project/hadoop-common/target/test-dir/bats.86220.1498
[exec] #
>/opt/rh/gcc-toolset-9/root/usr/lib64:/opt/rh/gcc-toolset-9/root/usr/lib:/opt/rh/gcc-toolset-9/root/usr/lib64/dyninst:/opt/rh/gcc-toolset-9/root/usr/lib/dyninst:/opt/rh/gcc-toolset-9/root/usr/lib64:/opt/rh/gcc-toolset-9/root/usr/lib:/home/rocky/srcs/hadoop/hadoop-common-project/hadoop-common/target/test-dir/bats.86220.1498<
[exec] not ok 4 hadoop_add_ldlibpath (default order)
[exec] # (in test file hadoop_add_ldlibpath.bats, line 42)
[exec] # `[ "${LD_LIBRARY_PATH}" = "${TMP}:/tmp" ]' failed
[exec] # bindir:
/home/rocky/srcs/hadoop/hadoop-common-project/hadoop-common/src/test/scripts
[exec] # DEBUG: Append colonpath(LD_LIBRARY_PATH):
/home/rocky/srcs/hadoop/hadoop-common-project/hadoop-common/target/test-dir/bats.86234.14082
[exec] # DEBUG: Append colonpath(LD_LIBRARY_PATH): /tmp
[exec] #
>/opt/rh/gcc-toolset-9/root/usr/lib64:/opt/rh/gcc-toolset-9/root/usr/lib:/opt/rh/gcc-toolset-9/root/usr/lib64/dyninst:/opt/rh/gcc-toolset-9/root/usr/lib/dyninst:/opt/rh/gcc-toolset-9/root/usr/lib64:/opt/rh/gcc-toolset-9/root/usr/lib:/home/rocky/srcs/hadoop/hadoop-common-project/hadoop-common/target/test-dir/bats.86234.14082:/tmp<
[exec] not ok 5 hadoop_add_ldlibpath (after order)
[exec] # (in test file hadoop_add_ldlibpath.bats, line 49)
[exec] # `[ "${LD_LIBRARY_PATH}" = "${TMP}:/tmp" ]' failed
[exec] # bindir:
/home/rocky/srcs/hadoop/hadoop-common-project/hadoop-common/src/test/scripts
[exec] # DEBUG: Append colonpath(LD_LIBRARY_PATH):
/home/rocky/srcs/hadoop/hadoop-common-project/hadoop-common/target/test-dir/bats.86248.22235
[exec] # DEBUG: Append colonpath(LD_LIBRARY_PATH): /tmp
[exec] #
>/opt/rh/gcc-toolset-9/root/usr/lib64:/opt/rh/gcc-toolset-9/root/usr/lib:/opt/rh/gcc-toolset-9/root/usr/lib64/dyninst:/opt/rh/gcc-toolset-9/root/usr/lib/dyninst:/opt/rh/gcc-toolset-9/root/usr/lib64:/opt/rh/gcc-toolset-9/root/usr/lib:/home/rocky/srcs/hadoop/hadoop-common-project/hadoop-common/target/test-dir/bats.86248.22235:/tmp<
[exec] not ok 6 hadoop_add_ldlibpath (before order)
[exec] # (in test file hadoop_add_ldlibpath.bats, line 56)
[exec] # `[ "${LD_LIBRARY_PATH}" = "/tmp:${TMP}" ]' failed
[exec] # bindir:
/home/rocky/srcs/hadoop/hadoop-common-project/hadoop-common/src/test/scripts
[exec] # DEBUG: Append colonpath(LD_LIBRARY_PATH):
/home/rocky/srcs/hadoop/hadoop-common-project/hadoop-common/target/test-dir/bats.86262.28642
[exec] # DEBUG: Prepend colonpath(LD_LIBRARY_PATH): /tmp
[exec] #
>/tmp:/opt/rh/gcc-toolset-9/root/usr/lib64:/opt/rh/gcc-toolset-9/root/usr/lib:/opt/rh/gcc-toolset-9/root/usr/lib64/dyninst:/opt/rh/gcc-toolset-9/root/usr/lib/dyninst:/opt/rh/gcc-toolset-9/root/usr/lib64:/opt/rh/gcc-toolset-9/root/usr/lib:/home/rocky/srcs/hadoop/hadoop-common-project/hadoop-common/target/test-dir/bats.86262.28642<
[exec] not ok 7 hadoop_add_ldlibpath (simple dupecheck 2)
[exec] # (from function `hadoop_add_colonpath' in file
../../main/bin/hadoop-functions.sh, line 1251,
[exec] # from function `hadoop_add_ldlibpath' in file
../../main/bin/hadoop-functions.sh, line 1286,
[exec] # in test file hadoop_add_ldlibpath.bats, line 63)
[exec] # `hadoop_add_ldlibpath "${TMP}"' failed
[exec] # bindir:
/home/rocky/srcs/hadoop/hadoop-common-project/hadoop-common/src/test/scripts
[exec] # DEBUG: Append colonpath(LD_LIBRARY_PATH):
/home/rocky/srcs/hadoop/hadoop-common-project/hadoop-common/target/test-dir/bats.86276.18537
[exec] # DEBUG: Append colonpath(LD_LIBRARY_PATH): /tmp
[exec] # DEBUG: Rejected colonpath(LD_LIBRARY_PATH):
/home/rocky/srcs/hadoop/hadoop-common-project/hadoop-common/target/test-dir/bats.86276.18537
[exec] #
[jira] [Updated] (HADOOP-16235) ABFS VersionedFileStatus to declare that it isEncrypted()
[ https://issues.apache.org/jira/browse/HADOOP-16235?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Masatake Iwasaki updated HADOOP-16235: -- Target Version/s: 3.2.5 (was: 3.2.4) > ABFS VersionedFileStatus to declare that it isEncrypted() > - > > Key: HADOOP-16235 > URL: https://issues.apache.org/jira/browse/HADOOP-16235 > Project: Hadoop Common > Issue Type: Sub-task > Components: fs/azure >Affects Versions: 3.2.0 >Reporter: Steve Loughran >Assignee: Masatake Iwasaki >Priority: Minor > Attachments: HADOOP-16235.001.patch, HADOOP-16235.002.patch, > HADOOP-16235.003.patch > > > Files in ABFS are always encrypted; have VersionedFileStatus.isEncrypted() > declare this, presumably just by changing the flag passed to the superclass's > constructor -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-16235) ABFS VersionedFileStatus to declare that it isEncrypted()
[ https://issues.apache.org/jira/browse/HADOOP-16235?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17565911#comment-17565911 ] Masatake Iwasaki commented on HADOOP-16235: --- update the targets to 3.2.5 for preparing 3.2.4 release. > ABFS VersionedFileStatus to declare that it isEncrypted() > - > > Key: HADOOP-16235 > URL: https://issues.apache.org/jira/browse/HADOOP-16235 > Project: Hadoop Common > Issue Type: Sub-task > Components: fs/azure >Affects Versions: 3.2.0 >Reporter: Steve Loughran >Assignee: Masatake Iwasaki >Priority: Minor > Attachments: HADOOP-16235.001.patch, HADOOP-16235.002.patch, > HADOOP-16235.003.patch > > > Files in ABFS are always encrypted; have VersionedFileStatus.isEncrypted() > declare this, presumably just by changing the flag passed to the superclass's > constructor -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-17446) Print the thread parker and lock information in stacks page
[ https://issues.apache.org/jira/browse/HADOOP-17446?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17565910#comment-17565910 ] Masatake Iwasaki commented on HADOOP-17446: --- update the targets to 3.2.5 for preparing 3.2.4 release. > Print the thread parker and lock information in stacks page > --- > > Key: HADOOP-17446 > URL: https://issues.apache.org/jira/browse/HADOOP-17446 > Project: Hadoop Common > Issue Type: New Feature > Components: common >Affects Versions: 3.4.0 >Reporter: Baolong Mao >Assignee: Baolong Mao >Priority: Major > Labels: pull-request-available > Attachments: image-2020-12-25-08-32-32-982.png > > Time Spent: 0.5h > Remaining Estimate: 0h > > Sometimes, our service stuck because of some lock held by other thread, but > we can get nothing from "stacks" for ReadWriteLock, and it is widely used in > our services, like the fslock, cplock, dirlock of namenode. > Luckily, we can get thread parker from Thread object, it can help us see the > thread parker clearly. > !image-2020-12-25-08-32-32-982.png! -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-17706) Problem in installation of Hadoop 3.2 with docket- libc-bin bug
[
https://issues.apache.org/jira/browse/HADOOP-17706?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Masatake Iwasaki updated HADOOP-17706:
--
Target Version/s: 3.2.5 (was: 3.2.4)
> Problem in installation of Hadoop 3.2 with docket- libc-bin bug
> --
>
> Key: HADOOP-17706
> URL: https://issues.apache.org/jira/browse/HADOOP-17706
> Project: Hadoop Common
> Issue Type: Bug
>Affects Versions: 3.2.2
>Reporter: Yuval Rochman
>Priority: Blocker
>
> Hi,
> I got the following bug while installing Hadoop 3.2.2 with docker:
> Processing triggers for libc-bin (2.23-0ubuntu11.2) ...
> WARN engine npm@7.13.0: wanted: \{"node":">=10"} (current:
> \{"node":"4.2.6","npm":"3.5.2"})
> WARN engine npm@7.13.0: wanted: \{"node":">=10"} (current:
> \{"node":"4.2.6","npm":"3.5.2"})
> /usr/local/lib
> `-- (empty)
> npm ERR! Linux 4.15.0-29-generic
> npm ERR! argv "/usr/bin/nodejs" "/usr/bin/npm" "install" "npm@latest" "-g"
> npm ERR! node v4.2.6
> npm ERR! npm v3.5.2
> npm ERR! path /usr/local/lib/node_modules/.staging/@npmcli/ci-detect-c7bf9552
> npm ERR! code ENOENT
> npm ERR! errno -2
> npm ERR! syscall rename
> npm ERR! enoent ENOENT: no such file or directory, rename
> '/usr/local/lib/node_modules/.staging/@npmcli/ci-detect-c7bf9552' ->
> '/usr/local/lib/node_modules/npm/node_modules/@npmcli/ci-detect'
> npm ERR! enoent ENOENT: no such file or directory, rename
> '/usr/local/lib/node_modules/.staging/@npmcli/ci-detect-c7bf9552' ->
> '/usr/local/lib/node_modules/npm/node_modules/@npmcli/ci-detect'
> npm ERR! enoent This is most likely not a problem with npm itself
> npm ERR! enoent and is related to npm not being able to find a file.
> npm ERR! enoent
> npm ERR! Please include the following file with any support request:
> npm ERR! /root/npm-debug.log
> npm ERR! code 1
>
>
> What can I do?
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-17706) Problem in installation of Hadoop 3.2 with docket- libc-bin bug
[
https://issues.apache.org/jira/browse/HADOOP-17706?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17565908#comment-17565908
]
Masatake Iwasaki commented on HADOOP-17706:
---
update the targets to 3.2.5 for preparing 3.2.4 release.
> Problem in installation of Hadoop 3.2 with docket- libc-bin bug
> --
>
> Key: HADOOP-17706
> URL: https://issues.apache.org/jira/browse/HADOOP-17706
> Project: Hadoop Common
> Issue Type: Bug
>Affects Versions: 3.2.2
>Reporter: Yuval Rochman
>Priority: Major
>
> Hi,
> I got the following bug while installing Hadoop 3.2.2 with docker:
> Processing triggers for libc-bin (2.23-0ubuntu11.2) ...
> WARN engine npm@7.13.0: wanted: \{"node":">=10"} (current:
> \{"node":"4.2.6","npm":"3.5.2"})
> WARN engine npm@7.13.0: wanted: \{"node":">=10"} (current:
> \{"node":"4.2.6","npm":"3.5.2"})
> /usr/local/lib
> `-- (empty)
> npm ERR! Linux 4.15.0-29-generic
> npm ERR! argv "/usr/bin/nodejs" "/usr/bin/npm" "install" "npm@latest" "-g"
> npm ERR! node v4.2.6
> npm ERR! npm v3.5.2
> npm ERR! path /usr/local/lib/node_modules/.staging/@npmcli/ci-detect-c7bf9552
> npm ERR! code ENOENT
> npm ERR! errno -2
> npm ERR! syscall rename
> npm ERR! enoent ENOENT: no such file or directory, rename
> '/usr/local/lib/node_modules/.staging/@npmcli/ci-detect-c7bf9552' ->
> '/usr/local/lib/node_modules/npm/node_modules/@npmcli/ci-detect'
> npm ERR! enoent ENOENT: no such file or directory, rename
> '/usr/local/lib/node_modules/.staging/@npmcli/ci-detect-c7bf9552' ->
> '/usr/local/lib/node_modules/npm/node_modules/@npmcli/ci-detect'
> npm ERR! enoent This is most likely not a problem with npm itself
> npm ERR! enoent and is related to npm not being able to find a file.
> npm ERR! enoent
> npm ERR! Please include the following file with any support request:
> npm ERR! /root/npm-debug.log
> npm ERR! code 1
>
>
> What can I do?
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-17446) Print the thread parker and lock information in stacks page
[ https://issues.apache.org/jira/browse/HADOOP-17446?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Masatake Iwasaki updated HADOOP-17446: -- Target Version/s: 3.4.0, 3.3.9, 3.2.5 (was: 3.4.0, 3.2.4, 3.3.9) > Print the thread parker and lock information in stacks page > --- > > Key: HADOOP-17446 > URL: https://issues.apache.org/jira/browse/HADOOP-17446 > Project: Hadoop Common > Issue Type: New Feature > Components: common >Affects Versions: 3.4.0 >Reporter: Baolong Mao >Assignee: Baolong Mao >Priority: Major > Labels: pull-request-available > Attachments: image-2020-12-25-08-32-32-982.png > > Time Spent: 0.5h > Remaining Estimate: 0h > > Sometimes, our service stuck because of some lock held by other thread, but > we can get nothing from "stacks" for ReadWriteLock, and it is widely used in > our services, like the fslock, cplock, dirlock of namenode. > Luckily, we can get thread parker from Thread object, it can help us see the > thread parker clearly. > !image-2020-12-25-08-32-32-982.png! -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-17706) Problem in installation of Hadoop 3.2 with docket- libc-bin bug
[
https://issues.apache.org/jira/browse/HADOOP-17706?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Masatake Iwasaki updated HADOOP-17706:
--
Priority: Major (was: Blocker)
> Problem in installation of Hadoop 3.2 with docket- libc-bin bug
> --
>
> Key: HADOOP-17706
> URL: https://issues.apache.org/jira/browse/HADOOP-17706
> Project: Hadoop Common
> Issue Type: Bug
>Affects Versions: 3.2.2
>Reporter: Yuval Rochman
>Priority: Major
>
> Hi,
> I got the following bug while installing Hadoop 3.2.2 with docker:
> Processing triggers for libc-bin (2.23-0ubuntu11.2) ...
> WARN engine npm@7.13.0: wanted: \{"node":">=10"} (current:
> \{"node":"4.2.6","npm":"3.5.2"})
> WARN engine npm@7.13.0: wanted: \{"node":">=10"} (current:
> \{"node":"4.2.6","npm":"3.5.2"})
> /usr/local/lib
> `-- (empty)
> npm ERR! Linux 4.15.0-29-generic
> npm ERR! argv "/usr/bin/nodejs" "/usr/bin/npm" "install" "npm@latest" "-g"
> npm ERR! node v4.2.6
> npm ERR! npm v3.5.2
> npm ERR! path /usr/local/lib/node_modules/.staging/@npmcli/ci-detect-c7bf9552
> npm ERR! code ENOENT
> npm ERR! errno -2
> npm ERR! syscall rename
> npm ERR! enoent ENOENT: no such file or directory, rename
> '/usr/local/lib/node_modules/.staging/@npmcli/ci-detect-c7bf9552' ->
> '/usr/local/lib/node_modules/npm/node_modules/@npmcli/ci-detect'
> npm ERR! enoent ENOENT: no such file or directory, rename
> '/usr/local/lib/node_modules/.staging/@npmcli/ci-detect-c7bf9552' ->
> '/usr/local/lib/node_modules/npm/node_modules/@npmcli/ci-detect'
> npm ERR! enoent This is most likely not a problem with npm itself
> npm ERR! enoent and is related to npm not being able to find a file.
> npm ERR! enoent
> npm ERR! Please include the following file with any support request:
> npm ERR! /root/npm-debug.log
> npm ERR! code 1
>
>
> What can I do?
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-17583) Enable shelldoc check in GitHub PR
[ https://issues.apache.org/jira/browse/HADOOP-17583?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17565909#comment-17565909 ] Masatake Iwasaki commented on HADOOP-17583: --- update the targets to 3.2.5 for preparing 3.2.4 release. > Enable shelldoc check in GitHub PR > -- > > Key: HADOOP-17583 > URL: https://issues.apache.org/jira/browse/HADOOP-17583 > Project: Hadoop Common > Issue Type: Bug > Components: build >Reporter: Akira Ajisaka >Priority: Major > > After HADOOP-17570, we can enable shelldoc check again because the commit > hash of Yetus includes YETUS-1099. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-17583) Enable shelldoc check in GitHub PR
[ https://issues.apache.org/jira/browse/HADOOP-17583?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Masatake Iwasaki updated HADOOP-17583: -- Target Version/s: 3.4.0, 3.3.9, 3.2.5 (was: 3.4.0, 3.2.4, 3.3.9) > Enable shelldoc check in GitHub PR > -- > > Key: HADOOP-17583 > URL: https://issues.apache.org/jira/browse/HADOOP-17583 > Project: Hadoop Common > Issue Type: Bug > Components: build >Reporter: Akira Ajisaka >Priority: Major > > After HADOOP-17570, we can enable shelldoc check again because the commit > hash of Yetus includes YETUS-1099. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-18301) Upgrade commons-io to 2.11.0
[ https://issues.apache.org/jira/browse/HADOOP-18301?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Masatake Iwasaki updated HADOOP-18301: -- Target Version/s: 3.3.4, 3.2.5 (was: 3.2.4, 3.3.4) > Upgrade commons-io to 2.11.0 > > > Key: HADOOP-18301 > URL: https://issues.apache.org/jira/browse/HADOOP-18301 > Project: Hadoop Common > Issue Type: Improvement >Affects Versions: 3.2.3, 3.3.3 >Reporter: Ashutosh Gupta >Assignee: Ashutosh Gupta >Priority: Minor > Labels: pull-request-available > Time Spent: 1h 50m > Remaining Estimate: 0h > > Current version 2.8.0 is almost ~2 years old > Upgrading to new release to keep up for new features and bug fixes. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-18301) Upgrade commons-io to 2.11.0
[ https://issues.apache.org/jira/browse/HADOOP-18301?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17565907#comment-17565907 ] Masatake Iwasaki commented on HADOOP-18301: --- update the targets to 3.2.5 for preparing 3.2.4 release. > Upgrade commons-io to 2.11.0 > > > Key: HADOOP-18301 > URL: https://issues.apache.org/jira/browse/HADOOP-18301 > Project: Hadoop Common > Issue Type: Improvement >Affects Versions: 3.2.3, 3.3.3 >Reporter: Ashutosh Gupta >Assignee: Ashutosh Gupta >Priority: Minor > Labels: pull-request-available > Time Spent: 1h 50m > Remaining Estimate: 0h > > Current version 2.8.0 is almost ~2 years old > Upgrading to new release to keep up for new features and bug fixes. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-18334) Fix create-release to address removal of GPG_AGENT_INFO in branch-3.2
[
https://issues.apache.org/jira/browse/HADOOP-18334?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Masatake Iwasaki updated HADOOP-18334:
--
Fix Version/s: 3.2.4
Hadoop Flags: Reviewed
Resolution: Fixed
Status: Resolved (was: Patch Available)
> Fix create-release to address removal of GPG_AGENT_INFO in branch-3.2
> -
>
> Key: HADOOP-18334
> URL: https://issues.apache.org/jira/browse/HADOOP-18334
> Project: Hadoop Common
> Issue Type: Bug
> Components: build
>Affects Versions: 3.2.3
>Reporter: Masatake Iwasaki
>Assignee: Masatake Iwasaki
>Priority: Major
> Labels: pull-request-available
> Fix For: 3.2.4
>
> Time Spent: 1h
> Remaining Estimate: 0h
>
> gpg v2.1 and above does not export GPG_AGENT_INFO. create-release script need
> to export the info by itself to make {{--sign}} work. It was addressed as
> part of HADOOP-16797 in branch-3.3 and trunk. Since we can not backport
> aarch64 support to branch-3.2, I filed this issue for branch-3.2 only.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-17126) implement non-guava Precondition checkNotNull
[
https://issues.apache.org/jira/browse/HADOOP-17126?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Masatake Iwasaki updated HADOOP-17126:
--
Fix Version/s: 3.2.4
(was: 3.2.3)
> implement non-guava Precondition checkNotNull
> -
>
> Key: HADOOP-17126
> URL: https://issues.apache.org/jira/browse/HADOOP-17126
> Project: Hadoop Common
> Issue Type: Sub-task
>Reporter: Ahmed Hussein
>Assignee: Ahmed Hussein
>Priority: Major
> Labels: pull-request-available
> Fix For: 3.3.2, 3.2.4
>
> Attachments: HADOOP-17126.001.patch, HADOOP-17126.002.patch
>
> Time Spent: 3h 20m
> Remaining Estimate: 0h
>
> As part In order to replace Guava Preconditions, we need to implement our own
> versions of the API.
> This Jira is to create {{checkNotNull}} in a new package dubbed {{unguava}}.
> +The plan is as follows+
> * create a new {{package org.apache.hadoop.util.unguava;}}
> * {{create class Validate}}
> * implement {{package org.apache.hadoop.util.unguava.Validate;}} with the
> following interface
> ** {{checkNotNull(final T obj)}}
> ** {{checkNotNull(final T reference, final Object errorMessage)}}
> ** {{checkNotNull(final T obj, final String message, final Object...
> values)}}
> ** {{checkNotNull(final T obj,final Supplier msgSupplier)}}
> * {{guava.preconditions used String.lenientformat which suppressed
> exceptions caused by string formatting of the exception message . So, in
> order to avoid changing the behavior, the implementation catches Exceptions
> triggered by building the message (IllegalFormat, InsufficientArg,
> NullPointer..etc)}}
> * {{After merging the new class, we can replace
> guava.Preconditions.checkNotNull by {{unguava.Validate.checkNotNull
> * We need the change to go into trunk, 3.1, 3.2, and 3.3
>
> Similar Jiras will be created to implement checkState, checkArgument,
> checkIndex
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Resolved] (HADOOP-18100) Change scope of inner classes in InodeTree to make them accessible outside package
[ https://issues.apache.org/jira/browse/HADOOP-18100?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Masatake Iwasaki resolved HADOOP-18100. --- Fix Version/s: 3.4.0 3.2.4 3.3.9 Resolution: Fixed > Change scope of inner classes in InodeTree to make them accessible outside > package > -- > > Key: HADOOP-18100 > URL: https://issues.apache.org/jira/browse/HADOOP-18100 > Project: Hadoop Common > Issue Type: Bug >Reporter: Abhishek Das >Assignee: Abhishek Das >Priority: Major > Labels: pull-request-available > Fix For: 3.4.0, 3.2.4, 3.3.9 > > Time Spent: 1h 40m > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-18313) AliyunOSS: AliyunOSSBlockOutputStream should not mark the temporary file for deletion
[ https://issues.apache.org/jira/browse/HADOOP-18313?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Masatake Iwasaki updated HADOOP-18313: -- Affects Version/s: 3.2.4 (was: 3.2.3) > AliyunOSS: AliyunOSSBlockOutputStream should not mark the temporary file for > deletion > - > > Key: HADOOP-18313 > URL: https://issues.apache.org/jira/browse/HADOOP-18313 > Project: Hadoop Common > Issue Type: Bug > Components: fs/oss >Affects Versions: 2.10.2, 3.2.4, 3.3.3 >Reporter: wujinhu >Assignee: wujinhu >Priority: Major > Labels: pull-request-available > Time Spent: 50m > Remaining Estimate: 0h > > AliyunOSSBlockOutputStream buffers data in local directory before uploading > to OSS. It uses LocalDirAllocator.createTmpFileForWrite which will create a > temp file on this directory and invoke DeleteOnExitHook.add to add the path > to a set and will be deleted on VM exit through a shutdown hook. However, the > size of set will be growing if the VM does not exit. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-18334) Fix create-release to address removal of GPG_AGENT_INFO in branch-3.2
[
https://issues.apache.org/jira/browse/HADOOP-18334?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Masatake Iwasaki updated HADOOP-18334:
--
Status: Patch Available (was: Open)
> Fix create-release to address removal of GPG_AGENT_INFO in branch-3.2
> -
>
> Key: HADOOP-18334
> URL: https://issues.apache.org/jira/browse/HADOOP-18334
> Project: Hadoop Common
> Issue Type: Bug
> Components: build
>Affects Versions: 3.2.3
>Reporter: Masatake Iwasaki
>Assignee: Masatake Iwasaki
>Priority: Major
> Labels: pull-request-available
> Time Spent: 20m
> Remaining Estimate: 0h
>
> gpg v2.1 and above does not export GPG_AGENT_INFO. create-release script need
> to export the info by itself to make {{--sign}} work. It was addressed as
> part of HADOOP-16797 in branch-3.3 and trunk. Since we can not backport
> aarch64 support to branch-3.2, I filed this issue for branch-3.2 only.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-18334) Fix create-release to address removal of GPG_AGENT_INFO in branch-3.2
[
https://issues.apache.org/jira/browse/HADOOP-18334?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17565259#comment-17565259
]
Masatake Iwasaki commented on HADOOP-18334:
---
{noformat}
Hadoop Release Creator
Version to create : 3.2.4
Release Candidate Label:
Source Version : 3.2.4
starting gpg agent
ERROR: Unable to launch or acquire gpg-agent. Disable signing.
{noformat}
> Fix create-release to address removal of GPG_AGENT_INFO in branch-3.2
> -
>
> Key: HADOOP-18334
> URL: https://issues.apache.org/jira/browse/HADOOP-18334
> Project: Hadoop Common
> Issue Type: Bug
> Components: build
>Affects Versions: 3.2.3
>Reporter: Masatake Iwasaki
>Assignee: Masatake Iwasaki
>Priority: Major
>
> gpg v2.1 and above does not export GPG_AGENT_INFO. create-release script need
> to export the info by itself to make {{--sign}} work. It was addressed as
> part of HADOOP-16797 in branch-3.3 and trunk. Since we can not backport
> aarch64 support to branch-3.2, I filed this issue for branch-3.2 only.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Created] (HADOOP-18334) Fix create-release to address removal of GPG_AGENT_INFO in branch-3.2
Masatake Iwasaki created HADOOP-18334:
-
Summary: Fix create-release to address removal of GPG_AGENT_INFO
in branch-3.2
Key: HADOOP-18334
URL: https://issues.apache.org/jira/browse/HADOOP-18334
Project: Hadoop Common
Issue Type: Bug
Components: build
Affects Versions: 3.2.3
Reporter: Masatake Iwasaki
Assignee: Masatake Iwasaki
gpg v2.1 and above does not export GPG_AGENT_INFO. create-release script need
to export the info by itself to make {{--sign}} work. It was addressed as part
of HADOOP-16797 in branch-3.3 and trunk. Since we can not backport aarch64
support to branch-3.2, I filed this issue for branch-3.2 only.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-17569) Building native code fails on Fedora 33
[
https://issues.apache.org/jira/browse/HADOOP-17569?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17562843#comment-17562843
]
Masatake Iwasaki commented on HADOOP-17569:
---
cherry-picked to branch-2.10.
> Building native code fails on Fedora 33
> ---
>
> Key: HADOOP-17569
> URL: https://issues.apache.org/jira/browse/HADOOP-17569
> Project: Hadoop Common
> Issue Type: Improvement
> Components: build, common
>Reporter: Kengo Seki
>Assignee: Masatake Iwasaki
>Priority: Major
> Labels: pull-request-available
> Fix For: 3.3.1, 3.4.0, 3.2.3, 2.10.3
>
> Time Spent: 1h
> Remaining Estimate: 0h
>
> I tried to build native code on Fedora 33, in which glibc 2.32 is installed
> by default, but it failed with the following error.
> {code:java}
> $ cat /etc/redhat-release
> Fedora release 33 (Thirty Three)
> $ sudo dnf info --installed glibc
> Installed Packages
> Name : glibc
> Version : 2.32
> Release : 1.fc33
> Architecture : x86_64
> Size : 17 M
> Source : glibc-2.32-1.fc33.src.rpm
> Repository : @System
> From repo: anaconda
> Summary : The GNU libc libraries
> URL : http://www.gnu.org/software/glibc/
> License : LGPLv2+ and LGPLv2+ with exceptions and GPLv2+ and GPLv2+ with
> exceptions and BSD and Inner-Net and ISC and Public Domain and GFDL
> Description : The glibc package contains standard libraries which are used by
> : multiple programs on the system. In order to save disk space
> and
> : memory, as well as to make upgrading easier, common system
> code is
> : kept in one place and shared between programs. This particular
> package
> : contains the most important sets of shared libraries: the
> standard C
> : library and the standard math library. Without these two
> libraries, a
> : Linux system will not function.
> $ mvn clean compile -Pnative
> ...
> [INFO] Running make -j 1 VERBOSE=1
> [WARNING] /usr/bin/cmake
> -S/home/vagrant/hadoop/hadoop-common-project/hadoop-common/src
> -B/home/vagrant/hadoop/hadoop-common-project/hadoop-common/target/native
> --check-build-system CMakeFiles/Makefile.cmake 0
> [WARNING] /usr/bin/cmake -E cmake_progress_start
> /home/vagrant/hadoop/hadoop-common-project/hadoop-common/target/native/CMakeFiles
>
> /home/vagrant/hadoop/hadoop-common-project/hadoop-common/target/native//CMakeFiles/progress.marks
> [WARNING] make -f CMakeFiles/Makefile2 all
> [WARNING] make[1]: Entering directory
> '/home/vagrant/hadoop/hadoop-common-project/hadoop-common/target/native'
> [WARNING] make -f CMakeFiles/hadoop_static.dir/build.make
> CMakeFiles/hadoop_static.dir/depend
> [WARNING] make[2]: Entering directory
> '/home/vagrant/hadoop/hadoop-common-project/hadoop-common/target/native'
> [WARNING] cd
> /home/vagrant/hadoop/hadoop-common-project/hadoop-common/target/native &&
> /usr/bin/cmake -E cmake_depends "Unix Makefiles"
> /home/vagrant/hadoop/hadoop-common-project/hadoop-common/src
> /home/vagrant/hadoop/hadoop-common-project/hadoop-common/src
> /home/vagrant/hadoop/hadoop-common-project/hadoop-common/target/native
> /home/vagrant/hadoop/hadoop-common-project/hadoop-common/target/native
> /home/vagrant/hadoop/hadoop-common-project/hadoop-common/target/native/CMakeFiles/hadoop_static.dir/DependInfo.cmake
> --color=
> [WARNING] Dependee
> "/home/vagrant/hadoop/hadoop-common-project/hadoop-common/target/native/CMakeFiles/hadoop_static.dir/DependInfo.cmake"
> is newer than depender
> "/home/vagrant/hadoop/hadoop-common-project/hadoop-common/target/native/CMakeFiles/hadoop_static.dir/depend.internal".
> [WARNING] Dependee
> "/home/vagrant/hadoop/hadoop-common-project/hadoop-common/target/native/CMakeFiles/CMakeDirectoryInformation.cmake"
> is newer than depender
> "/home/vagrant/hadoop/hadoop-common-project/hadoop-common/target/native/CMakeFiles/hadoop_static.dir/depend.internal".
> [WARNING] Scanning dependencies of target hadoop_static
> [WARNING] make[2]: Leaving directory
> '/home/vagrant/hadoop/hadoop-common-project/hadoop-common/target/native'
> [WARNING] make -f CMakeFiles/hadoop_static.dir/build.make
> CMakeFiles/hadoop_static.dir/build
> [WARNING] make[2]: Entering directory
> '/home/vagrant/hadoop/hadoop-common-project/hadoop-common/target/native'
> [WARNING] [ 2%] Building C object
> CMakeFiles/hadoop_static.dir/main/native/src/exception.c.o
> [WARNING] /usr/bin/cc
> -I/home/vagrant/hadoop/hadoop-common-project/hadoop-common/target/native/javah
>
> -I/home/vagrant/hadoop/hadoop-common-project/hadoop-common/src/main/native/src
> -I/home/vagrant/hadoop/hadoop-common-project/hadoop-common/src
> -I/home/vagrant/hadoop/hadoop-common-project/hadoop-common/src/src
>
[jira] [Updated] (HADOOP-17569) Building native code fails on Fedora 33
[
https://issues.apache.org/jira/browse/HADOOP-17569?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Masatake Iwasaki updated HADOOP-17569:
--
Fix Version/s: 2.10.3
> Building native code fails on Fedora 33
> ---
>
> Key: HADOOP-17569
> URL: https://issues.apache.org/jira/browse/HADOOP-17569
> Project: Hadoop Common
> Issue Type: Improvement
> Components: build, common
>Reporter: Kengo Seki
>Assignee: Masatake Iwasaki
>Priority: Major
> Labels: pull-request-available
> Fix For: 3.3.1, 3.4.0, 3.2.3, 2.10.3
>
> Time Spent: 1h
> Remaining Estimate: 0h
>
> I tried to build native code on Fedora 33, in which glibc 2.32 is installed
> by default, but it failed with the following error.
> {code:java}
> $ cat /etc/redhat-release
> Fedora release 33 (Thirty Three)
> $ sudo dnf info --installed glibc
> Installed Packages
> Name : glibc
> Version : 2.32
> Release : 1.fc33
> Architecture : x86_64
> Size : 17 M
> Source : glibc-2.32-1.fc33.src.rpm
> Repository : @System
> From repo: anaconda
> Summary : The GNU libc libraries
> URL : http://www.gnu.org/software/glibc/
> License : LGPLv2+ and LGPLv2+ with exceptions and GPLv2+ and GPLv2+ with
> exceptions and BSD and Inner-Net and ISC and Public Domain and GFDL
> Description : The glibc package contains standard libraries which are used by
> : multiple programs on the system. In order to save disk space
> and
> : memory, as well as to make upgrading easier, common system
> code is
> : kept in one place and shared between programs. This particular
> package
> : contains the most important sets of shared libraries: the
> standard C
> : library and the standard math library. Without these two
> libraries, a
> : Linux system will not function.
> $ mvn clean compile -Pnative
> ...
> [INFO] Running make -j 1 VERBOSE=1
> [WARNING] /usr/bin/cmake
> -S/home/vagrant/hadoop/hadoop-common-project/hadoop-common/src
> -B/home/vagrant/hadoop/hadoop-common-project/hadoop-common/target/native
> --check-build-system CMakeFiles/Makefile.cmake 0
> [WARNING] /usr/bin/cmake -E cmake_progress_start
> /home/vagrant/hadoop/hadoop-common-project/hadoop-common/target/native/CMakeFiles
>
> /home/vagrant/hadoop/hadoop-common-project/hadoop-common/target/native//CMakeFiles/progress.marks
> [WARNING] make -f CMakeFiles/Makefile2 all
> [WARNING] make[1]: Entering directory
> '/home/vagrant/hadoop/hadoop-common-project/hadoop-common/target/native'
> [WARNING] make -f CMakeFiles/hadoop_static.dir/build.make
> CMakeFiles/hadoop_static.dir/depend
> [WARNING] make[2]: Entering directory
> '/home/vagrant/hadoop/hadoop-common-project/hadoop-common/target/native'
> [WARNING] cd
> /home/vagrant/hadoop/hadoop-common-project/hadoop-common/target/native &&
> /usr/bin/cmake -E cmake_depends "Unix Makefiles"
> /home/vagrant/hadoop/hadoop-common-project/hadoop-common/src
> /home/vagrant/hadoop/hadoop-common-project/hadoop-common/src
> /home/vagrant/hadoop/hadoop-common-project/hadoop-common/target/native
> /home/vagrant/hadoop/hadoop-common-project/hadoop-common/target/native
> /home/vagrant/hadoop/hadoop-common-project/hadoop-common/target/native/CMakeFiles/hadoop_static.dir/DependInfo.cmake
> --color=
> [WARNING] Dependee
> "/home/vagrant/hadoop/hadoop-common-project/hadoop-common/target/native/CMakeFiles/hadoop_static.dir/DependInfo.cmake"
> is newer than depender
> "/home/vagrant/hadoop/hadoop-common-project/hadoop-common/target/native/CMakeFiles/hadoop_static.dir/depend.internal".
> [WARNING] Dependee
> "/home/vagrant/hadoop/hadoop-common-project/hadoop-common/target/native/CMakeFiles/CMakeDirectoryInformation.cmake"
> is newer than depender
> "/home/vagrant/hadoop/hadoop-common-project/hadoop-common/target/native/CMakeFiles/hadoop_static.dir/depend.internal".
> [WARNING] Scanning dependencies of target hadoop_static
> [WARNING] make[2]: Leaving directory
> '/home/vagrant/hadoop/hadoop-common-project/hadoop-common/target/native'
> [WARNING] make -f CMakeFiles/hadoop_static.dir/build.make
> CMakeFiles/hadoop_static.dir/build
> [WARNING] make[2]: Entering directory
> '/home/vagrant/hadoop/hadoop-common-project/hadoop-common/target/native'
> [WARNING] [ 2%] Building C object
> CMakeFiles/hadoop_static.dir/main/native/src/exception.c.o
> [WARNING] /usr/bin/cc
> -I/home/vagrant/hadoop/hadoop-common-project/hadoop-common/target/native/javah
>
> -I/home/vagrant/hadoop/hadoop-common-project/hadoop-common/src/main/native/src
> -I/home/vagrant/hadoop/hadoop-common-project/hadoop-common/src
> -I/home/vagrant/hadoop/hadoop-common-project/hadoop-common/src/src
>
[jira] [Updated] (HADOOP-17196) Fix C/C++ standard warnings
[
https://issues.apache.org/jira/browse/HADOOP-17196?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Masatake Iwasaki updated HADOOP-17196:
--
Fix Version/s: (was: 3.3.1)
> Fix C/C++ standard warnings
> ---
>
> Key: HADOOP-17196
> URL: https://issues.apache.org/jira/browse/HADOOP-17196
> Project: Hadoop Common
> Issue Type: Bug
> Components: build
>Affects Versions: 3.1.3
> Environment: Windows 10 Pro 64-bit
>Reporter: Gautham Banasandra
>Assignee: Gautham Banasandra
>Priority: Major
> Fix For: 3.4.0
>
> Original Estimate: 168h
> Remaining Estimate: 168h
>
> The C/C++ language standard is not specified in a cross-compiler manner. Even
> though it's as straight forward as passing *-std* as compiler arguments, not
> all the values are supported by all the compilers. For example, compilation
> with the Visual C++ compiler on Windows with *-std=gnu99* flag causes the
> following warning -
> {code:java}
> cl : command line warning D9002: ignoring unknown option '-std=gnu99'
> [Z:\hadoop-hdfs-project\hadoop-hdfs-native-client\target\native\main\native\libhdfs-examples\hdfs_read.vcxproj]
> {code}
> Thus, we need to use the appropriate flags provided by CMake to specify the
> C/C++ standards so that it is compiler friendly.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-17196) Fix C/C++ standard warnings
[
https://issues.apache.org/jira/browse/HADOOP-17196?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Masatake Iwasaki updated HADOOP-17196:
--
Fix Version/s: (was: 3.2.2)
> Fix C/C++ standard warnings
> ---
>
> Key: HADOOP-17196
> URL: https://issues.apache.org/jira/browse/HADOOP-17196
> Project: Hadoop Common
> Issue Type: Bug
> Components: build
>Affects Versions: 3.1.3
> Environment: Windows 10 Pro 64-bit
>Reporter: Gautham Banasandra
>Assignee: Gautham Banasandra
>Priority: Major
> Fix For: 3.3.1, 3.4.0
>
> Original Estimate: 168h
> Remaining Estimate: 168h
>
> The C/C++ language standard is not specified in a cross-compiler manner. Even
> though it's as straight forward as passing *-std* as compiler arguments, not
> all the values are supported by all the compilers. For example, compilation
> with the Visual C++ compiler on Windows with *-std=gnu99* flag causes the
> following warning -
> {code:java}
> cl : command line warning D9002: ignoring unknown option '-std=gnu99'
> [Z:\hadoop-hdfs-project\hadoop-hdfs-native-client\target\native\main\native\libhdfs-examples\hdfs_read.vcxproj]
> {code}
> Thus, we need to use the appropriate flags provided by CMake to specify the
> C/C++ standards so that it is compiler friendly.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-17196) Fix C/C++ standard warnings
[
https://issues.apache.org/jira/browse/HADOOP-17196?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17560682#comment-17560682
]
Masatake Iwasaki commented on HADOOP-17196:
---
We should accept it if we can not fix critical bug/regression without the
change. It is not the case here.
> Fix C/C++ standard warnings
> ---
>
> Key: HADOOP-17196
> URL: https://issues.apache.org/jira/browse/HADOOP-17196
> Project: Hadoop Common
> Issue Type: Bug
> Components: build
>Affects Versions: 3.1.3
> Environment: Windows 10 Pro 64-bit
>Reporter: Gautham Banasandra
>Assignee: Gautham Banasandra
>Priority: Major
> Fix For: 3.2.2, 3.3.1, 3.4.0
>
> Original Estimate: 168h
> Remaining Estimate: 168h
>
> The C/C++ language standard is not specified in a cross-compiler manner. Even
> though it's as straight forward as passing *-std* as compiler arguments, not
> all the values are supported by all the compilers. For example, compilation
> with the Visual C++ compiler on Windows with *-std=gnu99* flag causes the
> following warning -
> {code:java}
> cl : command line warning D9002: ignoring unknown option '-std=gnu99'
> [Z:\hadoop-hdfs-project\hadoop-hdfs-native-client\target\native\main\native\libhdfs-examples\hdfs_read.vcxproj]
> {code}
> Thus, we need to use the appropriate flags provided by CMake to specify the
> C/C++ standards so that it is compiler friendly.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-17196) Fix C/C++ standard warnings
[
https://issues.apache.org/jira/browse/HADOOP-17196?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17560678#comment-17560678
]
Masatake Iwasaki commented on HADOOP-17196:
---
[~gaurava] I meant the change of building requirement like you mentioned should
not be done in a *point release*. I will revert this from branch-3.2 and
branch-3.3.
> Fix C/C++ standard warnings
> ---
>
> Key: HADOOP-17196
> URL: https://issues.apache.org/jira/browse/HADOOP-17196
> Project: Hadoop Common
> Issue Type: Bug
> Components: build
>Affects Versions: 3.1.3
> Environment: Windows 10 Pro 64-bit
>Reporter: Gautham Banasandra
>Assignee: Gautham Banasandra
>Priority: Major
> Fix For: 3.2.2, 3.3.1, 3.4.0
>
> Original Estimate: 168h
> Remaining Estimate: 168h
>
> The C/C++ language standard is not specified in a cross-compiler manner. Even
> though it's as straight forward as passing *-std* as compiler arguments, not
> all the values are supported by all the compilers. For example, compilation
> with the Visual C++ compiler on Windows with *-std=gnu99* flag causes the
> following warning -
> {code:java}
> cl : command line warning D9002: ignoring unknown option '-std=gnu99'
> [Z:\hadoop-hdfs-project\hadoop-hdfs-native-client\target\native\main\native\libhdfs-examples\hdfs_read.vcxproj]
> {code}
> Thus, we need to use the appropriate flags provided by CMake to specify the
> C/C++ standards so that it is compiler friendly.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-18309) Upgrade bundled Tomcat to 8.5.76 or higher
[
https://issues.apache.org/jira/browse/HADOOP-18309?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Masatake Iwasaki updated HADOOP-18309:
--
Priority: Major (was: Critical)
> Upgrade bundled Tomcat to 8.5.76 or higher
> --
>
> Key: HADOOP-18309
> URL: https://issues.apache.org/jira/browse/HADOOP-18309
> Project: Hadoop Common
> Issue Type: Improvement
> Components: httpfs, kms
>Affects Versions: 2.10.1, 2.10.2
>Reporter: Ashutosh Gupta
>Assignee: Ashutosh Gupta
>Priority: Major
> Labels: pull-request-available
> Time Spent: 0.5h
> Remaining Estimate: 0h
>
> Currently we are using 8.5.75 which is affected by
> {color:#33}CVE-2022-25762{color}
> More Details -
> [https://lists.apache.org/thread/qzkqh2819x6zsmj7vwdf14ng2fdgckw7]
> Lets upgrade 8.5.76 or higher
>
>
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Resolved] (HADOOP-18309) Upgrade bundled Tomcat to 8.5.76 or higher
[
https://issues.apache.org/jira/browse/HADOOP-18309?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Masatake Iwasaki resolved HADOOP-18309.
---
Fix Version/s: 2.10.3
Hadoop Flags: Reviewed
Resolution: Fixed
> Upgrade bundled Tomcat to 8.5.76 or higher
> --
>
> Key: HADOOP-18309
> URL: https://issues.apache.org/jira/browse/HADOOP-18309
> Project: Hadoop Common
> Issue Type: Improvement
> Components: httpfs, kms
>Affects Versions: 2.10.1, 2.10.2
>Reporter: Ashutosh Gupta
>Assignee: Ashutosh Gupta
>Priority: Major
> Labels: pull-request-available
> Fix For: 2.10.3
>
> Time Spent: 0.5h
> Remaining Estimate: 0h
>
> Currently we are using 8.5.75 which is affected by
> {color:#33}CVE-2022-25762{color}
> More Details -
> [https://lists.apache.org/thread/qzkqh2819x6zsmj7vwdf14ng2fdgckw7]
> Lets upgrade 8.5.76 or higher
>
>
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-18299) Fix unexpected contents in hadoop-client-check-test-invariants on aarch64
[
https://issues.apache.org/jira/browse/HADOOP-18299?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17555849#comment-17555849
]
Masatake Iwasaki commented on HADOOP-18299:
---
[~ayushtkn] Since I got no issue on x86_64, I assumed that platform specific
issue first. It turned out that something wrong in my ~/.m2 on aarch64 VM.
> Fix unexpected contents in hadoop-client-check-test-invariants on aarch64
> -
>
> Key: HADOOP-18299
> URL: https://issues.apache.org/jira/browse/HADOOP-18299
> Project: Hadoop Common
> Issue Type: Bug
>Affects Versions: 3.3.3
>Reporter: Masatake Iwasaki
>Priority: Major
> Labels: pull-request-available
> Time Spent: 0.5h
> Remaining Estimate: 0h
>
> Build on aarch64 failed due to banned classes.
> {{com.github.stephenc.findbugs:findbugs-annotations}} seems to be pulled
> somewhere.
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Resolved] (HADOOP-18299) Fix unexpected contents in hadoop-client-check-test-invariants on aarch64
[
https://issues.apache.org/jira/browse/HADOOP-18299?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Masatake Iwasaki resolved HADOOP-18299.
---
Assignee: (was: Masatake Iwasaki)
Resolution: Cannot Reproduce
I can not reproduce this after wiping out ~/.m2/repository.
> Fix unexpected contents in hadoop-client-check-test-invariants on aarch64
> -
>
> Key: HADOOP-18299
> URL: https://issues.apache.org/jira/browse/HADOOP-18299
> Project: Hadoop Common
> Issue Type: Bug
>Affects Versions: 3.3.3
>Reporter: Masatake Iwasaki
>Priority: Major
> Labels: pull-request-available
> Time Spent: 20m
> Remaining Estimate: 0h
>
> Build on aarch64 failed due to banned classes.
> {{com.github.stephenc.findbugs:findbugs-annotations}} seems to be pulled
> somewhere.
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-18299) Fix unexpected contents in hadoop-client-check-test-invariants on aarch64
[
https://issues.apache.org/jira/browse/HADOOP-18299?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17555825#comment-17555825
]
Masatake Iwasaki commented on HADOOP-18299:
---
I can reproduce this only in branch-3.3 by the following cmdline.
{noformat}
$ mvn -Pdist -Psrc -Dtar -Dzookeeper.version=3.5.9 -DskipTests -DskipTest
-DskipITs install
{noformat}
Since the value of zookeeper.version in hadoop-project/pom.xml is 3.5.6, the
issue looks only related to newer zookeeper 3.5.x.
There is no issue in trunk in which the zookeeper.version is 3.6.3.
> Fix unexpected contents in hadoop-client-check-test-invariants on aarch64
> -
>
> Key: HADOOP-18299
> URL: https://issues.apache.org/jira/browse/HADOOP-18299
> Project: Hadoop Common
> Issue Type: Bug
>Affects Versions: 3.3.3
>Reporter: Masatake Iwasaki
>Assignee: Masatake Iwasaki
>Priority: Major
>
> Build on aarch64 failed due to banned classes.
> {{com.github.stephenc.findbugs:findbugs-annotations}} seems to be pulled
> somewhere.
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-18299) Fix unexpected contents in hadoop-client-check-test-invariants on aarch64
[
https://issues.apache.org/jira/browse/HADOOP-18299?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17555735#comment-17555735
]
Masatake Iwasaki commented on HADOOP-18299:
---
{noformat}
[ERROR] Found artifact with unexpected contents:
'/home/ubuntu/srcs/bigtop/output/hadoop/hadoop-3.3.3/hadoop-client-modules/hadoop-client-minicluster/target/hadoop-client-minicluster-3.3.3.jar'
Please check the following and either correct the build or update
the allowed list with reasoning.
edu/
edu/umd/
edu/umd/cs/
edu/umd/cs/findbugs/
edu/umd/cs/findbugs/annotations/
edu/umd/cs/findbugs/annotations/NonNull.class
edu/umd/cs/findbugs/annotations/CheckForNull.class
edu/umd/cs/findbugs/annotations/DefaultAnnotationForFields.class
edu/umd/cs/findbugs/annotations/PossiblyNull.class
edu/umd/cs/findbugs/annotations/OverrideMustInvoke.class
edu/umd/cs/findbugs/annotations/DesireNoWarning.class
edu/umd/cs/findbugs/annotations/DischargesObligation.class
edu/umd/cs/findbugs/annotations/DefaultAnnotationForMethods.class
edu/umd/cs/findbugs/annotations/CreatesObligation.class
edu/umd/cs/findbugs/annotations/DefaultAnnotationForParameters.class
edu/umd/cs/findbugs/annotations/ExpectWarning.class
edu/umd/cs/findbugs/annotations/NoWarning.class
edu/umd/cs/findbugs/annotations/ReturnValuesAreNonnullByDefault.class
edu/umd/cs/findbugs/annotations/CleanupObligation.class
edu/umd/cs/findbugs/annotations/SuppressWarnings.class
edu/umd/cs/findbugs/annotations/CheckReturnValue.class
edu/umd/cs/findbugs/annotations/DesireWarning.class
edu/umd/cs/findbugs/annotations/SuppressFBWarnings.class
edu/umd/cs/findbugs/annotations/When.class
edu/umd/cs/findbugs/annotations/DefaultAnnotation.class
edu/umd/cs/findbugs/annotations/UnknownNullness.class
edu/umd/cs/findbugs/annotations/Priority.class
edu/umd/cs/findbugs/annotations/Confidence.class
edu/umd/cs/findbugs/annotations/Nullable.class
{noformat}
> Fix unexpected contents in hadoop-client-check-test-invariants on aarch64
> -
>
> Key: HADOOP-18299
> URL: https://issues.apache.org/jira/browse/HADOOP-18299
> Project: Hadoop Common
> Issue Type: Bug
>Affects Versions: 3.3.3
>Reporter: Masatake Iwasaki
>Assignee: Masatake Iwasaki
>Priority: Major
>
> Build on aarch64 failed due to banned classes.
> {{com.github.stephenc.findbugs:findbugs-annotations}} seems to be pulled
> somewhere.
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Created] (HADOOP-18299) Fix unexpected contents in hadoop-client-check-test-invariants on aarch64
Masatake Iwasaki created HADOOP-18299:
-
Summary: Fix unexpected contents in
hadoop-client-check-test-invariants on aarch64
Key: HADOOP-18299
URL: https://issues.apache.org/jira/browse/HADOOP-18299
Project: Hadoop Common
Issue Type: Bug
Affects Versions: 3.3.3
Reporter: Masatake Iwasaki
Assignee: Masatake Iwasaki
Build on aarch64 failed due to banned classes.
{{com.github.stephenc.findbugs:findbugs-annotations}} seems to be pulled
somewhere.
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-18276) Rename CHANGES to CHANGELOG in branch-2.10
[ https://issues.apache.org/jira/browse/HADOOP-18276?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Masatake Iwasaki updated HADOOP-18276: -- Fix Version/s: 2.10.3 Hadoop Flags: Reviewed Resolution: Fixed Status: Resolved (was: Patch Available) > Rename CHANGES to CHANGELOG in branch-2.10 > -- > > Key: HADOOP-18276 > URL: https://issues.apache.org/jira/browse/HADOOP-18276 > Project: Hadoop Common > Issue Type: Bug > Components: documentation >Affects Versions: 2.10.2 >Reporter: Masatake Iwasaki >Assignee: Masatake Iwasaki >Priority: Minor > Labels: pull-request-available > Fix For: 2.10.3 > > Time Spent: 1h > Remaining Estimate: 0h > > The file names of changelog generated by releasedocmaker of Yetus was changed > from CHANGES.md to CHANGELOG.md. Hyperlinks in the generated index.html of > "Changelog and Release Notes" page were also affected. The CHANGES*.md > generated in previous releases must be renamed to avoid 404. -- This message was sent by Atlassian Jira (v8.20.7#820007) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-18276) Rename CHANGES to CHANGELOG in branch-2.10
[ https://issues.apache.org/jira/browse/HADOOP-18276?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Masatake Iwasaki updated HADOOP-18276: -- Status: Patch Available (was: Open) > Rename CHANGES to CHANGELOG in branch-2.10 > -- > > Key: HADOOP-18276 > URL: https://issues.apache.org/jira/browse/HADOOP-18276 > Project: Hadoop Common > Issue Type: Bug > Components: documentation >Affects Versions: 2.10.2 >Reporter: Masatake Iwasaki >Assignee: Masatake Iwasaki >Priority: Minor > Labels: pull-request-available > Time Spent: 20m > Remaining Estimate: 0h > > The file names of changelog generated by releasedocmaker of Yetus was changed > from CHANGES.md to CHANGELOG.md. Hyperlinks in the generated index.html of > "Changelog and Release Notes" page were also affected. The CHANGES*.md > generated in previous releases must be renamed to avoid 404. -- This message was sent by Atlassian Jira (v8.20.7#820007) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-18276) Rename CHANGES to CHANGELOG in branch-2.10
[ https://issues.apache.org/jira/browse/HADOOP-18276?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17550135#comment-17550135 ] Masatake Iwasaki commented on HADOOP-18276: --- While the link to CHANGELOG of the latest release is valid: https://hadoop.apache.org/docs/r2.10.2/hadoop-project-dist/hadoop-common/release/2.10.2/CHANGELOG.2.10.2.html hyperlinks to CHANGELOGs of previous releases are not correct: https://hadoop.apache.org/docs/r2.10.2/hadoop-project-dist/hadoop-common/release/2.10.0/CHANGELOG.2.10.0.html > Rename CHANGES to CHANGELOG in branch-2.10 > -- > > Key: HADOOP-18276 > URL: https://issues.apache.org/jira/browse/HADOOP-18276 > Project: Hadoop Common > Issue Type: Bug > Components: documentation >Affects Versions: 2.10.2 >Reporter: Masatake Iwasaki >Assignee: Masatake Iwasaki >Priority: Minor > > The file names of changelog generated by releasedocmaker of Yetus was changed > from CHANGES.md to CHANGELOG.md. Hyperlinks in the generated index.html of > "Changelog and Release Notes" page were also affected. The CHANGES*.md > generated in previous releases must be renamed to avoid 404. -- This message was sent by Atlassian Jira (v8.20.7#820007) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Created] (HADOOP-18276) Rename CHANGES to CHANGELOG in branch-2.10
Masatake Iwasaki created HADOOP-18276: - Summary: Rename CHANGES to CHANGELOG in branch-2.10 Key: HADOOP-18276 URL: https://issues.apache.org/jira/browse/HADOOP-18276 Project: Hadoop Common Issue Type: Bug Components: documentation Affects Versions: 2.10.2 Reporter: Masatake Iwasaki Assignee: Masatake Iwasaki The file names of changelog generated by releasedocmaker of Yetus was changed from CHANGES.md to CHANGELOG.md. Hyperlinks in the generated index.html of "Changelog and Release Notes" page were also affected. The CHANGES*.md generated in previous releases must be renamed to avoid 404. -- This message was sent by Atlassian Jira (v8.20.7#820007) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-18267) Fix failure of build of hadoop-hdfs-bkjournal in branch-2.10 without ptotoc in the PATH
[ https://issues.apache.org/jira/browse/HADOOP-18267?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Masatake Iwasaki updated HADOOP-18267: -- Target Version/s: 2.10.3 Status: Patch Available (was: Open) > Fix failure of build of hadoop-hdfs-bkjournal in branch-2.10 without ptotoc > in the PATH > --- > > Key: HADOOP-18267 > URL: https://issues.apache.org/jira/browse/HADOOP-18267 > Project: Hadoop Common > Issue Type: Sub-task > Components: build >Affects Versions: 2.10.1 >Reporter: Masatake Iwasaki >Assignee: Masatake Iwasaki >Priority: Major > Labels: pull-request-available > Time Spent: 10m > Remaining Estimate: 0h > > We still need protoc in branch-2.10 even after HADOOP-16598. -- This message was sent by Atlassian Jira (v8.20.7#820007) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Assigned] (HADOOP-18267) Fix failure of build of hadoop-hdfs-bkjournal in branch-2.10 without ptotoc in the PATH
[ https://issues.apache.org/jira/browse/HADOOP-18267?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Masatake Iwasaki reassigned HADOOP-18267: - Assignee: Masatake Iwasaki > Fix failure of build of hadoop-hdfs-bkjournal in branch-2.10 without ptotoc > in the PATH > --- > > Key: HADOOP-18267 > URL: https://issues.apache.org/jira/browse/HADOOP-18267 > Project: Hadoop Common > Issue Type: Sub-task > Components: build >Affects Versions: 2.10.1 >Reporter: Masatake Iwasaki >Assignee: Masatake Iwasaki >Priority: Major > > We still need protoc in branch-2.10 even after HADOOP-16598. -- This message was sent by Atlassian Jira (v8.20.7#820007) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-18267) Fix failure of build of hadoop-hdfs-bkjournal in branch-2.10 without ptotoc in the PATH
[
https://issues.apache.org/jira/browse/HADOOP-18267?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17543608#comment-17543608
]
Masatake Iwasaki commented on HADOOP-18267:
---
{noformat}
[INFO] --- maven-antrun-plugin:1.7:run (create-testdirs) @
hadoop-hdfs-bkjournal ---
[INFO] Executing tasks
main:
[mkdir] Created dir:
/ext/srcs/hadoop/hadoop-hdfs-project/hadoop-hdfs/src/contrib/bkjournal/target/test-dir
[INFO] Executed tasks
[INFO]
[INFO] --- hadoop-maven-plugins:2.10.3-SNAPSHOT:protoc (compile-protoc) @
hadoop-hdfs-bkjournal ---
[WARNING] [protoc, --version] failed: java.io.IOException: Cannot run program
"protoc": error=2, No such file or directory
{noformat}
> Fix failure of build of hadoop-hdfs-bkjournal in branch-2.10 without ptotoc
> in the PATH
> ---
>
> Key: HADOOP-18267
> URL: https://issues.apache.org/jira/browse/HADOOP-18267
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: build
>Affects Versions: 2.10.1
>Reporter: Masatake Iwasaki
>Priority: Major
>
> We still need protoc in branch-2.10 even after HADOOP-16598.
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Created] (HADOOP-18267) Fix failure of build of hadoop-hdfs-bkjournal in branch-2.10 without ptotoc in the PATH
Masatake Iwasaki created HADOOP-18267: - Summary: Fix failure of build of hadoop-hdfs-bkjournal in branch-2.10 without ptotoc in the PATH Key: HADOOP-18267 URL: https://issues.apache.org/jira/browse/HADOOP-18267 Project: Hadoop Common Issue Type: Sub-task Components: build Affects Versions: 2.10.1 Reporter: Masatake Iwasaki We still need protoc in branch-2.10 even after HADOOP-16598. -- This message was sent by Atlassian Jira (v8.20.7#820007) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-16664) Backport "HADOOP-16561 [MAPREDUCE] use protobuf-maven-plugin to generate protobuf classes" to all active branches
[ https://issues.apache.org/jira/browse/HADOOP-16664?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Masatake Iwasaki updated HADOOP-16664: -- Fix Version/s: 3.2.4 2.10.3 Hadoop Flags: Reviewed Resolution: Fixed Status: Resolved (was: Patch Available) > Backport "HADOOP-16561 [MAPREDUCE] use protobuf-maven-plugin to generate > protobuf classes" to all active branches > - > > Key: HADOOP-16664 > URL: https://issues.apache.org/jira/browse/HADOOP-16664 > Project: Hadoop Common > Issue Type: Sub-task >Reporter: Duo Zhang >Assignee: Duo Zhang >Priority: Major > Fix For: 3.2.4, 2.10.3 > > Attachments: HADOOP-16664-branch-2.patch, > HADOOP-16664-branch-3.2.patch > > -- This message was sent by Atlassian Jira (v8.20.7#820007) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-16664) Backport "HADOOP-16561 [MAPREDUCE] use protobuf-maven-plugin to generate protobuf classes" to all active branches
[
https://issues.apache.org/jira/browse/HADOOP-16664?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17543601#comment-17543601
]
Masatake Iwasaki commented on HADOOP-16664:
---
The patches are still applicable to branch-3.2 and branch-2.10. {{mvn clean
install -DskipTests -DskipShade}} worked without protoc in the PATH.
+1. Thanks, [~zhangduo].
> Backport "HADOOP-16561 [MAPREDUCE] use protobuf-maven-plugin to generate
> protobuf classes" to all active branches
> -
>
> Key: HADOOP-16664
> URL: https://issues.apache.org/jira/browse/HADOOP-16664
> Project: Hadoop Common
> Issue Type: Sub-task
>Reporter: Duo Zhang
>Assignee: Duo Zhang
>Priority: Major
> Attachments: HADOOP-16664-branch-2.patch,
> HADOOP-16664-branch-3.2.patch
>
>
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-16663) Backport "HADOOP-16560 [YARN] use protobuf-maven-plugin to generate protobuf classes" to all active branches
[ https://issues.apache.org/jira/browse/HADOOP-16663?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Masatake Iwasaki updated HADOOP-16663: -- Fix Version/s: 3.2.4 2.10.3 Hadoop Flags: Reviewed Resolution: Fixed Status: Resolved (was: Patch Available) > Backport "HADOOP-16560 [YARN] use protobuf-maven-plugin to generate protobuf > classes" to all active branches > > > Key: HADOOP-16663 > URL: https://issues.apache.org/jira/browse/HADOOP-16663 > Project: Hadoop Common > Issue Type: Sub-task >Reporter: Duo Zhang >Assignee: Duo Zhang >Priority: Major > Fix For: 3.2.4, 2.10.3 > > Attachments: HADOOP-16663-branch-2.patch, > HADOOP-16663-branch-3.2.patch > > -- This message was sent by Atlassian Jira (v8.20.7#820007) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-16663) Backport "HADOOP-16560 [YARN] use protobuf-maven-plugin to generate protobuf classes" to all active branches
[ https://issues.apache.org/jira/browse/HADOOP-16663?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17543597#comment-17543597 ] Masatake Iwasaki commented on HADOOP-16663: --- The patches are still applicable to branch-3.2 and branch-2.10. +1. Thanks, [~zhangduo]. > Backport "HADOOP-16560 [YARN] use protobuf-maven-plugin to generate protobuf > classes" to all active branches > > > Key: HADOOP-16663 > URL: https://issues.apache.org/jira/browse/HADOOP-16663 > Project: Hadoop Common > Issue Type: Sub-task >Reporter: Duo Zhang >Assignee: Duo Zhang >Priority: Major > Attachments: HADOOP-16663-branch-2.patch, > HADOOP-16663-branch-3.2.patch > > -- This message was sent by Atlassian Jira (v8.20.7#820007) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-16985) Handle release package related issues
[
https://issues.apache.org/jira/browse/HADOOP-16985?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17543588#comment-17543588
]
Masatake Iwasaki commented on HADOOP-16985:
---
cherry-picked to branch-3.2 and branch-2.10.
> Handle release package related issues
> -
>
> Key: HADOOP-16985
> URL: https://issues.apache.org/jira/browse/HADOOP-16985
> Project: Hadoop Common
> Issue Type: Bug
>Reporter: Vinayakumar B
>Assignee: Vinayakumar B
>Priority: Major
> Fix For: 3.3.0, 3.2.4, 2.10.3
>
>
> Same issue as mentioned in HADOOP-16919 is present in hadoop distribution
> generation as well.
> Handle following comments from [~elek] in 1.0.0-RC0 voting mail thread
> here[[https://lists.apache.org/thread.html/r1f2e8325ecef239f0d713c683a16336e2a22431a9f6bfbde3c763816%40%3Ccommon-dev.hadoop.apache.org%3E]]
> {quote}3. Yetus seems to be included in the source package. I am not sure if
> it's intentional but I would remove the patchprocess directory from the
> tar file.
> 7. Minor nit: I would suggest to use only the filename in the sha512
> files (instead of having the /build/source/target prefix). It would help
> to use `sha512 -c` command to validate the checksum.
> {quote}
>
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-16985) Handle release package related issues
[
https://issues.apache.org/jira/browse/HADOOP-16985?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Masatake Iwasaki updated HADOOP-16985:
--
Fix Version/s: 3.2.4
2.10.3
> Handle release package related issues
> -
>
> Key: HADOOP-16985
> URL: https://issues.apache.org/jira/browse/HADOOP-16985
> Project: Hadoop Common
> Issue Type: Bug
>Reporter: Vinayakumar B
>Assignee: Vinayakumar B
>Priority: Major
> Fix For: 3.3.0, 3.2.4, 2.10.3
>
>
> Same issue as mentioned in HADOOP-16919 is present in hadoop distribution
> generation as well.
> Handle following comments from [~elek] in 1.0.0-RC0 voting mail thread
> here[[https://lists.apache.org/thread.html/r1f2e8325ecef239f0d713c683a16336e2a22431a9f6bfbde3c763816%40%3Ccommon-dev.hadoop.apache.org%3E]]
> {quote}3. Yetus seems to be included in the source package. I am not sure if
> it's intentional but I would remove the patchprocess directory from the
> tar file.
> 7. Minor nit: I would suggest to use only the filename in the sha512
> files (instead of having the /build/source/target prefix). It would help
> to use `sha512 -c` command to validate the checksum.
> {quote}
>
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-18251) Fix failure of extracting JIRA id from commit message in git_jira_fix_version_check.py
[
https://issues.apache.org/jira/browse/HADOOP-18251?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Masatake Iwasaki updated HADOOP-18251:
--
Fix Version/s: 3.4.0
3.2.4
3.3.4
2.10.3
Hadoop Flags: Reviewed
Resolution: Fixed
Status: Resolved (was: Patch Available)
> Fix failure of extracting JIRA id from commit message in
> git_jira_fix_version_check.py
> --
>
> Key: HADOOP-18251
> URL: https://issues.apache.org/jira/browse/HADOOP-18251
> Project: Hadoop Common
> Issue Type: Bug
> Components: build
>Reporter: Masatake Iwasaki
>Assignee: Masatake Iwasaki
>Priority: Minor
> Labels: pull-request-available
> Fix For: 3.4.0, 3.2.4, 3.3.4, 2.10.3
>
> Time Spent: 50m
> Remaining Estimate: 0h
>
> git_jira_fix_version_check.py is confused by commit message like
> {{"YARN-1151. Ability to configure auxiliary services from HDFS-based JAR
> files."}} which contains both {{YARN-}} and {{{}HDFS-{}}}. The latter
> {{HDFS-}} is unexpectedly picked as JIRA issue id then 404 is thrown on
> accessing invalid URL like
> "https://issues.apache.org/jira/rest/api/2/issue/HDFS-;.
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-10738) Dynamically adjust distcp configuration by adding distcp-site.xml into code base
[ https://issues.apache.org/jira/browse/HADOOP-10738?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17541344#comment-17541344 ] Masatake Iwasaki commented on HADOOP-10738: --- updated the target version for preparing 2.10.2 release. > Dynamically adjust distcp configuration by adding distcp-site.xml into code > base > > > Key: HADOOP-10738 > URL: https://issues.apache.org/jira/browse/HADOOP-10738 > Project: Hadoop Common > Issue Type: Bug >Affects Versions: 2.4.0 >Reporter: Siqi Li >Priority: Major > Labels: BB2015-05-TBR > Attachments: HADOOP-10738-branch-2.001.patch, HADOOP-10738.v1.patch, > HADOOP-10738.v2.patch > > > For now, the configuration of distcp resides in hadoop-distcp.jar. This makes > it difficult to adjust the configuration dynamically. -- This message was sent by Atlassian Jira (v8.20.7#820007) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-10738) Dynamically adjust distcp configuration by adding distcp-site.xml into code base
[ https://issues.apache.org/jira/browse/HADOOP-10738?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Masatake Iwasaki updated HADOOP-10738: -- Target Version/s: 2.10.3 (was: 2.10.2) > Dynamically adjust distcp configuration by adding distcp-site.xml into code > base > > > Key: HADOOP-10738 > URL: https://issues.apache.org/jira/browse/HADOOP-10738 > Project: Hadoop Common > Issue Type: Bug >Affects Versions: 2.4.0 >Reporter: Siqi Li >Priority: Major > Labels: BB2015-05-TBR > Attachments: HADOOP-10738-branch-2.001.patch, HADOOP-10738.v1.patch, > HADOOP-10738.v2.patch > > > For now, the configuration of distcp resides in hadoop-distcp.jar. This makes > it difficult to adjust the configuration dynamically. -- This message was sent by Atlassian Jira (v8.20.7#820007) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-13091) DistCp masks potential CRC check failures
[
https://issues.apache.org/jira/browse/HADOOP-13091?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Masatake Iwasaki updated HADOOP-13091:
--
Target Version/s: 2.10.3 (was: 2.10.2)
> DistCp masks potential CRC check failures
> -
>
> Key: HADOOP-13091
> URL: https://issues.apache.org/jira/browse/HADOOP-13091
> Project: Hadoop Common
> Issue Type: Bug
>Affects Versions: 2.7.1
>Reporter: Elliot West
>Assignee: Yiqun Lin
>Priority: Major
> Labels: distcp
> Attachments: HADOOP-13091.003.patch, HADOOP-13091.004.patch,
> HDFS-10338.001.patch, HDFS-10338.002.patch
>
>
> There appear to be edge cases whereby CRC checks may be circumvented when
> requests for checksums from the source or target file system fail. In this
> event CRCs could differ between the source and target and yet the DistCp copy
> would succeed, even when the 'skip CRC check' option is not being used.
> The code in question is contained in the method
> [{{org.apache.hadoop.tools.util.DistCpUtils#checksumsAreEqual(...)}}|https://github.com/apache/hadoop/blob/release-2.7.1/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/util/DistCpUtils.java#L457]
> Specifically this code block suggests that if there is a failure when trying
> to read the source or target checksum then the method will return {{true}}
> (i.e. the checksums are equal), implying that the check succeeded. In actual
> fact we just failed to obtain the checksum and could not perform the check.
> {code}
> try {
> sourceChecksum = sourceChecksum != null ? sourceChecksum :
> sourceFS.getFileChecksum(source);
> targetChecksum = targetFS.getFileChecksum(target);
> } catch (IOException e) {
> LOG.error("Unable to retrieve checksum for " + source + " or "
> + target, e);
> }
> return (sourceChecksum == null || targetChecksum == null ||
> sourceChecksum.equals(targetChecksum));
> {code}
> I believe that at the very least the caught {{IOException}} should be
> re-thrown. If this is not deemed desirable then I believe an option
> ({{--strictCrc}}?) should be added to enforce a strict check where we require
> that both the source and target CRCs are retrieved, are not null, and are
> then compared for equality. If for any reason either of the CRCs retrievals
> fail then an exception is thrown.
> Clearly some {{FileSystems}} do not support CRCs and invocations to
> {{FileSystem.getFileChecksum(...)}} return {{null}} in these instances. I
> would suggest that these should fail a strict CRC check to prevent users
> developing a false sense of security in their copy pipeline.
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
