[jira] [Commented] (HADOOP-19164) Hadoop CLI MiniCluster is broken
[
https://issues.apache.org/jira/browse/HADOOP-19164?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17843362#comment-17843362
]
Ayush Saxena commented on HADOOP-19164:
---
Looks pretty much same as HDFS-16050, need to add mockito dependency I believe.
> Hadoop CLI MiniCluster is broken
>
>
> Key: HADOOP-19164
> URL: https://issues.apache.org/jira/browse/HADOOP-19164
> Project: Hadoop Common
> Issue Type: Bug
>Reporter: Ayush Saxena
>Priority: Major
>
> Documentation is also broken & it doesn't work either
> (https://apache.github.io/hadoop/hadoop-project-dist/hadoop-common/CLIMiniCluster.html)
> *Fails with:*
> {noformat}
> Exception in thread "main" java.lang.NoClassDefFoundError:
> org/mockito/stubbing/Answer
> at
> org.apache.hadoop.hdfs.MiniDFSCluster.isNameNodeUp(MiniDFSCluster.java:2666)
> at
> org.apache.hadoop.hdfs.MiniDFSCluster.isClusterUp(MiniDFSCluster.java:2680)
> at
> org.apache.hadoop.hdfs.MiniDFSCluster.waitClusterUp(MiniDFSCluster.java:1510)
> at
> org.apache.hadoop.hdfs.MiniDFSCluster.initMiniDFSCluster(MiniDFSCluster.java:989)
> at org.apache.hadoop.hdfs.MiniDFSCluster.(MiniDFSCluster.java:588)
> at
> org.apache.hadoop.hdfs.MiniDFSCluster$Builder.build(MiniDFSCluster.java:530)
> at
> org.apache.hadoop.mapreduce.MiniHadoopClusterManager.start(MiniHadoopClusterManager.java:160)
> at
> org.apache.hadoop.mapreduce.MiniHadoopClusterManager.run(MiniHadoopClusterManager.java:132)
> at
> org.apache.hadoop.mapreduce.MiniHadoopClusterManager.main(MiniHadoopClusterManager.java:320)
> Caused by: java.lang.ClassNotFoundException: org.mockito.stubbing.Answer
> at java.net.URLClassLoader.findClass(URLClassLoader.java:387)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:419)
> at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:352)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:352)
> ... 9 more{noformat}
> {*}Command executed:{*}
> {noformat}
> bin/mapred minicluster -format{noformat}
> *Documentation Issues:*
> {noformat}
> bin/mapred minicluster -rmport RM_PORT -jhsport JHS_PORT{noformat}
> Without -format option it doesn't work the first time telling Namenode isn't
> formatted, So, this should be corrected.
> {noformat}
> 2024-05-04 00:35:52,933 WARN namenode.FSNamesystem: Encountered exception
> loading fsimage
> java.io.IOException: NameNode is not formatted.
> at
> org.apache.hadoop.hdfs.server.namenode.FSImage.recoverTransitionRead(FSImage.java:253)
> {noformat}
> This isn't required either:
> {noformat}
> NOTE: You will need protoc 2.5.0 installed.
> {noformat}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-19164) Hadoop CLI MiniCluster is broken
[
https://issues.apache.org/jira/browse/HADOOP-19164?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ayush Saxena updated HADOOP-19164:
--
Description:
Documentation is also broken & it doesn't work either
(https://apache.github.io/hadoop/hadoop-project-dist/hadoop-common/CLIMiniCluster.html)
*Fails with:*
{noformat}
Exception in thread "main" java.lang.NoClassDefFoundError:
org/mockito/stubbing/Answer
at
org.apache.hadoop.hdfs.MiniDFSCluster.isNameNodeUp(MiniDFSCluster.java:2666)
at
org.apache.hadoop.hdfs.MiniDFSCluster.isClusterUp(MiniDFSCluster.java:2680)
at
org.apache.hadoop.hdfs.MiniDFSCluster.waitClusterUp(MiniDFSCluster.java:1510)
at
org.apache.hadoop.hdfs.MiniDFSCluster.initMiniDFSCluster(MiniDFSCluster.java:989)
at org.apache.hadoop.hdfs.MiniDFSCluster.(MiniDFSCluster.java:588)
at
org.apache.hadoop.hdfs.MiniDFSCluster$Builder.build(MiniDFSCluster.java:530)
at
org.apache.hadoop.mapreduce.MiniHadoopClusterManager.start(MiniHadoopClusterManager.java:160)
at
org.apache.hadoop.mapreduce.MiniHadoopClusterManager.run(MiniHadoopClusterManager.java:132)
at
org.apache.hadoop.mapreduce.MiniHadoopClusterManager.main(MiniHadoopClusterManager.java:320)
Caused by: java.lang.ClassNotFoundException: org.mockito.stubbing.Answer
at java.net.URLClassLoader.findClass(URLClassLoader.java:387)
at java.lang.ClassLoader.loadClass(ClassLoader.java:419)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:352)
at java.lang.ClassLoader.loadClass(ClassLoader.java:352)
... 9 more{noformat}
{*}Command executed:{*}
{noformat}
bin/mapred minicluster -format{noformat}
*Documentation Issues:*
{noformat}
bin/mapred minicluster -rmport RM_PORT -jhsport JHS_PORT{noformat}
Without -format option it doesn't work the first time telling Namenode isn't
formatted, So, this should be corrected.
{noformat}
2024-05-04 00:35:52,933 WARN namenode.FSNamesystem: Encountered exception
loading fsimage
java.io.IOException: NameNode is not formatted.
at
org.apache.hadoop.hdfs.server.namenode.FSImage.recoverTransitionRead(FSImage.java:253)
{noformat}
This isn't required either:
{noformat}
NOTE: You will need protoc 2.5.0 installed.
{noformat}
was:
Documentation is also broken & it doesn't work either
(https://apache.github.io/hadoop/hadoop-project-dist/hadoop-common/CLIMiniCluster.html)
*Fails with:*
{noformat}
Exception in thread "main" java.lang.NoClassDefFoundError:
org/mockito/stubbing/Answer
at
org.apache.hadoop.hdfs.MiniDFSCluster.isNameNodeUp(MiniDFSCluster.java:2666)
at
org.apache.hadoop.hdfs.MiniDFSCluster.isClusterUp(MiniDFSCluster.java:2680)
at
org.apache.hadoop.hdfs.MiniDFSCluster.waitClusterUp(MiniDFSCluster.java:1510)
at
org.apache.hadoop.hdfs.MiniDFSCluster.initMiniDFSCluster(MiniDFSCluster.java:989)
at org.apache.hadoop.hdfs.MiniDFSCluster.(MiniDFSCluster.java:588)
at
org.apache.hadoop.hdfs.MiniDFSCluster$Builder.build(MiniDFSCluster.java:530)
at
org.apache.hadoop.mapreduce.MiniHadoopClusterManager.start(MiniHadoopClusterManager.java:160)
at
org.apache.hadoop.mapreduce.MiniHadoopClusterManager.run(MiniHadoopClusterManager.java:132)
at
org.apache.hadoop.mapreduce.MiniHadoopClusterManager.main(MiniHadoopClusterManager.java:320)
Caused by: java.lang.ClassNotFoundException: org.mockito.stubbing.Answer
at java.net.URLClassLoader.findClass(URLClassLoader.java:387)
at java.lang.ClassLoader.loadClass(ClassLoader.java:419)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:352)
at java.lang.ClassLoader.loadClass(ClassLoader.java:352)
... 9 more{noformat}
{*}Command executed:{*}
{noformat}
bin/mapred minicluster -format{noformat}
*Documentation Issues:*
{noformat}
bin/mapred minicluster -rmport RM_PORT -jhsport JHS_PORT{noformat}
Without -format option it doesn't work the first time telling Namenode isn't
formatted, So, this should be corrected.
{noformat}
2024-05-04 00:35:52,933 WARN namenode.FSNamesystem: Encountered exception
loading fsimage
java.io.IOException: NameNode is not formatted.
at
org.apache.hadoop.hdfs.server.namenode.FSImage.recoverTransitionRead(FSImage.java:253)
{noformat}
> Hadoop CLI MiniCluster is broken
>
>
> Key: HADOOP-19164
> URL: https://issues.apache.org/jira/browse/HADOOP-19164
> Project: Hadoop Common
> Issue Type: Bug
>Reporter: Ayush Saxena
>Priority: Major
>
> Documentation is also broken & it doesn't work either
> (https://apache.github.io/hadoop/hadoop-project-dist/hadoop-common/CLIMiniCluster.html)
> *Fails with:*
> {noformat}
> Exception in thread "main" java.lang.NoClassDefFoundError:
> org/mockito/stubbing/Answer
> at
>
[jira] [Created] (HADOOP-19164) Hadoop CLI MiniCluster is broken
Ayush Saxena created HADOOP-19164:
-
Summary: Hadoop CLI MiniCluster is broken
Key: HADOOP-19164
URL: https://issues.apache.org/jira/browse/HADOOP-19164
Project: Hadoop Common
Issue Type: Bug
Reporter: Ayush Saxena
Documentation is also broken & it doesn't work either
(https://apache.github.io/hadoop/hadoop-project-dist/hadoop-common/CLIMiniCluster.html)
*Fails with:*
{noformat}
Exception in thread "main" java.lang.NoClassDefFoundError:
org/mockito/stubbing/Answer
at
org.apache.hadoop.hdfs.MiniDFSCluster.isNameNodeUp(MiniDFSCluster.java:2666)
at
org.apache.hadoop.hdfs.MiniDFSCluster.isClusterUp(MiniDFSCluster.java:2680)
at
org.apache.hadoop.hdfs.MiniDFSCluster.waitClusterUp(MiniDFSCluster.java:1510)
at
org.apache.hadoop.hdfs.MiniDFSCluster.initMiniDFSCluster(MiniDFSCluster.java:989)
at org.apache.hadoop.hdfs.MiniDFSCluster.(MiniDFSCluster.java:588)
at
org.apache.hadoop.hdfs.MiniDFSCluster$Builder.build(MiniDFSCluster.java:530)
at
org.apache.hadoop.mapreduce.MiniHadoopClusterManager.start(MiniHadoopClusterManager.java:160)
at
org.apache.hadoop.mapreduce.MiniHadoopClusterManager.run(MiniHadoopClusterManager.java:132)
at
org.apache.hadoop.mapreduce.MiniHadoopClusterManager.main(MiniHadoopClusterManager.java:320)
Caused by: java.lang.ClassNotFoundException: org.mockito.stubbing.Answer
at java.net.URLClassLoader.findClass(URLClassLoader.java:387)
at java.lang.ClassLoader.loadClass(ClassLoader.java:419)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:352)
at java.lang.ClassLoader.loadClass(ClassLoader.java:352)
... 9 more{noformat}
{*}Command executed:{*}
{noformat}
bin/mapred minicluster -format{noformat}
*Documentation Issues:*
{noformat}
bin/mapred minicluster -rmport RM_PORT -jhsport JHS_PORT{noformat}
Without -format option it doesn't work the first time telling Namenode isn't
formatted, So, this should be corrected.
{noformat}
2024-05-04 00:35:52,933 WARN namenode.FSNamesystem: Encountered exception
loading fsimage
java.io.IOException: NameNode is not formatted.
at
org.apache.hadoop.hdfs.server.namenode.FSImage.recoverTransitionRead(FSImage.java:253)
{noformat}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-19107) Drop support for HBase v1 & upgrade HBase v2
[
https://issues.apache.org/jira/browse/HADOOP-19107?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17843349#comment-17843349
]
Ayush Saxena commented on HADOOP-19107:
---
{quote}add in release notes?
{quote}
Done.
{quote}backport to 3.4.1?
{quote}
It isn't just a version upgrade, but removal of HBase v1 & changing the default
Hbase jar to HBase v2, not sure if compat allows that.
{quote}does this mean we can strip out parquet 2.5 from our redistributed
artifacts?
{quote}
I think you mean Protobuf, HBase still defines 2.5.0 in their pom as a compile
time dependency, but don't use it internally. We can give it a shot by
excluding it explicitly, they might have kept that for some of their downstream
consumers (maybe), I am pretty sure not just for compat sake of transitive
dependency. little bit risky, if it creates runtime issues but I can create a
ticket and experiment a bit if you say
[https://github.com/apache/hbase/blob/rel/2.5.8/pom.xml#L603]
> Drop support for HBase v1 & upgrade HBase v2
>
>
> Key: HADOOP-19107
> URL: https://issues.apache.org/jira/browse/HADOOP-19107
> Project: Hadoop Common
> Issue Type: Task
>Reporter: Ayush Saxena
>Assignee: Ayush Saxena
>Priority: Major
> Labels: pull-request-available
> Fix For: 3.5.0
>
>
> Drop support for Hbase V1 and make building Hbase v2 default.
> Dev List:
> [https://lists.apache.org/thread/vb2gh5ljwncbrmqnk0oflb8ftdz64hhs]
> https://lists.apache.org/thread/o88hnm7q8n3b4bng81q14vsj3fbhfx5w
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-19107) Drop support for HBase v1 & upgrade HBase v2
[ https://issues.apache.org/jira/browse/HADOOP-19107?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ayush Saxena updated HADOOP-19107: -- Flags: Important Release Note: Drops supports for HBase 1.x release line, The supported version HBase version is 2.5.8 > Drop support for HBase v1 & upgrade HBase v2 > > > Key: HADOOP-19107 > URL: https://issues.apache.org/jira/browse/HADOOP-19107 > Project: Hadoop Common > Issue Type: Task >Reporter: Ayush Saxena >Assignee: Ayush Saxena >Priority: Major > Labels: pull-request-available > Fix For: 3.5.0 > > > Drop support for Hbase V1 and make building Hbase v2 default. > Dev List: > [https://lists.apache.org/thread/vb2gh5ljwncbrmqnk0oflb8ftdz64hhs] > https://lists.apache.org/thread/o88hnm7q8n3b4bng81q14vsj3fbhfx5w -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-19107) Drop support for HBase v1 & upgrade HBase v2
[ https://issues.apache.org/jira/browse/HADOOP-19107?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ayush Saxena updated HADOOP-19107: -- Summary: Drop support for HBase v1 & upgrade HBase v2 (was: Drop support for HBase v1) > Drop support for HBase v1 & upgrade HBase v2 > > > Key: HADOOP-19107 > URL: https://issues.apache.org/jira/browse/HADOOP-19107 > Project: Hadoop Common > Issue Type: Task >Reporter: Ayush Saxena >Assignee: Ayush Saxena >Priority: Major > Labels: pull-request-available > Fix For: 3.5.0 > > > Drop support for Hbase V1 and make building Hbase v2 default. > Dev List: > [https://lists.apache.org/thread/vb2gh5ljwncbrmqnk0oflb8ftdz64hhs] > https://lists.apache.org/thread/o88hnm7q8n3b4bng81q14vsj3fbhfx5w -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Resolved] (HADOOP-19107) Drop support for HBase v1
[ https://issues.apache.org/jira/browse/HADOOP-19107?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ayush Saxena resolved HADOOP-19107. --- Fix Version/s: 3.5.0 Hadoop Flags: Reviewed Resolution: Fixed > Drop support for HBase v1 > - > > Key: HADOOP-19107 > URL: https://issues.apache.org/jira/browse/HADOOP-19107 > Project: Hadoop Common > Issue Type: Task >Reporter: Ayush Saxena >Assignee: Ayush Saxena >Priority: Major > Labels: pull-request-available > Fix For: 3.5.0 > > > Drop support for Hbase V1 and make building Hbase v2 default. > Dev List: > [https://lists.apache.org/thread/vb2gh5ljwncbrmqnk0oflb8ftdz64hhs] > https://lists.apache.org/thread/o88hnm7q8n3b4bng81q14vsj3fbhfx5w -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-19107) Drop support for HBase v1
[ https://issues.apache.org/jira/browse/HADOOP-19107?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17840374#comment-17840374 ] Ayush Saxena commented on HADOOP-19107: --- Committed to trunk. Thanx [~vinayakumarb] & [~ste...@apache.org] for the reviews!!! > Drop support for HBase v1 > - > > Key: HADOOP-19107 > URL: https://issues.apache.org/jira/browse/HADOOP-19107 > Project: Hadoop Common > Issue Type: Task >Reporter: Ayush Saxena >Assignee: Ayush Saxena >Priority: Major > Labels: pull-request-available > > Drop support for Hbase V1 and make building Hbase v2 default. > Dev List: > [https://lists.apache.org/thread/vb2gh5ljwncbrmqnk0oflb8ftdz64hhs] > https://lists.apache.org/thread/o88hnm7q8n3b4bng81q14vsj3fbhfx5w -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Assigned] (HADOOP-19107) Drop support for HBase v1
[ https://issues.apache.org/jira/browse/HADOOP-19107?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ayush Saxena reassigned HADOOP-19107: - Assignee: Ayush Saxena > Drop support for HBase v1 > - > > Key: HADOOP-19107 > URL: https://issues.apache.org/jira/browse/HADOOP-19107 > Project: Hadoop Common > Issue Type: Task >Reporter: Ayush Saxena >Assignee: Ayush Saxena >Priority: Major > Labels: pull-request-available > > Drop support for Hbase V1 and make building Hbase v2 default. > Dev List: > [https://lists.apache.org/thread/vb2gh5ljwncbrmqnk0oflb8ftdz64hhs] > https://lists.apache.org/thread/o88hnm7q8n3b4bng81q14vsj3fbhfx5w -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Assigned] (HADOOP-19130) FTPFileSystem rename with full qualified path broken
[
https://issues.apache.org/jira/browse/HADOOP-19130?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ayush Saxena reassigned HADOOP-19130:
-
Assignee: shawn
> FTPFileSystem rename with full qualified path broken
>
>
> Key: HADOOP-19130
> URL: https://issues.apache.org/jira/browse/HADOOP-19130
> Project: Hadoop Common
> Issue Type: Bug
> Components: fs
>Affects Versions: 0.20.2, 3.3.3, 3.3.4, 3.3.6
>Reporter: shawn
>Assignee: shawn
>Priority: Major
> Labels: pull-request-available
> Attachments: image-2024-03-27-09-59-12-381.png,
> image-2024-03-28-09-58-19-721.png
>
> Original Estimate: 2h
> Remaining Estimate: 2h
>
> When use fs shell to put/rename file in ftp server with full qualified
> path , it always get "Input/output error"(eg.
> [ftp://user:password@localhost/pathxxx]), the reason is that
> changeWorkingDirectory command underneath is being passed a string with
> [file://|file:///] uri prefix which will not be understand by ftp server
> !image-2024-03-27-09-59-12-381.png|width=948,height=156!
>
> in our case, after
> client.changeWorkingDirectory("ftp://mytest:myt...@10.5.xx.xx/files;)
> executed, the workingDirectory of ftp server is still "/", which is
> incorrect(not understand by ftp server)
> !image-2024-03-28-09-58-19-721.png|width=745,height=431!
> the solution should be pass
> absoluteSrc.getParent().toUri().getPath().toString to avoid
> [file://|file:///] uri prefix, like this:
> {code:java}
> --- a/FTPFileSystem.java
> +++ b/FTPFileSystem.java
> @@ -549,15 +549,15 @@ public class FTPFileSystem extends FileSystem {
> throw new IOException("Destination path " + dst
> + " already exist, cannot rename!");
> }
> - String parentSrc = absoluteSrc.getParent().toUri().toString();
> - String parentDst = absoluteDst.getParent().toUri().toString();
> + URI parentSrc = absoluteSrc.getParent().toUri();
> + URI parentDst = absoluteDst.getParent().toUri();
> String from = src.getName();
> String to = dst.getName();
> - if (!parentSrc.equals(parentDst)) {
> + if (!parentSrc.toString().equals(parentDst.toString())) {
> throw new IOException("Cannot rename parent(source): " + parentSrc
> + ", parent(destination): " + parentDst);
> }
> - client.changeWorkingDirectory(parentSrc);
> + client.changeWorkingDirectory(parentSrc.getPath().toString());
> boolean renamed = client.rename(from, to);
> return renamed;
> }{code}
> already related issue as follows
> https://issues.apache.org/jira/browse/HADOOP-8653
> I create this issue and add related unit test.
>
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-19130) FTPFileSystem rename with full qualified path broken
[
https://issues.apache.org/jira/browse/HADOOP-19130?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17838327#comment-17838327
]
Ayush Saxena commented on HADOOP-19130:
---
Committed to trunk.
Thanx [~zhangjian16] for the contribution & [~hiwangzhihui] for the review!!!
Note: Added [~zhangjian16] as Hadoop Common Contributor to assign the ticket.
Welcome to Hadoop!!!
> FTPFileSystem rename with full qualified path broken
>
>
> Key: HADOOP-19130
> URL: https://issues.apache.org/jira/browse/HADOOP-19130
> Project: Hadoop Common
> Issue Type: Bug
> Components: fs
>Affects Versions: 0.20.2, 3.3.3, 3.3.4, 3.3.6
>Reporter: shawn
>Assignee: shawn
>Priority: Major
> Labels: pull-request-available
> Attachments: image-2024-03-27-09-59-12-381.png,
> image-2024-03-28-09-58-19-721.png
>
> Original Estimate: 2h
> Remaining Estimate: 2h
>
> When use fs shell to put/rename file in ftp server with full qualified
> path , it always get "Input/output error"(eg.
> [ftp://user:password@localhost/pathxxx]), the reason is that
> changeWorkingDirectory command underneath is being passed a string with
> [file://|file:///] uri prefix which will not be understand by ftp server
> !image-2024-03-27-09-59-12-381.png|width=948,height=156!
>
> in our case, after
> client.changeWorkingDirectory("ftp://mytest:myt...@10.5.xx.xx/files;)
> executed, the workingDirectory of ftp server is still "/", which is
> incorrect(not understand by ftp server)
> !image-2024-03-28-09-58-19-721.png|width=745,height=431!
> the solution should be pass
> absoluteSrc.getParent().toUri().getPath().toString to avoid
> [file://|file:///] uri prefix, like this:
> {code:java}
> --- a/FTPFileSystem.java
> +++ b/FTPFileSystem.java
> @@ -549,15 +549,15 @@ public class FTPFileSystem extends FileSystem {
> throw new IOException("Destination path " + dst
> + " already exist, cannot rename!");
> }
> - String parentSrc = absoluteSrc.getParent().toUri().toString();
> - String parentDst = absoluteDst.getParent().toUri().toString();
> + URI parentSrc = absoluteSrc.getParent().toUri();
> + URI parentDst = absoluteDst.getParent().toUri();
> String from = src.getName();
> String to = dst.getName();
> - if (!parentSrc.equals(parentDst)) {
> + if (!parentSrc.toString().equals(parentDst.toString())) {
> throw new IOException("Cannot rename parent(source): " + parentSrc
> + ", parent(destination): " + parentDst);
> }
> - client.changeWorkingDirectory(parentSrc);
> + client.changeWorkingDirectory(parentSrc.getPath().toString());
> boolean renamed = client.rename(from, to);
> return renamed;
> }{code}
> already related issue as follows
> https://issues.apache.org/jira/browse/HADOOP-8653
> I create this issue and add related unit test.
>
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Resolved] (HADOOP-19130) FTPFileSystem rename with full qualified path broken
[
https://issues.apache.org/jira/browse/HADOOP-19130?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ayush Saxena resolved HADOOP-19130.
---
Fix Version/s: 3.5.0
Hadoop Flags: Reviewed
Resolution: Fixed
> FTPFileSystem rename with full qualified path broken
>
>
> Key: HADOOP-19130
> URL: https://issues.apache.org/jira/browse/HADOOP-19130
> Project: Hadoop Common
> Issue Type: Bug
> Components: fs
>Affects Versions: 0.20.2, 3.3.3, 3.3.4, 3.3.6
>Reporter: shawn
>Assignee: shawn
>Priority: Major
> Labels: pull-request-available
> Fix For: 3.5.0
>
> Attachments: image-2024-03-27-09-59-12-381.png,
> image-2024-03-28-09-58-19-721.png
>
> Original Estimate: 2h
> Remaining Estimate: 2h
>
> When use fs shell to put/rename file in ftp server with full qualified
> path , it always get "Input/output error"(eg.
> [ftp://user:password@localhost/pathxxx]), the reason is that
> changeWorkingDirectory command underneath is being passed a string with
> [file://|file:///] uri prefix which will not be understand by ftp server
> !image-2024-03-27-09-59-12-381.png|width=948,height=156!
>
> in our case, after
> client.changeWorkingDirectory("ftp://mytest:myt...@10.5.xx.xx/files;)
> executed, the workingDirectory of ftp server is still "/", which is
> incorrect(not understand by ftp server)
> !image-2024-03-28-09-58-19-721.png|width=745,height=431!
> the solution should be pass
> absoluteSrc.getParent().toUri().getPath().toString to avoid
> [file://|file:///] uri prefix, like this:
> {code:java}
> --- a/FTPFileSystem.java
> +++ b/FTPFileSystem.java
> @@ -549,15 +549,15 @@ public class FTPFileSystem extends FileSystem {
> throw new IOException("Destination path " + dst
> + " already exist, cannot rename!");
> }
> - String parentSrc = absoluteSrc.getParent().toUri().toString();
> - String parentDst = absoluteDst.getParent().toUri().toString();
> + URI parentSrc = absoluteSrc.getParent().toUri();
> + URI parentDst = absoluteDst.getParent().toUri();
> String from = src.getName();
> String to = dst.getName();
> - if (!parentSrc.equals(parentDst)) {
> + if (!parentSrc.toString().equals(parentDst.toString())) {
> throw new IOException("Cannot rename parent(source): " + parentSrc
> + ", parent(destination): " + parentDst);
> }
> - client.changeWorkingDirectory(parentSrc);
> + client.changeWorkingDirectory(parentSrc.getPath().toString());
> boolean renamed = client.rename(from, to);
> return renamed;
> }{code}
> already related issue as follows
> https://issues.apache.org/jira/browse/HADOOP-8653
> I create this issue and add related unit test.
>
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Resolved] (HADOOP-19123) Update commons-configuration2 to 2.10.1 due to CVE
[ https://issues.apache.org/jira/browse/HADOOP-19123?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ayush Saxena resolved HADOOP-19123. --- Fix Version/s: 3.5.0 3.4.1 Hadoop Flags: Reviewed Resolution: Fixed > Update commons-configuration2 to 2.10.1 due to CVE > -- > > Key: HADOOP-19123 > URL: https://issues.apache.org/jira/browse/HADOOP-19123 > Project: Hadoop Common > Issue Type: Task >Reporter: PJ Fanning >Assignee: PJ Fanning >Priority: Major > Labels: pull-request-available > Fix For: 3.5.0, 3.4.1 > > > https://github.com/advisories/GHSA-9w38-p64v-xpmv -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-19123) Update commons-configuration2 to 2.10.1 due to CVE
[ https://issues.apache.org/jira/browse/HADOOP-19123?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ayush Saxena updated HADOOP-19123: -- Summary: Update commons-configuration2 to 2.10.1 due to CVE (was: update commons-configuration2 to 2.10.1 due to CVE) > Update commons-configuration2 to 2.10.1 due to CVE > -- > > Key: HADOOP-19123 > URL: https://issues.apache.org/jira/browse/HADOOP-19123 > Project: Hadoop Common > Issue Type: Task >Reporter: PJ Fanning >Assignee: PJ Fanning >Priority: Major > Labels: pull-request-available > > https://github.com/advisories/GHSA-9w38-p64v-xpmv -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-19077) Remove use of javax.ws.rs.core.HttpHeaders
[ https://issues.apache.org/jira/browse/HADOOP-19077?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ayush Saxena updated HADOOP-19077: -- Fix Version/s: 3.5.0 (was: 3.4.1) > Remove use of javax.ws.rs.core.HttpHeaders > -- > > Key: HADOOP-19077 > URL: https://issues.apache.org/jira/browse/HADOOP-19077 > Project: Hadoop Common > Issue Type: Task > Components: io >Reporter: PJ Fanning >Assignee: PJ Fanning >Priority: Major > Labels: pull-request-available > Fix For: 3.5.0 > > > One step towards removing Hadoop's dependence on Jersey1 and jsr311-api. > We have other classes where we can get HTTP header names. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Resolved] (HADOOP-19077) Remove use of javax.ws.rs.core.HttpHeaders
[ https://issues.apache.org/jira/browse/HADOOP-19077?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ayush Saxena resolved HADOOP-19077. --- Fix Version/s: 3.4.1 Hadoop Flags: Reviewed Resolution: Fixed > Remove use of javax.ws.rs.core.HttpHeaders > -- > > Key: HADOOP-19077 > URL: https://issues.apache.org/jira/browse/HADOOP-19077 > Project: Hadoop Common > Issue Type: Task > Components: io >Reporter: PJ Fanning >Assignee: PJ Fanning >Priority: Major > Labels: pull-request-available > Fix For: 3.4.1 > > > One step towards removing Hadoop's dependence on Jersey1 and jsr311-api. > We have other classes where we can get HTTP header names. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-19077) Remove use of javax.ws.rs.core.HttpHeaders
[ https://issues.apache.org/jira/browse/HADOOP-19077?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17832757#comment-17832757 ] Ayush Saxena commented on HADOOP-19077: --- Committed to trunk. Thanx [~pj.fanning] for the contribution!!! > Remove use of javax.ws.rs.core.HttpHeaders > -- > > Key: HADOOP-19077 > URL: https://issues.apache.org/jira/browse/HADOOP-19077 > Project: Hadoop Common > Issue Type: Task > Components: io >Reporter: PJ Fanning >Assignee: PJ Fanning >Priority: Major > Labels: pull-request-available > > One step towards removing Hadoop's dependence on Jersey1 and jsr311-api. > We have other classes where we can get HTTP header names. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-19077) Remove use of javax.ws.rs.core.HttpHeaders
[ https://issues.apache.org/jira/browse/HADOOP-19077?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ayush Saxena updated HADOOP-19077: -- Summary: Remove use of javax.ws.rs.core.HttpHeaders (was: remove use of javax.ws.rs.core.HttpHeaders) > Remove use of javax.ws.rs.core.HttpHeaders > -- > > Key: HADOOP-19077 > URL: https://issues.apache.org/jira/browse/HADOOP-19077 > Project: Hadoop Common > Issue Type: Task > Components: io >Reporter: PJ Fanning >Assignee: PJ Fanning >Priority: Major > Labels: pull-request-available > > One step towards removing Hadoop's dependence on Jersey1 and jsr311-api. > We have other classes where we can get HTTP header names. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Assigned] (HADOOP-19077) remove use of javax.ws.rs.core.HttpHeaders
[ https://issues.apache.org/jira/browse/HADOOP-19077?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ayush Saxena reassigned HADOOP-19077: - Assignee: PJ Fanning > remove use of javax.ws.rs.core.HttpHeaders > -- > > Key: HADOOP-19077 > URL: https://issues.apache.org/jira/browse/HADOOP-19077 > Project: Hadoop Common > Issue Type: Task > Components: io >Reporter: PJ Fanning >Assignee: PJ Fanning >Priority: Major > Labels: pull-request-available > > One step towards removing Hadoop's dependence on Jersey1 and jsr311-api. > We have other classes where we can get HTTP header names. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-19024) Use bouncycastle jdk18 1.77
[ https://issues.apache.org/jira/browse/HADOOP-19024?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ayush Saxena updated HADOOP-19024: -- Fix Version/s: 3.4.1 > Use bouncycastle jdk18 1.77 > --- > > Key: HADOOP-19024 > URL: https://issues.apache.org/jira/browse/HADOOP-19024 > Project: Hadoop Common > Issue Type: Task >Reporter: PJ Fanning >Assignee: PJ Fanning >Priority: Major > Labels: pull-request-available > Fix For: 3.5.0, 3.4.1 > > > They have stopped patching the JDK 1.5 jars that Hadoop uses (see > https://issues.apache.org/jira/browse/HADOOP-18540). > The new artifacts have similar names - but the names are like bcprov-jdk18on > as opposed to bcprov-jdk15on. > CVE-2023-33201 is an example of a security issue that seems only to be fixed > in the JDK 1.8 artifacts (ie no JDK 1.5 jar has the fix). > https://www.bouncycastle.org/releasenotes.html#r1rv77 latest current release > but the CVE was fixed in 1.74. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Resolved] (HADOOP-19024) Use bouncycastle jdk18 1.77
[ https://issues.apache.org/jira/browse/HADOOP-19024?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ayush Saxena resolved HADOOP-19024. --- Fix Version/s: 3.5.0 Hadoop Flags: Reviewed Resolution: Fixed > Use bouncycastle jdk18 1.77 > --- > > Key: HADOOP-19024 > URL: https://issues.apache.org/jira/browse/HADOOP-19024 > Project: Hadoop Common > Issue Type: Task >Reporter: PJ Fanning >Assignee: PJ Fanning >Priority: Major > Labels: pull-request-available > Fix For: 3.5.0 > > > They have stopped patching the JDK 1.5 jars that Hadoop uses (see > https://issues.apache.org/jira/browse/HADOOP-18540). > The new artifacts have similar names - but the names are like bcprov-jdk18on > as opposed to bcprov-jdk15on. > CVE-2023-33201 is an example of a security issue that seems only to be fixed > in the JDK 1.8 artifacts (ie no JDK 1.5 jar has the fix). > https://www.bouncycastle.org/releasenotes.html#r1rv77 latest current release > but the CVE was fixed in 1.74. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-19024) Use bouncycastle jdk18 1.77
[ https://issues.apache.org/jira/browse/HADOOP-19024?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17832453#comment-17832453 ] Ayush Saxena commented on HADOOP-19024: --- Committed to trunk & branch-3.4 Thanx [~pj.fanning] for the contribution!!! > Use bouncycastle jdk18 1.77 > --- > > Key: HADOOP-19024 > URL: https://issues.apache.org/jira/browse/HADOOP-19024 > Project: Hadoop Common > Issue Type: Task >Reporter: PJ Fanning >Assignee: PJ Fanning >Priority: Major > Labels: pull-request-available > > They have stopped patching the JDK 1.5 jars that Hadoop uses (see > https://issues.apache.org/jira/browse/HADOOP-18540). > The new artifacts have similar names - but the names are like bcprov-jdk18on > as opposed to bcprov-jdk15on. > CVE-2023-33201 is an example of a security issue that seems only to be fixed > in the JDK 1.8 artifacts (ie no JDK 1.5 jar has the fix). > https://www.bouncycastle.org/releasenotes.html#r1rv77 latest current release > but the CVE was fixed in 1.74. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Assigned] (HADOOP-19024) Use bouncycastle jdk18 1.77
[ https://issues.apache.org/jira/browse/HADOOP-19024?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ayush Saxena reassigned HADOOP-19024: - Assignee: PJ Fanning > Use bouncycastle jdk18 1.77 > --- > > Key: HADOOP-19024 > URL: https://issues.apache.org/jira/browse/HADOOP-19024 > Project: Hadoop Common > Issue Type: Task >Reporter: PJ Fanning >Assignee: PJ Fanning >Priority: Major > Labels: pull-request-available > > They have stopped patching the JDK 1.5 jars that Hadoop uses (see > https://issues.apache.org/jira/browse/HADOOP-18540). > The new artifacts have similar names - but the names are like bcprov-jdk18on > as opposed to bcprov-jdk15on. > CVE-2023-33201 is an example of a security issue that seems only to be fixed > in the JDK 1.8 artifacts (ie no JDK 1.5 jar has the fix). > https://www.bouncycastle.org/releasenotes.html#r1rv77 latest current release > but the CVE was fixed in 1.74. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-19024) Use bouncycastle jdk18 1.77
[ https://issues.apache.org/jira/browse/HADOOP-19024?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ayush Saxena updated HADOOP-19024: -- Summary: Use bouncycastle jdk18 1.77 (was: change to bouncy castle jdk1.8 jars) > Use bouncycastle jdk18 1.77 > --- > > Key: HADOOP-19024 > URL: https://issues.apache.org/jira/browse/HADOOP-19024 > Project: Hadoop Common > Issue Type: Task >Reporter: PJ Fanning >Priority: Major > Labels: pull-request-available > > They have stopped patching the JDK 1.5 jars that Hadoop uses (see > https://issues.apache.org/jira/browse/HADOOP-18540). > The new artifacts have similar names - but the names are like bcprov-jdk18on > as opposed to bcprov-jdk15on. > CVE-2023-33201 is an example of a security issue that seems only to be fixed > in the JDK 1.8 artifacts (ie no JDK 1.5 jar has the fix). > https://www.bouncycastle.org/releasenotes.html#r1rv77 latest current release > but the CVE was fixed in 1.74. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-19107) Drop support for HBase v1
[ https://issues.apache.org/jira/browse/HADOOP-19107?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17828232#comment-17828232 ] Ayush Saxena commented on HADOOP-19107: --- I tried removing HBase v1 & upgrading v2 as part of: https://github.com/apache/hadoop/pull/6629 It shows some spotbugs warnings: HADOOP-19100 Had to go near RBF in this, because my build started failing without that: HDFS-17370 Rest, there were lot of HBase compatible. declared and used, I removed/cleaned all of that. I don't think that is required HBase shades most of them & they weren't creating any issue. The test failures in the build: I don't think they are related, I had 2 consecutive run, both had different tests failing, with one with "Unable to create native thread", shouldn't be me. I will check on the related tickets to see, If I can get a green build. > Drop support for HBase v1 > - > > Key: HADOOP-19107 > URL: https://issues.apache.org/jira/browse/HADOOP-19107 > Project: Hadoop Common > Issue Type: Task >Reporter: Ayush Saxena >Priority: Major > > Drop support for Hbase V1 and make building Hbase v2 default. > Dev List: > [https://lists.apache.org/thread/vb2gh5ljwncbrmqnk0oflb8ftdz64hhs] > https://lists.apache.org/thread/o88hnm7q8n3b4bng81q14vsj3fbhfx5w -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-19100) Fix Spotbugs warnings in the build
[ https://issues.apache.org/jira/browse/HADOOP-19100?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ayush Saxena updated HADOOP-19100: -- Description: We are getting spotbugs warnings in every PR. [https://ci-hadoop.apache.org/job/hadoop-qbt-trunk-java8-linux-x86_64/1532/artifact/out/branch-spotbugs-hadoop-common-project_hadoop-common-warnings.html] [https://ci-hadoop.apache.org/job/hadoop-qbt-trunk-java8-linux-x86_64/1532/artifact/out/branch-spotbugs-hadoop-common-project-warnings.html] [https://ci-hadoop.apache.org/job/hadoop-qbt-trunk-java8-linux-x86_64/1532/artifact/out/branch-spotbugs-hadoop-hdfs-project_hadoop-hdfs-client-warnings.html] [https://ci-hadoop.apache.org/job/hadoop-qbt-trunk-java8-linux-x86_64/1532/artifact/out/branch-spotbugs-hadoop-hdfs-project_hadoop-hdfs-httpfs-warnings.html] [https://ci-hadoop.apache.org/job/hadoop-qbt-trunk-java8-linux-x86_64/1532/artifact/out/branch-spotbugs-hadoop-yarn-project_hadoop-yarn-warnings.html] [https://ci-hadoop.apache.org/job/hadoop-qbt-trunk-java8-linux-x86_64/1532/artifact/out/branch-spotbugs-hadoop-hdfs-project_hadoop-hdfs-rbf-warnings.html] [https://ci-hadoop.apache.org/job/hadoop-qbt-trunk-java8-linux-x86_64/1532/artifact/out/branch-spotbugs-hadoop-hdfs-project-warnings.html] [https://ci-hadoop.apache.org/job/hadoop-qbt-trunk-java8-linux-x86_64/1532/artifact/out/branch-spotbugs-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-applications-warnings.html] [https://ci-hadoop.apache.org/job/hadoop-qbt-trunk-java8-linux-x86_64/1532/artifact/out/branch-spotbugs-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-applications_hadoop-yarn-services-warnings.html] [https://ci-hadoop.apache.org/job/hadoop-qbt-trunk-java8-linux-x86_64/1532/artifact/out/branch-spotbugs-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-applications_hadoop-yarn-services_hadoop-yarn-services-core-warnings.html] [https://ci-hadoop.apache.org/job/hadoop-qbt-trunk-java8-linux-x86_64/1517/artifact/out/branch-spotbugs-hadoop-yarn-project-warnings.html] [https://ci-hadoop.apache.org/job/hadoop-qbt-trunk-java8-linux-x86_64/1517/artifact/out/branch-spotbugs-root-warnings.html] Source: https://ci-hadoop.apache.org/view/Hadoop/job/hadoop-qbt-trunk-java8-linux-x86_64/1532/console was: We are getting spotbugs warnings in every PR. [https://ci-hadoop.apache.org/job/hadoop-qbt-trunk-java8-linux-x86_64/1517/artifact/out/branch-spotbugs-hadoop-common-project_hadoop-common-warnings.html] [https://ci-hadoop.apache.org/job/hadoop-qbt-trunk-java8-linux-x86_64/1517/artifact/out/branch-spotbugs-hadoop-common-project-warnings.html] [https://ci-hadoop.apache.org/job/hadoop-qbt-trunk-java8-linux-x86_64/1517/artifact/out/branch-spotbugs-hadoop-hdfs-project_hadoop-hdfs-client-warnings.html] [https://ci-hadoop.apache.org/job/hadoop-qbt-trunk-java8-linux-x86_64/1517/artifact/out/branch-spotbugs-hadoop-hdfs-project_hadoop-hdfs-httpfs-warnings.html] [https://ci-hadoop.apache.org/job/hadoop-qbt-trunk-java8-linux-x86_64/1517/artifact/out/branch-spotbugs-hadoop-yarn-project_hadoop-yarn-warnings.html] [https://ci-hadoop.apache.org/job/hadoop-qbt-trunk-java8-linux-x86_64/1517/artifact/out/branch-spotbugs-hadoop-hdfs-project_hadoop-hdfs-rbf-warnings.html] [https://ci-hadoop.apache.org/job/hadoop-qbt-trunk-java8-linux-x86_64/1517/artifact/out/branch-spotbugs-hadoop-hdfs-project-warnings.html] [https://ci-hadoop.apache.org/job/hadoop-qbt-trunk-java8-linux-x86_64/1517/artifact/out/branch-spotbugs-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-applications-warnings.html] [https://ci-hadoop.apache.org/job/hadoop-qbt-trunk-java8-linux-x86_64/1517/artifact/out/branch-spotbugs-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-applications_hadoop-yarn-services-warnings.html] [https://ci-hadoop.apache.org/job/hadoop-qbt-trunk-java8-linux-x86_64/1517/artifact/out/branch-spotbugs-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-applications_hadoop-yarn-services_hadoop-yarn-services-core-warnings.html] [https://ci-hadoop.apache.org/job/hadoop-qbt-trunk-java8-linux-x86_64/1517/artifact/out/branch-spotbugs-hadoop-yarn-project-warnings.html] [https://ci-hadoop.apache.org/job/hadoop-qbt-trunk-java8-linux-x86_64/1517/artifact/out/branch-spotbugs-root-warnings.html] Source: https://ci-hadoop.apache.org/view/Hadoop/job/hadoop-qbt-trunk-java8-linux-x86_64/1517/console > Fix Spotbugs warnings in the build > -- > > Key: HADOOP-19100 > URL: https://issues.apache.org/jira/browse/HADOOP-19100 > Project: Hadoop Common > Issue Type: Bug >Reporter: Ayush Saxena >Priority: Major > > We are getting spotbugs warnings in every PR. > [https://ci-hadoop.apache.org/job/hadoop-qbt-trunk-java8-linux-x86_64/1532/artifact/out/branch-spotbugs-hadoop-common-project_hadoop-common-warnings.html] >
[jira] [Comment Edited] (HADOOP-19100) Fix Spotbugs warnings in the build
[ https://issues.apache.org/jira/browse/HADOOP-19100?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17828098#comment-17828098 ] Ayush Saxena edited comment on HADOOP-19100 at 3/18/24 9:01 PM: Thanx [~slfan1989] for volunteering. I think these are coming post hadoop-thirdparty upgrade, maybe that is bringing in some dependency transitively which is causing this, need to find that & exclude in the pom or fix/ignore all of these warnings These first showed on 10th Feb build here: [https://lists.apache.org/thread/3kcbt5hrbxyhv9gndnxlzosfbysxgjbz] Which has Hadoop-Thirdparty upgrade. The build for 9th Feb doesn't have these: [https://lists.apache.org/thread/0tv991c8mvfcfwh3ndyg776qfjh3jmbw] I reverted locally the commit & those spotbugs warnings disappeared was (Author: ayushtkn): Thanx [~slfan1989] for volunteering. I think these are coming post hadoop-thirdparty upgrade, maybe that is bringing in some dependency transitively which is causing this. These first showed on 10th Feb build here: [https://lists.apache.org/thread/3kcbt5hrbxyhv9gndnxlzosfbysxgjbz] Which has Hadoop-Thirdparty upgrade. The build for 9th Feb doesn't have these: [https://lists.apache.org/thread/0tv991c8mvfcfwh3ndyg776qfjh3jmbw] I reverted locally the commit & those spotbugs warnings disappeared > Fix Spotbugs warnings in the build > -- > > Key: HADOOP-19100 > URL: https://issues.apache.org/jira/browse/HADOOP-19100 > Project: Hadoop Common > Issue Type: Bug >Reporter: Ayush Saxena >Priority: Major > > We are getting spotbugs warnings in every PR. > [https://ci-hadoop.apache.org/job/hadoop-qbt-trunk-java8-linux-x86_64/1517/artifact/out/branch-spotbugs-hadoop-common-project_hadoop-common-warnings.html] > [https://ci-hadoop.apache.org/job/hadoop-qbt-trunk-java8-linux-x86_64/1517/artifact/out/branch-spotbugs-hadoop-common-project-warnings.html] > [https://ci-hadoop.apache.org/job/hadoop-qbt-trunk-java8-linux-x86_64/1517/artifact/out/branch-spotbugs-hadoop-hdfs-project_hadoop-hdfs-client-warnings.html] > [https://ci-hadoop.apache.org/job/hadoop-qbt-trunk-java8-linux-x86_64/1517/artifact/out/branch-spotbugs-hadoop-hdfs-project_hadoop-hdfs-httpfs-warnings.html] > [https://ci-hadoop.apache.org/job/hadoop-qbt-trunk-java8-linux-x86_64/1517/artifact/out/branch-spotbugs-hadoop-yarn-project_hadoop-yarn-warnings.html] > [https://ci-hadoop.apache.org/job/hadoop-qbt-trunk-java8-linux-x86_64/1517/artifact/out/branch-spotbugs-hadoop-hdfs-project_hadoop-hdfs-rbf-warnings.html] > [https://ci-hadoop.apache.org/job/hadoop-qbt-trunk-java8-linux-x86_64/1517/artifact/out/branch-spotbugs-hadoop-hdfs-project-warnings.html] > [https://ci-hadoop.apache.org/job/hadoop-qbt-trunk-java8-linux-x86_64/1517/artifact/out/branch-spotbugs-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-applications-warnings.html] > [https://ci-hadoop.apache.org/job/hadoop-qbt-trunk-java8-linux-x86_64/1517/artifact/out/branch-spotbugs-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-applications_hadoop-yarn-services-warnings.html] > [https://ci-hadoop.apache.org/job/hadoop-qbt-trunk-java8-linux-x86_64/1517/artifact/out/branch-spotbugs-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-applications_hadoop-yarn-services_hadoop-yarn-services-core-warnings.html] > [https://ci-hadoop.apache.org/job/hadoop-qbt-trunk-java8-linux-x86_64/1517/artifact/out/branch-spotbugs-hadoop-yarn-project-warnings.html] > [https://ci-hadoop.apache.org/job/hadoop-qbt-trunk-java8-linux-x86_64/1517/artifact/out/branch-spotbugs-root-warnings.html] > > Source: > https://ci-hadoop.apache.org/view/Hadoop/job/hadoop-qbt-trunk-java8-linux-x86_64/1517/console -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Comment Edited] (HADOOP-19100) Fix Spotbugs warnings in the build
[ https://issues.apache.org/jira/browse/HADOOP-19100?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17828098#comment-17828098 ] Ayush Saxena edited comment on HADOOP-19100 at 3/18/24 9:00 PM: Thanx [~slfan1989] for volunteering. I think these are coming post hadoop-thirdparty upgrade, maybe that is bringing in some dependency transitively which is causing this. These first showed on 10th Feb build here: [https://lists.apache.org/thread/3kcbt5hrbxyhv9gndnxlzosfbysxgjbz] Which has Hadoop-Thirdparty upgrade. The build for 9th Feb doesn't have these: [https://lists.apache.org/thread/0tv991c8mvfcfwh3ndyg776qfjh3jmbw] I reverted locally the commit & those spotbugs warnings disappeared was (Author: ayushtkn): Thanx [~slfan1989] for volunteering. I think these are coming post hadoop-thirdparty upgrade, maybe that is bringing in some dependency transitively which is causing this. These first showed on 10th Feb build here: [https://lists.apache.org/thread/3kcbt5hrbxyhv9gndnxlzosfbysxgjbz] Which has Hadoop-Thirdparty upgrade. The build for 9th Feb doesn't have these: https://lists.apache.org/thread/0tv991c8mvfcfwh3ndyg776qfjh3jmbw > Fix Spotbugs warnings in the build > -- > > Key: HADOOP-19100 > URL: https://issues.apache.org/jira/browse/HADOOP-19100 > Project: Hadoop Common > Issue Type: Bug >Reporter: Ayush Saxena >Priority: Major > > We are getting spotbugs warnings in every PR. > [https://ci-hadoop.apache.org/job/hadoop-qbt-trunk-java8-linux-x86_64/1517/artifact/out/branch-spotbugs-hadoop-common-project_hadoop-common-warnings.html] > [https://ci-hadoop.apache.org/job/hadoop-qbt-trunk-java8-linux-x86_64/1517/artifact/out/branch-spotbugs-hadoop-common-project-warnings.html] > [https://ci-hadoop.apache.org/job/hadoop-qbt-trunk-java8-linux-x86_64/1517/artifact/out/branch-spotbugs-hadoop-hdfs-project_hadoop-hdfs-client-warnings.html] > [https://ci-hadoop.apache.org/job/hadoop-qbt-trunk-java8-linux-x86_64/1517/artifact/out/branch-spotbugs-hadoop-hdfs-project_hadoop-hdfs-httpfs-warnings.html] > [https://ci-hadoop.apache.org/job/hadoop-qbt-trunk-java8-linux-x86_64/1517/artifact/out/branch-spotbugs-hadoop-yarn-project_hadoop-yarn-warnings.html] > [https://ci-hadoop.apache.org/job/hadoop-qbt-trunk-java8-linux-x86_64/1517/artifact/out/branch-spotbugs-hadoop-hdfs-project_hadoop-hdfs-rbf-warnings.html] > [https://ci-hadoop.apache.org/job/hadoop-qbt-trunk-java8-linux-x86_64/1517/artifact/out/branch-spotbugs-hadoop-hdfs-project-warnings.html] > [https://ci-hadoop.apache.org/job/hadoop-qbt-trunk-java8-linux-x86_64/1517/artifact/out/branch-spotbugs-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-applications-warnings.html] > [https://ci-hadoop.apache.org/job/hadoop-qbt-trunk-java8-linux-x86_64/1517/artifact/out/branch-spotbugs-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-applications_hadoop-yarn-services-warnings.html] > [https://ci-hadoop.apache.org/job/hadoop-qbt-trunk-java8-linux-x86_64/1517/artifact/out/branch-spotbugs-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-applications_hadoop-yarn-services_hadoop-yarn-services-core-warnings.html] > [https://ci-hadoop.apache.org/job/hadoop-qbt-trunk-java8-linux-x86_64/1517/artifact/out/branch-spotbugs-hadoop-yarn-project-warnings.html] > [https://ci-hadoop.apache.org/job/hadoop-qbt-trunk-java8-linux-x86_64/1517/artifact/out/branch-spotbugs-root-warnings.html] > > Source: > https://ci-hadoop.apache.org/view/Hadoop/job/hadoop-qbt-trunk-java8-linux-x86_64/1517/console -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-19100) Fix Spotbugs warnings in the build
[ https://issues.apache.org/jira/browse/HADOOP-19100?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17828098#comment-17828098 ] Ayush Saxena commented on HADOOP-19100: --- Thanx [~slfan1989] for volunteering. I think these are coming post hadoop-thirdparty upgrade, maybe that is bringing in some dependency transitively which is causing this. These first showed on 10th Feb build here: [https://lists.apache.org/thread/3kcbt5hrbxyhv9gndnxlzosfbysxgjbz] Which has Hadoop-Thirdparty upgrade. The build for 9th Feb doesn't have these: https://lists.apache.org/thread/0tv991c8mvfcfwh3ndyg776qfjh3jmbw > Fix Spotbugs warnings in the build > -- > > Key: HADOOP-19100 > URL: https://issues.apache.org/jira/browse/HADOOP-19100 > Project: Hadoop Common > Issue Type: Bug >Reporter: Ayush Saxena >Priority: Major > > We are getting spotbugs warnings in every PR. > [https://ci-hadoop.apache.org/job/hadoop-qbt-trunk-java8-linux-x86_64/1517/artifact/out/branch-spotbugs-hadoop-common-project_hadoop-common-warnings.html] > [https://ci-hadoop.apache.org/job/hadoop-qbt-trunk-java8-linux-x86_64/1517/artifact/out/branch-spotbugs-hadoop-common-project-warnings.html] > [https://ci-hadoop.apache.org/job/hadoop-qbt-trunk-java8-linux-x86_64/1517/artifact/out/branch-spotbugs-hadoop-hdfs-project_hadoop-hdfs-client-warnings.html] > [https://ci-hadoop.apache.org/job/hadoop-qbt-trunk-java8-linux-x86_64/1517/artifact/out/branch-spotbugs-hadoop-hdfs-project_hadoop-hdfs-httpfs-warnings.html] > [https://ci-hadoop.apache.org/job/hadoop-qbt-trunk-java8-linux-x86_64/1517/artifact/out/branch-spotbugs-hadoop-yarn-project_hadoop-yarn-warnings.html] > [https://ci-hadoop.apache.org/job/hadoop-qbt-trunk-java8-linux-x86_64/1517/artifact/out/branch-spotbugs-hadoop-hdfs-project_hadoop-hdfs-rbf-warnings.html] > [https://ci-hadoop.apache.org/job/hadoop-qbt-trunk-java8-linux-x86_64/1517/artifact/out/branch-spotbugs-hadoop-hdfs-project-warnings.html] > [https://ci-hadoop.apache.org/job/hadoop-qbt-trunk-java8-linux-x86_64/1517/artifact/out/branch-spotbugs-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-applications-warnings.html] > [https://ci-hadoop.apache.org/job/hadoop-qbt-trunk-java8-linux-x86_64/1517/artifact/out/branch-spotbugs-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-applications_hadoop-yarn-services-warnings.html] > [https://ci-hadoop.apache.org/job/hadoop-qbt-trunk-java8-linux-x86_64/1517/artifact/out/branch-spotbugs-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-applications_hadoop-yarn-services_hadoop-yarn-services-core-warnings.html] > [https://ci-hadoop.apache.org/job/hadoop-qbt-trunk-java8-linux-x86_64/1517/artifact/out/branch-spotbugs-hadoop-yarn-project-warnings.html] > [https://ci-hadoop.apache.org/job/hadoop-qbt-trunk-java8-linux-x86_64/1517/artifact/out/branch-spotbugs-root-warnings.html] > > Source: > https://ci-hadoop.apache.org/view/Hadoop/job/hadoop-qbt-trunk-java8-linux-x86_64/1517/console -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-19107) Drop support for HBase v1
[
https://issues.apache.org/jira/browse/HADOOP-19107?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17825580#comment-17825580
]
Ayush Saxena commented on HADOOP-19107:
---
Currently v2 is broken for me
{noformat}
[INFO] --- enforcer:3.0.0:enforce (depcheck) @
hadoop-yarn-server-timelineservice-hbase-client ---
[WARNING] Rule 1: org.apache.maven.plugins.enforcer.BannedDependencies failed
with message:
Found Banned Dependency: org.slf4j:slf4j-log4j12:jar:1.7.25
Use 'mvn dependency:tree' to locate the source of the banned dependencies.
{noformat}
Tried "mvn clean install -DskipTests -Dhbase.profile=2.0", need some exclusion
for sl4j somewhere I believe
> Drop support for HBase v1
> -
>
> Key: HADOOP-19107
> URL: https://issues.apache.org/jira/browse/HADOOP-19107
> Project: Hadoop Common
> Issue Type: Task
>Reporter: Ayush Saxena
>Priority: Major
>
> Drop support for Hbase V1 and make building Hbase v2 default.
> Dev List:
> [https://lists.apache.org/thread/vb2gh5ljwncbrmqnk0oflb8ftdz64hhs]
> https://lists.apache.org/thread/o88hnm7q8n3b4bng81q14vsj3fbhfx5w
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Created] (HADOOP-19107) Drop support for HBase v1
Ayush Saxena created HADOOP-19107: - Summary: Drop support for HBase v1 Key: HADOOP-19107 URL: https://issues.apache.org/jira/browse/HADOOP-19107 Project: Hadoop Common Issue Type: Task Reporter: Ayush Saxena Drop support for Hbase V1 and make building Hbase v2 default. Dev List: [https://lists.apache.org/thread/vb2gh5ljwncbrmqnk0oflb8ftdz64hhs] https://lists.apache.org/thread/o88hnm7q8n3b4bng81q14vsj3fbhfx5w -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Resolved] (HADOOP-19090) Update Protocol Buffers installation to 3.23.4
[ https://issues.apache.org/jira/browse/HADOOP-19090?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ayush Saxena resolved HADOOP-19090. --- Fix Version/s: 3.4.1 Hadoop Flags: Reviewed Resolution: Fixed > Update Protocol Buffers installation to 3.23.4 > -- > > Key: HADOOP-19090 > URL: https://issues.apache.org/jira/browse/HADOOP-19090 > Project: Hadoop Common > Issue Type: Improvement > Components: build >Affects Versions: 3.4.0, 3.3.9 >Reporter: PJ Fanning >Assignee: PJ Fanning >Priority: Major > Labels: pull-request-available > Fix For: 3.4.1 > > Time Spent: 1h 10m > Remaining Estimate: 0h > > We are seeing issues with Java 8 usage of protobuf-java > See https://issues.apache.org/jira/browse/HADOOP-18197 and comments about > java.lang.NoSuchMethodError: > java.nio.ByteBuffer.position(I)Ljava/nio/ByteBuffer; -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-19090) Update Protocol Buffers installation to 3.23.4
[ https://issues.apache.org/jira/browse/HADOOP-19090?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17824343#comment-17824343 ] Ayush Saxena commented on HADOOP-19090: --- Committed to trunk. Thanx [~pj.fanning] for the contribution & [~hexiaoqiao] for the review!!! > Update Protocol Buffers installation to 3.23.4 > -- > > Key: HADOOP-19090 > URL: https://issues.apache.org/jira/browse/HADOOP-19090 > Project: Hadoop Common > Issue Type: Improvement > Components: build >Affects Versions: 3.4.0, 3.3.9 >Reporter: PJ Fanning >Assignee: PJ Fanning >Priority: Major > Labels: pull-request-available > Time Spent: 1h 10m > Remaining Estimate: 0h > > We are seeing issues with Java 8 usage of protobuf-java > See https://issues.apache.org/jira/browse/HADOOP-18197 and comments about > java.lang.NoSuchMethodError: > java.nio.ByteBuffer.position(I)Ljava/nio/ByteBuffer; -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-19100) Fix Spotbugs warnings in the build
[ https://issues.apache.org/jira/browse/HADOOP-19100?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17823050#comment-17823050 ] Ayush Saxena commented on HADOOP-19100: --- not sure which commit induced these, but we should fix/ignore these in the code, so as to get green builds > Fix Spotbugs warnings in the build > -- > > Key: HADOOP-19100 > URL: https://issues.apache.org/jira/browse/HADOOP-19100 > Project: Hadoop Common > Issue Type: Bug >Reporter: Ayush Saxena >Priority: Major > > We are getting spotbugs warnings in every PR. > [https://ci-hadoop.apache.org/job/hadoop-qbt-trunk-java8-linux-x86_64/1517/artifact/out/branch-spotbugs-hadoop-common-project_hadoop-common-warnings.html] > [https://ci-hadoop.apache.org/job/hadoop-qbt-trunk-java8-linux-x86_64/1517/artifact/out/branch-spotbugs-hadoop-common-project-warnings.html] > [https://ci-hadoop.apache.org/job/hadoop-qbt-trunk-java8-linux-x86_64/1517/artifact/out/branch-spotbugs-hadoop-hdfs-project_hadoop-hdfs-client-warnings.html] > [https://ci-hadoop.apache.org/job/hadoop-qbt-trunk-java8-linux-x86_64/1517/artifact/out/branch-spotbugs-hadoop-hdfs-project_hadoop-hdfs-httpfs-warnings.html] > [https://ci-hadoop.apache.org/job/hadoop-qbt-trunk-java8-linux-x86_64/1517/artifact/out/branch-spotbugs-hadoop-yarn-project_hadoop-yarn-warnings.html] > [https://ci-hadoop.apache.org/job/hadoop-qbt-trunk-java8-linux-x86_64/1517/artifact/out/branch-spotbugs-hadoop-hdfs-project_hadoop-hdfs-rbf-warnings.html] > [https://ci-hadoop.apache.org/job/hadoop-qbt-trunk-java8-linux-x86_64/1517/artifact/out/branch-spotbugs-hadoop-hdfs-project-warnings.html] > [https://ci-hadoop.apache.org/job/hadoop-qbt-trunk-java8-linux-x86_64/1517/artifact/out/branch-spotbugs-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-applications-warnings.html] > [https://ci-hadoop.apache.org/job/hadoop-qbt-trunk-java8-linux-x86_64/1517/artifact/out/branch-spotbugs-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-applications_hadoop-yarn-services-warnings.html] > [https://ci-hadoop.apache.org/job/hadoop-qbt-trunk-java8-linux-x86_64/1517/artifact/out/branch-spotbugs-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-applications_hadoop-yarn-services_hadoop-yarn-services-core-warnings.html] > [https://ci-hadoop.apache.org/job/hadoop-qbt-trunk-java8-linux-x86_64/1517/artifact/out/branch-spotbugs-hadoop-yarn-project-warnings.html] > [https://ci-hadoop.apache.org/job/hadoop-qbt-trunk-java8-linux-x86_64/1517/artifact/out/branch-spotbugs-root-warnings.html] > > Source: > https://ci-hadoop.apache.org/view/Hadoop/job/hadoop-qbt-trunk-java8-linux-x86_64/1517/console -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Created] (HADOOP-19100) Fix Spotbugs warnings in the build
Ayush Saxena created HADOOP-19100: - Summary: Fix Spotbugs warnings in the build Key: HADOOP-19100 URL: https://issues.apache.org/jira/browse/HADOOP-19100 Project: Hadoop Common Issue Type: Bug Reporter: Ayush Saxena We are getting spotbugs warnings in every PR. [https://ci-hadoop.apache.org/job/hadoop-qbt-trunk-java8-linux-x86_64/1517/artifact/out/branch-spotbugs-hadoop-common-project_hadoop-common-warnings.html] [https://ci-hadoop.apache.org/job/hadoop-qbt-trunk-java8-linux-x86_64/1517/artifact/out/branch-spotbugs-hadoop-common-project-warnings.html] [https://ci-hadoop.apache.org/job/hadoop-qbt-trunk-java8-linux-x86_64/1517/artifact/out/branch-spotbugs-hadoop-hdfs-project_hadoop-hdfs-client-warnings.html] [https://ci-hadoop.apache.org/job/hadoop-qbt-trunk-java8-linux-x86_64/1517/artifact/out/branch-spotbugs-hadoop-hdfs-project_hadoop-hdfs-httpfs-warnings.html] [https://ci-hadoop.apache.org/job/hadoop-qbt-trunk-java8-linux-x86_64/1517/artifact/out/branch-spotbugs-hadoop-yarn-project_hadoop-yarn-warnings.html] [https://ci-hadoop.apache.org/job/hadoop-qbt-trunk-java8-linux-x86_64/1517/artifact/out/branch-spotbugs-hadoop-hdfs-project_hadoop-hdfs-rbf-warnings.html] [https://ci-hadoop.apache.org/job/hadoop-qbt-trunk-java8-linux-x86_64/1517/artifact/out/branch-spotbugs-hadoop-hdfs-project-warnings.html] [https://ci-hadoop.apache.org/job/hadoop-qbt-trunk-java8-linux-x86_64/1517/artifact/out/branch-spotbugs-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-applications-warnings.html] [https://ci-hadoop.apache.org/job/hadoop-qbt-trunk-java8-linux-x86_64/1517/artifact/out/branch-spotbugs-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-applications_hadoop-yarn-services-warnings.html] [https://ci-hadoop.apache.org/job/hadoop-qbt-trunk-java8-linux-x86_64/1517/artifact/out/branch-spotbugs-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-applications_hadoop-yarn-services_hadoop-yarn-services-core-warnings.html] [https://ci-hadoop.apache.org/job/hadoop-qbt-trunk-java8-linux-x86_64/1517/artifact/out/branch-spotbugs-hadoop-yarn-project-warnings.html] [https://ci-hadoop.apache.org/job/hadoop-qbt-trunk-java8-linux-x86_64/1517/artifact/out/branch-spotbugs-root-warnings.html] Source: https://ci-hadoop.apache.org/view/Hadoop/job/hadoop-qbt-trunk-java8-linux-x86_64/1517/console -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-18197) Update protobuf 3.7.1 to a version without CVE-2021-22569
[ https://issues.apache.org/jira/browse/HADOOP-18197?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17820711#comment-17820711 ] Ayush Saxena commented on HADOOP-18197: --- If protobuf 3.23 has a fix & released, I think we should prefer that rather than patching protobuf ourselves in our code. I think we should attempt upgrading protobuf version to the one which has the fix, if that fails then we can try the pathing approach > Update protobuf 3.7.1 to a version without CVE-2021-22569 > - > > Key: HADOOP-18197 > URL: https://issues.apache.org/jira/browse/HADOOP-18197 > Project: Hadoop Common > Issue Type: Improvement > Components: hadoop-thirdparty >Affects Versions: thirdparty-1.2.0 >Reporter: Ivan Viaznikov >Assignee: PJ Fanning >Priority: Major > Labels: pull-request-available, security > Fix For: thirdparty-1.2.0 > > Time Spent: 2h 20m > Remaining Estimate: 0h > > The artifact `org.apache.hadoop:hadoop-common` brings in a dependency > `com.google.protobuf:protobuf-java:2.5.0`, which is an outdated version > released in 2013 and it contains a vulnerability > [CVE-2021-22569|https://nvd.nist.gov/vuln/detail/CVE-2021-22569]. > Therefore, requesting you to clarify if this library version is going to be > updated in the following releases -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-18197) Update protobuf 3.7.1 to a version without CVE-2021-22569
[
https://issues.apache.org/jira/browse/HADOOP-18197?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17818651#comment-17818651
]
Ayush Saxena commented on HADOOP-18197:
---
I think this is causing some trouble due to some incompatibilities introduced
in Java
https://ci-hadoop.apache.org/job/hadoop-qbt-trunk-java8-linux-x86_64/1503/testReport/junit/org.apache.hadoop.hdfs.protocol/TestBlockListAsLongs/testFuzz/
{noformat}
java.lang.NoSuchMethodError:
java.nio.ByteBuffer.position(I)Ljava/nio/ByteBuffer;
at
org.apache.hadoop.thirdparty.protobuf.IterableByteBufferInputStream.read(IterableByteBufferInputStream.java:143)
at
org.apache.hadoop.thirdparty.protobuf.CodedInputStream$StreamDecoder.read(CodedInputStream.java:2080)
at
org.apache.hadoop.thirdparty.protobuf.CodedInputStream$StreamDecoder.tryRefillBuffer(CodedInputStream.java:2831)
at
org.apache.hadoop.thirdparty.protobuf.CodedInputStream$StreamDecoder.refillBuffer(CodedInputStream.java:2777)
at
org.apache.hadoop.thirdparty.protobuf.CodedInputStream$StreamDecoder.readRawByte(CodedInputStream.java:2859)
at
org.apache.hadoop.thirdparty.protobuf.CodedInputStream$StreamDecoder.readRawVarint64SlowPath(CodedInputStream.java:2648)
at
org.apache.hadoop.thirdparty.protobuf.CodedInputStream$StreamDecoder.readRawVarint64(CodedInputStream.java:2641)
at
org.apache.hadoop.thirdparty.protobuf.CodedInputStream$StreamDecoder.readSInt64(CodedInputStream.java:2497)
{noformat}
Rolling back the thirdparty upgrade from trunk, did solve this, so I doubt it
could be because of this change (didn't investigate further), there are lot of
other failures as well in the daily build, which I have just started to look
> Update protobuf 3.7.1 to a version without CVE-2021-22569
> -
>
> Key: HADOOP-18197
> URL: https://issues.apache.org/jira/browse/HADOOP-18197
> Project: Hadoop Common
> Issue Type: Improvement
> Components: hadoop-thirdparty
>Affects Versions: thirdparty-1.2.0
>Reporter: Ivan Viaznikov
>Assignee: PJ Fanning
>Priority: Major
> Labels: pull-request-available, security
> Fix For: thirdparty-1.2.0
>
> Time Spent: 2h 20m
> Remaining Estimate: 0h
>
> The artifact `org.apache.hadoop:hadoop-common` brings in a dependency
> `com.google.protobuf:protobuf-java:2.5.0`, which is an outdated version
> released in 2013 and it contains a vulnerability
> [CVE-2021-22569|https://nvd.nist.gov/vuln/detail/CVE-2021-22569].
> Therefore, requesting you to clarify if this library version is going to be
> updated in the following releases
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-18716) [JDK-17] Failed unit tests , with Java 17 runtime and compiled Java 8
[ https://issues.apache.org/jira/browse/HADOOP-18716?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17812647#comment-17812647 ] Ayush Saxena commented on HADOOP-18716: --- Nopes, feel free to take over If you plan to chase > [JDK-17] Failed unit tests , with Java 17 runtime and compiled Java 8 > - > > Key: HADOOP-18716 > URL: https://issues.apache.org/jira/browse/HADOOP-18716 > Project: Hadoop Common > Issue Type: Bug >Reporter: Vinay Devadiga >Priority: Critical > > Compiled Hadoop - Hadoop branch 3.3.3 > mvn clean install - DskipTests=True > Java_Home -> points to Java-8 > maven version - 3.8.8 (Quite latest) > > Ran various whole test suit on my private cloud environment - > Changed Java_Home to Java-17 > > mvn surefire:test > > Out of 22k tests - 2.5 k tests failed . -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-18991) Remove commons-beanutils dependency from Hadoop 3
[ https://issues.apache.org/jira/browse/HADOOP-18991?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17801768#comment-17801768 ] Ayush Saxena commented on HADOOP-18991: --- >From hive perspective, I don't think removing this would be a problem, >hadoop-3.3.x+ won't work out of the box with lower hive versions, so it would >anyway require code changes & hive-4.x is already sorted, it has an explicit >dependency & if needed that could be backported > Remove commons-beanutils dependency from Hadoop 3 > - > > Key: HADOOP-18991 > URL: https://issues.apache.org/jira/browse/HADOOP-18991 > Project: Hadoop Common > Issue Type: Improvement > Components: common >Reporter: Istvan Toth >Priority: Major > > Hadoop doesn't acually use it, and it pollutes the classpath of dependent > projects. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-19020) Update the year to 2024
[ https://issues.apache.org/jira/browse/HADOOP-19020?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17801505#comment-17801505 ] Ayush Saxena commented on HADOOP-19020: --- Committed to trunk, branch-3.3 & 2.10 Thanx [~groot] & [~slfan1989] for the reviews!!! > Update the year to 2024 > --- > > Key: HADOOP-19020 > URL: https://issues.apache.org/jira/browse/HADOOP-19020 > Project: Hadoop Common > Issue Type: Task >Reporter: Ayush Saxena >Assignee: Ayush Saxena >Priority: Major > Labels: pull-request-available > > Update the year to 2024 -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Resolved] (HADOOP-19020) Update the year to 2024
[ https://issues.apache.org/jira/browse/HADOOP-19020?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ayush Saxena resolved HADOOP-19020. --- Fix Version/s: 3.4.0 2.10.3 3.3.7 Hadoop Flags: Reviewed Resolution: Fixed > Update the year to 2024 > --- > > Key: HADOOP-19020 > URL: https://issues.apache.org/jira/browse/HADOOP-19020 > Project: Hadoop Common > Issue Type: Task >Reporter: Ayush Saxena >Assignee: Ayush Saxena >Priority: Major > Labels: pull-request-available > Fix For: 3.4.0, 2.10.3, 3.3.7 > > > Update the year to 2024 -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Created] (HADOOP-19020) Update the year to 2024
Ayush Saxena created HADOOP-19020: - Summary: Update the year to 2024 Key: HADOOP-19020 URL: https://issues.apache.org/jira/browse/HADOOP-19020 Project: Hadoop Common Issue Type: Task Reporter: Ayush Saxena Assignee: Ayush Saxena Update the year to 2024 -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-18754) Make DistCp Split files hidden to prevent interfering with query engines
[ https://issues.apache.org/jira/browse/HADOOP-18754?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17799575#comment-17799575 ] Ayush Saxena commented on HADOOP-18754: --- which query engine? the split files aren't generated in the table directory that it can bother the query engine, anyway those directory paths are configurable like normal MR jobs. share more details... > Make DistCp Split files hidden to prevent interfering with query engines > - > > Key: HADOOP-18754 > URL: https://issues.apache.org/jira/browse/HADOOP-18754 > Project: Hadoop Common > Issue Type: Improvement > Components: tools/distcp, util >Reporter: Chayanika Bhandary >Priority: Major > > As of today, the distcp split files are not hidden, and hence could interfere > with query engines. Making them hidden. > > Contributing a fix to it. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-17424) Replace HTrace with No-Op tracer
[
https://issues.apache.org/jira/browse/HADOOP-17424?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17799029#comment-17799029
]
Ayush Saxena commented on HADOOP-17424:
---
What is the fix to this incompatibility?
It creates issues with Tez which is on 3.3.1 & Hive which is on 3.3.6, both
hive & tez jars are required during runtime,
{noformat}
2023-12-20T19:36:01,147 WARN [main] server.HiveServer2: Error starting
HiveServer2 on attempt 3, will retry in 6ms
java.lang.NoSuchMethodError:
org.apache.hadoop.fs.FsTracer.get(Lorg/apache/hadoop/conf/Configuration;)Lorg/apache/htrace/core/Tracer;
at org.apache.hadoop.hdfs.DFSClient.(DFSClient.java:323)
~[hadoop-hdfs-client-3.3.1.jar:?]
at org.apache.hadoop.hdfs.DFSClient.(DFSClient.java:308)
~[hadoop-hdfs-client-3.3.1.jar:?]
at
org.apache.hadoop.hdfs.DistributedFileSystem.initDFSClient(DistributedFileSystem.java:201)
~[hadoop-hdfs-client-3.3.1.jar:?]
at
org.apache.hadoop.hdfs.DistributedFileSystem.initialize(DistributedFileSystem.java:186)
~[hadoop-hdfs-client-3.3.1.jar:?]
at
org.apache.hadoop.fs.FileSystem.createFileSystem(FileSystem.java:3611)
~[hadoop-common-3.3.6.jar:?]
at org.apache.hadoop.fs.FileSystem.access$300(FileSystem.java:174)
~[hadoop-common-3.3.6.jar:?]
at
org.apache.hadoop.fs.FileSystem$Cache.getInternal(FileSystem.java:3712)
~[hadoop-common-3.3.6.jar:?]
at org.apache.hadoop.fs.FileSystem$Cache.get(FileSystem.java:3663)
~[hadoop-common-3.3.6.jar:?]
at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:557)
~[hadoop-common-3.3.6.jar:?]
at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:290)
~[hadoop-common-3.3.6.jar:?]
at
org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:705)
~[hive-exec-4.0.0-beta-2-SNAPSHOT.jar:4.0.0-beta-2-SNAPSHOT]
at
org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:669)
~[hive-exec-4.0.0-beta-2-SNAPSHOT.jar:4.0.0-beta-2-SNAPSHOT]
at
org.apache.hive.service.cli.CLIService.applyAuthorizationConfigPolicy(CLIService.java:134)
~[hive-service-4.0.0-beta-2-SNAPSHOT.jar:4.0.0-beta-2-SNAPSHOT]
at org.apache.hive.service.cli.CLIService.init(CLIService.java:119)
~[hive-service-4.0.0-beta-2-SNAPSHOT.jar:4.0.0-beta-2-SNAPSHOT]
at
org.apache.hive.service.CompositeService.init(CompositeService.java:59)
~[hive-service-4.0.0-beta-2-SNAPSHOT.jar:4.0.0-beta-2-SNAPSHOT]
at
org.apache.hive.service.server.HiveServer2.init(HiveServer2.java:243)
~[hive-service-4.0.0-beta-2-SNAPSHOT.jar:4.0.0-beta-2-SNAPSHOT]
at
org.apache.hive.service.server.HiveServer2.startHiveServer2(HiveServer2.java:1114)
~[hive-service-4.0.0-beta-2-SNAPSHOT.jar:4.0.0-beta-2-SNAPSHOT]
at
org.apache.hive.service.server.HiveServer2.access$1500(HiveServer2.java:146)
~[hive-service-4.0.0-beta-2-SNAPSHOT.jar:4.0.0-beta-2-SNAPSHOT]
at
org.apache.hive.service.server.HiveServer2$StartOptionExecutor.execute(HiveServer2.java:1476)
~[hive-service-4.0.0-beta-2-SNAPSHOT.jar:4.0.0-beta-2-SNAPSHOT]
at
org.apache.hive.service.server.HiveServer2.main(HiveServer2.java:1283)
~[hive-service-4.0.0-beta-2-SNAPSHOT.jar:4.0.0-beta-2-SNAPSHOT]
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
~[?:1.8.0_342]
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
~[?:1.8.0_342]
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
~[?:1.8.0_342]
at java.lang.reflect.Method.invoke(Method.java:498) ~[?:1.8.0_342]
at org.apache.hadoop.util.RunJar.run(RunJar.java:328)
~[hadoop-common-3.3.6.jar:?]
at org.apache.hadoop.util.RunJar.main(RunJar.java:241)
~[hadoop-common-3.3.6.jar:?]{noformat}
Similar stuff here as well:
[https://github.com/exasol/cloud-storage-extension/issues/219]
We could have explore keeping the package name same right?
> Replace HTrace with No-Op tracer
>
>
> Key: HADOOP-17424
> URL: https://issues.apache.org/jira/browse/HADOOP-17424
> Project: Hadoop Common
> Issue Type: Sub-task
>Reporter: Siyao Meng
>Assignee: Siyao Meng
>Priority: Major
> Labels: pull-request-available
> Fix For: 3.4.0, 3.3.2
>
> Time Spent: 10.5h
> Remaining Estimate: 0h
>
> Remove HTrace dependency as it is depending on old jackson jars. Use a no-op
> tracer for now to eliminate potential security issues.
> The plan is to move part of the code in
> [PR#1846|https://github.com/apache/hadoop/pull/1846] out here for faster
> review.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail:
[jira] [Commented] (HADOOP-18613) Upgrade ZooKeeper to version 3.8.3
[ https://issues.apache.org/jira/browse/HADOOP-18613?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17798670#comment-17798670 ] Ayush Saxena commented on HADOOP-18613: --- Committed to trunk. Thanx [~BilwaST] for the contribution!!! > Upgrade ZooKeeper to version 3.8.3 > -- > > Key: HADOOP-18613 > URL: https://issues.apache.org/jira/browse/HADOOP-18613 > Project: Hadoop Common > Issue Type: Improvement > Components: common >Affects Versions: 3.3.4 >Reporter: Tamas Penzes >Assignee: Bilwa S T >Priority: Major > Labels: pull-request-available > -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Resolved] (HADOOP-18613) Upgrade ZooKeeper to version 3.8.3
[ https://issues.apache.org/jira/browse/HADOOP-18613?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ayush Saxena resolved HADOOP-18613. --- Fix Version/s: 3.4.0 Hadoop Flags: Reviewed Resolution: Fixed > Upgrade ZooKeeper to version 3.8.3 > -- > > Key: HADOOP-18613 > URL: https://issues.apache.org/jira/browse/HADOOP-18613 > Project: Hadoop Common > Issue Type: Improvement > Components: common >Affects Versions: 3.3.4 >Reporter: Tamas Penzes >Assignee: Bilwa S T >Priority: Major > Labels: pull-request-available > Fix For: 3.4.0 > > -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-18924) Upgrade grpc jars to v1.53.0 due to CVEs
[ https://issues.apache.org/jira/browse/HADOOP-18924?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ayush Saxena updated HADOOP-18924: -- Summary: Upgrade grpc jars to v1.53.0 due to CVEs (was: upgrade grpc jars to v1.53.0 due to CVEs) > Upgrade grpc jars to v1.53.0 due to CVEs > > > Key: HADOOP-18924 > URL: https://issues.apache.org/jira/browse/HADOOP-18924 > Project: Hadoop Common > Issue Type: Improvement > Components: build >Reporter: PJ Fanning >Assignee: PJ Fanning >Priority: Major > Labels: pull-request-available > > https://mvnrepository.com/artifact/io.grpc/grpc-protobuf -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-18991) Remove commons-benautils dependency from Hadoop 3
[
https://issues.apache.org/jira/browse/HADOOP-18991?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17790907#comment-17790907
]
Ayush Saxena commented on HADOOP-18991:
---
The error stack from the previous ticket, which Wei-Chiu posted
{noformat}
/container.common/build/cdh/hive/2.1.1-cdh6.x-SNAPSHOT/source/metastore/src/java/org/apache/hadoop/hive/metastore/MetaStoreUtils.java:[57,36]
package org.apache.commons.beanutils does not exist{noformat}
It was from metastore sever, & now it has a direct dependency on bean-utils now
[https://github.com/apache/hive/blob/master/standalone-metastore/metastore-server/pom.xml#L226-L229]
but the version then tried seems to be hive 2.1.1
> Remove commons-benautils dependency from Hadoop 3
> -
>
> Key: HADOOP-18991
> URL: https://issues.apache.org/jira/browse/HADOOP-18991
> Project: Hadoop Common
> Issue Type: Improvement
> Components: common
>Reporter: Istvan Toth
>Priority: Major
>
> Hadoop doesn't acually use it, and it pollutes the classpath of dependent
> projects.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Comment Edited] (HADOOP-18991) Remove commons-benautils dependency from Hadoop 3
[ https://issues.apache.org/jira/browse/HADOOP-18991?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17790904#comment-17790904 ] Ayush Saxena edited comment on HADOOP-18991 at 11/29/23 5:49 AM: - Hive latest shouldn't be bothered post HIVE-23268, though I haven't tried, the usage is in Metastore, which isn't just a Hive thing, not sure what will happen to older Metastore if they try to run on a newer hadoop, if it works today. Neverthless, it had a valid veto in the previous ticket which attempted this, so if that wasn't irrelevant, then make sure those concerns aren't outsmarted here. Though I don't think removing a transitive dependency which isn't used would be a problem, but the previous ticket say so, we can accept. If we plan to chase this, should be reviewed carefully by "some safe hands & proper testing" was (Author: ayushtkn): Hive latest shouldn't be bothered post HIVE-23268, though I haven't tried, the usage is in Metastore, which isn't just a Hive thing, not sure what will happen to older Metastore if they try to run on a newer hadoop, if it works today. Neverthless, it had a valid veto in the previous ticket which attempted this, so if that wasn't irrelevant, then make sure those concerns aren't outsmarted here. > Remove commons-benautils dependency from Hadoop 3 > - > > Key: HADOOP-18991 > URL: https://issues.apache.org/jira/browse/HADOOP-18991 > Project: Hadoop Common > Issue Type: Improvement > Components: common >Reporter: Istvan Toth >Priority: Major > > Hadoop doesn't acually use it, and it pollutes the classpath of dependent > projects. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-18991) Remove commons-benautils dependency from Hadoop 3
[ https://issues.apache.org/jira/browse/HADOOP-18991?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17790906#comment-17790906 ] Ayush Saxena commented on HADOOP-18991: --- [~stoty] the hive one is: HIVE-23268 > Remove commons-benautils dependency from Hadoop 3 > - > > Key: HADOOP-18991 > URL: https://issues.apache.org/jira/browse/HADOOP-18991 > Project: Hadoop Common > Issue Type: Improvement > Components: common >Reporter: Istvan Toth >Priority: Major > > Hadoop doesn't acually use it, and it pollutes the classpath of dependent > projects. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-18991) Remove commons-benautils dependency from Hadoop 3
[ https://issues.apache.org/jira/browse/HADOOP-18991?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17790904#comment-17790904 ] Ayush Saxena commented on HADOOP-18991: --- Hive latest shouldn't be bothered post HIVE-23268, though I haven't tried, the usage is in Metastore, which isn't just a Hive thing, not sure what will happen to older Metastore if they try to run on a newer hadoop, if it works today. Neverthless, it had a valid veto in the previous ticket which attempted this, so if that wasn't irrelevant, then make sure those concerns aren't outsmarted here. > Remove commons-benautils dependency from Hadoop 3 > - > > Key: HADOOP-18991 > URL: https://issues.apache.org/jira/browse/HADOOP-18991 > Project: Hadoop Common > Issue Type: Improvement > Components: common >Reporter: Istvan Toth >Priority: Major > > Hadoop doesn't acually use it, and it pollutes the classpath of dependent > projects. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-18957) Use StandardCharsets.UTF_8 constant
[ https://issues.apache.org/jira/browse/HADOOP-18957?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17788119#comment-17788119 ] Ayush Saxena commented on HADOOP-18957: --- Committed to trunk. Thanx [~pj.fanning] for the contribution!!! > Use StandardCharsets.UTF_8 constant > --- > > Key: HADOOP-18957 > URL: https://issues.apache.org/jira/browse/HADOOP-18957 > Project: Hadoop Common > Issue Type: Improvement >Reporter: PJ Fanning >Assignee: PJ Fanning >Priority: Major > Labels: pull-request-available > > * there are some places in the code that have to check for > UnsupportedCharsetException when explicitly using the charset name "UTF-8" > * using StandardCharsets.UTF_8 is more efficient because the Java libs > usually have to look up the charsets when you provide it as String param > instead > * also stop using Guava Charsets and use StandardCharsets -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Resolved] (HADOOP-18957) Use StandardCharsets.UTF_8 constant
[ https://issues.apache.org/jira/browse/HADOOP-18957?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ayush Saxena resolved HADOOP-18957. --- Fix Version/s: 3.4.0 Hadoop Flags: Reviewed Resolution: Fixed > Use StandardCharsets.UTF_8 constant > --- > > Key: HADOOP-18957 > URL: https://issues.apache.org/jira/browse/HADOOP-18957 > Project: Hadoop Common > Issue Type: Improvement >Reporter: PJ Fanning >Assignee: PJ Fanning >Priority: Major > Labels: pull-request-available > Fix For: 3.4.0 > > > * there are some places in the code that have to check for > UnsupportedCharsetException when explicitly using the charset name "UTF-8" > * using StandardCharsets.UTF_8 is more efficient because the Java libs > usually have to look up the charsets when you provide it as String param > instead > * also stop using Guava Charsets and use StandardCharsets -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Assigned] (HADOOP-18957) Use StandardCharsets.UTF_8 constant
[ https://issues.apache.org/jira/browse/HADOOP-18957?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ayush Saxena reassigned HADOOP-18957: - Assignee: PJ Fanning > Use StandardCharsets.UTF_8 constant > --- > > Key: HADOOP-18957 > URL: https://issues.apache.org/jira/browse/HADOOP-18957 > Project: Hadoop Common > Issue Type: Improvement >Reporter: PJ Fanning >Assignee: PJ Fanning >Priority: Major > Labels: pull-request-available > > * there are some places in the code that have to check for > UnsupportedCharsetException when explicitly using the charset name "UTF-8" > * using StandardCharsets.UTF_8 is more efficient because the Java libs > usually have to look up the charsets when you provide it as String param > instead > * also stop using Guava Charsets and use StandardCharsets -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-18964) Update plugin for SBOM generation to 2.7.10 #6235
[ https://issues.apache.org/jira/browse/HADOOP-18964?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17786375#comment-17786375 ] Ayush Saxena commented on HADOOP-18964: --- Committed to trunk. Thanx [~vinodanandan] for the contribution!!! > Update plugin for SBOM generation to 2.7.10 #6235 > - > > Key: HADOOP-18964 > URL: https://issues.apache.org/jira/browse/HADOOP-18964 > Project: Hadoop Common > Issue Type: Improvement >Reporter: Vinod Anandan >Assignee: Vinod Anandan >Priority: Major > Labels: pull-request-available > > Update the CycloneDX Maven plugin for SBOM generation to 2.7.10 -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Resolved] (HADOOP-18964) Update plugin for SBOM generation to 2.7.10 #6235
[ https://issues.apache.org/jira/browse/HADOOP-18964?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ayush Saxena resolved HADOOP-18964. --- Fix Version/s: 3.4.0 Hadoop Flags: Reviewed Resolution: Fixed > Update plugin for SBOM generation to 2.7.10 #6235 > - > > Key: HADOOP-18964 > URL: https://issues.apache.org/jira/browse/HADOOP-18964 > Project: Hadoop Common > Issue Type: Improvement >Reporter: Vinod Anandan >Assignee: Vinod Anandan >Priority: Major > Labels: pull-request-available > Fix For: 3.4.0 > > > Update the CycloneDX Maven plugin for SBOM generation to 2.7.10 -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-18964) Update plugin for SBOM generation to 2.7.10
[ https://issues.apache.org/jira/browse/HADOOP-18964?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ayush Saxena updated HADOOP-18964: -- Summary: Update plugin for SBOM generation to 2.7.10 (was: Update plugin for SBOM generation to 2.7.10 #6235) > Update plugin for SBOM generation to 2.7.10 > --- > > Key: HADOOP-18964 > URL: https://issues.apache.org/jira/browse/HADOOP-18964 > Project: Hadoop Common > Issue Type: Improvement >Reporter: Vinod Anandan >Assignee: Vinod Anandan >Priority: Major > Labels: pull-request-available > Fix For: 3.4.0 > > > Update the CycloneDX Maven plugin for SBOM generation to 2.7.10 -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-18973) Clean yum cache after installing which in hadoop2 docker image
[
https://issues.apache.org/jira/browse/HADOOP-18973?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17786362#comment-17786362
]
Ayush Saxena commented on HADOOP-18973:
---
[~adoroszlai] done!!!
> Clean yum cache after installing which in hadoop2 docker image
> --
>
> Key: HADOOP-18973
> URL: https://issues.apache.org/jira/browse/HADOOP-18973
> Project: Hadoop Common
> Issue Type: Improvement
>Reporter: Attila Doroszlai
>Assignee: Attila Doroszlai
>Priority: Minor
> Labels: pull-request-available
>
> Minor addendum to HADOOP-18970: clean yum cache after installing {{which}} to
> reduce image size by ~230MB.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Created] (HADOOP-18974) Replace commons-lang imports with commons-lang3
Ayush Saxena created HADOOP-18974: - Summary: Replace commons-lang imports with commons-lang3 Key: HADOOP-18974 URL: https://issues.apache.org/jira/browse/HADOOP-18974 Project: Hadoop Common Issue Type: Improvement Reporter: Ayush Saxena we have a direct dependency on commons-lang3 but still we are using commons-lang at a couple of places, which is a transitive dependency coming from hbase, solr, and couple of other places. good to replace those with lang3, rather than relying on older transitive dependency -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-18958) UserGroupInformation debug log improve
[ https://issues.apache.org/jira/browse/HADOOP-18958?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ayush Saxena updated HADOOP-18958: -- Fix Version/s: (was: 3.3.4) > UserGroupInformation debug log improve > -- > > Key: HADOOP-18958 > URL: https://issues.apache.org/jira/browse/HADOOP-18958 > Project: Hadoop Common > Issue Type: Improvement > Components: common >Affects Versions: 3.3.5, 3.3.3, 3.3.4 >Reporter: wangzhihui >Priority: Minor > Labels: pull-request-available > Attachments: 20231029-122825-1.jpeg, 20231029-122825.jpeg, > 20231030-143525.jpeg, image-2023-10-29-09-47-56-489.png, > image-2023-10-30-14-35-11-161.png > > Original Estimate: 1h > Remaining Estimate: 1h > > Using “new Exception( )” to print the call stack of "doAs Method " in > the UserGroupInformation class. Using this way will print meaningless > Exception information and too many call stacks, This is not conducive to > troubleshooting > *example:* > !20231029-122825.jpeg|width=991,height=548! > > *improved result* : > > !image-2023-10-29-09-47-56-489.png|width=1099,height=156! > !20231030-143525.jpeg|width=572,height=674! -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Resolved] (HADOOP-18963) Fix typos in .gitignore
[ https://issues.apache.org/jira/browse/HADOOP-18963?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ayush Saxena resolved HADOOP-18963. --- Fix Version/s: 3.4.0 (was: 3.3.6) Hadoop Flags: Reviewed Resolution: Fixed > Fix typos in .gitignore > --- > > Key: HADOOP-18963 > URL: https://issues.apache.org/jira/browse/HADOOP-18963 > Project: Hadoop Common > Issue Type: Improvement > Components: common >Affects Versions: 3.3.6 >Reporter: 袁焊忠 >Assignee: 袁焊忠 >Priority: Major > Labels: pull-request-available > Fix For: 3.4.0 > > > DS_Store is auto generated by Mac in every opened folder, which is useless > but annoying. Not only DS_Store file in the repository root directory should > be ignored but DS_Store file in its subfolders. > -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-18963) Fix typos in .gitignore
[ https://issues.apache.org/jira/browse/HADOOP-18963?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ayush Saxena updated HADOOP-18963: -- Summary: Fix typos in .gitignore (was: Fix typos in .gitignore #6243) > Fix typos in .gitignore > --- > > Key: HADOOP-18963 > URL: https://issues.apache.org/jira/browse/HADOOP-18963 > Project: Hadoop Common > Issue Type: Improvement > Components: common >Affects Versions: 3.3.6 >Reporter: 袁焊忠 >Assignee: 袁焊忠 >Priority: Major > Labels: pull-request-available > Fix For: 3.3.6 > > > DS_Store is auto generated by Mac in every opened folder, which is useless > but annoying. Not only DS_Store file in the repository root directory should > be ignored but DS_Store file in its subfolders. > -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Assigned] (HADOOP-18963) Fix typos in .gitignore #6243
[ https://issues.apache.org/jira/browse/HADOOP-18963?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ayush Saxena reassigned HADOOP-18963: - Assignee: 袁焊忠 > Fix typos in .gitignore #6243 > - > > Key: HADOOP-18963 > URL: https://issues.apache.org/jira/browse/HADOOP-18963 > Project: Hadoop Common > Issue Type: Improvement > Components: common >Affects Versions: 3.3.6 >Reporter: 袁焊忠 >Assignee: 袁焊忠 >Priority: Major > Labels: pull-request-available > Fix For: 3.3.6 > > > DS_Store is auto generated by Mac in every opened folder, which is useless > but annoying. Not only DS_Store file in the repository root directory should > be ignored but DS_Store file in its subfolders. > -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-18963) Fix typos in .gitignore
[ https://issues.apache.org/jira/browse/HADOOP-18963?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17782791#comment-17782791 ] Ayush Saxena commented on HADOOP-18963: --- Committed to trunk. Thanx [~yuanhanzhong666] for the contribution!!! > Fix typos in .gitignore > --- > > Key: HADOOP-18963 > URL: https://issues.apache.org/jira/browse/HADOOP-18963 > Project: Hadoop Common > Issue Type: Improvement > Components: common >Affects Versions: 3.3.6 >Reporter: 袁焊忠 >Assignee: 袁焊忠 >Priority: Major > Labels: pull-request-available > Fix For: 3.3.6 > > > DS_Store is auto generated by Mac in every opened folder, which is useless > but annoying. Not only DS_Store file in the repository root directory should > be ignored but DS_Store file in its subfolders. > -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-18936) Upgrade to jetty 9.4.53
[ https://issues.apache.org/jira/browse/HADOOP-18936?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ayush Saxena updated HADOOP-18936: -- Fix Version/s: 3.3.9 > Upgrade to jetty 9.4.53 > --- > > Key: HADOOP-18936 > URL: https://issues.apache.org/jira/browse/HADOOP-18936 > Project: Hadoop Common > Issue Type: Improvement > Components: build >Reporter: PJ Fanning >Assignee: PJ Fanning >Priority: Major > Labels: pull-request-available > Fix For: 3.4.0, 3.3.9 > > > 2 CVE fixes in > https://github.com/jetty/jetty.project/releases/tag/jetty-9.4.53.v20231009 > 4 more security fixes in > https://github.com/jetty/jetty.project/releases/tag/jetty-9.4.52.v20230823 -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-18917) upgrade to commons-io 2.14.0
[ https://issues.apache.org/jira/browse/HADOOP-18917?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ayush Saxena updated HADOOP-18917: -- Fix Version/s: 3.3.9 > upgrade to commons-io 2.14.0 > > > Key: HADOOP-18917 > URL: https://issues.apache.org/jira/browse/HADOOP-18917 > Project: Hadoop Common > Issue Type: Improvement > Components: build >Reporter: PJ Fanning >Assignee: PJ Fanning >Priority: Major > Labels: pull-request-available > Fix For: 3.4.0, 3.3.9 > > > The release contains some hardening of support in some areas -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-18905) Negative timeout in ZKFailovercontroller due to overflow
[ https://issues.apache.org/jira/browse/HADOOP-18905?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17780698#comment-17780698 ] Ayush Saxena commented on HADOOP-18905: --- Committed to trunk. Thanx [~FuzzingTeam] for the contribution!!! > Negative timeout in ZKFailovercontroller due to overflow > > > Key: HADOOP-18905 > URL: https://issues.apache.org/jira/browse/HADOOP-18905 > Project: Hadoop Common > Issue Type: Bug >Affects Versions: 3.3.6 >Reporter: ConfX >Assignee: ConfX >Priority: Major > Labels: pull-request-available > > Graceful fence timeout of FailoverController in ZKFailovercontroller equals > to `ha.failover-controller.graceful-fence.rpc-timeout.ms` * 2. Since users > are unaware of this calculation, it thus has risks of overflowing to a > negative number if users set > `ha.failover-controller.graceful-fence.rpc-timeout.ms` to a large value. > > To reproduce: > 1. set `ha.failover-controller.graceful-fence.rpc-timeout.ms` to 1092752431 > 2. run `mvn surefire:test > -Dtest=org.apache.hadoop.ha.TestZKFailoverController#testGracefulFailoverFailBecomingStandby` > > We create a PR that provides a fix by checking the timeout after > multiplication is at least 0. > -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Resolved] (HADOOP-18905) Negative timeout in ZKFailovercontroller due to overflow
[ https://issues.apache.org/jira/browse/HADOOP-18905?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ayush Saxena resolved HADOOP-18905. --- Fix Version/s: 3.4.0 Hadoop Flags: Reviewed Resolution: Fixed > Negative timeout in ZKFailovercontroller due to overflow > > > Key: HADOOP-18905 > URL: https://issues.apache.org/jira/browse/HADOOP-18905 > Project: Hadoop Common > Issue Type: Bug >Affects Versions: 3.3.6 >Reporter: ConfX >Assignee: ConfX >Priority: Major > Labels: pull-request-available > Fix For: 3.4.0 > > > Graceful fence timeout of FailoverController in ZKFailovercontroller equals > to `ha.failover-controller.graceful-fence.rpc-timeout.ms` * 2. Since users > are unaware of this calculation, it thus has risks of overflowing to a > negative number if users set > `ha.failover-controller.graceful-fence.rpc-timeout.ms` to a large value. > > To reproduce: > 1. set `ha.failover-controller.graceful-fence.rpc-timeout.ms` to 1092752431 > 2. run `mvn surefire:test > -Dtest=org.apache.hadoop.ha.TestZKFailoverController#testGracefulFailoverFailBecomingStandby` > > We create a PR that provides a fix by checking the timeout after > multiplication is at least 0. > -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Assigned] (HADOOP-18905) Negative timeout in ZKFailovercontroller due to overflow
[ https://issues.apache.org/jira/browse/HADOOP-18905?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ayush Saxena reassigned HADOOP-18905: - Assignee: ConfX > Negative timeout in ZKFailovercontroller due to overflow > > > Key: HADOOP-18905 > URL: https://issues.apache.org/jira/browse/HADOOP-18905 > Project: Hadoop Common > Issue Type: Bug >Affects Versions: 3.3.6 >Reporter: ConfX >Assignee: ConfX >Priority: Major > Labels: pull-request-available > > Graceful fence timeout of FailoverController in ZKFailovercontroller equals > to `ha.failover-controller.graceful-fence.rpc-timeout.ms` * 2. Since users > are unaware of this calculation, it thus has risks of overflowing to a > negative number if users set > `ha.failover-controller.graceful-fence.rpc-timeout.ms` to a large value. > > To reproduce: > 1. set `ha.failover-controller.graceful-fence.rpc-timeout.ms` to 1092752431 > 2. run `mvn surefire:test > -Dtest=org.apache.hadoop.ha.TestZKFailoverController#testGracefulFailoverFailBecomingStandby` > > We create a PR that provides a fix by checking the timeout after > multiplication is at least 0. > -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-18936) Upgrade to jetty 9.4.53
[ https://issues.apache.org/jira/browse/HADOOP-18936?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ayush Saxena updated HADOOP-18936: -- Summary: Upgrade to jetty 9.4.53 (was: upgrade jetty to 9.4.53 due to CVEs) > Upgrade to jetty 9.4.53 > --- > > Key: HADOOP-18936 > URL: https://issues.apache.org/jira/browse/HADOOP-18936 > Project: Hadoop Common > Issue Type: Improvement > Components: build >Reporter: PJ Fanning >Assignee: PJ Fanning >Priority: Major > Labels: pull-request-available > > 2 CVE fixes in > https://github.com/jetty/jetty.project/releases/tag/jetty-9.4.53.v20231009 > 4 more security fixes in > https://github.com/jetty/jetty.project/releases/tag/jetty-9.4.52.v20230823 -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-18936) Upgrade to jetty 9.4.53
[ https://issues.apache.org/jira/browse/HADOOP-18936?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17780696#comment-17780696 ] Ayush Saxena commented on HADOOP-18936: --- Committed to trunk. Thanx [~pj.fanning] for the contribution!!! > Upgrade to jetty 9.4.53 > --- > > Key: HADOOP-18936 > URL: https://issues.apache.org/jira/browse/HADOOP-18936 > Project: Hadoop Common > Issue Type: Improvement > Components: build >Reporter: PJ Fanning >Assignee: PJ Fanning >Priority: Major > Labels: pull-request-available > > 2 CVE fixes in > https://github.com/jetty/jetty.project/releases/tag/jetty-9.4.53.v20231009 > 4 more security fixes in > https://github.com/jetty/jetty.project/releases/tag/jetty-9.4.52.v20230823 -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Resolved] (HADOOP-18936) Upgrade to jetty 9.4.53
[ https://issues.apache.org/jira/browse/HADOOP-18936?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ayush Saxena resolved HADOOP-18936. --- Fix Version/s: 3.4.0 Hadoop Flags: Reviewed Resolution: Fixed > Upgrade to jetty 9.4.53 > --- > > Key: HADOOP-18936 > URL: https://issues.apache.org/jira/browse/HADOOP-18936 > Project: Hadoop Common > Issue Type: Improvement > Components: build >Reporter: PJ Fanning >Assignee: PJ Fanning >Priority: Major > Labels: pull-request-available > Fix For: 3.4.0 > > > 2 CVE fixes in > https://github.com/jetty/jetty.project/releases/tag/jetty-9.4.53.v20231009 > 4 more security fixes in > https://github.com/jetty/jetty.project/releases/tag/jetty-9.4.52.v20230823 -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Assigned] (HADOOP-18936) upgrade jetty to 9.4.53 due to CVEs
[ https://issues.apache.org/jira/browse/HADOOP-18936?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ayush Saxena reassigned HADOOP-18936: - Assignee: PJ Fanning > upgrade jetty to 9.4.53 due to CVEs > --- > > Key: HADOOP-18936 > URL: https://issues.apache.org/jira/browse/HADOOP-18936 > Project: Hadoop Common > Issue Type: Improvement > Components: build >Reporter: PJ Fanning >Assignee: PJ Fanning >Priority: Major > Labels: pull-request-available > > 2 CVE fixes in > https://github.com/jetty/jetty.project/releases/tag/jetty-9.4.53.v20231009 > 4 more security fixes in > https://github.com/jetty/jetty.project/releases/tag/jetty-9.4.52.v20230823 -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Resolved] (HADOOP-18916) module-info classes from external dependencies appearing in uber jars
[ https://issues.apache.org/jira/browse/HADOOP-18916?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ayush Saxena resolved HADOOP-18916. --- Fix Version/s: 3.4.0 3.3.9 Hadoop Flags: Reviewed Resolution: Fixed > module-info classes from external dependencies appearing in uber jars > - > > Key: HADOOP-18916 > URL: https://issues.apache.org/jira/browse/HADOOP-18916 > Project: Hadoop Common > Issue Type: Improvement > Components: build >Reporter: PJ Fanning >Assignee: PJ Fanning >Priority: Major > Labels: pull-request-available > Fix For: 3.4.0, 3.3.9 > > > hadoop-client-minicluster and hadoop-client-runtime try unsuccessfully to > exclude module-info classes from dependencies. Over time, more and more jars > have these classes. > The module-info classes are causing issue with CI builds. Builds can fail if > there are more than module-inf class that is not excluded. > It seems better to exclude them all, especially since they will be affected > by shading anyway. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-18916) module-info classes from external dependencies appearing in uber jars
[ https://issues.apache.org/jira/browse/HADOOP-18916?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17775164#comment-17775164 ] Ayush Saxena commented on HADOOP-18916: --- Committed to trunk & branch-3.3. Thanx [~pj.fanning] for the contribution!!! > module-info classes from external dependencies appearing in uber jars > - > > Key: HADOOP-18916 > URL: https://issues.apache.org/jira/browse/HADOOP-18916 > Project: Hadoop Common > Issue Type: Improvement > Components: build >Reporter: PJ Fanning >Assignee: PJ Fanning >Priority: Major > Labels: pull-request-available > > hadoop-client-minicluster and hadoop-client-runtime try unsuccessfully to > exclude module-info classes from dependencies. Over time, more and more jars > have these classes. > The module-info classes are causing issue with CI builds. Builds can fail if > there are more than module-inf class that is not excluded. > It seems better to exclude them all, especially since they will be affected > by shading anyway. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Assigned] (HADOOP-18916) module-info classes from external dependencies appearing in uber jars
[ https://issues.apache.org/jira/browse/HADOOP-18916?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ayush Saxena reassigned HADOOP-18916: - Assignee: PJ Fanning > module-info classes from external dependencies appearing in uber jars > - > > Key: HADOOP-18916 > URL: https://issues.apache.org/jira/browse/HADOOP-18916 > Project: Hadoop Common > Issue Type: Improvement > Components: build >Reporter: PJ Fanning >Assignee: PJ Fanning >Priority: Major > Labels: pull-request-available > > hadoop-client-minicluster and hadoop-client-runtime try unsuccessfully to > exclude module-info classes from dependencies. Over time, more and more jars > have these classes. > The module-info classes are causing issue with CI builds. Builds can fail if > there are more than module-inf class that is not excluded. > It seems better to exclude them all, especially since they will be affected > by shading anyway. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Created] (HADOOP-18934) Add documentation for building hadoop with docker
Ayush Saxena created HADOOP-18934: - Summary: Add documentation for building hadoop with docker Key: HADOOP-18934 URL: https://issues.apache.org/jira/browse/HADOOP-18934 Project: Hadoop Common Issue Type: Improvement Reporter: Ayush Saxena We have documentation for Single Node setup & stuff, maybe good if we can add documentation to get started via docker: https://hub.docker.com/r/apache/hadoop -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Comment Edited] (HADOOP-18929) Build failure while trying to create apache 3.3.7 release locally.
[
https://issues.apache.org/jira/browse/HADOOP-18929?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17773746#comment-17773746
]
Ayush Saxena edited comment on HADOOP-18929 at 10/10/23 4:22 PM:
-
Hi [~mthakur]
I think it is due to HADOOP-18895
The build failed in the branch-3.3 PR here:
[https://github.com/apache/hadoop/pull/6073#issuecomment-1721005715] it had -1
for mvn install & shadedclient
Can you revert that & try, it works for me.
cc. [~ste...@apache.org]
was (Author: ayushtkn):
Hi [~mthakur]
I think it is due to HADOOP-
The build failed in the branch-3.3 PR here:
https://github.com/apache/hadoop/pull/6073#issuecomment-1721005715 it had -1
for mvn install & shadedclient
Can you revert that & try, it works for me.
cc. [~ste...@apache.org]
> Build failure while trying to create apache 3.3.7 release locally.
> --
>
> Key: HADOOP-18929
> URL: https://issues.apache.org/jira/browse/HADOOP-18929
> Project: Hadoop Common
> Issue Type: Bug
>Affects Versions: 3.3.6
>Reporter: Mukund Thakur
>Priority: Critical
>
> {noformat}
> [ESC[1;34mINFOESC[m] ESC[1m---<
> ESC[0;36morg.apache.hadoop:hadoop-client-check-test-invariantsESC[0;1m
> >ESC[m
> [ESC[1;34mINFOESC[m] ESC[1mBuilding Apache Hadoop Client Packaging Invariants
> for Test 3.3.9-SNAPSHOT [105/111]ESC[m
> [ESC[1;34mINFOESC[m] ESC[1m[ pom
> ]-ESC[m
> [ESC[1;34mINFOESC[m]
> [ESC[1;34mINFOESC[m] ESC[1m---
> ESC[0;32mmaven-enforcer-plugin:3.0.0-M1:enforceESC[m
> ESC[1m(enforce-banned-dependencies)ESC[m @
> ESC[36mhadoop-client-check-test-invariantsESC[0;1m ---ESC[m
> [ESC[1;34mINFOESC[m] Adding ignorable dependency:
> org.apache.hadoop:hadoop-annotations:null
> [ESC[1;34mINFOESC[m] Adding ignore: *
> [ESC[1;33mWARNINGESC[m] Rule 1:
> org.apache.maven.plugins.enforcer.BanDuplicateClasses failed with message:
> Duplicate classes found:
> Found in:
> org.apache.hadoop:hadoop-client-minicluster:jar:3.3.9-SNAPSHOT:compile
> org.apache.hadoop:hadoop-client-runtime:jar:3.3.9-SNAPSHOT:compile
> Duplicate classes:
> META-INF/versions/9/module-info.class
> {noformat}
> CC [~ste...@apache.org] [~weichu]
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-18929) Build failure while trying to create apache 3.3.7 release locally.
[
https://issues.apache.org/jira/browse/HADOOP-18929?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17773746#comment-17773746
]
Ayush Saxena commented on HADOOP-18929:
---
Hi [~mthakur]
I think it is due to HADOOP-
The build failed in the branch-3.3 PR here:
https://github.com/apache/hadoop/pull/6073#issuecomment-1721005715 it had -1
for mvn install & shadedclient
Can you revert that & try, it works for me.
cc. [~ste...@apache.org]
> Build failure while trying to create apache 3.3.7 release locally.
> --
>
> Key: HADOOP-18929
> URL: https://issues.apache.org/jira/browse/HADOOP-18929
> Project: Hadoop Common
> Issue Type: Bug
>Affects Versions: 3.3.6
>Reporter: Mukund Thakur
>Priority: Critical
>
> {noformat}
> [ESC[1;34mINFOESC[m] ESC[1m---<
> ESC[0;36morg.apache.hadoop:hadoop-client-check-test-invariantsESC[0;1m
> >ESC[m
> [ESC[1;34mINFOESC[m] ESC[1mBuilding Apache Hadoop Client Packaging Invariants
> for Test 3.3.9-SNAPSHOT [105/111]ESC[m
> [ESC[1;34mINFOESC[m] ESC[1m[ pom
> ]-ESC[m
> [ESC[1;34mINFOESC[m]
> [ESC[1;34mINFOESC[m] ESC[1m---
> ESC[0;32mmaven-enforcer-plugin:3.0.0-M1:enforceESC[m
> ESC[1m(enforce-banned-dependencies)ESC[m @
> ESC[36mhadoop-client-check-test-invariantsESC[0;1m ---ESC[m
> [ESC[1;34mINFOESC[m] Adding ignorable dependency:
> org.apache.hadoop:hadoop-annotations:null
> [ESC[1;34mINFOESC[m] Adding ignore: *
> [ESC[1;33mWARNINGESC[m] Rule 1:
> org.apache.maven.plugins.enforcer.BanDuplicateClasses failed with message:
> Duplicate classes found:
> Found in:
> org.apache.hadoop:hadoop-client-minicluster:jar:3.3.9-SNAPSHOT:compile
> org.apache.hadoop:hadoop-client-runtime:jar:3.3.9-SNAPSHOT:compile
> Duplicate classes:
> META-INF/versions/9/module-info.class
> {noformat}
> CC [~ste...@apache.org] [~weichu]
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Resolved] (HADOOP-18923) Switch to SPDX identifier for license name
[ https://issues.apache.org/jira/browse/HADOOP-18923?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ayush Saxena resolved HADOOP-18923. --- Hadoop Flags: Reviewed Resolution: Fixed > Switch to SPDX identifier for license name > -- > > Key: HADOOP-18923 > URL: https://issues.apache.org/jira/browse/HADOOP-18923 > Project: Hadoop Common > Issue Type: Task >Reporter: Colm O hEigeartaigh >Assignee: Colm O hEigeartaigh >Priority: Minor > Labels: pull-request-available > Fix For: 3.4.0, 3.2.5, 3.3.7 > > > [https://maven.apache.org/pom.html#Licenses] > "Using an [SPDX identifier|https://spdx.org/licenses/] as the license name is > recommended." > The Apache parent pom is already using this identifier -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-18923) Switch to SPDX identifier for license name
[ https://issues.apache.org/jira/browse/HADOOP-18923?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17772853#comment-17772853 ] Ayush Saxena commented on HADOOP-18923: --- Committed to trunk, branch-3.3 & 3.2 Thanx [~coheigea] for the contribution!!! > Switch to SPDX identifier for license name > -- > > Key: HADOOP-18923 > URL: https://issues.apache.org/jira/browse/HADOOP-18923 > Project: Hadoop Common > Issue Type: Task >Reporter: Colm O hEigeartaigh >Assignee: Colm O hEigeartaigh >Priority: Minor > Labels: pull-request-available > Fix For: 3.4.0, 3.2.5, 3.3.7 > > > [https://maven.apache.org/pom.html#Licenses] > "Using an [SPDX identifier|https://spdx.org/licenses/] as the license name is > recommended." > The Apache parent pom is already using this identifier -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Resolved] (HADOOP-18917) upgrade to commons-io 2.14.0
[ https://issues.apache.org/jira/browse/HADOOP-18917?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ayush Saxena resolved HADOOP-18917. --- Fix Version/s: 3.4.0 Hadoop Flags: Reviewed Resolution: Fixed > upgrade to commons-io 2.14.0 > > > Key: HADOOP-18917 > URL: https://issues.apache.org/jira/browse/HADOOP-18917 > Project: Hadoop Common > Issue Type: Improvement > Components: build >Reporter: PJ Fanning >Assignee: PJ Fanning >Priority: Major > Labels: pull-request-available > Fix For: 3.4.0 > > > The release contains some hardening of support in some areas -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Assigned] (HADOOP-18917) upgrade to commons-io 2.14.0
[ https://issues.apache.org/jira/browse/HADOOP-18917?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ayush Saxena reassigned HADOOP-18917: - Assignee: PJ Fanning > upgrade to commons-io 2.14.0 > > > Key: HADOOP-18917 > URL: https://issues.apache.org/jira/browse/HADOOP-18917 > Project: Hadoop Common > Issue Type: Improvement > Components: build >Reporter: PJ Fanning >Assignee: PJ Fanning >Priority: Major > Labels: pull-request-available > > The release contains some hardening of support in some areas -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-18917) upgrade to commons-io 2.14.0
[ https://issues.apache.org/jira/browse/HADOOP-18917?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17772372#comment-17772372 ] Ayush Saxena commented on HADOOP-18917: --- Committed to trunk. Thanx [~pj.fanning] for the contribution!!! > upgrade to commons-io 2.14.0 > > > Key: HADOOP-18917 > URL: https://issues.apache.org/jira/browse/HADOOP-18917 > Project: Hadoop Common > Issue Type: Improvement > Components: build >Reporter: PJ Fanning >Assignee: PJ Fanning >Priority: Major > Labels: pull-request-available > > The release contains some hardening of support in some areas -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Resolved] (HADOOP-18912) upgrade snappy-java to 1.1.10.4 due to CVE
[ https://issues.apache.org/jira/browse/HADOOP-18912?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ayush Saxena resolved HADOOP-18912. --- Fix Version/s: 3.4.0 Hadoop Flags: Reviewed Resolution: Fixed > upgrade snappy-java to 1.1.10.4 due to CVE > -- > > Key: HADOOP-18912 > URL: https://issues.apache.org/jira/browse/HADOOP-18912 > Project: Hadoop Common > Issue Type: Bug > Components: build >Reporter: PJ Fanning >Assignee: PJ Fanning >Priority: Major > Labels: pull-request-available > Fix For: 3.4.0 > > > follow up to HADOOP-18782 > https://github.com/xerial/snappy-java/security/advisories/GHSA-55g7-9cwv-5qfv -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-18912) upgrade snappy-java to 1.1.10.4 due to CVE
[ https://issues.apache.org/jira/browse/HADOOP-18912?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17769912#comment-17769912 ] Ayush Saxena commented on HADOOP-18912: --- Committed to trunk. Thanx [~pj.fanning] for the contribution!!! > upgrade snappy-java to 1.1.10.4 due to CVE > -- > > Key: HADOOP-18912 > URL: https://issues.apache.org/jira/browse/HADOOP-18912 > Project: Hadoop Common > Issue Type: Bug > Components: build >Reporter: PJ Fanning >Assignee: PJ Fanning >Priority: Major > Labels: pull-request-available > > follow up to HADOOP-18782 > https://github.com/xerial/snappy-java/security/advisories/GHSA-55g7-9cwv-5qfv -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Assigned] (HADOOP-18912) upgrade snappy-java to 1.1.10.4 due to CVE
[ https://issues.apache.org/jira/browse/HADOOP-18912?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ayush Saxena reassigned HADOOP-18912: - Assignee: PJ Fanning > upgrade snappy-java to 1.1.10.4 due to CVE > -- > > Key: HADOOP-18912 > URL: https://issues.apache.org/jira/browse/HADOOP-18912 > Project: Hadoop Common > Issue Type: Bug > Components: build >Reporter: PJ Fanning >Assignee: PJ Fanning >Priority: Major > Labels: pull-request-available > > follow up to HADOOP-18782 > https://github.com/xerial/snappy-java/security/advisories/GHSA-55g7-9cwv-5qfv -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-16062) Add ability to disable Configuration reload registry
[
https://issues.apache.org/jira/browse/HADOOP-16062?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17767724#comment-17767724
]
Ayush Saxena commented on HADOOP-16062:
---
[~zabetak] found this ticket while investigating some memory issue in Hive & I
see it isn't committed yet, Going through the ticket, I don't find any
objections around this.
[~kgyrtkirk] Will you be able to create a PR for this? Patches don't trigger
jenkins now.
Regarding the patch
{code:java}
+ public static synchronized void setConfigurationRegistryEnabled(boolean
enable) {
+if (enable) {
+ REGISTRY = new WeakHashMap(); {code}
I don't think we should do {{new WeakHashMap();}} only
if {{REGISTRY}} is {{null}} else this will overwrite the map if it was already
enabled.
> Add ability to disable Configuration reload registry
>
>
> Key: HADOOP-16062
> URL: https://issues.apache.org/jira/browse/HADOOP-16062
> Project: Hadoop Common
> Issue Type: Improvement
>Reporter: Zoltan Haindrich
>Assignee: Zoltan Haindrich
>Priority: Major
> Attachments: HADOOP-16062.01.patch, jxray_registry.png
>
>
> In Hive we see that after an extensive amount of usage there are a lot of
> Configuration objects not fully reclaimed because of Configuration's REGISTRY
> weak hashmap.
> https://github.com/apache/hadoop/blob/abde1e1f58d5b699e4b8e460cff68e154738169b/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/conf/Configuration.java#L321
> !jxray_registry.png!
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Resolved] (HADOOP-18823) Add Labeler Github Action.
[ https://issues.apache.org/jira/browse/HADOOP-18823?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ayush Saxena resolved HADOOP-18823. --- Fix Version/s: 3.4.0 Hadoop Flags: Reviewed Resolution: Fixed > Add Labeler Github Action. > -- > > Key: HADOOP-18823 > URL: https://issues.apache.org/jira/browse/HADOOP-18823 > Project: Hadoop Common > Issue Type: Bug > Components: build >Affects Versions: 3.4.0 >Reporter: Ayush Saxena >Priority: Major > Labels: pull-request-available > Fix For: 3.4.0 > > > Add auto github lablel for ease of reviewing: > Pull Request Labeler Github Action Configuration: > https://github.com/marketplace/actions/labeler -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-18823) Add Labeler Github Action.
[ https://issues.apache.org/jira/browse/HADOOP-18823?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17746661#comment-17746661 ] Ayush Saxena commented on HADOOP-18823: --- Committed to trunk. Thanx [~hexiaoqiao] for the review!!! > Add Labeler Github Action. > -- > > Key: HADOOP-18823 > URL: https://issues.apache.org/jira/browse/HADOOP-18823 > Project: Hadoop Common > Issue Type: Bug > Components: build >Affects Versions: 3.4.0 >Reporter: Ayush Saxena >Priority: Major > Labels: pull-request-available > > Add auto github lablel for ease of reviewing: > Pull Request Labeler Github Action Configuration: > https://github.com/marketplace/actions/labeler -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Created] (HADOOP-18823) Add Labeler Github Action.
Ayush Saxena created HADOOP-18823: - Summary: Add Labeler Github Action. Key: HADOOP-18823 URL: https://issues.apache.org/jira/browse/HADOOP-18823 Project: Hadoop Common Issue Type: Bug Components: build Affects Versions: 3.4.0 Reporter: Ayush Saxena Add auto github lablel for ease of reviewing: Pull Request Labeler Github Action Configuration: https://github.com/marketplace/actions/labeler -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-18801) Delete path directly when it can not be parsed in trash
[
https://issues.apache.org/jira/browse/HADOOP-18801?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17742307#comment-17742307
]
Ayush Saxena commented on HADOOP-18801:
---
yep, the PR title
> Delete path directly when it can not be parsed in trash
> ---
>
> Key: HADOOP-18801
> URL: https://issues.apache.org/jira/browse/HADOOP-18801
> Project: Hadoop Common
> Issue Type: Improvement
>Reporter: farmmamba
>Assignee: farmmamba
>Priority: Major
> Labels: pull-request-available
>
> If we move path to trash dir directly rather than use delete API or rm
> command, when
> invoke deleteCheckpoint method, it will catch ParseException and ignore
> deleting the path. It will never be deleted, so we should do something to
> prevent or monitor it.
> Some logs are listed below.
>
> {code:java}
> WARN org.apache.hadoop.fs.TrashPolicyDefault: Unexpected item in trash:
> /user/de_eight/.Trash/college_geek_job_recall_als_modelres_5_2_6.del.
> Ignoring.
> WARN org.apache.hadoop.fs.TrashPolicyDefault: Unexpected item in trash:
> /user/de_eight/.Trash/college_geek_addf_vector. Ignoring.
> {code}
>
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-18801) Delete path directly when it can not be parsed in trash
[
https://issues.apache.org/jira/browse/HADOOP-18801?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17742300#comment-17742300
]
Ayush Saxena commented on HADOOP-18801:
---
Thanx [~hexiaoqiao] for the details, I would prefer second option and that
being covered under a config, which stays by default false, to maintain compat.
> Delete path directly when it can not be parsed in trash
> ---
>
> Key: HADOOP-18801
> URL: https://issues.apache.org/jira/browse/HADOOP-18801
> Project: Hadoop Common
> Issue Type: Improvement
>Reporter: farmmamba
>Assignee: farmmamba
>Priority: Major
> Labels: pull-request-available
>
> If we move path to trash dir directly rather than use delete API or rm
> command, when
> invoke deleteCheckpoint method, it will catch ParseException and ignore
> deleting the path. It will never be deleted, so we should do something to
> prevent or monitor it.
> Some logs are listed below.
>
> {code:java}
> WARN org.apache.hadoop.fs.TrashPolicyDefault: Unexpected item in trash:
> /user/de_eight/.Trash/college_geek_job_recall_als_modelres_5_2_6.del.
> Ignoring.
> WARN org.apache.hadoop.fs.TrashPolicyDefault: Unexpected item in trash:
> /user/de_eight/.Trash/college_geek_addf_vector. Ignoring.
> {code}
>
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-18801) Delete path directly when it can not be parsed in trash
[
https://issues.apache.org/jira/browse/HADOOP-18801?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17742268#comment-17742268
]
Ayush Saxena commented on HADOOP-18801:
---
Moved to Hadoop-Common, this isn't just HDFS specific. Out of curiosity why is
some putting something into trash directly? If he wanted it to get deleted
immediately, he could have done skipTrash
> Delete path directly when it can not be parsed in trash
> ---
>
> Key: HADOOP-18801
> URL: https://issues.apache.org/jira/browse/HADOOP-18801
> Project: Hadoop Common
> Issue Type: Improvement
>Reporter: farmmamba
>Assignee: farmmamba
>Priority: Major
> Labels: pull-request-available
>
> If we move path to trash dir directly rather than use delete API or rm
> command, when
> invoke deleteCheckpoint method, it will catch ParseException and ignore
> deleting the path. It will never be deleted, so we should do something to
> prevent or monitor it.
> Some logs are listed below.
>
> {code:java}
> WARN org.apache.hadoop.fs.TrashPolicyDefault: Unexpected item in trash:
> /user/de_eight/.Trash/college_geek_job_recall_als_modelres_5_2_6.del.
> Ignoring.
> WARN org.apache.hadoop.fs.TrashPolicyDefault: Unexpected item in trash:
> /user/de_eight/.Trash/college_geek_addf_vector. Ignoring.
> {code}
>
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Moved] (HADOOP-18801) Delete path directly when it can not be parsed in trash
[
https://issues.apache.org/jira/browse/HADOOP-18801?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ayush Saxena moved HDFS-17046 to HADOOP-18801:
--
Key: HADOOP-18801 (was: HDFS-17046)
Project: Hadoop Common (was: Hadoop HDFS)
> Delete path directly when it can not be parsed in trash
> ---
>
> Key: HADOOP-18801
> URL: https://issues.apache.org/jira/browse/HADOOP-18801
> Project: Hadoop Common
> Issue Type: Improvement
>Reporter: farmmamba
>Assignee: farmmamba
>Priority: Major
> Labels: pull-request-available
>
> If we move path to trash dir directly rather than use delete API or rm
> command, when
> invoke deleteCheckpoint method, it will catch ParseException and ignore
> deleting the path. It will never be deleted, so we should do something to
> prevent or monitor it.
> Some logs are listed below.
>
> {code:java}
> WARN org.apache.hadoop.fs.TrashPolicyDefault: Unexpected item in trash:
> /user/de_eight/.Trash/college_geek_job_recall_als_modelres_5_2_6.del.
> Ignoring.
> WARN org.apache.hadoop.fs.TrashPolicyDefault: Unexpected item in trash:
> /user/de_eight/.Trash/college_geek_addf_vector. Ignoring.
> {code}
>
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Resolved] (HADOOP-18794) ipc.server.handler.queue.size missing from core-default.xml
[ https://issues.apache.org/jira/browse/HADOOP-18794?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ayush Saxena resolved HADOOP-18794. --- Fix Version/s: 3.4.0 Hadoop Flags: Reviewed Resolution: Fixed > ipc.server.handler.queue.size missing from core-default.xml > --- > > Key: HADOOP-18794 > URL: https://issues.apache.org/jira/browse/HADOOP-18794 > Project: Hadoop Common > Issue Type: Improvement > Components: rpc-server >Reporter: WangYuanben >Assignee: WangYuanben >Priority: Major > Labels: pull-request-available > Fix For: 3.4.0 > > > This config should be documented. It is necessary to help users gain a > clearer understanding of the usage of this property. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-18794) ipc.server.handler.queue.size missing from core-default.xml
[ https://issues.apache.org/jira/browse/HADOOP-18794?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17741947#comment-17741947 ] Ayush Saxena commented on HADOOP-18794: --- Committed to trunk. Thanx [~wangyuanben] for the contribution, [~slfan1989] & [~zhtttylzz] for the reviews!!! > ipc.server.handler.queue.size missing from core-default.xml > --- > > Key: HADOOP-18794 > URL: https://issues.apache.org/jira/browse/HADOOP-18794 > Project: Hadoop Common > Issue Type: Improvement > Components: rpc-server >Reporter: WangYuanben >Assignee: WangYuanben >Priority: Major > Labels: pull-request-available > > This config should be documented. It is necessary to help users gain a > clearer understanding of the usage of this property. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Assigned] (HADOOP-18794) ipc.server.handler.queue.size missing from core-default.xml
[ https://issues.apache.org/jira/browse/HADOOP-18794?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ayush Saxena reassigned HADOOP-18794: - Assignee: WangYuanben > ipc.server.handler.queue.size missing from core-default.xml > --- > > Key: HADOOP-18794 > URL: https://issues.apache.org/jira/browse/HADOOP-18794 > Project: Hadoop Common > Issue Type: Improvement > Components: rpc-server >Reporter: WangYuanben >Assignee: WangYuanben >Priority: Major > Labels: pull-request-available > > This config should be documented. It is necessary to help users gain a > clearer understanding of the usage of this property. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-18796) Compilation fails with triple-beam@1.4.0: The engine "node" is incompatible with this module. Expected version ">= 16.0.0". Got "12.22.1"
[
https://issues.apache.org/jira/browse/HADOOP-18796?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17741518#comment-17741518
]
Ayush Saxena commented on HADOOP-18796:
---
this doesn't fail for me locally, unless I go and explicitily put
"triple-beam": "^1.4.0" in the dependency section
[https://github.com/apache/hadoop/blob/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-catalog/hadoop-yarn-applications-catalog-webapp/package.json#L15]
It doesn't seems to be defined explicitly in the code, so may be it randomly
picks. May be we should define this explicitly to be 1.3.0.
from the git log [~iwasakims] did some changes in the past(long long ago),
Masatake do you have any pointers around this, this doesn't repro for me
locally out of the box
> Compilation fails with triple-beam@1.4.0: The engine "node" is incompatible
> with this module. Expected version ">= 16.0.0". Got "12.22.1"
> -
>
> Key: HADOOP-18796
> URL: https://issues.apache.org/jira/browse/HADOOP-18796
> Project: Hadoop Common
> Issue Type: Bug
> Components: build
>Affects Versions: 3.4.0
>Reporter: Ayush Saxena
>Priority: Major
>
> {code:java}
> 2023-07-10T07:32:20.1069856Z [INFO] yarn install v1.22.5
> 2023-07-10T07:32:20.1435049Z [INFO] info No lockfile found.
> 2023-07-10T07:32:20.1489685Z [INFO] [1/4] Resolving packages...
> 2023-07-10T07:32:20.9138642Z [INFO] warning angular-route@1.6.10: For the
> actively supported Angular, see https://www.npmjs.com/package/@angular/core.
> AngularJS support has officially ended. For extended AngularJS support
> options, see https://goo.gle/angularjs-path-forward.
> 2023-07-10T07:32:20.9433934Z [INFO] warning angular@1.6.10: For the actively
> supported Angular, see https://www.npmjs.com/package/@angular/core. AngularJS
> support has officially ended. For extended AngularJS support options, see
> https://goo.gle/angularjs-path-forward.
> 2023-07-10T07:32:21.5051267Z [INFO] [2/4] Fetching packages...
> 2023-07-10T07:32:23.2554282Z [INFO] error triple-beam@1.4.0: The engine
> "node" is incompatible with this module. Expected version ">= 16.0.0". Got
> "12.22.1"
> 2023-07-10T07:32:23.2651719Z [INFO] error Found incompatible module.
> 2023-07-10T07:32:23.2664307Z [INFO] info Visit
> https://yarnpkg.com/en/docs/cli/install for documentation about this command.
> {code}
> Ref:
> https://pipelines.actions.githubusercontent.com/serviceHosts/a3203f99-c5b3-4def-b81d-c5a8ebd0356d/_apis/pipelines/1/runs/294/signedlogcontent/2?urlExpires=2023-07-10T09%3A08%3A57.5185178Z=HMACV1=OHImbS%2FdsTBFKRxwRnV6jFPnumE0GxbB%2BalrVpAgmyw%3D
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Created] (HADOOP-18796) Compilation fails with triple-beam@1.4.0: The engine "node" is incompatible with this module. Expected version ">= 16.0.0". Got "12.22.1"
Ayush Saxena created HADOOP-18796:
-
Summary: Compilation fails with triple-beam@1.4.0: The engine
"node" is incompatible with this module. Expected version ">= 16.0.0". Got
"12.22.1"
Key: HADOOP-18796
URL: https://issues.apache.org/jira/browse/HADOOP-18796
Project: Hadoop Common
Issue Type: Bug
Components: build
Affects Versions: 3.4.0
Reporter: Ayush Saxena
{code:java}
2023-07-10T07:32:20.1069856Z [INFO] yarn install v1.22.5
2023-07-10T07:32:20.1435049Z [INFO] info No lockfile found.
2023-07-10T07:32:20.1489685Z [INFO] [1/4] Resolving packages...
2023-07-10T07:32:20.9138642Z [INFO] warning angular-route@1.6.10: For the
actively supported Angular, see https://www.npmjs.com/package/@angular/core.
AngularJS support has officially ended. For extended AngularJS support options,
see https://goo.gle/angularjs-path-forward.
2023-07-10T07:32:20.9433934Z [INFO] warning angular@1.6.10: For the actively
supported Angular, see https://www.npmjs.com/package/@angular/core. AngularJS
support has officially ended. For extended AngularJS support options, see
https://goo.gle/angularjs-path-forward.
2023-07-10T07:32:21.5051267Z [INFO] [2/4] Fetching packages...
2023-07-10T07:32:23.2554282Z [INFO] error triple-beam@1.4.0: The engine "node"
is incompatible with this module. Expected version ">= 16.0.0". Got "12.22.1"
2023-07-10T07:32:23.2651719Z [INFO] error Found incompatible module.
2023-07-10T07:32:23.2664307Z [INFO] info Visit
https://yarnpkg.com/en/docs/cli/install for documentation about this command.
{code}
Ref:
https://pipelines.actions.githubusercontent.com/serviceHosts/a3203f99-c5b3-4def-b81d-c5a8ebd0356d/_apis/pipelines/1/runs/294/signedlogcontent/2?urlExpires=2023-07-10T09%3A08%3A57.5185178Z=HMACV1=OHImbS%2FdsTBFKRxwRnV6jFPnumE0GxbB%2BalrVpAgmyw%3D
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
