Re: Possibility of using ci-hadoop.a.o for Nutch integration tests

2022-01-05 Thread lewis john mcgibbney
Thank you for the information. I agree with you. I totally misunderstood what ci-hadoop.a.o was. Sorry for the noise. lewismc On Wed, Jan 5, 2022 at 21:56 Akira Ajisaka wrote: > (Adding builds@) > > Hi Lewis, > > Nutch is already using ci-builds.apache.org, so I think Nutch can > continue using

Re: Possibility of using ci-hadoop.a.o for Nutch integration tests

2022-01-05 Thread Akira Ajisaka
(Adding builds@) Hi Lewis, Nutch is already using ci-builds.apache.org, so I think Nutch can continue using it. ci-hadoop.apache.org provides almost the same functionality as ci-builds.apache.org and there is no non-production Hadoop cluster running there. Therefore moving to ci-hadoop does not

Re: Possibility of using ci-hadoop.a.o for Nutch integration tests

2022-01-05 Thread lewis john mcgibbney
Thank you for the response and for directing the conversation to the correct places. I may have misunderstood what ci-hadoop.apache.org actually is. We are looking for a non-production Hadoop cluster which we can use to simulate Nutch jobs. I am not sure if this is what ci-hadoop.apache.org

Re: Possibility of using ci-hadoop.a.o for Nutch integration tests

2022-01-05 Thread Lewis John Mcgibbney
Thank you for the response and for directing the conversation to the correct places. I may have misunderstood what ci-hadoop.apache.org actually is. We are looking for a non-production Hadoop cluster which we can use to simulate Nutch jobs. I am not sure if this is what ci-hadoop.apache.org

Re: Possibility of using ci-hadoop.a.o for Nutch integration tests

2022-01-05 Thread Ayush Saxena
Moved to Dev lists. Not sure about this though: when a PR is submitted to Nutch project it will run some MR job in Hadoop CI. Whatever that PR requires should run as part of Nutch Infra. Why in Hadoop CI? Our CI is already loaded with our own workloads. If by any chance the above assertion gets

[jira] [Created] (HADOOP-18071) ABFS: Set driver global timeout for ITestAzureBlobFileSystemBasics

2022-01-05 Thread Sumangala Patki (Jira)
Sumangala Patki created HADOOP-18071: Summary: ABFS: Set driver global timeout for ITestAzureBlobFileSystemBasics Key: HADOOP-18071 URL: https://issues.apache.org/jira/browse/HADOOP-18071

Apache Hadoop qbt Report: branch-3.2+JDK8 on Linux/x86_64

2022-01-05 Thread Apache Jenkins Server
For more details, see https://ci-hadoop.apache.org/job/hadoop-qbt-branch-3.2-java8-linux-x86_64/28/ [Dec 30, 2021 10:06:01 PM] (Chris Nauroth) HADOOP-13464. Upgrade Gson dependency to version 2.8.9 (#2524) [Jan 4, 2022 2:27:35 AM] (Akira Ajisaka) HADOOP-18063. Remove unused import

[jira] [Created] (HADOOP-18070) libhdfspp authentication failed

2022-01-05 Thread wondertx (Jira)
wondertx created HADOOP-18070: - Summary: libhdfspp authentication failed Key: HADOOP-18070 URL: https://issues.apache.org/jira/browse/HADOOP-18070 Project: Hadoop Common Issue Type: Bug

Apache Hadoop qbt Report: trunk+JDK11 on Linux/x86_64

2022-01-05 Thread Apache Jenkins Server
For more details, see https://ci-hadoop.apache.org/job/hadoop-qbt-trunk-java11-linux-x86_64/238/ [Jan 4, 2022 2:18:03 AM] (noreply) HADOOP-18061. Update the year to 2022. (#3845). Contributed by Ayush Saxena. [Jan 4, 2022 2:20:01 AM] (noreply) HDFS-16393. RBF: Fix

Apache Hadoop qbt Report: trunk+JDK8 on Linux/x86_64

2022-01-05 Thread Apache Jenkins Server
For more details, see https://ci-hadoop.apache.org/job/hadoop-qbt-trunk-java8-linux-x86_64/741/ [Jan 4, 2022 2:18:03 AM] (noreply) HADOOP-18061. Update the year to 2022. (#3845). Contributed by Ayush Saxena. [Jan 4, 2022 2:20:01 AM] (noreply) HDFS-16393. RBF: Fix

[jira] [Created] (HADOOP-18069) CVE-2021-0341 in okhttp@2.7.5 detected in hdfs-client

2022-01-05 Thread Eugene Shinn (Truveta) (Jira)
Eugene Shinn (Truveta) created HADOOP-18069: --- Summary: CVE-2021-0341 in okhttp@2.7.5 detected in hdfs-client Key: HADOOP-18069 URL: https://issues.apache.org/jira/browse/HADOOP-18069

[jira] [Resolved] (HADOOP-18056) DistCp: Filter duplicates in the source paths

2022-01-05 Thread Ayush Saxena (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18056?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ayush Saxena resolved HADOOP-18056. --- Fix Version/s: 3.4.0 3.3.3 Hadoop Flags: Reviewed

[jira] [Created] (HADOOP-18068) upgrade AWS SDK to 1.12.x

2022-01-05 Thread Steve Loughran (Jira)
Steve Loughran created HADOOP-18068: --- Summary: upgrade AWS SDK to 1.12.x Key: HADOOP-18068 URL: https://issues.apache.org/jira/browse/HADOOP-18068 Project: Hadoop Common Issue Type:

[jira] [Resolved] (HADOOP-17566) Über-jira: S3A Hadoop 3.3.2 features

2022-01-05 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17566?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran resolved HADOOP-17566. - Fix Version/s: 3.3.2 Resolution: Done > Über-jira: S3A Hadoop 3.3.2 features >

[jira] [Resolved] (HADOOP-15751) AWS Data read stack trace in S3a putObjectDirect

2022-01-05 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-15751?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran resolved HADOOP-15751. - Resolution: Cannot Reproduce not seen for a long time; closing. re-open if it surfaces

[jira] [Resolved] (HADOOP-15955) declare that fs.s3a.ext. is a prefix for arbitrary extensions

2022-01-05 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-15955?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran resolved HADOOP-15955. - Fix Version/s: 3.3.2 Resolution: Done > declare that fs.s3a.ext. is a prefix

[jira] [Resolved] (HADOOP-16370) S3AFileSystem copyFile to propagate etag/version from getObjectMetadata to copy request

2022-01-05 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-16370?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran resolved HADOOP-16370. - Resolution: Duplicate > S3AFileSystem copyFile to propagate etag/version from

[jira] [Resolved] (HADOOP-14070) S3a: Failed to reset the request input stream/make S3A uploadPart() retriable

2022-01-05 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-14070?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran resolved HADOOP-14070. - Resolution: Cannot Reproduce not seen for a long time; closing. re-open if it surfaces

[jira] [Resolved] (HADOOP-16188) s3a rename failed during copy, "Unable to copy part" + 200 error code

2022-01-05 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-16188?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran resolved HADOOP-16188. - Resolution: Cannot Reproduce > s3a rename failed during copy, "Unable to copy part" +

[jira] [Resolved] (HADOOP-17226) Failure of ITestAssumeRole.testRestrictedCommitActions

2022-01-05 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17226?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran resolved HADOOP-17226. - Fix Version/s: 3.3.2 Resolution: Fixed > Failure of

[jira] [Resolved] (HADOOP-17263) transient ITestS3AFileContextStatistics failure -read buffer not filled

2022-01-05 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17263?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran resolved HADOOP-17263. - Resolution: Won't Fix usual transient buffer underrun issue > transient

[jira] [Resolved] (HADOOP-17403) S3A ITestPartialRenamesDeletes.testRenameDirFailsInDelete failure: missing directory marker

2022-01-05 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17403?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran resolved HADOOP-17403. - Resolution: Won't Fix remove s3guard and this whole feature gets cut, so no test

[jira] [Resolved] (HADOOP-17818) ITestS3AContractDistCp failing IllegalStateException: WFOPENSSL0030 Running handshake with buffered unwrapped data

2022-01-05 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17818?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran resolved HADOOP-17818. - Resolution: Cannot Reproduce > ITestS3AContractDistCp failing IllegalStateException:

[jira] [Created] (HADOOP-18067) Über-jira: S3A Hadoop 3.3.3 features

2022-01-05 Thread Steve Loughran (Jira)
Steve Loughran created HADOOP-18067: --- Summary: Über-jira: S3A Hadoop 3.3.3 features Key: HADOOP-18067 URL: https://issues.apache.org/jira/browse/HADOOP-18067 Project: Hadoop Common Issue

[jira] [Created] (HADOOP-18066) AbstractJavaKeyStoreProvider: need a way to read credential store password from Configuration

2022-01-05 Thread Jira
László Bodor created HADOOP-18066: - Summary: AbstractJavaKeyStoreProvider: need a way to read credential store password from Configuration Key: HADOOP-18066 URL: https://issues.apache.org/jira/browse/HADOOP-18066

Apache Hadoop qbt Report: branch-2.10+JDK7 on Linux/x86_64

2022-01-05 Thread Apache Jenkins Server
For more details, see https://ci-hadoop.apache.org/job/hadoop-qbt-branch-2.10-java7-linux-x86_64/533/ [Jan 4, 2022 2:36:27 AM] (Ayush Saxena) HADOOP-18061. Update the year to 2022. (#3845). Contributed by Ayush Saxena. -1 overall The following subsystems voted -1: asflicense compile

[jira] [Created] (HADOOP-18065) ExecutorHelper.logThrowableFromAfterExecute() is too noisy.

2022-01-05 Thread Mukund Thakur (Jira)
Mukund Thakur created HADOOP-18065: -- Summary: ExecutorHelper.logThrowableFromAfterExecute() is too noisy. Key: HADOOP-18065 URL: https://issues.apache.org/jira/browse/HADOOP-18065 Project: Hadoop