I compiled and ran s3 tests using mvn clean verify -Ds3guard -Ddynamo -Dauth. I see some failures. I ran these separately as well but it still fails for me.
Also I am trying to run the abfs tests but all tests are getting skipped even if I think I have right auth-keys.xml. I will debug this and update. [*ERROR*] *Failures: * [*ERROR*] * ITestS3AMiscOperations.testEmptyFileChecksums:147->Assert.assertEquals:118->Assert.failNotEquals:743->Assert.fail:88 checksums expected:<etag: "4eb50cc3151f8d174a2ff79a7c16277e"> but was:<etag: "cb190410cc285024b2066360c44c006a">* [*ERROR*] * ITestS3AMiscOperations.testNonEmptyFileChecksumsUnencrypted:199->Assert.assertEquals:118->Assert.failNotEquals:743->Assert.fail:88 checksums expected:<etag: "90d890f880e286b6d62534dfde5ce720"> but was:<etag: "e9181a730651153656d3570226667ef0">* [*INFO*] [*ERROR*] *Tests run: 12, Failures: 2, Errors: 0, Skipped: 0* [*ERROR*] *Errors: * [*ERROR*] * ITestS3GuardToolDynamoDB.testDynamoDBInitDestroyCycle:224->AbstractS3GuardToolTestBase.exec:286->AbstractS3GuardToolTestBase.exec:308 ยป AWSServiceIO* On Tue, May 5, 2020 at 11:48 PM Steve Loughran <ste...@cloudera.com.invalid> wrote: > mvn -T 1 -Phadoop-3.2 -Dhadoop.version=3.1.4 -Psnapshots-and-staging > -Phadoop-cloud,yarn,kinesis-asl,yarn clean install -DskipTests > > Then a test run of the cloud bits > > mvn -T 1 -Phadoop-3.2 -Dhadoop.version=3.1.4 -Psnapshots-and-staging > -Phadoop-cloud,yarn,kinesis-asl test --pl hadoop-cloud > > And I got a guava binding stack trace (joy!) > > > CommitterBindingSuite: > *** RUN ABORTED *** > java.lang.NoSuchMethodError: > > com.google.common.base.Preconditions.checkArgument(ZLjava/lang/String;Ljava/lang/Object;)V > at org.apache.hadoop.conf.Configuration.set(Configuration.java:1357) > at org.apache.hadoop.conf.Configuration.set(Configuration.java:1338) > at > org.apache.spark.internal.io > .cloud.CommitterBindingSuite.newJob(CommitterBindingSuite.scala:89) > at > org.apache.spark.internal.io > .cloud.CommitterBindingSuite.$anonfun$new$1(CommitterBindingSuite.scala:55) > at org.scalatest.OutcomeOf.outcomeOf(OutcomeOf.scala:85) > at org.scalatest.OutcomeOf.outcomeOf$(OutcomeOf.scala:83) > at org.scalatest.OutcomeOf$.outcomeOf(OutcomeOf.scala:104) > at org.scalatest.Transformer.apply(Transformer.scala:22) > at org.scalatest.Transformer.apply(Transformer.scala:20) > at org.scalatest.FunSuiteLike$$anon$1.apply(FunSuiteLike.scala:186) > ... > > Fix: spark's guava version needs to be bumped up. There's a bigger patch > for that, but I've got a minor PR which lets someone change it on the maven > CLI to match the hadoop version, e.g. -Dguava.version=27.0-jre > > https://issues.apache.org/jira/browse/SPARK-31644 > > There's nothing which can be done except highlight to all that this has > happened and what that stack trace means. > > On Tue, 5 May 2020 at 02:08, Wei-Chiu Chuang <weic...@cloudera.com.invalid > > > wrote: > > > Gabor, I'm sorry there's a test failure in branch-3.1 HDFS-14599 > > <https://issues.apache.org/jira/browse/HDFS-14599> > > > > 1. > > 1. I just cherrypicked the fix to branch-3.2 branch-3.1. It's a > test > > only fix so technically I could live with it. But it would be best > > to add > > the fix to 3.1.4 as well. > > > > > > On Mon, May 4, 2020 at 3:20 PM Gabor Bota <gab...@apache.org> wrote: > > > > > Hi folks, > > > > > > I have put together a release candidate (RC0) for Hadoop 3.1.4. > > > > > > The RC is available at: > > http://people.apache.org/~gabota/hadoop-3.1.4-RC0/ > > > The RC tag in git is here: > > > https://github.com/apache/hadoop/releases/tag/release-3.1.4-RC0 > > > The maven artifacts are staged at > > > > https://repository.apache.org/content/repositories/orgapachehadoop-1266/ > > > > > > You can find my public key at: > > > https://dist.apache.org/repos/dist/release/hadoop/common/KEYS > > > and http://keys.gnupg.net/pks/lookup?op=get&search=0xB86249D83539B38C > > > > > > Please try the release and vote. The vote will run for 5 weekdays, > > > until May 11. 2020. 23:00 CET. > > > > > > Thanks, > > > Gabor > > > > > > --------------------------------------------------------------------- > > > To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org > > > For additional commands, e-mail: common-dev-h...@hadoop.apache.org > > > > > > > > >
