Mukund -thank you for running these tests. Both of them are things we've fixed, and in both cases, problems in the tests, not the production code
On Wed, 1 Jul 2020 at 14:22, Mukund Madhav Thakur <mtha...@cloudera.com> wrote: > Compile the distribution using mvn package -Pdist -DskipTests > -Dmaven.javadoc.skip=true -DskipShade and run some hadoop fs commands. All > good there. > > Then I ran the hadoop-aws tests and saw following failures: > > [*ERROR*] *Failures: * > > [*ERROR*] * > ITestS3AMiscOperations.testEmptyFileChecksums:147->Assert.assertEquals:118->Assert.failNotEquals:743->Assert.fail:88 > checksums expected:<etag: "6dd081d9f4abc2fb88fb75f94c84a85f"> but > was:<etag: "aa7c140fc86610c1d0d188acb572036c">* > > [*ERROR*] * > ITestS3AMiscOperations.testNonEmptyFileChecksumsUnencrypted:199->Assert.assertEquals:118->Assert.failNotEquals:743->Assert.fail:88 > checksums expected:<etag: "381e9886ed6117722fa9080e5234202f"> but > was:<etag: "10908d6e1c24362a79a3cd4c5aafb1a1">* > you've got a bucket encrypting things so checksums some back different. We've tweaked those tests so on 3.3 we look @ the bucket and skip the test if there's any default encryption policy https://issues.apache.org/jira/browse/HADOOP-16319 > These were the same failures which I saw in RC0 as well. I think these are > known failures. > > > Apart from that, all of my AssumedRole tests are failing AccessDenied > exception like > > [*ERROR*] > testPartialDeleteSingleDelete(org.apache.hadoop.fs.s3a.auth.ITestAssumeRole) > Time elapsed: 3.359 s <<< ERROR! > > org.apache.hadoop.fs.s3a.AWSServiceIOException: initTable on mthakur-data: > com.amazonaws.services.dynamodbv2.model.AmazonDynamoDBException: User: > arn:aws:sts::152813717728:assumed-role/mthakur-assumed-role/valid is not > authorized to perform: dynamodb:DescribeTable on resource: > arn:aws:dynamodb:ap-south-1:152813717728:table/mthakur-data (Service: > AmazonDynamoDBv2; Status Code: 400; Error Code: AccessDeniedException; > Request ID: UJLKVGJ9I1S9TQF3AEPHVGENVJVV4KQNSO5AEMVJF66Q9ASUAAJG): User: > arn:aws:sts::152813717728:assumed-role/mthakur-assumed-role/valid is not > authorized to perform: dynamodb:DescribeTable on resource: > arn:aws:dynamodb:ap-south-1:152813717728:table/mthakur-data (Service: > AmazonDynamoDBv2; Status Code: 400; Error Code: AccessDeniedException; > Request ID: UJLKVGJ9I1S9TQF3AEPHVGENVJVV4KQNSO5AEMVJF66Q9ASUAAJG) > > at > org.apache.hadoop.fs.s3a.auth.ITestAssumeRole.executePartialDelete(ITestAssumeRole.java:759) > > at > org.apache.hadoop.fs.s3a.auth.ITestAssumeRole.testPartialDeleteSingleDelete(ITestAssumeRole.java:735) > > > I checked my policy and could verify that dynamodb:DescribeTable access is > present there. > > > So just to cross check, I ran the AssumedRole test with the same configs > for apache/trunk and it succeeded. Not sure if this is a false alarm but I > think it would be better if someone else run these AssumedRole tests as > well and verify. > That's https://issues.apache.org/jira/browse/HADOOP-15583 nothing to worry about >>
