steveloughran commented on issue #1820: HADOOP-16830. Add public IOStatistics API + S3A implementation URL: https://github.com/apache/hadoop/pull/1820#issuecomment-613469465 * tweak of the EmptyIOStatistics. We do need tests of the dynamic stuff next * S3A stats -findbugs changed to turn off complaints about volatile the change to the builder API needed to wire up AWS stats collection is broken regarding region signing calculation: ``` [ERROR] testSelectNothing(org.apache.hadoop.fs.s3a.select.ITestS3SelectCLI) Time elapsed: 2.059 s <<< ERROR! org.apache.hadoop.fs.s3a.AWSBadRequestException: getFileStatus on s3a://landsat-pds/scene_list.gz: com.amazonaws.services.s3.model.AmazonS3Exception: Bad Request (Service: Amazon S3; Status Code: 400; Error Code: 400 Bad Request; Request ID: 9BDEA365F9EE00C9; S3 Extended Request ID: oeSi4G7orAk2b3ogGQp9ydQf6a0ktIuQPFKKQ+tox6OngBliI6NNPcZTIPLI37U21ffivz/HDw8=), S3 Extended Request ID: oeSi4G7orAk2b3ogGQp9ydQf6a0ktIuQPFKKQ+tox6OngBliI6NNPcZTIPLI37U21ffivz/HDw8=:400 Bad Request: Bad Request (Service: Amazon S3; Status Code: 400; Error Code: 400 Bad Request; Request ID: 9BDEA365F9EE00C9; S3 Extended Request ID: oeSi4G7orAk2b3ogGQp9ydQf6a0ktIuQPFKKQ+tox6OngBliI6NNPcZTIPLI37U21ffivz/HDw8=) at org.apache.hadoop.fs.s3a.S3AUtils.translateException(S3AUtils.java:241) at org.apache.hadoop.fs.s3a.S3AUtils.translateException(S3AUtils.java:168) at org.apache.hadoop.fs.s3a.S3AFileSystem.s3GetFileStatus(S3AFileSystem.java:3014) at org.apache.hadoop.fs.s3a.S3AFileSystem.innerGetFileStatus(S3AFileSystem.java:2932) at org.apache.hadoop.fs.s3a.S3AFileSystem.extractOrFetchSimpleFileStatus(S3AFileSystem.java:4636) at org.apache.hadoop.fs.s3a.S3AFileSystem.select(S3AFileSystem.java:4562) at org.apache.hadoop.fs.s3a.S3AFileSystem.lambda$null$24(S3AFileSystem.java:4719) at org.apache.hadoop.util.LambdaUtils.eval(LambdaUtils.java:52) at org.apache.hadoop.fs.s3a.S3AFileSystem.lambda$openFileWithOptions$25(S3AFileSystem.java:4718) at java.util.concurrent.FutureTask.run(FutureTask.java:266) ``` need to understand more; may want to do an SDK updated too
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
