Steve Loughran created HADOOP-16133: ---------------------------------------
Summary: S3A statistic collection underrecords bytes written in helper threads Key: HADOOP-16133 URL: https://issues.apache.org/jira/browse/HADOOP-16133 Project: Hadoop Common Issue Type: Sub-task Components: fs/s3 Affects Versions: 3.1.2 Reporter: Steve Loughran Applications collecting per-thread statistics from S3A get underreporting of bytes written, as all byte written in the worker call update those in a different thread. Proposed: * the bytes upload statistics are uploaded in the primary thread as a block is queued for write, not after in the completion phase in the other thread * final {{WriteOperationsHelper.writeSuccessful()}} takes the final statistics for its own entertainment Really I want context-specific storage statistics. -- This message was sent by Atlassian JIRA (v7.6.3#76005) --------------------------------------------------------------------- To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-dev-h...@hadoop.apache.org