Yida Wu has uploaded this change for review. ( http://gerrit.cloudera.org:8080/16078
Change subject: IMPALA-9829: Add metrics for Spilling to S3 ...................................................................... IMPALA-9829: Add metrics for Spilling to S3 Metrics of Spilling to HDFS and S3 are added when the HDFS or S3 path has been passed in scatch_dir, like --scratch_dirs='hdfs://hdfspath, s3a://s3path'. Three types of metrics added in this case: 1. tmp-file-mgr.s3/hdfs.write-latency, unit: ns 2. tmp-file-mgr.s3/hdfs.write-size, unit: Bytes 3. tmp-file-mgr.s3/hdfs.write-io-error It is a mock function for spill to s3, more metrics or test cases might be added later. Also, the remote paths should be verified to be valid later. Limitations: 1. No support for HDFS 'port number' by now, due to the current delimiter ':' may conflict the use of 'port number', the rule of the format might be changed later. 2. Only one HDFS disk and one S3 disk can be supported by now. Change-Id: Ia8d15c691d52af00c51af489cb67386772f3dec4 --- M be/src/runtime/tmp-file-mgr-test.cc M be/src/runtime/tmp-file-mgr.cc M be/src/runtime/tmp-file-mgr.h M be/src/util/hdfs-util.cc M be/src/util/hdfs-util.h M common/thrift/metrics.json 6 files changed, 315 insertions(+), 72 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/78/16078/1 -- To view, visit http://gerrit.cloudera.org:8080/16078 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newchange Gerrit-Change-Id: Ia8d15c691d52af00c51af489cb67386772f3dec4 Gerrit-Change-Number: 16078 Gerrit-PatchSet: 1 Gerrit-Owner: Yida Wu <wydbaggio...@gmail.com>
