[
https://issues.apache.org/jira/browse/HADOOP-14425?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16024676#comment-16024676
]
Steve Loughran commented on HADOOP-14425:
-----------------------------------------
Latest dump of stats from a committer test run. Note how the latency values are
in a different case from the others, including the s3guard_ props. They should
be downcased
{code}
2017-05-25 12:24:08,466 [ScalaTest-main-running-S3ACommitDataframeSuite] INFO
s3.S3AOperations (CloudLogging.scala:logInfo(56)) -
S3guard_metadatastore_put_path_latency50thPercentileLatency = 0
S3guard_metadatastore_put_path_latency75thPercentileLatency = 0
S3guard_metadatastore_put_path_latency90thPercentileLatency = 0
S3guard_metadatastore_put_path_latency95thPercentileLatency = 0
S3guard_metadatastore_put_path_latency99thPercentileLatency = 0
S3guard_metadatastore_put_path_latencyNumOps = 0
committer_bytes_committed = 314
committer_commits_aborted = 0
committer_commits_completed = 1
committer_commits_created = 1
committer_commits_failed = 0
committer_commits_reverted = 0
committer_jobs_completed = 1
committer_jobs_failed = 0
committer_tasks_completed = 1
committer_tasks_failed = 0
directories_created = 0
directories_deleted = 0
fake_directories_deleted = 4
files_copied = 0
files_copied_bytes = 0
files_created = 0
files_deleted = 2
ignored_errors = 1
object_continue_list_requests = 0
object_copy_requests = 0
object_delete_requests = 2
object_list_requests = 5
object_metadata_requests = 8
object_multipart_aborted = 0
object_put_bytes = 314
object_put_bytes_pending = 0
object_put_requests = 1
object_put_requests_active = 0
object_put_requests_completed = 1
op_copy_from_local_file = 0
op_exists = 1
op_get_file_status = 3
op_glob_status = 0
op_is_directory = 0
op_is_file = 0
op_list_files = 0
op_list_located_status = 0
op_list_status = 0
op_mkdirs = 0
op_rename = 0
s3guard_metadatastore_initialization = 0
s3guard_metadatastore_put_path_request = 1
stream_aborted = 0
stream_backward_seek_operations = 0
stream_bytes_backwards_on_seek = 0
stream_bytes_discarded_in_abort = 0
stream_bytes_read = 0
stream_bytes_read_in_close = 0
stream_bytes_skipped_on_seek = 0
stream_close_operations = 0
stream_closed = 0
stream_forward_seek_operations = 0
stream_opened = 0
stream_read_exceptions = 0
stream_read_fully_operations = 0
stream_read_operations = 0
stream_read_operations_incomplete = 0
stream_seek_operations = 0
stream_write_block_uploads = 0
stream_write_block_uploads_aborted = 0
stream_write_block_uploads_active = 0
stream_write_block_uploads_committed = 0
stream_write_block_uploads_data_pending = 0
stream_write_block_uploads_pending = 0
stream_write_failures = 0
stream_write_total_data = 0
stream_write_total_time = 0
{code}
> Add more s3guard metrics
> ------------------------
>
> Key: HADOOP-14425
> URL: https://issues.apache.org/jira/browse/HADOOP-14425
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: fs/s3
> Reporter: Ai Deng
>
> The metrics suggested to add:
> Status:
> S3GUARD_METADATASTORE_ENABLED
> S3GUARD_METADATASTORE_IS_AUTHORITATIVE
> Operations:
> S3GUARD_METADATASTORE_INITIALIZATION
> S3GUARD_METADATASTORE_DELETE_PATH
> S3GUARD_METADATASTORE_DELETE_PATH_LATENCY
> S3GUARD_METADATASTORE_DELETE_SUBTREE_PATCH
> S3GUARD_METADATASTORE_GET_PATH
> S3GUARD_METADATASTORE_GET_PATH_LATENCY
> S3GUARD_METADATASTORE_GET_CHILDREN_PATH
> S3GUARD_METADATASTORE_GET_CHILDREN_PATH_LATENCY
> S3GUARD_METADATASTORE_MOVE_PATH
> S3GUARD_METADATASTORE_PUT_PATH
> S3GUARD_METADATASTORE_PUT_PATH_LATENCY
> S3GUARD_METADATASTORE_CLOSE
> S3GUARD_METADATASTORE_DESTORY
> From S3Guard:
> S3GUARD_METADATASTORE_MERGE_DIRECTORY
> For the failures:
> S3GUARD_METADATASTORE_DELETE_FAILURE
> S3GUARD_METADATASTORE_GET_FAILURE
> S3GUARD_METADATASTORE_PUT_FAILURE
> Etc:
> S3GUARD_METADATASTORE_PUT_RETRY_TIMES
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
