[
https://issues.apache.org/jira/browse/HADOOP-14425?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16015573#comment-16015573
]
Steve Loughran commented on HADOOP-14425:
-----------------------------------------
BTW, dump of stats on a test run without s3guard enabled; committer stats added
{code}
2017-05-18 11:53:05,631 [JUnit-ITestStagingCommitProtocol-testCommitLifecycle]
INFO commit.AbstractITCommitProtocol
(AbstractITCommitProtocol.java:teardown(169)) - Statistics for
s3a://hwdev-steve-ireland-new:
Context = S3AFileSystem
FileSystemId = fc085a79-a825-4edf-9c99-bd1d5afead56-hwdev-steve-ireland-new
fsURI = s3a://hwdev-steve-ireland-new/
files_created = 1
files_copied = 0
files_copied_bytes = 0
files_deleted = 2
fake_directories_deleted = 4
directories_created = 2
directories_deleted = 0
ignored_errors = 3
op_copy_from_local_file = 0
op_exists = 0
op_get_file_status = 11
op_glob_status = 0
op_is_directory = 0
op_is_file = 0
op_list_files = 1
op_list_located_status = 0
op_list_status = 1
op_mkdirs = 1
op_rename = 0
object_copy_requests = 0
object_delete_requests = 3
object_list_requests = 12
object_continue_list_requests = 0
object_metadata_requests = 20
object_multipart_aborted = 0
object_put_bytes = 2897
object_put_requests = 4
object_put_requests_completed = 4
stream_write_failures = 0
stream_write_block_uploads = 0
stream_write_block_uploads_committed = 0
stream_write_block_uploads_aborted = 0
stream_write_total_time = 0
stream_write_total_data = 2857
s3guard_committer_commits_created = 0
s3guard_committer_commits_completed = 1
s3guard_committer_jobs_completed = 1
s3guard_committer_jobs_failed = 0
s3guard_committer_tasks_completed = 1
s3guard_committer_tasks_failed = 0
s3guard_committer_bytes_committed = 1
s3guard_committer_commits_failed = 0
s3guard_committer_commits_aborted = 0
s3guard_committer_commits_reverted = 0
s3guard_metadatastore_put_path_request = 5
s3guard_metadatastore_initialization = 0
object_put_requests_active = 0
object_put_bytes_pending = 0
stream_write_block_uploads_active = 0
stream_write_block_uploads_pending = 1
stream_write_block_uploads_data_pending = 0
S3guard_metadatastore_put_path_latencyNumOps = 4
S3guard_metadatastore_put_path_latency50thPercentileLatency = 5384
S3guard_metadatastore_put_path_latency75thPercentileLatency = 5950
S3guard_metadatastore_put_path_latency90thPercentileLatency = 5950
S3guard_metadatastore_put_path_latency95thPercentileLatency = 5950
S3guard_metadatastore_put_path_latency99thPercentileLatency = 5950
stream_read_fully_operations = 1
stream_opened = 1
stream_bytes_skipped_on_seek = 0
stream_closed = 1
stream_bytes_backwards_on_seek = 0
stream_bytes_read = 40
stream_read_operations_incomplete = 0
stream_bytes_discarded_in_abort = 0
stream_close_operations = 1
stream_read_operations = 1
stream_aborted = 0
stream_forward_seek_operations = 0
stream_backward_seek_operations = 0
stream_seek_operations = 0
stream_bytes_read_in_close = 0
stream_read_exceptions = 0
{code}
> Add more s3guard metrics
> ------------------------
>
> Key: HADOOP-14425
> URL: https://issues.apache.org/jira/browse/HADOOP-14425
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: fs/s3
> Reporter: Ai Deng
>
> The metrics suggested to add:
> Status:
> S3GUARD_METADATASTORE_ENABLED
> S3GUARD_METADATASTORE_IS_AUTHORITATIVE
> Operations:
> S3GUARD_METADATASTORE_INITIALIZATION
> S3GUARD_METADATASTORE_DELETE_PATH
> S3GUARD_METADATASTORE_DELETE_PATH_LATENCY
> S3GUARD_METADATASTORE_DELETE_SUBTREE_PATCH
> S3GUARD_METADATASTORE_GET_PATH
> S3GUARD_METADATASTORE_GET_PATH_LATENCY
> S3GUARD_METADATASTORE_GET_CHILDREN_PATH
> S3GUARD_METADATASTORE_GET_CHILDREN_PATH_LATENCY
> S3GUARD_METADATASTORE_MOVE_PATH
> S3GUARD_METADATASTORE_PUT_PATH
> S3GUARD_METADATASTORE_PUT_PATH_LATENCY
> S3GUARD_METADATASTORE_CLOSE
> S3GUARD_METADATASTORE_DESTORY
> From S3Guard:
> S3GUARD_METADATASTORE_MERGE_DIRECTORY
> For the failures:
> S3GUARD_METADATASTORE_DELETE_FAILURE
> S3GUARD_METADATASTORE_GET_FAILURE
> S3GUARD_METADATASTORE_PUT_FAILURE
> Etc:
> S3GUARD_METADATASTORE_PUT_RETRY_TIMES
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
