[
https://issues.apache.org/jira/browse/HADOOP-18915?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17790720#comment-17790720
]
ASF GitHub Bot commented on HADOOP-18915:
-----------------------------------------
steveloughran commented on PR #6180:
URL: https://github.com/apache/hadoop/pull/6180#issuecomment-1830466764
+delete cost tests are wrong, presumably because fs binding is single object
delete only.
would be interesting to think about having the delete cost test skip bulk
delete profile if target fs is set up for single object delete only.
```
[ERROR] Tests run: 8, Failures: 2, Errors: 0, Skipped: 0, Time elapsed:
47.084 s <<< FAILURE! - in
org.apache.hadoop.fs.s3a.performance.ITestS3ADeleteCost
[ERROR]
testDeleteSingleFileInDir[keep-markers](org.apache.hadoop.fs.s3a.performance.ITestS3ADeleteCost)
Time elapsed: 4.244 s <<< FAILURE!
java.lang.AssertionError: operation returning after fs.delete(simpleFile)
action_executor_acquired starting=0 current=0 diff=0, action_file_opened
starting=0 current=0 diff=0, action_http_get_request starting=0 current=0
diff=0, action_http_head_request starting=2 current=3 diff=1,
audit_access_check_failure starting=0 current=0 diff=0, audit_failure
starting=0 current=0 diff=0, audit_request_execution starting=10 current=13
diff=3, audit_span_creation starting=5 current=6 diff=1,
committer_bytes_committed starting=0 current=0 diff=0, committer_bytes_uploaded
starting=0 current=0 diff=0, committer_commit_job starting=0 current=0 diff=0,
committer_commits.failures starting=0 current=0 diff=0,
committer_commits_aborted starting=0 current=0 diff=0,
committer_commits_completed starting=0 current=0 diff=0,
committer_commits_created starting=0 current=0 diff=0,
committer_commits_reverted starting=0 current=0 diff=0,
committer_jobs_completed starting=0 current=0 diff=0, committer_jobs_failed
starting=0 current=0 diff=0, committer_load_single_pending_file starting=0
current=0 diff=0, committer_magic_files_created starting=0 current=0 diff=0,
committer_magic_marker_put starting=0 current=0 diff=0,
committer_materialize_file starting=0 current=0 diff=0,
committer_stage_file_upload starting=0 current=0 diff=0,
committer_tasks_completed starting=0 current=0 diff=0, committer_tasks_failed
starting=0 current=0 diff=0, delegation_tokens_issued starting=0 current=0
diff=0, directories_created starting=2 current=2 diff=0, directories_deleted
starting=0 current=0 diff=0, fake_directories_created starting=0 current=0
diff=0, fake_directories_deleted starting=0 current=0 diff=0, files_copied
starting=0 current=0 diff=0, files_copied_bytes starting=0 current=0 diff=0,
files_created starting=1 current=1 diff=0, files_delete_rejected starting=0
current=0 diff=0, files_deleted starting=0 current=1 diff=1, ignored_errors
starting=0 current=0 diff=0, multipart_instantiated starting=0 current=0
diff=0, multipart_upload_abort_under_path_invoked starting=0 current=0 diff=0,
multipart_upload_aborted starting=0 current=0 diff=0,
multipart_upload_completed starting=0 current=0 diff=0, multipart_upload_list
starting=0 current=0 diff=0, multipart_upload_part_put starting=0 current=0
diff=0, multipart_upload_part_put_bytes starting=0 current=0 diff=0,
multipart_upload_started starting=0 current=0 diff=0,
object_bulk_delete_request starting=0 current=0 diff=0,
object_continue_list_request starting=0 current=0 diff=0, object_copy_requests
starting=0 current=0 diff=0, object_delete_objects starting=0 current=1 diff=1,
object_delete_request starting=0 current=1 diff=1, object_list_request
starting=5 current=6 diff=1, object_metadata_request starting=2 current=3
diff=1, object_multipart_aborted starting=0 current=0 diff=0,
object_multipart_initiated starting=0 current=0 diff=0, object_multipart_list
starting=0 current=0 diff=0, object_put_bytes starting=0 current=0 diff=0,
object_put_request starting=2 current=2 diff=0, object_put_request_completed
starting=3 current=3 diff=0, object_select_requests starting=0 current=0
diff=0, op_abort starting=0 current=0 diff=0, op_access starting=0 current=0
diff=0, op_copy_from_local_file starting=0 current=0 diff=0, op_create
starting=1 current=1 diff=0, op_create_non_recursive starting=0 current=0
diff=0, op_createfile starting=0 current=0 diff=0, op_delete starting=0
current=1 diff=1, op_exists starting=0 current=0 diff=0, op_get_content_summary
starting=0 current=0 diff=0, op_get_delegation_token starting=0 current=0
diff=0, op_get_file_checksum starting=0 current=0 diff=0, op_get_file_status
starting=0 current=0 diff=0, op_glob_status starting=0 current=0 diff=0,
op_hflush starting=0 current=0 diff=0, op_hsync starting=0 current=0 diff=0,
op_is_directory starting=0 current=0 diff=0, op_is_file starting=0 current=0
diff=0, op_list_files starting=0 current=0 diff=0, op_list_located_status
starting=0 current=0 diff=0, op_list_status starting=0 current=0 diff=0,
op_mkdirs starting=2 current=2 diff=0, op_open starting=0 current=0 diff=0,
op_openfile starting=0 current=0 diff=0, op_rename starting=0 current=0 diff=0,
op_xattr_get_map starting=0 current=0 diff=0, op_xattr_get_named starting=0
current=0 diff=0, op_xattr_get_named_map starting=0 current=0 diff=0,
op_xattr_list starting=0 current=0 diff=0, store_exists_probe starting=0
current=0 diff=0, store_io_request starting=10 current=13 diff=3,
store_io_retry starting=0 current=0 diff=0, store_io_throttled starting=0
current=0 diff=0, stream_aborted starting=0 current=0 diff=0,
stream_evict_blocks_from_cache starting=0 current=0 diff=0,
stream_file_cache_eviction starting=0 current=0 diff=0, stream_read_bytes
starting=0 current=0 diff=0, stream_read_bytes_backwards_on_seek starting=0
current=0 diff=0, stream_read_bytes_discarded_in_abort starting=0 current=0
diff=0, stream_read_bytes_discarded_in_close starting=0 current=0 diff=0,
stream_read_close_operations starting=0 current=0 diff=0, stream_read_closed
starting=0 current=0 diff=0, stream_read_exceptions starting=0 current=0
diff=0, stream_read_fully_operations starting=0 current=0 diff=0,
stream_read_opened starting=0 current=0 diff=0, stream_read_operations
starting=0 current=0 diff=0, stream_read_operations_incomplete starting=0
current=0 diff=0, stream_read_remote_stream_aborted starting=0 current=0
diff=0, stream_read_remote_stream_drain starting=0 current=0 diff=0,
stream_read_seek_backward_operations starting=0 current=0 diff=0,
stream_read_seek_bytes_discarded starting=0 current=0 diff=0,
stream_read_seek_bytes_skipped starting=0 current=0 diff=0,
stream_read_seek_forward_operations starting=0 current=0 diff=0,
stream_read_seek_operations starting=0 current=0 diff=0,
stream_read_seek_policy_changed starting=0 current=0 diff=0,
stream_read_total_bytes starting=0 current=0 diff=0, stream_read_unbuffered
starting=0 current=0 diff=0, stream_read_vectored_combined_ranges starting=0
current=0 diff=0, stream_read_vectored_incoming_ranges starting=0 current=0
diff=0, stream_read_vectored_operations starting=0 current=0 diff=0,
stream_read_vectored_read_bytes_discarded starting=0 current=0 diff=0,
stream_read_version_mismatches starting=0 current=0 diff=0,
stream_write_block_uploads starting=0 current=0 diff=0,
stream_write_block_uploads_aborted starting=0 current=0 diff=0,
stream_write_block_uploads_committed starting=0 current=0 diff=0,
stream_write_bytes starting=0 current=0 diff=0, stream_write_exceptions
starting=0 current=0 diff=0, stream_write_exceptions_completing_upload
starting=0 current=0 diff=0, stream_write_queue_duration starting=0 current=0
diff=0, stream_write_total_data starting=0 current=0 diff=0,
stream_write_total_time starting=0 current=0 diff=0: object_delete_request
expected:<2> but was:<1>
at org.junit.Assert.fail(Assert.java:89)
at org.junit.Assert.failNotEquals(Assert.java:835)
at org.junit.Assert.assertEquals(Assert.java:647)
at
org.apache.hadoop.fs.s3a.S3ATestUtils$MetricDiff.assertDiffEquals(S3ATestUtils.java:1054)
at
org.apache.hadoop.fs.s3a.performance.OperationCostValidator$ExpectSingleStatistic.verify(OperationCostValidator.java:418)
at
org.apache.hadoop.fs.s3a.performance.OperationCostValidator.exec(OperationCostValidator.java:177)
at
org.apache.hadoop.fs.s3a.performance.AbstractS3ACostTest.verifyMetrics(AbstractS3ACostTest.java:318)
at
org.apache.hadoop.fs.s3a.performance.ITestS3ADeleteCost.testDeleteSingleFileInDir(ITestS3ADeleteCost.java:116)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at
org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59)
at
org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
at
org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56)
at
org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
at
org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
at
org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
at org.junit.rules.TestWatcher$1.evaluate(TestWatcher.java:61)
at
org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:299)
at
org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:293)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.lang.Thread.run(Thread.java:750)
[ERROR]
testDeleteSingleFileInDir[delete-markers](org.apache.hadoop.fs.s3a.performance.ITestS3ADeleteCost)
Time elapsed: 4.79 s <<< FAILURE!
java.lang.AssertionError: operation returning after fs.delete(simpleFile)
action_executor_acquired starting=0 current=0 diff=0, action_file_opened
starting=0 current=0 diff=0, action_http_get_request starting=0 current=0
diff=0, action_http_head_request starting=2 current=3 diff=1,
audit_access_check_failure starting=0 current=0 diff=0, audit_failure
starting=0 current=0 diff=0, audit_request_execution starting=16 current=22
diff=6, audit_span_creation starting=5 current=6 diff=1,
committer_bytes_committed starting=0 current=0 diff=0, committer_bytes_uploaded
starting=0 current=0 diff=0, committer_commit_job starting=0 current=0 diff=0,
committer_commits.failures starting=0 current=0 diff=0,
committer_commits_aborted starting=0 current=0 diff=0,
committer_commits_completed starting=0 current=0 diff=0,
committer_commits_created starting=0 current=0 diff=0,
committer_commits_reverted starting=0 current=0 diff=0,
committer_jobs_completed starting=0 current=0 diff=0, committer_jobs_failed
starting=0 current=0 diff=0, committer_load_single_pending_file starting=0
current=0 diff=0, committer_magic_files_created starting=0 current=0 diff=0,
committer_magic_marker_put starting=0 current=0 diff=0,
committer_materialize_file starting=0 current=0 diff=0,
committer_stage_file_upload starting=0 current=0 diff=0,
committer_tasks_completed starting=0 current=0 diff=0, committer_tasks_failed
starting=0 current=0 diff=0, delegation_tokens_issued starting=0 current=0
diff=0, directories_created starting=2 current=3 diff=1, directories_deleted
starting=0 current=0 diff=0, fake_directories_created starting=0 current=0
diff=0, fake_directories_deleted starting=6 current=8 diff=2, files_copied
starting=0 current=0 diff=0, files_copied_bytes starting=0 current=0 diff=0,
files_created starting=1 current=1 diff=0, files_delete_rejected starting=0
current=0 diff=0, files_deleted starting=0 current=1 diff=1, ignored_errors
starting=0 current=0 diff=0, multipart_instantiated starting=0 current=0
diff=0, multipart_upload_abort_under_path_invoked starting=0 current=0 diff=0,
multipart_upload_aborted starting=0 current=0 diff=0,
multipart_upload_completed starting=0 current=0 diff=0, multipart_upload_list
starting=0 current=0 diff=0, multipart_upload_part_put starting=0 current=0
diff=0, multipart_upload_part_put_bytes starting=0 current=0 diff=0,
multipart_upload_started starting=0 current=0 diff=0,
object_bulk_delete_request starting=0 current=0 diff=0,
object_continue_list_request starting=0 current=0 diff=0, object_copy_requests
starting=0 current=0 diff=0, object_delete_objects starting=6 current=9 diff=3,
object_delete_request starting=6 current=9 diff=3, object_list_request
starting=5 current=6 diff=1, object_metadata_request starting=2 current=3
diff=1, object_multipart_aborted starting=0 current=0 diff=0,
object_multipart_initiated starting=0 current=0 diff=0, object_multipart_list
starting=0 current=0 diff=0, object_put_bytes starting=0 current=0 diff=0,
object_put_request starting=2 current=3 diff=1, object_put_request_completed
starting=3 current=4 diff=1, object_select_requests starting=0 current=0
diff=0, op_abort starting=0 current=0 diff=0, op_access starting=0 current=0
diff=0, op_copy_from_local_file starting=0 current=0 diff=0, op_create
starting=1 current=1 diff=0, op_create_non_recursive starting=0 current=0
diff=0, op_createfile starting=0 current=0 diff=0, op_delete starting=0
current=1 diff=1, op_exists starting=0 current=0 diff=0, op_get_content_summary
starting=0 current=0 diff=0, op_get_delegation_token starting=0 current=0
diff=0, op_get_file_checksum starting=0 current=0 diff=0, op_get_file_status
starting=0 current=0 diff=0, op_glob_status starting=0 current=0 diff=0,
op_hflush starting=0 current=0 diff=0, op_hsync starting=0 current=0 diff=0,
op_is_directory starting=0 current=0 diff=0, op_is_file starting=0 current=0
diff=0, op_list_files starting=0 current=0 diff=0, op_list_located_status
starting=0 current=0 diff=0, op_list_status starting=0 current=0 diff=0,
op_mkdirs starting=2 current=2 diff=0, op_open starting=0 current=0 diff=0,
op_openfile starting=0 current=0 diff=0, op_rename starting=0 current=0 diff=0,
op_xattr_get_map starting=0 current=0 diff=0, op_xattr_get_named starting=0
current=0 diff=0, op_xattr_get_named_map starting=0 current=0 diff=0,
op_xattr_list starting=0 current=0 diff=0, store_exists_probe starting=0
current=0 diff=0, store_io_request starting=16 current=22 diff=6,
store_io_retry starting=0 current=0 diff=0, store_io_throttled starting=0
current=0 diff=0, stream_aborted starting=0 current=0 diff=0,
stream_evict_blocks_from_cache starting=0 current=0 diff=0,
stream_file_cache_eviction starting=0 current=0 diff=0, stream_read_bytes
starting=0 current=0 diff=0, stream_read_bytes_backwards_on_seek starting=0
current=0 diff=0, stream_read_bytes_discarded_in_abort starting=0 current=0
diff=0, stream_read_bytes_discarded_in_close starting=0 current=0 diff=0,
stream_read_close_operations starting=0 current=0 diff=0, stream_read_closed
starting=0 current=0 diff=0, stream_read_exceptions starting=0 current=0
diff=0, stream_read_fully_operations starting=0 current=0 diff=0,
stream_read_opened starting=0 current=0 diff=0, stream_read_operations
starting=0 current=0 diff=0, stream_read_operations_incomplete starting=0
current=0 diff=0, stream_read_remote_stream_aborted starting=0 current=0
diff=0, stream_read_remote_stream_drain starting=0 current=0 diff=0,
stream_read_seek_backward_operations starting=0 current=0 diff=0,
stream_read_seek_bytes_discarded starting=0 current=0 diff=0,
stream_read_seek_bytes_skipped starting=0 current=0 diff=0,
stream_read_seek_forward_operations starting=0 current=0 diff=0,
stream_read_seek_operations starting=0 current=0 diff=0,
stream_read_seek_policy_changed starting=0 current=0 diff=0,
stream_read_total_bytes starting=0 current=0 diff=0, stream_read_unbuffered
starting=0 current=0 diff=0, stream_read_vectored_combined_ranges starting=0
current=0 diff=0, stream_read_vectored_incoming_ranges starting=0 current=0
diff=0, stream_read_vectored_operations starting=0 current=0 diff=0,
stream_read_vectored_read_bytes_discarded starting=0 current=0 diff=0,
stream_read_version_mismatches starting=0 current=0 diff=0,
stream_write_block_uploads starting=0 current=0 diff=0,
stream_write_block_uploads_aborted starting=0 current=0 diff=0,
stream_write_block_uploads_committed starting=0 current=0 diff=0,
stream_write_bytes starting=0 current=0 diff=0, stream_write_exceptions
starting=0 current=0 diff=0, stream_write_exceptions_completing_upload
starting=0 current=0 diff=0, stream_write_queue_duration starting=0 current=0
diff=0, stream_write_total_data starting=0 current=0 diff=0,
stream_write_total_time starting=0 current=0 diff=0: object_delete_request
expected:<2> but was:<3>
at org.junit.Assert.fail(Assert.java:89)
at org.junit.Assert.failNotEquals(Assert.java:835)
at org.junit.Assert.assertEquals(Assert.java:647)
at
org.apache.hadoop.fs.s3a.S3ATestUtils$MetricDiff.assertDiffEquals(S3ATestUtils.java:1054)
at
org.apache.hadoop.fs.s3a.performance.OperationCostValidator$ExpectSingleStatistic.verify(OperationCostValidator.java:418)
at
org.apache.hadoop.fs.s3a.performance.OperationCostValidator.exec(OperationCostValidator.java:177)
at
org.apache.hadoop.fs.s3a.performance.AbstractS3ACostTest.verifyMetrics(AbstractS3ACostTest.java:318)
at
org.apache.hadoop.fs.s3a.performance.ITestS3ADeleteCost.testDeleteSingleFileInDir(ITestS3ADeleteCost.java:116)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at
org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59)
at
org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
at
org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56)
at
org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
at
org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
at
org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
at org.junit.rules.TestWatcher$1.evaluate(TestWatcher.java:61)
at
org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:299)
at
org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:293)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.lang.Thread.run(Thread.java:750)
```
> Extend S3A http client connection timeouts
> ------------------------------------------
>
> Key: HADOOP-18915
> URL: https://issues.apache.org/jira/browse/HADOOP-18915
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: fs/s3
> Affects Versions: 3.4.0
> Reporter: Ahmar Suhail
> Assignee: Steve Loughran
> Priority: Major
> Labels: pull-request-available
>
> * Add ability to configure *all* timeouts, especially acquisition time
> * recognise ApiCallTimeout and map tp a retryable exception
> * use getDuration so suffixes can be used -so remove all ambiguity about time
> unit
> * use units in core-default.xml so warnings aren't printed
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
