[
https://issues.apache.org/jira/browse/IMPALA-6331?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Joe McDonnell resolved IMPALA-6331.
-----------------------------------
Fix Version/s: Not Applicable
Resolution: Cannot Reproduce
This has not reproduced anytime recently. This may have been related to some
tests running on an s3 bucket in a different region.
> slow s3 operations in unit tests
> --------------------------------
>
> Key: IMPALA-6331
> URL: https://issues.apache.org/jira/browse/IMPALA-6331
> Project: IMPALA
> Issue Type: Bug
> Components: Backend, Infrastructure, Perf Investigation
> Affects Versions: Impala 2.10.0
> Reporter: Vuk Ercegovac
> Priority: Major
> Fix For: Not Applicable
>
>
> Seeing some slowness for certain fs tests when run against S3. Since they in
> the hour range, I think its useful to track it down and distinguish between a
> test infrastructure issue vs. an impala/s3 issue.
> Here are logs from a recent run (note the time gaps in the log entries):
> 20:07:37
> query_test/test_hdfs_file_mods.py::TestHdfsFileMods::test_file_modifications[modification_type:
> move_file | exec_option: {'batch_size': 0, 'num_nodes': 0,
> 'disable_codegen_rows_threshold': 0, 'disable_codegen': False,
> 'abort_on_error': 1, 'exec_single_node_rows_threshold': 0} | table_format:
> text/none] PASSED
> 20:08:19
> query_test/test_hdfs_file_mods.py::TestHdfsFileMods::test_file_modifications[modification_type:
> append | exec_option: {'batch_size': 0, 'num_nodes': 0,
> 'disable_codegen_rows_threshold': 0, 'disable_codegen': False,
> 'abort_on_error': 1, 'exec_single_node_rows_threshold': 0} | table_format:
> text/none] PASSED
> 22:59:55
> query_test/test_insert.py::TestInsertQueries::test_insert[exec_option:
> {'sync_ddl': 0, 'batch_size': 0, 'num_nodes': 0,
> 'disable_codegen_rows_threshold': 0, 'disable_codegen': True,
> 'abort_on_error': 1, 'exec_single_node_rows_threshold': 0} | table_format:
> text/none] PASSED
> 03:40:11
> query_test/test_insert.py::TestInsertQueries::test_insert[exec_option:
> {'sync_ddl': 0, 'batch_size': 0, 'num_nodes': 0,
> 'disable_codegen_rows_threshold': 0, 'disable_codegen': False,
> 'abort_on_error': 1, 'exec_single_node_rows_threshold': 0} | table_format:
> text/none] PASSED
> 03:41:12
> query_test/test_insert.py::TestInsertQueries::test_insert_overwrite[exec_option:
> {'sync_ddl': 0, 'batch_size': 0, 'num_nodes': 0,
> 'disable_codegen_rows_threshold': 0, 'disable_codegen': True,
> 'abort_on_error': 1, 'exec_single_node_rows_threshold': 0} | table_format:
> text/none] PASSED
> There are several other tests that take in the 10 minute to minute range.
> Highlighting above the longest ones (> hour).
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
