[
https://issues.apache.org/jira/browse/IMPALA-12453?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17771215#comment-17771215
]
ASF subversion and git services commented on IMPALA-12453:
----------------------------------------------------------
Commit 782cda44996ac73c0c14cc0727683c9e3ef1fd1a in impala's branch
refs/heads/master from Michael Smith
[ https://gitbox.apache.org/repos/asf?p=impala.git;h=782cda449 ]
IMPALA-12453: Increase yarn high disk threshold
Yarn will mark a node as unavailable for running jobs if it exceeds the
disk threshold. We only have one node, so that means jobs hang until
disk frees up. Exhaustive test runs are approaching using 300GB, which
is a common size for our test environments, and we see some tests
hanging on yarn jobs.
Increases the disk threshold from default of 90% to 99%; in test
environments the remaining 1% is ~3GB. That should still be sufficient
in case yarn needs to throttle workloads to avoid using all the disk,
while leaving most of the disk available.
Change-Id: I06b723e4ecea94ce95fb2bd41bd356fba694e87a
Reviewed-on: http://gerrit.cloudera.org:8080/20501
Reviewed-by: Impala Public Jenkins <[email protected]>
Tested-by: Impala Public Jenkins <[email protected]>
> TestIcebergV2Table::test_delete_partitioned slows down then times out in
> exhaustive mode
> ----------------------------------------------------------------------------------------
>
> Key: IMPALA-12453
> URL: https://issues.apache.org/jira/browse/IMPALA-12453
> Project: IMPALA
> Issue Type: Task
> Affects Versions: Impala 4.3.0
> Reporter: Laszlo Gaal
> Priority: Blocker
> Labels: broken-build
> Fix For: Impala 4.4.0
>
>
> Exhaustive runs slow down, then eventually time out during Iceberg deletion
> tests. First the interval between reported test steps increased from minutes
> to two hours (maybe an internal timeout?), then the build's internal timeout
> for the whole test phase was triggered, which killed the test phase with core
> dumps.
> This is the transition from PASSED to FAILED in the test run:{code}
> 19:44:16
> query_test/test_iceberg.py::TestIcebergV2Table::test_delete_partitioned[protocol:
> beeswax | exec_option: {'test_replan': 1, 'batch_size': 0, 'num_nodes': 0,
> 'disable_codegen_rows_threshold': 0, 'disable_codegen': False,
> 'abort_on_error': 1, 'exec_single_node_rows_threshold': 0} | table_format:
> parquet/none | disable_optimized_iceberg_v2_read: 1]
> 19:46:37 [gw4] PASSED
> query_test/test_iceberg.py::TestIcebergV2Table::test_delete_partitioned[protocol:
> beeswax | exec_option: {'test_replan': 1, 'batch_size': 0, 'num_nodes': 0,
> 'disable_codegen_rows_threshold': 0, 'disable_codegen': False,
> 'abort_on_error': 1, 'exec_single_node_rows_threshold': 0} | table_format:
> parquet/none | disable_optimized_iceberg_v2_read: 1]
> 19:46:37
> query_test/test_iceberg.py::TestIcebergV2Table::test_delete_partitioned[protocol:
> beeswax | exec_option: {'test_replan': 1, 'batch_size': 0, 'num_nodes': 0,
> 'disable_codegen_rows_threshold': 0, 'disable_codegen': False,
> 'abort_on_error': 1, 'exec_single_node_rows_threshold': 0} | table_format:
> parquet/none | disable_optimized_iceberg_v2_read: 0]
> 19:48:18 [gw4] FAILED
> query_test/test_iceberg.py::TestIcebergV2Table::test_delete_partitioned[protocol:
> beeswax | exec_option: {'test_replan': 1, 'batch_size': 0, 'num_nodes': 0,
> 'disable_codegen_rows_threshold': 0, 'disable_codegen': False,
> 'abort_on_error': 1, 'exec_single_node_rows_threshold': 0} | table_format:
> parquet/none | disable_optimized_iceberg_v2_read: 0]
> {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
