[
https://issues.apache.org/jira/browse/IMPALA-13643?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17910574#comment-17910574
]
ASF subversion and git services commented on IMPALA-13643:
----------------------------------------------------------
Commit e82cf3b3a1c2860c722d74e33cf9816d6abfe7af in impala's branch
refs/heads/master from Riza Suminto
[ https://gitbox.apache.org/repos/asf?p=impala.git;h=e82cf3b3a ]
IMPALA-13643: Add scan_multithread_constraint in test_scanners.py
Debug action HDFS_SCANNER_THREAD_CHECK_SOFT_MEM_LIMIT exist inside
HdfsScanNode (hdfs-scan-node.cc) code path. MT_DOP > 0 executes using
HdfsScanNodeMt (hdfs-scan-node-mt.cc) rather than HdfsScanNode, and
always start single scanner thread per ScanNode. Thus, there is no need
to exercise HDFS_SCANNER_THREAD_CHECK_SOFT_MEM_LIMIT and MT_DOP > 0
combination.
This patch adds scan_multithread_constraint in test_scanners.py where
'mt_dop' exec option dimension is declared. This reduce core test vector
combination from 1254 to 1138 and exhaustive test vector combination
from 7530 to 6774.
Testing:
- Run and pass test_scanners.py in core exploration.
Change-Id: I77c2e6f9bbd4bc1825fa1f006a22ee1a6ea5a606
Reviewed-on: http://gerrit.cloudera.org:8080/22300
Reviewed-by: Impala Public Jenkins <[email protected]>
Reviewed-by: Jason Fehr <[email protected]>
Tested-by: Impala Public Jenkins <[email protected]>
> HDFS_SCANNER_THREAD_CHECK_SOFT_MEM_LIMIT is ineffective in MT_DOP > 0
> ---------------------------------------------------------------------
>
> Key: IMPALA-13643
> URL: https://issues.apache.org/jira/browse/IMPALA-13643
> Project: IMPALA
> Issue Type: Improvement
> Components: Test
> Reporter: Riza Suminto
> Assignee: Riza Suminto
> Priority: Minor
> Fix For: Impala 4.5.0
>
>
> Debug action HDFS_SCANNER_THREAD_CHECK_SOFT_MEM_LIMIT exist inside
> HdfsScanNode (hdfs-scan-node.cc) code path. MT_DOP > 0 executes using
> HdfsScanNodeMt (hdfs-scan-node-mt.cc) rather than HdfsScanNode, and always
> start single scanner thread per ScanNode. Thus, no need to exercise
> HDFS_SCANNER_THREAD_CHECK_SOFT_MEM_LIMIT and MT_DOP > 0 combination.
> [https://github.com/apache/impala/blob/99529db6ad62ddc34cbfd924d7e41b1fce5b60a2/tests/query_test/test_scanners.py#L70]
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
