[jira] [Commented] (IMPALA-13058) TestRuntimeFilters.test_basic_filters failed on ARM builds
[
https://issues.apache.org/jira/browse/IMPALA-13058?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17844689#comment-17844689
]
ASF subversion and git services commented on IMPALA-13058:
--
Commit fdb87a755a2ac581b3144f1506b6e901ec8d2da2 in impala's branch
refs/heads/master from Riza Suminto
[ https://gitbox.apache.org/repos/asf?p=impala.git;h=fdb87a755 ]
IMPALA-13058: Init first_arrival_time_ and completion_time_ with -1
Impala run over ARM machine shows 'arch_sys_counter' clock source being
used rather than more precise 'tsc'. This cause
MonotonicStopwatch::Now() to use 'CLOCK_MONOTONIC_COARSE' rather than
'CLOCK_MONOTONIC'. This is what printed near the beginning of impalad
log:
I0506 13:49:15.429359 355337 init.cc:600] OS distribution: Red Hat Enterprise
Linux 8.8 (Ootpa)
OS version: Linux version 4.18.0-477.15.1.el8_8.aarch64 ...
Clock: clocksource: 'arch_sys_counter', clockid_t: CLOCK_MONOTONIC_COARSE
This difference in clock source causes test failure in
test_runtime_filters.py::TestRuntimeFilters::test_basic_filters. This
patch fixes the issue by initializing first_arrival_time_ and
completion_time_ fields of Coordinator::FilterState with -1 and accept 0
as valid value for those fields.
query_events_ initialization and start are also moved to the beginning
of ClientRequestState's contructor.
Testing:
- Tweak row_regex pattern in runtime_filters.test.
- Loop and pass test_runtime_filters.py in exhaustive mode 3 times
in ARM machine.
Change-Id: I1176e2118bb03414ab35049f50009ff0e8c63f58
Reviewed-on: http://gerrit.cloudera.org:8080/21405
Reviewed-by: Wenzhe Zhou
Tested-by: Impala Public Jenkins
> TestRuntimeFilters.test_basic_filters failed on ARM builds
> --
>
> Key: IMPALA-13058
> URL: https://issues.apache.org/jira/browse/IMPALA-13058
> Project: IMPALA
> Issue Type: Bug
> Components: Backend
>Reporter: Wenzhe Zhou
>Assignee: Riza Suminto
>Priority: Major
>
> Stacktrace:
> query_test/test_runtime_filters.py:98: in test_basic_filters
> self.run_test_case('QueryTest/runtime_filters', vector,
> test_file_vars=vars)
> /data/jenkins/workspace/impala-cdw-master-exhaustive-release-arm/repos/Impala/tests/common/impala_test_suite.py:851:
> in run_test_case
> update_section=pytest.config.option.update_results)
> /data/jenkins/workspace/impala-cdw-master-exhaustive-release-arm/repos/Impala/tests/common/test_result_verifier.py:701:
> in verify_runtime_profile
> actual))
> E AssertionError: Did not find matches for lines in runtime profile:
> E EXPECTED LINES:
> E row_regex: .*REMOTE.*ms.*ms.*true
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Commented] (IMPALA-13058) TestRuntimeFilters.test_basic_filters failed on ARM builds
[
https://issues.apache.org/jira/browse/IMPALA-13058?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17844132#comment-17844132
]
Wenzhe Zhou commented on IMPALA-13058:
--
Thanks Riza to look in this. The unit-test was very old test. It failed only
for ARM builds, which were added to nightly builds since last week.
> TestRuntimeFilters.test_basic_filters failed on ARM builds
> --
>
> Key: IMPALA-13058
> URL: https://issues.apache.org/jira/browse/IMPALA-13058
> Project: IMPALA
> Issue Type: Bug
> Components: Backend
>Reporter: Wenzhe Zhou
>Assignee: Riza Suminto
>Priority: Major
>
> Stacktrace:
> query_test/test_runtime_filters.py:98: in test_basic_filters
> self.run_test_case('QueryTest/runtime_filters', vector,
> test_file_vars=vars)
> /data/jenkins/workspace/impala-cdw-master-exhaustive-release-arm/repos/Impala/tests/common/impala_test_suite.py:851:
> in run_test_case
> update_section=pytest.config.option.update_results)
> /data/jenkins/workspace/impala-cdw-master-exhaustive-release-arm/repos/Impala/tests/common/test_result_verifier.py:701:
> in verify_runtime_profile
> actual))
> E AssertionError: Did not find matches for lines in runtime profile:
> E EXPECTED LINES:
> E row_regex: .*REMOTE.*ms.*ms.*true
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Commented] (IMPALA-13058) TestRuntimeFilters.test_basic_filters failed on ARM builds
[
https://issues.apache.org/jira/browse/IMPALA-13058?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17844060#comment-17844060
]
Riza Suminto commented on IMPALA-13058:
---
This seems unrelated to runtime filter. It is strange that Query Compilation:
2.858ms, but Planning finished: 0.000ns.
{code:java}
E Query Compilation: 2.858ms
E - Metadata of all 2 tables cached: 274.516us (274.516us)
E - Analysis finished: 543.727us (269.211us)
E - Authorization finished (noop): 599.562us (55.835us)
E - Value transfer graph computed: 665.553us (65.991us)
E - Single node plan created: 1.105ms (439.572us)
E - Runtime filters computed: 1.748ms (643.246us)
E - Distributed plan created: 1.768ms (20.414us)
E - Parallel plans created: 1.782ms (13.834us)
E - Planning finished: 2.858ms (1.075ms)
E Query Timeline: 50.001ms
E - Query submitted: 0.000ns (0.000ns)
E - Planning finished: 0.000ns (0.000ns)
E - Submit for admission: 0.000ns (0.000ns)
E - Completed admission: 0.000ns (0.000ns)
E - Ready to start on 3 backends: 0.000ns (0.000ns)
E - All 3 execution backends (13 fragment instances) started: 0.000ns
(0.000ns)
E - First dynamic filter received: 0.000ns (0.000ns)
E - Rows available: 10.000ms (10.000ms)
E - First row fetched: 50.001ms (40.000ms)
E - Last row fetched: 50.001ms (0.000ns)
E - Released admission control resources: 50.001ms (0.000ns)
E - AdmissionControlTimeSinceLastUpdate: 26.000ms
E - ComputeScanRangeAssignmentTimer: 0.000ns {code}
> TestRuntimeFilters.test_basic_filters failed on ARM builds
> --
>
> Key: IMPALA-13058
> URL: https://issues.apache.org/jira/browse/IMPALA-13058
> Project: IMPALA
> Issue Type: Bug
> Components: Backend
>Reporter: Wenzhe Zhou
>Assignee: Riza Suminto
>Priority: Major
>
> Stacktrace:
> query_test/test_runtime_filters.py:98: in test_basic_filters
> self.run_test_case('QueryTest/runtime_filters', vector,
> test_file_vars=vars)
> /data/jenkins/workspace/impala-cdw-master-exhaustive-release-arm/repos/Impala/tests/common/impala_test_suite.py:851:
> in run_test_case
> update_section=pytest.config.option.update_results)
> /data/jenkins/workspace/impala-cdw-master-exhaustive-release-arm/repos/Impala/tests/common/test_result_verifier.py:701:
> in verify_runtime_profile
> actual))
> E AssertionError: Did not find matches for lines in runtime profile:
> E EXPECTED LINES:
> E row_regex: .*REMOTE.*ms.*ms.*true
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Commented] (IMPALA-13058) TestRuntimeFilters.test_basic_filters failed on ARM builds
[
https://issues.apache.org/jira/browse/IMPALA-13058?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17843833#comment-17843833
]
Riza Suminto commented on IMPALA-13058:
---
Coordinator received filter update very fast within 2ms, but column "First
arrived" and "Completed" stays "N/A" in "Final filter table".
{code:java}
E Filter routing table:
EID Src. Node Tgt. Node(s) Target type Partition filter Pending
(Expected) First arrived Completed Enabled Bloom Size Est fpp Min value
Max value In-list size
E
-
E 0 2 0 REMOTE true 1
(3)N/AN/A true 1.00 MB 1.13e-16
E Backend startup latencies: Count: 3, sum: 0, min / max: 0 / 0, 25th
%-ile: 0, 50th %-ile: 0, 75th %-ile: 0, 90th %-ile: 0, 95th %-ile: 0, 99.9th
%-ile: 0
E Slowest backend to start up:
E Final filter table:
EID Src. Node Tgt. Node(s) Target type Partition filter Pending
(Expected) First arrived Completed Enabled Bloom Size Est fpp Min value
Max value In-list size
E
-
E 0 2 0 REMOTE true 0
(3)N/AN/A true 1.00 MB 1.13e-16 {code}
I think what happen is, the filter arrived so fast such that
first_arrival_time() and completion_time() stays at 0 and printed as "N/A"
[https://github.com/apache/impala/blob/0d01f5e82967b84b182d40de1ce213367e0b6181/be/src/runtime/coordinator.cc#L675-L684]
> TestRuntimeFilters.test_basic_filters failed on ARM builds
> --
>
> Key: IMPALA-13058
> URL: https://issues.apache.org/jira/browse/IMPALA-13058
> Project: IMPALA
> Issue Type: Bug
> Components: Backend
>Reporter: Wenzhe Zhou
>Priority: Major
>
> Stacktrace:
> query_test/test_runtime_filters.py:98: in test_basic_filters
> self.run_test_case('QueryTest/runtime_filters', vector,
> test_file_vars=vars)
> /data/jenkins/workspace/impala-cdw-master-exhaustive-release-arm/repos/Impala/tests/common/impala_test_suite.py:851:
> in run_test_case
> update_section=pytest.config.option.update_results)
> /data/jenkins/workspace/impala-cdw-master-exhaustive-release-arm/repos/Impala/tests/common/test_result_verifier.py:701:
> in verify_runtime_profile
> actual))
> E AssertionError: Did not find matches for lines in runtime profile:
> E EXPECTED LINES:
> E row_regex: .*REMOTE.*ms.*ms.*true
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
