Fang-Yu Rao created IMPALA-13276:
------------------------------------
Summary: Revise the description of the query option of
'RUNTIME_FILTER_WAIT_TIME_MS'
Key: IMPALA-13276
URL: https://issues.apache.org/jira/browse/IMPALA-13276
Project: IMPALA
Issue Type: Task
Components: Docs
Reporter: Fang-Yu Rao
Assignee: Fang-Yu Rao
The documentation of the query option 'RUNTIME_FILTER_WAIT_TIME_MS' at
[https://github.com/apache/impala/blob/master/docs/topics/impala_runtime_filter_wait_time_ms.xml#L37-L43]
as provided in the following describes the meaning of this query option.
{code:java}
The <codeph>RUNTIME_FILTER_WAIT_TIME_MS</codeph> query option
adjusts the settings for the runtime filtering feature.
It specifies a time in milliseconds that each scan node waits for
runtime filters to be produced by other plan fragments.
{code}
However the description above is not entirely accurate in that the wait time is
with respect to the time when a runtime filter was registered (within
[QueryState::InitFilterBank()|https://github.com/apache/impala/blob/master/be/src/runtime/query-state.cc#L381])
instead of the time when a scan node is calling
[ScanNode::WaitForRuntimeFilters()|https://github.com/apache/impala/blob/master/be/src/exec/scan-node.cc#L212].
For instance if a scan node started so late that when
ScanNode::WaitForRuntimeFilters() was called, the amount of time passed since
the registration of this runtime filter was already greater than the value of
'RUNTIME_FILTER_WAIT_TIME_MS', this scan node would not be waiting for the
runtime filter. Refer to
[https://github.com/apache/impala/blob/master/be/src/runtime/runtime-filter.cc#L86-L87]
for further details.
{code:java}
bool RuntimeFilter::WaitForArrival(int32_t timeout_ms) const {
unique_lock<mutex> l(arrival_mutex_);
while (arrival_time_.Load() == 0) {
int64_t ms_since_registration = MonotonicMillis() - registration_time_;
int64_t ms_remaining = timeout_ms - ms_since_registration;
if (ms_remaining <= 0) break;
if (injection_delay_ > 0) SleepForMs(injection_delay_);
arrival_cv_.WaitFor(l, ms_remaining * MICROS_PER_MILLI);
}
return arrival_time_.Load() != 0;
}
{code}
We should revise the documentation to make it a bit clearer.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
