[
https://issues.apache.org/jira/browse/IMPALA-4816?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Tim Armstrong resolved IMPALA-4816.
-----------------------------------
Resolution: Fixed
Assignee: Bikramjeet Vig (was: Henry Robinson)
Fix Version/s: Impala 2.12.0
IMPALA-5519 avoids this entirely by reserving the memory.
> There is no way to determine if a filter was dropped due to OOM based on
> profile
> --------------------------------------------------------------------------------
>
> Key: IMPALA-4816
> URL: https://issues.apache.org/jira/browse/IMPALA-4816
> Project: IMPALA
> Issue Type: Bug
> Components: Backend
> Affects Versions: Impala 2.6.0, Impala 2.7.0, Impala 2.8.0, Impala 2.9.0
> Reporter: Tim Armstrong
> Assignee: Bikramjeet Vig
> Priority: Major
> Labels: supportability
> Fix For: Impala 2.12.0
>
>
> I just spent some time debugging a test failure that was the result of a
> bloom filter not being sent due to out of memory. There's no obvious way to
> tell if this happened by looking at the profile - it looks like the filter
> arrived and was just ineffective. It looks like the filter can be dropped in
> RuntimeFilterBank::PublishGlobalFilter() and
> PhjBuilder::AllocateRuntimeFilters()
> {code}
> Filter 0 (128.00 MB):
> - Files processed: 8 (8)
> - Files rejected: 0 (0)
> - Files total: 8 (8)
> - RowGroups processed: 8 (8)
> - RowGroups rejected: 0 (0)
> - RowGroups total: 8 (8)
> - Rows processed: 2.46K (2460)
> - Rows rejected: 0 (0)
> - Rows total: 2.46K (2460)
> - Splits processed: 8 (8)
> - Splits rejected: 0 (0)
> - Splits total: 8 (8)
> {code}
> I ran into it with the follow query on my buffer pool branch. You can
> probably replicate it on master by tweaking mem_limit.
> {code}
> [localhost:21000] > use functional_parquet; SET RUNTIME_FILTER_MODE=GLOBAL;
> SET RUNTIME_FILTER_WAIT_TIME_MS=30000; SET RUNTIME_FILTER_MIN_SIZE=128MB; SET
> RUNTIME_FILTER_MAX_SIZE=500MB; SET MEM_LIMIT=140MB; select STRAIGHT_JOIN *
> from alltypes a join [BROADCAST] alltypes b on a.month = b.id and b.int_col =
> -3;
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
