Yujiang Zhong created PARQUET-2244:
--------------------------------------
Summary: Dictionary filter may skip row-groups incorrectly when
evaluating notIn
Key: PARQUET-2244
URL: https://issues.apache.org/jira/browse/PARQUET-2244
Project: Parquet
Issue Type: Bug
Components: parquet-mr
Affects Versions: 1.12.2
Reporter: Yujiang Zhong
Dictionary filter may skip row-groups incorrectly when evaluating `notIn` on
optional columns with null values. Here is an example:
Say there is a optional column `c1` with all pages dict encoded, `c1` has and
only has two distinct values: ['foo', null], and the predicate is `c1 not in
('foo', 'bar')`.
Now dictionary filter may skip this row-group that is actually should not be
skipped, because there are nulls in the column.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
