Repository: incubator-impala Updated Branches: refs/heads/master eb54287fb -> b4343895d
IMPALA-5324: Fix version check in EvalDictionaryFilters Due to a bootstrapping issue with the dictionary filtering code change, the parquet version check used in EvalDictionaryFilters was checking for < 2.10. However, the impala 2.9 parquet contains the appropriate encoding, so this changes the version check to be < 2.9. Change-Id: Icc216332171038f74ff1d2ce3066da8167095361 Reviewed-on: http://gerrit.cloudera.org:8080/6969 Reviewed-by: Alex Behm <[email protected]> Tested-by: Impala Public Jenkins Project: http://git-wip-us.apache.org/repos/asf/incubator-impala/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-impala/commit/a173f70a Tree: http://git-wip-us.apache.org/repos/asf/incubator-impala/tree/a173f70a Diff: http://git-wip-us.apache.org/repos/asf/incubator-impala/diff/a173f70a Branch: refs/heads/master Commit: a173f70a0ccacff2de978d171d6bb42349917471 Parents: eb54287 Author: Joe McDonnell <[email protected]> Authored: Tue May 23 16:00:55 2017 -0700 Committer: Impala Public Jenkins <[email protected]> Committed: Thu May 25 01:37:25 2017 +0000 ---------------------------------------------------------------------- be/src/exec/hdfs-parquet-scanner.cc | 7 +++---- 1 file changed, 3 insertions(+), 4 deletions(-) ---------------------------------------------------------------------- http://git-wip-us.apache.org/repos/asf/incubator-impala/blob/a173f70a/be/src/exec/hdfs-parquet-scanner.cc ---------------------------------------------------------------------- diff --git a/be/src/exec/hdfs-parquet-scanner.cc b/be/src/exec/hdfs-parquet-scanner.cc index 1d5d807..4106943 100644 --- a/be/src/exec/hdfs-parquet-scanner.cc +++ b/be/src/exec/hdfs-parquet-scanner.cc @@ -834,11 +834,10 @@ Status HdfsParquetScanner::EvalDictionaryFilters(const parquet::RowGroup& row_gr bool* row_group_eliminated) { *row_group_eliminated = false; - // TODO: Bootstrapping problem: existing 2.9 files don't have the encoding - // stats or encodings set properly, but after this goes in, they will. - // Change to 2.9 later. + // Legacy impala files (< 2.9) require special handling, because they do not encode + // information about whether the column is 100% dictionary encoded. bool is_legacy_impala = false; - if (file_version_.application == "impala" && file_version_.VersionLt(2,10,0)) { + if (file_version_.application == "impala" && file_version_.VersionLt(2,9,0)) { is_legacy_impala = true; }
