This is an automated email from the ASF dual-hosted git repository. joemcdonnell pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/impala.git
commit bf945824e62a48822414c9375d6641dc499846a9 Author: Joe McDonnell <joemcdonn...@cloudera.com> AuthorDate: Tue Jun 16 08:18:36 2020 -0700 IMPALA-9831: Fix off by one error in condition for ValidateColumnOffsets() ParquetMetadataUtils::ValidateColumnOffsets() returns an error if the end of the column is beyond the end of the file (i.e. offset > end_of_file). Instead, because there is a footer, the end of column must not be the end of the file either, so it should use offset >= end_of_file. Otherwise, a subsequent DCHECK in ParquetPageReader using the stricter condition will fire. Testing: - Core job Change-Id: I16bd6dfbb8eeacc1cb854ed4a3c2ed9f1c3aa11f Reviewed-on: http://gerrit.cloudera.org:8080/16086 Reviewed-by: Impala Public Jenkins <impala-public-jenk...@cloudera.com> Tested-by: Impala Public Jenkins <impala-public-jenk...@cloudera.com> --- be/src/exec/parquet/parquet-metadata-utils.cc | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/be/src/exec/parquet/parquet-metadata-utils.cc b/be/src/exec/parquet/parquet-metadata-utils.cc index aece0e1..7ec32b6 100644 --- a/be/src/exec/parquet/parquet-metadata-utils.cc +++ b/be/src/exec/parquet/parquet-metadata-utils.cc @@ -245,7 +245,7 @@ Status ParquetMetadataUtils::ValidateColumnOffsets(const string& filename, } int64_t col_len = col_chunk.meta_data.total_compressed_size; int64_t col_end = col_start + col_len; - if (col_end <= 0 || col_end > file_length) { + if (col_end <= 0 || col_end >= file_length) { return Status(Substitute("Parquet file '$0': metadata is corrupt. Column $1 has " "invalid column offsets (offset=$2, size=$3, file_size=$4).", filename, i, col_start, col_len, file_length));