Repository: incubator-impala Updated Branches: refs/heads/master d8bc570b6 -> cb645b1bc
IMPALA-5798: ASAN use-after-poison in Parquet decoder In ParquetLevelDecoder::Init() for RLE encoding, we read the metadata size and advance the data buffer past it. If the metadata size is corrupted, it can cause us to incorrectly read past the end of the buffer. This patch checks that the metadata size is less than the total size of the buffer, and returns an error if it isn't. Testing: - Ran test_scanners_fuzz.py under ASAN 500 times without hitting the use-after-poison (previously it would usually hit in < 100 runs). Change-Id: I3f3d0d998f7581c7c935d98fde886f145efd61a8 Reviewed-on: http://gerrit.cloudera.org:8080/7769 Reviewed-by: Alex Behm <[email protected]> Reviewed-by: Matthew Jacobs <[email protected]> Tested-by: Impala Public Jenkins Project: http://git-wip-us.apache.org/repos/asf/incubator-impala/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-impala/commit/cb645b1b Tree: http://git-wip-us.apache.org/repos/asf/incubator-impala/tree/cb645b1b Diff: http://git-wip-us.apache.org/repos/asf/incubator-impala/diff/cb645b1b Branch: refs/heads/master Commit: cb645b1bcb3e18123bee0916da9fbcf7ff55050d Parents: d8bc570 Author: Thomas Tauber-Marshall <[email protected]> Authored: Mon Aug 21 17:58:09 2017 -0700 Committer: Impala Public Jenkins <[email protected]> Committed: Wed Aug 23 20:15:37 2017 +0000 ---------------------------------------------------------------------- be/src/exec/parquet-column-readers.cc | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) ---------------------------------------------------------------------- http://git-wip-us.apache.org/repos/asf/incubator-impala/blob/cb645b1b/be/src/exec/parquet-column-readers.cc ---------------------------------------------------------------------- diff --git a/be/src/exec/parquet-column-readers.cc b/be/src/exec/parquet-column-readers.cc index 9f00762..d55b545 100644 --- a/be/src/exec/parquet-column-readers.cc +++ b/be/src/exec/parquet-column-readers.cc @@ -90,7 +90,7 @@ Status ParquetLevelDecoder::Init(const string& filename, if (!ReadWriteUtil::Read(data, data_size, &num_bytes, &status)) { return status; } - if (num_bytes < 0) { + if (num_bytes < 0 || num_bytes > *data_size) { return Status(TErrorCode::PARQUET_CORRUPT_RLE_BYTES, filename, num_bytes); } int bit_width = BitUtil::Log2Ceiling64(max_level + 1);
