Tianyi Wang has uploaded a new change for review. http://gerrit.cloudera.org:8080/8030
Change subject: IMPALA-5250: Unify decompressor output_length semantics ...................................................................... IMPALA-5250: Unify decompressor output_length semantics This patch makes the semantics output_length parameter in Codec::ProcessBlock to be the same. In existing code different decompressor treats output_length differently: 1. SnappyDecompressor needs output_length to be greater than or equal to actual decompressed length, but it does not set it to actual decompressed length after decompression. 2. SnappyBlockDecompressor and Lz4Decompressor require output_length to be exactly the same as actual decompressed length, otherwise decompression fails. 3. Other decompressors need output_length to be greater than or equal to actual decompressed length and will set it to actual decompressed length if oversized. This inconsistency leads to a bug where the error message is undeterministic when the compressed block is corrupted. This patch makes all decompressor behave as 3 and adds a testcase checking that decompressors can handle oversized output buffer correctly. Lz4Decompressor will use the "safe" version of decompression function instead of the "fast" version, for the latter is insecure with corrupted data and requires decompressed length to be known. Change-Id: Ifd42942b169921a7eb53940c3762bc45bb82a993 --- M be/src/util/decompress-test.cc M be/src/util/decompress.cc 2 files changed, 31 insertions(+), 24 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/30/8030/1 -- To view, visit http://gerrit.cloudera.org:8080/8030 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-MessageType: newchange Gerrit-Change-Id: Ifd42942b169921a7eb53940c3762bc45bb82a993 Gerrit-PatchSet: 1 Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-Owner: Tianyi Wang <tw...@cloudera.com>
