Impala Public Jenkins has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/8748 )
Change subject: IMPALA-5987: LZ4 Codec silently produces bogus compressed data for large inputs ...................................................................... IMPALA-5987: LZ4 Codec silently produces bogus compressed data for large inputs When Lz4Compressor::MaxOutputLen returns 0, it means that the input is too large to compress. When invoked Lz4Compressor::ProcessBlock with an input too large, it silently produced a bogus result. This bogus result even decompresses successfully, but not to the data that was originally compressed. After this commit, Lz4Compressor::ProcessBlock will return error if it cannot compress the input. I also added a comment on Codec::MaxOutputLen() that return value 0 means that the input is too large. I added some checks after the invocations of MaxOutputLen() where the compressor can be a Lz4Compressor. I added an automated test case to be/src/util/decompress-test.cc. Change-Id: Ifb0bc4ed98c5d7b628b791aa90ead36347b9fbb8 Reviewed-on: http://gerrit.cloudera.org:8080/8748 Reviewed-by: Tim Armstrong <tarmstr...@cloudera.com> Tested-by: Impala Public Jenkins --- M be/src/benchmarks/row-batch-serialize-benchmark.cc M be/src/catalog/catalog-util.cc M be/src/runtime/row-batch.cc M be/src/util/codec.h M be/src/util/compress.cc M be/src/util/decompress-test.cc M be/src/util/runtime-profile.cc M common/thrift/generate_error_codes.py 8 files changed, 39 insertions(+), 1 deletion(-) Approvals: Tim Armstrong: Looks good to me, approved Impala Public Jenkins: Verified -- To view, visit http://gerrit.cloudera.org:8080/8748 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: merged Gerrit-Change-Id: Ifb0bc4ed98c5d7b628b791aa90ead36347b9fbb8 Gerrit-Change-Number: 8748 Gerrit-PatchSet: 3 Gerrit-Owner: Zoltan Borok-Nagy <borokna...@cloudera.com> Gerrit-Reviewer: Alex Behm <alex.b...@cloudera.com> Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Tim Armstrong <tarmstr...@cloudera.com> Gerrit-Reviewer: Zoltan Borok-Nagy <borokna...@cloudera.com>
