Adar Dembo has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/15647 )
Change subject: bitshuffle: stop using uninitialized data as padding ...................................................................... bitshuffle: stop using uninitialized data as padding The resize() incorrectly accounted for a block header that was never actually written to data_. The result was that added padding was actually kHeaderSize bytes "to the right", and the call to compress_lz4() read uninitialized data from this part of data_ rather than the added padding. What's the effect? Up to padding_bytes of uninitialized data gets bitshuffled, compressed, and written to the block. At read time, it is decompressed, debitshuffled, but ultimately ignored, as it was expected to be just padded zeroes. I observed this in cfile-test's TestMetadata built with MSAN instrumentation, but oddly enough not in any other test in cfile-test, even though others use bitshuffle and padding. Change-Id: I25eba027ba356774173b2313c68436d7baddaddc Reviewed-on: http://gerrit.cloudera.org:8080/15647 Reviewed-by: Andrew Wong <[email protected]> Tested-by: Kudu Jenkins --- M src/kudu/cfile/bshuf_block.h 1 file changed, 1 insertion(+), 1 deletion(-) Approvals: Andrew Wong: Looks good to me, approved Kudu Jenkins: Verified -- To view, visit http://gerrit.cloudera.org:8080/15647 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: merged Gerrit-Change-Id: I25eba027ba356774173b2313c68436d7baddaddc Gerrit-Change-Number: 15647 Gerrit-PatchSet: 2 Gerrit-Owner: Adar Dembo <[email protected]> Gerrit-Reviewer: Adar Dembo <[email protected]> Gerrit-Reviewer: Andrew Wong <[email protected]> Gerrit-Reviewer: Bankim Bhavsar <[email protected]> Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Reviewer: Todd Lipcon <[email protected]>
