Tim Armstrong has posted comments on this change. Change subject: IMPALA-3441, IMPALA-3659: check for malformed Avro data ......................................................................
Patch Set 11: (2 comments) http://gerrit.cloudera.org:8080/#/c/3072/13/be/src/exec/hdfs-avro-scanner.cc File be/src/exec/hdfs-avro-scanner.cc: Line 533: if (decompressor_.get() != NULL && !decompressor_->reuse_output_buffer()) { > I didn't add a test file that's long enough to trigger multiple iterations I added a test that triggers the bug with batch_size=1. The trick was to truncate the compressed data for the second fixed-len float row by 1 byte. With the bug, it doesn't notice that it's reading past the end of the buffer for the second row. After the fix, it notices the file is truncated. I had to fix a couple of things in the test result verifier to get this to work, since we may return a subset of results in that case. http://gerrit.cloudera.org:8080/#/c/3072/11/be/src/exec/read-write-util.cc File be/src/exec/read-write-util.cc: Line 87: } > If it's inlined, then this is fine. But wouldn't it work to also just forwa Done -- To view, visit http://gerrit.cloudera.org:8080/3072 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-MessageType: comment Gerrit-Change-Id: I801a11c496a128e02c564c2a9c44baa5a97be132 Gerrit-PatchSet: 11 Gerrit-Project: Impala Gerrit-Branch: cdh5-trunk Gerrit-Owner: Skye Wanderman-Milne <[email protected]> Gerrit-Reviewer: Alex Behm <[email protected]> Gerrit-Reviewer: Dan Hecht <[email protected]> Gerrit-Reviewer: Skye Wanderman-Milne <[email protected]> Gerrit-Reviewer: Tim Armstrong <[email protected]> Gerrit-HasComments: Yes
