Skye Wanderman-Milne has posted comments on this change. Change subject: IMPALA-3441, IMPALA-3659: check for malformed Avro data ......................................................................
Patch Set 12: (4 comments) http://gerrit.cloudera.org:8080/#/c/3072/12/be/src/exec/hdfs-avro-scanner.cc File be/src/exec/hdfs-avro-scanner.cc: Line 123: if (num_entries < 1) return Status("File header metadata has no data"); > Can we add an error code for this and the other similar check. Something ge Done http://gerrit.cloudera.org:8080/#/c/3072/12/be/src/exec/read-write-util.cc File be/src/exec/read-write-util.cc: Line 46: shift += 7; > I'm pretty sure this can overflow. E.g. if all bytes before 0x80 are 0x7f. Done, as per your suggestion. I have this case in the unit test too but don't check for a particular result. http://gerrit.cloudera.org:8080/#/c/3072/12/be/src/exec/read-write-util.h File be/src/exec/read-write-util.h: Line 110: /// (i.e. the first invalid byte). > Document the behaviour in error cases. If the zig-zag encoded number has to Done http://gerrit.cloudera.org:8080/#/c/3072/12/be/src/exec/scanner-context.cc File be/src/exec/scanner-context.cc: Line 128: DCHECK(!completed_io_buffers_.empty()); > I'm not sure I understand this DCHECK. It seems like we could maybe hit it I put it just to have some extra validation that the condition correctly specifies that we're at eof, since I don't think we create scan ranges for empty files. I'll remove it since it's confusing though. -- To view, visit http://gerrit.cloudera.org:8080/3072 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-MessageType: comment Gerrit-Change-Id: I801a11c496a128e02c564c2a9c44baa5a97be132 Gerrit-PatchSet: 12 Gerrit-Project: Impala Gerrit-Branch: cdh5-trunk Gerrit-Owner: Skye Wanderman-Milne <[email protected]> Gerrit-Reviewer: Alex Behm <[email protected]> Gerrit-Reviewer: Dan Hecht <[email protected]> Gerrit-Reviewer: Skye Wanderman-Milne <[email protected]> Gerrit-Reviewer: Tim Armstrong <[email protected]> Gerrit-HasComments: Yes
