Skye Wanderman-Milne has posted comments on this change. Change subject: IMPALA-1578: fix text scanner to handle "\r\n" delimiters split across blocks ......................................................................
Patch Set 8: (1 comment) http://gerrit.cloudera.org:8080/#/c/2803/8/be/src/exec/hdfs-text-scanner.cc File be/src/exec/hdfs-text-scanner.cc: Line 608: bytes_left -= next_tuple_offset; > why don't we need to update byte_buffer_read_size_ like the old code? Before Lars' patch, we didn't update byte_buffer_read_size_. The comment describes it as "Actual bytes received from last file read.", and I use it CheckForSplitDelimiter() to check if the current byte buffer is 0-length. Updating it here didn't change anything before my patch though since it's not used downstream anywhere. I'm reverting to the original behavior since that's easiest for my patch, and you can compute the updated version with byte_buffer_end_ - byte_buffer_ptr_ if you really want. -- To view, visit http://gerrit.cloudera.org:8080/2803 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-MessageType: comment Gerrit-Change-Id: Id42b441674bb21517ad2788b99942a4b5dc55420 Gerrit-PatchSet: 8 Gerrit-Project: Impala Gerrit-Branch: cdh5-trunk Gerrit-Owner: Skye Wanderman-Milne <[email protected]> Gerrit-Reviewer: Dan Hecht <[email protected]> Gerrit-Reviewer: Internal Jenkins Gerrit-Reviewer: Skye Wanderman-Milne <[email protected]> Gerrit-Reviewer: Tim Armstrong <[email protected]> Gerrit-HasComments: Yes
