Yida Wu has posted comments on this change. ( http://gerrit.cloudera.org:8080/21754 )
Change subject: IMPALA-12908: Add correctness check for tuple cache ...................................................................... Patch Set 2: (6 comments) Rebased, and updated the cluster scripts for testing with this feature. http://gerrit.cloudera.org:8080/#/c/21754/1/be/src/exec/tuple-cache-node.cc File be/src/exec/tuple-cache-node.cc: http://gerrit.cloudera.org:8080/#/c/21754/1/be/src/exec/tuple-cache-node.cc@200 PS1, Line 200: cache[str].reference_rows_count = 1; > Couldn't this just be Done http://gerrit.cloudera.org:8080/#/c/21754/1/be/src/exec/tuple-cache-node.cc@216 PS1, Line 216: } > This should be impossible. Also both counts are uint64_t, so <0 is meaningl Done http://gerrit.cloudera.org:8080/#/c/21754/1/be/src/exec/tuple-cache-node.cc@260 PS1, Line 260: << new_path << ". Error code: " << result; > It's possible for external interaction to cause an error (another process d Done http://gerrit.cloudera.org:8080/#/c/21754/1/be/src/exec/tuple-cache-node.cc@351 PS1, Line 351: // If reader_ is set, then the child was never opened and needs to be opened now > Can you add a comment about when this would produce different behavior? I t Somehow the reader won't return error but just set the eos if nothing to read, so it is fine for the next iteration https://github.com/apache/impala/blob/30ccfce590903ab64af1ca10b9d42fe3f665aac3/be/src/exec/tuple-file-reader.cc#L64. I changed the logic to use flags to avoid additional GetNext when eos is set. Also added some comments. http://gerrit.cloudera.org:8080/#/c/21754/1/be/src/exec/tuple-cache-node.cc@448 PS1, Line 448: if (debug_dump_text_writer_) { > I think this should be true now, based on scan ranges. Made the changes to use combined_key_ as the sub_dir. http://gerrit.cloudera.org:8080/#/c/21754/1/be/src/exec/tuple-text-file-reader.h File be/src/exec/tuple-text-file-reader.h: http://gerrit.cloudera.org:8080/#/c/21754/1/be/src/exec/tuple-text-file-reader.h@44 PS1, Line 44: // Returns the size of the file in bytes. > Should mention that this leaves the reader at the end of the file. Or updat Made the change to resume the original position. -- To view, visit http://gerrit.cloudera.org:8080/21754 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ied074e274ebf99fb57e3ee41a13148725775b77c Gerrit-Change-Number: 21754 Gerrit-PatchSet: 2 Gerrit-Owner: Yida Wu <[email protected]> Gerrit-Reviewer: Impala Public Jenkins <[email protected]> Gerrit-Reviewer: Joe McDonnell <[email protected]> Gerrit-Reviewer: Kurt Deschler <[email protected]> Gerrit-Reviewer: Michael Smith <[email protected]> Gerrit-Reviewer: Yida Wu <[email protected]> Gerrit-Comment-Date: Fri, 13 Sep 2024 15:46:19 +0000 Gerrit-HasComments: Yes
