Philip Zeyliger has posted comments on this change. ( http://gerrit.cloudera.org:8080/12097 )
Change subject: IMPALA-7980: Fix spinning because of buggy num_unqueued_files_. ...................................................................... Patch Set 5: (3 comments) Thanks for the reviews. The non-assertion (but rather printing) caught one more bug, wherein the Avro stuff wasn't decrementing when skipping files. I've fixed that. I also renamed a variable that was the same name in a base class and a subclass. I was using the file_type_counter_ business to debug something. It's very minor, so I snuck it in. http://gerrit.cloudera.org:8080/#/c/12097/5//COMMIT_MSG Commit Message: http://gerrit.cloudera.org:8080/#/c/12097/5//COMMIT_MSG@85 PS5, Line 85: adn > Nit: and Done http://gerrit.cloudera.org:8080/#/c/12097/5/be/src/exec/hdfs-scan-node-base.h File be/src/exec/hdfs-scan-node-base.h: http://gerrit.cloudera.org:8080/#/c/12097/5/be/src/exec/hdfs-scan-node-base.h@465 PS5, Line 465: remaining_scan_range_issuances_ > Here is how it works (as I understand it, which might be incomplete): I am now trying remaining_scan_range_submissions_. I'd argue that it's all better than num_unqueued_files_. :) http://gerrit.cloudera.org:8080/#/c/12097/5/be/src/exec/hdfs-scan-node.cc File be/src/exec/hdfs-scan-node.cc: http://gerrit.cloudera.org:8080/#/c/12097/5/be/src/exec/hdfs-scan-node.cc@79 PS5, Line 79: auto remaining = remaining_scan_range_issuances_.Load(); : if (remaining != 0) { : LOG(INFO) << "XXX Assertion Failed " << remaining << " " << GetStackTrace(); : } > If you take the suggestion of using the above DCHECK, I don't think you nee You were right and we can move this into Close() once the threads have been joined. I did switch to DCHECK, but I'm still using the ifdef. I could write a more complicated boolean expression rather than using the if(... && ... && ...) business, but I think it's clearer with the ifndef. -- To view, visit http://gerrit.cloudera.org:8080/12097 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I133de13238d3d05c510e2ff771d48979125735b1 Gerrit-Change-Number: 12097 Gerrit-PatchSet: 5 Gerrit-Owner: Philip Zeyliger <[email protected]> Gerrit-Reviewer: Impala Public Jenkins <[email protected]> Gerrit-Reviewer: Joe McDonnell <[email protected]> Gerrit-Reviewer: Michael Ho <[email protected]> Gerrit-Reviewer: Philip Zeyliger <[email protected]> Gerrit-Reviewer: Tim Armstrong <[email protected]> Gerrit-Reviewer: Todd Lipcon <[email protected]> Gerrit-Comment-Date: Tue, 29 Jan 2019 23:11:31 +0000 Gerrit-HasComments: Yes
