Pooja Nilangekar has posted comments on this change. ( http://gerrit.cloudera.org:8080/11517 )
Change subject: [WIP] IMPALA-6932: Speed up scans for sequence datasets with many files ...................................................................... Patch Set 3: (4 comments) http://gerrit.cloudera.org:8080/#/c/11517/2/be/src/exec/hdfs-scan-node-base.h File be/src/exec/hdfs-scan-node-base.h: http://gerrit.cloudera.org:8080/#/c/11517/2/be/src/exec/hdfs-scan-node-base.h@262 PS2, Line 262: EnqueueLocation enqueue_location = EnqueueLocation::TAIL) WARN_UNUSED_RESULT; > I think this is a reasonable place to use a default argument value but I'm Agreed. Actually, thinking about it made me realize that this is probably a bad idea. If we were to add another scanner for a new file format, we'd definitely use some variant of the AddDiskIoRanges() function. At that point it would make sense for the caller to think/reason about the queueing order. (Hopefully avoiding behavior like this issue.) I tried getting rid of the default argument but it caused issues with the HdfsLzoTextScanner::IssueFileRanges() in Impala-lzo. I have changed all the invocations to pass in the EnqueueLocation and will get rid of the default arguments once Impala-lzo is modified. Does that make sense? Or should I let the default parameters remain as is? http://gerrit.cloudera.org:8080/#/c/11517/2/be/src/exec/hdfs-scan-node.h File be/src/exec/hdfs-scan-node.h: http://gerrit.cloudera.org:8080/#/c/11517/2/be/src/exec/hdfs-scan-node.h@90 PS2, Line 90: int num_files_queued, EnqueueLocation enqueue_location = > weird line wrapping. Ok to leave if clang-format did this. Yes, it was because of clang-format. http://gerrit.cloudera.org:8080/#/c/11517/2/be/src/runtime/io/request-context.h File be/src/runtime/io/request-context.h: http://gerrit.cloudera.org:8080/#/c/11517/2/be/src/runtime/io/request-context.h@28 PS2, Line 28: L > extra space Done http://gerrit.cloudera.org:8080/#/c/11517/2/be/src/runtime/io/request-context.cc File be/src/runtime/io/request-context.cc: http://gerrit.cloudera.org:8080/#/c/11517/2/be/src/runtime/io/request-context.cc@358 PS2, Line 358: DCHECK_ENUM_EQ(schedule_mode, ScheduleMode::UPON_GETNEXT_HEAD); > Add a DCHECK_ENUM_EQ to assert that it's UPON_GETNEXT_HEAD so that the assu Done -- To view, visit http://gerrit.cloudera.org:8080/11517 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I211e2511ea3bb5edea29f1bd63e6b1fa4c4b1965 Gerrit-Change-Number: 11517 Gerrit-PatchSet: 3 Gerrit-Owner: Pooja Nilangekar <[email protected]> Gerrit-Reviewer: Bikramjeet Vig <[email protected]> Gerrit-Reviewer: Impala Public Jenkins <[email protected]> Gerrit-Reviewer: Pooja Nilangekar <[email protected]> Gerrit-Reviewer: Tim Armstrong <[email protected]> Gerrit-Comment-Date: Mon, 22 Oct 2018 20:58:37 +0000 Gerrit-HasComments: Yes
