Michael Ho has posted comments on this change. Change subject: IMPALA-2831: Bound the number of scanner threads per scan node. ......................................................................
Patch Set 1: (2 comments) http://gerrit.cloudera.org:8080/#/c/4174/1/be/src/exec/hdfs-scan-node.cc File be/src/exec/hdfs-scan-node.cc: Line 262: RETURN_IF_ERROR(HdfsParquetScanner::IssueInitialRanges(this, > Is the order important? Would be helpful to have a comment here if there is Yes, mostly for slight preference for parquet files. This mostly makes up for the difference in behavior due to the change in line 906. Previously, we would have spun up those scanner threads (unintentionally) when calling IssueInitalRanges() for Text file format. We also start issuing the initial IO ranges slightly earlier now for parquet files. Line 740: runtime_state_->resource_pool()->set_max_quota(CpuInfo::num_cores() + 1); > Nm, you are already doing that. It seems weird that the scanner threads quo Yes, I agree that we probably should have the + 1 for the query option too. -- To view, visit http://gerrit.cloudera.org:8080/4174 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-MessageType: comment Gerrit-Change-Id: I191988ad18d6b4caf892fc967258823edcf9681f Gerrit-PatchSet: 1 Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-Owner: Michael Ho <k...@cloudera.com> Gerrit-Reviewer: Michael Ho <k...@cloudera.com> Gerrit-Reviewer: Tim Armstrong <tarmstr...@cloudera.com> Gerrit-HasComments: Yes
