Michael Ho has posted comments on this change. ( http://gerrit.cloudera.org:8080/13882 )
Change subject: IMPALA-7984: Port runtime filter from Thrift RPC to KRPC ...................................................................... Patch Set 22: (10 comments) http://gerrit.cloudera.org:8080/#/c/13882/22/be/src/runtime/coordinator-backend-state.cc File be/src/runtime/coordinator-backend-state.cc: http://gerrit.cloudera.org:8080/#/c/13882/22/be/src/runtime/coordinator-backend-state.cc@565 PS22, Line 565: LOG(WARNING) LOG(ERROR) http://gerrit.cloudera.org:8080/#/c/13882/22/be/src/runtime/coordinator-backend-state.cc@572 PS22, Line 572: LOG(WARNING) LOG(ERROR) http://gerrit.cloudera.org:8080/#/c/13882/22/be/src/runtime/coordinator-backend-state.cc@575 PS22, Line 575: LOG(WARNING) LOG(ERROR) http://gerrit.cloudera.org:8080/#/c/13882/22/be/src/runtime/runtime-filter-bank.cc File be/src/runtime/runtime-filter-bank.cc: http://gerrit.cloudera.org:8080/#/c/13882/22/be/src/runtime/runtime-filter-bank.cc@120 PS22, Line 120: INFO ERROR http://gerrit.cloudera.org:8080/#/c/13882/22/be/src/runtime/runtime-filter-bank.cc@126 PS22, Line 126: std::unique_lock<SpinLock> l(num_inflight_rpcs_lock_); DCHECK_GT(num_inflight_rpcs_, 0); http://gerrit.cloudera.org:8080/#/c/13882/22/be/src/runtime/runtime-filter-bank.cc@127 PS22, Line 127: num_inflight_rpcs_-- nit: pre-increment/decrement: --num_inflight_rpcs_; http://gerrit.cloudera.org:8080/#/c/13882/22/be/src/runtime/runtime-filter-bank.cc@201 PS22, Line 201: std::unique_lock<SpinLock> l(num_inflight_rpcs_lock_); Please add a comment on why this is necessary: Increment num_inflight_rpcs_ to make sure that the filter will not be deallocated in Close() until all in-flight RPC completes. http://gerrit.cloudera.org:8080/#/c/13882/22/be/src/runtime/runtime-filter-bank.cc@202 PS22, Line 202: num_inflight_rpcs_++; ++num_inflight_rpcs_; http://gerrit.cloudera.org:8080/#/c/13882/22/be/src/runtime/runtime-filter-bank.cc@321 PS22, Line 321: std::chrono::milliseconds(50)) In theory, we don't need a timeout here as we need to wait till the RPC is done. That said, it may not hurt to log a statement after waiting for extended period of time. 50 ms may be a bit iffy if the network is slow or if there is a lot of outbound traffic. May be 60s ? One potential improvement is that we may as well cancel the RPC after waiting for too long. http://gerrit.cloudera.org:8080/#/c/13882/22/be/src/util/bloom-filter.h File be/src/util/bloom-filter.h: http://gerrit.cloudera.org:8080/#/c/13882/22/be/src/util/bloom-filter.h@46 PS22, Line 46: namespace bloom_filter_test_util { : void BfUnion(const impala::BloomFilter& x, const impala::BloomFilter& y, : int64_t directory_size, bool* success, impala::BloomFilterPB* protobuf, : std::string* directory); : } // namespace bloom_filter_test_util : : namespace either { : struct TestData; : } // namespace either Can these be moved to the test file ? -- To view, visit http://gerrit.cloudera.org:8080/13882 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I6b394796d250286510e157ae326882bfc01d387a Gerrit-Change-Number: 13882 Gerrit-PatchSet: 22 Gerrit-Owner: Fang-Yu Rao <[email protected]> Gerrit-Reviewer: Fang-Yu Rao <[email protected]> Gerrit-Reviewer: Impala Public Jenkins <[email protected]> Gerrit-Reviewer: Michael Ho <[email protected]> Gerrit-Reviewer: Thomas Tauber-Marshall <[email protected]> Gerrit-Comment-Date: Mon, 23 Sep 2019 20:52:45 +0000 Gerrit-HasComments: Yes
