Dan Hecht has posted comments on this change. Change subject: IMPALA-4026: Implement double-buffering for BlockingQueue ......................................................................
Patch Set 5: (7 comments) http://gerrit.cloudera.org:8080/#/c/4350/5/be/src/exec/hdfs-scan-node.cc File be/src/exec/hdfs-scan-node.cc: Line 79: row_batches_put_timer_ = runtime_profile()->AddCounter("QueuePutTime", TUnit::TIME_NS); we usually do this in Open() or Prepare() (see other counters in e.g. HdfsScanNodeBase. Also usually use ADD_COUNTER() macro (though I think we should get rid of those soon). http://gerrit.cloudera.org:8080/#/c/4350/5/be/src/util/blocking-queue.h File be/src/util/blocking-queue.h: PS5, Line 90: DCHECK nit: DCHECK_NE Line 102: put_cv_.NotifyOne(); is it worth explaining this race rather than fixing it? Doesn't pthreads optimize this case correctly (when holding the "wait lock" when doing a signal, it should just transfer the destination thread to the lock's wait queue rather than waking it up). But maybe we aren't sure that optimization is implemented? PS5, Line 137: GetSize this is an unfortunate name. I read it to be the size of the "Get" list. Maybe we could rename it Size()? Line 171: // the queue's size could have changed once the lock is dropped. how do you know the deque::size() method doesn't need the synchronization (on x86 it's probably okay but maybe not other platforms). Would taking both locks here be prohibitively expensive? Line 197: boost::scoped_ptr<std::deque<T>> put_list_; why add this extra indirection? couldn't we just do deque::swap() directly on the deque, rather than using the ptr swap? http://gerrit.cloudera.org:8080/#/c/4350/5/be/src/util/condition-variable.h File be/src/util/condition-variable.h: PS5, Line 29: doesn't have any logic to deal with thread interruption what's the implication of that? are signals not handled properly? -- To view, visit http://gerrit.cloudera.org:8080/4350 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-MessageType: comment Gerrit-Change-Id: Ib9f4cf351455efefb0f3bb791cf9bc82d1421d54 Gerrit-PatchSet: 5 Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-Owner: Michael Ho <k...@cloudera.com> Gerrit-Reviewer: Alex Behm <alex.b...@cloudera.com> Gerrit-Reviewer: Chen Huang <paulhuan...@utexas.edu> Gerrit-Reviewer: Dan Hecht <dhe...@cloudera.com> Gerrit-Reviewer: Michael Ho <k...@cloudera.com> Gerrit-Reviewer: Mostafa Mokhtar <mmokh...@cloudera.com> Gerrit-Reviewer: Tim Armstrong <tarmstr...@cloudera.com> Gerrit-HasComments: Yes
