Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/21579 )
Change subject: POC: Add queuing to broadcast data sender ...................................................................... Patch Set 2: (16 comments) http://gerrit.cloudera.org:8080/#/c/21579/2/be/src/runtime/krpc-data-stream-sender.h File be/src/runtime/krpc-data-stream-sender.h: http://gerrit.cloudera.org:8080/#/c/21579/2/be/src/runtime/krpc-data-stream-sender.h@170 PS2, Line 170: OutboundQueue(const std::vector<std::unique_ptr<Channel>>& channels, int queue_size, KrpcDataStreamSender* parent); line too long (119 > 90) http://gerrit.cloudera.org:8080/#/c/21579/2/be/src/runtime/krpc-data-stream-sender.h@173 PS2, Line 173: OutboundRowBatch* RpcFinished(OutboundRowBatch* batch, Channel* channel, Status status, bool closed); line too long (105 > 90) http://gerrit.cloudera.org:8080/#/c/21579/2/be/src/runtime/krpc-data-stream-sender.cc File be/src/runtime/krpc-data-stream-sender.cc: http://gerrit.cloudera.org:8080/#/c/21579/2/be/src/runtime/krpc-data-stream-sender.cc@192 PS2, Line 192: Status TransmitData(std::unique_ptr<OutboundRowBatch>* outbound_batch, bool swap_batch, bool* channel_closed=nullptr); line too long (120 > 90) http://gerrit.cloudera.org:8080/#/c/21579/2/be/src/runtime/krpc-data-stream-sender.cc@390 PS2, Line 390: RuntimeState* state, std::shared_ptr<CharMemTrackerAllocator> allocator, OutboundQueue* queue) { line too long (100 > 90) http://gerrit.cloudera.org:8080/#/c/21579/2/be/src/runtime/krpc-data-stream-sender.cc@539 PS2, Line 539: if (next_batch == nullptr || !status.ok()) return; // No batch in queue, wait for next AddBlocked(). line too long (102 > 90) http://gerrit.cloudera.org:8080/#/c/21579/2/be/src/runtime/krpc-data-stream-sender.cc@887 PS2, Line 887: RETURN_IF_ERROR(channels_[i]->Init(state, char_mem_tracker_allocator_, broadcast_queue_.get())); line too long (100 > 90) http://gerrit.cloudera.org:8080/#/c/21579/2/be/src/runtime/krpc-data-stream-sender.cc@1111 PS2, Line 1111: // TODO: update comment - TransmitData() will block if there are still in-flight rpcs (and those will line too long (105 > 90) http://gerrit.cloudera.org:8080/#/c/21579/2/be/src/runtime/krpc-data-stream-sender.cc@1333 PS2, Line 1333: while(queue_.size() == max_queue_size_ && !closed_ && !parent_->state_->is_cancelled() && status_.ok()) { line too long (107 > 90) http://gerrit.cloudera.org:8080/#/c/21579/2/be/src/runtime/krpc-data-stream-sender.cc@1361 PS2, Line 1361: //LOG(INFO) << "Add blocking free_batches_" << free_batches_.size() << " queue " << queue_.size(); line too long (100 > 90) http://gerrit.cloudera.org:8080/#/c/21579/2/be/src/runtime/krpc-data-stream-sender.cc@1382 PS2, Line 1382: OutboundRowBatch* KrpcDataStreamSender::OutboundQueue::RpcFinished(OutboundRowBatch* batch, Channel* channel, Status status, bool closed) { line too long (140 > 90) http://gerrit.cloudera.org:8080/#/c/21579/2/be/src/runtime/krpc-data-stream-sender.cc@1415 PS2, Line 1415: // Detect the case when there is another batch in the queue, but it is processed by all line too long (91 > 90) http://gerrit.cloudera.org:8080/#/c/21579/2/be/src/runtime/krpc-data-stream-sender.cc@1416 PS2, Line 1416: // open channels. As this channel won't processed it, it can be removed from the queue. line too long (91 > 90) http://gerrit.cloudera.org:8080/#/c/21579/2/be/src/runtime/krpc-data-stream-sender.cc@1417 PS2, Line 1417: // TODO: this only works with queue size of 2, will have to come up with something better line too long (93 > 90) http://gerrit.cloudera.org:8080/#/c/21579/2/be/src/runtime/krpc-data-stream-sender.cc@1420 PS2, Line 1420: if (queue_.size() == 1 && channels_.size() == closed_channel_count_ + free_channels_.size()) { line too long (98 > 90) http://gerrit.cloudera.org:8080/#/c/21579/2/be/src/runtime/krpc-data-stream-sender.cc@1437 PS2, Line 1437: //LOG(INFO) << "RpcFinished, found in queue free batches:" << free_batches_.size() << " queue " << queue_.size(); line too long (122 > 90) http://gerrit.cloudera.org:8080/#/c/21579/2/be/src/runtime/krpc-data-stream-sender.cc@1445 PS2, Line 1445: //LOG(INFO) << "RpcFinished,. wait for addblocked free batches:" << free_batches_.size() << " queue " << queue_.size(); line too long (123 > 90) -- To view, visit http://gerrit.cloudera.org:8080/21579 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ibe3d01fada31f1b48de6c1403e7d2955114d8078 Gerrit-Change-Number: 21579 Gerrit-PatchSet: 2 Gerrit-Owner: Csaba Ringhofer <[email protected]> Gerrit-Reviewer: Csaba Ringhofer <[email protected]> Gerrit-Reviewer: Impala Public Jenkins <[email protected]> Gerrit-Reviewer: Michael Smith <[email protected]> Gerrit-Comment-Date: Fri, 12 Jul 2024 08:29:10 +0000 Gerrit-HasComments: Yes
