Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/21579 )
Change subject: POC: Add queuing to broadcast data sender ...................................................................... Patch Set 4: (15 comments) http://gerrit.cloudera.org:8080/#/c/21579/4/be/src/runtime/krpc-data-stream-sender.h File be/src/runtime/krpc-data-stream-sender.h: http://gerrit.cloudera.org:8080/#/c/21579/4/be/src/runtime/krpc-data-stream-sender.h@170 PS4, Line 170: OutboundQueue(const std::vector<std::unique_ptr<Channel>>& channels, int queue_size, KrpcDataStreamSender* parent); line too long (119 > 90) http://gerrit.cloudera.org:8080/#/c/21579/4/be/src/runtime/krpc-data-stream-sender.h@173 PS4, Line 173: OutboundRowBatch* RpcFinished(OutboundRowBatch* batch, Channel* channel, Status status, bool closed); line too long (105 > 90) http://gerrit.cloudera.org:8080/#/c/21579/4/be/src/runtime/krpc-data-stream-sender.cc File be/src/runtime/krpc-data-stream-sender.cc: http://gerrit.cloudera.org:8080/#/c/21579/4/be/src/runtime/krpc-data-stream-sender.cc@192 PS4, Line 192: Status TransmitData(std::unique_ptr<OutboundRowBatch>* outbound_batch, bool swap_batch, bool* channel_closed=nullptr); line too long (120 > 90) http://gerrit.cloudera.org:8080/#/c/21579/4/be/src/runtime/krpc-data-stream-sender.cc@390 PS4, Line 390: RuntimeState* state, std::shared_ptr<CharMemTrackerAllocator> allocator, OutboundQueue* queue) { line too long (100 > 90) http://gerrit.cloudera.org:8080/#/c/21579/4/be/src/runtime/krpc-data-stream-sender.cc@539 PS4, Line 539: if (next_batch == nullptr || !status.ok()) return; // No batch in queue, wait for next AddBlocked(). line too long (102 > 90) http://gerrit.cloudera.org:8080/#/c/21579/4/be/src/runtime/krpc-data-stream-sender.cc@887 PS4, Line 887: RETURN_IF_ERROR(channels_[i]->Init(state, char_mem_tracker_allocator_, broadcast_queue_.get())); line too long (100 > 90) http://gerrit.cloudera.org:8080/#/c/21579/4/be/src/runtime/krpc-data-stream-sender.cc@1111 PS4, Line 1111: // TODO: update comment - TransmitData() will block if there are still in-flight rpcs (and those will line too long (105 > 90) http://gerrit.cloudera.org:8080/#/c/21579/4/be/src/runtime/krpc-data-stream-sender.cc@1358 PS4, Line 1358: //LOG(INFO) << "Add blocking free_batches_" << free_batches_.size() << " queue " << queue_.size(); line too long (100 > 90) http://gerrit.cloudera.org:8080/#/c/21579/4/be/src/runtime/krpc-data-stream-sender.cc@1373 PS4, Line 1373: OutboundRowBatch* KrpcDataStreamSender::OutboundQueue::RpcFinished(OutboundRowBatch* batch, Channel* channel, Status status, bool closed) { line too long (140 > 90) http://gerrit.cloudera.org:8080/#/c/21579/4/be/src/runtime/krpc-data-stream-sender.cc@1406 PS4, Line 1406: // Detect the case when there is another batch in the queue, but it is processed by all line too long (91 > 90) http://gerrit.cloudera.org:8080/#/c/21579/4/be/src/runtime/krpc-data-stream-sender.cc@1407 PS4, Line 1407: // open channels. As this channel won't processed it, it can be removed from the queue. line too long (91 > 90) http://gerrit.cloudera.org:8080/#/c/21579/4/be/src/runtime/krpc-data-stream-sender.cc@1408 PS4, Line 1408: // TODO: this only works with queue size of 2, will have to come up with something better line too long (93 > 90) http://gerrit.cloudera.org:8080/#/c/21579/4/be/src/runtime/krpc-data-stream-sender.cc@1411 PS4, Line 1411: if (removed_batch && queue_.size() == 1 && channels_.size() == closed_channel_count_ + free_channels_.size()) { line too long (115 > 90) http://gerrit.cloudera.org:8080/#/c/21579/4/be/src/runtime/krpc-data-stream-sender.cc@1428 PS4, Line 1428: //LOG(INFO) << "RpcFinished, found in queue free batches:" << free_batches_.size() << " queue " << queue_.size(); line too long (122 > 90) http://gerrit.cloudera.org:8080/#/c/21579/4/be/src/runtime/krpc-data-stream-sender.cc@1436 PS4, Line 1436: //LOG(INFO) << "RpcFinished,. wait for addblocked free batches:" << free_batches_.size() << " queue " << queue_.size(); line too long (123 > 90) -- To view, visit http://gerrit.cloudera.org:8080/21579 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ibe3d01fada31f1b48de6c1403e7d2955114d8078 Gerrit-Change-Number: 21579 Gerrit-PatchSet: 4 Gerrit-Owner: Csaba Ringhofer <[email protected]> Gerrit-Reviewer: Csaba Ringhofer <[email protected]> Gerrit-Reviewer: Impala Public Jenkins <[email protected]> Gerrit-Reviewer: Michael Smith <[email protected]> Gerrit-Comment-Date: Fri, 12 Jul 2024 17:15:51 +0000 Gerrit-HasComments: Yes
