morningman commented on a change in pull request #7134:
URL: https://github.com/apache/incubator-doris/pull/7134#discussion_r752399627
##########
File path:
fe/fe-core/src/main/java/org/apache/doris/load/loadv2/BrokerLoadJob.java
##########
@@ -346,6 +346,16 @@ private void
updateLoadingStatus(BrokerLoadingTaskAttachment attachment) {
}
}
+ @Override
+ public void updateProgress(Long beId, TUniqueId loadId, TUniqueId
fragmentId, long scannedRows,
+ long scannedBytes, boolean isDone) {
+ super.updateProgress(beId, loadId, fragmentId, scannedRows,
scannedBytes, isDone);
+ progress = (int) ((double) loadStatistic.getLoadBytes() /
loadStatistic.totalFileSizeB * 100);
Review comment:
This may always larger than 100, when loading parquet or orc format
files.
##########
File path: be/src/exec/tablet_sink.cpp
##########
@@ -707,13 +707,15 @@ Status OlapTableSink::open(RuntimeState* state) {
Status OlapTableSink::send(RuntimeState* state, RowBatch* input_batch) {
SCOPED_TIMER(_profile->total_time_counter());
- _number_input_rows += input_batch->num_rows();
// update incrementally so that FE can get the progress.
// the real 'num_rows_load_total' will be set when sink being closed.
- state->update_num_rows_load_total(input_batch->num_rows());
- state->update_num_bytes_load_total(input_batch->total_byte_size());
- DorisMetrics::instance()->load_rows->increment(input_batch->num_rows());
-
DorisMetrics::instance()->load_bytes->increment(input_batch->total_byte_size());
+ int64_t num_rows = input_batch->num_rows();
+ int64_t num_bytes = input_batch->total_byte_size();
Review comment:
Pay attention that `total_byte_size()` may cost times.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
