EmmyMiao87 commented on a change in pull request #703: Change the relationship
between txn and task
URL: https://github.com/apache/incubator-doris/pull/703#discussion_r263337211
##########
File path:
fe/src/main/java/org/apache/doris/load/routineload/KafkaProgress.java
##########
@@ -57,7 +58,7 @@ public void setPartitionIdToOffset(Map<Integer, Long>
partitionIdToOffset) {
public void update(RoutineLoadProgress progress) {
KafkaProgress newProgress = (KafkaProgress) progress;
newProgress.getPartitionIdToOffset().entrySet().parallelStream()
- .forEach(entity -> partitionIdToOffset.put(entity.getKey(),
entity.getValue()));
+ .forEach(entity -> partitionIdToOffset.put(entity.getKey(),
entity.getValue() + 1));
Review comment:
The offset of txn is the end offset while the kafka offset of progress is
begin offset . The next task will read data from this begin offset.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
