github-actions[bot] commented on code in PR #66099:
URL: https://github.com/apache/doris/pull/66099#discussion_r3699016758
##########
fe/fe-core/src/main/java/org/apache/doris/load/routineload/kafka/KafkaTaskInfo.java:
##########
@@ -126,9 +130,13 @@ public TRoutineLoadTask createRoutineLoadTask() throws
UserException {
@Override
public void updateAdaptiveTimeout(RoutineLoadJob routineLoadJob) {
- if (!isEof) {
+ adaptiveMinBatchInterval =
Config.routine_load_adaptive_min_batch_interval_sec;
+ KafkaRoutineLoadJob kafkaRoutineLoadJob = (KafkaRoutineLoadJob)
routineLoadJob;
+ isAdaptiveBatch =
DebugPointUtil.isEnable("KafkaTaskInfo.shouldUseAdaptiveBatch")
Review Comment:
[P2] Validate both sides of the adaptive timeout contract
This new lag gate can apply an internally invalid timing tuple to the first
formerly EOF task. For example, an accepted adaptive interval of 30000 with the
default multiplier 10 derives 300000 seconds, above `max_load_timeout_second`
(259200); `beginTxn()` rejects it and every delayed retry fails before BE
submission. Conversely, the accepted multiplier 0 derives the valid 60-second
minimum while Thrift still sends `maxIntervalS=360`; a slow task can time out
before any adaptive batch boundary, its ABORTED Kafka progress is ignored, and
renewal repeats from the same offsets. Snapshotting once does not make either
tuple coherent. Please validate these mutable values together so the
transaction/planner timeout both stays within admission bounds and outlives the
advertised BE interval (or pause/reject deterministically), with upper-bound,
zero/negative-multiplier, and abort/renew coverage.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
