水位线对齐与空闲问题

Thu, 16 Mar 2023 19:48:15 -0700 

hi,


我在1.15.x和1.16.1对水位线策略进行测试发现水位线对齐和idleness同时使用会造成变成空闲的source无法再消费kafka数据。这是一个bug吗？
我的水位线策略如下：
           WatermarkStrategy
                .<String>forBoundedOutOfOrderness(Duration.ofMillis(0))
                .withTimestampAssigner((element, recordTimestamp) -> 
Long.parseLong(element))
                .withWatermarkAlignment("default", Duration.ofMillis(10), 
Duration.ofSeconds(2))
                .withIdleness(Duration.ofSeconds(30));


在经过30s以后，水位线对齐的日志如下：
10:31:25,717 INFO  
org.apache.flink.runtime.source.coordinator.SourceCoordinator [] - Distributing 
maxAllowedWatermark=11 to subTaskIds=[0]
10:31:25,717 INFO  
org.apache.flink.runtime.source.coordinator.SourceCoordinator [] - Distributing 
maxAllowedWatermark=11 to subTaskIds=[0]
10:31:27,710 INFO  
org.apache.flink.runtime.source.coordinator.SourceCoordinator [] - Distributing 
maxAllowedWatermark=-9223372036854775799 to subTaskIds=[0]
10:31:27,710 INFO  
org.apache.flink.runtime.source.coordinator.SourceCoordinator [] - Distributing 
maxAllowedWatermark=-9223372036854775799 to subTaskIds=[0]


此时再向topic里面写数据已经不再消费

回复