yunfengzhou-hub commented on code in PR #23521:
URL: https://github.com/apache/flink/pull/23521#discussion_r1371166494
##########
flink-streaming-java/src/main/java/org/apache/flink/streaming/runtime/streamrecord/StreamElementSerializer.java:
##########
@@ -146,6 +147,8 @@ public void copy(DataInputView source, DataOutputView
target) throws IOException
target.writeLong(source.readLong());
target.writeLong(source.readLong());
target.writeInt(source.readInt());
+ } else if (tag == TAG_RECORD_ATTRIBUTES) {
+ target.writeBoolean(source.readBoolean());
Review Comment:
Do we need to serialize/deserialize InternalRecordAttributes as well?
##########
flink-streaming-java/src/test/java/org/apache/flink/streaming/api/operators/InternalBacklogAwareTimerServiceImplTest.java:
##########
@@ -0,0 +1,115 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.flink.streaming.api.operators;
+
+import org.apache.flink.api.common.typeutils.base.IntSerializer;
+import org.apache.flink.api.common.typeutils.base.StringSerializer;
+import org.apache.flink.api.java.tuple.Tuple2;
+import org.apache.flink.runtime.state.KeyGroupRange;
+import org.apache.flink.runtime.state.KeyGroupedInternalPriorityQueue;
+import org.apache.flink.runtime.state.heap.HeapPriorityQueueSetFactory;
+import org.apache.flink.streaming.runtime.tasks.TestProcessingTimeService;
+
+import org.junit.Test;
+
+import java.util.ArrayList;
+import java.util.List;
+
+import static
org.apache.flink.streaming.api.operators.TimeServiceTestUtils.createBacklogTimerService;
+import static
org.apache.flink.streaming.api.operators.TimeServiceTestUtils.createInternalTimerService;
+import static
org.apache.flink.streaming.api.operators.TimeServiceTestUtils.createTimerQueue;
+import static org.assertj.core.api.Assertions.assertThat;
+
+/** Tests for {@link InternalBacklogAwareTimerServiceImpl}. */
+public class InternalBacklogAwareTimerServiceImplTest extends
InternalTimerServiceImplTest {
Review Comment:
Let's override the way `InternalTimerServiceImplTest` uses to create timer
service into producing `InternalBacklogAwareTimerServiceImpl` as well.
Otherwise extending `InternalTimerServiceImplTest` will not test
`InternalBacklogAwareTimerServiceImpl` against more test cases.
##########
flink-tests/src/test/java/org/apache/flink/test/streaming/api/datastream/StreamingWithBacklogITCase.java:
##########
@@ -0,0 +1,180 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.flink.test.streaming.api.datastream;
+
+import org.apache.flink.api.common.eventtime.WatermarkStrategy;
+import org.apache.flink.api.common.functions.AggregateFunction;
+import org.apache.flink.api.common.functions.ReduceFunction;
+import org.apache.flink.api.common.typeinfo.Types;
+import org.apache.flink.api.java.tuple.Tuple2;
+import org.apache.flink.api.java.tuple.Tuple3;
+import org.apache.flink.configuration.Configuration;
+import org.apache.flink.connector.base.source.hybrid.HybridSource;
+import org.apache.flink.connector.datagen.source.DataGeneratorSource;
+import org.apache.flink.connector.datagen.source.GeneratorFunction;
+import org.apache.flink.streaming.api.datastream.SingleOutputStreamOperator;
+import org.apache.flink.streaming.api.environment.StreamExecutionEnvironment;
+import org.apache.flink.streaming.api.functions.windowing.WindowFunction;
+import
org.apache.flink.streaming.api.windowing.assigners.TumblingEventTimeWindows;
+import org.apache.flink.streaming.api.windowing.time.Time;
+import org.apache.flink.streaming.api.windowing.windows.TimeWindow;
+import org.apache.flink.util.CloseableIterator;
+import org.apache.flink.util.CollectionUtil;
+
+import org.junit.jupiter.api.Test;
+
+import java.time.Duration;
+import java.util.ArrayList;
+import java.util.List;
+
+import static
org.apache.flink.streaming.api.environment.ExecutionCheckpointingOptions.CHECKPOINTING_INTERVAL_DURING_BACKLOG;
+import static org.assertj.core.api.Assertions.assertThat;
+
+/** Integration test for streaming job with backlog. */
+public class StreamingWithBacklogITCase {
+ @Test
+ void testKeyedAggregationWithBacklog() throws Exception {
+ final Configuration config = new Configuration();
+ config.set(CHECKPOINTING_INTERVAL_DURING_BACKLOG, Duration.ZERO);
Review Comment:
The test cases in this class can also pass without this configuration, which
means we are not sure whether the changes in this PR took effect. Would it be
better if we add some verifications that are exclusive to backlog cases? For
example, the frequency of switching keys is much lower than normal cases, and
the watermark happens to stop advancing until backlog status changes to false.
##########
flink-streaming-java/src/test/java/org/apache/flink/streaming/api/operators/LambdaTrigger.java:
##########
@@ -0,0 +1,60 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.flink.streaming.api.operators;
+
+import org.assertj.core.api.Assertions;
+
+import java.util.function.Consumer;
+
+/** Triggerable for test. */
+public class LambdaTrigger<K, N> implements Triggerable<K, N> {
Review Comment:
It might be better to improve the naming and JavaDoc of this class, since it
is a shared utility class instead of a private class now.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
