amaliujia commented on code in PR #5137:
URL: https://github.com/apache/calcite/pull/5137#discussion_r3701268485
##########
core/src/main/java/org/apache/calcite/adapter/enumerable/EnumUtils.java:
##########
@@ -1173,21 +1181,26 @@ private void initialize() {
elements.add(inputEnumerator.current());
}
- Map<@Nullable Object, SortedMultiMap<Pair<Long, Long>, @Nullable
Object[]>> sessionKeyMap =
- new HashMap<>();
+ // The windows of each key are kept sorted by start time; the merge
+ // below only compares a window with the one that precedes it.
+ Map<@Nullable Object, NavigableMap<Pair<Long, Long>, List<@Nullable
Object[]>>>
+ sessionKeyMap = new HashMap<>();
Review Comment:
I am a bit confused: how do we achieved sorted timestamp here?
##########
core/src/main/java/org/apache/calcite/adapter/enumerable/EnumUtils.java:
##########
@@ -1173,21 +1181,26 @@ private void initialize() {
elements.add(inputEnumerator.current());
}
- Map<@Nullable Object, SortedMultiMap<Pair<Long, Long>, @Nullable
Object[]>> sessionKeyMap =
- new HashMap<>();
+ // The windows of each key are kept sorted by start time; the merge
+ // below only compares a window with the one that precedes it.
+ Map<@Nullable Object, NavigableMap<Pair<Long, Long>, List<@Nullable
Object[]>>>
+ sessionKeyMap = new HashMap<>();
for (@Nullable Object[] element : elements) {
- SortedMultiMap<Pair<Long, Long>, @Nullable Object[]> session =
- sessionKeyMap.computeIfAbsent(element[indexOfKeyColumn], k -> new
SortedMultiMap<>());
- Object watermark =
- requireNonNull(element[indexOfWatermarkedColumn],
- "element[indexOfWatermarkedColumn]");
+ Object watermark = element[indexOfWatermarkedColumn];
+ if (watermark == null) {
+ // A row without a timestamp belongs to no session, and window_start
Review Comment:
This is changing behavior:
The previous we expect all the rows should have timestamp otherwise will
fail by `requireNonNull`.
Can you remind me when a row without timestamp could hit here?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
