Github user xccui commented on the issue:
https://github.com/apache/flink/pull/5094
Never mind~ Thanks for reminding me of the ideas :-)
---
Github user fhueske commented on the issue:
https://github.com/apache/flink/pull/5094
No worries! It's always good to have another pair of eyes looking over
complex logic like this :-)
---
Github user hequn8128 commented on the issue:
https://github.com/apache/flink/pull/5094
Ops... finally understand what you mean. Thank your very much for your
explain. @fhueske @xccui . I will close this pr :)
---
Github user fhueske commented on the issue:
https://github.com/apache/flink/pull/5094
Yes, the records emitted in your test are late. But the reason for that is
that also records that you give into the join are already late. You set the
watermarks on both input to `6000` and
Github user hequn8128 commented on the issue:
https://github.com/apache/flink/pull/5094
Thanks for your replies.
I agree with that it is valid to join late input data.What I concern is the
watermark has not been hold back correctly.
Take `testRowTimeJoinWithCommonBounds2`
Github user fhueske commented on the issue:
https://github.com/apache/flink/pull/5094
I had a look at `testRowTimeJoinWithCommonBounds2`.
The test processes starts by processing two watermarks for `6000` for the
first and second input. After that three records are added for
Github user xccui commented on the issue:
https://github.com/apache/flink/pull/5094
Hi @hequn8128, as @fhueske explained, watermarks should not define the
semantics of the join. We just emit join results, in spite of whether they are
late and the join function should not care about
Github user hequn8128 commented on the issue:
https://github.com/apache/flink/pull/5094
Thanks for your replies.
@fhueske : The watermark must be aligned with timestamps, and it is the
main reason why watermarks are hold back(right?). Current window join may
output a record with
Github user fhueske commented on the issue:
https://github.com/apache/flink/pull/5094
To add some more context, we decided to also join late records with records
that are still in the cache to have a more complete join result (watermarks
should not define the semantics of the query
Github user xccui commented on the issue:
https://github.com/apache/flink/pull/5094
Hi @hequn8128, let me try to explain this.
1. In current implementation, the join process just relies on the cached
rows instead of the watermarks. Specifically, when receiving a record, the
Github user hequn8128 commented on the issue:
https://github.com/apache/flink/pull/5094
Hi @xccui , thanks for your reply. Feel free to take it if you wish. I
still have some confusions.
1. Considering the test `testRowTimeJoinWithCommonBounds2` in
`JoinHarnessTest`, do you mean
Github user xccui commented on the issue:
https://github.com/apache/flink/pull/5094
Hi @hequn8128, thanks for looking into this.
I've checked the current implementation and found that it really may emit
late data. However, that was caused by the checkings below:
12 matches
Mail list logo
