[
https://issues.apache.org/jira/browse/FLINK-29692?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17623052#comment-17623052
]
Canope Nerda edited comment on FLINK-29692 at 10/24/22 9:47 AM:
----------------------------------------------------------------
[~jingzhang] Let me elaborate my case a little bit.
First of all, it's a case in high frequency trading. We deployed multiple
agents globally to individually collect data from exchanges, so inherently
there are duplicates and each data entry has a field that uniquely identifies
the data from biz perspective, and an event time column in microsecond. The
event time column may vary for entries with the same unique column value. So we
would like to get the entry with the earliest event time for each unique column
value.
was (Author: JIRAUSER297248):
[~jingzhang] Let me elaborate my case a little bit.
First of all, it's a case in high frequency trading. We deployed multiple
agents globally to individually collect data from exchanges, so inherently
there are duplicates and each data entry has a field that uniquely identifies
the data from biz perspective, and an event time column in microsecond. The
event time column may vary for entries with the same unique column value. So we
would like to first the entry with the earliest event time for each unique
column value.
> Support early/late fires for Windowing TVFs
> -------------------------------------------
>
> Key: FLINK-29692
> URL: https://issues.apache.org/jira/browse/FLINK-29692
> Project: Flink
> Issue Type: New Feature
> Components: Table SQL / Planner
> Affects Versions: 1.15.2
> Reporter: Canope Nerda
> Priority: Major
>
> I have cases where I need to 1) output data as soon as possible and 2) handle
> late arriving data to achieve eventual correctness. In the logic, I need to
> do window deduplication which is based on Windowing TVFs and according to
> source code, early/late fires are not supported yet in Windowing TVFs.
> Actually 1) contradicts with 2). Without early/late fires, we had to
> compromise, either live with fresh incorrect data or tolerate excess latency
> for correctness.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
