[
https://issues.apache.org/jira/browse/CALCITE-3509?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17020527#comment-17020527
]
Rui Wang edited comment on CALCITE-3509 at 1/21/20 7:56 PM:
------------------------------------------------------------
I am trying to propose something about EMIT syntax in dev@. If that idea looks
good then watermark propagation will become an implementation detail that
engines will need to implement their own version of "current_watermark()"
function, which will make this Jira very trivial.
was (Author: amaliujia):
I am trying to proposal something about EMIT syntax in dev@. If that idea looks
good then watermark propagation will become an implementation detail that
engines will need to implement their own version of "current_watermark()"
function, which will make this Jira very trivial.
> Watermark Propagation
> ----------------------
>
> Key: CALCITE-3509
> URL: https://issues.apache.org/jira/browse/CALCITE-3509
> Project: Calcite
> Issue Type: Sub-task
> Reporter: Rui Wang
> Priority: Major
>
> To have a useful enumerable implementation for streaming windowing semantic
> in Calcite, and also make Emit syntax useful, we will need to introduce
> watermark into Calcite. More specifically, the following should be discussed:
> 1. How is watermark generated? What changes are required on source/table scan?
> 2. How does watermark propagate through the whole plan? How are watermarks
> from multiple inputs aggregated together?
> 3. Where watermark is used to control the time to emit data?
> 4. How are event timestamp combined per key (typical in aggregation/join).
> From Julian:
> 0. Can watermarks be handled using algebraic rewrites? I.e. can we use the
> usual operators but restart them, giving them just the rows that changed,
> when watermarks arrive.
> This is consistent with Calcite's general approach, which is to solve things
> at a high level using algebra rather than dive into the details of a
> particular kind of engine.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
