matriv commented on pull request #18058:
URL: https://github.com/apache/flink/pull/18058#issuecomment-999406346
> > > Ah, I forgot, I think we should document explicitly the potential
issues regarding the update/delete behaviour.
> >
> >
> > Or even don't allow other than Insert only streams in this case.
>
> It's a good point, this is an existed issue for all dynamic index pattern,
for example the changed field is a computed column like `ts as proctime()` in
update_before and update after and which is used in
`dynamic-index-{ts|yyyy-MM-dd}`, its hard to avoid because Flink SQL treats a
update event as two records rather than single record and the return value of
`proctime()`/`now()` is nondeterministic, the later one is more relative. I
think giving a limitation tips about this case is more useful comparing to only
allow append-only stream. @matriv HDYT?
Imho, a limitation beforehand makes it more clear, and prevents from
surprises when/if this happens. As you said, if for example the pattern is not
daily but monthly, then a user can have a smoothly running system until the
month changes, and would come up with surprises, and struggle to recover
(erroneous updates), potentially on a production system.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
