hackergin commented on pull request #18058: URL: https://github.com/apache/flink/pull/18058#issuecomment-999481611
> > > > Ah, I forgot, I think we should document explicitly the potential issues regarding the update/delete behaviour. > > > > > > > > > Or even don't allow other than Insert only streams in this case. > > > > > > Indeed, this can avoid some errors, but if we support meta-column, do we have to do the same restriction? At first glance, it's not easy to implement. > > This is independent of the metadata column, and is a generic issue with dynamic index names based on proctime, which is the time a record reaches the connector, so it's not deterministic if the event will fall onto the current or the next index, when tries to update a previous record. Yes, this problem may exist for all dynamic indexes generated based on proctime. The point is, we seem to be unable to distinguish whether a field uses proctime. Therefore, if we want to do this restriction, we may need to prohibit all dynamic indexes in non-append only datastream. But I think the restriction is necessary to avoid causing users to encounter unexpected and unreproducible problems. Since the current dynamic index also has this problem, I think we can create a new jira to fix this problem. 1. Do we need to prohibit all dynamic indexes in non-append-only stream? 2. Do we need to create a new jira/issue to fix the problem? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
