Github user sunjincheng121 commented on the issue:
https://github.com/apache/flink/pull/3765
Hi @haohui @fhueske I am very interested in `DISTINCT`, Let me share some
ideas about this:
First up, in standard database there are two situations can using
`DISTINCT` keyword.
* in `SELECT Clause`, e.g.: `SELECT DISTINCT name FROM table`
* in `AGG Clause`, e.g.: `COUNT([ALL|DISTINCT]
expression)`,`SUM([ALL|DISTINCT] expression)`, etc.
In this post,we talk about `AGG Clause`. The `DISTINCT` keyword tells the
database system to aggregate only the distinct, or unique, values within the
scope of the aggregate function. i.e. database system will deal with the
`DISTINCT` keyword, and put the unique value into `AGG`. Based on this
understanding, I think FLINK FRAMEWORK(not the AGG) should deal with the
`DISTINCT` keyword. we do not need `DistinctAccumulator.java`. About GROUP
WINDOW, I think I like analyze whether the data is duplicated in
`XXXWindowFunction` and `DataSetXXXAggFunction`, And add boolean variable
`isFirstTimeProcess` identifies whether the data is duplicated as a parameter
of `GeneratedAggregationsFunction`. `GeneratedAggregationsFunction` process
data according to `aggCall.isDistinct` and `isFirstTimeProcess`. What do you
think? @haohui @fhueske
Best,
SunJincheng
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
