GitHub user walterddr opened a pull request:
https://github.com/apache/flink/pull/5555
[FLINK-8689][table]Add runtime support of distinct filter using MapView for
codegen
## What is the purpose of the change
- Adding in runtime support using distinct aggregation delegate to support
distinct filtering using MapView. This is only fixing the currently broken
over-window aggregate with distinct filter, however this change is meant to be
used by other distinct operations on datastream such as group window aggregate,
group aggregate, etc.
## Brief change log
- Adding a new DistinctAggDelegateFunction and DistinctAccumulator that
encapsulates any real aggregate function.
- change codegen to specifically generate distinct filter before invoking
the actual aggregate function.
- adding in more codegen specifically for using delegates on merge and
reset.
## Verifying this change
This change added tests and can be verified as follows:
- Added over-window unit-test to verify generated plan before codegen.
- Added integration tests for testing distinct over-window aggregate
end-to-end.
## Does this pull request potentially affect one of the following parts:
no
## Documentation
this does not expose and additional external facing functionality, which
will come in separated PR.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/walterddr/flink FLINK-8689
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/flink/pull/5555.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #5555
----
commit cb853e6d1adfbb04819a234f8fbced32a8ffb21f
Author: Rong Rong <rongr@...>
Date: 2018-02-21T21:40:29Z
support distinct agg function delegate using mapview accumulator in runtime
environment
commit fff7fa48f07b350f6fd4565f58d6a35af88faa50
Author: Rong Rong <rongr@...>
Date: 2018-02-21T21:43:02Z
add over window aggregate and unbounded aggregate ITCase
commit 03fc641ac01af6867d2516263bcea0ac96f1802c
Author: Rong Rong <walter_ddr@...>
Date: 2018-02-22T05:15:26Z
adding in overwindow logical test cases to verify distinct modifier
----
---
