[
https://issues.apache.org/jira/browse/FLINK-5955?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15893780#comment-15893780
]
ASF GitHub Bot commented on FLINK-5955:
---------------------------------------
GitHub user shaoxuan-wang opened a pull request:
https://github.com/apache/flink/pull/3464
[FLINK-5955] [table] Merging a list of buffered records will have problem
when ObjectReuse is turned on
This PR changes the dataSet AGG merge to pair-merge.
If we buffer the iterated records for group-merge, we will get wrong error
when ObjectReuse is turned on. Alternatively, we could deep-copy every record
and buffer them for group-merge. But I think that is expense in terms of memory
and also CPU. We could later add group-merge when needed (in the future we
should add rules to select either pair-merge or group-merge, but for now all
built-in aggregates should work fine with pair-merge).
Thanks for contributing to Apache Flink. Before you open your pull request,
please take the following check list into consideration.
If your changes take all of the items into account, feel free to open your
pull request. For more information and/or questions please refer to the [How To
Contribute guide](http://flink.apache.org/how-to-contribute.html).
In addition to going through the list, please provide a meaningful
description of your changes.
- [X] General
- The pull request references the related JIRA issue ("[FLINK-XXX] Jira
title text")
- The pull request addresses only one issue
- Each commit in the PR has a meaningful commit message (including the
JIRA id)
- [ ] Documentation
- Documentation has been added for new functionality
- Old documentation affected by the pull request has been updated
- JavaDoc for public methods has been added
- [X] Tests & Build
- Functionality added by the pull request is covered by tests
- `mvn clean verify` has been executed successfully locally or a Travis
build has passed
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/shaoxuan-wang/flink F5955-submit
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/flink/pull/3464.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #3464
----
commit e6cdab7cd309f16d028894943f177f4321889630
Author: shaoxuan-wang <[email protected]>
Date: 2017-03-03T05:50:29Z
[FLINK-5955] [table] Merging a list of buffered records will have problem
when ObjectReuse is turned on
----
> Merging a list of buffered records will have problem when ObjectReuse is
> turned on
> ----------------------------------------------------------------------------------
>
> Key: FLINK-5955
> URL: https://issues.apache.org/jira/browse/FLINK-5955
> Project: Flink
> Issue Type: Bug
> Components: Table API & SQL
> Reporter: Shaoxuan Wang
> Assignee: Shaoxuan Wang
>
> Turn on ObjectReuse in MultipleProgramsTestBase:
> TestEnvironment clusterEnv = new TestEnvironment(cluster, 4, true);
> Then the tests "testEventTimeSessionGroupWindow",
> "testEventTimeSessionGroupWindow", and
> "testEventTimeTumblingGroupWindowOverTime" will fail.
> The reason is that we have buffered iterated records for group-merge. I think
> we should change the Agg merge to pair-merge, and later add group-merge when
> needed (in the future we should add rules to select either pair-merge or
> group-merge, but for now all built-in aggregates should work fine with
> pair-merge).
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
