[
https://issues.apache.org/jira/browse/FLINK-4265?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15402174#comment-15402174
]
ASF GitHub Bot commented on FLINK-4265:
---------------------------------------
Github user greghogan commented on the issue:
https://github.com/apache/flink/pull/2294
This would replace `Delegate` and is much simpler. I placed it in the Java
API package but it would not be visible to users through the `DataSet` API.
The ultimate goal is to implicitly reuse operations performing the same
computation on the same input `DataSet`. The optimizer cannot do this without
understanding the UDF configuration and logic. Two instances of a UDF may be
* incompatible, in which case a new result must be computed
* the same, in which case the old result can be reused
* different but compatible, in which case the UDF can merge the
configurations and the new, shared result replaces the old result
Replacing the old result requires a wrapper. Using `ProxyFactory` in
`Delegate` has limitations as documented in FLINK-4257. With a `NoOpOperator`
we can perform the same function by appending and then ignoring a dummy POJO
operator.
> Add a NoOpOperator
> ------------------
>
> Key: FLINK-4265
> URL: https://issues.apache.org/jira/browse/FLINK-4265
> Project: Flink
> Issue Type: New Feature
> Components: DataSet API
> Affects Versions: 1.1.0
> Reporter: Greg Hogan
> Assignee: Greg Hogan
> Priority: Minor
> Fix For: 1.1.0
>
>
> One recent feature of Gelly is algorithms which detect duplicated or similar
> computation which can be shared. My initial implementation could only reuse a
> {{DataSet}} result. Before committing to Flink this was updated to use a
> javassist {{ProxyFactory}} allowing configuration to be merged and results to
> be replaced. There were some issues, as identified in FLINK-4257. With a
> {{NoOpOperator}} we can remove the use of {{ProxyFactory}} and resolve the
> identified issues.
> This ticket adds a {{NoOpOperator}} which is unwound in
> {{OperatorTranslation.translate}}. The {{NoOpOperator}} contains a
> {{DataSet}} which is accessed by a getter and setter.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
