GitHub user ktzoumas opened a pull request:
https://github.com/apache/incubator-flink/pull/85
Java api functions to sams
This is FLINK-701. This is not good to merge yet, I am putting it out for
comments and assistance. However, it should be part of release 0.6 as it breaks
compatibility.
The patch changes the Java and Record APIs to work on top of SAM (Single
Abstract Method) interfaces rather than abstract classes. The SAMs are named
Fooable for an operation Foo (e.g., Mappable for Map, Reducible for Reduce),
and they replace the former GenericFooer (e.g., GenericMapper) interfaces.
The original "rich functions" still exist and work as usual, as they
implement hte aforementioned interfaces. They are called FooFunction (e.g.,
MapFunction, ReduceFunction), and contain the open(), close(), etc methods as
well.
As part of the refactoring, Cross was changed to return exactly one value
rather than taking a collector as input.
GenericCombiner is renamed to FlatCombinable (general naming rule:
FlatFooable takes a Collector as parameter and does not return a value, Fooable
returns exactly one value if both interfaces
exist). This PR does *not* add a Combinable and does *not* solve FLINK-848,
these can be added later without breaking compatibility.
This PR does add an explicit FlatJoinable that is at the same level of
inheritance as Joiable. The runtime works only on FlatJoinable objects,
Joinables are shallowly transformed to FlatJoinables (see
GeneratedFlatJoinFunction).
Two consequences:
(1) FlatCross is removed from the Scala API, as it cannot be supported by
the new Cross signature. This is an API design choice that cannot be rectified
immediately.
(2) As a side-effect, this PR does add support for Java 8 lambdas in the
filter and reduce operators. Lambdas in the other operators do not work yet, as
the current TypeExctractor implementation needs to be adapted to extract the
return types of lambdas (filter and reduce have both known return types). With
that solved, FLINK-612 will probably be resolved as well.
Things that need to be fixed:
(1) Currently, several POM files require Java 8. This will be changed. The
goal is to have some lambda tests that are executed only if Java 8 is present.
(2) If a lambda function is currently provided as an argument to a method
other than filter or reduce, an error is displayed to the user. Detecting that
the input is a lambda is currently done via string matching on the function
name (see FunctionUtils.isLambdaFunction) which (a) is quite iffy, and (b)
might not work with all JVMSs/break in several occasions.
If someone knows a good way to "detect if something is a lambda", that
would be great! The internet was not too much help until now.
(3) Could I have an extra set of eyes on RegularPactTask, line 531?
(4) DeltaIterationTranslationTest currently does not pass, this is simple
fix, I will do this asap.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/ktzoumas/incubator-flink
java_api_functions_to_sams
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/incubator-flink/pull/85.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #85
----
commit 9bdc93166a9dfc25b0e83cb86769151281663563
Author: Kostas Tzoumas <[email protected]>
Date: 2014-07-18T13:37:19Z
Renamed of Function to RichFunction, created empty Function interface
commit d8bdd90bc35446d65d8a0bbb3a75daa23f9f5b2a
Author: Kostas Tzoumas <[email protected]>
Date: 2014-07-18T13:37:26Z
Renamed of Function to RichFunction, created empty Function interface
commit 56b27cc46665fcbc5cd69bfeb19fe695c89d64a3
Author: Kostas Tzoumas <[email protected]>
Date: 2014-07-18T14:48:48Z
changes to wrapping function
commit 2eb3aa8a7d301fa5f52c0311ee5cb4fd9920065d
Author: Kostas Tzoumas <[email protected]>
Date: 2014-07-19T11:22:38Z
map and reduce operators
commit 762c33b37f78e82af76690a8b6e188c361b02f15
Author: Kostas Tzoumas <[email protected]>
Date: 2014-07-23T13:39:21Z
group reduce refactoring
commit d041d15cc16c16ea026d7b73673b0a592ba545d7
Author: Kostas Tzoumas <[email protected]>
Date: 2014-07-24T09:34:41Z
group reduce as interface -- shallow implementation
commit aed25492c994c200a50ff4972abb274fc0387c4e
Author: Kostas Tzoumas <[email protected]>
Date: 2014-07-29T16:34:26Z
Java and record API based on SAM interfaces
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
