Hi Manu,
awesome !
Thanks for sharing this !
I will take a look.
Regards
JB
On 04/08/2016 03:38 AM, Manu Zhang wrote:
Hi JB,
- it seems the Gearpump Serializer (based on Kryo) can be expressed as a
Coder. You mean because you have only binary "input", whereas the Coder
needs a specific type. Right ?
Yes. I'm thinking about adding a typed serializer interface to Gearpump and
the default serializer based on Kryo could be type of "Any" in Scala. Then
it's trivial to add support for Coder.
- Any plan for the side inputs (I don't know if we can do it easily in
Gearpump) ?
Certainly, it's in our plan. Off the top of my head, Gearpump is built on
Akka Actors and side inputs could be actor states which we can query with
messages. Any good ideas about how a do it in a streaming engine (not
specifically on Gearpump) ?
The code is at
https://github.com/manuzhang/incubator-beam/tree/gearpump_runner. Nothing
is working yet and I still need to fix styles, add docs and tests. The
focus now is to get a working example first.
I also created an umbrella jira (
https://issues.apache.org/jira/browse/GEARPUMP-21) to track status and
related issues at Gearpump side and notify the Gearpump community.
Regards,
Manu
On Thu, Apr 7, 2016 at 1:45 PM, Jean-Baptiste Onofré <[email protected]>
wrote:
Hi Manu,
I replied in the Jira.
It sounds good to me.
Just a question about the gaps:
- it seems the Gearpump Serializer (based on Kryo) can be expressed as a
Coder. You mean because you have only binary "input", whereas the Coder
needs a specific type. Right ?
- Any plan for the side inputs (I don't know if we can do it easily in
Gearpump) ?
Is it possible to see the code in progress ?
Thanks !
Regards
JB
On 04/07/2016 06:24 AM, Manu Zhang wrote:
Hi all,
I've drafted a design document
<
https://drive.google.com/open?id=1nw64QUWVfT8L7FUprPGLEeNjSBpDMkn1otfLt2rHM5g
for
Gearpump Runner (https://issues.apache.org/jira/browse/BEAM-79). Please
kindly offer your comment.
Thanks,
Manu
--
Jean-Baptiste Onofré
[email protected]
http://blog.nanthrax.net
Talend - http://www.talend.com
--
Jean-Baptiste Onofré
[email protected]
http://blog.nanthrax.net
Talend - http://www.talend.com
