Re: A small Project I've been working on

2015-04-01 Thread Ufuk Celebi
Little side projects ftw. Very nice :-) Can you give some points on how this works internally? Is it making use of anything generic from the Python API pull request? On Wednesday, April 1, 2015, Márton Balassi balassi.mar...@gmail.com wrote: Woot! On Wed, Apr 1, 2015 at 9:01 AM, Aljoscha

Re: A small Project I've been working on

2015-04-01 Thread Stephan Ewen
It April, 1st, right? On Wed, Apr 1, 2015 at 9:47 AM, Gyula Fóra gyf...@apache.org wrote: Amazing! :) On Wed, Apr 1, 2015 at 9:41 AM, fhue...@gmail.com wrote: My ruby skills are a bit rust(y) but I’d love to contribute. Can you point me to a repository that I can fork?

Re: [DISCUSS] Inconsistent naming of intermediate results

2015-04-01 Thread Ufuk Celebi
To summarize so far: all are in favor of a rename. I agree with both of Henry's points regarding the docs. @Stephan: what would you suggest? I would trust your gut feeling on this one. ;) JobResult, ExecutionJobResult, ExecutionResult, etc.? On Tue, Mar 31, 2015 at 8:16 PM, Henry Saputra

Re: A small Project I've been working on

2015-04-01 Thread Aljoscha Krettek
Right now, runtime is roughly thrice that of equivalent java programs. But I plan on bringing that to the same ballpark using code generation. On Wed, Apr 1, 2015 at 8:54 AM, Fabian Hueske fhue...@gmail.com wrote: :-D This is awesome! Do you have some performance numbers? On Apr 1, 2015 8:43

A small Project I've been working on

2015-04-01 Thread Aljoscha Krettek
Hi, I've been working on a little side project in my free time: Ruby on Flink (RoF). This should finally allow us to tap into the whole web developer ruby world. The design is also flexible enough to easily port this to node.js. The code is not yet in shape to be made public but this is a small

Re: A small Project I've been working on

2015-04-01 Thread Fabian Hueske
:-D This is awesome! Do you have some performance numbers? On Apr 1, 2015 8:43 AM, Aljoscha Krettek aljos...@apache.org wrote: Hi, I've been working on a little side project in my free time: Ruby on Flink (RoF). This should finally allow us to tap into the whole web developer ruby world. The

Re: A small Project I've been working on

2015-04-01 Thread fhueske
My ruby skills are a bit rust(y) but I’d love to contribute. Can you point me to a repository that I can fork? From: Till Rohrmann Sent: ‎Wednesday‎, ‎1‎. ‎April‎, ‎2015 ‎09‎:‎29 To: dev@flink.apache.org Where can I start contributing. I've got the feeling that it will be inherently

Re: A small Project I've been working on

2015-04-01 Thread Stephan Ewen
For the Ruby interpreter on Flink, I suggest to implement this in Rust. To implement a system that analyzes data at web scale, what would be a better language than one used to implement the engine of a web browser. On Wed, Apr 1, 2015 at 10:05 AM, Stephan Ewen se...@apache.org wrote: It April,

Re: A small Project I've been working on

2015-04-01 Thread Márton Balassi
Woot! On Wed, Apr 1, 2015 at 9:01 AM, Aljoscha Krettek aljos...@apache.org wrote: Right now, runtime is roughly thrice that of equivalent java programs. But I plan on bringing that to the same ballpark using code generation. On Wed, Apr 1, 2015 at 8:54 AM, Fabian Hueske fhue...@gmail.com

Re: A small Project I've been working on

2015-04-01 Thread Maximilian Michels
I'm in love with the API. So innovative and it feels very Rubyesk, even though I'm a Python lover. Your WordCount shows some really nice features which we should port to the Flink Java and Scala API. Perhaps, by embedding a little Ruby interpreter. Great idea also to port it to Node.js. Together

Re: Make docs searchable

2015-04-01 Thread Till Rohrmann
I also like the idea. +1 On Wed, Apr 1, 2015 at 12:20 PM, Robert Metzger rmetz...@apache.org wrote: Cool. I would like to have the ability to search the docs, so +1 for this idea! On Wed, Apr 1, 2015 at 12:10 PM, Ufuk Celebi u...@apache.org wrote: Hey all, I think our documentation

Re: [DISCUSS] Gelly iteration abstractions

2015-04-01 Thread Stephan Ewen
Wow, Vasia, a GC limit error in the Optimizer (pre-flight), not the runtime. That is cannot have been easy to produce ;-) The program does not really look that tricky, I need to take some closer look at this... BTW: Before this algorithm runs well, we really need pinning intermediate results in

Re: Question about Infinite Streaming Job on Mini Cluster and ITCase

2015-04-01 Thread Márton Balassi
Hey Matthias, Thanks for reporting the Exception thrown, we were not preparing for this use case yet. We fixed it with Gyula, he is pushing a fix for it right now: When the job is cancelled (for example due to shutting down the executor underneath) you should not see that InterruptedException as

Make docs searchable

2015-04-01 Thread Ufuk Celebi
Hey all, I think our documentation has grown to a point where we need to think about how to make it more accessible. I would like to add a custom Google search limited to the docs *excluding* the API docs (otherwise the results are very noisy). Mocks: http://tinypic.com/view.php?pic=2qtlg1ks=8

Re: Make docs searchable

2015-04-01 Thread Robert Metzger
Cool. I would like to have the ability to search the docs, so +1 for this idea! On Wed, Apr 1, 2015 at 12:10 PM, Ufuk Celebi u...@apache.org wrote: Hey all, I think our documentation has grown to a point where we need to think about how to make it more accessible. I would like to add a

Re: Question about Infinite Streaming Job on Mini Cluster and ITCase

2015-04-01 Thread Stephan Ewen
As a followup - I think it would be a good thing to add a way to gracefully stop a streaming job. Something that sends close to the sources, and they quit. We can use this for graceful shutdown wen re-partitioninig / scaling in or out, ... On Wed, Apr 1, 2015 at 1:29 PM, Matthias J. Sax

Re: HBase TableOutputFormat fix (Flink 0.8.1)

2015-04-01 Thread Flavio Pompermaier
Ok..I'd like to have this fix in the next release. Should I branch Flink 0.8.1 or 0.9 or which version? On Wed, Apr 1, 2015 at 2:04 PM, Maximilian Michels m...@apache.org wrote: Hi Flavio, Thanks for looking into this problem. Actually, it's a bit difficult to discuss your changes here

Re: HBase TableOutputFormat fix (Flink 0.8.1)

2015-04-01 Thread Fabian Hueske
What ever works best for you. We can easily backport or forwardport the patch. 2015-04-01 14:12 GMT+02:00 Flavio Pompermaier pomperma...@okkam.it: Ok..I'd like to have this fix in the next release. Should I branch Flink 0.8.1 or 0.9 or which version? On Wed, Apr 1, 2015 at 2:04 PM,

Re: HBase TableOutputFormat fix (Flink 0.8.1)

2015-04-01 Thread Maximilian Michels
Just base your changes on the current master. On Wed, Apr 1, 2015 at 2:12 PM, Flavio Pompermaier pomperma...@okkam.it wrote: Ok..I'd like to have this fix in the next release. Should I branch Flink 0.8.1 or 0.9 or which version? On Wed, Apr 1, 2015 at 2:04 PM, Maximilian Michels

Re: A small Project I've been working on

2015-04-01 Thread Gyula Fóra
Amazing! :) On Wed, Apr 1, 2015 at 9:41 AM, fhue...@gmail.com wrote: My ruby skills are a bit rust(y) but I’d love to contribute. Can you point me to a repository that I can fork? From: Till Rohrmann Sent: ‎Wednesday‎, ‎1‎. ‎April‎, ‎2015 ‎09‎:‎29 To: dev@flink.apache.org

Re: HBase TableOutputFormat fix (Flink 0.8.1)

2015-04-01 Thread Fabian Hueske
As I said before, I think the configure() method of the original HadoopOutputFormat should be called in the configure() method of the Flink HadoopOutputFormatBase. Flink calls configure() before open() and finalizeOnMaster(), so that should work. Have you checked if that fixes your problem? If

Re: Make docs searchable

2015-04-01 Thread Fabian Hueske
+1! Would also be good if we could make the documentation less monolithic. There are some really large pages which would benefit from an in-page search ;-) 2015-04-01 13:57 GMT+02:00 Maximilian Michels m...@apache.org: +1 Nice idea! We would just have to filter the /api/java and /api/scala

[jira] [Created] (FLINK-1810) UpdateTask message contains unused information

2015-04-01 Thread Ufuk Celebi (JIRA)
Ufuk Celebi created FLINK-1810: -- Summary: UpdateTask message contains unused information Key: FLINK-1810 URL: https://issues.apache.org/jira/browse/FLINK-1810 Project: Flink Issue Type:

Re: Make docs searchable

2015-04-01 Thread Stephan Ewen
big +1 On Wed, Apr 1, 2015 at 12:25 PM, Till Rohrmann trohrm...@apache.org wrote: I also like the idea. +1 On Wed, Apr 1, 2015 at 12:20 PM, Robert Metzger rmetz...@apache.org wrote: Cool. I would like to have the ability to search the docs, so +1 for this idea! On Wed, Apr 1, 2015

Re: HBase TableOutputFormat fix (Flink 0.8.1)

2015-04-01 Thread Flavio Pompermaier
Any feedback about this? On Tue, Mar 31, 2015 at 7:07 PM, Flavio Pompermaier pomperma...@okkam.it wrote: Hi Flink devs, this is my final report about the HBaseOutputFormat problem (with Flink 0.8.1) and I hope you could suggest me the best way to make a PR: 1) The following code produce the

Re: [QUESTION] In the DataExchangeMode.BATCH how does the consumer being notified to start consumer the data?

2015-04-01 Thread Henry Saputra
Ah cool. Thanks for the explanation, Ufuk. Really helpful. - Henry On Tuesday, March 31, 2015, Ufuk Celebi u...@apache.org wrote: Hey Henry, 1) There is no extra message, but this is piggy backed with the finished state transition (see Execution#markAsFinished). There it is essentially the

Re: [DISCUSS] Gelly iteration abstractions

2015-04-01 Thread Vasiliki Kalavri
Hey, Wow, Vasia, a GC limit error in the Optimizer (pre-flight), not the runtime. That is cannot have been easy to produce ;-) The program does not really look that tricky, I need to take some closer look at this... ​It was pretty easy actually :P The program is really simple: in each

[jira] [Created] (FLINK-1813) Barrier broadcast sometimes causes illegal state exception in source RecordWriter

2015-04-01 Thread Gyula Fora (JIRA)
Gyula Fora created FLINK-1813: - Summary: Barrier broadcast sometimes causes illegal state exception in source RecordWriter Key: FLINK-1813 URL: https://issues.apache.org/jira/browse/FLINK-1813 Project: