New Flink Runtime Monitor Dashboard

2015-07-22 Thread Stephan Ewen
Hi all! Yesterday, we merged the framework for the new Flink runtime monitor dashboard. New stub, based on angular.js and coffee script. https://github.com/apache/flink/tree/master/flink-runtime-web Anyone who is savvy with these technologies and wants to work on that part, please ping! We'd be

[jira] [Created] (FLINK-2393) Add a stateless at-least-once mode for streaming

2015-07-22 Thread Stephan Ewen (JIRA)
Stephan Ewen created FLINK-2393: --- Summary: Add a stateless at-least-once mode for streaming Key: FLINK-2393 URL: https://issues.apache.org/jira/browse/FLINK-2393 Project: Flink Issue Type: New

Windowing Design Document

2015-07-22 Thread Aljoscha Krettek
Hi All, I added a new document along the existing design documents in the wiki: https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=60624830. It's very bare-bones right now but we can flesh it out together. I put some requirements that I think are good but please add other stuff or

Any planning for Apache Flink 1.0 release?

2015-07-22 Thread Slim Baltagi
Hi Once DataStream API is out of beta and both DataStream and DataSet API are stabilized, will we be seeing Flink 1.0 release? On April 28, 2014, Fabian Hueske personal guess of Flink 1.0 release in his info.com interview was April 2016? Reference:

Re: New Flink Runtime Monitor Dashboard

2015-07-22 Thread Kostas Tzoumas
Finally, Flink is a JavaScript project :-) On Wed, Jul 22, 2015 at 6:22 PM, Stephan Ewen se...@apache.org wrote: Hi all! Yesterday, we merged the framework for the new Flink runtime monitor dashboard. New stub, based on angular.js and coffee script.

Theoretical complexity of a coGroup

2015-07-22 Thread Andra Lungu
Hi everyone, I am not 100% sure about this one, so I thought that I could set my thoughts straight via the mailing list. Here's the use case. You coGroup a data set of vertices with a data set of edges. That gives you a complexity of* O(|V| * |E|)*, where |V| is the total number of vertices and

Re: serialization issue

2015-07-22 Thread Robert Metzger
Were you able to resolve the issue? I first thought its an JVM-version incompatibility, but this thread suggests a serialization issue: http://stackoverflow.com/questions/851491/java-deserialization-of-java-lang-integer-exception By the way: your emails to the dev@ list sometimes end up in the

Re: Theoretical complexity of a coGroup

2015-07-22 Thread Kostas Tzoumas
Hi Andra, CoGroup is not implemented as a Cartesian product, so O(V*E) is not a very accurate approximation. All this depends on what you count. Let's assume single-node execution and that everything fits in memory, and let's count comparisons and UDFs on groups. Then, coGroup sorts both

Re: serialization issue

2015-07-22 Thread Stephan Ewen
Hmmm, this seams like the stream is read differently than it is written. Maybe one of the involved classes overrides the readObject() and writeObject() methods, and implements them inconsistently. And you see if that is the case in one of your data types? On Wed, Jul 22, 2015 at 11:02 AM, Robert

[jira] [Created] (FLINK-2392) Instable test in flink-yarn-tests

2015-07-22 Thread Matthias J. Sax (JIRA)
Matthias J. Sax created FLINK-2392: -- Summary: Instable test in flink-yarn-tests Key: FLINK-2392 URL: https://issues.apache.org/jira/browse/FLINK-2392 Project: Flink Issue Type: Bug

Re: New Flink Runtime Monitor Dashboard

2015-07-22 Thread Matthias J. Sax
Haha! On 07/22/2015 07:02 PM, Kostas Tzoumas wrote: Finally, Flink is a JavaScript project :-) On Wed, Jul 22, 2015 at 6:22 PM, Stephan Ewen se...@apache.org wrote: Hi all! Yesterday, we merged the framework for the new Flink runtime monitor dashboard. New stub, based on angular.js and