Re: Drafting a roadmap for Flink

2015-01-08 Thread Fabian Hueske
Hi, I think the roadmap should show the long-term development goals of Flink, i.e., show that we are going for a ML library, SQL support, Batch-Streaming integration, etc. Right now, it is quite detailed and with very optimistic time estimates, IMO. If we would do everything in time, we would be

[jira] [Created] (FLINK-1370) Build website automatically

2015-01-08 Thread Robert Metzger (JIRA)
Robert Metzger created FLINK-1370: - Summary: Build website automatically Key: FLINK-1370 URL: https://issues.apache.org/jira/browse/FLINK-1370 Project: Flink Issue Type: Task Compon

Re: Drafting a roadmap for Flink

2015-01-08 Thread Robert Metzger
Hi, I very much like the "PIG Journal" here: https://cwiki.apache.org/confluence/display/PIG/Pig+Journal Its basically a nice view (however outdated in that case) on whats going on in the PIG community. You can see finished features on the top, current features being developed in the middle and id

Re: Drafting a roadmap for Flink

2015-01-08 Thread Stephan Ewen
The Pig Journal is nice, we could have this evolve into something like that. I think that we need to give some time estimate on the features / issues. Otherwise, it is of rather little value - all it says is that people thought about that, no one knows when you can plan with it. We can coarsen th

Re: Drafting a roadmap for Flink

2015-01-08 Thread Fabian Hueske
Having an estimate assigned to an issue might give the impression that it is already assigned to somebody. This would not help to find external contributors who are interested in helping with a certain feature. Issues without estimates are still useful as they show in which direction the project p

Re: Drafting a roadmap for Flink

2015-01-08 Thread Stephan Ewen
Okay, I see what you are going. Some issues have people working on them, some do not. How about we add a "responsible person" to the items that have someone in charge already, and mark others as open? Associating a responsible person (that need not be the one that does all the work, but the one t

Re: Drafting a roadmap for Flink

2015-01-08 Thread Fabian Hueske
Adding a responsible person sounds good. We should make sure that this role is clearly communicated though. How about adding priorities instead of time estimates? That would help to see how the priorities are set in Flink and which features to expect next (without having a date assigned to it thou

Re: [VOTE] Release Apache Flink 0.8.0 (RC1)

2015-01-08 Thread Stephan Ewen
After the dependencies are cleaned up (Robert's pull request https://github.com/apache/flink/pull/268 solves that), we need to also do the following: - Make a pass over dependencies to ensure that NOTICE and LICENSE files are up to date (both source and bin distribution) - Adjust the variables i

Re: [VOTE] Release Apache Flink 0.8.0 (RC1)

2015-01-08 Thread Robert Metzger
+1 for merging pull request 268. It makes Flink even more compatible with hadoop. regarding the documentation: I already updated the build instructions. But we should double check everything again. I probably won't have time to look into this today however. On Thu, Jan 8, 2015 at 11:50 AM, Steph

[jira] [Created] (FLINK-1371) KryoSerializer swallows/wraps IOExceptions

2015-01-08 Thread Stephan Ewen (JIRA)
Stephan Ewen created FLINK-1371: --- Summary: KryoSerializer swallows/wraps IOExceptions Key: FLINK-1371 URL: https://issues.apache.org/jira/browse/FLINK-1371 Project: Flink Issue Type: Bug

Re: Kicking off the Machine Learning Library

2015-01-08 Thread Till Rohrmann
Great Henry, Sparkling Water looks really interesting. We would probably have to take a similar approach to make Flink interact with H2O. I briefly looked into the code and that's how they do it (or at least how I understood it ;-). Correct me if I got it wrong: They first start up a Spark clust

Re: [VOTE] Release Apache Flink 0.8.0 (RC1)

2015-01-08 Thread Stephan Ewen
We just found and fixed another highly critical bug. I vote to also cherry pick commit 19066b520435528e104a69ccf372f56811123ee3 ([FLINK-1371] [runtime] Fix KryoSerializer to not swallow EOFExceptions) On Thu, Jan 8, 2015 at 11:55 AM, Robert Metzger wrote: > +1 for merging pull request 268. It

[jira] [Created] (FLINK-1372) TaskManager and JobManager do not log startup settings any more

2015-01-08 Thread Stephan Ewen (JIRA)
Stephan Ewen created FLINK-1372: --- Summary: TaskManager and JobManager do not log startup settings any more Key: FLINK-1372 URL: https://issues.apache.org/jira/browse/FLINK-1372 Project: Flink

Re: [VOTE] Release Apache Flink 0.8.0 (RC1)

2015-01-08 Thread Márton Balassi
+1 for merging PR 268, volunteering for the merge. It should also solve the issue mentioned by Stephan with the yarn uberjar. Updating the release scripts and doing a double-check on the cosmetics. On Thu, Jan 8, 2015 at 12:55 PM, Stephan Ewen wrote: > We just found and fixed another highly cri

Re: [VOTE] Release Apache Flink 0.8.0 (RC1)

2015-01-08 Thread Robert Metzger
Please wait with merging 268. There is still a minor open issue. On Thu, Jan 8, 2015 at 1:28 PM, Márton Balassi wrote: > +1 for merging PR 268, volunteering for the merge. It should also solve the > issue mentioned by Stephan with the yarn uberjar. > > Updating the release scripts and doing a do

Re: [VOTE] Release Apache Flink 0.8.0 (RC1)

2015-01-08 Thread Stephan Ewen
Robert, can you let us know once the PR is merged? Then we can start with the next steps... On Thu, Jan 8, 2015 at 2:07 PM, Robert Metzger wrote: > Please wait with merging 268. There is still a minor open issue. > > On Thu, Jan 8, 2015 at 1:28 PM, Márton Balassi > wrote: > > > +1 for mergin

Re: [VOTE] Release Apache Flink 0.8.0 (RC1)

2015-01-08 Thread Robert Metzger
Allright. The travis tests are green and I tested it again with Tachyon on a cluster. My pull request also fixes some of the issues mentioned earlier in this thread by Stephan (the warnings from shading regarding duplicate classes). I can not fix all of them because some projects like hadoop just

Re: [VOTE] Release Apache Flink 0.8.0 (RC1)

2015-01-08 Thread Stephan Ewen
Nice. @Marton: As soon as as you are done, I make a pass over the licenses... Stephan On Thu, Jan 8, 2015 at 4:42 PM, Robert Metzger wrote: > Allright. The travis tests are green and I tested it again with Tachyon on > a cluster. > > My pull request also fixes some of the issues mentioned ear

[jira] [Created] (FLINK-1373) Add documentation for ntermediate results and network stack to internals

2015-01-08 Thread Ufuk Celebi (JIRA)
Ufuk Celebi created FLINK-1373: -- Summary: Add documentation for ntermediate results and network stack to internals Key: FLINK-1373 URL: https://issues.apache.org/jira/browse/FLINK-1373 Project: Flink

[jira] [Created] (FLINK-1374) Please delete old releases from mirroring system

2015-01-08 Thread Sebb (JIRA)
Sebb created FLINK-1374: --- Summary: Please delete old releases from mirroring system Key: FLINK-1374 URL: https://issues.apache.org/jira/browse/FLINK-1374 Project: Flink Issue Type: Task Enviro