Re: Restructuring the maven projects

2015-03-17 Thread Stephan Ewen
The good thing about the API projects is that there is no dependency from Java code to Scala code. I think that caused most of the issues. We may still want to keep it separate. I am not fully decided on this yet... Stephan On Tue, Mar 17, 2015 at 3:52 PM, Ufuk Celebi u...@apache.org wrote:

Re: Website documentation minor bug

2015-03-17 Thread Aljoscha Krettek
+1 On Mar 17, 2015 12:28 PM, Maximilian Michels m...@apache.org wrote: Another proposal in addition to the recent changes: How about we remove the fixed attribute from the navigation of the docs? Then the nav will just scroll away and give space for the documentation. After all, how useful is

Re: Introduction of new Gelly developper

2015-03-17 Thread Till Rohrmann
Hi Zhou Yi, welcome to the Flink community. Great to hear that you're gonna work on Gelly. If you have any problems getting started, then let us know. Cheers, Till On Tue, Mar 17, 2015 at 9:22 AM, Stephan Ewen se...@apache.org wrote: Hi Zhou Yi! Welcome to the Flink community. Gelly (and

Blog post about Parallel Joins in Flink - Mechanisms and Performance

2015-03-17 Thread Stephan Ewen
Hello Squirrels! Flink committer Fabian Hueske has written a very nice article about joins in Apache Flink. The article talks about joins in the APIs, the join algorithms, memory management, and performance experiments on a small cluster. A good read for everyone with SQL/ETL-style use cases and

Re: Restructuring the maven projects

2015-03-17 Thread Stephan Ewen
To not let this discussion die, here is a concrete JIRA and a proposed layout to restructure to. What remains to be discusses is whether we want to keep the Scala/Java APIs for batch/streaming in separate projects or in one project. Also, we need to find a good time to do this, when we are low

[jira] [Created] (FLINK-1712) Restructure Maven Projects

2015-03-17 Thread Stephan Ewen (JIRA)
Stephan Ewen created FLINK-1712: --- Summary: Restructure Maven Projects Key: FLINK-1712 URL: https://issues.apache.org/jira/browse/FLINK-1712 Project: Flink Issue Type: Improvement Affects

Re: Introduction of new Gelly developper

2015-03-17 Thread Vasiliki Kalavri
Hello Zhou Yi! welcome to the Flink community! Thanks for your interest in Gelly and for choosing to work on Affinity Propagation. It'll be a great addition to the library ^^ Cheers, Vasia. On 17 March 2015 at 10:41, Till Rohrmann trohrm...@apache.org wrote: Hi Zhou Yi, welcome to the Flink

[jira] [Created] (FLINK-1711) Replace all usages off commons.Validate with guava.check

2015-03-17 Thread Stephan Ewen (JIRA)
Stephan Ewen created FLINK-1711: --- Summary: Replace all usages off commons.Validate with guava.check Key: FLINK-1711 URL: https://issues.apache.org/jira/browse/FLINK-1711 Project: Flink Issue

Re: Restructuring the maven projects

2015-03-17 Thread Márton Balassi
+1 for the proposed structure. I have no explicit preference for having batch and streaming scala together or separated. That said streaming scala is considerably thin, it does not really require an own maven submodule. Marked an older JIRA for the same issue as duplicate. [1] [1]

Re: Improve the documentation of the Flink Architecture and internals

2015-03-17 Thread Kostas Tzoumas
+1 for the Wiki. When these have been stabilized we can move them to the docs if we decide to do so. On Mon, Mar 16, 2015 at 10:07 PM, Stephan Ewen se...@apache.org wrote: I have put my suggested version of an outline for the docs into the wiki. Regardless where the docs end up (wiki or

Re: [Multiple Programs Test Base] Consequence of Shading Guava away

2015-03-17 Thread Robert Metzger
Hey Andra, I've checked out your repository and made some changes. It seems to compile, also the Files thing seems to work (at least thats what IntelliJ is telling me). https://github.com/rmetzger/scratch/commit/203d647086d089575fb27223462d79c87771f1d1 Let me know if this is sufficient or if you

Re: Restructuring the maven projects

2015-03-17 Thread Fabian Hueske
I agree that it's a good idea to move the APIs into one module. But why should we merge client and compiler (optimizer) and the examples into one module? I think modules with clearly separated responsibilities can also help new contributors to navigate the code. 2015-03-17 16:16 GMT+01:00

Re: [DISCUSS] Issues with heterogeneity of the code

2015-03-17 Thread Fabian Hueske
Touching every file of the code would also be a good opportunity to switch from tab to space indention. So if we enforce a strict style, we could also address this issue which causes discussions every now and then. 2015-03-16 21:53 GMT+01:00 Aljoscha Krettek aljos...@apache.org: No, but I don't

Re: Blog post about Parallel Joins in Flink - Mechanisms and Performance

2015-03-17 Thread Henry Saputra
Great post. Small comment, @Fabian I think the image is flipped. The image for Repartition-Repartition is shown in the Broadcast-Forward section. - Henry On Tue, Mar 17, 2015 at 2:22 PM, Stephan Ewen se...@apache.org wrote: Hello Squirrels! Flink committer Fabian Hueske has written a very

Re: [Multiple Programs Test Base] Consequence of Shading Guava away

2015-03-17 Thread Andra Lungu
Hi Robert, Forgot to answer :) Worked like a charm. Thanks a lot! On Tue, Mar 17, 2015 at 11:41 AM, Robert Metzger rmetz...@apache.org wrote: Hey Andra, I've checked out your repository and made some changes. It seems to compile, also the Files thing seems to work (at least thats what

Re: Website documentation minor bug

2015-03-17 Thread Maximilian Michels
Another proposal in addition to the recent changes: How about we remove the fixed attribute from the navigation of the docs? Then the nav will just scroll away and give space for the documentation. After all, how useful is it to constantly have the Java and Scala doc links available? Actually,

Re: Restructuring the maven projects

2015-03-17 Thread Ufuk Celebi
+1 I like the proposed structure. The only thing I was wondering about is whether to name core = batch? On Tue, Mar 17, 2015 at 11:37 AM, Márton Balassi balassi.mar...@gmail.com wrote: +1 for the proposed structure. I have no explicit preference for having batch and streaming scala together

Re: Restructuring the maven projects

2015-03-17 Thread Ufuk Celebi
Thanks for bringing this up, Till. You are right, but I think the main issue was that tight interaction between Java and Scala was problematic. I am not sure whether this is such a big problem for the the APIs. On Tue, Mar 17, 2015 at 3:21 PM, Till Rohrmann trohrm...@apache.org wrote: Putting

Re: Restructuring the maven projects

2015-03-17 Thread Till Rohrmann
Putting the Scala and Java API into the same module means that we'll have more mixed Java/Scala projects, right? I just want to check if everyone is aware of it considering our latest experiences with these kind of modules. On Tue, Mar 17, 2015 at 2:21 PM, Ufuk Celebi u...@apache.org wrote: +1