Re: Improve the documentation of the Flink Architecture and internals

2015-03-23 Thread Ufuk Celebi
I couldn't have a look at it earlier, because the Wiki was down. Very nice overview of the flow of things. I like the text and pictures a lot. I will add content about: 1) The way that we do the network transfers with Netty 2) A more detailed message flow for pipelined vs. blocking results.

Re: Improve the documentation of the Flink Architecture and internals

2015-03-23 Thread Maximilian Michels
Very insightful post. Thank you! On Sat, Mar 21, 2015 at 5:30 PM, Till Rohrmann trohrm...@apache.org wrote: I wrote some internal documentation for Akka and the distributed communication [1]. Cheers, Till [1] https://cwiki.apache.org/confluence/display/FLINK/Akka+and+Actors On Fri, Mar

Re: Improve the documentation of the Flink Architecture and internals

2015-03-21 Thread Stephan Ewen
Very nice post, Till! We are starting to get much better with this... On Sat, Mar 21, 2015 at 6:45 PM, Henry Saputra henry.sapu...@gmail.com wrote: Awesome, thanks Till On Saturday, March 21, 2015, Till Rohrmann trohrm...@apache.org wrote: I wrote some internal documentation for Akka and

Re: Improve the documentation of the Flink Architecture and internals

2015-03-20 Thread Kostas Tzoumas
I added a document for data exchange between tasks: https://cwiki.apache.org/confluence/display/FLINK/Data+exchange+between+tasks Feel free to edit. I plan to link the class names to the class files in github. On Tue, Mar 17, 2015 at 11:17 AM, Kostas Tzoumas ktzou...@apache.org wrote: +1 for

Re: Improve the documentation of the Flink Architecture and internals

2015-03-20 Thread Maximilian Michels
+1 for the initiative and for the wiki. At the moment, the wiki's barrier to entry is like the git repository's. Contributors need to explicitly ask comitters for access to the wiki. Is there a way we could open up the wiki for contributors without having to face too much spam? (e.g. have changes

Re: Improve the documentation of the Flink Architecture and internals

2015-03-20 Thread Henry Saputra
Ah the Tweet infra bot just announce extended downtime for Confluence [1] - Henry [1] https://twitter.com/infrabot/status/578983473970475008 On Fri, Mar 20, 2015 at 11:27 AM, Stephan Ewen se...@apache.org wrote: For me as well. Earlier today it said down for maintenance On Fri, Mar 20, 2015

Re: Improve the documentation of the Flink Architecture and internals

2015-03-20 Thread Kostas Tzoumas
it's down for me as well On Fri, Mar 20, 2015 at 7:12 PM, Henry Saputra henry.sapu...@gmail.com wrote: Is the wiki down for any of you? I can't access https://cwiki.apache.org/confluence/display/FLINK/Apache+Flink+Home 404 - Henry On Fri, Mar 20, 2015 at 4:46 AM, Kostas Tzoumas

Re: Improve the documentation of the Flink Architecture and internals

2015-03-20 Thread Henry Saputra
I don't think so. But adding new contributors wiki username to edit is much easier to merging changes to website, I hope. - Henry On Fri, Mar 20, 2015 at 8:53 AM, Maximilian Michels m...@apache.org wrote: +1 for the initiative and for the wiki. At the moment, the wiki's barrier to entry is

Re: Improve the documentation of the Flink Architecture and internals

2015-03-17 Thread Kostas Tzoumas
+1 for the Wiki. When these have been stabilized we can move them to the docs if we decide to do so. On Mon, Mar 16, 2015 at 10:07 PM, Stephan Ewen se...@apache.org wrote: I have put my suggested version of an outline for the docs into the wiki. Regardless where the docs end up (wiki or

Re: Improve the documentation of the Flink Architecture and internals

2015-03-16 Thread Aljoscha Krettek
Why do you wan't to split stuff between the doc in the repository and the wiki. I for one would always be to lazy to check stuff in a wiki when there is also a documentation. Plus, this would lead to additional overhead in deciding what goes where and syncing between the two places for

Re: Improve the documentation of the Flink Architecture and internals

2015-03-16 Thread Stephan Ewen
I have put my suggested version of an outline for the docs into the wiki. Regardless where the docs end up (wiki or repository), we can use the wiki to outline the docs. https://cwiki.apache.org/confluence/display/FLINK/Flink+Internals Some pages contain some stub or outline, others are

Re: Improve the documentation of the Flink Architecture and internals

2015-03-16 Thread Stephan Ewen
I think the Wiki has a much lower barrier of entry to fix docs, especially for external people. The docs, with the Jekyll setup, is rather tricky. I would very much like that all kinds of people contribute to the docs about the internals, not just the usual three suspects that have done this so