+1 I don't have much to add to the debate, except to state that Beam currently has a fragmentation problem for user documentation. I would support limiting the wiki to non-user focused docs to prevent additional fragmentation.
We have first-class website pages [1], documentation embedded in Javadocs [2], and in some cases cloud.google.com/dataflow/docs has more context than official Beam docs [3]. [1] https://beam.apache.org/documentation/programming-guide/ [2] https://beam.apache.org/documentation/sdks/javadoc/2.4.0/org/apache/beam/sdk/io/TextIO.html [3] https://cloud.google.com/dataflow/pipelines/specifying-exec-params On Fri, Jun 8, 2018 at 2:18 PM Kenneth Knowles <k...@google.com> wrote: > Replies in line to keep on the discussion of what we might want to put in > different venues. But it does sound like there's enough support to at least > get started on getting our wiki set up to be accessible. > > For general context > > - Our wiki "exists" here: > https://cwiki.apache.org/confluence/display/BEAM/ > - It was turned on during incubation: > https://issues.apache.org/jira/browse/INFRA-11181 > > I can't seem to edit even though it says anyone can. JB - are you still > admin? I'd be happy to help. > > On Fri, Jun 8, 2018 at 12:45 PM Robert Bradshaw <rober...@google.com> > wrote: > >> The use of wiki vs. docs vs. the (repo-backed) website seems to be one of >> convenience vs. polish >> > > Wiki vs. docs vs. web site is about other things, too: > > - presentation style (slightly different from polish) > - target audience for each piece of content better (get started > contributing vs deep dive tutorials) > - separating large-scale "worlds" of content (vs just different pages on > the site) to avoid distractions (even polished design docs are a > distraction for a user-facing site) > > >> , and totally orthogonal to dev vs. user-facing stuff. >> > > User-facing content requires way more polish! But it also has other > important differences, like expectations, and the relative importance of > concision vs. comprehensiveness. > > I'm not opposed to a wiki, but personally I think a lot of our dev-facing >> docs (e.g. testing, ptransform style guide, portability overview) benefit >> from being in a more polished, permanent form (in particular, have been >> improved going through the code review process). >> > > I do like having review for things that are meant to last. I don't have a > technical solution in mind for that. I don't think they are mutually > exclusive anyhow. > > >> Very technical stuff like asf-site/README.md are IMHO best put right next >> to the sources/artifacts they describe. >> > > Yea the site README is an example that isn't so bad. I do still get a lot > of questions, because people don't know it is where they should look. The > worse examples might be the main beam/README.md and sdk/CONTAINERS.md which > are basically invisible. I think a big problem here is discoverability > which is very poor for sprinkled docs. It could remain poor on the wiki if > we make disjoint pages without good breadcrumb paths, but at least there's > search. > > > On the flip side, no need to limit the wiki to contributor-focused >> material. >> > > I disagree strongly here - I don't think the wiki will have appropriate > polish for users. Even if carefully polished I don't think the presentation > style is right, and it is not flexible. Power users will find it, of course. > > Kenn > > > > >> On Fri, Jun 8, 2018 at 12:05 PM Thomas Weise <t...@apache.org> wrote: >> >>> +1 most of the contributor material could live on Wiki and there it will >>> be easier to maintain (perhaps the lower bar for updates will lead to more >>> information and increased maintenance). Contribution policy related >>> material should remain on the website and go through proper >>> review/versioning. >>> >>> >>> On Fri, Jun 8, 2018 at 11:44 AM, Udi Meiri <eh...@google.com> wrote: >>> >>>> (a) Yes. >>>> (b) I'm interested in putting documentation for contributors there. >>>> (test triage guide, precommit and postcommit guidelines, processes, etc.) >>>> It'd be faster than having to go through the motions of a github pull >>>> request and a review process. >>>> (c) Anything that goes to a wide audience, such as SDK users. That >>>> would need review. >>>> >>>> Also, have you looked at https://wiki.apache.org/general/ ? (not sure >>>> if that's Confluence) >>>> >>>> >>>> On Fri, Jun 8, 2018 at 10:07 AM Andrew Pilloud <apill...@google.com> >>>> wrote: >>>> >>>>> +1 It would be really nice to have a lightweight place to share that >>>>> is more searchable then random Google docs. >>>>> >>>>> Andrew >>>>> >>>>> On Fri, Jun 8, 2018 at 9:35 AM Anton Kedin <ke...@google.com> wrote: >>>>> >>>>>> +1 >>>>>> >>>>>> (a) we should; >>>>>> (b) I think it will be a good place for all of the things you list; >>>>>> (c) introductory things, like "getting started", or "programming >>>>>> guide" that people not deeply involved in the project would expect to >>>>>> find >>>>>> on beam.apache.org should stay there, not in the wiki; >>>>>> >>>>>> On Fri, Jun 8, 2018 at 12:56 AM Etienne Chauchot < >>>>>> echauc...@apache.org> wrote: >>>>>> >>>>>>> Hi Kenn, >>>>>>> I'm +1 of course. I remember that you and I and others discussed in >>>>>>> a similar thread about dev facing docs but it got lost at some point in >>>>>>> time. >>>>>>> IMHO >>>>>>> >>>>>>> I would add >>>>>>> - runners specifics (e.g. how runners implement state or timer, how >>>>>>> they split data into bundles, etc...) >>>>>>> - probably putting online the doc I did for nexmark that summarizes >>>>>>> the architecture and pseudo code of the queries (because some are >>>>>>> several >>>>>>> thousand lines of code). I often use it to recall what a given query >>>>>>> does. >>>>>>> >>>>>>> I would remove >>>>>>> - BIPs / summaries of collections of JIRA >>>>>>> because it is hard to maintain and will end up being out of date I >>>>>>> think. >>>>>>> >>>>>>> Etienne >>>>>>> >>>>>>> Le jeudi 07 juin 2018 à 13:23 -0700, Kenneth Knowles a écrit : >>>>>>> >>>>>>> Hi all, >>>>>>> >>>>>>> I've been in half a dozen conversations recently about whether to >>>>>>> have a wiki and what to use it for. Some things I've heard: >>>>>>> >>>>>>> - "why is all this stuff that users don't care about here?" >>>>>>> - "can we have a lighter weight place to put technical references >>>>>>> for contributors" >>>>>>> >>>>>>> So I want to consider as a community starting up our wiki. Ideas for >>>>>>> what could go there: >>>>>>> >>>>>>> - Collection of links to design docs like >>>>>>> https://beam.apache.org/contribute/design-documents/ >>>>>>> - Specialized walkthroughs like >>>>>>> https://beam.apache.org/contribute/docker-images/ >>>>>>> - Best-effort notes that just try to help out like >>>>>>> https://beam.apache.org/contribute/intellij/ >>>>>>> - Docs on in-progress stuff like >>>>>>> https://beam.apache.org/documentation/runners/jstorm/ >>>>>>> - Expanded instructions for committers, more than >>>>>>> https://beam.apache.org/contribute/committer-guide/ >>>>>>> - BIPs / summaries of collections of JIRA >>>>>>> - Docs sitting in markdown in the repo like >>>>>>> https://github.com/apache/beam/blob/master/sdks/CONTAINERS.md and >>>>>>> https://github.com/apache/beam-site/blob/asf-site/README.md (which >>>>>>> will soon not be a toplevel README) >>>>>>> >>>>>>> What do you think? >>>>>>> >>>>>>> (a) should we do it? >>>>>>> (b) what should go there? >>>>>>> (c) what should not go there? >>>>>>> >>>>>>> Kenn >>>>>>> >>>>>>> >>>
