AFAIR, the main point to use Wiki last time was a lower entry bar, comparing to the main website, for everybody to add a new information related to any of aspects of Beam.
On the flip side, as it was mentioned above, this information is mostly not reviewed, bad structured and not easy to find in the end. IMHO, Wiki should be used just as an “archive" in case if we something is actual for the moment and we don’t want to miss it (e.g. design documents list or git/gradle/etc tips) or for very internal things the can “pollute” the website. I believe the main entry point should be a Beam website that accumulates most of the things that are needed for users and contributors frequently and are pretty stable in terms of content changing. The review of new doc updates is also useful in terms of knowledge sharing. I'm also against another framework or a tool for similar purposes, it will just complicate the things on all levels. — Alexey > On 22 Sep 2023, at 18:14, Robert Bradshaw via dev <dev@beam.apache.org> wrote: > > On Fri, Sep 22, 2023 at 8:05 AM Danny McCormick via dev <dev@beam.apache.org > <mailto:dev@beam.apache.org>> wrote: >> > I do feel strongly that https://beam.apache.org/contribute/ should remain >> > on the main site, as it's aimed at users (who hopefully want to step up >> > and contribute) >> >> To be clear, I don't think anyone is suggesting getting rid of the section, >> my comments were about replacing the side panel links with links to the wiki >> (or now markdown or wherever we put our docs) instead of hosting those >> things as part of our site. >> >> > Related, I stumbled across this the other day: >> > https://github.com/apache/beam-site which appears to be unused which could >> > probably even have different review and committer sets if we wanted? >> >> That actually holds our published release docs, just not on master - >> https://github.com/apache/beam-site/tree/release-docs. > > Yeah. Basically we're using it as hosting for our voluminous auto-generated > docs. > >> A separate repo is always an option regardless, though I don't see a ton of >> advantages and it moves us further from the core codebase. > > I don't see any advantage, and plenty of downsides, to a separate repo. What > is the issue we're trying to solve here? > >> > I feel like that's actually pretty easy with Github actions? I think maybe >> > there's even one that exists Github Pages and probably any other static >> > site generator thingy we could care to name. >> >> Do you know of any actions that do this? >> https://github.com/kamranahmedse/github-pages-blog-action is kinda close, >> but not obviously better than a folder of markdown docs (no side nav AFAIK). >> I'm not sure if actions are really helpful here anyways. >> >> Building our own is definitely doable, but maybe not trivial (feel free to >> fact check that) and does introduce a second website framework (hugo vs >> jekyll). > > Yeah, -1 on introducing yet another framework. Mostly, we need to prioritize > a place to push content that's easy to keep up to date. > >> On Fri, Sep 22, 2023 at 10:42 AM Byron Ellis via dev <dev@beam.apache.org >> <mailto:dev@beam.apache.org>> wrote: >>> I feel like that's actually pretty easy with Github actions? I think maybe >>> there's even one that exists Github Pages and probably any other static >>> site generator thingy we could care to name. Related, I stumbled across >>> this the other day: https://github.com/apache/beam-site which appears to be >>> unused which could probably even have different review and committer sets >>> if we wanted? >>> >>> On Thu, Sep 21, 2023 at 3:19 PM Robert Bradshaw via dev >>> <dev@beam.apache.org <mailto:dev@beam.apache.org>> wrote: >>>> TBH, I'm not a huge fan of the wikis either. My ideal flow would be >>>> something like g3doc, and markdown files in github do a reasonable enough >>>> job emulating that. (I don't think the overhead of having to do a PR for >>>> small edits like typos is oneros, as those are super easy reviews to do as >>>> well...) For anything in-depth, a pointer to an "actual" doc with better >>>> collaborative editing tools is generally in order anyway. >>>> >>>> I do feel strongly that https://beam.apache.org/contribute/ should remain >>>> on the main site, as it's aimed at users (who hopefully want to step up >>>> and contribute). The top level should probably mostly be a pointer to >>>> this, but I think it's valuable (for the audience that reaches it from >>>> github) to be a bit taylored to that audience (e.g. assume they just >>>> forked/downloaded the repository and want to edit-build-push. Generally a >>>> more advanced user than would find the page on the website.) >>>> >>>> The release guide? Meh. Wherever those doing releases find it most >>>> convenient. If that was me I'd probably put a markdown file right in the >>>> release directory next to the relevant scripts... (If not jump to literate >>>> programming right there :). >>>> >>>> On Thu, Sep 21, 2023 at 1:20 PM Kenneth Knowles <k...@apache.org >>>> <mailto:k...@apache.org>> wrote: >>>>> >>>>> >>>>> On Thu, Sep 21, 2023 at 3:55 PM Danny McCormick >>>>> <dannymccorm...@google.com <mailto:dannymccorm...@google.com>> wrote: >>>>>> > - reviewed >>>>>> >>>>>> Generally, I'm actually probably -0 on this one - it depends on context, >>>>>> but things that are for other developers only are usually better off >>>>>> without this requirement IMO since you get more contributions and more >>>>>> useful/unpolished things. Unfortunately, I'm not sure if confluence >>>>>> actually meets the bar for easy to update though because getting an >>>>>> account/initial setup is a pain. So I'm -0 since I don't know of a tool >>>>>> that both allows people to easily edit and avoids spam, but if such a >>>>>> tool exists I'd strongly prefer that. >>>>>> >>>>>> > - discoverable/orientable aka top/side nav >>>>>> >>>>>> I'm -1 on this requirement. A structured in-repo `docs` folder and/or a >>>>>> dedicated developer documentation repo have worked well on teams I've >>>>>> been on in the past and it avoids having to maintain additional >>>>>> infrastructure for a website. It also brings folks closer to the code, >>>>>> making edits more likely. These look nice, but I don't know how much >>>>>> value they actually add. >>>>>> >>>>>> > I did a quick search to see if there was a standard answer to having >>>>>> > top and side nav for a docs/ folder of markdown in your github repo. I >>>>>> > guess that is GitHub Pages? TBH I have used them happily in the >>>>>> > distant past but somehow I thought they had been deprecated or >>>>>> > something. >>>>>> >>>>>> I'm probably -1 on pages because at that point we've got 2 different >>>>>> website setups, one using hugo (https://beam.apache.org/) and one using >>>>>> Jekyl (pages); at that point, we might as well just move things totally >>>>>> back into the website and just have it live under a separate section of >>>>>> the site. >>>>>> >>>>>> My vote if we're moving away from confluence (which seems fine) would be >>>>>> either a dedicated `docs` or `developer-docs` folder or a separate >>>>>> markdown only repo. >>>>> >>>>> I could go for this. I'm pretty -1 on a soup of files without any >>>>> information architecture or scattered throughout random folders. But I'm >>>>> probably -2 on the confluence wiki if such a thing is possible and it >>>>> would also remove a piece from our infra, so... I think I'd call it an >>>>> upgrade to have a folder full of docs. If we then make taxonomic >>>>> subfolders that hide all the information I'll be sad again. >>>>> >>>>> Ideally the developer-docs/ folder could be read as text, lightly >>>>> rendered like GH does, or fully rendered with navs. Yes, I am describing >>>>> g3doc (which is talked about publicly so I can name it, but I don't know >>>>> what the publicly-available equivalent is). None of the website-building >>>>> not-human-readable stuff from jekyll and hugo. >>>>> >>>>> Kenn >>>>> >>>>>> >>>>>> On Thu, Sep 21, 2023 at 3:30 PM Kenneth Knowles <k...@apache.org >>>>>> <mailto:k...@apache.org>> wrote: >>>>>>> OK so this did turn into a discussion all about the tech/hosting :-). >>>>>>> It has been 5 years and we have experience of the wiki now so maybe >>>>>>> that is fair anyhow. And perhaps the preference of where to put >>>>>>> information cannot be separated from it. >>>>>>> >>>>>>> Top posting because there was so much in common across the responses >>>>>>> and I agree mostly too so I'll merge & paraphrase. >>>>>>> >>>>>>> > Focusing the main website primarily toward users is good >>>>>>> >>>>>>> Seems everyone still agrees with this >>>>>>> >>>>>>> > The wiki is not reviewed and our docs we care about should be >>>>>>> >>>>>>> Agree. >>>>>>> >>>>>>> > Wiki syntax is an old thing that is not quite markdown and should >>>>>>> > just be markdown >>>>>>> >>>>>>> Agree. >>>>>>> >>>>>>> > Wiki is yet another place to go, hard to navigate, not discoverable. >>>>>>> >>>>>>> Agree. >>>>>>> >>>>>>> So the "neverending argument" is so far unanimous on this particular >>>>>>> thread :-) >>>>>>> >>>>>>> --------------- >>>>>>> >>>>>>> My personal preferences are: >>>>>>> >>>>>>> - markdown >>>>>>> - reviewed >>>>>>> - organized... >>>>>>> - ...independently of code folders >>>>>>> - discoverable/orientable aka top/side nav >>>>>>> >>>>>>> So large markdown files don't meet "organized" and collections of >>>>>>> READMEs don't meet "independently of code folders" and a docs/ folder >>>>>>> in the repo doesn't meet "discoverable/orientable aka top/side nav". >>>>>>> Seems like a new place is needed to meet all the desires. >>>>>>> >>>>>>> CONTRIBUTING.md is a good example to dissect. The integration with >>>>>>> GitHub is great, but it should be super *concise* (so as not to >>>>>>> discourage anyone) and have only information that *every* contributor >>>>>>> should learn when they are *new*. Any information not meeting all those >>>>>>> criteria needs a different home. >>>>>>> >>>>>>> I did a quick search to see if there was a standard answer to having >>>>>>> top and side nav for a docs/ folder of markdown in your github repo. I >>>>>>> guess that is GitHub Pages? TBH I have used them happily in the distant >>>>>>> past but somehow I thought they had been deprecated or something. >>>>>>> >>>>>>> Kenn >>>>>>> >>>>>>> On Thu, Sep 21, 2023 at 1:18 PM Danny McCormick via dev >>>>>>> <dev@beam.apache.org <mailto:dev@beam.apache.org>> wrote: >>>>>>>> > I might be wrong but I think of wiki as a more volatile and a less >>>>>>>> > reliable place than the Website >>>>>>>> >>>>>>>> I agree, the counterpoint is that docs that require more work to >>>>>>>> update are more likely to go stale since there is higher friction to >>>>>>>> update. There's also more of an expectation that everything is >>>>>>>> polished, which may or may not be desirable. >>>>>>>> >>>>>>>> In practice, the end result is that wiki guides are more comprehensive >>>>>>>> but messier (and to your point a little less reliable and I'd add less >>>>>>>> discoverable, though that's fixable). To me, that is an ok tradeoff to >>>>>>>> make with developer guides. Also, note that the contribution guide >>>>>>>> itself is in GitHub markdown - CONTRIBUTING.md >>>>>>>> <https://github.com/apache/beam/blob/master/CONTRIBUTING.md> - to me >>>>>>>> that's something we should definitely not change since that is the >>>>>>>> broadly agreed upon standard for GitHub projects and gets special >>>>>>>> treatment from GitHub. >>>>>>>> >>>>>>>> ------------------------------------------------------------------------------------------------------------------------------------ >>>>>>>> >>>>>>>> Mostly, my vote is predicated on maintaining consistency with the >>>>>>>> decision in >>>>>>>> https://lists.apache.org/thread/w4g8xpg4215nlq86hxbd6n3q7jfnylny and >>>>>>>> wanting to avoid relitigating that decision (since code review vs no >>>>>>>> code review on dev docs is a neverending argument that has played out >>>>>>>> many times across many projects with no clear winner and it is tightly >>>>>>>> coupled with personal preference). If the decision was "dev stuff" >>>>>>>> goes to confluence, then the contribution section seems to be a clear >>>>>>>> place to draw the line since that is all by definition "dev stuff". >>>>>>>> >>>>>>>> Thanks, >>>>>>>> Danny >>>>>>>> >>>>>>>> On Thu, Sep 21, 2023 at 12:58 PM Chamikara Jayalath >>>>>>>> <chamik...@google.com <mailto:chamik...@google.com>> wrote: >>>>>>>>> I might be wrong but I think of wiki as a more volatile and a less >>>>>>>>> reliable place than the Website (can be updated without a review by >>>>>>>>> any committer and we do that quite often). I think things in the >>>>>>>>> contribution guide are key to a healthy Beam community so I'd like >>>>>>>>> them to be in a more stable place that gets reviewed appropriately >>>>>>>>> when updated. >>>>>>>>> >>>>>>>>> Thanks, >>>>>>>>> Cham >>>>>>>>> >>>>>>>>> On Thu, Sep 21, 2023 at 9:14 AM Danny McCormick via dev >>>>>>>>> <dev@beam.apache.org <mailto:dev@beam.apache.org>> wrote: >>>>>>>>>> +1 on moving the release guide. I'd argue that everything under the >>>>>>>>>> `contribute` tag other than the main page >>>>>>>>>> (https://beam.apache.org/contribute/) and the link to >>>>>>>>>> CONTRIBUTING.md >>>>>>>>>> <https://github.com/apache/beam/blob/master/CONTRIBUTING.md> makes >>>>>>>>>> more sense on the wiki (we can keep the section with the sidebar >>>>>>>>>> links just redirecting to the wiki). I don't think it makes sense to >>>>>>>>>> move anything else, but the contributing section is inherently "dev >>>>>>>>>> focused". >>>>>>>>>> >>>>>>>>>> Thanks, >>>>>>>>>> Danny >>>>>>>>>> >>>>>>>>>> On Thu, Sep 21, 2023 at 11:58 AM Kenneth Knowles <k...@apache.org >>>>>>>>>> <mailto:k...@apache.org>> wrote: >>>>>>>>>>> Hello! >>>>>>>>>>> >>>>>>>>>>> I am reviving a discussion that began at >>>>>>>>>>> https://lists.apache.org/thread/w4g8xpg4215nlq86hxbd6n3q7jfnylny >>>>>>>>>>> when we started our Confluence wiki and has even been revived once >>>>>>>>>>> before. >>>>>>>>>>> >>>>>>>>>>> The conclusion of that thread was basically "yes, let us separate >>>>>>>>>>> the contributor-facing stuff to a different site". It also was the >>>>>>>>>>> boot up of the Confluence wiki but I want to not discuss >>>>>>>>>>> tech/hosting for this thread. I want to focus on the issue of >>>>>>>>>>> having a separate user-facing website vs a contributor-facing >>>>>>>>>>> website. Some things like issue priorities are user-and-dev facing >>>>>>>>>>> and they require review for changes and should stay on the user >>>>>>>>>>> site. I also don't want to get into those more complex cases. >>>>>>>>>>> >>>>>>>>>>> We are basically in a halfway state today because I didn't have >>>>>>>>>>> enough volunteer time to finish everything and I did not wrangle >>>>>>>>>>> enough help. >>>>>>>>>>> >>>>>>>>>>> So now I am release manager and encountering the docs more closely >>>>>>>>>>> again. The release docs really blend stuff. >>>>>>>>>>> >>>>>>>>>>> - The main release guide is on the website. >>>>>>>>>>> - Some steps, though, are GitHub Issues that we push along from >>>>>>>>>>> release Milestone to the next one. >>>>>>>>>>> - The actual technical bits to do the steps are sometimes on the >>>>>>>>>>> confluence wiki >>>>>>>>>>> - I expect I will also be touching README files in various folders >>>>>>>>>>> of the repo >>>>>>>>>>> >>>>>>>>>>> So I just want to make some more steps, and I wanted to ask the >>>>>>>>>>> community for their current thoughts. I think one big step could be >>>>>>>>>>> to move the release guide itself to the dev site, which is >>>>>>>>>>> currently the wiki. >>>>>>>>>>> >>>>>>>>>>> What do you think? Are there any other areas of the website that >>>>>>>>>>> you think could just move to the wiki today? >>>>>>>>>>> >>>>>>>>>>> Kenn >>>>>>>>>>> >>>>>>>>>>> p.s. Some time in the past I saw an upper right corner fold (like >>>>>>>>>>> https://www.istockphoto.com/illustrations/paper-corner-fold) that >>>>>>>>>>> took you to the dev site that looked the same with different color >>>>>>>>>>> scheme. That was fun :-)
