I'm ok with the proposed approach. Btw. if we want to use Githu wiki then maybe we can move there all contributing guides etc? I personally found it easier to navigate between small pages than long documents with no easy access to the navigation menu.
T. On Mon, Aug 3, 2020 at 4:58 PM Jarek Potiuk <jarek.pot...@polidea.com> wrote: > > Tomek, I would love to leave aside ADR snapshots for now - we can have a > separate discussion about it later on how to do it. > > I think we need to clean up the mess we have now in CWIKI. I would love to > focus only on this for now. > > I think we all agree we should only keep planning/overview of the effort. > However, currently, it's NOT the case for Airflow 2.0 and Cwiki (we have > way too much overlapping information): > > 1) https://cwiki.apache.org/confluence/display/AIRFLOW/Roadmap - general > outdated roadmap > 2) https://cwiki.apache.org/confluence/display/AIRFLOW/Airflow+2.0 - > High-level features (Pretty much the same as in the > https://github.com/apache/airflow/issues/10085) > 3) > https://cwiki.apache.org/confluence/display/AIRFLOW/Airflow+2.0+-+Planning - > this is I think a bit different - recently updated by Kaxil and it is > mostly about general approach we should take (and overlap with the rest is > minimal). > > My proposal: > > 3) should stay and we should archive/deprecate 1) and 2). > > I also have the feeling that we must archive all the stuff that is > outdated because it is really confusing now which information is outdated > or not. > > My proposal: > > * AIPs - we keep them for now as they are (And make it part of later ADR > discussion). > * Airflow Links - archive & deprecate > * Airflow Release Planning - we leave it for now and update it as part of > 2.0 discussion > * Building Docs - archive & deprecate > * Releasing Airflow - I can move it to "dev" together with planned backport > release doc updates > * Announcements - leave it for now (maybe forever) > * API conventions - archive & deprecate > * Committers/Commiter's Guide -> archive & deprecate (review if some > information can be move to CONTRIBUTING.rst) > * Common Pitfalls -> move it to docs > * Community Gudelines, contributor's Guide -> archive & deprecate (review > if some information can be move to CONTRIBUTING.rst) > * First time contributor's workshop -> move it as blog to apache.airflow.org > * File lists - > move it to Airflow repo as resources. > * Meeting notes -> archive & deprecate. > * Meetups -> archive & deprecate. > * Product requirements, Roadmap Airflow 2.0 -> archive & deprecate > * Roles -> archive & deprecate (some stuff moved to CONTRIBUTING.rst) > * Scheduler Basics - > move it to docs > * Season of Docs 2019 -> archive & deprecate > > Does it sound reasonable? Does anyone think some other things should stay? > I am happy to do it if no-one objects. > > J. > > > On Mon, Aug 3, 2020 at 3:42 PM Tomasz Urbaszek <turbas...@apache.org> wrote: > > > I agree with Ry. Moreover,we should adjust cwiki information after a > > discussion on devlist/github. So I think in the case of AIPs the cwiki > > should work as someting like architecture decision records. > > However, I'm afraid that there will be no way to automate or enforce > > this synchronization. > > > > Tomek > > > > > > On Mon, Aug 3, 2020 at 3:32 PM Ry Walker <r...@rywalker.com> wrote: > > > > > > I'd say the cwiki should provide an overview of the effort, as it does > > now, > > > and that we should keep track of the work in a Github project using > > github > > > issues. The cwiki should link to that project board as the source of > > truth > > > for project status. This will help the wiki page to be perceived as up to > > > date as it won't need to be updated with each bit of progress. > > > > > > -Ry > > > > > > > > > On Mon, Aug 3, 2020 at 4:29 AM Jarek Potiuk <jarek.pot...@polidea.com> > > > wrote: > > > > > > > Yeah. After experimenting a bit with it - seems that Wiki in Github is > > > > a bit "abandoned" place and omissions like lack of auto-linking issues > > > > and PRs is a big bummer. > > > > > > > > Kamil - would you mind re-creating the issue based on the old issue? I > > > > - unfortunately - added all "Apache Committers" to it so we cannot > > > > re-open it. > > > > > > > > But I have another question here: > > > > > > > > 1) Should we remove > > > > > > https://cwiki.apache.org/confluence/display/AIRFLOW/Airflow+2.0+-+Planning > > > > completely? > > > > 2) More than that - should we archive and move everything else from > > > > CWiki to Github Issues? > > > > > > > > > > > > I think it will be very confusing (especially for new contributors) if > > > > we keep some information in CWiki but also start using Github Issues > > > > for similar purpose. So I would be for archiving all content in the > > > > CWiki and moving it all to Issues. > > > > > > > > I took a look at the kind of documents we have in CWiki and we have a > > > > LOT of information there that is outdated or could live elsewhere. > > > > Here are my proposals: > > > > > > > > * Airflow 2.0 planing - we could completely move it to "Airflow 2.0 > > > > Release" issue > > > > * AIPs - we could keep all the completed AIP-s there (And keep the > > > > "proposed" ones for the future) but we could move all the "active" > > > > AIPs to Github Issues and add all the new AIPs there. > > > > * Airflow Links - we can abandon it (It's already abandoned in fact - > > > > last update May 2017) > > > > * Airflow Release Planning - we could review it and turn it into a > > > > "meta" issue - it has a lot fo information about pre-1.10 releases > > > > which we can remove (And we will have to redefine it after we agree > > > > release schedule and versioning for 2.* series) > > > > * Building Docs - is outdated > > > > * Releasing Airflow - I think we can move it to Airflow's source code > > > > in "dev" folder (like I did for the Backport Packages) > > > > * Announcements -> that one we might do on "airflow.apache.org" site > > > > as a Blog post ? > > > > * API conventions - outdated > > > > * Committers/Commiter's Guide -> we could have it in the > > > > "CONTRIBUTING.rst" documentation of Airflow (some of the information > > > > there is not valid anyway and CONTRIBUTING documentation is much more > > > > updated) > > > > * Common Pitfalls -> I think that one belongs to the documentation of > > > > Airflow not to Wiki and we could select/move some still valid > > > > information from there to the documentation > > > > * Community Gudelines, contributor's Guide -> this all in > > CONTRIBUTING.rst > > > > * First time contributor's workshop -> this can be moved to a > > > > "apache.airflow.org" as a Blog Post. > > > > * File lists - > those files can be all added to the airflow > > > > repository in "resources" folder or smth. > > > > * Meeting notes - Those could be added to relevant issues in GitHub. > > > > We could have "meta" issues for "special interest groups" and add > > > > meeting notes there. > > > > * Meetups -> already part of airflow.apache.org > > > > * Product requirements, Roadmap Airflow 2.0 -> this all could be moved > > > > to "meta" issues > > > > * Roles -> should be added to CONTRIBUTING.rst > > > > * Scheduler Basics - > should be part of Airflow Documentation > > > > * Season of Docs 2019 -> we can archive it. > > > > > > > > We could also use Github Wiki to only have "Index" of all important > > > > issues that are "permanent" - Airflow 2.0 roadmap, Special interest > > > > groups, AIPs, > > > > > > > > Let me know what you think? > > > > > > > > J. > > > > > > > > > > > > > > > > On Sun, Aug 2, 2020 at 12:31 PM Kaxil Naik <kaxiln...@gmail.com> > > wrote: > > > > > > > > > > I agree with Tomek and feel Github issues ("meta"-issue) is a better > > > > place > > > > > than Github Wiki. > > > > > > > > > > On Sun, Aug 2, 2020 at 11:26 AM Tomasz Urbaszek < > > turbas...@apache.org> > > > > > wrote: > > > > > > > > > > > I see the advantage of having no comment in wiki but in the longer > > > > > > run, I think this will create confusion. Where should I discuss a > > > > > > particular thing? On devlist? Slack? In issue? How should a new > > > > > > contributor know this? > > > > > > > > > > > > After giving some thought to that I'm leaning towards the > > meta-issue: > > > > > > - they are clear (no need to go to wiki) > > > > > > - give possibilit to link other issues/PRs that shows their > > content on > > > > > > hover > > > > > > - this is great advantage as we can see how our work is > > interconnected > > > > > > - having an issue make it explicit to where contributors should > > leave > > > > > > their comments > > > > > > > > > > > > No matter what we decide, we should thrive to limit the places > > where > > > > > > information is available. > > > > > > > > > > > > Bests, > > > > > > Tomek > > > > > > > > > > > > > > > > > > On Sun, Aug 2, 2020 at 12:00 PM Jarek Potiuk < > > jarek.pot...@polidea.com > > > > > > > > > > > wrote: > > > > > > > > > > > > > > Question. Should we move over Airflow 2.0 Status and other > > > > "permanent" > > > > > > > information to Github Wiki? See here for example: > > > > > > > https://github.com/apache/airflow/wiki/Airflow-2.0 > > > > > > > > > > > > > > The discussion originated by Kamil creating an issue for Airflow > > 2.0 > > > > - > > > > > > > which was essentially overriding the page we had in > > > > > > > > > > > > > > > > > > > https://cwiki.apache.org/confluence/display/AIRFLOW/Airflow+2.0+-+Planning > > > > > > > and adding more "status" information in > > > > > > > https://github.com/apache/airflow/issues/10085. This was more > > of a > > > > > > > "meta" issue as it has a lot of unrelated issues / projects > > mentioned > > > > > > > - the only common thing for those was that it was "Airflow 2.0". > > But > > > > > > > we already have "Milestone 2.0" and CWIKI page. > > > > > > > > > > > > > > My proposal was that since we have 2.0 Milestone already we > > should > > > > use > > > > > > > this one to mark issues for 2.0 and in order to keep > > > > > > > Roadmap/Plans/Status we can use Github's Wiki instead. IMHO it is > > > > much > > > > > > > better as it does not allow comments - which is good IMHO. For > > this > > > > > > > jind of "permanent" pages, comments and discussion should happen > > for > > > > > > > the individual issues not for the page itself (especially when > > you > > > > do > > > > > > > not have in-line comments). > > > > > > > > > > > > > > And this page should always be "current" - with the old roadmap > > in > > > > > > > CWIKI and the issue 10085 when you add comments, you quickly lose > > > > > > > track whether the comments are more important than the overview, > > and > > > > > > > how accurate the "overview" is. When you just edit the wiki - > > you > > > > > > > always do it deliberately - because you want to update status > > rather > > > > > > > than make a comment or discuss, > > > > > > > > > > > > > > So I created this as copy of the issue: > > > > > > > https://github.com/apache/airflow/wiki/Airflow-2.0 so that we > > can > > > > > > > compare it - can you please compare it with > > > > > > > https://github.com/apache/airflow/issues/10085 and voice your > > > > opinion > > > > > > > what's better? > > > > > > > > > > > > > > I think it's also a great opportunity to archive a lot of the > > old and > > > > > > > not up-to-date from the old Wiki and migrate it to GitHub. We > > could > > > > > > > move AIPs to Github issues (as needed) - AIPS are fine for > > > > > > > discussion/issues/comments, but when they got implemented we > > could > > > > > > > move it over to wiki as "Implemented" status for history. > > > > > > > > > > > > > > Let me know what you think. > > > > > > > > > > > > > > BTW. PLEASE do NOT comment on that #10085 issue (it's now locked > > and > > > > > > > closed). I accidentally (shame on me) notified all Apache > > Committers. > > > > > > > Happened twice today (also for someone else) so I opened a > > ticket to > > > > > > > Infra to restrict that (If only possible) because it's all too > > easy > > > > to > > > > > > > notify everyone @Apache). If you comment there 3K+ people get > > > > > > > notified. > > > > > > > > > > > > > > But feel free to upvote the infra ticket: > > > > > > > https://issues.apache.org/jira/browse/INFRA-20623 > > > > > > > > > > > > > > > > > > > > > J. > > > > > > > > > > > > > > > > > > > > > > > > > > > > -- > > > > > > > Jarek Potiuk > > > > > > > Polidea | Principal Software Engineer > > > > > > > > > > > > > > M: +48 660 796 129 > > > > > > > > > > > > > > > > > > > > > > -- > > > > > > > > Jarek Potiuk > > > > Polidea | Principal Software Engineer > > > > > > > > M: +48 660 796 129 > > > > > > > > > -- > > Jarek Potiuk > Polidea <https://www.polidea.com/> | Principal Software Engineer > > M: +48 660 796 129 <+48660796129> > [image: Polidea] <https://www.polidea.com/>
