Comment on that: I looked at how towncrier has been integrated with PyPi (I followed what's going on there for a while). I like the approach and simplicity of "towncrier" and I think we might want to incorporate it in our changelog process. I think we can try it out for providers after we release "August" release (I think it needs to get incorporated into the tooling PR process so that any change in "providers" is accompanied by "news" piece (or marked as trivial). I think it should be rather easy to incorporate and automate it. Then we can also apply it to Airflow starting from 2.2 release (I think the tooling has to be implemented at the moment we branch-off for the 2.2 release, so that all the 2.3-targeted changes already follow the process.
J On Tue, Jun 15, 2021 at 9:57 PM Jarek Potiuk <[email protected]> wrote: > Just to clarify :) we have 70 + CHANGELOGs now :D - one per Airflow > and one per each provider (and I guess one should be needed for helm > chart). > > On Tue, Jun 15, 2021 at 9:34 PM Andrew Godwin > <[email protected]> wrote: > > > > I was going to suggest a solution like Towncrier - I feel the > collecting-news-entries model works well and is proven across several > different large projects, it's more a matter of choosing _how_ to actually > run it. > > > > On Django we simply have people commit to the future-changelog document > in the docs/ directory, which is nice as it lets you interlink all the big > news with other Sphinx pages, but there we have no top-level CHANGELOG file > like Airflow does so the process has to be a bit different by nature. > > > > Andrew > > > > On Tue, Jun 15, 2021 at 1:08 PM Jarek Potiuk <[email protected]> wrote: > >> > >> Hmm interesting. The real advantage of the "news" or blurb files is > >> what Ash mentioned is that you can change them after they've been > >> committed. > >> > >> I am not totally against it, that does sound interesting, and forces > >> you to think a bit when you realise that the file is missing. I wonder > >> what others think about it? > >> > >> On Mon, Jun 14, 2021 at 4:51 PM Tzu-ping Chung <[email protected]> > wrote: > >> > > >> > "Jarek Potiuk" <[email protected]> (14/06/21 20:06:14): > >> > > > >> > >I never used tools like Reno (so I might be wrong) but I think they > >> > >add extra effort and requirements (and confusion) for contributors to > >> > >also modify the changelog pretty much always. > >> > >IMHO for new contributors it will lead to "default" one more extra > >> > >iteration of review ("and please update the changelog") with any code > >> > >change which otherwise would not be needed. This will be especially > >> > >confusing because we have now 70+ changelogs and sometimes the change > >> > >applies to several providers (for transfer operators) so several > >> > >changelogs should be updated. > >> > > > >> > >Is there anyone who used such tools in a sizable project with > multiple > >> > >contributors (including a lot of first-time comitters) and can share > >> > >their experiences? > >> > > >> > Projects I involved with generally use bots to automatically go > through > >> > this “hey you need to add a changelog” phase. It is indeed a hassle > for > >> > contributors, but Airflow already has a ton of checks a new > contributor > >> > needs to be aware of, the overhead of adding yet another one is less > of > >> > an issue (compared of from zero to one). > >> > > >> > CPython uses a home-brewed tool called Blurb > >> > <https://pypi.org/project/blurb/> for changelog management, and a bot > >> > called Bedevere handles the news entries review (and a bit more). It > >> > automatically comments under a PR after checking, and edits the top > >> > message to supply relevant information when checks pass. > >> > > >> > Pip uses Towncrier <https://github.com/twisted/towncrier> (developed > by > >> > the same folks maintaining Twisted) and uses a check to mark the PR > with > >> > a red cross if the news entry is missing. There is a pre-commit check > as > >> > well (but as you know, very few contributors would actually set > >> > pre-commit up, so that’s less effective). > >> > > >> > The most lightweight setup I’m in touch with is pipx, where we just > add > >> > a checkbox to the PR template and tells the contributor to check it > >> > after following the news entry instruction. Up until now every new > >> > contributor I can remember automatically does the right thing without > >> > issues. > >> > > >> > All those tools work more or less the same (a file is added for each > >> > feature, and those are collected into a section in the actual release > >> > note file when a release is made), so I’d imagine it shouldn’t be too > >> > difficult for Airflow to implement some kind of automation to keep the > >> > friction low for both contributors and reviewers. > >> > > >> > TP > >> > > >> > > >> > > > >> > > > >> > >J. > >> > > > >> > >On Mon, Jun 14, 2021 at 1:30 PM Ash Berlin-Taylor <[email protected]> > wrote: > >> > >> > >> > >> Would it be possible to use the branch name for the > "conventional" part -- i.e. fix/my-branch - feat/something? I guess not as > once the PR is merged the name of the branch is not easily accessible > anymore, right? > >> > >> > >> > >> -ash > >> > >> > >> > >> On Mon, Jun 14 2021 at 09:08:45 +0100, Ash Berlin-Taylor < > [email protected]> wrote: > >> > >> > >> > >> I also don't like conventional commits - and ultimately I don't > think for commit logs are the right tool for building a changelog - they > are too granular and also too noisy. > >> > >> > >> > >> Primary reasons for not liking them: it uses up a lot of space in > the already short commit subject "limit", it's subjective, it can't be > changed after the commit is merged. > >> > >> > >> > >> If the goal is to make building change logs easier and more > useful I'd rather we used a tool designed for that - > https://docs.openstack.org/reno/latest/ > >> > >> > >> > >> This also natively handles our cheery pick process, and also > gives us the ability to edit changelogs if we notice a typo for instance. > >> > >> > >> > >> -ash > >> > >> > >> > >> On 14 June 2021 03:39:50 BST, Tzu-ping Chung > <[email protected]> wrote: > >> > >>> > >> > >>> There is also a tool called commitizen that helps teams > standardise commit messages. There is a Python implementation: > >> > >>> > >> > >>>https://commitizen-tools.github.io/commitizen/ > >> > >>> > >> > >>> I’m not a huge fan of semantic commit messages and never > personally used any tooling for it, but one of commitizen’s co-maintainers > is a friend of mine, and I’ve been listening to his pitches for a while > now, so I guess I don’t really mind and can probably adapt to it if needed. > As long as we don’t use emoji tags. Those are terrible. > >> > >>> > >> > >>> TP > >> > >>> > >> > >>> > >> > >>> "Janardhan" <[email protected]> (13/06/21 19:31:52): > >> > >>> > >> > >>> Hi, > >> > >>> > >> > >>> I am new to Apache Airflow. > >> > >>> > >> > >>> May be the following example guide would also be helpful. > >> > >>> > >> > >>> There is a commit style guide[1] for Apache SystemDS based it's > commit history over the last 10 years. > >> > >>> > >> > >>> It lists the tags used, and provides the type of commits > including a list of attributes to be included in its description and > example commits. > >> > >>> > >> > >>> Thank you, > >> > >>> Janardhan > >> > >>> > >> > >>> > >> > >>> [1] > https://github.com/apache/systemds/blob/master/CONTRIBUTING.md#commit-style > >> > >>> > >> > >>> > >> > >>> On Sunday, June 13, 2021, Tomasz Urbaszek <[email protected]> > wrote: > >> > >>>> > >> > >>>> Related discussion from last year ("Use semantic pull request"): > >> > >>>> > https://lists.apache.org/thread.html/r076232c60600238f37277497f66fb7eb9507869b92403c5ef96dcb3e%40%3Cdev.airflow.apache.org%3E > >> > >>>> > >> > >>>> > >> > >>>> On Sun, 13 Jun 2021 at 12:56, Jarek Potiuk <[email protected]> > wrote: > >> > >>>> > > >> > >>>> > Hey everyone, > >> > >>>> > > >> > >>>> > I would like to hear people's opinions on using > semantic/conventional > >> > >>>> > commits. I see people occasionally using it, but unless we > make it a > >> > >>>> > "standard" and mandatory (and fail CI if commits are not > following > >> > >>>> > it), IMHO there is virtually no benefit for the whole > community. > >> > >>>> > > >> > >>>> > I am now preparing the June provider's release (a little > delayed due > >> > >>>> > to my unavailability - sorry) and with 60+ providers it's > somewhat > >> > >>>> > manageable without it. I semi-automatically prepare and > maintain all > >> > >>>> > the changelogs now for all providers (I implemented a very > simple > >> > >>>> > heuristics to help with it and classify the commits based on > the > >> > >>>> > commit message) but it requires quite some effort to > re-classify the > >> > >>>> > changes. Not much, it's manageable, but having > semantic/conventional > >> > >>>> > commits would make my (and other release managers) life a bit > easier. > >> > >>>> > > >> > >>>> > For those who are not familiar with - here is the "gist" of > it with links: > >> > >>>> > > https://gist.github.com/joshbuchea/6f47e86d2510bce28f8e7f42ae84c716 > >> > >>>> > > >> > >>>> > In short - here are examples of semantic/conventional commit > messages: > >> > >>>> > > >> > >>>> > feat: add hat wobble > >> > >>>> > fix: fix the hole eaten by moles > >> > >>>> > doc: describe the hat etiquette > >> > >>>> > style: make hat follow latest hat conventions > >> > >>>> > refactor: replace hat underlying construction to be more > sturdy > >> > >>>> > test: test the hat when it's raining > >> > >>>> > chore: cleanup the hat, it became dusty a bit > >> > >>>> > > >> > >>>> > Questions: > >> > >>>> > > >> > >>>> > * What's your experience with using the semantic/conventional > commits? > >> > >>>> > * Do you like/dislike the semantic/conventional commits? > >> > >>>> > * Should we make them mandatory? > >> > >>>> > * Maybe there are other ways we can achieve the same results? > >> > >>>> > > >> > >>>> > J. > >> > >>>> > > >> > >>>> > > >> > >>>> > -- > >> > >>>> > +48 660 796 129 > >> > > > >> > > > >> > > > >> > >-- > >> > >+48 660 796 129 > >> > > >> > >> > >> -- > >> +48 660 796 129 > > > > -- > +48 660 796 129 > -- +48 660 796 129
