Hi Christoph, Joe, and everyone, > Let’s take a step back and reflect on what we actually need. Some of you > have expressed the desire to have a more web-based or modern (or > github-like…) Kwant development workflow.
Christoph, I believe you're downplaying the reasoning here, it's not as arbitrary. Ultimately, the reason why I raised this question together with others is not the similarity of the workflow to github or it being web-based, but the entry level for contributing. I have just checked with the Kwant survey that approximately half of the potential contributors don't use version control. While I believe the physics community as a whole benefits from the proliferation of version control, Kwant itself benefits from having contributors more than from enforcing best practices. So I want to make it reasonably easy to contribute to Kwant for people who don't have time or wish to learn details of git usage, PEP 8, PEP 257 (as long as the contributions are actually useful of course). My own first commit to a public project, a docstring edit, was done when I didn't feel comfortable with git, and it was made using entirely web interface of github. You may consider it not relevant, but I think checking such a pull request is less work for a developer than going the complete thing from scratch. Making another step back you could ask why we bother using git at all if it's so hard to learn, and for me the answer is that it makes some tasks easier for us. The aspect relevant to our discussion is that it allows to easily see what exactly was done, ensure that it's easy to modify, reapply, or undo the changes, and to see who did what. I agree with you that it is not a matter of pedantry, but a very useful feature. This still leaves us with freedom to decide how we use git, and the current approach consists of maintaining a linear history where every commit is meaningful. The "github" approach results in a history with a lot of merge commits and many commits on feature branches breaking things or being generally of poor quality, so it appears to hurt the usefulness of version control. However, looking around, I have found that git allows to achieve comparable results with dirty branches. For example, '--first-parent' will make the history easy to navigate (e.g. like this: http://antonakhmerov.org/misc/first-parent.png), and rebase replaces cherry-pick. So it is possible to keep the history clean using github approach, but it relies on using different conventions for what is considered clean. I do realize that this approach shifts the some of the burden from the contributors to developers, since manipulating github-style history is marginally harder. Let me now comment on the current Kwant workflow aspects (a1-a4 in your email). As I have discussed above the workflow aspects a2 (all commits clean) and a3 (minimal amount of merges) aren't of universal importance being instead a part of an agreement on how to interpret development history. While Peter Hintjens in his EuroScipy presentation argued that there is no need to rely on experienced reviewers accepting patches (a1), I think this aspect has no impact on our tool choice and I don't have a strong opinion about it. Finally I agree that there's a merit in not resctricting the tools contributors use (a4), but as long as we have operational email accounts, anybody may just email us and not use any of the tools. As Joe also writes in his reply, we aim to simplify the workflow for people who do not have a preference of the tools they want to use. Proceeding to the things that should be improved from status quo: > • (p1) There is no good way to query all existing bugs and issues... > • Even more relevant IMHO are not bugs but tasks/wishlist items... I agree. > • (p2) The github crowd has no obvious way to contribute to Kwant. I do not > think that this is a big problem, but it would be certainly nice to allow > contributions in a way that is familiar to many people. BUT I strongly > insist on not giving up nice aspects of our own way of working because > “everybody else is doing it like that”. Also, I think that it is crucial > that it’s possible to contribute to a piece of software without having to > accept the terms & conditions of some specific non-free software. Alas, > many projects do not see this problem. I would extend this: not only the github users have no obvious way to contribute, but also physicists with limited programming skills. Further "Everybody else is doing it like that" is indeed not an ultimate argument, but it is an argument nonetheless since it saves time to contributors familiar with a common convention. As I argued above, I believe we can adopt the other workflow while not giving up the practical consequences of our current workflow. > • (p3) Currently, most code review is done in person-to-person > conversations. As we are spread geographically, it would be nice to make > this more traceable for others. Also, the stages of evolution of a > patchset are not preserved through multiple rewrites. I agree. > • (p4) It could be beneficial to setup some way of automated testing of the > code base (confusingly, that often is called “continuous integration”): If > a patchset gets proposed, it would be nice to see whether all tests pass. > But then, if a contributor is unable to internalize the simple rule that > all tests should always pass, that’s a bad contributor. Also, if code > review is done in a reasonable way, it should be trivial for the reviewer > to execute the tests himself (as it is now). As I already mentioned to you in person, this makes me a bad contributor :-) I try my best, but sometimes I forget. Automated tests allow both contributors and reviewers to spare some mental effort. Since the goal of programming is to make routine tasks automated, I'm very much in favor of automated testing. > Before we continue with evaluating various tools, let’s first discuss what > we would like to have. I’m looking forward to your comments. While I was typing this email, Joe replied, and Christoph as well. I will discuss software details as a reply to those message. Best, Anton
