Hi Ben,
It isn't just the tool itself which has to be maintained: we have commit hooks, integration with other bits of infrastructure and so forth which also needs to both be implemented and maintained.
In case of Gerrit, there is no need for custom hooks as they stay on git.kde.org, and therefore I believe this point is not relevant to its adoption. The whole setup has been designed and implemented in a way suitable for long-term parallel operation alongside KDE's git.
As for the integration bits, they're done now. The tool just talks to LDAP, and maintenance of that connection is effectively zero, unless a dramatic revamp of our LDAP is planned. The repo mirroring was a matter of setting up a single user account and configuring proper ACLs, and these are also finished already.
I can understand the general reasons for limitting the number of services which we offer and support. However, I would appreciate if we mentioned how big these costs are, as there's some room of misinterpretation otherwise.
The more custom work we have, the harder it is to upgrade things.
While true in general, I fail to see how it is relevant to Gerrit. What custom bits are involved here?
We'll confuse newcomers if projects A, B and C are reviewed on tool X while projects D, E and F are reviewed on tool Y.
I haven't received a single complaint from the Trojita GCI students about any hardness in this. They did struggle with making an actual change to the code, with proper coding style, with commit message formatting, with git in general, they even failed to understand the written text about CCBUG and BUG keywords in the our wiki, but nope, I haven't seen them struggle with a need to use Gerrit or its differences to RB. YMMV, of course.
Because the majority of complaints actually came from people who are well-versed with ReviewBoard, my best guess is that there's muscle memory at play here. This is supported by an anecdote -- when I was demoing the RB interface to a colleague who maintains Gerrit at $differentOrg, we both struggled with finding buttons for managing a list of issues within RB. It's been some time since I worked with RB, and it showed.
I remember having hard time grokking the relation between a "review" and "attaching/updating a file" on RB. I didn't read the docs, and it showed.
A single tool would be best here. Let me make clear that it is not a case of Reviewboard vs. Gerrit here - as other options need to be evaluated too.
I understand that people would like to avoid migrating to Gerrit if a migration to a $better-tool was coming. Each migration hurts, and it makes a lot of sense to reduce the number of hops.
However, what I'm slightly worried about is postponing Gerrit indefinitely until all future replacements are evaluated. I don't see people putting significant time into any alternative for code review right now. Do we have any chances of these people making themselves known in close future? How long would be a reasonable waiting time for a testing deployment of alternate tools? When are we going to reduce our candidates just to the contenders which have been deployed and tested by some projects?
In regards to the difficulty of Gerrit - I tend to agree with this argument (it took me at least a minute to find the comment button, and I didn't even get to reviewing a diff).
The documentation, however, explains the functionality in a pretty clean manner, see https://gerrit.vesnicky.cesnet.cz/r/Documentation/user-review-ui.html .
We also aren't the first project trying to work with Gerrit, so there's plenty of tooling available right now, not "to be written". There's text-mode interface, the "gertty" project, there's integration in QtCreator, there are pure-CLI tools for creating reviews, another web UIs in development, there are even Android clients.
Plus there are major concerns with integration into our infrastructure, such as pulling SSH keys from LDAP for instance (no, you can't have the tool maintain them itself - as mainline Git and Subversion need the keys too).
Yes, SSH-keys-in-LDAP is a PITA, but given that one needs a patched OpenSSH to look up keys from LDAP anyway, I don't think this is a blocker issue. The situation is exactly the same with the Gitolite setup which currently runs on git.k.o though, as that doesn't talk to LDAP either. As you mentioned during our IRC chat, there's a Python daemon which polls for changes in LDAP, and propagates these into Gitolite's config backend in a non-public git repo. Why wouldn't this be enough for Gerrit, then?
Gerrit has both SSH-authenticated API and a REST HTTPS API for adding and removing of SSH keys by an admin account. *If* this is needed, I'll be happy to make it work, it's simply a matter of calling two trivial scripts. Would you see any problems with hooking into the identity webapp or its backend, if there's any, for this? An edge trigger would be cool.
Are there any other concerns?
Please note that any discussion of tools should be on the merits of the tools themselves. Things like CI integration are addons, which both Reviewboard and Gerrit are capable of.
With varying levels of ease, I should add. In the end, everything is achievable, and you can write tools which automatically pull patches sent through mailing lists and build that, but the question is who is going to do the work, and when it's going to be ready. I know that the CI+Gerrit thing is now done and solved, and I also know that I won't be spending my time in redoing the same for ReviewBoard. Nobody bothered to do this pre-merge CI for RB in the past years. Do we have some volunteers now?
The only reason we don't have Reviewboard integration yet is a combination of technical issues (lack of SNI support in Java 6)
And nobody caring enough to do the work, I suppose. AFAIK Jenkins runs on Java 7 just fine, but apparently nobody found time for such an upgrade. There's nothing bad with this, of course, but it doesn't suggest that suddenly these people will have time for setting up early CI with RB.
and resource ones (some projects take a long time to complete, and i'm concerned we don't have the processing power).
This is not really an all-or-nothing question. If there's a project which exceeds our current HW possibilities (you mentioned Calligra before, right?) and we cannot easily get more HW (did someone ask the foundation's treasurers for funds for HW rental, or approached some of the obvious candidates such as RedHat or SuSE asking for HW access?), perhaps that project can simply be omitted from these pre-merge CI runs.
We've chatted a couple of times about the limits of the current CI setup, about its inability to perform checks in parallel. Is the existing architecture going to scale with these pre-merge CI runs without substantial changes?
Again, this is something which is solved now with the CI setup that is behind Gerrit. The changes were into the glue code, the code which schedules the builds, distributes the jobs and which decides when to build what. It's still using the KDE CI's Python scripts for managing library deps and for actually launching the build (I sent the necessary patches your way).
In terms of a modern and consistent project tool - I agree here. A long term todo item of sysadmin is to replace projects.kde.org. The question is of course - with what. Chiliproject is now unmaintained, so we do have to migrate off to another solution at some point. If the new tool happens to be more integrated in terms of code review, that is a bonus from my point of view (as it means the integration will be better, and there is one less piece of infrastructure to maintain).
See above for my view of mixing the quest for finding a decent project management tool and for finding a good code review system. To go a bit to the meta side, IMHO, a universal tool which does plenty of things in a not-excellent manner is worse than using diverse set of tools which do one thing each, and do it well. That's why I see a Chilliproject replacement to be an orthogonal topic to the choice of a code review tool.
@Jan: could you please outline what you consider to be the key advantages? At the moment I understand that you are after: 1) CI integration to pre-validate the change before it gets reviewed
- With the CI actually testing not just the change in isolation (as a result of it on top of what was at the repo at the time the change was made), but the result of the change as applied to the current state of the repo at merge time, and with user-visible possibility of retruggering a check job.
- Being able to do "trunk gating" for projects which care enough. That is, there's a tool which makes sure that there are no regressions, and which won't let in commits which break the build or cause tests to fail. (And yes, I know that there'll always be an option of direct pushes, I am not pushing against that, it appears to be a point which people require. OK with me.)
- Being able to do cross-project verification, i.e. "does this change of kio break plasma-framework?"
- Performing builds on various base OSes, against the Qt version provided by system (Trojita aims at supporting Qt 4.6-4.8 and 5.2+), using ancient compilers (yes, C++11 and GCC 4.4 are so much fun when taken together) and different mixes of optional dependencies and features to be built. In short, testing in the env people will be using, including Windows and Mac OS X.
2) Ability to directly "git pull" the patch (which Phabricator's arc tool would meet I believe?)
With Gerrit, one has an access to the full history of each change, including accessing them in offline mode. This is opt-in, so people who don't care will not have their clones "polluted by this nonsense", while people who do care have them "enhanced by this valuable data". This happens with no extra tooling. I'm sure I could come up with e.g. local scripts that build these git refs from the (history of) patches on RB, Phabricator, GitHub, Gitorious or whatever, but native support trumps scripting each time.
According to the docs, `arc` is something which just pushes patches around. Working with patches is different from having a git ref to work on.
Have you ever used OpenSUSE's Build Service and its CLI client, the `osc`? That's what happens when one tries to reimplement a SCM when working on a tool whose primary focus is something else. My favorite misfeature is the "expand and unexpand linked packages" thingy, and the associated hiding of merge conflicts when a source package changes. That's not fun to debug, and it won't ever happen with plain git because git comes with excellent support for merges. That support is a result of many years of extremely heavy use of git by a ton of developers. I don't expect Facebook to be able to cope with *that* regardless of their engineering size, and therefore I expect that `arc` will fail when people use it for non-obvious stuff.
I suppose most of our developers are either already familiar with git, or they have to learn it anyhow to be able to participate in our community in an efficient manner. Introducing another patch manager to the mix doesn't help, IMHO. This is just to illustrate my experience with tools that behave-quite-like-a-SCM but do not actually implement full SCM functionality. It works well for quick demos, but it sucks when you start using them seriously.
I would encourage anyone who evaluates these tools to pay attention to these not-so-obvious usage issues. Having a CLI tool that can fetch a patch and apply it to a local checkout is not equivalent to native git refs. It's an important building block, but not a finished tool.
I know that I won't be in a business of building these tools. I'm quite happy with having them out-of-box with Gerrit.
Cheers, Jan -- Trojitá, a fast Qt IMAP e-mail client -- http://trojita.flaska.net/
