A Dijous, 2 de juliol de 2009, Jeff Mitchell va escriure: > The purpose of this proposal is to encourage KDE, as it migrates to Git, > to migrate to Gitorious.org as a hosting solution. Although I will not > be able to attend the KDE SCM BoF at Akademy/GCDS, Johan Sørensen (the > developer of Gitorious and founder of Gitorious.org) will be in > attendance. I hope that people will read this proposal and consider its > merit, then discuss the items presented here both on this list (for > initial discussion, and to come up with further questions) and at the > relevant SCM BoF when Johan can be present. > > ==Background== > > Gitorious is software that sits as middleware between Git and the user. > On the Git side, it manages access control to repositories and contains > macros and scripts that allow things like merge requests to be handled > intuitively through the web. On the web side, it manages authentication, > administrative control over users/groups/repositories, provides an > interface to code browsing that exceeds the capabilities of GitWeb, and > more. It's an excellent piece of software that is actively developed and > ever-increasing in features. It is already serving many projects well, > including Qt, and many of the workflow issues discovered as a result of > Qt's transition to it are actively being fixed. > > The current plan for KDE Git hosting is to host an instance of Gitorious > local to KDE's systems. Dirk Mueller has been tasked with looking into > this. > > However, hosting Gitorious within KDE has some drawbacks. The most > egregious is that it places burden on our system administrators, mainly > by requiring constant merging of the Gitorious code to keep up-to-date.
You mean git pull --rebase is time consuming? > This increases the time commitments required by a sysadmin team that is > already spread thin. It also means that support for modifications or > fixes needed by KDE is likely to be thin, in a more "when I get to it" > fashion by the Gitorious developers. > > In contrast, the most actively supported version of Gitorious is always > going to be the instance on Gitorious.org. Gitorious is self-hosted, and > used constantly by the main Gitorious developer, Johan Sørensen. In > addition, problems encountered by KDE would most likely be affecting > other projects as well, meaning that they would be likely to be fixed > quickly on the production site. > > > ==Proposal== > > KDE should choose Gitorious.org as its Git hosting solution. > > This proposal is organized as follows: First, explicit advantages are > discussed. Second, possible drawbacks as identified by Dirk (who has > looked into a Git migration) are presented. Third, concerns with the > migration itself are discussed. Finally, further considerations are > enumerated. > > > ==Advantages== > > Using Gitorious.org as a hosting solution would bring numerous technical > and community advantages. > > A1) The people managing our SCM instance are Git (and Gitorious) > experts. This lessens the need for our own sysadmins to become Git > experts simply to keep the site running. > > A2) Developers can share accounts between Qt and KDE, allowing for > easier cross-library collaboration and patches. > > A3) Developers use the same accounts when comitting code to KDE and > other (non-Qt) projects. This helps build community because it helps > build cross-references between KDE and other projects, and show > collaboration that is taking place. There is really any significant project we could contribute to on gitorious besides Qt? Having a look at active projects summary on gitorious seems not > A4) Nokia pays for hosted support and a SLA from Gitorious. According to > Thomas Zander, there is a possibility that they would be interested in > contributing further to cover KDE's costs as well. This would ease our > own hosting and support burden and would increase Gitorious' capacity. > This doesn't preclude the possibility of us collaborating with > Gitorious.org as described in the Migration and Capabilities section. > > A5) KDE and Gitorious (and Qt) benefit from cross-marketing and > promotion. We'd be the most visible project on the site (most likely > above Qt in terms of activity), and our involvement would help make > Gitorious more visible and legitimate. That's not an advantage for us it's an advantage for gitorious. > A6) It makes KDE development look more community-related and less of a > walled garden. See my comment to A3, as there's not much big projects in gitorious other than Qt and KDE it would just be a different walled garden. > > It is obvious that many of these points are wins in terms of marketing. > In fact, in discussions with members of the Marketing Working Group, I > posed the question as to what the marketing advantages and disadvantages > of following in Qt's footsteps and using Gitorious.org as our hosting > provider would be, and many of the above points came directly from this > conversation. > > > ==Drawbacks== > > These are some of the drawbacks of using Gitoriorious.org as a hosting > provider. > > When discussing the Git testing with Dirk, I broached the topic of using > Gitorious.org instead of KDE hosting its own instance. The reasons I was > told that it was not considered were as follows (verbatim): > > D1) "I think it is important to have full control over our core > infrastructure to stay independent." > > While it is true that control over our core infrastructure helps ensure > our independence, part of the draw of Git is that clones are identical > copies of the repository. Should we ever actually need to switch hosting > solutions, migration of the actual source code should be relatively > simple -- pulling the repositories from one place and pushing them to > another. > > Of course, this would entail a loss of the extra value provided by > Gitorious.org. Since Gitorious is open-source we could still use it, but > it may require some significant effort to regain the extra value built > up on Gitorious.org. > > D2) "There is not much other reason than that we have the hardware and > the hosting, so we can do it." > > This is provided for completeness, but there isn't much discussion > necessary. I agree with dirk, it's our code, let's be ourself that hosts it. > ==Migration and Capabilities== > > I've discussed the possibility of migration with Johan to think of the > issues that could arise and how they could be avoided. These are the > migration concerns that I posed to Johan, and his advice/answers. > > S1) Commitfilter > > Although running arbitrary shellscripts on Gitorious.org as git hooks is > not allowed for security reasons, Johan is planning to add support to > POST the data of commits to URLs (it's in a branch that will be > integrated soon). This will allow you to supply one or more callback > URLs for a repository, so whenever someone pushes the details of the > commit are POSTed via HTTP to the callback URL(s) as JSON. It includes > items like the actual commit, who pushed it, and the before/after SHAs. > > This could be used (with IP-based access control for security) to still > be able to provide commitfilter-like functionality (although > commitfilter as it exists would likely need to be updated as a result of > the SCM difference in the first place). Afair commitfilter works at kde-commits mailing list level so no rewriting needed for commitfilter (though you still need to rewrite kde-commits mailing list sending) > > S2) Capacity > > Capacity is a concern, as KDE would become the largest project both in > terms of source code and activity. When these issues was posed to Johan, > here was his answer: > > 'We should look into some kind of general mirroring facility in the long > run (not really because of bandwidth used, but also more to speed up > transfers on other continents). We do pay for SAN storage and bandwidth, > but there's "plenty available" as long as we pay for it.' > > However, there are two ways this could be mitigated: > > * Johan said that extra capacity (bandwidth and disk space) is available > for purchase in his datacenter, which will ensure capacity is not a > problem if KDE/Nokia are providing monetary support in return for the > hosting. Bandwidth for the site is currently costing them about > $120USD/month. > > * The servers currently used by KDE for distributing code via Subversion > could become a part of a wider Gitorious.org infrastructure, acting as > local mirrors and distribution centers for the subset of KDE > repositories and clones. This benefits both KDE and Gitorious.org by > speeding up transfers to various parts of the world, increasing > redundancy, and easing the burden on the main site. For instance, KDE > could handle reads on the various KDE repository and clones, while > pushes still go to the main server. This would significantly cut down on > bandwidth at the main Gitorious.org site. > > Johan is working on getting an estimate for the cost of a SLA for KDE. Why pay 120$/month if we have KDE servers hosted by free? > > S3) Scripty > > Chani Armitage did some wonderful work porting Scripty to Git. Part of > the work includes support for different backends, so that the > translators can still use SVN while projects slowly migrate. > > S4) Identity > > It is understandable if people would like our Git repositories to be > available at git.kde.org. HTTP redirects/proxying or other techniques > could be used to redirect (transparently or otherwise) git.kde.org to > e.g. kde.gitorious.org. Johan provided some further information: > > 'It would be a similar approach such as how http://qt.gitorious.org is > being done. In nutshell we wrap that site in a custom layout > (markup+css+images) and projects are then associated with that site, > which in turn means that they'll be presented in the same layout and > domain.' > > S5) Testing the Waters > > Amarok has been chomping at the bit to switch to Git, and the Amarok > developers are willing to be the guinea pigs. If it is decided that > Gitorious.org would be a desirable hosting platform, Amarok will > immediately port to using it and start ironing out any issues. TagLib > would also be switching shortly afterwards. > > > ==Further Considerations== > > These are further considerations, questions that may be asked that have > been posed to Johan, or just relevant points about Gitorious. > > C1) Are we able to fix technical problems with Gitorious.org or get them > fixed in a timely manner? > > Johan's answer: > > 'I think we could find some kind of arrangement here perhaps, but we'd > have to look into how to solve practically it first, since there are few > things worth keeping in mind: > * We do have an SLA towards companies such as Nokia > * We do have a privacy policy that would need to be governed > * We do pay a good amount of money for system administration already (as > a result of #1) > ** It only goes so far when it comes to actual > gitorious-the-application-itself things, but that would be the same > situation with any potential KDE admin (at least to begin with) > > The upside is that apart from personal information supplied by users, > there aren't any secret IP used or stored, since it is all under some > kind of OSS license' > > What Johan is referring to is a suggestion by me that trusted KDE > sysadmins could have access to admin Gitoroius.org as well. He is not > opposed to the idea, but the exact parameters would need to be worked > out (which may just entail ensuring that appropriate KDE sysadmins are > versed in how the site works and can be trusted not to make any problems > that happen *worse*). > > C2) What happens if Gitorious.org goes down, how long would it take to > get it up and running again? > > Johan's answer: > > '"if there's a bug, how long does it take to fix?" :) > > In the case of hardware failures, we do have redundant hardware. We > currently run Gitorious.org on two sun x4150's with 32GB of ram, > connected to a SAN (over FiberChannel) where the actual Xen images and > repository data etc is stored. The machines only utilitize 50% of the > resources that can't be grown or shrunk, so we can move the images > between them in case of hardware failure.' > > C3) How will account management and access control work with > Gitorious.org? Do we have full control over that? > > Johan's answer: > > "The easiest way would probably be to create one or more teams, such as > the one Thiago has created here: http://gitorious.org/+kde-developers > (or use that). That would allow anyone with an administrator role within > that team to add/remove/promote/demote members and all members would > have commits rights to any projects or repositories either owned by that > team, or where the team is added as committers to a repository. Whoever > has commits rights to repositories is entirely governed by the owner of > that repo, which can be either a single user or a team (in which case > the admins of that team would have the ability to do admin type things > on the repo or project)." > > To me, this sounds like an entirely reasonable method of managing > committers. Established KDE developers could be part of the > kde-developers group, SoC students could be part of a kde-soc-students > group while SoC is going on, and those that continue comitting could be > moved into kde-developers, etc. Why give newbies less commit rights, kde has never worked this way. I like the way we have now (with very few ACLs) > C4) What happens if the Gitorious.org maintainer goes mad and tries to > harm KDE or some such thing? > > Johan's answer: > > 'There is more than one maintainer of Gitorious these days, and we > (Shortcut, a company I co-founded, which manage on Gitorious.org) do > have commercial interests and contracts with companies such as Qt > software, doing anything harmful or destructive would be a, well, bad > business move. Any agreement we come to should probably include some > kind of exit strategy for KDE, meaning anything that isn't blocked by > the privacy policy could get transfered back to KDE. And the actual Git > repositories would be in more than one place anyway (as opposed to a > single central SVN repository).' > > C5) Will 'push -f' be disabled? We can't have a public repository with > that allowed. > > Johan's answer: > > 'Yes, we're refactoring the pre-receive hook handling quite a bit right > now (for the web-hooks and new merge-request features, among other > things) so it is something that'll get implemented within a week I think.' > > C6) What are some pros/cons vs. GitHub? Obviously being open-source is a > killer feature, but it'd be good to know what is coming at Gitorious > that GitHub doesn't have, since there are many things GitHub has that > Gitorious doesn't. What's the Gitorious roadmap? > > Johan's answer: > > 'Well, what features do you miss the most from Github then? > > I think the fact that Gitorious pivots around projects as a whole, > rather than individual repositories is a major advantage. Since it helps > in keeping everyone moving towards to the same thing ("the project") in > the long run, rather than encouraging lots of siloed development. > Personally, I also think it works better for collaborating on large > projects. > > I also think that the merge-requests on gitorious is a major advantage, > rather than simply being a simple notification feature they can actually > be used to implement workflow features such as code-review (with inline > commenting) and pushing directly to a magic ref in the target > repository, which creates a new merge request (or updates an existing > one). The last two items are something we're working hard on right now, > as part of our consulting work for Qt. > > We're also gearing up for some more design and UI improvements. > > I also think that the fact that Qt is there would be a major advantage > for KDE.' > > I'd like to second something he said there. Being a user of both GitHub > and Gitorious, I find that despite what GitHub advertises "Social > Coding", the Gitorious paradigm feels much more social. On GitHub, they > encourage you to "fork" projects (their word for what everyone else > calls "clone" and have no mechanism for real collaboration on the site > other than sending merge requests and sending notes. (There are some > extras, like the ability to have commits go into chatrooms and such that > help with collaboration.) You can have multiple people with commit > rights to a project, but they're simply extra allowed SSH keys. In > short, GitHub feels more oriented around individual users, and in a > sense almost feels isolating, not social. > > Gitorious has a different feel in that not only are there teams, but > teams can own projects. The team paradigm helps people feel like they > are really part of a collaborative effort, especially when a team owns a > project. It may not be a huge difference from GitHub in terms of the > paradigm, but it really gives it a different feel in my experience, and > one that makes it feel far more social than GitHub. > > > ==Summary== > > Gitoroius.org as a hosting platform has many advantages and few, if any, > real drawbacks. The developer of Gitorious would be thrilled to have KDE > hosted there, and there are many indications that it would be a mutually > beneficial relationship. It would ease account management and sysadmin > burden while not significantly impacting our autonomy or identity, and > would allow KDE and Qt collaboration and development to happen more > easily. Johan has indicated that he would be willing to help us migrate > and be willing to prioritize adding features or fixing bugs that we > would discover or need short-term as we migrate; in addition, he is also > happy to discuss having our help and input with system administration > should we want it, which helps ensure that we have both a stake in the > project and an exit plan should an unfortunate situation arise. > > Gitorious.org is the logical choice for hosting our Git repositories > compared to hosting it ourselves, and should be given serious > consideration. What about pre-commit hooks? We have some of those i would not like to loose. Albert _______________________________________________ Kde-scm-interest mailing list [email protected] https://mail.kde.org/mailman/listinfo/kde-scm-interest
