Re: [igt-dev] RFC: Migration to Gitlab
On Fri, Aug 24, 2018 at 8:52 AM, Jani Nikula wrote: > On Wed, 22 Aug 2018, Rodrigo Vivi wrote: >> On Wed, Aug 22, 2018 at 10:19:19AM -0400, Adam Jackson wrote: >>> On Wed, 2018-08-22 at 16:13 +0300, Jani Nikula wrote: >>> >>> > - Sticking to fdo bugzilla and disabling gitlab issues for at least >>> > drm-intel for the time being. Doing that migration in the same go is a >>> > bit much I think. Reassignment across bugzilla and gitlab will be an >>> > issue. >>> >>> Can you elaborate a bit on the issues here? The actual move-the-bugs >>> process has been pretty painless for the parts of xorg we've done so >>> far. >> >> I guess there is nothing against moving the bugs there. The concern is only >> on >> doing everything at once. > > No, it's not just that. > > We have some automation using the bugzilla APIs directly, and > someone(tm) needs to figure out how this should work with gitlab. Maybe > we have a better chance of doing things with gitlab's APIs, maybe we can > reduce our dependence on external logic altogether. > > We have special "i915 platform" and "i915 features" fields in > bugzilla. We use a mailing list default assignee. Some of us use the > "whiteboard" and "keywords" fields. Etc. > > I don't think figuring all this out is rocket science, but someone needs > to actually do it, and get our workflows straightened out *before* we > flip the switch. I'm just trying to figure out if that is a blocker to > migrating the repos. I think the mesa approach of flat-out disabling gitlab issues and merge request is solid. That way there's no confusions with contributions and bug reports stranded in the wrong place. This should allow us to handle all the different feature sets gitlab provides independently and as we see fit: Repo hosting, documentation/artifact hosting, CI integration, patch submissions, issue tracking. We can choose what we think is good, and ignore everything else. And of course if 1-2 years down the road things change, we can reevaluate. I think for a rough timeline if things go well we'll have igt and maintainer-tools migrated by end of year (just the repos, nothing else), and a solid plan for migrating all the drm* repos next year. Everything else is too far away to have solid plans for, or even just a timeline. -Daniel -- Daniel Vetter Software Engineer, Intel Corporation +41 (0) 79 365 57 48 - http://blog.ffwll.ch ___ amd-gfx mailing list amd-gfx@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/amd-gfx
Re: [igt-dev] RFC: Migration to Gitlab
On Wed, 22 Aug 2018, Rodrigo Vivi wrote: > On Wed, Aug 22, 2018 at 10:19:19AM -0400, Adam Jackson wrote: >> On Wed, 2018-08-22 at 16:13 +0300, Jani Nikula wrote: >> >> > - Sticking to fdo bugzilla and disabling gitlab issues for at least >> > drm-intel for the time being. Doing that migration in the same go is a >> > bit much I think. Reassignment across bugzilla and gitlab will be an >> > issue. >> >> Can you elaborate a bit on the issues here? The actual move-the-bugs >> process has been pretty painless for the parts of xorg we've done so >> far. > > I guess there is nothing against moving the bugs there. The concern is only on > doing everything at once. No, it's not just that. We have some automation using the bugzilla APIs directly, and someone(tm) needs to figure out how this should work with gitlab. Maybe we have a better chance of doing things with gitlab's APIs, maybe we can reduce our dependence on external logic altogether. We have special "i915 platform" and "i915 features" fields in bugzilla. We use a mailing list default assignee. Some of us use the "whiteboard" and "keywords" fields. Etc. I don't think figuring all this out is rocket science, but someone needs to actually do it, and get our workflows straightened out *before* we flip the switch. I'm just trying to figure out if that is a blocker to migrating the repos. BR, Jani. -- Jani Nikula, Intel Open Source Graphics Center ___ amd-gfx mailing list amd-gfx@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/amd-gfx
Re: [igt-dev] RFC: Migration to Gitlab
On Wed, Aug 22, 2018 at 05:37:22PM +0100, Daniel Stone wrote: > Hi Rodrigo, > > On Wed, 22 Aug 2018 at 17:06, Rodrigo Vivi wrote: > > On Wed, Aug 22, 2018 at 10:19:19AM -0400, Adam Jackson wrote: > > > On Wed, 2018-08-22 at 16:13 +0300, Jani Nikula wrote: > > > > - Sticking to fdo bugzilla and disabling gitlab issues for at least > > > > drm-intel for the time being. Doing that migration in the same go is a > > > > bit much I think. Reassignment across bugzilla and gitlab will be an > > > > issue. > > > > > > Can you elaborate a bit on the issues here? The actual move-the-bugs > > > process has been pretty painless for the parts of xorg we've done so > > > far. > > > > I guess there is nothing against moving the bugs there. The concern is only > > on > > doing everything at once. > > > > I'm in favor of moving gits for now and after we are confident that > > everything is there and working we move the bugs. > > As Daniel alluded to, the only issue I really have is moving _all_ the > kernel repos at once. At the end of the year we'll have easy automatic > scaling thanks to the independent services being separated. As it is, > all the GitLab services (apart from CI runners) run on a single > machine, so we have limited options if it becomes overwhelmed with > load. > > Do you have a particular concern about the repos? no concerns from my side about the repos itself. From my committer perspective on libdrm, mesa the migration was really smooth. > e.g. what would you > check for to make sure things are 'there and working'? more in terms of other committers getting used to it, dim working for most of committers, links in documentations and wikis updated... but no concerns with the infra itself. > > > One question about the bugzilla: > > > > Will all the referrences on all commit messages get outdated after > > bugzilla is dead? > > Or bugzilla will stay up for referrence but closed for interaction? > > or all old closed stuff are always moved and bugzilla.fd.o as well and > > bugzilla.fd.o will be mirroring gitlab? > > When bugs are migrated from Bugzilla to GitLab, only open bugs are > migrated. Closed ones are left in place, as is; open ones have a > comment at the end saying that the bug has moved to GitLab, a URL > linking to the new GitLab issue, and telling them to please chase it > up there. > > Even when we move everyone completely off Bugzilla, we will keep it as > a read-only mirror forever. Even with Phabricator, which very few > people ever used, has had all its bugs and code review captured and > archived, so we can continue to preserve all the old content and > links, without having to run the actual service. Great! Thanks for all clarification, Rodrigo. > > Cheers, > Daniel ___ amd-gfx mailing list amd-gfx@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/amd-gfx
Re: [igt-dev] RFC: Migration to Gitlab
Hi Rodrigo, On Wed, 22 Aug 2018 at 17:06, Rodrigo Vivi wrote: > On Wed, Aug 22, 2018 at 10:19:19AM -0400, Adam Jackson wrote: > > On Wed, 2018-08-22 at 16:13 +0300, Jani Nikula wrote: > > > - Sticking to fdo bugzilla and disabling gitlab issues for at least > > > drm-intel for the time being. Doing that migration in the same go is a > > > bit much I think. Reassignment across bugzilla and gitlab will be an > > > issue. > > > > Can you elaborate a bit on the issues here? The actual move-the-bugs > > process has been pretty painless for the parts of xorg we've done so > > far. > > I guess there is nothing against moving the bugs there. The concern is only on > doing everything at once. > > I'm in favor of moving gits for now and after we are confident that > everything is there and working we move the bugs. As Daniel alluded to, the only issue I really have is moving _all_ the kernel repos at once. At the end of the year we'll have easy automatic scaling thanks to the independent services being separated. As it is, all the GitLab services (apart from CI runners) run on a single machine, so we have limited options if it becomes overwhelmed with load. Do you have a particular concern about the repos? e.g. what would you check for to make sure things are 'there and working'? > One question about the bugzilla: > > Will all the referrences on all commit messages get outdated after > bugzilla is dead? > Or bugzilla will stay up for referrence but closed for interaction? > or all old closed stuff are always moved and bugzilla.fd.o as well and > bugzilla.fd.o will be mirroring gitlab? When bugs are migrated from Bugzilla to GitLab, only open bugs are migrated. Closed ones are left in place, as is; open ones have a comment at the end saying that the bug has moved to GitLab, a URL linking to the new GitLab issue, and telling them to please chase it up there. Even when we move everyone completely off Bugzilla, we will keep it as a read-only mirror forever. Even with Phabricator, which very few people ever used, has had all its bugs and code review captured and archived, so we can continue to preserve all the old content and links, without having to run the actual service. Cheers, Daniel ___ amd-gfx mailing list amd-gfx@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/amd-gfx
Re: [igt-dev] RFC: Migration to Gitlab
Hi, On Wed, 22 Aug 2018 at 15:44, Daniel Vetter wrote: > On Wed, Aug 22, 2018 at 3:13 PM, Jani Nikula > wrote: > > Just a couple of concerns from drm/i915 perspective for starters: > > > > - Patchwork integration. I think we'll want to keep patchwork for at > > least intel-gfx etc. for the time being. IIUC the one thing we need is > > some server side hook to update patchwork on git push. > > > > - Sticking to fdo bugzilla and disabling gitlab issues for at least > > drm-intel for the time being. Doing that migration in the same go is a > > bit much I think. Reassignment across bugzilla and gitlab will be an > > issue. > > Good points, forgot about both. Patchwork reading the mailing list > should keep working as-is, but the update hook needs looking into. All the hooks are retained. gitlab.fd.o pushes to git.fd.o, and git.fd.o still executes all the old hooks. This includes Patchwork, readthedocs, AppVeyor, and whatever else. > For merge requests I think best approach is to enable them very > selectively at first for testing out, and then making a per-subproject > decision whether they make sense. E.g. I think for maintainer-tools > integrating make check and the doc building into gitlab CI would be > sweet, and worth looking into gitlab merge requests just to automate > that. Again best left out of scope for the initial migration. You don't need MRs to do that. You can build a .gitlab-ci.yml file which will run make check or build the docs or whatever, and have that run on pushes. Anyone can then fork the repo, push their changes to that fork, and see the CI results from there. It's like Travis: the CI configuration is a (mutable) part of the codebase, not a separate 'thing' hanging off a specific repo. So if you write the CI pipeline first, you can have people running CI on push completely independently of switching the workflow to use MRs. Cheers, Daniel ___ amd-gfx mailing list amd-gfx@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/amd-gfx
Re: [igt-dev] RFC: Migration to Gitlab
On Wed, Aug 22, 2018 at 10:19:19AM -0400, Adam Jackson wrote: > On Wed, 2018-08-22 at 16:13 +0300, Jani Nikula wrote: > > > - Sticking to fdo bugzilla and disabling gitlab issues for at least > > drm-intel for the time being. Doing that migration in the same go is a > > bit much I think. Reassignment across bugzilla and gitlab will be an > > issue. > > Can you elaborate a bit on the issues here? The actual move-the-bugs > process has been pretty painless for the parts of xorg we've done so > far. I guess there is nothing against moving the bugs there. The concern is only on doing everything at once. I'm in favor of moving gits for now and after we are confident that everything is there and working we move the bugs. One question about the bugzilla: Will all the referrences on all commit messages get outdated after bugzilla is dead? Or bugzilla will stay up for referrence but closed for interaction? or all old closed stuff are always moved and bugzilla.fd.o as well and bugzilla.fd.o will be mirroring gitlab? Thanks, Rodrigo. > > - ajax > ___ > dim-tools mailing list > dim-to...@lists.freedesktop.org > https://lists.freedesktop.org/mailman/listinfo/dim-tools ___ amd-gfx mailing list amd-gfx@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/amd-gfx
Re: [igt-dev] RFC: Migration to Gitlab
On Wed, Aug 22, 2018 at 3:13 PM, Jani Nikula wrote: > On Wed, 22 Aug 2018, Daniel Vetter wrote: >> Hi all, >> >> I think it's time to brainstorm a bit about the gitlab migration. Basic >> reasons: >> >> - fd.o admins want to deprecate shell accounts and hand-rolled >> infrastructure, because it's a pain to keep secure >> >> - gitlab will allow us to add committers on our own, greatly >> simplifying that process (and offloading that task from fd.o admins). >> >> There's also some more benefits we might want to reap, like better CI >> integration for basic build testing - no more "oops didn't build >> drm-misc defconfigs" or "sry, forgot make check in maintainer-tools". >> But that's all fully optional. >> >> For the full in-depth writeup of everything, see >> >> https://www.fooishbar.org/blog/gitlab-fdo-introduction/ >> >> I think now is also a good time, with mesa, xorg, wayland/weston and >> others moved, to start thinking about how we'll move drm. There's a >> few things to figure out though: >> >> - We probably want to split out maintainer-tools. That would address >> the concern that there's 50+ committers to an auto-updating shell >> script ... >> >> - We need to figure out how to handle the ACL trickery around drm-tip in >> gitlab. >> >> - Probably good to stage the migration, with maintainer-tools, igt >> leading. That will also make fd.o admins happy, who want to rework >> their cloud infrastructure a bit before migrating the big kernel repos >> over. >> >> - Figuring out the actual migration - we've been adding a pile of >> committers since fd.o LDAP was converted to gitlab once back in >> spring. We need to at least figure out how to move the new >> accounts/committers. >> >> - Similar, maintainer-tools needs to move. We probably want to move >> all the dim maintained kernel repos in one go, to avoid headaches with >> double-accounts needed for committers. >> >> - CI, linux-next and everyone else should be fine, since the >> cgit/non-ssh paths will keep working (they'll be read-only mirrors). >> Need to double-check that with everyone. >> >> - Some organization structure would be good. >> >> https://cgit.freedesktop.org/drm >> >> libdrm won't be part of the gitlab drm group because that's already >> moved under mesa (and you can't symlink/mulit-home anymore on gitlab): >> >> https://gitlab.freedesktop.org/mesa/drm >> >> But there's also drm_hwcomposer, which we might want to migrate into >> drm too - gitlab requires a containing group, and >> drm_hwcomposer/drm_hwcomposer is a bit silly. >> >> Note: Access rights can be done at any level in the hierarchy, the >> organization is orthogonal to commit rights. >> >> - Anything else I've forgotten. >> >> A lot of this still needs to be figured out first. As a first step I'm >> looking for volunteers who want to join the fun, besides comments and >> thoughts on the overall topic of course. > > Just a couple of concerns from drm/i915 perspective for starters: > > - Patchwork integration. I think we'll want to keep patchwork for at > least intel-gfx etc. for the time being. IIUC the one thing we need is > some server side hook to update patchwork on git push. > > - Sticking to fdo bugzilla and disabling gitlab issues for at least > drm-intel for the time being. Doing that migration in the same go is a > bit much I think. Reassignment across bugzilla and gitlab will be an > issue. Good points, forgot about both. Patchwork reading the mailing list should keep working as-is, but the update hook needs looking into. Disabling gitlab issues is a non-brainer, same with merge requests. Mesa is already doing that. For bugs I think it's best to entirely leave them out for now, and maybe reconsider when/if mesa has moved. Before that I don't think gitlab issues make any sense at all. For merge requests I think best approach is to enable them very selectively at first for testing out, and then making a per-subproject decision whether they make sense. E.g. I think for maintainer-tools integrating make check and the doc building into gitlab CI would be sweet, and worth looking into gitlab merge requests just to automate that. Again best left out of scope for the initial migration. -Daniel -- Daniel Vetter Software Engineer, Intel Corporation +41 (0) 79 365 57 48 - http://blog.ffwll.ch ___ amd-gfx mailing list amd-gfx@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/amd-gfx
Re: [igt-dev] RFC: Migration to Gitlab
On Wed, 2018-08-22 at 16:13 +0300, Jani Nikula wrote: > - Sticking to fdo bugzilla and disabling gitlab issues for at least > drm-intel for the time being. Doing that migration in the same go is a > bit much I think. Reassignment across bugzilla and gitlab will be an > issue. Can you elaborate a bit on the issues here? The actual move-the-bugs process has been pretty painless for the parts of xorg we've done so far. - ajax ___ amd-gfx mailing list amd-gfx@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/amd-gfx
Re: [igt-dev] RFC: Migration to Gitlab
On Wed, 22 Aug 2018, Daniel Vetter wrote: > Hi all, > > I think it's time to brainstorm a bit about the gitlab migration. Basic > reasons: > > - fd.o admins want to deprecate shell accounts and hand-rolled > infrastructure, because it's a pain to keep secure > > - gitlab will allow us to add committers on our own, greatly > simplifying that process (and offloading that task from fd.o admins). > > There's also some more benefits we might want to reap, like better CI > integration for basic build testing - no more "oops didn't build > drm-misc defconfigs" or "sry, forgot make check in maintainer-tools". > But that's all fully optional. > > For the full in-depth writeup of everything, see > > https://www.fooishbar.org/blog/gitlab-fdo-introduction/ > > I think now is also a good time, with mesa, xorg, wayland/weston and > others moved, to start thinking about how we'll move drm. There's a > few things to figure out though: > > - We probably want to split out maintainer-tools. That would address > the concern that there's 50+ committers to an auto-updating shell > script ... > > - We need to figure out how to handle the ACL trickery around drm-tip in > gitlab. > > - Probably good to stage the migration, with maintainer-tools, igt > leading. That will also make fd.o admins happy, who want to rework > their cloud infrastructure a bit before migrating the big kernel repos > over. > > - Figuring out the actual migration - we've been adding a pile of > committers since fd.o LDAP was converted to gitlab once back in > spring. We need to at least figure out how to move the new > accounts/committers. > > - Similar, maintainer-tools needs to move. We probably want to move > all the dim maintained kernel repos in one go, to avoid headaches with > double-accounts needed for committers. > > - CI, linux-next and everyone else should be fine, since the > cgit/non-ssh paths will keep working (they'll be read-only mirrors). > Need to double-check that with everyone. > > - Some organization structure would be good. > > https://cgit.freedesktop.org/drm > > libdrm won't be part of the gitlab drm group because that's already > moved under mesa (and you can't symlink/mulit-home anymore on gitlab): > > https://gitlab.freedesktop.org/mesa/drm > > But there's also drm_hwcomposer, which we might want to migrate into > drm too - gitlab requires a containing group, and > drm_hwcomposer/drm_hwcomposer is a bit silly. > > Note: Access rights can be done at any level in the hierarchy, the > organization is orthogonal to commit rights. > > - Anything else I've forgotten. > > A lot of this still needs to be figured out first. As a first step I'm > looking for volunteers who want to join the fun, besides comments and > thoughts on the overall topic of course. Just a couple of concerns from drm/i915 perspective for starters: - Patchwork integration. I think we'll want to keep patchwork for at least intel-gfx etc. for the time being. IIUC the one thing we need is some server side hook to update patchwork on git push. - Sticking to fdo bugzilla and disabling gitlab issues for at least drm-intel for the time being. Doing that migration in the same go is a bit much I think. Reassignment across bugzilla and gitlab will be an issue. BR, Jani. -- Jani Nikula, Intel Open Source Graphics Center ___ amd-gfx mailing list amd-gfx@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/amd-gfx
