On Thu, 2 Jun 2016 07:46:33 -0400 (EDT) Kamil Paral <kpa...@redhat.com> wrote:
> > Hi, > > > > we have deployed task result namespaces support a while ago and put > > our checks (depcheck, upgradepath, rpmlint) into the 'qa' namespace. > > With newly added tasks like task-abicheck and task-dockerautotest, > > we weren't sure where to put them and so we used the scratch > > namespace which is supposed to be used for testing purposes. Now > > with those "3rd party" tasks deployed to production, we need to > > decide what namespaces should they and other future tasks belong in. > > > > The starting namespaces, and maintainers of tasks belonging to that > > namespace, follows: > > qa.* - Fedora QA > > pkgs.<pkgname>.* - <pkgname> maintainers > > fas.<username>.* - <username> > > fasgroup.<groupname>.* - fas members belonging to <groupname> > > scratch.* - anyone > > > > Now, since we maintain task-dockerautotest, it should go into > > qa.*, > > Agreed. Once we have dist-git checks implemented, we can ask docker > maintainers to keep it and maintain it in their repo. If they don't > want, we can keep it in our domain (same as anyone will be able to > run any test against any package in their personal space). It seems really weird to have one effectively package-specific in qa.* but there are worse things. > > I am not sure though where does task-abicheck belong. I see three > > options here: > > 1. we can create fas group and put it there, > > More on this below. > > > 2. create additional dist.* namespace where tasks like abicheck > > and/or rpmgrill would be, or > > What is the use case for "dist."? Which tasks would go there and > which would not? Is this just about "scratch" not sounding that good? I'm pretty anti-fas-name on the release blocking checks (will detail below) so in my mind, dist.* is a good place to put things which need to be run on all the packages/updates/composes etc. - something that is pretty much global. Other ideas on how to split that stuff up are welcome, though. Having scratch.* be the catch-all also allows us to say "scratch.* probably isn't important" while having another option for where to put results. I think of it as kinda similar to the concept of a scratch build in koji. > > 3. change semantics of qa.* to 'anything Fedora QA maintains or > > important, not package-specific, tasks". > > Personally, I always considered our namespaces to have two functions: > 1. show ownership - it's clear who owns qa.depcheck, or > fas.ksinny.abicheck, or group.workstation.gnome-startup Why is it important to show ownership? In some cases (mostly non-release-blocking stuff), I can kind of see a case for this but for abicheck in particular, I don't see the point. We don't encode the maintainer(s) into package names for fedora, why should we for checks/tests/tasks? From a more (perhaps overly) cynical PoV, I suspect that we're going to get the lion's share of complaints about failing tasks, no matter what the namespace happens to be. I don't really have an issue with the group names. If that name changes or the task is handed off to some other group, I think that there will be bigger changes afoot where changing filters etc. aren't the biggest concern. FWIW, I don't remember the concept of ownership coming up in the original discussions of namespaces - mostly "security" and being able to tell which results are more important in terms of release blocking vs. package blocking. > 2. increase > security and eliminate mistakes - by allowing certain people or > groups to write results only into their own namespace, we reduce > attack vectors (random people can't send fake results for > qa.depcheck) and we eliminate mistakes (people will not create a > thousand test cases in "qa." by accident) I think this is the primary use case for namespaces, honestly. > I didn't intend namespaces to reflect task importance in any way, > e.g. to put all gating tasks into "qa.". It sounds tempting, but it's > likely to violate the two functions mentioned above. For release > critical tasks, it's possible that we will take their ownership and > maintain them, but I don't think it's going to happen for all of > them. I don't see a problem in Bodhi or some releng tools monitoring > qa.depcheck, qa.upgradepath and fas.ksinny.abicheck. It's all about > trust, and the exact testcase name doesn't matter. You can't trust a > random person that he/she will maintain the task and quickly fix all > issues, but once you know who you're dealing with and that he/she is > willing and capable of doing that work, it doesn't matter whether the > namespace is "qa." or "fas.". Let's pretend that everyone who maintains task-abicheck decides to go on vacation for a month. One week into that month, task-abicheck breaks horribly and is no longer producing results that we can believe. If we're relying on results from task-abicheck to do things like gate non-rawhide builds moving into stable tags, I really don't think that the attitude will be "oh well, none of the owners are around, I guess we'll just turn off this check for a couple of weeks or not allow anything to go stable until the maintainers return". For abicheck in particular, I think that once it's integrated into various Fedora workflows, it's going to stay there unless libabigail stops being maintained for some reason and breaks horribly. We're going to be on the hook for that task, regardless of whether its in the qa.* namespace or not - I'm not saying that I think that something will happen, just using it as an example. I don't much care what the namespace ends up being, so long as it's not tied to a fas username and makes sense. Any suggestions? > Of course there's the issue of people coming and going, and it would > be nice to have the testcase name changing as little as possible. For > that reason, I imagine that really important tasks will move to some > group ownership, where people can maintain it and the namespace > doesn't need to change. Let's imagine "group.workstation" or > "group.dnf" (which can match fas groups, or be created manually, or > both - the implementation is up to discussion). The primary issue I have with fas usernames as namespaces is the churn that can represent. If task-abicheck reported to fas.ksinny.abicheck, what happens if ksinny hands of maintenance to someone else for whatever reason? Do we change the namespace for that check just because the maintainer changes? That would have cascading effects in notifications, filtering of fedmsgs/notifications by all users and any systems which query resultsdb for abicheck results. Once there's a namespace set for a certain task, that namespace shouldn't change unless there's a _really_ good reason and I don't think that change-of-maintainer is a good enough reason. If we do go the group/area route, we're going to have to figure out how the group names/membership will be maintained. > So, personally, what I would *not* do is to place task-abicheck into > "qa." namespace while having Dodji or Sinny still maintain it. That > breaks the two primary rules of namespaces (as I see it). We should > either move it to "qa." and start maintaining it ourselves, or put it > into Sinny's/Dodji's personal namespace and let them maintain it. > This is nothing personal against Sinny or Dodji (you two are doing > great work), and it's not that I don't trust them, I'm using a > concrete example but trying to show the principle in general. The primary issue I have with fas usernames as namespaces is the churn that can represent. If task-abicheck reported to fas.ksinny.abicheck, what happens if ksinny hands of maintenance to someone else for whatever reason? Do we change the namespace for that check just because the maintainer changes? That would have cascading effects in notifications, filtering of fedmsgs/notifications by all users and any systems which query resultsdb for abicheck results. Once there's a namespace set for a certain task, that namespace shouldn't change unless there's a really good reason and change-of-maintainer isn't a good enough reason. I don't really care if abicheck is in the qa.* namespace or not, though. It just seemed like a decent place and it's not like we wouldn't be on the hook if the proverbial poo hit the fan. > The only thing is that "fas." is not implemented yet and that's why > it is in "scratch." now. I don't see a problem with it, and we can > move it once we implement "fas.". Once abicheck proves to be useful > and reliable (or before, once we agree on what we really want), we > can suggest them to come up with some maintenance fas group so that > the namespace doesn't need to change, in case they want somebody else > to take it over. I could be wrong, but I have a hard time seeing a huge use case for fas username namespaces. If contributors write tasks, I really think they're going to be for a group/package/image/etc. and using scratch.* for things that don't have a group or other functional namespace works well enough. > Of course you might have a different opinion on what the namespaces > are useful for, and what are their important a unimportant features. > Please feel free to disagree with me and present your views. I do not > feel strongly about this, I was just trying to describe what I > currently see as the best (safest, least error-prone, least > maintenance) way forward. Thanks. It feels like we're starting to have another of those huge discussions that isn't worth a huge discussion but hopefully it's just a lot of words around a similar goal :) From my PoV, the things I'd fight for are: - fas username namespaced tasks cannot ever be release blocking - anything release-blocking should have a pretty much immutable namespace and name - making sure that enough people can change release blocking tasks that we don't hit a bus number problem on task ACL - keep things as consistent as we can so we can avoid future incidents of "oh, by the way, if you have any filters for results ... you'll want to change those" The things that I'd like to see but don't care about so much: - no fas username namespacing at all - namespace checks by functional use/area rather than "owner" even though that would be similar in many cases > PS: There's one case where I can imagine we should do a different > solution. If we wanted to show abicheck results in Bodhi *asap* and > were concerned about the risks of displaying results from "scratch." > namespace, then it would make sense to create some separate temporary > namespace, place abicheck into in (and set up permissions so that > it's not publicly accessible) and have it there until we implement > "fas.". This involves changing the namespace and I don't think that's a good idea. Let's change it once and (hopefully) only once. Tim
pgpx2ZF2nW176.pgp
Description: OpenPGP digital signature
_______________________________________________ qa-devel mailing list qa-devel@lists.fedoraproject.org https://lists.fedoraproject.org/admin/lists/qa-devel@lists.fedoraproject.org
