Re: Planning: Fedmsg Integration
On Thu, Dec 11, 2014 at 11:10:55PM -0700, Tim Flink wrote: Almost forgot about this, even though I was talking about it earlier today. The current plan for bodhi2, as I understand it, is shooting for production deployment in January or February. This affects us because we will no longer be submitting bodhi comments as a feedback mechanism for completed tasks. Before this happens, we will need to enhance resultsdb-backend to send out fedmsgs that can be processed by fedmsg-notifier (fmn [1]) [1] https://apps.fedoraproject.org/notifications/ Outside of the technical requirements here, we need to figure out whether or not receiving notifications will be optional for maintainers or not. This will require some conversation with FESCo as it's more of a policy thing but will need driving from our end. IMHO, it should be optional, but opt-out. (So we would need to include it in the new defaults that go out with the next release of fmn). The more technical bits that I'm aware of at this point are: - Decide on a schema for the messages we emit * org.fedoraproject.taskotron.result.package.foo-1.2-3.fc99 etc. (not even sure that's a valid message) I'd rather not have a variable in the package topic -- that will give us an enormous number of total unique topic in the datagrepper database. How about something like: * org.fedoraproject.taskotron.result.new ..and embed the update, build, and/or package name in the body of the message. - Figure out what information should be in the emitted messages - Figure out where the messages should be emitted from and add/modify code to get that happening (resultsdb-backend makes the most sense to me right now but am open to other ideas) - Get certs for the emitting machines - Start emitting messages in staging As a reward when it's done: - rip out the bodhi comment code from litaskotron and burn it over a bonfire while we toast marshmallows for s'mores I'd like to have this ready for deployment in staging by early January, so that it can run for a while before we move it into production but I'm not sure if that's compatible with vacation schedules. For what its worth, bodhi2 (unreleased) currently just queries the resultsdb JSON API to figure out how it should mark or block updates... all without fedmsg. fedmsg just gives us a nice notification layer for end users. There is another layer of integration that we've not yet begun to stub out in bodhi2, which is to publish a message when its backend begins a mash. We have been hoping that taskotron could listen for that new message and begin a depcheck run against all the packages in the koji tag that bodhi is about to mash -- and then later, publish a message when that check is done. Bodhi would block, waiting for that response, and then only proceed with the mash if that depcheck run passes. pgpPH_OZT4ZgK.pgp Description: PGP signature ___ qa-devel mailing list qa-devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/qa-devel
Re: Planning: Fedmsg Integration
On Fri, 12 Dec 2014 11:59:30 -0500 Ralph Bean rb...@redhat.com wrote: On Thu, Dec 11, 2014 at 11:10:55PM -0700, Tim Flink wrote: Almost forgot about this, even though I was talking about it earlier today. The current plan for bodhi2, as I understand it, is shooting for production deployment in January or February. This affects us because we will no longer be submitting bodhi comments as a feedback mechanism for completed tasks. Before this happens, we will need to enhance resultsdb-backend to send out fedmsgs that can be processed by fedmsg-notifier (fmn [1]) [1] https://apps.fedoraproject.org/notifications/ Outside of the technical requirements here, we need to figure out whether or not receiving notifications will be optional for maintainers or not. This will require some conversation with FESCo as it's more of a policy thing but will need driving from our end. IMHO, it should be optional, but opt-out. (So we would need to include it in the new defaults that go out with the next release of fmn). Normally, I'd agree with you but if we're going to start gating updates or builds based on automated check results, that changes things a bit. Then again, it's not like that's going to happen right away so it's probably something we can deal with later. The more technical bits that I'm aware of at this point are: - Decide on a schema for the messages we emit * org.fedoraproject.taskotron.result.package.foo-1.2-3.fc99 etc. (not even sure that's a valid message) I'd rather not have a variable in the package topic -- that will give us an enormous number of total unique topic in the datagrepper database. How about something like: * org.fedoraproject.taskotron.result.new ..and embed the update, build, and/or package name in the body of the message. Yeah, that would work for me. I've not spent much time looking into fedmsg structure yet and the one I listed was just the first thing I thought of. If we did embed the task name and other data in the message itself, would fmn be able to differentiate between results for different affected packages? I don't think that anyone wants emails for every result from taskotron - I don't even have that set up. Another thing we're going to have to figure out is how to send notifications for depcheck. Update title isn't always consistent, so it makes for a poor method to create notification rules by. We could start posting results per-package but that is going to require some new code to figure out what all would be affected if a build was witheld. - Figure out what information should be in the emitted messages - Figure out where the messages should be emitted from and add/modify code to get that happening (resultsdb-backend makes the most sense to me right now but am open to other ideas) - Get certs for the emitting machines - Start emitting messages in staging As a reward when it's done: - rip out the bodhi comment code from litaskotron and burn it over a bonfire while we toast marshmallows for s'mores I'd like to have this ready for deployment in staging by early January, so that it can run for a while before we move it into production but I'm not sure if that's compatible with vacation schedules. For what its worth, bodhi2 (unreleased) currently just queries the resultsdb JSON API to figure out how it should mark or block updates... all without fedmsg. fedmsg just gives us a nice notification layer for end users. Yeah, I'm looking forward to that. The code we're currently using to make bodhi comments is ... not well liked or a great piece of code but it was the best reasonable solution we had at the time. There is another layer of integration that we've not yet begun to stub out in bodhi2, which is to publish a message when its backend begins a mash. We have been hoping that taskotron could listen for that new message and begin a depcheck run against all the packages in the koji tag that bodhi is about to mash -- and then later, publish a message when that check is done. Bodhi would block, waiting for that response, and then only proceed with the mash if that depcheck run passes. Yeah, we've talked about doing that in the past. Most of the stuff to do that is in place for Taskotron but it would probably require some tweaking to return depcheck results in a useful format for mash - which builds in this set are OK to push?. Has there been any farther discussion on when we'd want to start doing that and how we'd handle the nitty-gritty of actually doing it (overriding bad results, exact policy on what gets pushed and what doesn't etc.)? Tim pgpXwn9VeUhf7.pgp Description: OpenPGP digital signature ___ qa-devel mailing list qa-devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/qa-devel
Planning: Fedmsg Integration
Almost forgot about this, even though I was talking about it earlier today. The current plan for bodhi2, as I understand it, is shooting for production deployment in January or February. This affects us because we will no longer be submitting bodhi comments as a feedback mechanism for completed tasks. Before this happens, we will need to enhance resultsdb-backend to send out fedmsgs that can be processed by fedmsg-notifier (fmn [1]) [1] https://apps.fedoraproject.org/notifications/ Outside of the technical requirements here, we need to figure out whether or not receiving notifications will be optional for maintainers or not. This will require some conversation with FESCo as it's more of a policy thing but will need driving from our end. The more technical bits that I'm aware of at this point are: - Decide on a schema for the messages we emit * org.fedoraproject.taskotron.result.package.foo-1.2-3.fc99 etc. (not even sure that's a valid message) - Figure out what information should be in the emitted messages - Figure out where the messages should be emitted from and add/modify code to get that happening (resultsdb-backend makes the most sense to me right now but am open to other ideas) - Get certs for the emitting machines - Start emitting messages in staging As a reward when it's done: - rip out the bodhi comment code from litaskotron and burn it over a bonfire while we toast marshmallows for s'mores I'd like to have this ready for deployment in staging by early January, so that it can run for a while before we move it into production but I'm not sure if that's compatible with vacation schedules. Tim pgpCn3pOlFoHz.pgp Description: OpenPGP digital signature ___ qa-devel mailing list qa-devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/qa-devel
