Dear Kamil, On Thu, Feb 12, 2015 at 03:41:38AM -0500, Kamil Paral wrote: > > [...] > > I assume, running rpmgrill in Taskotron could mean, that each rpmgrill > > test result translates to an entry in Resultsdb as an outcome (PASSED, > > FAILED). > > If you are interested, we support more outcome keywords, like INFO or > NEEDS_INSPECTION: > https://docs.qadevel.cloud.fedoraproject.org/libtaskotron/latest/library.html#module-libtaskotron.check
Great. Thanks for the pointer. > > Yet the question for the developer as to why it failed is > > important. > > > > I have mainly three main questions around Taskatron and specifically > > Resultsdb: > > > > 1. Ed Santiago is currently running RPMGrill here: > > > > http://rpmgrill-fc20.edsantiago.com:5000/recent > > > > which in terms of presenting results to the user provides a different > > experience. The results are grouped by test and provide more insight > > as to why a test failed. Test results in Resultsdb currently seem to > > have only one outcome and technical details are left on the build > > master. > > > > How does a representation like RPMGrill translate into Resultsdb? Are > > there plans/ideas on how to provide a more diversified result > > representation in Resultsdb (e.g. error reason, hint on how to solve > > an error). > > At the moment, each result contains a link to the task log, which you can > inspect, e.g.: > https://taskotron.fedoraproject.org/resultsdb/results?job_id=45746 > points to > https://taskotron.fedoraproject.org/taskmaster//builders/x86_64/builds/36226/steps/runtask/logs/stdio > > That log contains both taskotron execution info and above statements and task > output. If you know an undocumented trick, you can also strip down the path a > bit arrive to the buildbot job info page: > https://taskotron.fedoraproject.org/taskmaster//builders/x86_64/builds/36226/ > and there you have access to taskotron.log, which contains full debug info, > which we use for debugging libtaskotron issues. This is usually not needed by > package maintainers, so we don't advertise this too much yet. Thanks. > I agree it would be great if taskotron could execute rpmgrill, and > then upload the results to your special application and nicely display > them. Or alternatively generate the html report locally. Unfortunately > we don't have a support for storing and displaying artifacts (e.g. a > html page) at the moment -- but we plan to do it in the future. > > If you want to have something working right now, I see several ways to > go: > > 1. You could print out a text representation of the rpmgrill results > into the log. You could also post the results into your web > applications, and then include a hyperlink in the log in order to > point people to a better visualization. > > 2. I guess we could change the "Log →" hyperlink in resultsdb to point > to your web application directly, rather than into our buildbot log. > However, I see two concerns with that, one is that accessing the task > log itself would be non-trivial, and second that displaying any > results at all would be relying on your web app availability. If it > went down, nobody could see any results at all. > > In the future, generating the html result locally and then showing it > from resultsdb interface is probably the way to go. That's were we would like to help to make this happen. Before discussing any technical details, my first e-mail was aimed to figure out if this is a desirable feature or not. If that's something which looks like it is part of where resultsdb is going, I could start a technical discussion in another thread? > > 3. Are there plans/ideas on implementing a waiver mechanism? > > We will definitely need this, for most of the checks. But we haven't > started yet designing that feature, so it won't be available any time > soon. Thanks for sharing. > > BTW, I talked to Miroslav Vadkerti on DevConf about rpmgrill. We > should meet up again in the following weeks and try to cook some > actionable plan for rpmgrill in Taskotron. Wow great! Would it make sense to create a roadmap document or is there one already? Kind Regards, -- Róman Joost Software Engineer, HSS - Software Engineering and Development (Brisbane) email: rjo...@redhat.com | tz: UTC+10 irc: #hss #rpmdiff
pgpKpKb_pbMVx.pgp
Description: PGP signature
_______________________________________________ qa-devel mailing list qa-devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/qa-devel
