Re: What do rawhide testers want and expect?
Hi Stephen, Thanks for putting this issue on the table. I am using Rawhide as my primary development environment. Indeed it offers a mostly up-to-date development target for the next gnome release. But the situation is far from ideal, rawhide expectantly breaks major components once in a while (Xorg, gnome, emacs, empathy, evolution...), making my workday often begin with fiddling with koji builds and/or version reverting. Gnome offers jhbuild for that use-case but it has its shortcomings too. Obviously the major one being the need for a daily long compilation step. Still rawhide is the closest to a daily gnome build with latest and greatest libs (and their API/ABI breaks :/). Hopefully the Gnome OS initiative will fix that with tested SDK daily images not impacted by subsystems (kernel and co) breaks. -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: What do rawhide testers want and expect?
On Mon, Sep 12, 2011 at 09:57:34AM -0600, Stephen John Smoogen wrote: In reading the long trash each other fest that accompanies pre-release jitters, could we start on a cleaner plate? What do the people who are using rawhide day-in/day-out expect out of the channel? What level of pain does someone like Jonathan Corbet expect and how can we better serve them? [And no this isnt me trying for another distro quote of the week, he is just my reference point of someone I know who runs rawhide day in and out.] In some cases, expectations may be off which means we need to market our deliverables better. In other cases, they may be looking for a better way to get attention to rawhide issues when everyone else is focused on F-XX-beta. In that case we can look at a mechanism that allows for less zero-sum game antics of elementary school yard you suck, no you suck more that the threads head towards. I use Rawhide packages ad hoc on my laptop. I also build and test packages in Koji using full-up Rawhide. What would make my life a bit easier would be if the kernel and qemu packages had a %check section that booted one on top of the other to ensure that the combination vaguely works. And fix it if it doesn't, don't push out those broken packages. Further along, it would be good if a few other critical bits did the same thing (glibc and a few core libraries and tools). Rich. -- Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones Read my programming blog: http://rwmj.wordpress.com Fedora now supports 80 OCaml packages (the OPEN alternative to F#) http://cocan.org/getting_started_with_ocaml_on_red_hat_and_fedora -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Fwd: What do rawhide testers want and expect?
Not sure if this went through either so forwarding as requested. -- Forwarded message -- From: Jonathan Corbet cor...@lwn.net Date: Tue, Sep 13, 2011 at 15:50 Subject: Re: What do rawhide testers want and expect? To: Development discussions related to Fedora devel@lists.fedoraproject.org Cc: smo...@gmail.com [ I'm not directly subscribed to devel, so this may not go through; if so, Stephen, can you forward it on if it seems worthwhile? ] On Mon, 12 Sep 2011 09:57:34 -0600 Stephen John Smoogen smo...@gmail.com wrote: What do the people who are using rawhide day-in/day-out expect out of the channel? What level of pain does someone like Jonathan Corbet expect and how can we better serve them? FWIW, I've been running Rawhide for many years. I've found it to be a good early warning system for what's coming, a good way to easily mess with current software, and, often, just fun. It's also a decent platform for kernel development and a way to, hopefully, help push things forward by finding and reporting bugs in the distribution. I fully expect it to break occasionally; I'm prepared for that and know how to recover. In the last year or two I'm finding that Rawhide is not as useful to me and not as much fun. Things break more often, and they tend to stay broken for a long time. I don't remember when gnucash last worked, for example; the bug I filed has not gotten a great deal of attention. The fact that X was broken for a long time and nobody seemed to care spoke volumes to me. I've been told several times now that Fedora developers don't bother with the Rawhide dumping ground and that I should be running the next-release branched version. Sometimes it seems like I'm about the only person left. That may be exaggerated, but it is clear that no frozen Rawhide has translated to wilder and slower-to-fix Rawhide with fewer users. As a result, I have, for a little while now, been thinking about moving on to something else. I don't know what the value of $SOMETHING_ELSE is yet, but it may well not be Fn+1. I need a sustained period where I'm not in an airplane - or preparing to get into an airplane - to figure that out. Naturally, Rawhide doesn't exist to make my life easier or more interesting. When I raised a similar issue some months ago, the responses suggested that, for Fedora developers, it's doing what it's supposed to do and that they were happy. So perhaps there's nothing really wrong with Rawhide, even if it no longer quite fills the niche that it once did. Thanks, jon -- Stephen J Smoogen. The core skill of innovators is error recovery, not failure avoidance. Randy Nelson, President of Pixar University. Let us be kind, one to another, for most of us are fighting a hard battle. -- Ian MacLaren -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
What do rawhide testers want and expect?
In reading the long trash each other fest that accompanies pre-release jitters, could we start on a cleaner plate? What do the people who are using rawhide day-in/day-out expect out of the channel? What level of pain does someone like Jonathan Corbet expect and how can we better serve them? [And no this isnt me trying for another distro quote of the week, he is just my reference point of someone I know who runs rawhide day in and out.] In some cases, expectations may be off which means we need to market our deliverables better. In other cases, they may be looking for a better way to get attention to rawhide issues when everyone else is focused on F-XX-beta. In that case we can look at a mechanism that allows for less zero-sum game antics of elementary school yard you suck, no you suck more that the threads head towards. -- Stephen J Smoogen. The core skill of innovators is error recovery, not failure avoidance. Randy Nelson, President of Pixar University. Let us be kind, one to another, for most of us are fighting a hard battle. -- Ian MacLaren -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: What do rawhide testers want and expect?
On Mon, Sep 12, 2011 at 08:57, Stephen John Smoogen smo...@gmail.comwrote: In reading the long trash each other fest that accompanies pre-release jitters, could we start on a cleaner plate? What do the people who are using rawhide day-in/day-out expect out of the channel? What level of pain does someone like Jonathan Corbet expect and how can we better serve them? [And no this isnt me trying for another distro quote of the week, he is just my reference point of someone I know who runs rawhide day in and out.] In some cases, expectations may be off which means we need to market our deliverables better. In other cases, they may be looking for a better way to get attention to rawhide issues when everyone else is focused on F-XX-beta. In that case we can look at a mechanism that allows for less zero-sum game antics of elementary school yard you suck, no you suck more that the threads head towards. Background I've been doing daily updates from rawhide for the past 5+ years. Over the last couple of years I've become a koji junkie, sometimes doing updates a few times a day. I never run an actual release version of fedora. Expectations I recognize that fedora (and even more so rawhide) represent the latest in software. I expect that when I run rawhide there will be many components that are broken. As as rawhide tester I generally try to report packaging and functionality problems as quickly as possible. There are occasional issues that can cost me a great deal of time to work around, diagnose and report. I expect that package maintainers will do their best to keep a minimum of breakage. I also expect that when a critical problem is discovered that they will update their packages in a timely manner. I recognize that some of the problems that I report might be interesting to me, but they may have a limited impact and so may not be fixed quickly. I generally have a backup plan that ensures that when breakage occurs I will have a workaround or backout, or just live with the problem. Commentary I've never been much of a joiner or wanting to be part of an official process. I have no interest in being a proven tester. I'm surprised that the maintainers of the critical systems don't use the pool of rawhide testers more to help them monitor the problem with packages. It is common to see maintainers letting other maintainers know about dependency updates, but it is rare to see a maintainer give a heads-up for feedback about a package they are updating. Summary Rawhide isn't really all that broken and the state of the current processes is actually quite good. People who use rawhide regularly learn to cope with a certain amount of broken issues. The current state of testing for many packages doesn't allow for updates to be error-free, so people who want more stable systems should stick with one of the release versions. -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: What do rawhide testers want and expect?
On Mon, Sep 12, 2011 at 09:57:34 -0600, Stephen John Smoogen smo...@gmail.com wrote: In reading the long trash each other fest that accompanies pre-release jitters, could we start on a cleaner plate? What do the people who are using rawhide day-in/day-out expect out of the channel? What level of pain does someone like Jonathan Corbet expect and how can we better serve them? [And no this isnt me trying for another distro quote of the week, he is just my reference point of someone I know who runs rawhide day in and out.] I am currently using rawhide for my main home desktop. My use of it changes over time. Sometimes it's because I want to start testing what will be branched before branching has occurred. Sometimes I want to test bigger changes there to evaluate whether a major update is baked enough for the next release or to see if it is suitable for previous releases. Things that cause problems for me are broken dependencies and people not doing rawhide builds, causing fixes to be delayed. (Note that rawhide doesn't inherit stuff in updates-testing.) I seem to run into kernel issues more than other people. Though I am now finding that working with upstream directly for regessions in development kernels seems to be a better process than going through Fedora. Changes in gnome have had significant negative impact for periods during this development cycle. But that isn't much different than for branched and probably will be less likely to be an issue again for several releases. -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: What do rawhide testers want and expect?
I run rawhide full time on a test machine here. rawhide-test.scrye.com, see: http://fedoraproject.org/wiki/Test_Machine_Resources_For_Package_Maintainers Usually I use it to test my packages and confirm/fix bugs that are reported. It's a vm, but vncserver works for testing a lot. I do notice when things like sshd break, or kernels don't boot. I don't use it day to day as a desktop however. kevin signature.asc Description: PGP signature -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: What do rawhide testers want and expect?
On 09/12/2011 11:57 AM, Stephen John Smoogen wrote: In reading the long trash each other fest that accompanies pre-release jitters, could we start on a cleaner plate? What do the people who are using rawhide day-in/day-out expect out of the channel? What level of pain does someone like Jonathan Corbet expect and how can we better serve them? [And no this isnt me trying for another distro quote of the week, he is just my reference point of someone I know who runs rawhide day in and out.] In some cases, expectations may be off which means we need to market our deliverables better. In other cases, they may be looking for a better way to get attention to rawhide issues when everyone else is focused on F-XX-beta. In that case we can look at a mechanism that allows for less zero-sum game antics of elementary school yard you suck, no you suck more that the threads head towards. I use rawhide daily, primarily for mail, web and document creation. I also enjoy trying new software in rawhide which leads to some interesting situations. I like to test the packages in updates-testing as a proven packager, but lately have been afraid to do so since what I think is a problem is criticized as not being a problem or if the package works for me but I can't test the bug it fixes then I am criticized for not testing the bug and passing the package on. I expect rawhide to just, minimally, work, i.e., boot at least to a command line. I also expect breakage, but I also expect that whatever breaks it to get attention and will file a bug if I can figure out what broke it. Before there was no-frozen-rawhide, we would sometimes get a heads up that something would break rawhide and a workaround or advice to avoid the specific update. That doesn't seem to happen any more. It seems that too much software is just dumped into rawhide without any sanity checking before hand. I do see notices on the development list about abi breakage or a change in an .so-n file. Should new packages, bug fixes, new capabilities and enhancements destined for the current development release (F16) be required to work in rawhide first? Why not have FESCO develop a definitive statement on what rawhide is, how to use it and what it expects from developers/maintainers and testers and then publish it on all Fedora lists, wiki's, LWN and the New York Times and Washington Post. -- Regards, OldFart -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: What do rawhide testers want and expect?
On Mon, Sep 12, 2011 at 7:57 AM, Stephen John Smoogen smo...@gmail.comwrote: In some cases, expectations may be off which means we need to market our deliverables better. In other cases, they may be looking for a better way to get attention to rawhide issues when everyone else is focused on F-XX-beta. In that case we can look at a mechanism that allows for less zero-sum game antics of elementary school yard you suck, no you suck more that the threads head towards. I'll ask a related question. What can we do to help maintainers more effectively catch hold of brown bag issues for their untested packages? I've read the discussion from Richard Jones which regard to how virtualization plays a role for libguestfs dev and how they are using using %check section and catching problems. I think there is a lot there to mull over. I've also used Kevin's rawhide instance to do pre-rawhide submission package testing on occasion when I had reason to believe my (admittedly non-critical path) packages might be be a bit wonky across an upstream release bump or something. So with all that in mind, does Fedora as a project have the ability/resources to make the use of throw-away virtual instance for pre-submission testing available as part of best practice rawhide workflow? Would wide spread use of virtualized rawhide help bridge a gap? -jef -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: What do rawhide testers want and expect?
On Mon, 2011-09-12 at 09:57 -0600, Stephen John Smoogen wrote: What do the people who are using rawhide day-in/day-out expect out of the channel? My expectation is something that basically works. Like others have said, I expect occasional breakage, but my rawhide criterion is latest version that works, not latest version, let's see if it works. Ideally, when stuff breaks, there should be some centralised way of co-ordinating fixes/workarounds like common bugs in the release docs. All that's probably needed is a wiki page - I know test@ exists, but I don't particularly want to a. follow another list, b. rely on mail working :D to fix a broken rawhide. I think this description from the wikipage is pretty much the ideal statement of the tension in rawhide: End users should not use Rawhide as their main day-to-day workstation. Because Rawhide is a development branch, many changes are not heavily tested (or tested at all) before being released to Rawhide, and packages in Rawhide can and do break without warning. It is even possible that bugs in Rawhide could cause data loss. However, testing Rawhide is a very valuable activity which helps direct Fedora development and ensure that the quality of the stable releases is high. I think the more people on rawhide, testing, the better. So that implies to me that the barrier to using rawhide be as low as possible, which means avoiding breaking changes - in particular, many changes ... not tested at all just isn't appropriate IMHO. Cheers Alex. -- This message was scanned by Better Hosted and is believed to be clean. http://www.betterhosted.com -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: What do rawhide testers want and expect?
Le lundi 12 septembre 2011 à 09:57 -0600, Stephen John Smoogen a écrit : In reading the long trash each other fest that accompanies pre-release jitters, could we start on a cleaner plate? What do the people who are using rawhide day-in/day-out expect out of the channel? What level of pain does someone like Jonathan Corbet expect and how can we better serve them? [And no this isnt me trying for another distro quote of the week, he is just my reference point of someone I know who runs rawhide day in and out.] I'd expect apps to works, with breakage limited to non-mainstream or brand-new options (in other words app crash on startup = no good, some feature not baked yet = why not) When there is breakage, I expect to have someone benefit from the breakage inflicted to me. That means abrt and sealert *must* work 100% of the time in rawhide and maintainers *must* look at problem reports (if only to indicate workarounds). There is nothing more depressing than spending time fighting a problem to see no one cares and there is no way to report it, or having the same app crash for months in a row (I'm thinking about you empathy) When rawhide crashes too much I spend all my time repairing it which means I have no time left for my other Fedora activities. After a while I don't have the energy to check if it's still broken I just procastinate and push tasks to the next week. It's easy to get into the habit of not caring when others do not care about you. -- Nicolas Mailhot -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: What do rawhide testers want and expect?
On Mon, 2011-09-12 at 13:01 -0400, Clyde E. Kunkel wrote: I use rawhide daily, primarily for mail, web and document creation. I also enjoy trying new software in rawhide which leads to some interesting situations. I like to test the packages in updates-testing as a proven packager, but lately have been afraid to do so since what I think is a problem is criticized as not being a problem It's not that it's not a problem, but it's not always something the update should be held up for. This is partly a problem of the over-simplistic current Bodhi karma system, but generally, don't -1 an update unless you're sure the bug you're hitting is a valid reason to reject the update. At a minimum, it should certainly be a problem which makes the update *worse* than the package it supersedes. 'The update doesn't fix bug 123456' is never a reason for a -1 if 123456 was also present in the previous stable build. or if the package works for me but I can't test the bug it fixes then I am criticized for not testing the bug and passing the package on. You shouldn't be criticized for that, usually, as this isn't really the function of Bodhi testing. The only time it makes sense is if the update is non-critpath, and the *sole* change in the update is to fix a single bug. If the update contains a single bugfix and no other change, then there's no real point accepting the update unless we know it actually fixes the bug. If anyone's criticized you for 'not testing that the update fixes the bug' in any other circumstances, feel free to blow raspberries at them. =) -- Adam Williamson Fedora QA Community Monkey IRC: adamw | Twitter: AdamW_Fedora | identi.ca: adamwfedora http://www.happyassassin.net -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
