Re: Blocking criteria proposal for F30+: Printing
On Mon, 2019-02-11 at 11:56 -0500, Stephen Gallagher wrote: > On Fri, Nov 16, 2018 at 7:18 PM Adam Williamson > wrote: > > On Thu, 2018-09-20 at 08:33 -0400, Stephen Gallagher wrote: > > > There was a bug[1] filed recently that indicated that printing was > > > broken on certain printers. As a result of that discussion, it became > > > apparent that there was no criteria for printing to work at all, which > > > seems like an oversight. > > > > > > I discussed this briefly with Matthias Clasen this morning and he > > > agreed that this should be treated as blocking for Workstation. > > > > > > I'd like to propose that we add the following criteria to Beta for Fedora > > > 30+: > > > * Printing must work on at least one printer available to Fedora QA. > > > "Work" is defined as the output from the device matching a preview > > > shown on the GNOME print preview display. (Note that differences in > > > color reproduction are not considered "non-working".) > > > > > > and this to Final for Fedora 30+: > > > * Printing must work on at least one printer using each of the > > > following drivers: > > > (I don't know which ones to specify here, but we ought to try to > > > figure out a cross-section that covers a large swath of our expected > > > user base). > > > > So as with the optical media proposal we had quite a lively discussion > > on this one, then it got stuck a bit. Stephen, can you take a look at > > all the followups and either restate or revise the proposal? Thanks! > > Sorry that it's taken me so long to get back to this. > > I think the feedback on this has been mostly positive on the Beta > criteria, but I'd like to tweak the phrasing a bit and see if this > comes off more favorable: > > I'd like to propose that we add the following criteria to Beta for Fedora 30+: > * Printing must work on at least one printer available to Fedora QA. > "Work" is defined as the output from the device matching a preview > shown on the GNOME print preview display. (Note that differences in > color reproduction are not considered "non-working".) > > and this to Final for Fedora 30+: > * Printing must work on at least one printer using each of the > following drivers: > - The built-in print-to-PDF driver > - The generic IPP driver > > To clarify, this does not mean that all printers need to function > properly that use the IPP driver, just that at least one does (so we > know that printing as a whole is unbroken). Contrary to the first > proposal, we won't specify any particular hardware makes or models > that must work. > > How does that sound to people? There was broad support for this proposal both on lists and in meetings, so I am now implementing it with minor tweaks. Thanks Stephen! -- Adam Williamson Fedora QA Community Monkey IRC: adamw | Twitter: AdamW_Fedora | XMPP: adamw AT happyassassin . net http://www.happyassassin.net ___ test mailing list -- test@lists.fedoraproject.org To unsubscribe send an email to test-le...@lists.fedoraproject.org Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/test@lists.fedoraproject.org
Re: Blocking criteria proposal for F30+: Printing
On Tue, Mar 12, 2019 at 2:53 AM Zdenek Dohnal wrote: > > IMHO Stephen meant it as driverless 'driver' or IPP everywhere enabled > printer, since 'generic IPP driver' does not exist. OK. > > > > What supports IPP Everywhere out of the box? > > > > Any computer running CUPS 1.5 or later > I beg to differ that it is not entirely correct. I got that straight out of the IPP Everywhere FAQ, but the point I did not state and should have is, Fedora 30 definitely far exceeds the minimum requirement. That was also the point of pointing out Android 4.4 supports it. > > Proposal for Fedora 30: If anyone is able to, with reasonable effort, > > successfully run the agreed test cases to any printer supporting IPP > > 2.0 or higher, using whatever driver is required, then we don't block. > > I would go with 'if printing works on IPP everywhere printer available > for Fedora QA' (hooray, we have one :) ) 'then do not block'. But it > seems as technicality... I'm completely fine with narrowing this to an IPP Everywhere printer for Fedora 30. From yesterday's QA meeting, I was understanding they don't have an IPP Everywhere printer, but figured it should be possible to track down an IPP 2.0+ printer. So yeah if there's an IPP Everwhere test printer handy, just go with that from the outset. -- Chris Murphy ___ test mailing list -- test@lists.fedoraproject.org To unsubscribe send an email to test-le...@lists.fedoraproject.org Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/test@lists.fedoraproject.org
Re: Blocking criteria proposal for F30+: Printing
On Thu, Feb 28, 2019 at 2:56 PM Stephen Gallagher wrote: > * Printing must work on at least one printer available to Fedora QA. > "Work" is defined as the output from the device matching a preview > shown on the GNOME print preview display. (Note that non-ridiculous > differences in color reproduction are not considered "non-working". In > general, we'll apply the "last blocker at Go/No-Go" principle here > when deciding whether a print glitch is truly blocking.) > > and this to Final for Fedora 30+: > * Printing must work (as defined above) on at least one printer using > each of the following drivers: > - The built-in print-to-PDF driver > - The generic IPP driver > * For each blocking desktop, it must be possible to print: > - A test page from the desktop environment's built-in "test page" > feature, if such a feature exists. > - A simple text document of at least 100 words (lorem ipsum) from > the standard basic text editor accompanying that desktop. > > This does not mean that all printers need to function properly that > use the IPP driver, just that at least one does (so we > know that printing as a whole is unbroken). We won't specify any > particular hardware makes or models that must work. I think "generic IPP driver" needs to be more specifically stated. There's IPP protocol versions 1.1, 2.0, 2.1 and 2.2. The "driverless" specification is IPP Everywhere, which uses IPP protocol versions 2.0 or higher, along with additional requirements to support driverless device discovery and printing of text and images. There isn't strictly speaking a generic IPP driver, although PCL and PostScript are common. What supports IPP Everywhere out of the box? Any computer running CUPS 1.5 or later Mobile devices running Android 4.4 and later Proposal for Fedora 30: If anyone is able to, with reasonable effort, successfully run the agreed test cases to any printer supporting IPP 2.0 or higher, using whatever driver is required, then we don't block. Proposal for Fedora 31: If anyone is able to, with reasonable effort, successfully run the agreed test cases to any IPP Everywhere printer, then we don't block. Something I need to dig into deeper is how to create a virtual IPP Everywhere printer (a service); which would be useful for testing, in particular automated testing such that the virtual printer itself says "yes this is a valid print job". But also it might be possible to bridge the virtual IPP Everywhere printer with a conventional CUPS+gimp/foomatic driver based printer. If so, I'm thinking Fedora IoT on a Raspberry Pi Zero W, using a static containerized approach, that once working, should be a reliable indicator that any failures coinciding with print pipeline changes in the client, are in fact client bugs. But we'll see about that. This might be useful: IPP Everywhere mini-tutorial https://github.com/apple/cups/wiki/IPP-(Everywhere)-Mini-Tutorial Other references: IPP Everywhere FAQ: https://www.pwg.org/ipp/evefaq.html IPP Everywhere self-certified printers list: https://www.pwg.org/dynamo/eveprinters.php -- Chris Murphy ___ test mailing list -- test@lists.fedoraproject.org To unsubscribe send an email to test-le...@lists.fedoraproject.org Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/test@lists.fedoraproject.org
Re: Blocking criteria proposal for F30+: Printing
I just realized I only responded to Zdenek the other day. Re-sending my response now. On Tue, Feb 12, 2019 at 9:13 AM Zdenek Dohnal wrote: > > Hi, > > comments are in the text: > > On 2/11/19 9:17 PM, Stephen Gallagher wrote: > > On Mon, Feb 11, 2019 at 2:24 PM Chris Murphy > > wrote: > >> On Mon, Feb 11, 2019 at 9:58 AM Stephen Gallagher > >> wrote: > >>> Sorry that it's taken me so long to get back to this. > >>> > >>> I think the feedback on this has been mostly positive on the Beta > >>> criteria, but I'd like to tweak the phrasing a bit and see if this > >>> comes off more favorable: > >>> > >>> I'd like to propose that we add the following criteria to Beta for Fedora > >>> 30+: > >>> * Printing must work on at least one printer available to Fedora QA. > >>> "Work" is defined as the output from the device matching a preview > >>> shown on the GNOME print preview display. (Note that differences in > >>> color reproduction are not considered "non-working".) > >> Does the criterion pply strictly to the printing of text and line > >> art, or does it also apply to gross departures in photographs? If the > >> latter: > >> > >> ^minor differences in color reproduction are not considered "non-working"; > >> or > >> ^only major differences in color reproduction are considered "non-working" > >> > >> Major defined as any of: > >> obvious and grossly incorrect scaling (e.g. +/- 20%) > >> color inversion, torqued primaries (white becomes black, black becomes > >> white; red becomes blue, blue becomes green, etc) > >> tone reproduction that obliterates relevant identifying detail in two > >> or more test images > >> > >> With that language I'm trying to carve out only remarkable, WTF level, > >> bugs as blockers. > >> > > I think we can *probably* leave this as a thing to be decided at a > > blocker bug review. I really want to avoid trying to set a hard line > > on a topic that is inherently subjective. In general, I think we can > > just rely on the "last blocker at Go/No-Go" test for this. > I agree with Stephen - such topics can be really subjective and even the > fault does not have to be on Fedora side (f.e. when you catch the file > which goes to the printer, you look into it and it looks fine, but > output paper has 'slightly' different colors, scale etc... - so there > can be issues in the printer itself). > > > >> Next question is what applications to use for printing, since the > >> initiating application matters. What if there's a bug in just one > >> application? That shouldn't be a printing blocker (it might be a basic > >> functionality blocker for that application if it's included in default > >> installations). So I'd say pick two. Firefox and LibreOffice? Firefox > >> and evince? > >> > > How about "Desktop environment's 'test page' functionality" and > > whichever basic text editor comes with it. > > IMHO it is not correct blocker criteria for printing as itself, but it > is more like blocker for these applications. AFAIK blocker is the issue, > which can not be worked around - if the file is printable by CUPS CLI > commands 'lp'/'lpr', but not from a app, IMHO it is not blocker for > printing. > > IMO issues like 'not being able to print from X application' should be > blocking/release criteria for some common/widely used apps like > Firefox/evince/libreoffice, not for printing itself. (If the issue would > be actually connected to CUPS, I'll cooperate with them to fix the issue). > Well, we don't have to be that specific in the release criteria, honestly. We're talking about blocker criteria specifically for blocking desktops, so in my opinion it's okay to have "test page" and "basic text editor" as the stand-ins for this. (This is similar to how we have "package manager must be able to download and apply updates" as a stand-in for "the network must not be totally broken".) I'd be fine if we wanted to add a corollary that either of these are not blockers if it can be shown that other applications can print successfully. I just wanted to suggest those as the basic litmus test. > > > >> Next question, test document(s). European Color Initiative has several > >> test PDFs already prepared, perhaps the most applicable for our > >> purposes is the visual test (and a subset of it).And for font scaling > >> and reproduction, Ghent Working Group has test GWG 9.1 which tests > >> various encodings of TrueType, PostScript, and OpenType rendering. > >> Also, there's a suite of LibreOffice test files, and while I haven't > >> gone through it, I'm willing to bet there's one or two that'd serve as > >> a decent sanity tester (in any case I'm not proposing printing out > >> entire test suites): > >> https://github.com/freedesktop/libreoffice-test-files > > Chris, would you mind elaborating more on the topic of these test files > and tests from these sources? Martin (mosvald in CC) currently does only > comparing sample file and output file in ghostscript and I'm on my way > to do it the similar
Re: Blocking criteria proposal for F30+: Printing
On 2/11/19 3:17 PM, Stephen Gallagher wrote: On Mon, Feb 11, 2019 at 2:24 PM Chris Murphy wrote: On Mon, Feb 11, 2019 at 9:58 AM Stephen Gallagher wrote: Sorry that it's taken me so long to get back to this. I think the feedback on this has been mostly positive on the Beta criteria, but I'd like to tweak the phrasing a bit and see if this comes off more favorable: I'd like to propose that we add the following criteria to Beta for Fedora 30+: * Printing must work on at least one printer available to Fedora QA. "Work" is defined as the output from the device matching a preview shown on the GNOME print preview display. (Note that differences in color reproduction are not considered "non-working".) Does the criterion pply strictly to the printing of text and line art, or does it also apply to gross departures in photographs? If the latter: ^minor differences in color reproduction are not considered "non-working"; or ^only major differences in color reproduction are considered "non-working" Major defined as any of: obvious and grossly incorrect scaling (e.g. +/- 20%) color inversion, torqued primaries (white becomes black, black becomes white; red becomes blue, blue becomes green, etc) tone reproduction that obliterates relevant identifying detail in two or more test images With that language I'm trying to carve out only remarkable, WTF level, bugs as blockers. I think we can *probably* leave this as a thing to be decided at a blocker bug review. I really want to avoid trying to set a hard line on a topic that is inherently subjective. In general, I think we can just rely on the "last blocker at Go/No-Go" test for this. Next question is what applications to use for printing, since the initiating application matters. What if there's a bug in just one application? That shouldn't be a printing blocker (it might be a basic functionality blocker for that application if it's included in default installations). So I'd say pick two. Firefox and LibreOffice? Firefox and evince? How about "Desktop environment's 'test page' functionality" and whichever basic text editor comes with it. Next question, test document(s). European Color Initiative has several test PDFs already prepared, perhaps the most applicable for our purposes is the visual test (and a subset of it).And for font scaling and reproduction, Ghent Working Group has test GWG 9.1 which tests various encodings of TrueType, PostScript, and OpenType rendering. Also, there's a suite of LibreOffice test files, and while I haven't gone through it, I'm willing to bet there's one or two that'd serve as a decent sanity tester (in any case I'm not proposing printing out entire test suites): https://github.com/freedesktop/libreoffice-test-files The nice thing about standardized tests is the far lower risk of bugs in the test file itself, and for sure the applicable developers are familiar with them so as they get escalated, it eliminates the kick back "how did you create this test file? can you attach it to the bug?" etc. This sounds useful for automating the tests, but I think in general we don't need to write this into the criteria. They don't need to be that specific. and this to Final for Fedora 30+: * Printing must work on at least one printer using each of the following drivers: - The built-in print-to-PDF driver - The generic IPP driver To clarify, this does not mean that all printers need to function properly that use the IPP driver, just that at least one does (so we know that printing as a whole is unbroken). Contrary to the first proposal, we won't specify any particular hardware makes or models that must work. I agree with this. One possible sanity test: 1. "Print" the standardized test file to a PDF file (using the built-in print to PDF driver) 2. Print both the resulting PDF from 1, and the original standardized test file, to the designated IPP printer. i.e. two physical prints on paper. And within some ballpark on scaling, they should appear the same. Some of the subcriteria: a. PDF file is created from test document b. PDF file is viewable with the default PDF viewer c. PDF file is printed d. Test document is printed e. minor differences aside: b, c, and d should not cause a WTF reaction by a human That seems reasonable, though I'd rather have Master Wordsmith Adam Williamson phrase that better. ___ test mailing list -- test@lists.fedoraproject.org To unsubscribe send an email to test-le...@lists.fedoraproject.org Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/test@lists.fedoraproject.org I support the idea of having basic printer functionality as a blocker for Workstation. I haven't been able to use gnome to install printers here since we bought new printers (Brother HL-L6200DW) I think one
Re: Blocking criteria proposal for F30+: Printing
On Mon, Feb 11, 2019 at 2:24 PM Chris Murphy wrote: > > On Mon, Feb 11, 2019 at 9:58 AM Stephen Gallagher wrote: > > > > Sorry that it's taken me so long to get back to this. > > > > I think the feedback on this has been mostly positive on the Beta > > criteria, but I'd like to tweak the phrasing a bit and see if this > > comes off more favorable: > > > > I'd like to propose that we add the following criteria to Beta for Fedora > > 30+: > > * Printing must work on at least one printer available to Fedora QA. > > "Work" is defined as the output from the device matching a preview > > shown on the GNOME print preview display. (Note that differences in > > color reproduction are not considered "non-working".) > > Does the criterion pply strictly to the printing of text and line > art, or does it also apply to gross departures in photographs? If the > latter: > > ^minor differences in color reproduction are not considered "non-working"; or > ^only major differences in color reproduction are considered "non-working" > > Major defined as any of: > obvious and grossly incorrect scaling (e.g. +/- 20%) > color inversion, torqued primaries (white becomes black, black becomes > white; red becomes blue, blue becomes green, etc) > tone reproduction that obliterates relevant identifying detail in two > or more test images > > With that language I'm trying to carve out only remarkable, WTF level, > bugs as blockers. > I think we can *probably* leave this as a thing to be decided at a blocker bug review. I really want to avoid trying to set a hard line on a topic that is inherently subjective. In general, I think we can just rely on the "last blocker at Go/No-Go" test for this. > Next question is what applications to use for printing, since the > initiating application matters. What if there's a bug in just one > application? That shouldn't be a printing blocker (it might be a basic > functionality blocker for that application if it's included in default > installations). So I'd say pick two. Firefox and LibreOffice? Firefox > and evince? > How about "Desktop environment's 'test page' functionality" and whichever basic text editor comes with it. > Next question, test document(s). European Color Initiative has several > test PDFs already prepared, perhaps the most applicable for our > purposes is the visual test (and a subset of it).And for font scaling > and reproduction, Ghent Working Group has test GWG 9.1 which tests > various encodings of TrueType, PostScript, and OpenType rendering. > Also, there's a suite of LibreOffice test files, and while I haven't > gone through it, I'm willing to bet there's one or two that'd serve as > a decent sanity tester (in any case I'm not proposing printing out > entire test suites): > https://github.com/freedesktop/libreoffice-test-files > > The nice thing about standardized tests is the far lower risk of bugs > in the test file itself, and for sure the applicable developers are > familiar with them so as they get escalated, it eliminates the kick > back "how did you create this test file? can you attach it to the > bug?" etc. > > This sounds useful for automating the tests, but I think in general we don't need to write this into the criteria. They don't need to be that specific. > > > > > and this to Final for Fedora 30+: > > * Printing must work on at least one printer using each of the > > following drivers: > > - The built-in print-to-PDF driver > > - The generic IPP driver > > > > To clarify, this does not mean that all printers need to function > > properly that use the IPP driver, just that at least one does (so we > > know that printing as a whole is unbroken). Contrary to the first > > proposal, we won't specify any particular hardware makes or models > > that must work. > > I agree with this. One possible sanity test: > > 1. "Print" the standardized test file to a PDF file (using the > built-in print to PDF driver) > 2. Print both the resulting PDF from 1, and the original standardized > test file, to the designated IPP printer. > > i.e. two physical prints on paper. And within some ballpark on > scaling, they should appear the same. Some of the subcriteria: > > a. PDF file is created from test document > b. PDF file is viewable with the default PDF viewer > c. PDF file is printed > d. Test document is printed > e. minor differences aside: b, c, and d should not cause a WTF > reaction by a human > That seems reasonable, though I'd rather have Master Wordsmith Adam Williamson phrase that better. ___ test mailing list -- test@lists.fedoraproject.org To unsubscribe send an email to test-le...@lists.fedoraproject.org Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/test@lists.fedoraproject.org
Re: Blocking criteria proposal for F30+: Printing
On Mon, Feb 11, 2019 at 9:58 AM Stephen Gallagher wrote: > > On Fri, Nov 16, 2018 at 7:18 PM Adam Williamson > wrote: > > > > On Thu, 2018-09-20 at 08:33 -0400, Stephen Gallagher wrote: > > > There was a bug[1] filed recently that indicated that printing was > > > broken on certain printers. As a result of that discussion, it became > > > apparent that there was no criteria for printing to work at all, which > > > seems like an oversight. > > > > > > I discussed this briefly with Matthias Clasen this morning and he > > > agreed that this should be treated as blocking for Workstation. > > > > > > I'd like to propose that we add the following criteria to Beta for Fedora > > > 30+: > > > * Printing must work on at least one printer available to Fedora QA. > > > "Work" is defined as the output from the device matching a preview > > > shown on the GNOME print preview display. (Note that differences in > > > color reproduction are not considered "non-working".) > > > > > > and this to Final for Fedora 30+: > > > * Printing must work on at least one printer using each of the > > > following drivers: > > > (I don't know which ones to specify here, but we ought to try to > > > figure out a cross-section that covers a large swath of our expected > > > user base). > > > > So as with the optical media proposal we had quite a lively discussion > > on this one, then it got stuck a bit. Stephen, can you take a look at > > all the followups and either restate or revise the proposal? Thanks! > > > Sorry that it's taken me so long to get back to this. > > I think the feedback on this has been mostly positive on the Beta > criteria, but I'd like to tweak the phrasing a bit and see if this > comes off more favorable: > > I'd like to propose that we add the following criteria to Beta for Fedora 30+: > * Printing must work on at least one printer available to Fedora QA. > "Work" is defined as the output from the device matching a preview > shown on the GNOME print preview display. (Note that differences in > color reproduction are not considered "non-working".) Does the criterion pply strictly to the printing of text and line art, or does it also apply to gross departures in photographs? If the latter: ^minor differences in color reproduction are not considered "non-working"; or ^only major differences in color reproduction are considered "non-working" Major defined as any of: obvious and grossly incorrect scaling (e.g. +/- 20%) color inversion, torqued primaries (white becomes black, black becomes white; red becomes blue, blue becomes green, etc) tone reproduction that obliterates relevant identifying detail in two or more test images With that language I'm trying to carve out only remarkable, WTF level, bugs as blockers. Next question is what applications to use for printing, since the initiating application matters. What if there's a bug in just one application? That shouldn't be a printing blocker (it might be a basic functionality blocker for that application if it's included in default installations). So I'd say pick two. Firefox and LibreOffice? Firefox and evince? Next question, test document(s). European Color Initiative has several test PDFs already prepared, perhaps the most applicable for our purposes is the visual test (and a subset of it).And for font scaling and reproduction, Ghent Working Group has test GWG 9.1 which tests various encodings of TrueType, PostScript, and OpenType rendering. Also, there's a suite of LibreOffice test files, and while I haven't gone through it, I'm willing to bet there's one or two that'd serve as a decent sanity tester (in any case I'm not proposing printing out entire test suites): https://github.com/freedesktop/libreoffice-test-files The nice thing about standardized tests is the far lower risk of bugs in the test file itself, and for sure the applicable developers are familiar with them so as they get escalated, it eliminates the kick back "how did you create this test file? can you attach it to the bug?" etc. > and this to Final for Fedora 30+: > * Printing must work on at least one printer using each of the > following drivers: > - The built-in print-to-PDF driver > - The generic IPP driver > > To clarify, this does not mean that all printers need to function > properly that use the IPP driver, just that at least one does (so we > know that printing as a whole is unbroken). Contrary to the first > proposal, we won't specify any particular hardware makes or models > that must work. I agree with this. One possible sanity test: 1. "Print" the standardized test file to a PDF file (using the built-in print to PDF driver) 2. Print both the resulting PDF from 1, and the original standardized test file, to the designated IPP printer. i.e. two physical prints on paper. And within some ballpark on scaling, they should appear the same. Some of the subcriteria: a. PDF file is created from test document b. PDF file is viewable with the default PDF viewer c. PDF
Re: Blocking criteria proposal for F30+: Printing
On Fri, Nov 16, 2018 at 7:18 PM Adam Williamson wrote: > > On Thu, 2018-09-20 at 08:33 -0400, Stephen Gallagher wrote: > > There was a bug[1] filed recently that indicated that printing was > > broken on certain printers. As a result of that discussion, it became > > apparent that there was no criteria for printing to work at all, which > > seems like an oversight. > > > > I discussed this briefly with Matthias Clasen this morning and he > > agreed that this should be treated as blocking for Workstation. > > > > I'd like to propose that we add the following criteria to Beta for Fedora > > 30+: > > * Printing must work on at least one printer available to Fedora QA. > > "Work" is defined as the output from the device matching a preview > > shown on the GNOME print preview display. (Note that differences in > > color reproduction are not considered "non-working".) > > > > and this to Final for Fedora 30+: > > * Printing must work on at least one printer using each of the > > following drivers: > > (I don't know which ones to specify here, but we ought to try to > > figure out a cross-section that covers a large swath of our expected > > user base). > > So as with the optical media proposal we had quite a lively discussion > on this one, then it got stuck a bit. Stephen, can you take a look at > all the followups and either restate or revise the proposal? Thanks! Sorry that it's taken me so long to get back to this. I think the feedback on this has been mostly positive on the Beta criteria, but I'd like to tweak the phrasing a bit and see if this comes off more favorable: I'd like to propose that we add the following criteria to Beta for Fedora 30+: * Printing must work on at least one printer available to Fedora QA. "Work" is defined as the output from the device matching a preview shown on the GNOME print preview display. (Note that differences in color reproduction are not considered "non-working".) and this to Final for Fedora 30+: * Printing must work on at least one printer using each of the following drivers: - The built-in print-to-PDF driver - The generic IPP driver To clarify, this does not mean that all printers need to function properly that use the IPP driver, just that at least one does (so we know that printing as a whole is unbroken). Contrary to the first proposal, we won't specify any particular hardware makes or models that must work. How does that sound to people? ___ test mailing list -- test@lists.fedoraproject.org To unsubscribe send an email to test-le...@lists.fedoraproject.org Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/test@lists.fedoraproject.org
Re: Blocking criteria proposal for F30+: Printing
On Thu, 2018-09-20 at 08:33 -0400, Stephen Gallagher wrote: > There was a bug[1] filed recently that indicated that printing was > broken on certain printers. As a result of that discussion, it became > apparent that there was no criteria for printing to work at all, which > seems like an oversight. > > I discussed this briefly with Matthias Clasen this morning and he > agreed that this should be treated as blocking for Workstation. > > I'd like to propose that we add the following criteria to Beta for Fedora 30+: > * Printing must work on at least one printer available to Fedora QA. > "Work" is defined as the output from the device matching a preview > shown on the GNOME print preview display. (Note that differences in > color reproduction are not considered "non-working".) > > and this to Final for Fedora 30+: > * Printing must work on at least one printer using each of the > following drivers: > (I don't know which ones to specify here, but we ought to try to > figure out a cross-section that covers a large swath of our expected > user base). So as with the optical media proposal we had quite a lively discussion on this one, then it got stuck a bit. Stephen, can you take a look at all the followups and either restate or revise the proposal? Thanks! -- Adam Williamson Fedora QA Community Monkey IRC: adamw | Twitter: AdamW_Fedora | XMPP: adamw AT happyassassin . net http://www.happyassassin.net ___ test mailing list -- test@lists.fedoraproject.org To unsubscribe send an email to test-le...@lists.fedoraproject.org Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/test@lists.fedoraproject.org
Re: Blocking criteria proposal for F30+: Printing
On Thu, Sep 20, 2018 at 10:22 AM, Peter Robinson wrote: >> There was a bug[1] filed recently that indicated that printing was >> broken on certain printers. As a result of that discussion, it became >> apparent that there was no criteria for printing to work at all, which >> seems like an oversight. >> >> I discussed this briefly with Matthias Clasen this morning and he >> agreed that this should be treated as blocking for Workstation. >> >> I'd like to propose that we add the following criteria to Beta for Fedora >> 30+: >> * Printing must work on at least one printer available to Fedora QA. >> "Work" is defined as the output from the device matching a preview >> shown on the GNOME print preview display. (Note that differences in >> color reproduction are not considered "non-working".) >> >> and this to Final for Fedora 30+: >> * Printing must work on at least one printer using each of the >> following drivers: >> (I don't know which ones to specify here, but we ought to try to >> figure out a cross-section that covers a large swath of our expected >> user base). > > I'm against this as a blocker for a number of reasons: > * When we've tried to do hardware specific blocking at the time like > dual boot with MacOS this has not worked well and the dual boot is > testable with one piece of hardware > * It's easy to do a zero day update or a standard update to fix it > post release as doesn't affect the install path > * We don't do it for other non critical hardware selections such as > digital cameras, video cameras, and other such things > * Hardware availability, I don't see blocking for one type of printer > over another type is a good use of our time. I think it's reasonable to block on some really basic aspects of printing breakage, like not being able to print to a PDF file, and possibly being unable to print to the far simpler realm of IPP Everywhere printers. But model specific stuff. No way. I'd be generous with freeze exceptions, but not blocking the release. I'm even on the fence if I'd actually block on IPP Everywhere printing being broken. Once we're at a blocker, do we have the resources to get it fixed within a few days? If not, forget it. It can't be a blocker if we don't have the resources to support fixing the blocker in a time escalated manner. -- Chris Murphy ___ test mailing list -- test@lists.fedoraproject.org To unsubscribe send an email to test-le...@lists.fedoraproject.org Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/test@lists.fedoraproject.org
Re: Blocking criteria proposal for F30+: Printing
> There was a bug[1] filed recently that indicated that printing was > broken on certain printers. As a result of that discussion, it became > apparent that there was no criteria for printing to work at all, which > seems like an oversight. > > I discussed this briefly with Matthias Clasen this morning and he > agreed that this should be treated as blocking for Workstation. > > I'd like to propose that we add the following criteria to Beta for Fedora 30+: > * Printing must work on at least one printer available to Fedora QA. > "Work" is defined as the output from the device matching a preview > shown on the GNOME print preview display. (Note that differences in > color reproduction are not considered "non-working".) > > and this to Final for Fedora 30+: > * Printing must work on at least one printer using each of the > following drivers: > (I don't know which ones to specify here, but we ought to try to > figure out a cross-section that covers a large swath of our expected > user base). I'm against this as a blocker for a number of reasons: * When we've tried to do hardware specific blocking at the time like dual boot with MacOS this has not worked well and the dual boot is testable with one piece of hardware * It's easy to do a zero day update or a standard update to fix it post release as doesn't affect the install path * We don't do it for other non critical hardware selections such as digital cameras, video cameras, and other such things * Hardware availability, I don't see blocking for one type of printer over another type is a good use of our time. ___ test mailing list -- test@lists.fedoraproject.org To unsubscribe send an email to test-le...@lists.fedoraproject.org Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/test@lists.fedoraproject.org
Re: Blocking criteria proposal for F30+: Printing
On Thu, Sep 20, 2018, 7:50 AM Stephen Gallagher wrote: > On Thu, Sep 20, 2018 at 9:47 AM Ben Cotton wrote: > > > > On Thu, Sep 20, 2018 at 8:33 AM Stephen Gallagher > wrote: > > > > > > I'd like to propose that we add the following criteria to Beta for > Fedora 30+: > > > * Printing must work on at least one printer available to Fedora QA. > > > "Work" is defined as the output from the device matching a preview > > > shown on the GNOME print preview display. (Note that differences in > > > color reproduction are not considered "non-working".) > > > > > +1 to this > > > > > and this to Final for Fedora 30+: > > > * Printing must work on at least one printer using each of the > > > following drivers: > > > (I don't know which ones to specify here, but we ought to try to > > > figure out a cross-section that covers a large swath of our expected > > > user base). > Print to file (PDF) is available by default and should be in the list. " work" means - creates a file that, when opened with the default PDF reader and in Firefox using its built-in PDF support, is reasonably similar to the preview shown on the GNOME print preview display. As for a real printer, I suggest limiting it to an IPP Everywhere printer (any make and model), also known as driverless printing. Otherwise you can quickly get stuck in the mud. So I'd suggest that this criteria > essentially means "We block if it is *known* to fail". +1 Chris Murphy ___ test mailing list -- test@lists.fedoraproject.org To unsubscribe send an email to test-le...@lists.fedoraproject.org Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/test@lists.fedoraproject.org
Re: Blocking criteria proposal for F30+: Printing
On Thu, Sep 20, 2018 at 9:47 AM Ben Cotton wrote: > > On Thu, Sep 20, 2018 at 8:33 AM Stephen Gallagher wrote: > > > > I'd like to propose that we add the following criteria to Beta for Fedora > > 30+: > > * Printing must work on at least one printer available to Fedora QA. > > "Work" is defined as the output from the device matching a preview > > shown on the GNOME print preview display. (Note that differences in > > color reproduction are not considered "non-working".) > > > +1 to this > > > and this to Final for Fedora 30+: > > * Printing must work on at least one printer using each of the > > following drivers: > > (I don't know which ones to specify here, but we ought to try to > > figure out a cross-section that covers a large swath of our expected > > user base). > > > My main concern here is making sure QA has at least one of each of the > necessary printers. That could get large pretty quickly if we're not > careful. I'm also concerned that we could end up blocking the final > because a printer broke or is out of ink, or other hardware failure. I > think I'd rather keep the Beta proposal for Final. Since we have no > criterion currently, adding the Beta criterion is an improvement. We > can always make the requirement more aggressive if it turns out to be > insufficient. We do in fact have criteria that we cannot always verify (like the Serial-Attached SCSI criteria). In this case, we don't always test it, but if someone who does have that hardware reports that it doesn't work, we generally will block on it. So I'd suggest that this criteria essentially means "We block if it is *known* to fail". ___ test mailing list -- test@lists.fedoraproject.org To unsubscribe send an email to test-le...@lists.fedoraproject.org Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/test@lists.fedoraproject.org
Re: Blocking criteria proposal for F30+: Printing
On Thu, Sep 20, 2018 at 8:33 AM Stephen Gallagher wrote: > > I'd like to propose that we add the following criteria to Beta for Fedora 30+: > * Printing must work on at least one printer available to Fedora QA. > "Work" is defined as the output from the device matching a preview > shown on the GNOME print preview display. (Note that differences in > color reproduction are not considered "non-working".) > +1 to this > and this to Final for Fedora 30+: > * Printing must work on at least one printer using each of the > following drivers: > (I don't know which ones to specify here, but we ought to try to > figure out a cross-section that covers a large swath of our expected > user base). > My main concern here is making sure QA has at least one of each of the necessary printers. That could get large pretty quickly if we're not careful. I'm also concerned that we could end up blocking the final because a printer broke or is out of ink, or other hardware failure. I think I'd rather keep the Beta proposal for Final. Since we have no criterion currently, adding the Beta criterion is an improvement. We can always make the requirement more aggressive if it turns out to be insufficient. -- Ben Cotton Fedora Program Manager TZ=America/Indiana/Indianapolis ___ test mailing list -- test@lists.fedoraproject.org To unsubscribe send an email to test-le...@lists.fedoraproject.org Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/test@lists.fedoraproject.org
Re: Blocking criteria proposal for F30+: Printing
On Thu, Sep 20, 2018 at 8:40 AM Stephen Gallagher wrote: > I'd like to propose that we add the following criteria to Beta for Fedora > 30+: > * Printing must work on at least one printer available to Fedora QA. > > and this to Final for Fedora 30+: > * Printing must work on at least one printer using each of the > following drivers: > I'm in agreement here -- they seem like very reasonable criteria. As for the printer drivers, I think that the Postscript ought to be in the list, and perhaps something along the lines of HP LaserJet 4, as there are lots and lots and lots of printers that try to stay compatible with that. I'd also love to see "Print to PDF" added to that list, as I find myself using that more and more as time goes on. -Jared ___ test mailing list -- test@lists.fedoraproject.org To unsubscribe send an email to test-le...@lists.fedoraproject.org Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/test@lists.fedoraproject.org
Blocking criteria proposal for F30+: Printing
There was a bug[1] filed recently that indicated that printing was broken on certain printers. As a result of that discussion, it became apparent that there was no criteria for printing to work at all, which seems like an oversight. I discussed this briefly with Matthias Clasen this morning and he agreed that this should be treated as blocking for Workstation. I'd like to propose that we add the following criteria to Beta for Fedora 30+: * Printing must work on at least one printer available to Fedora QA. "Work" is defined as the output from the device matching a preview shown on the GNOME print preview display. (Note that differences in color reproduction are not considered "non-working".) and this to Final for Fedora 30+: * Printing must work on at least one printer using each of the following drivers: (I don't know which ones to specify here, but we ought to try to figure out a cross-section that covers a large swath of our expected user base). [1] https://bugzilla.redhat.com/show_bug.cgi?id=1628255 ___ test mailing list -- test@lists.fedoraproject.org To unsubscribe send an email to test-le...@lists.fedoraproject.org Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/test@lists.fedoraproject.org
Re: Blocking criteria proposal for F30+: Printing
On Thu, Sep 20, 2018 at 08:33:05AM -0400, Stephen Gallagher wrote: > There was a bug[1] filed recently that indicated that printing was > broken on certain printers. As a result of that discussion, it became > apparent that there was no criteria for printing to work at all, which > seems like an oversight. > > I discussed this briefly with Matthias Clasen this morning and he > agreed that this should be treated as blocking for Workstation. > > I'd like to propose that we add the following criteria to Beta for Fedora 30+: > * Printing must work on at least one printer available to Fedora QA. > "Work" is defined as the output from the device matching a preview > shown on the GNOME print preview display. (Note that differences in > color reproduction are not considered "non-working".) > > and this to Final for Fedora 30+: > * Printing must work on at least one printer using each of the > following drivers: > (I don't know which ones to specify here, but we ought to try to > figure out a cross-section that covers a large swath of our expected > user base). Makes sense overall. Perhaps we could compose a list of major CUPS drivers and make sure we test each with at least one printer. P > [1] https://bugzilla.redhat.com/show_bug.cgi?id=1628255 signature.asc Description: PGP signature ___ test mailing list -- test@lists.fedoraproject.org To unsubscribe send an email to test-le...@lists.fedoraproject.org Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/test@lists.fedoraproject.org
