Yeah, makes sense. I've never used it so have no idea if it's any good but there seems to be something called Allure (https://github.com/allure-framework/allure2) which has Spock 1.3 support ( https://github.com/allure-framework/allure-java/tree/master/allure-spock, for Spock 2.0 you probably could use JUnit 5 support) and it has ability to add attachments (https://docs.qameta.io/allure-report/#_attachments). You could probably plug into that via an implementation of geb.report.ReportingListener ( https://gebish.org/manual/current/#listening-to-reporting).
Hope this helps, Marcin On Fri, Jun 17, 2022 at 4:55 PM Nick Bensema <nick.bensema....@gmail.com> wrote: > actually, it looks like geb-spock-reports is the only thing in my way, > both for multi-window reporting, and for reporting downloaded files. > Indeed, the custom reporter that saves a ZIP file works great, but the > reports show links to nonexistent screenshots. Since it's not being > maintained, maybe I just need to switch to something else. There's > probably no need to add features to Geb for this case. > > Can you recommend a reporting extension for the geb-spock stack that > includes links to Geb artifacts in the output? > > On Friday, June 17, 2022 at 2:44:09 AM UTC-7 marcin....@proxerd.pl wrote: > >> Nick, >> >> So as far as I'm aware, geb-spock-reports is no longer maintained ( >> https://github.com/AOEpeople/geb-spock-reports/issues/37#issuecomment-950887176) >> thus trying to use it will become harder and harder over time, especially >> with newer versions of Geb. I've never been involved with geb-spock-report >> or has been a user of it so unfortunately won't be able to help with that. >> I can help with what reports and how are being generated from Geb but not >> with what geb-spock-reports then does with that. >> >> With regards to extending GebTestManager - it's not designed to be >> extended but it's designed to be configured via the builder you mention in >> your email. This is because the old "test framework support is scattered in >> base test classes" mechanism that was used prior to introduction of >> GebTestManager lead to duplication. It also meant that users would be >> encouraged to extend and override code from these base classes which then >> makes it hard to track all possible use cases and makes it harder to evolve >> the code of the framework without inadvertently introducing breaking >> changes. >> >> Unfortunately GebTestManager currently does not cater for your usecase >> of having some custom logic for generating reports upon failure but which >> seems like a completely valid usecase. My idea would be to introduce >> >> GebTestManagerBuilder withReportFailure(Consumer<GebTestManager> >> reportOnFailure) { >> ... >> } >> >> to GebTestManagerBuilder. Then you would use that in custom base classes >> in your project, for example >> >> @DynamicallyDispatchesToBrowser >> class DesignerPageReportFailureHandlingSpec extends Specification >> implements ManagedGebTest { >> >> private final static GebTestManager TEST_MANAGER = new >> SpockGebTestManagerBuilder() >> .withReportingEnabled(true) >> .withReportFailure { testManager -> >> def page = testManager.browser.page >> >> if( page instanceOf DesignerPage ) { >> if(page.preview.displayed) { >> testManager.report "preview" >> page.preview.close() >> } >> } >> >> testManager.report "failure" >> } >> .build() >> >> @Override >> @Delegate(includes = ["getBrowser", "report"]) >> GebTestManager getTestManager() { >> TEST_MANAGER >> } >> } >> >> I believe that would allow you to achieve what you are after. Would that >> work for you? If that's the case then I can create an issue for this and >> incorporate this into the upcoming Geb 6.0 release which will be based on >> Spock 2.1 and WebDriver 4. I know that this would force you to do a big >> version jump and will be somewhat inconvenient but I unfortunately do not >> have capacity to backport this. >> >> Thanks for bearing with me on this. >> >> On Tue, Jun 14, 2022 at 9:55 PM Nick Bensema <nick.ben...@gmail.com> >> wrote: >> >>> so, I've looked at the default template in the source code, and I've >>> even added my own listener object to confirm that both "main" and >>> "designer" were being caught by listeners, but still only "main" appeared >>> in the report HTML. If I suppress the "main" window, then "designer" does >>> show up. I poked around the source code and think there might be a problem >>> with GebReportingListener in geb-spock-reports; there's a lot of strange >>> logic in there and I can see how it might not do the right thing if >>> findArtifactByNumber already returns something... >>> >>> having hit a dead end there, I took another look at how to write a >>> GebTestManager. I poked around the source code some more and thought I >>> could extend GebTestManager and move my custom reportFailure() method into >>> there, but there's this whole GebTestManagerBuilder thing that populates a >>> private, not protected, singleton, and it doesn't seem to have a mechanism >>> to just override one method. Since getTestReporter()'s return type is >>> GebTestManager, I have to return a subclass of that, and I can't just write >>> a one-method class and @Delegate to call the real test manager for >>> everything else. And the real test manager is a private static final >>> singleton, and so I'd have to delegate everything to that specific >>> instance. does @Delegate take precedence over superclass methods or do I >>> have to write a dozen @Override methods to shunt every other method call to >>> the delegate? Either way it seems fragile, and smells like the wrong way >>> to do it. >>> >>> so, either I have to find the right way to change reportFailure(), or >>> fix the bug in geb-spock-reports. am I missing something here? >>> >>> On Friday, June 10, 2022 at 1:22:31 PM UTC-7 Nick Bensema wrote: >>> >>>> I did indeed fix that problem by moving it to just setup() -- moving it >>>> to the config isn't yet appropriate because I want to report different test >>>> classes differently; I have different tests for the UI and for the >>>> downloads. Maybe that will work if I can figure out how to modularize this >>>> project, but that's another thread... >>>> >>>> Now I'm having a different problem with the MultiWindowReporter. It's >>>> writing the HTML and PNG for both windows, but only one of them is showing >>>> up in my geb-spock-reports output -- always the same one, the main window >>>> rather than the editor window. If I add an extra report "notFailure" >>>> call in my test, both reports show up, but still only the first window. I >>>> went so far as to upgrade to Spock 2.0 and Groovy 3.0, but ended up right >>>> back where I started. For what it's worth, I went from version 1.8.0 to >>>> version 2.0-groovy-3.0 of this: >>>> https://github.com/AOEpeople/geb-spock-reports >>>> >>>> I thought it might have to do with the long window name or the space in >>>> the name or something, so I tried writing my own writeWindowReport call. >>>> I'd need to do this anyway because "main" and "design" will be more helpful >>>> names than the window ID. My reporter correctly determines the correct >>>> window label, writes the logging message, and writes both HTML and PNG >>>> files for each one, but only the "main" shows up in the HTML report for >>>> that test class: >>>> >>>> class DesignerReporter extends MultiWindowReporter { >>>> public DesignerReporter(Reporter r) { >>>> super(r) >>>> } >>>> void writeWindowReport(ReportState reportState, String >>>> windowId) { >>>> //geb.Browser browser = reportState.browser >>>> reportState.browser.withWindow(windowId) { >>>> def windowLabel = "${reportState.label}.main" >>>> if(isAt(DesignerPage)) { >>>> windowLabel = "${reportState.label}.designer" >>>> } >>>> log.info "reporting $windowLabel with window $windowId" >>>> def windowState = new ReportState(reportState.browser, >>>> windowLabel, reportState.outputDir) >>>> backing.writeReport(windowState) >>>> } >>>> } >>>> >>>> On Friday, June 10, 2022 at 12:35:41 AM UTC-7 marcin....@proxerd.pl >>>> wrote: >>>> >>>>> There is a new instance of browser and config created per test, so >>>>> modifying config in setupSpec() will only have for the first test method >>>>> in >>>>> a spec. You might get more luck by putting that code in setup but there >>>>> still will be cases where that code might not be applied to a browser >>>>> instance, like for example when reporting is triggered from cleanupSpec. I >>>>> would therefore consider registering your custom reporter in your config >>>>> script like I initially suggested. >>>>> >>>>> Marcin >>>>> >>>>> On Mon, Jun 6, 2022 at 11:05 PM Nick Bensema <nick.ben...@gmail.com> >>>>> wrote: >>>>> >>>>>> Thanks, you've made it a lot clearer for me, and I think I'm getting >>>>>> close. I've created a custom reporter that extends ReporterSupport, with >>>>>> its own writeReport method that does its own thing and calls >>>>>> notifyListeners. I created a constructor where I can pass the existing >>>>>> reporter, so in my setupSpec I have something like this: >>>>>> >>>>>> browser.config.reporter = new MyReporter(browser.config.reporter) >>>>>> >>>>>> I can even call methods, like when I've downloaded the file I want to >>>>>> test, I can call browser.config.reporter.setDownloadedFile(file) and the >>>>>> log statements say it's executing. But when I test it by introducing a >>>>>> purposeful failure, it still calls the ordinary ScreenshotReporter and >>>>>> skips my code entirely, and once again I get a png of the download page. >>>>>> >>>>>> So, what am I missing, why might my reporter not be activating? >>>>>> On Thursday, June 2, 2022 at 12:22:30 PM UTC-7 marcin....@proxerd.pl >>>>>> wrote: >>>>>> >>>>>>> Hi Nick, >>>>>>> >>>>>>> Sorry for a late reply. >>>>>>> >>>>>>> So for the reporting on multiple windows you can use a built-in >>>>>>> capability - MultiWindowReporter. I don't know when it was introduced >>>>>>> but >>>>>>> it's definitely available in 4.1 because its usage is described in the >>>>>>> manual for that version, see >>>>>>> https://gebish.org/manual/4.1/#reporting-on-multiple-windows. >>>>>>> >>>>>>> With regards to your second requirement of triggering another report >>>>>>> before performing an action and doing the report proper I would suggest >>>>>>> implementing another `geb.report.Reporter` and then plugging it in >>>>>>> together >>>>>>> with MultiWindowReporter via configuration in a similar way to the one >>>>>>> reported in >>>>>>> https://gebish.org/manual/4.1/#reporting-on-multiple-windows. I >>>>>>> think you should be able to get ideas how to implement that reporter >>>>>>> from >>>>>>> looking at the sources of MultiWindowReporter ( >>>>>>> https://github.com/geb/geb/blob/master/module/geb-core/src/main/groovy/geb/report/MultiWindowReporter.groovy) >>>>>>> but let me know if you struggle with it and I will help out with some >>>>>>> code. >>>>>>> >>>>>>> Thanks, >>>>>>> Marcin >>>>>>> >>>>>>> >>>>>>> >>>>>>> On Fri, May 27, 2022 at 9:39 PM Nick Bensema <nick.ben...@gmail.com> >>>>>>> wrote: >>>>>>> >>>>>>>> It looks like Geb development has come a long way, so I'm trying to >>>>>>>> keep my codebase current, and I've tried a few times to run my tests >>>>>>>> with >>>>>>>> geb 4.1 instead of 3.4.1 -- baby steps on the way to 5.1. So far, it >>>>>>>> won't >>>>>>>> even compile, and I need some help. >>>>>>>> >>>>>>>> The first problem I have is that I have several specs, both >>>>>>>> abstract and concrete, which try to override the `reportFailure` >>>>>>>> method, >>>>>>>> which no longer compiles. I know that I'm supposed to use testManager, >>>>>>>> but >>>>>>>> it's not quite obvious how to do it; there's very little documentation >>>>>>>> about it. >>>>>>>> >>>>>>>> First of all, I have a main abstract class that also overrides >>>>>>>> reportFailure(), because those tests all have two browser windows >>>>>>>> open, and >>>>>>>> I want to make sure to screenshot both of them. >>>>>>>> >>>>>>>> But also, sometimes I override reportFailure in a concrete test >>>>>>>> class, so that I can get screenshots of dialog boxes, and close them. >>>>>>>> That >>>>>>>> method also calls super.reportFailure(), meaning both of my custom >>>>>>>> methods >>>>>>>> will run. >>>>>>>> >>>>>>>> So, given a test class that overrides reportFailure(), how do I >>>>>>>> move that method into a test manager object that the class can use? >>>>>>>> and >>>>>>>> will it support the kind of inheritance I had before? >>>>>>>> >>>>>>>> Example of the kind of thing my method will contain: calling the >>>>>>>> main report function, doing stuff with page content, and calling the >>>>>>>> superclass method which might also be an override: >>>>>>>> >>>>>>>> @Override >>>>>>>> void reportFailure() { >>>>>>>> if( page instanceOf DesignerPage ) { >>>>>>>> if(preview.displayed) { >>>>>>>> report "preview" >>>>>>>> preview.close() >>>>>>>> } >>>>>>>> } >>>>>>>> super.reportFailure() >>>>>>>> } >>>>>>>> >>>>>>>> -- >>>>>>>> You received this message because you are subscribed to the Google >>>>>>>> Groups "Geb User Mailing List" group. >>>>>>>> To unsubscribe from this group and stop receiving emails from it, >>>>>>>> send an email to geb-user+u...@googlegroups.com. >>>>>>>> To view this discussion on the web visit >>>>>>>> https://groups.google.com/d/msgid/geb-user/d3a5566a-672e-433b-a9d2-322da900e12en%40googlegroups.com >>>>>>>> <https://groups.google.com/d/msgid/geb-user/d3a5566a-672e-433b-a9d2-322da900e12en%40googlegroups.com?utm_medium=email&utm_source=footer> >>>>>>>> . >>>>>>>> >>>>>>> -- >>>>>> You received this message because you are subscribed to the Google >>>>>> Groups "Geb User Mailing List" group. >>>>>> To unsubscribe from this group and stop receiving emails from it, >>>>>> send an email to geb-user+u...@googlegroups.com. >>>>>> >>>>> To view this discussion on the web visit >>>>>> https://groups.google.com/d/msgid/geb-user/7488922e-f26e-445c-a7bc-dc947ad1029an%40googlegroups.com >>>>>> <https://groups.google.com/d/msgid/geb-user/7488922e-f26e-445c-a7bc-dc947ad1029an%40googlegroups.com?utm_medium=email&utm_source=footer> >>>>>> . >>>>>> >>>>> -- >>> You received this message because you are subscribed to the Google >>> Groups "Geb User Mailing List" group. >>> To unsubscribe from this group and stop receiving emails from it, send >>> an email to geb-user+u...@googlegroups.com. >>> >> To view this discussion on the web visit >>> https://groups.google.com/d/msgid/geb-user/9892e5ba-6bff-448c-ab50-9c972eb8828fn%40googlegroups.com >>> <https://groups.google.com/d/msgid/geb-user/9892e5ba-6bff-448c-ab50-9c972eb8828fn%40googlegroups.com?utm_medium=email&utm_source=footer> >>> . >>> >> -- > You received this message because you are subscribed to the Google Groups > "Geb User Mailing List" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to geb-user+unsubscr...@googlegroups.com. > To view this discussion on the web visit > https://groups.google.com/d/msgid/geb-user/1737e7e1-95e0-483f-a74f-417456d30e82n%40googlegroups.com > <https://groups.google.com/d/msgid/geb-user/1737e7e1-95e0-483f-a74f-417456d30e82n%40googlegroups.com?utm_medium=email&utm_source=footer> > . > -- You received this message because you are subscribed to the Google Groups "Geb User Mailing List" group. To unsubscribe from this group and stop receiving emails from it, send an email to geb-user+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/geb-user/CA%2B52dQScrQo6E2LudwASYPRYyZuDSPdAfjvKfbX0dM%2B01tob0w%40mail.gmail.com.
