If you have any tips, then please do tell! The history of this is that back when I set up Appveyor to do CI tests, it was extremely painful and time consuming, since I don't know much about Windows and don't have access to a Windows machine. So it was an endless cycle of push, watch to see where the appveyor job failed, take a guess at what to change next, push again, wait another 20 minutes... etc. By the time I got the whole package to *build* I was pretty weary and needed to take a break before continuing the work to make the *tests* also run. And somehow I never quite got back to it.
Getting just the Windows builds on CI was already enormously helpful; now that this is the case, we rarely check in things that break the build itself on Windows (that used to happen all the time). But of course it would be better to run the full testsuite as well. I'm going to take a quick stab that that and see how far I get. But if you already have necessary changes to the test scripts, that would be very welcome. > On Apr 17, 2019, at 5:39 PM, Jonathan Tilden (2K) <[email protected]> > wrote: > > At some point (around 1.7 release – that’s the last one we took) I did take > some time to fix up the tests on Windows. Not sure how much they’ve changed > since then, but at the very least, I’d be able to talk about the work I did > to ensure they ran. LMK if that is of interest. Thanks! > -J > > From: Oiio-dev <[email protected]> On Behalf Of Larry > Gritz > Sent: Wednesday, April 17, 2019 4:10 PM > To: OpenImageIO developers <[email protected]> > Subject: Re: [Oiio-dev] Tests failing > > Oh, I see. Now that you mention it, we run the full testsuite as CI on every > PR and checkin on both Linux and Windows, but we only build (not run tests) > on Windows. (Aside: I'm not proud of this; we should fix so it runs full > tests. It's just something that slipped through the cracks.) > > So I'd like to revise my previous statement: It may in fact be expected to > fail a whole bunch of tests on Windows, not because the software doesn't > work, but because the testing system itself is not exercised there and may at > the very least be more sensitive to differences in your environment than we'd > like. Also, it is possible that many tests depend on the oiio-images project > being installed "adjacent to" the oiio project. That's certainly how it is > set up for the CI tests. > > If you're getting failures on legit things (you said something about memory > exception when reading dpx?), then we should concentrate directly on that > rather than get sucked into a possibly irrelevant chasing down why the > testsuite is not configured properly for Windows. > > -- lg > > > > On Apr 17, 2019, at 2:52 PM, Mathieu Prevot <[email protected] > <mailto:[email protected]>> wrote: > > To start with, it seems that runtest.py made wrong assumptions about path. > > python.exe "I:/src/master-oiio/testsuite/runtest.py" > "I:/src/master-oiio/build/testsuite/nonwhole > -tiles" "--devenv-config" "Debug" "--solution-path" "I:/src/master-oiio/build" > Traceback (most recent call last): > File "I:/src/master-oiio/testsuite/runtest.py", line 426, in <module> > with open(os.path.join(test_source_dir,"run.py")) as f: > IOError: [Errno 2] No such file or directory: > '../../../../testsuite\\nonwhole-tiles\\run.py' > > Furthermore, it seems that I need image tests (from > https://github.com/OpenImageIO/oiio-images > <https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_OpenImageIO_oiio-2Dimages&d=DwMFaQ&c=RKDswobrOGdp5vDCbl5XjxW8HqrsRSr80dGTvu3rE9Q&r=tQiBwrZewsSf_IBuvm1YFdww6ZCWQXpKnQkEmS1jaH4&m=XUL0sBe3INyfxHV1VDHZEOASj-duecx4cr76kW2Y4bc&s=ETiAWn3XWWlUjnELvFG5qRl5veHH7n060m-1kEKNTfk&e=>) > Is that correct ? > > I'll make sure that path and images are correctly targetted and proceed > further. > > M > > > Le mer. 17 avr. 2019 à 23:31, Mathieu Prevot <[email protected] > <mailto:[email protected]>> a écrit : > I built successfully releases 2.0.7 and 2.0.6; for both tests fail. > > 25% tests passed, 40 tests failed out of 53 for 2.0.7 > 23% tests passed, 40 tests failed out of 52 for 2.0.6 > > I add that I'm running all that in windows, and I run tests with: > `MSBuild.exe .\build\RUN_TESTS.vcxproj` > > Attached the test.log (result of `MSBuild.exe .\build\RUN_TESTS.vcxproj | > Tee-Object -filepath tests.log`) > > I'll run individual commands in a debugger and collect traces. > > M > > > Le mer. 17 avr. 2019 à 17:18, Larry Gritz <[email protected] > <mailto:[email protected]>> a écrit : > Also: If instead of the trunk, you try building from tag "Release-2.0.7", > does that work? > > > > > On Apr 17, 2019, at 8:14 AM, Larry Gritz <[email protected] > <mailto:[email protected]>> wrote: > > Definitely not expected. > > Can you do a 'make test > test.log' and send it to me? Maybe if I see the > output of the tests that fail, it will be more clear where to start. > > Is it possible to make a debug build, run any individual command that seems > to fail (for example, just 'oiiotool -info fails.tiff') in the debugger, and > send us the stack trace (which functions it's in, what lines) so we know > where it's crashing? > > > > On Apr 17, 2019, at 3:09 AM, Mathieu Prevot <[email protected] > <mailto:[email protected]>> wrote: > > Hello, > > I managed to build OIIO, using tiff 4.0.10, openexr 2.3.0, zlib 1.2.11, > libpng 1.6.35, boost 1.70, libjpegturbo 2.0.3, (tbb 2019.3 was ignored); > cmake 3.13.4, VS2017. > > However, most of the tests are failing and I could not even open an image > (tried tiff, and DPX), for which I had memory exception. > > I used the trunk-master version of OIIO of 15th april 2019. > > Is is expected or not ? How to solve this (have a working OIIO) ? > > Please cc me since I did not subscribe to the list. > Regards > M > > > ---------------------------------------------------------------- > 24% tests passed, 42 tests failed out of 55 > > Total Test time (real) = 54.19 sec > > The following tests FAILED: > Errors while running CTest > 1 - nonwhole-tiles (Failed) > 2 - oiiotool-composite (Failed) > 3 - oiiotool-copy (Failed) > 4 - oiiotool-pattern (Failed) > 5 - oiiotool-subimage (Failed) > 6 - oiiotool-text (Failed) > 7 - diff (Failed) > 8 - dither (Failed) > 9 - dup-channels (Failed) > 10 - jpeg-corrupt (Failed) > 11 - null (Failed) > 12 - psd-colormodes (Failed) > 13 - rational (Failed) > 14 - jpeg-corrupt-header (Failed) > 15 - texture-half (Failed) > 16 - texture-half.batch (Not Run) > 17 - texture-uint16 (Failed) > 18 - texture-uint16.batch (Not Run) > 19 - texture-interp-bilinear (Failed) > 20 - texture-interp-bilinear.batch (Not Run) > 21 - texture-interp-closest (Failed) > 22 - texture-interp-closest.batch (Not Run) > 23 - texture-mip-onelevel (Failed) > 24 - texture-mip-onelevel.batch (Not Run) > 25 - texture-icwrite (Failed) > 26 - texture-icwrite.batch (Not Run) > 27 - heif-broken (Failed) > 28 - ptex-broken (Failed) > 29 - texture-field3d-broken (Failed) > 30 - texture-field3d-broken.batch (Not Run) > 31 - field3d-broken (Failed) > 32 - openvdb-broken (Failed) > 39 - unit_strutil (SEGFAULT) > 41 - unit_filesystem (Failed) > 45 - unit_timer (Failed) > 47 - unit_simd (SEGFAULT) > 48 - unit_filter (SEGFAULT) > 49 - unit_paramlist (SEGFAULT) > 50 - unit_imagebuf (Exit code 0xc0000374 > ) > 51 - unit_imagecache (Failed) > 52 - unit_imagebufalgo (Failed) > 54 - unit_imageinout (Exit code 0xc0000409 > ) > _______________________________________________ > Oiio-dev mailing list > [email protected] <mailto:[email protected]> > http://lists.openimageio.org/listinfo.cgi/oiio-dev-openimageio.org > <https://urldefense.proofpoint.com/v2/url?u=http-3A__lists.openimageio.org_listinfo.cgi_oiio-2Ddev-2Dopenimageio.org&d=DwMFaQ&c=RKDswobrOGdp5vDCbl5XjxW8HqrsRSr80dGTvu3rE9Q&r=tQiBwrZewsSf_IBuvm1YFdww6ZCWQXpKnQkEmS1jaH4&m=XUL0sBe3INyfxHV1VDHZEOASj-duecx4cr76kW2Y4bc&s=CDlrEhSGvT0BVWspCB3f01tiGrTOEzVbAaJCEn49cy0&e=> > > -- > Larry Gritz > [email protected] <mailto:[email protected]> > > > > > _______________________________________________ > Oiio-dev mailing list > [email protected] <mailto:[email protected]> > http://lists.openimageio.org/listinfo.cgi/oiio-dev-openimageio.org > <https://urldefense.proofpoint.com/v2/url?u=http-3A__lists.openimageio.org_listinfo.cgi_oiio-2Ddev-2Dopenimageio.org&d=DwMFaQ&c=RKDswobrOGdp5vDCbl5XjxW8HqrsRSr80dGTvu3rE9Q&r=tQiBwrZewsSf_IBuvm1YFdww6ZCWQXpKnQkEmS1jaH4&m=XUL0sBe3INyfxHV1VDHZEOASj-duecx4cr76kW2Y4bc&s=CDlrEhSGvT0BVWspCB3f01tiGrTOEzVbAaJCEn49cy0&e=> > > -- > Larry Gritz > [email protected] <mailto:[email protected]> > > > > > _______________________________________________ > Oiio-dev mailing list > [email protected] <mailto:[email protected]> > http://lists.openimageio.org/listinfo.cgi/oiio-dev-openimageio.org > <https://urldefense.proofpoint.com/v2/url?u=http-3A__lists.openimageio.org_listinfo.cgi_oiio-2Ddev-2Dopenimageio.org&d=DwMFaQ&c=RKDswobrOGdp5vDCbl5XjxW8HqrsRSr80dGTvu3rE9Q&r=tQiBwrZewsSf_IBuvm1YFdww6ZCWQXpKnQkEmS1jaH4&m=XUL0sBe3INyfxHV1VDHZEOASj-duecx4cr76kW2Y4bc&s=CDlrEhSGvT0BVWspCB3f01tiGrTOEzVbAaJCEn49cy0&e=> > _______________________________________________ > Oiio-dev mailing list > [email protected] <mailto:[email protected]> > http://lists.openimageio.org/listinfo.cgi/oiio-dev-openimageio.org > <https://urldefense.proofpoint.com/v2/url?u=http-3A__lists.openimageio.org_listinfo.cgi_oiio-2Ddev-2Dopenimageio.org&d=DwQFaQ&c=RKDswobrOGdp5vDCbl5XjxW8HqrsRSr80dGTvu3rE9Q&r=tQiBwrZewsSf_IBuvm1YFdww6ZCWQXpKnQkEmS1jaH4&m=XUL0sBe3INyfxHV1VDHZEOASj-duecx4cr76kW2Y4bc&s=CDlrEhSGvT0BVWspCB3f01tiGrTOEzVbAaJCEn49cy0&e=> > > -- > Larry Gritz > [email protected] <mailto:[email protected]> > > > > > _______________________________________________ > Oiio-dev mailing list > [email protected] <mailto:[email protected]> > http://lists.openimageio.org/listinfo.cgi/oiio-dev-openimageio.org > <http://lists.openimageio.org/listinfo.cgi/oiio-dev-openimageio.org> -- Larry Gritz [email protected]
_______________________________________________ Oiio-dev mailing list [email protected] http://lists.openimageio.org/listinfo.cgi/oiio-dev-openimageio.org
