Again, these are against a super old version and done back in 2016. Also, apologies, there are some cases where I’m failing to recall why something changed. Anyhow here is a good gist of changes:
- In testsuite/runtest.py o Created variable “concatstr” to be “ & “ on windows and “ ;\n “ otherwise o Replaced all occurences of ‘ ;\n ‘ with new concatstr variable in this, and all run.py tests - Modified oiio-src/CMakeLists to only run python based tests when USE_PYTHON is set. - Oiio_app (for reasons I cannot remember) was failing to resolve app names, so I changed all occurances of oiio_app(“appname”) with the appropriate tool in all run.py tests - Added a platform check for windows in oiiotool-text/run.py to use Arial as the font, as opposed to doidserif – as Arial is a default font installed on windows. Also added corresponding test image - For oiio-subimage/run.py had to create _freetype_win.exr variants to allow for a difference in font rendering when comparing images. I cannot recall why the default font for that test is serifed on windows… - For reasons I do not recall, texture-icwrite/run.py generated different tiff outputs to the supplied reference. I simply just manually generated my own and added “–win.tif” variants. - Any test that generated an SHA for comparison was generating a different SHA on windows (compiled with a now 11 year old compiler…). So I added window-variant output text files where appropriate (such as gpsread, python-imageinput, etc) - Fixed relative pathing to textures in the python-imagebufalgo/run.py. For whatever reason, it wasn’t correctly resolving the paths to the reference textures. Some of these changes may or may not be applicable now, but it helped me in the past. If TPTB allow, I’ll share my modifications. Thanks! -J From: Oiio-dev <[email protected]> On Behalf Of Larry Gritz Sent: Thursday, April 18, 2019 10:21 AM To: OpenImageIO developers <[email protected]> Subject: Re: [Oiio-dev] Tests failing If you have any tips, then please do tell! The history of this is that back when I set up Appveyor to do CI tests, it was extremely painful and time consuming, since I don't know much about Windows and don't have access to a Windows machine. So it was an endless cycle of push, watch to see where the appveyor job failed, take a guess at what to change next, push again, wait another 20 minutes... etc. By the time I got the whole package to *build* I was pretty weary and needed to take a break before continuing the work to make the *tests* also run. And somehow I never quite got back to it. Getting just the Windows builds on CI was already enormously helpful; now that this is the case, we rarely check in things that break the build itself on Windows (that used to happen all the time). But of course it would be better to run the full testsuite as well. I'm going to take a quick stab that that and see how far I get. But if you already have necessary changes to the test scripts, that would be very welcome. On Apr 17, 2019, at 5:39 PM, Jonathan Tilden (2K) <[email protected]<mailto:[email protected]>> wrote: At some point (around 1.7 release – that’s the last one we took) I did take some time to fix up the tests on Windows. Not sure how much they’ve changed since then, but at the very least, I’d be able to talk about the work I did to ensure they ran. LMK if that is of interest. Thanks! -J From: Oiio-dev <[email protected]<mailto:[email protected]>> On Behalf Of Larry Gritz Sent: Wednesday, April 17, 2019 4:10 PM To: OpenImageIO developers <[email protected]<mailto:[email protected]>> Subject: Re: [Oiio-dev] Tests failing Oh, I see. Now that you mention it, we run the full testsuite as CI on every PR and checkin on both Linux and Windows, but we only build (not run tests) on Windows. (Aside: I'm not proud of this; we should fix so it runs full tests. It's just something that slipped through the cracks.) So I'd like to revise my previous statement: It may in fact be expected to fail a whole bunch of tests on Windows, not because the software doesn't work, but because the testing system itself is not exercised there and may at the very least be more sensitive to differences in your environment than we'd like. Also, it is possible that many tests depend on the oiio-images project being installed "adjacent to" the oiio project. That's certainly how it is set up for the CI tests. If you're getting failures on legit things (you said something about memory exception when reading dpx?), then we should concentrate directly on that rather than get sucked into a possibly irrelevant chasing down why the testsuite is not configured properly for Windows. -- lg On Apr 17, 2019, at 2:52 PM, Mathieu Prevot <[email protected]<mailto:[email protected]>> wrote: To start with, it seems that runtest.py made wrong assumptions about path. python.exe "I:/src/master-oiio/testsuite/runtest.py" "I:/src/master-oiio/build/testsuite/nonwhole -tiles" "--devenv-config" "Debug" "--solution-path" "I:/src/master-oiio/build" Traceback (most recent call last): File "I:/src/master-oiio/testsuite/runtest.py", line 426, in <module> with open(os.path.join(test_source_dir,"run.py")) as f: IOError: [Errno 2] No such file or directory: '../../../../testsuite\\nonwhole-tiles\\run.py' Furthermore, it seems that I need image tests (from https://github.com/OpenImageIO/oiio-images<https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_OpenImageIO_oiio-2Dimages&d=DwMFaQ&c=RKDswobrOGdp5vDCbl5XjxW8HqrsRSr80dGTvu3rE9Q&r=tQiBwrZewsSf_IBuvm1YFdww6ZCWQXpKnQkEmS1jaH4&m=XUL0sBe3INyfxHV1VDHZEOASj-duecx4cr76kW2Y4bc&s=ETiAWn3XWWlUjnELvFG5qRl5veHH7n060m-1kEKNTfk&e=>) Is that correct ? I'll make sure that path and images are correctly targetted and proceed further. M Le mer. 17 avr. 2019 à 23:31, Mathieu Prevot <[email protected]<mailto:[email protected]>> a écrit : I built successfully releases 2.0.7 and 2.0.6; for both tests fail. 25% tests passed, 40 tests failed out of 53 for 2.0.7 23% tests passed, 40 tests failed out of 52 for 2.0.6 I add that I'm running all that in windows, and I run tests with: `MSBuild.exe .\build\RUN_TESTS.vcxproj` Attached the test.log (result of `MSBuild.exe .\build\RUN_TESTS.vcxproj | Tee-Object -filepath tests.log`) I'll run individual commands in a debugger and collect traces. M Le mer. 17 avr. 2019 à 17:18, Larry Gritz <[email protected]<mailto:[email protected]>> a écrit : Also: If instead of the trunk, you try building from tag "Release-2.0.7", does that work? On Apr 17, 2019, at 8:14 AM, Larry Gritz <[email protected]<mailto:[email protected]>> wrote: Definitely not expected. Can you do a 'make test > test.log' and send it to me? Maybe if I see the output of the tests that fail, it will be more clear where to start. Is it possible to make a debug build, run any individual command that seems to fail (for example, just 'oiiotool -info fails.tiff') in the debugger, and send us the stack trace (which functions it's in, what lines) so we know where it's crashing? On Apr 17, 2019, at 3:09 AM, Mathieu Prevot <[email protected]<mailto:[email protected]>> wrote: Hello, I managed to build OIIO, using tiff 4.0.10, openexr 2.3.0, zlib 1.2.11, libpng 1.6.35, boost 1.70, libjpegturbo 2.0.3, (tbb 2019.3 was ignored); cmake 3.13.4, VS2017. However, most of the tests are failing and I could not even open an image (tried tiff, and DPX), for which I had memory exception. I used the trunk-master version of OIIO of 15th april 2019. Is is expected or not ? How to solve this (have a working OIIO) ? Please cc me since I did not subscribe to the list. Regards M ---------------------------------------------------------------- 24% tests passed, 42 tests failed out of 55 Total Test time (real) = 54.19 sec The following tests FAILED: Errors while running CTest 1 - nonwhole-tiles (Failed) 2 - oiiotool-composite (Failed) 3 - oiiotool-copy (Failed) 4 - oiiotool-pattern (Failed) 5 - oiiotool-subimage (Failed) 6 - oiiotool-text (Failed) 7 - diff (Failed) 8 - dither (Failed) 9 - dup-channels (Failed) 10 - jpeg-corrupt (Failed) 11 - null (Failed) 12 - psd-colormodes (Failed) 13 - rational (Failed) 14 - jpeg-corrupt-header (Failed) 15 - texture-half (Failed) 16 - texture-half.batch (Not Run) 17 - texture-uint16 (Failed) 18 - texture-uint16.batch (Not Run) 19 - texture-interp-bilinear (Failed) 20 - texture-interp-bilinear.batch (Not Run) 21 - texture-interp-closest (Failed) 22 - texture-interp-closest.batch (Not Run) 23 - texture-mip-onelevel (Failed) 24 - texture-mip-onelevel.batch (Not Run) 25 - texture-icwrite (Failed) 26 - texture-icwrite.batch (Not Run) 27 - heif-broken (Failed) 28 - ptex-broken (Failed) 29 - texture-field3d-broken (Failed) 30 - texture-field3d-broken.batch (Not Run) 31 - field3d-broken (Failed) 32 - openvdb-broken (Failed) 39 - unit_strutil (SEGFAULT) 41 - unit_filesystem (Failed) 45 - unit_timer (Failed) 47 - unit_simd (SEGFAULT) 48 - unit_filter (SEGFAULT) 49 - unit_paramlist (SEGFAULT) 50 - unit_imagebuf (Exit code 0xc0000374 ) 51 - unit_imagecache (Failed) 52 - unit_imagebufalgo (Failed) 54 - unit_imageinout (Exit code 0xc0000409 ) _______________________________________________ Oiio-dev mailing list [email protected]<mailto:[email protected]> http://lists.openimageio.org/listinfo.cgi/oiio-dev-openimageio.org<https://urldefense.proofpoint.com/v2/url?u=http-3A__lists.openimageio.org_listinfo.cgi_oiio-2Ddev-2Dopenimageio.org&d=DwMFaQ&c=RKDswobrOGdp5vDCbl5XjxW8HqrsRSr80dGTvu3rE9Q&r=tQiBwrZewsSf_IBuvm1YFdww6ZCWQXpKnQkEmS1jaH4&m=XUL0sBe3INyfxHV1VDHZEOASj-duecx4cr76kW2Y4bc&s=CDlrEhSGvT0BVWspCB3f01tiGrTOEzVbAaJCEn49cy0&e=> -- Larry Gritz [email protected]<mailto:[email protected]> _______________________________________________ Oiio-dev mailing list [email protected]<mailto:[email protected]> http://lists.openimageio.org/listinfo.cgi/oiio-dev-openimageio.org<https://urldefense.proofpoint.com/v2/url?u=http-3A__lists.openimageio.org_listinfo.cgi_oiio-2Ddev-2Dopenimageio.org&d=DwMFaQ&c=RKDswobrOGdp5vDCbl5XjxW8HqrsRSr80dGTvu3rE9Q&r=tQiBwrZewsSf_IBuvm1YFdww6ZCWQXpKnQkEmS1jaH4&m=XUL0sBe3INyfxHV1VDHZEOASj-duecx4cr76kW2Y4bc&s=CDlrEhSGvT0BVWspCB3f01tiGrTOEzVbAaJCEn49cy0&e=> -- Larry Gritz [email protected]<mailto:[email protected]> _______________________________________________ Oiio-dev mailing list [email protected]<mailto:[email protected]> http://lists.openimageio.org/listinfo.cgi/oiio-dev-openimageio.org<https://urldefense.proofpoint.com/v2/url?u=http-3A__lists.openimageio.org_listinfo.cgi_oiio-2Ddev-2Dopenimageio.org&d=DwMFaQ&c=RKDswobrOGdp5vDCbl5XjxW8HqrsRSr80dGTvu3rE9Q&r=tQiBwrZewsSf_IBuvm1YFdww6ZCWQXpKnQkEmS1jaH4&m=XUL0sBe3INyfxHV1VDHZEOASj-duecx4cr76kW2Y4bc&s=CDlrEhSGvT0BVWspCB3f01tiGrTOEzVbAaJCEn49cy0&e=> _______________________________________________ Oiio-dev mailing list [email protected]<mailto:[email protected]> http://lists.openimageio.org/listinfo.cgi/oiio-dev-openimageio.org<https://urldefense.proofpoint.com/v2/url?u=http-3A__lists.openimageio.org_listinfo.cgi_oiio-2Ddev-2Dopenimageio.org&d=DwQFaQ&c=RKDswobrOGdp5vDCbl5XjxW8HqrsRSr80dGTvu3rE9Q&r=tQiBwrZewsSf_IBuvm1YFdww6ZCWQXpKnQkEmS1jaH4&m=XUL0sBe3INyfxHV1VDHZEOASj-duecx4cr76kW2Y4bc&s=CDlrEhSGvT0BVWspCB3f01tiGrTOEzVbAaJCEn49cy0&e=> -- Larry Gritz [email protected]<mailto:[email protected]> _______________________________________________ Oiio-dev mailing list [email protected]<mailto:[email protected]> http://lists.openimageio.org/listinfo.cgi/oiio-dev-openimageio.org<https://urldefense.proofpoint.com/v2/url?u=http-3A__lists.openimageio.org_listinfo.cgi_oiio-2Ddev-2Dopenimageio.org&d=DwMFaQ&c=RKDswobrOGdp5vDCbl5XjxW8HqrsRSr80dGTvu3rE9Q&r=tQiBwrZewsSf_IBuvm1YFdww6ZCWQXpKnQkEmS1jaH4&m=yocIL80R3RMGVreR5QJ8YmFOd0YpgSS2lAMTKvoxEOw&s=8czPiCz9jCGNHUddb_J7EpwSNBowM_RZO6c9aZ0abrY&e=> -- Larry Gritz [email protected]<mailto:[email protected]>
_______________________________________________ Oiio-dev mailing list [email protected] http://lists.openimageio.org/listinfo.cgi/oiio-dev-openimageio.org
