Hi J-S,
very nice little app. May I suggest you make the app return -1 on
failure, or even better number of errors. That way your app is very
easy to configure as a testcase to be run by ctest. The basic idea
with ctest is to have a test program(s) that return 0 on success and
<>0 on failure, and then in some CMakeLists.txt do:
ADD_TEST("SomeTestName" target ARGS)
so we could have:
simplest case:
ADD_TEST("TestDefaultImageReadeWriters" osgpluginstest)
to get notification from entire test.
or better:
ADD_TEST("Plugin tiff reader" osgpluginstest --imageReaders tiff)
ADD_TEST("Plugin png reader" osgpluginstest --imageReaders png)
...
To get test results per plugin
The latter could be put in each plugin's CMakeLists.txt under src/osgPlugins/xxx
With that schema we could get tests run by all sites building and
reporting to our Dashboard. If you want I can help with the cmake
stuff once you get the app into svn.
Just a thought
cheers
Mattias
On Fri, Feb 20, 2009 at 10:54 PM, Jean-Sébastien Guay
<[email protected]> wrote:
> Hi all,
>
> As discussed in the "HDRloader error" thread, I've started a little program
> that tests every (potentially valid) combination of an image reader and
> writer. See
> http://thread.gmane.org/gmane.comp.graphics.openscenegraph.cvs/4897/focus=4923
>
> The hope is that it can turn into a way of automatically testing both model
> and image plugins prior to a release, to make sure what should read some
> data file actually does, and what should be able to write an image/3D model
> actually does.
>
> For now I started with images. I went through the plugins looking for
> readFile and writeFile methods. Sometimes I found some weirdness (for
> example the VTF plugin implements a writeFile method, but it just calls
> another function that always returns false, so the file is created but
> writing fails) and some combinations didn't work (possibly because of pixel
> format, but should the writing plugin convert pixel formats automatically,
> or should OSG have facilities to do that for the plugin?)
>
> Anyways, it's a start.
>
>
> Notes:
>
> The program reads files from a data/ directory, which must be called
> osgPluginsTest.<extension>. I've got about 15 images (all the same image
> just saved to the different formats) right now to test, but there are some
> formats I can't save to using the software I have, so perhaps we can build a
> collection of files to test collectively and we'll have better chances of
> getting all the formats OSG supports.
>
> The program will write output files to an output/ directory which it will
> create. I have added a command line argument to tell it to keep output files
> (the default would be to delete them after having tested the writer) but I
> didn't want to do the deletion itself right now (platform-independent
> way...).
>
> I've made the program build a default hard-coded list of readers and
> writers, and there's a way to override that list with extensions given on
> the command line (colon-separated list, untested).
>
> I've put a way of skipping combinations that I think are incompatible by
> design (though that needs to be verified, so there's a
> --testIncompatibleCombinations command line option and there's a TODO
> comment where the blacklisting occurs in the code).
>
> The program redirects cerr to cout so that any plugin error messages will
> also be picked up if you redirect stdout to a file. (I can never remember
> how to redirect stdout AND stderr to a file on Windows...)
>
>
> I've attached the code (for discussion more than actual submission) and also
> the results it prints out.
>
> Comments? Is this a good direction?
>
> J-S
> --
> ______________________________________________________
> Jean-Sebastien Guay [email protected]
> http://www.cm-labs.com/
> http://whitestar02.webhop.org/
>
> -------------------------------------------------------------
> Input image extension: bmp
> Testing read of [bmp] image - success
> Testing write of [bmp] image - success
> Testing write of [dds] image - success
> Testing write of [jpg] image - success
> Testing write of [png] image - success
> Testing write of [ppm] image - success
> Testing write of [rgb] image - success
> Testing write of [rgba] image - success
> Testing write of [sgi] image - success
> Testing write of [int] image - success
> Testing write of [inta] image - success
> Testing write of [bw] image - success
> Testing write of [tif] image - success
> -------------------------------------------------------------
> Input image extension: dds
> Testing read of [dds] image - success
> Testing write of [bmp] image - success
> Testing write of [dds] image - success
> Testing write of [jpg] image - success
> Skipping [png] writer based on blacklist
> Skipping [ppm] writer based on blacklist
> Testing write of [rgb] image - success
> Testing write of [rgba] image - success
> Testing write of [sgi] image - success
> Testing write of [int] image - success
> Testing write of [inta] image - success
> Testing write of [bw] image - success
> Skipping [tif] writer based on blacklist
> -------------------------------------------------------------
> Input image extension: gif
> Testing read of [gif] image - success
> Testing write of [bmp] image - success
> Testing write of [dds] image - success
> Testing write of [jpg] image - success
> Testing write of [png] image - success
> Skipping [ppm] writer based on blacklist
> Testing write of [rgb] image - success
> Testing write of [rgba] image - success
> Testing write of [sgi] image - success
> Testing write of [int] image - success
> Testing write of [inta] image - success
> Testing write of [bw] image - success
> Testing write of [tif] image - success
> -------------------------------------------------------------
> Input image extension: hdr
> Testing read of [hdr] image - success
> Testing write of [bmp] image - success
> Testing write of [dds] image - success
> Testing write of [jpg] image - success
> Testing write of [png] image - success
> Skipping [ppm] writer based on blacklist
> Testing write of [rgb] image - success
> Testing write of [rgba] image - success
> Testing write of [sgi] image - success
> Testing write of [int] image - success
> Testing write of [inta] image - success
> Testing write of [bw] image - success
> Testing write of [tif] image - success
> -------------------------------------------------------------
> Input image extension: jpg
> Testing read of [jpg] image - success
> Testing write of [bmp] image - success
> Testing write of [dds] image - success
> Testing write of [jpg] image - success
> Testing write of [png] image - success
> Testing write of [ppm] image - success
> Testing write of [rgb] image - success
> Testing write of [rgba] image - success
> Testing write of [sgi] image - success
> Testing write of [int] image - success
> Testing write of [inta] image - success
> Testing write of [bw] image - success
> Testing write of [tif] image - success
> -------------------------------------------------------------
> Input image extension: jpeg
> Testing read of [jpeg] image - success
> Testing write of [bmp] image - success
> Testing write of [dds] image - success
> Testing write of [jpg] image - success
> Testing write of [png] image - success
> Testing write of [ppm] image - success
> Testing write of [rgb] image - success
> Testing write of [rgba] image - success
> Testing write of [sgi] image - success
> Testing write of [int] image - success
> Testing write of [inta] image - success
> Testing write of [bw] image - success
> Testing write of [tif] image - success
> -------------------------------------------------------------
> Input image extension: pic
> Testing read of [pic] image - file not found
> -------------------------------------------------------------
> Input image extension: png
> Testing read of [png] image - success
> Testing write of [bmp] image - success
> Testing write of [dds] image - success
> Testing write of [jpg] image - success
> Testing write of [png] image - success
> Testing write of [ppm] image - success
> Testing write of [rgb] image - success
> Testing write of [rgba] image - success
> Testing write of [sgi] image - success
> Testing write of [int] image - success
> Testing write of [inta] image - success
> Testing write of [bw] image - success
> Testing write of [tif] image - success
> -------------------------------------------------------------
> Input image extension: pnm
> Testing read of [pnm] image - success
> Testing write of [bmp] image - success
> Testing write of [dds] image - success
> Testing write of [jpg] image - success
> Testing write of [png] image - success
> Testing write of [ppm] image - success
> Testing write of [rgb] image - success
> Testing write of [rgba] image - success
> Testing write of [sgi] image - success
> Testing write of [int] image - success
> Testing write of [inta] image - success
> Testing write of [bw] image - success
> Testing write of [tif] image - success
> -------------------------------------------------------------
> Input image extension: ppm
> Testing read of [ppm] image - success
> Testing write of [bmp] image - success
> Testing write of [dds] image - success
> Testing write of [jpg] image - success
> Testing write of [png] image - success
> Testing write of [ppm] image - success
> Testing write of [rgb] image - success
> Testing write of [rgba] image - success
> Testing write of [sgi] image - success
> Testing write of [int] image - success
> Testing write of [inta] image - success
> Testing write of [bw] image - success
> Testing write of [tif] image - success
> -------------------------------------------------------------
> Input image extension: pgm
> Testing read of [pgm] image - success
> Testing write of [bmp] image - success
> Testing write of [dds] image - success
> Testing write of [jpg] image - success
> Testing write of [png] image - success
> Skipping [ppm] writer based on blacklist
> Testing write of [rgb] image - success
> Testing write of [rgba] image - success
> Testing write of [sgi] image - success
> Testing write of [int] image - success
> Testing write of [inta] image - success
> Testing write of [bw] image - success
> Testing write of [tif] image - success
> -------------------------------------------------------------
> Input image extension: pbm
> Testing read of [pbm] image - success
> Testing write of [bmp] image - success
> Testing write of [dds] image - success
> Testing write of [jpg] image - success
> Testing write of [png] image - success
> Skipping [ppm] writer based on blacklist
> Testing write of [rgb] image - success
> Testing write of [rgba] image - success
> Testing write of [sgi] image - success
> Testing write of [int] image - success
> Testing write of [inta] image - success
> Testing write of [bw] image - success
> Testing write of [tif] image - success
> -------------------------------------------------------------
> Input image extension: rgb
> Testing read of [rgb] image - success
> Testing write of [bmp] image - success
> Testing write of [dds] image - success
> Testing write of [jpg] image - success
> Testing write of [png] image - success
> Testing write of [ppm] image - success
> Testing write of [rgb] image - success
> Testing write of [rgba] image - success
> Testing write of [sgi] image - success
> Testing write of [int] image - success
> Testing write of [inta] image - success
> Testing write of [bw] image - success
> Testing write of [tif] image - success
> -------------------------------------------------------------
> Input image extension: rgba
> Testing read of [rgba] image - file not found
> -------------------------------------------------------------
> Input image extension: sgi
> Testing read of [sgi] image - file not found
> -------------------------------------------------------------
> Input image extension: int
> Testing read of [int] image - file not found
> -------------------------------------------------------------
> Input image extension: inta
> Testing read of [inta] image - file not found
> -------------------------------------------------------------
> Input image extension: bw
> Testing read of [bw] image - file not found
> -------------------------------------------------------------
> Input image extension: tga
> Testing read of [tga] image - success
> Testing write of [bmp] image - success
> Testing write of [dds] image - success
> Testing write of [jpg] image - success
> Testing write of [png] image - success
> Testing write of [ppm] image - success
> Testing write of [rgb] image - success
> Testing write of [rgba] image - success
> Testing write of [sgi] image - success
> Testing write of [int] image - success
> Testing write of [inta] image - success
> Testing write of [bw] image - success
> Testing write of [tif] image - success
> -------------------------------------------------------------
> Input image extension: tif
> Testing read of [tif] image - success
> Testing write of [bmp] image - success
> Testing write of [dds] image - success
> Testing write of [jpg] image - success
> Testing write of [png] image - success
> Testing write of [ppm] image - success
> Testing write of [rgb] image - success
> Testing write of [rgba] image - success
> Testing write of [sgi] image - success
> Testing write of [int] image - success
> Testing write of [inta] image - success
> Testing write of [bw] image - success
> Testing write of [tif] image - success
> -------------------------------------------------------------
> Input image extension: tiff
> Testing read of [tiff] image - success
> Testing write of [bmp] image - success
> Testing write of [dds] image - success
> Testing write of [jpg] image - success
> Testing write of [png] image - success
> Testing write of [ppm] image - success
> Testing write of [rgb] image - success
> Testing write of [rgba] image - success
> Testing write of [sgi] image - success
> Testing write of [int] image - success
> Testing write of [inta] image - success
> Testing write of [bw] image - success
> Testing write of [tif] image - success
> -------------------------------------------------------------
> Input image extension: vtf
> Testing read of [vtf] image - file not found
>
> #include <osgDB/ReadFile>
> #include <osgDB/WriteFile>
> #include <osgDB/FileUtils>
> #include <osg/ArgumentParser>
>
> #include <iostream>
> #include <vector>
> #include <map>
>
> //---------------------------------------------------------------------------
> // Utilities
> //---------------------------------------------------------------------------
> char const kBlankChars[] = " \t\n\r";
>
> /// Returns a string with leading/trailing characters of a set stripped
> std::string trim( std::string const& str, char const* sepSet = kBlankChars )
> {
> std::string::size_type const first = str.find_first_not_of(sepSet);
> return ( first==std::string::npos )
> ? std::string()
> : str.substr(first, str.find_last_not_of(sepSet)-first+1);
> }
>
> std::string rtrim( std::string const& str, char const* sepSet = kBlankChars
> )
> {
> std::string::size_type const last = str.find_last_not_of(sepSet);
> return ( last==std::string::npos )
> ? std::string()
> : str.substr(0, last+1);
> }
>
> std::string ltrim( std::string const& str, char const* sepSet = kBlankChars
> )
> {
> std::string::size_type const first = str.find_first_not_of(sepSet);
> return ( first==std::string::npos )
> ? std::string()
> : str.substr( first );
> }
>
> //---------------------------------------------------------------------------
> //---------------------------------------------------------------------------
>
> std::vector<std::string> imageReaders;
> std::vector<std::string> imageWriters;
>
> typedef std::map<std::string, std::vector<std::string> > IgnoreWriterMap;
> IgnoreWriterMap ignoreWriter;
>
> void addImageReader(const std::string& extension)
> {
> imageReaders.push_back(extension);
> }
>
> void addImageWriter(const std::string& extension)
> {
> imageWriters.push_back(extension);
> }
>
> void addImageReaderAndWriter(const std::string& extension)
> {
> addImageReader(extension);
> addImageWriter(extension);
> }
>
> //---------------------------------------------------------------------------
>
> void addDefaultImageReadersAndWriters()
> {
> // ReaderWriterBMP
> addImageReaderAndWriter("bmp");
> // ReaderWriterDDS
> addImageReaderAndWriter("dds");
> // ReaderWriterGIF
> addImageReader("gif");
> // ReaderWriterHDR - Writes only float images unless we pass options...
> addImageReader("hdr");
> // ReaderWriterJPEG - just need to test writing once.
> addImageReaderAndWriter("jpg");
> addImageReader("jpeg");
> // ReaderWriterPIC
> addImageReader("pic");
> // ReaderWriterPNG
> addImageReaderAndWriter("png");
> // ReaderWriterPNM - reads all four formats, but writes only ppm
> addImageReader("pnm");
> addImageReaderAndWriter("ppm");
> addImageReader("pgm");
> addImageReader("pbm");
> // ReaderWriterRGB
> addImageReaderAndWriter("rgb");
> addImageReaderAndWriter("rgba");
> addImageReaderAndWriter("sgi");
> addImageReaderAndWriter("int");
> addImageReaderAndWriter("inta");
> addImageReaderAndWriter("bw");
> // ReaderWriterTGA
> addImageReader("tga");
> // ReaderWriterTIFF - just need to test writing once.
> addImageReaderAndWriter("tif");
> addImageReader("tiff");
> // ReaderWriterVTF - Has a writeFile() method, but it calls WriteVTF()
> // which does nothing and only returns false, which is
> // false advertisement dammit!
> addImageReader("vtf");
>
> std::vector<std::string> writersToIgnore;
>
> // DDS images cannot be written back as png, ppm or tif...
> // TODO: Check why and fix it if necessary.
> writersToIgnore.push_back("png");
> writersToIgnore.push_back("ppm");
> writersToIgnore.push_back("tif");
> ignoreWriter["dds"] = writersToIgnore;
> writersToIgnore.clear();
>
> // GIF images cannot be written back as ppm...
> // TODO: Check why and fix it if necessary.
> writersToIgnore.push_back("ppm");
> ignoreWriter["gif"] = writersToIgnore;
> writersToIgnore.clear();
>
> // HDR images cannot be written back as ppm...
> // TODO: Check why and fix it if necessary.
> writersToIgnore.push_back("ppm");
> ignoreWriter["hdr"] = writersToIgnore;
> writersToIgnore.clear();
>
> // PGM images cannot be written back as ppm...
> // TODO: Check why and fix it if necessary.
> writersToIgnore.push_back("ppm");
> ignoreWriter["pgm"] = writersToIgnore;
> writersToIgnore.clear();
>
> // PBM images cannot be written back as ppm...
> // TODO: Check why and fix it if necessary.
> writersToIgnore.push_back("ppm");
> ignoreWriter["pbm"] = writersToIgnore;
> writersToIgnore.clear();
> }
>
> //---------------------------------------------------------------------------
>
> std::string extractFirstExtension(std::string& extensionList)
> {
> // Extract the first extension, return it and remove it from the original
> string.
> std::string::size_type pos = extensionList.find_first_of(":");
> std::string ext = extensionList.substr(0, pos);
> extensionList = extensionList.substr(pos+1);
> return ext;
> }
>
> void addImageReaders(const std::string& imageReaderExtensions)
> {
> std::string extensions = imageReaderExtensions;
>
> while (!extensions.empty())
> {
> std::string ext = extractFirstExtension(extensions);
> addImageReader(ext);
> }
> }
>
> void addImageWriters(const std::string& imageWriterExtensions)
> {
> std::string extensions = imageWriterExtensions;
>
> while (!extensions.empty())
> {
> std::string ext = extractFirstExtension(extensions);
> addImageWriter(ext);
> }
> }
>
> //---------------------------------------------------------------------------
>
> osg::Image* testImageReader(const std::string& extension)
> {
> std::cout << "Testing read of [" << extension << "] image - ";
>
> // Find an image that matches the extension
> // TODO
> std::string filename = "data/osgPluginsTest." + extension;
>
> // Check if the file even exists
> if (!osgDB::fileExists(filename))
> {
> std::cout << "file not found" << std::endl;
> return 0;
> }
>
> // Try to read the image
> osg::Image* image = osgDB::readImageFile(filename);
>
> // Report whether it worked or not.
> std::cout << (image ? "success" : "failed") << std::endl;
>
> return image;
> }
>
> void testImageWriter(osg::Image* image, const std::string& extension,
> bool deleteOutputFile = true)
> {
> if (!image) return;
>
> std::cout << "Testing write of [" << extension << "] image - ";
>
> // Create a directory for output files
> bool created = osgDB::makeDirectory("output");
> if (!created)
> {
> std::cout << "could not create output directory" << std::endl;
> return;
> }
>
> // Try to write the image
> // Just create any filename so we can write the image
> std::string filename = "output/osgPluginsTest_output." + extension;
> bool success = osgDB::writeImageFile(*image, filename);
>
> if (success)
> {
> // Check that the file really exists?
> if (!osgDB::fileExists(filename))
> {
> std::cout << "no file written" << std::endl;
> return;
> }
>
> // Delete the file if requested
> if (deleteOutputFile)
> {
> // TODO
> }
> }
>
>
> // Report whether it worked or not.
> std::cout << (success ? "success" : "failed") << std::endl;
> }
>
> //---------------------------------------------------------------------------
>
> int main(int argc, char** argv)
> {
> std::streambuf* cerr_sbuf = std::cerr.rdbuf(); // save original sbuf
> std::cerr.rdbuf(std::cout.rdbuf()); // redirect 'cerr' to a
> 'cout'
>
> osg::ArgumentParser arguments(&argc,argv);
>
> std::string imageReaderExtensions = "";
> std::string imageWriterExtensions = "";
> while (arguments.read("--imageReaders", imageReaderExtensions));
> while (arguments.read("--imageWriters", imageWriterExtensions));
>
> bool deleteOutputFiles = true;
> while (arguments.read("--keepOutputFiles")) deleteOutputFiles = false;
>
> bool testIncompatibleCombinations = false;
> while (arguments.read("--testIncompatibleCombinations"))
> testIncompatibleCombinations = true;
>
> if (imageReaderExtensions.empty() && imageWriterExtensions.empty())
> addDefaultImageReadersAndWriters();
> else
> {
> addImageReaders(imageReaderExtensions);
> addImageWriters(imageWriterExtensions);
> }
>
> // Test every reader-writer combination.
> for (unsigned int i = 0; i < imageReaders.size(); i++)
> {
> std::cout <<
> "-------------------------------------------------------------" <<
> std::endl;
> std::cout << "Input image extension: " << imageReaders[i] <<
> std::endl;
>
> osg::ref_ptr<osg::Image> image = testImageReader(imageReaders[i]);
>
> if (image)
> {
> for (unsigned int j = 0; j < imageWriters.size(); j++)
> {
> // Check if we should skip this writer for this image.
> if (!testIncompatibleCombinations)
> {
> IgnoreWriterMap::const_iterator it =
> ignoreWriter.find(imageReaders[i]);
> if (it != ignoreWriter.end())
> {
> if (std::find(it->second.begin(), it->second.end(),
> imageWriters[j]) != it->second.end())
> {
> std::cout << "Skipping [" << imageWriters[j] <<
> "] writer based on blacklist" << std::endl;
> continue;
> }
> }
> }
>
> testImageWriter(image, imageWriters[j], deleteOutputFiles);
> }
> }
> }
>
> std::cerr.rdbuf(cerr_sbuf); // restore the original stream buffer
> return 0;
> }
> _______________________________________________
> osg-submissions mailing list
> [email protected]
> http://lists.openscenegraph.org/listinfo.cgi/osg-submissions-openscenegraph.org
>
>
_______________________________________________
osg-submissions mailing list
[email protected]
http://lists.openscenegraph.org/listinfo.cgi/osg-submissions-openscenegraph.org
