Hi Robert
2010/1/15 Robert Osfield <[email protected]>:
>> osgconv -e osg2 cow.osg cow.osgb -O WriteImageHint=IncludeFile
> OutputStream::writeImage(): Failed to open image file Images/reflect.rgb
> Data written to 'cow.osgb'.
>
> The error report suggests to me that the path to the original file is
> being lost and then not subsequently found. I'm surprised this is an
> issue as the .ive plugin just uses the image data in memory but the
> error from the osg2 plugin suggests that it's going back to the
> original image file. Using the image data that's loaded into memory
> would be probably a better approach.
Oops, I just miss to call osgDB::findDataFile in the writeImage()
function. I've updated it in the attachment.
The IncludeFile option works similar to the includeImageFileInIVEFile
option of ive plugin. It directly embeds the image file (which might
be small and portable). You may simply use WriteImageHint=IncludeData
to record image data in memory instead. :)
osgconv -e osg2 cow.osg cow.osgb -O WriteImageHint=IncludeData
> I've also done some quick comparisons of plugin size. I'm getting
>
> 883968 osgdb_osg.so
> 969502 osgdb_ive.so
> 2319842 osgdb_osg2.so
On my Windows XP SP3 and VS 2005 express, it is:
osgdb_osg.dll = 524288
osgdb_ive.dll = 692224
osgdb_osg2.dll = 839680
So I ignored the size difference because it seems acceptable. But on
my Arch Linux, I got the same result as you did:
osgdb_osg.so = 814115
osgdb_ive.so = 1016547
osgdb_osg2.so = 2842786
I think what causes this is the heavy reliance on serializer
templates. For each ADD_..._SERIALIZER macro of each class wrapper, A
unique class is generated with current mechanism, which may lead to
large consumption. With this idea, I have had another test with the
original osgdb_bin source code, which is the first version born in the
last year, without serialization operations and ascii support, but
with full osg core class wrappers.
osgdb_bin.so = 855455
I'll go deep with more experiments later. IMHO, A direct approach is
to try to reduce the number of inherited classes, especially those
derived from the template serializer class.
> Which... brings me on to me next observation, could the .osg2 scheme
> be added for generic querying and setting of scene graph properties?
> Could it unseat some of the need for osgIntrospection?
At present the osg2 schema is just simple key and value pairs,like:
osg::Object = Name DataVariance
It shows which properties are stored in a specified serializer and
their reading/writing orders (First getName(), and second
getDataVariance), which is used to resort the properties of the plugin
reader's serializers, which might be of another version.
Maybe it could be used for generic querying class methods without too
many modifications, because the setter and getter methods are already
recorded in the serializer class, and could be immediately put into
use if needed.
Thanks for so many suggestions and advices. I'm going to keep on
improving the new file format to make it competent for the 3.0
releases. :)
Cheers,
Wang Rui
#include <osg/Version>
#include <osg/Notify>
#include <osgDB/FileUtils>
#include <osgDB/WriteFile>
#include <ObjectWrapper>
#include <fstream>
OutputStream::OutputStream( std::ostream* ostream, const osgDB::Options*
options )
: _writeMode(WRITE_BINARY), _writeImageHint(WRITE_USE_IMAGE_HINT),
_readyForEndBracket(false), _indent(0),
_out(ostream)
{
if ( !_out )
throw OutputException(_currentField, "OutputStream: Null stream
specified.");
if ( !options ) return;
StringList optionList;
split( options->getOptionString(), optionList );
for ( StringList::iterator itr=optionList.begin(); itr!=optionList.end();
++itr )
{
const std::string& option = *itr;
if ( option=="Ascii" )
_writeMode = WRITE_ASCII;
else
{
StringList keyAndValues;
split( option, keyAndValues, '=' );
if ( keyAndValues.size()<2 ) continue;
if ( keyAndValues[0]=="SchemaFile" )
{
osgDB::ofstream schemaStream( keyAndValues[1].c_str(),
std::ios::out );
if ( !schemaStream.fail() ) writeSchema( schemaStream );
schemaStream.close();
}
else if ( keyAndValues[0]=="Compressor" )
_compressorName = keyAndValues[1];
else if ( keyAndValues[0]=="WriteImageHint" )
{
if ( keyAndValues[1]=="IncludeData" ) _writeImageHint =
WRITE_INLINE_DATA;
else if ( keyAndValues[1]=="IncludeFile" ) _writeImageHint =
WRITE_INLINE_FILE;
else if ( keyAndValues[1]=="UseExternal" ) _writeImageHint =
WRITE_USE_EXTERNAL;
else if ( keyAndValues[1]=="WriteOut" ) _writeImageHint =
WRITE_EXTERNAL_FILE;
}
else
osg::notify(osg::WARN) << "OutputStream: Unknown option " <<
option << std::endl;
}
}
}
OutputStream::~OutputStream()
{
}
OutputStream& OutputStream::operator<<( const osg::Vec2b& v )
{ *this << v.x() << v.y(); return *this; }
OutputStream& OutputStream::operator<<( const osg::Vec3b& v )
{ *this << v.x() << v.y() << v.z(); return *this; }
OutputStream& OutputStream::operator<<( const osg::Vec4b& v )
{ *this << v.x() << v.y() << v.z() << v.w(); return *this; }
OutputStream& OutputStream::operator<<( const osg::Vec4ub& v )
{ *this << v.r() << v.g() << v.b() << v.a(); return *this; }
OutputStream& OutputStream::operator<<( const osg::Vec2s& v )
{ *this << v.x() << v.y(); return *this; }
OutputStream& OutputStream::operator<<( const osg::Vec3s& v )
{ *this << v.x() << v.y() << v.z(); return *this; }
OutputStream& OutputStream::operator<<( const osg::Vec4s& v )
{ *this << v.x() << v.y() << v.z() << v.w(); return *this; }
OutputStream& OutputStream::operator<<( const osg::Vec2f& v )
{ *this << v.x() << v.y(); return *this; }
OutputStream& OutputStream::operator<<( const osg::Vec3f& v )
{ *this << v.x() << v.y() << v.z(); return *this; }
OutputStream& OutputStream::operator<<( const osg::Vec4f& v )
{ *this << v.x() << v.y() << v.z() << v.w(); return *this; }
OutputStream& OutputStream::operator<<( const osg::Vec2d& v )
{ *this << v.x() << v.y(); return *this; }
OutputStream& OutputStream::operator<<( const osg::Vec3d& v )
{ *this << v.x() << v.y() << v.z(); return *this; }
OutputStream& OutputStream::operator<<( const osg::Vec4d& v )
{ *this << v.x() << v.y() << v.z() << v.w(); return *this; }
OutputStream& OutputStream::operator<<( const osg::Quat& q )
{ *this << q.x() << q.y() << q.z() << q.w(); return *this; }
OutputStream& OutputStream::operator<<( const osg::Plane& p )
{ *this << (double)p[0] << (double)p[1] << (double)p[2] << (double)p[3]; return
*this; }
OutputStream& OutputStream::operator<<( const osg::Matrixf& mat )
{
*this << PROPERTY("Matrixf") << BEGIN_BRACKET << std::endl;
for ( int r=0; r<4; ++r )
{
*this << mat(r, 0) << mat(r, 1)
<< mat(r, 2) << mat(r, 3) << std::endl;
}
*this << END_BRACKET << std::endl;
return *this;
}
OutputStream& OutputStream::operator<<( const osg::Matrixd& mat )
{
*this << PROPERTY("Matrixd") << BEGIN_BRACKET << std::endl;
for ( int r=0; r<4; ++r )
{
*this << mat(r, 0) << mat(r, 1)
<< mat(r, 2) << mat(r, 3) << std::endl;
}
*this << END_BRACKET << std::endl;
return *this;
}
OutputStream& OutputStream::operator<<( const ObjectGLenum& value )
{
GLenum e = value.get();
if ( isBinary() )
{
_out->write((char*)&e, GLENUM_SIZE);
}
else
{
const std::string& enumString =
GlobalLookupTable::instance()->getString("GL", e);
*_out << enumString << ' ';
}
return *this;
}
OutputStream& OutputStream::operator<<( const ObjectProperty& prop )
{
if ( isBinary() )
{
if ( prop._mapProperty )
_out->write( (char*)&(prop._value), INT_SIZE );
}
else
{
std::string enumString = prop._name;
if ( prop._mapProperty )
{
enumString =
GlobalLookupTable::instance()->getString(prop._name,
prop._value);
}
*_out << enumString << ' ';
}
return *this;
}
OutputStream& OutputStream::operator<<( const ObjectMark& mark )
{
if ( !isBinary() )
{
int delta = mark._indentDelta;
if ( delta<0 && _readyForEndBracket )
{
if ( _indent<-delta ) delta = -_indent;
_readyForEndBracket = false;
_out->seekp( delta, std::ios::cur );
}
_indent += delta;
*this << mark._name;
}
return *this;
}
void OutputStream::writeArray( const osg::Array* a )
{
if ( !a ) return;
unsigned int id = findOrCreateArrayID( a );
*this << PROPERTY("ArrayID") << id;
if ( id<_arrayMap.size() ) // Shared array
{
*this << std::endl;
return;
}
switch ( a->getType() )
{
case osg::Array::ByteArrayType:
*this << MAPPEE(ArrayType, ID_BYTE_ARRAY);
writeArrayImplementation( static_cast<const osg::ByteArray*>(a),
a->getNumElements(), 4 );
break;
case osg::Array::UByteArrayType:
*this << MAPPEE(ArrayType, ID_UBYTE_ARRAY);
writeArrayImplementation( static_cast<const osg::UByteArray*>(a),
a->getNumElements(), 4 );
break;
case osg::Array::ShortArrayType:
*this << MAPPEE(ArrayType, ID_SHORT_ARRAY);
writeArrayImplementation( static_cast<const osg::ShortArray*>(a),
a->getNumElements(), 4 );
break;
case osg::Array::UShortArrayType:
*this << MAPPEE(ArrayType, ID_USHORT_ARRAY);
writeArrayImplementation( static_cast<const osg::UShortArray*>(a),
a->getNumElements(), 4 );
break;
case osg::Array::IntArrayType:
*this << MAPPEE(ArrayType, ID_INT_ARRAY);
writeArrayImplementation( static_cast<const osg::IntArray*>(a),
a->getNumElements(), 4 );
break;
case osg::Array::UIntArrayType:
*this << MAPPEE(ArrayType, ID_UINT_ARRAY);
writeArrayImplementation( static_cast<const osg::UIntArray*>(a),
a->getNumElements(), 4 );
break;
case osg::Array::FloatArrayType:
*this << MAPPEE(ArrayType, ID_FLOAT_ARRAY);
writeArrayImplementation( static_cast<const osg::FloatArray*>(a),
a->getNumElements(), 4 );
break;
case osg::Array::DoubleArrayType:
*this << MAPPEE(ArrayType, ID_DOUBLE_ARRAY);
writeArrayImplementation( static_cast<const osg::DoubleArray*>(a),
a->getNumElements(), 4 );
break;
case osg::Array::Vec2bArrayType:
*this << MAPPEE(ArrayType, ID_VEC2B_ARRAY);
writeArrayImplementation( static_cast<const osg::Vec2bArray*>(a),
a->getNumElements() );
break;
case osg::Array::Vec3bArrayType:
*this << MAPPEE(ArrayType, ID_VEC3B_ARRAY);
writeArrayImplementation( static_cast<const osg::Vec3bArray*>(a),
a->getNumElements() );
break;
case osg::Array::Vec4bArrayType:
*this << MAPPEE(ArrayType, ID_VEC4B_ARRAY);
writeArrayImplementation( static_cast<const osg::Vec4bArray*>(a),
a->getNumElements() );
break;
case osg::Array::Vec4ubArrayType:
*this << MAPPEE(ArrayType, ID_VEC4UB_ARRAY);
writeArrayImplementation( static_cast<const osg::Vec4ubArray*>(a),
a->getNumElements() );
break;
case osg::Array::Vec2sArrayType:
*this << MAPPEE(ArrayType, ID_VEC2S_ARRAY);
writeArrayImplementation( static_cast<const osg::Vec2sArray*>(a),
a->getNumElements() );
break;
case osg::Array::Vec3sArrayType:
*this << MAPPEE(ArrayType, ID_VEC3S_ARRAY);
writeArrayImplementation( static_cast<const osg::Vec3sArray*>(a),
a->getNumElements() );
break;
case osg::Array::Vec4sArrayType:
*this << MAPPEE(ArrayType, ID_VEC4S_ARRAY);
writeArrayImplementation( static_cast<const osg::Vec4sArray*>(a),
a->getNumElements() );
break;
case osg::Array::Vec2ArrayType:
*this << MAPPEE(ArrayType, ID_VEC2_ARRAY);
writeArrayImplementation( static_cast<const osg::Vec2Array*>(a),
a->getNumElements() );
break;
case osg::Array::Vec3ArrayType:
*this << MAPPEE(ArrayType, ID_VEC3_ARRAY);
writeArrayImplementation( static_cast<const osg::Vec3Array*>(a),
a->getNumElements() );
break;
case osg::Array::Vec4ArrayType:
*this << MAPPEE(ArrayType, ID_VEC4_ARRAY);
writeArrayImplementation( static_cast<const osg::Vec4Array*>(a),
a->getNumElements() );
break;
case osg::Array::Vec2dArrayType:
*this << MAPPEE(ArrayType, ID_VEC4D_ARRAY);
writeArrayImplementation( static_cast<const osg::Vec2dArray*>(a),
a->getNumElements() );
break;
case osg::Array::Vec3dArrayType:
*this << MAPPEE(ArrayType, ID_VEC4D_ARRAY);
writeArrayImplementation( static_cast<const osg::Vec3dArray*>(a),
a->getNumElements() );
break;
case osg::Array::Vec4dArrayType:
*this << MAPPEE(ArrayType, ID_VEC4D_ARRAY);
writeArrayImplementation( static_cast<const osg::Vec4dArray*>(a),
a->getNumElements() );
break;
default:
throw OutputException(_currentField, "OutputStream::writeArray():
Unsupported array type.");
}
}
void OutputStream::writePrimitiveSet( const osg::PrimitiveSet* p )
{
if ( !p ) return;
switch ( p->getType() )
{
case osg::PrimitiveSet::DrawArraysPrimitiveType:
*this << MAPPEE(PrimitiveType, ID_DRAWARRAYS);
{
const osg::DrawArrays* da = static_cast<const osg::DrawArrays*>(p);
*this << MAPPEE(PrimitiveType, da->getMode())
<< da->getFirst() << da->getCount() << std::endl;
}
break;
case osg::PrimitiveSet::DrawArrayLengthsPrimitiveType:
*this << MAPPEE(PrimitiveType, ID_DRAWARRAY_LENGTH);
{
const osg::DrawArrayLengths* dl = static_cast<const
osg::DrawArrayLengths*>(p);
*this << MAPPEE(PrimitiveType, dl->getMode()) << dl->getFirst();
writeArrayImplementation( dl, dl->size(), 4 );
}
break;
case osg::PrimitiveSet::DrawElementsUBytePrimitiveType:
*this << MAPPEE(PrimitiveType, ID_DRAWELEMENTS_UBYTE);
{
const osg::DrawElementsUByte* de = static_cast<const
osg::DrawElementsUByte*>(p);
*this << MAPPEE(PrimitiveType, de->getMode());
writeArrayImplementation( de, de->size(), 4 );
}
break;
case osg::PrimitiveSet::DrawElementsUShortPrimitiveType:
*this << MAPPEE(PrimitiveType, ID_DRAWELEMENTS_USHORT);
{
const osg::DrawElementsUShort* de = static_cast<const
osg::DrawElementsUShort*>(p);
*this << MAPPEE(PrimitiveType, de->getMode());
writeArrayImplementation( de, de->size(), 4 );
}
break;
case osg::PrimitiveSet::DrawElementsUIntPrimitiveType:
*this << MAPPEE(PrimitiveType, ID_DRAWELEMENTS_UINT);
{
const osg::DrawElementsUInt* de = static_cast<const
osg::DrawElementsUInt*>(p);
*this << MAPPEE(PrimitiveType, de->getMode());
writeArrayImplementation( de, de->size(), 4 );
}
break;
default:
throw OutputException(_currentField,
"OutputStream::writePrimitiveSet(): Unsupported primitive type.");
}
}
void OutputStream::writeImage( const osg::Image* img )
{
if ( !img ) return;
*this << PROPERTY("FileName"); writeWrappedString(img->getFileName());
*this << std::endl;
*this << PROPERTY("WriteHint") << (int)img->getWriteHint();
int decision = IMAGE_EXTERNAL;
switch ( _writeImageHint )
{
case OutputStream::WRITE_INLINE_DATA: decision = IMAGE_INLINE_DATA; break;
case OutputStream::WRITE_INLINE_FILE: decision = IMAGE_INLINE_FILE; break;
case OutputStream::WRITE_EXTERNAL_FILE: decision = IMAGE_EXTERNAL; break;
case OutputStream::WRITE_USE_EXTERNAL: decision = IMAGE_WRITE_OUT; break;
default:
if ( img->getWriteHint()==osg::Image::STORE_INLINE && isBinary() )
decision = IMAGE_INLINE_DATA;
else if ( img->getWriteHint()==osg::Image::EXTERNAL_FILE )
decision = IMAGE_WRITE_OUT;
break;
}
*this << decision << std::endl;
if ( decision==IMAGE_WRITE_OUT || _writeImageHint==WRITE_EXTERNAL_FILE )
{
bool result = osgDB::writeImageFile( *img, img->getFileName() );
osg::notify(osg::NOTICE) << "OutputStream::writeImage(): Write image
data to external file "
<< img->getFileName() << std::endl;
if ( !result )
{
osg::notify(osg::WARN) << "OutputStream::writeImage(): Failed to
write "
<< img->getFileName() << std::endl;
}
}
switch ( decision )
{
case IMAGE_INLINE_DATA:
if ( isBinary() )
{
*this << img->getOrigin(); // _origin
*this << img->s() << img->t() << img->r(); // _s & _t & _r
*this << img->getInternalTextureFormat(); // _internalTextureFormat
*this << img->getPixelFormat(); // _pixelFormat
*this << img->getDataType(); // _dataType
*this << img->getPacking(); // _packing
*this << img->getAllocationMode(); // _allocationMode
// _data
unsigned int size = img->getTotalSizeInBytesIncludingMipmaps();
*this << size; writeCharArray( (char*)img->data(), size );
// _mipmapData
const osg::Image::MipmapDataType& levels = img->getMipmapLevels();
*this << levels.size();
for ( osg::Image::MipmapDataType::const_iterator itr=levels.begin();
itr!=levels.end(); ++itr )
{
*this << *itr;
}
}
break;
case IMAGE_INLINE_FILE:
if ( isBinary() )
{
std::string fullPath = osgDB::findDataFile( img->getFileName() );
std::ifstream infile( fullPath.c_str(),
std::ios::in|std::ios::binary );
if ( infile )
{
infile.seekg( 0, std::ios::end );
unsigned int size = infile.tellg();
*this << size;
if ( size>0 )
{
char* data = new char[size];
if ( !data )
throw OutputException(_currentField,
"OutputStream::writeImage(): Out of memory.");
infile.seekg( 0, std::ios::beg );
infile.read( data, size );
writeCharArray( data, size );
delete[] data;
}
infile.close();
}
else
{
osg::notify(osg::WARN) << "OutputStream::writeImage(): Failed
to open image file "
<< img->getFileName() << std::endl;
*this << (unsigned int)0;
}
}
break;
case IMAGE_EXTERNAL:
break;
default:
break;
}
writeObject( img );
}
void OutputStream::writeObject( const osg::Object* obj )
{
if ( !obj ) return;
std::string name = obj->libraryName();
name += std::string("::") + obj->className();
unsigned int id = findOrCreateObjectID( obj );
*this << name << BEGIN_BRACKET << std::endl; // Write object name
*this << PROPERTY("UniqueID") << id << std::endl; // Write object ID
// Check whether this is a shared object or not
if ( id>=_objectMap.size() )
{
ObjectWrapper* wrapper = ObjectRegistry::instance()->findWrapper( name
);
if ( !wrapper )
{
osg::notify(osg::WARN) << "OutputStream::writeObject(): Unsupported
wrapper class "
<< name << std::endl;
*this << END_BRACKET << std::endl;
return;
}
const StringList& associates = wrapper->getAssociates();
for ( StringList::const_iterator itr=associates.begin();
itr!=associates.end(); ++itr )
{
ObjectWrapper* assocWrapper =
ObjectRegistry::instance()->findWrapper(*itr);
if ( !assocWrapper )
{
osg::notify(osg::WARN) << "OutputStream::writeObject():
Unsupported associated class "
<< *itr << std::endl;
continue;
}
_currentField = assocWrapper->getName();
assocWrapper->write( *this, *obj );
}
}
*this << END_BRACKET << std::endl;
}
void OutputStream::start( OutputStream::WriteType type )
{
_currentField = "Header";
if ( isBinary() )
{
*this << (unsigned int)OSG_HEADER_LOW << (unsigned int)OSG_HEADER_HIGH
<< (unsigned int)type << (unsigned int)PLUGIN_VERSION;
if ( sizeof(osg::Matrix::value_type)==FLOAT_SIZE ) *this << (unsigned
int)0;
else *this << (unsigned int)1; // Record matrix value type of current
built OSG
if ( !_compressorName.empty() )
{
BaseCompressor* compressor =
ObjectRegistry::instance()->findCompressor(_compressorName);
if ( !compressor )
{
osg::notify(osg::WARN) << "OutputStream::start(): No such
compressor "
<< _compressorName << std::endl;
}
else
{
*this << _compressorName;
_out->flush();
_out = &_compressSource;
return;
}
}
*this << std::string("0");
}
else
{
std::string typeString("Unknown");
switch ( type )
{
case WRITE_SCENE: typeString = "Scene"; break;
case WRITE_IMAGE: typeString = "Image"; break;
default: break;
}
*this << PROPERTY("#Ascii") << typeString << std::endl;
*this << PROPERTY("#Version") << (unsigned int)PLUGIN_VERSION <<
std::endl;
*this << PROPERTY("#Generator") << std::string("OpenSceneGraph")
<< std::string(osgGetVersion()) << std::endl;
*this << std::endl;
}
}
void OutputStream::compress( std::ostream* ostream )
{
_currentField = "Compression";
if ( _compressorName.empty() || !isBinary() ) return;
BaseCompressor* compressor =
ObjectRegistry::instance()->findCompressor(_compressorName);
if ( !compressor || !ostream ) return;
if ( !compressor->compress(*ostream, _compressSource.str()) )
throw OutputException(_currentField, "OutputStream: Failed to compress
stream.");
}
// PROTECTED METHODS
void OutputStream::writeSchema( std::ostream& fout )
{
// Write to external ascii stream
const ObjectRegistry::WrapperMap& wrappers =
ObjectRegistry::instance()->getWrapperMap();
for ( ObjectRegistry::WrapperMap::const_iterator itr=wrappers.begin();
itr!=wrappers.end(); ++itr )
{
ObjectWrapper* wrapper = itr->second.get();
fout << itr->first << " =";
StringList properties;
wrapper->writeSchema( properties );
if ( properties.size()>0 )
{
for ( StringList::iterator sitr=properties.begin();
sitr!=properties.end(); ++sitr )
{
fout << ' ' << *sitr;
}
}
fout << std::endl;
}
}
template<typename T>
void OutputStream::writeArrayImplementation( const T* a, int writeSize,
unsigned int numInRow )
{
*this << writeSize << BEGIN_BRACKET;
if ( numInRow>1 )
{
for ( int i=0; i<writeSize; ++i )
{
if ( !(i%numInRow) )
{
*this << std::endl << (*a)[i];
}
else
*this << (*a)[i];
}
*this << std::endl;
}
else
{
*this << std::endl;
for ( int i=0; i<writeSize; ++i )
*this << (*a)[i] << std::endl;
}
*this << END_BRACKET << std::endl;
}
unsigned int OutputStream::findOrCreateArrayID( const osg::Array* array )
{
ArrayMap::iterator itr = _arrayMap.find( array );
if ( itr==_arrayMap.end() )
{
unsigned int id = _arrayMap.size()+1;
_arrayMap[array] = id;
return id;
}
return itr->second;
}
unsigned int OutputStream::findOrCreateObjectID( const osg::Object* obj )
{
ObjectMap::iterator itr = _objectMap.find( obj );
if ( itr==_objectMap.end() )
{
unsigned int id = _objectMap.size()+1;
_objectMap[obj] = id;
return id;
}
return itr->second;
}
_______________________________________________
osg-submissions mailing list
[email protected]
http://lists.openscenegraph.org/listinfo.cgi/osg-submissions-openscenegraph.org
