[boost] Re: Serialization Library Draft # 10
Vladimir Prus wrote
1. When I run bjam in examples, I see:
don't know how to make libs!serialization!exampledemo_mi.cpp
don't know how to make libs!serialization!exampledemo_unregistered_ptrs.cpp
I moved these demos over to tests and forgot to update the jamfile.
2. Later, I get compile errors, with g++ 3.3 on Linux. Illustration of the problem
I've come to realize that making this work with all the compilers is going to be
a hellish effort. Finding a common subset of C++ that would compile under
both gcc and msvc 7.0 was a lot of effort. Hopefully that will be a lot of it.
class C {
public:
explicit C(int) {}
};
int main()
{
int i = 0;
static_castC(i);
return 0;
}
Does not work with 3.3. Works with 3.2. Works with 3.3 if explicit is
removed. I'd say it's a gcc regression, so I went ahead and filed
http://gcc.gnu.org/bugzilla/show_bug.cgi?id=12163
Having had other instances of this sort, I would guess you'll get
a response like the new behaviour is required by the standard
paragram 123.12321.1234 a) . So my inclination is to check
all the cases where I've used the above. I'll look into this.
I have as yet not been able to get this whole thing to build and
test on any system other than MSVC 7.0. I've used gcc on
cygwin to compile all the code. But cygwin gcc doesn't support
wide char i/o so I can't test that. Also I have a very wierd error
when I try to link any of the xml tests which I have so far been
unable to resolve. I've started to try to work with Comeau but
am having problems with that also.
I would very much like to hear from someone who has been able
to build and test on other platforms.
Robert Ramey
___
Unsubscribe other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
[boost] Re: boost::array
David B. Held wrote
Jeremy Siek [EMAIL PROTECTED] wrote in message
[...]
However, did he want the exact interface as boost::array? If so, I'd
say we need a new class called ptr_array that adapts a pointer
and a size into an array.
Or perhaps a policy-based smart pointer with an array-wrapping
policy and boost::array-like interface? ;) If the OP is using gcc 3.3,
I might be able to provide exactly such a beast.
Dave
Dave Gomboc wrote:
const int array[] = {1, 2, 3, 4};
std::copy(array, array + 4, ...);
Actually this suits my current need. It compiles in my environment.
Does this mean its universally OK?. I thought I had seen something
like this before but couldn't find it in any of my references so I had been
trying something variations on:
const int array[] = {1, 2, 3, 4};
std::copy(array.begin(), array.end(), ...);
without success - which of course makes sense.
which led me to boost::array. This I found unsuitable because
of the extra copy. I was also unenthusiastic about the intialization
syntax.
So now my immediate problem is addressed. However, now I've
got a couple of questions raised by the responses.
Presumably boost::array was motivated by the desire to make
an array look like a container - which is what I wanted. I found
it unsatisfactory because of the copying. Would it not be interesting
to make an equivalent that would take either an array reference
or a pointer and give it a ful container like interface so that one could
do something like:
const int array[] = {1, 2, 3, 4};
boost::container_facadeint cfa(array);
std::copy(cfa.begin(), cfa.end(), ...);
so that an array or pointer could be used anywhere a container
can be used?
I think it would be a relative easy task given that boost::array
already provides the interface. In fact, just factoring boost::array
into two layers (one to provide the common interface) and one
to provide different constuctors an array or array reference member
might do the trick.
Just a suggestion.
Robert Ramey
___
Unsubscribe other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
[boost] Re: boost::array
Ramey wrote:
Actually this suits my current need. It compiles in my environment.
Does this mean its universally OK?. I thought I had seen something
like this before but couldn't find it in any of my references so I had been
trying something variations on:
If you look at the regression tests you will find a couple of failures.
On windows Borland and MSVC6/7.0 cannot be initialised for anything but
the basic types (int, double, const char *, etc) The two tests that
fail are based on array std::string . If array const char * is
substituted they initialise correctly and pass the tests.
So yes, this is correct for conforming compilers, and so long as you are
not using user defined types will work even on the current compilers
that are broken.
Hmmm - are there any references or language in the standard that
supports this?
Presumably boost::array was motivated by the desire to make
an array look like a container - which is what I wanted. I found
it unsatisfactory because of the copying. Would it not be interesting
to make an equivalent that would take either an array reference
or a pointer and give it a ful container like interface so that one could
do something like:
const int array[] = {1, 2, 3, 4};
boost::container_facadeint cfa(array);
std::copy(cfa.begin(), cfa.end(), ...);
Ok, here's the awkward question. How do we determine cfa.end()?
boost::array works with fixed size arrays, so always knows the size of
the array.
If you want a variable sized array, then the standard library already
provides this with std::vector. So I assume you want either
OK, lets assume the above example were modified to something like
const int array[] = {1, 2, 3, 4};
boost::container_facadeint cfa(array, sizeof(array)/sizeof(int));
or perhaps
const int array[] = {1, 2, 3, 4};
boost::container_facadeint, sizeof(array)/sizeof(int); cfa(array);
or ???
its even concievable that the size could be determined at template
instantiation time using mpl magic. In anycase, I don't see
geting the size of the array at an early enough time to be
any serious obstacle.
a/ a fixed-size array, of size unknown at compile time.
or
b/ an array fixed at compile time, owned outside the container, but
supplying a container-like wrapper?
(a) is certainly possible, and something I have considered implementing
several times myself. However, given std::vector covers so much of the
need I find it hard to justify writing an extra class. AFAICT, the main
difference is the lack of push_back, pop_back, insert and erase.
std::vector is a run-time construct while array is essentially a compile
time construct. I see them as complementary rather than competing
methods.
The issue above is exacly the same in regards to boost::array.
I presume this issue was resolved in favor of adding boost::array to
boost. The only real issue is there place for a version of boost::array
that doesn't require copying.
(b) I am not sold on, as all other containers I am familiar with own
their contents. I feel this change of idiom might confuse otherwise
clear code.
We often use containers of pointers/ shared pointers etc. I don't know
if that qualifies as containers that own thier contents
b is what I'm refering to. To avoid confusion it might be given a different
name - e.g. container_facade or to emphasize identity of interface
and shared implementation it might be boost::array_ref. This is not
a huge issue for me. Though generally I prefer to leverage on
known names to save storage in my aging brain.
Also note that one of the listed advantages of boost::array is that it
does indeed copy-by-value. This means you need to pass-by-reference to
achieve 'traitional' array passing, but have a new facility available
that some of us do find useful g
It really comes down to this. I know I would find it useful. For me its not a great
problem now as it would be easy to build using boost::array as a basis. The
real question is whether sufficient number of other people would find it
sufficiently useful to justify enhancing boost::array (or making something else)
to support the container facade view point.
--
AlisdairM
___
Unsubscribe other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
[boost] Re: question regarding boost array
Robert Ramey wrote:
Currently boost array contains a copy of the array that initialized it.
Is there any reason that boost array can't be enhanced to contain a reference
to the original C array? I think this would make it more useful
to me as well as others.
AlisdairM wrote
I don't see the problem. boost::array is not a wrapper around an
existing array, it IS an array. In the same way, std::vector does not
take ownership of any existing memory you may want to initialise it
with.
The problem is that boost::array is not a wrapper around an
existing array, it IS an array. This implys making a superfluous
copy of the array in some cases.
Given an simple C array, I want a wrapper which will supply an
iterator interface so that I can do something like.
const in array{] = {1, 2, 3, 4};
boost::arrayint ba(array);
std::copy(ba.begin(), ba.end(), std_ostream_iteratorint(std::cout));
Is there already a way to do this? Or is there some reason why
one would never want to do this?
I am not sure what you are trying to achieve, why do scoped_array or
shared_array not serve in this case?
Hmmm - I didn't find shared_array nor scoped_array in my documenation
until you mentioned it. Perhaps it should be added to the index at a higher level.
___
Unsubscribe other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
[boost] question regarding boost array
Currently boost array contains a copy of the array that initialized it. Is there any reason that boost array can't be enhanced to contain a reference to the original C array? I think this would make it more useful to me as well as others. Robert Ramey ___ Unsubscribe other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
Re: [boost] BJAM help request
Beman Daws wrote: Robert, if you didn't get a response to this, you might want to try the jamboost mailing list Rene Rivera responded privately got me all straighened out on this in no time. Thanks Rene. And thanks to you Beman for asking. In the course of investigating this I did come up with another question someone might want to answer. There is a compile option in all versions of VC Enable Function Level LInking /Gy That I don't see referenced in the JAM files for MSVC et al. It would seem to me that this should be included in builds - at least for release mode in order that executable built with library don't include any more unused code than necessary. Is there some reason that this isn't included? Robert Ramey . ___ Unsubscribe other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
[boost] Re: GUI/GDI template library
Bohdan [EMAIL PROTECTED] wrote: BTW, there were some talks about second review of declined serialization library which can be used for resource files ... is this library being developed or it is dead ? I'm still working on it and hope to upload an updated/evolved version shortly (two weeks?) for examination by interested parties. Robert Ramey ___ Unsubscribe other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
Re: [boost] BOOST_STATIC_WARNING ?
John Maddok wrote:
Another problem (I think) is that with BOOST_STATIC_ASSERT you know compilation will
stop.
However, a BOOST_STATIC_WARNING would keep re-apearing in each translation unit
#including your file:
// boost_file.hpp
BOOST_STATIC_WARNING( we're in boost_file.hpp)
// file1.hpp
#include boost/boost_file.hpp
-- appears here
// file2.hpp
#include boost/boost_file.hpp
a) BOOST_STATIC_WARNING only appears once since include files are guarded by
#ifdef
#define
...
#endif
BOOST_STATIC_ASSERT doesn't stop compilation. It only appears once for the
same reason that BOOST_STATIC_WARNING does.
Here is my best effort at BOOST_STATIC_WARNING.
It suffers from the followng deficiencies:
a) On a Microsoft platform it only works within a function definition.
b) It gives a warning that points to the exact line, BUT doesn't have a particularly
eye catching message
Other than that, it seems to work as one would hope.
Robert Ramey
// (C) Copyright Robert Ramey 2003.
// Permission to copy, use, modify, sell and
// distribute this software is granted provided this copyright notice appears
// in all copies. This software is provided as is without express or implied
// warranty, and with no claim as to its suitability for any purpose.
// See http://www.boost.org/libs/static_assert for documentation.
/*
Revision history:
15 June 2003
Initial version.
*/
#ifndef BOOST_STATIC_WARNING_HPP
#define BOOST_STATIC_WARNING_HPP
#include boost/config.hpp
namespace boost {
// used to generate warning
templateconst char *T struct WARNING_MESSAGE;
templatebool
class STATIC_WARNING;
template
class STATIC_WARNINGfalse
{
typedef char type;
};
template
class STATIC_WARNINGtrue
{
typedef int type;
};
} // boost
// Implemenation
// this implementation relies on the warning issued by compilers
// when a constant is truncated. If the boolean is false, the
// compiler emits a warning message. The warning message points
// to the point where the macro was invoked. The warning message
// indicates data lost in an integral conversion. Unfortunately,
// we have yet to discover an way to invoke a more eye catching
// warning message a la BOOST_STATIC_ASSERT.
#if defined(BOOST_MSVC)
// __LINE__ macro broken when -ZI is used see Q199057
// Sooo for this compiler this only works inside a function
#define BOOST_STATIC_WARNING(B) \
{static_cast ::boost::STATIC_WARNING (bool)(B) ::type (0x);}
#else
#define BOOST_STATIC_WARNING(B) \
enum { \
BOOST_JOIN(boost_static_warning_enum_, __LINE__) \
= static_cast ::boost::STATIC_WARNING (bool)(B) ::type (0x) \
}
#endif
#endif // BOOST_STATIC_WARNING_HPP
/1/2/3/4/5/6/7/8
// test_static_warning.cpp:
// (C) Copyright 2002 Robert Ramey - http://www.rrsd.com . Permission to copy,
// use, modify, sell and distribute this software is granted provided this
// copyright notice appears in all copies. This software is provided as
is
// without express or implied warranty, and with no claim as to its
suitability
// for any purpose.
#include boost/static_warning.hpp
#if ! defined(BOOST_MSVC)
BOOST_STATIC_WARNING(false);
BOOST_STATIC_WARNING(true);
#endif
int
test_main( int argc, char* argv[] )
{
BOOST_STATIC_WARNING(true);
BOOST_STATIC_WARNING(false);
return 0;
}
___
Unsubscribe other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
[boost] WinCVS note
I also had problems getting WinCVS to work to taste. I was frustrating as I didn't need to go into the whole facility - I just wanted a sync'ed local copy. I bought a book on CVS but it started to consume a lot of time to use I finally got WinCVS to create a new local copy by using the command line functionality ( login, etc) included in WinCVS. From then on the sync stuff (update) worked fine (though I had to search around for an option in the update). I suspect that WinCVS is a little bit behind the complete CVS syntax. I suppose there might be a lesson here but I've move on. Robert Ramey Edward Diener [EMAIL PROTECTED] writes: How does one get the latest Boost CVS source ? I have WinCVS but have never been able to figure out how to use it to get CVS source on the Internet anywhere. Would anyone like to run me through it ? I know it has something to do with server access but I am dumbfounded by the WinCVS doc, The easiest and most reliable way to use CVS to get the Boost CVS sources is to use the cvs command-line tool (a cvs.exe is part of your WinCVS installation) and to simply follow verbatim the instructions for anonymous CVS access at: http://sourceforge.net/cvs/?group_id=7586 Using boost for the modulename. It's only two lines you need to type. I know WinCVS is supposed to make things easier, but for many jobs it really doesn't - you have to figure out how the instructions everyone else uses can be translated into equivalent checkboxes and menu items in WinCVS, and eventually you need to understand how the command-line works because the abstraction layer provided by WinCVS always leaks the underlying implementation details. Dave Abrahams ___ Unsubscribe other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
RE: [boost]Interest in library generating streambufs from objects
This a very ripe subject My interest stems from my efforts regarding a serialization library. I have had occasion to consider alternatives in this light. I have looked at Jeffs compression streambuf. I think its interesting, useful and nicely done. Given this, maybe you should set your sights even higher. I think it would be more interesting if the following were considered a) composability - suppose I have an encryption facility and a compression facility. I would like to be able to make a streambuf the compresses thhen encryrpts or vice versa. b) suppose I add other facilities such as escaping/unescaping ascii characters, convert to/from wide characters, etc. I should be able to compose these in an arbitray way. c) this suggests to me: i) a implemention set of composable iterator adaptors. ii) a streambuf that can filter input/output through an itererator adaptor. This MIGHT be implemented as a codecvt facet that takes an interator adaptor as an argument. d) This would have the advantage that the conversion facilities would be useful way beyond streambuf e) currently, the new iterator adaptor package doesn't include an example that is suitable for use as a basis for such conversion or transformational adaptors. But its not hard to make one. To summarize, I would like see the concept of adaptable sequence factored out of streambuf and a streambuf built that can use any adaptable sequence. Robert Ramey Jeff Garland wrote Jonathon T. wrote: I have several class templates for producing standard streambufs based on classes with read, write and seek functions (or a suitable subset thereof.) I have used them successfully to access tcp connections, cryptographic routines, OLE compound documents, zip files, etc. There are templates istreambuf and ostreambuf which perform either input or output exclusively, a template double_streambuf which behaves like an istreambuf glued to an ostreambuf, and a template bidirectional_streambuf which performs input and output using a single shared buffer with a single repositionable file pointer. In addition, template parameters can be used to specify code conversion and to turn off buffering, and a contained error object can be used to customize exception throwing. I there is any interest, I will post them at the vault. Jonathan -- I have an interest in this (sorry for the slow response) for a couple reasons. 1) I'd be interested in how this fits with the Boost Socket initiative http://www.crystalclearsoftware.com/cgi-bin/boost_wiki/wiki.pl?BoostSocket 2) I've written something similar for zip files, but looks like you may have a more general solution -- a toolkit for creating new streambuf types. If this is the case, it would be very handy. Jeff ___ Unsubscribe other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
[boost] library build issue
First issue:In building the libraries I had a couple of problems:
In building the boost_1_30_0 I had to make the following change in the jam file in
order for the build to work.
I believe previously I addressed this by specifying more information in the bjam
command line.
#file vc7-tools.jam
extends-toolset msvc ;
# singleton variables...
set-as-singleton VC7_ROOT ;
if ! $(MSVCDir)
{
VC7_ROOT ?= C:\\Program Files\\Microsoft Visual Studio.NET\\VC7 ;
/
#remove space here___/
VC_TOOL_PATH = $(VC7_ROOT)\\bin\\ ;
VC_SETUP = CALL \$(VC_TOOL_PATH)VCVARS32.BAT\ nul ;
}
VC_PDB_NAME = vc70 ;
flags vc7 CFLAGS : /Op ;
flags vc7 C++FLAGS : /Zc:wchar_t,forScope ;
# The following #// line will be used by the regression test table generation
# program as the column heading for HTML tables. Must not include version number.
#//a href=http://msdn.microsoft.com/vstudio/default.asp;Micro-brsoftbrVC++/a
Second issue:I added a new library to my local copy of boost-dev directory tree. In
trying to build I found
the the filesystem, optional and spirit libraries aren't included in the boost-dev
tree while they are included in
boost_1_30_0 tree. As my library used spirit, it wouldn't build. Is this an
oversight or is it simply that the development versions of these libraries live in a
different place.
Robert Ramey
___
Unsubscribe other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
[boost] RE: an XML API in boost
Having used the spirit lib to parse XML input in both mulit-byte and unicode input, (in connection with the serialization library). I want to second the suggestion to look into spirit in this context. The spirit package includes two examples for parsing XML. It seems to me that the more elaborate one would go a very long way to providing boost/standard XML parser. Robert Ramey ___ Unsubscribe other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
[boost] Re: an xml-binary persistence library
Vladimir Prus wrote: iii) requirement to pre-register classes to be saved as pointers through a base class Was it ever considered a problem. You surely have to register a class in order to deserialize it (Java can create a class given its name, but we're in C++). I refering to the issue of forward declaration vs module level registration We spent a quite of bit of time on this as well as type_info, etc. I have added the module level registration which I think would satisfy the concerns you raised and conflict with my concerns. But I want to pass through it one more time to unitify the treatment of external type ids. This is in part inspired by G Rozenthal's concern that RTTI should not be required. I don't really share the concern. But if things are factored in the right way the external type id issue becomes a replaceable component and we won't have to spend a lot of time on the issue. iv) requirement to have separt save/load code for serialization functions I though about it just recently, and stubmled upon the issue of const-correctness. Probably, separate save/load is OK, since I don't know any good alternative solution. I was always a fan of const-correctness that came with the save/load system. Also the save/load system permitted a finer granularity such that a program that only read would only have to import half the headers and library code. (Same for programs that only write archives). However, I was sensitive to the redundance of the save/load system and came to appreciate the usage of the operator for both saving an loading - even at the cost of const - correctness. (thanks to jens maurers effort) After trying out a number of alternatives I came to a solution which permits either to be used on a class by class basis. Can we have a look at it? Probably, putting it to boost-sandbox is good idea. I'm really interested to see how it applies to use case I have at hand. I don't feel its quite ready for that. Some things are still in the experimental stage and I'm still weighng alternatives. After these things are settled I would better be able to provide explanations for the choices I made. Please send me your use case. I am very interested in it. Hopefully soon I will have the package working at the level I desire. Then there's a huge amount of work (re)writing and expanding the documentation and finer granularity test cases. oh then there's the jam build. and then making things work on more compilers,etc. In all its a very large job. So my preferred plan would be: a) finish what I'm doing as far as features are concerned b) expand documentation a minium amount c) expand test cases a minimum amont d) send it privately to boosters who have shown a special interest in this package e) make some more changes f) If there's still interest, upload it to CVS g) see what happens. This would still be a couple of months away. b) versioning at the class level It would be nice, BTW, to have optional versioning The question about optionality features is much more intricate than it first appears. Other features that are candidates for optionality are: a) serialization of pointers b) versioning c) tracking of object addresses to eliminate extraneous copies d) bypassing the serialization system for special cases and some of these features interact with each other. After much experimentation I have made a system which permits selection of optional serialization features on a per class basis. It turns out that ALL the issues raised in the review, including those that I dismissed, are being addressed. I didn't really intend to do this I had resolved to improve the quality of the implementation and leave most of the feature decisions unchanged as I saw them as ireconcilable trade offs. I was much surprised to discover that improving the implementation made apparent that what I had thought were trade offs where in fact artifacts of implementation stratagy decisions. Note to potential library submitters: The boost review process is far more rigorous than what one is normally accustomed to and I suspect that few are really prepared for it. On the other hand, I have no doubt that this the reason for the incredible high quality and utility of the boost libraries. The only thing I can't explain why anyone would subject himself to this. Yet here I am. oh well. Robert Ramey ___ Unsubscribe other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
[boost] is_abstract_baseT ?
Is there such a thing as is_abstract_baseT similar to is_polymorphicT ? Is such a thing possible? I could use it but have been unable to figrure out how to do it. Robert Ramey ___ Unsubscribe other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
[boost] Boost Mailing List Archive question
What is with the Mailing List archive? It doesn't seem to have more than 50 messages in it for January. Also it seems the search function doesn't seem to work in a predictable way. Robert Ramey ___ Unsubscribe other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
[boost] Re:Serialization and Reflection
From: Matthias Troyer [EMAIL PROTECTED]
templateclass Archive
void save(Archive ar, T t)
{
ar member1;
...
}
Here we have a problem as far as I can see: if the class is
polymorphic, how can I serialize the derived class by calling the
save() function of the base class?
the current serialization code doesn't rely on save being virtual
to function. It downcasts the base class pointer to the most
derived class and calls the save function on the recast pointer.
This casting is implemented by void_cast.
Robert Ramey
___
Unsubscribe other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
[boost] Re:Serialization and Reflection
A very interesting post. I also believe it is interesting to consider whether it is valuable to make a system of reflection for C++. I'm not sure whether it is or not - but it is interesting non the less. In particular we would be interested in compile-time reflection as well as runtime reflection. I found the links on your subject interesting. Second, in terms of Serialization, I'm willing to create a portable, efficient binary archiver based on CDR (the format used by CORBA). Since I'm lazy, I will probably steal most of the code from ACE/TAO (http://www.cs.wustl.edu/~schmidt/ACE.html). Note that CDR has minimal overhead (eg: it has variable byte-ordering, so byte swapping may not be necessary). If there's interest, let me know. There was much interest in implementing XDR. I believe that initially it proceeded rapidly but bogged down because it depended on the existence of some code shipped only with unix. Ultimately, implemenation of a portable binary stream, requires specific coding for each machine architecture. This makes the task larger than it first appears. Robert Ramey ___ Unsubscribe other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
[boost] Re:What should Serialization do?
concern in a natural and complete way. A really, really fundamental issue in the submitted library is the usage of Archive as a virtual base class. This is the traditional way of separating interface from implementation. Advantages a) we're used to it b) it permits total separation of UDT serialization specification from archive implementation. UDT serialization specifications don't even have to be recompiled for different archives. c) logically decouples UDT serialization concept from archive implementation concept. d) permits any UDT serialization implementation to work with with any archive implementation e) less compile time dependency - implies simpler code and faster compilations. Disadvantages === a) Does not permit archive implementation and UDT serialization to be coupled. This is the fundamental obstacle to serialization in XML format. b) virtual functions incurr some extra overhead in calling A newer way would be to use template specialization rather than virtual base class to implement the interface / implementation paradigm Advantages a) Permits archive implementation and UDT serialization to be coupled thereby permitting archives to be smarter and facilitating implementation of something like XML. b) not virtual function call over head Disadvantages == a) we're really not used to it yet b) requires coupling of archive and UDT specification. This can make the system harder to understand and use in simple cases. System requires recompilation of the everything for every combination of UDT and archive used in a program. c) significantly larger executables d) much longer compile/build times In the submitted library, I chose option 1 primarily because of a) Whether or not this is the best choice really depends on the other factors mentioned above so I don't see an obvious answer here. In fact, for most situations either would work just as well. Robert Ramey ___ Unsubscribe other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
[boost] Re: What should Serialization do?
From: Ihsan Ali Al Darhi [EMAIL PROTECTED] I have a question for you. Do you think that the serialization library should save exceptions? I don't know if the question is for me but I'll respond anyway. I can't imagine what you mean by this. An exception is a class definition and can be save/restored as can any other class. I'm curious as to why you would want to do this. I'm not saying that there is anything wrong with it - I just wonder if you've come upon some application that hasn't occurred to me. Just to keep the pot boiling, along the same lines, I wonder if there is any sense to serializing iterators. I imagine its possible though I don't know for sure. I wonder if that has any application. Robert Ramey ___ Unsubscribe other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
[boost] Re:Serialization and Reflection
From: Matthias Troyer [EMAIL PROTECTED]
I would like to make a comment here: our aim was just to achieve
portability between all different UNIX variants. We have about ten
different UNIX architectures around, but actually no Windows machines.
Thus, for our purposes XDR provides a perfect and portable binary
format which we have ben using without problems for more than eight
years.
is it possible to snag the xdr source code from FreeBSD unix so it can
be compiled for windows? I believe that the license would be boost friendly
I would love to include an xdr portable binary implementation but I think that
its appeal would be much diminished if itcouldn't be used by windows.
After all, its whole motivation would be is archive portability.
I believe that this addition would make it portable to all platforms
in common usage today (including Apple - based on freeBSD?).
5) Versioning: [snip]
I was more concerned with runtime overhead, but this needs to be timed.
the version is read and stored the first time a class is used. From then
on it is available when references so I doubt there would be any
detectable implact. Of course, I never have any objection to someone
running such a test if there is any doubt about the issue.
I think we have one more disadvantage:
e) will not be able to deal with polymorphic objects, since there are
no virtual template functions
the virtual functions eliminated would be those currently in *archive.
user classes would be the same. Bascially instead of inheriting from
basic_[i|o] archives would be specialization of something like
class biarchive
{
biarchive operator (int i)
...
}
then save/load would look like
templateclass Archive
void save(Archive ar, T t)
{
ar member1;
...
}
So it would be quite similar in concept. I'm rather reluctant to delve into
this too much as it gets me into the position of comparing something that
is fully developed and working with something that is only an idea. Also
I can't ignore that I already invested much work implementing the first approach.
However I also can't ignore that addresses the two main complaints about
the submitted library. Users that want special functions added to the archive
to do things like fast arrays and/or XML tags can have them. Of course then
end up coupling the UDT serialization to a particular kind of archive. This
breaks a valuable feature of the library - independence of the archve
and UDT serialization. This feature makes the library easier to use and
understand in my opinion.
Robert Ramey
___
Unsubscribe other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
[boost] Re:Serialization and Reflection
From: [EMAIL PROTECTED] (Dave Harris) Will it be able to load files from existing 3rd party formats? For example, I currently have a lot of data stored in Microsoft's MFC archive format. This includes very little metadata beyond the version number. The meaning of fields is defined by their order in the archive, and they have a particular system for mapping class names onto object factories. Much of this is moderately lame - eg it doesn't support namespaces. It will be much easier to switch to the new boost framework if archives in the old format can still be loaded (without having 2 lots of code). Is this a reasonable design objective? I believe that it is more than reasonable. The latest submission could handle this pretty easily: When loading a) Add the new serialization declarations to your current MFC classes b) Derive from CArchive to read the first few bytes of data c) if it looks like the new serialization preamble, seek to 0 and invoke ar doc; where doc is the highest level object - usually derived from CDocument. When Saving a) create a new boost::archive leaving the file name the same. b) ar doc your done. Eventually when all your old files have been processed, you can remove the old MFC serialization code from your classes. I don't think it would be too much harder than that. To summarize, if one already has a method to build objects from a file then the serialization scheme doesn't have to re-implement it. The serialization scheme is not dependent on the file format but just the class structure once its loaded. Robert Ramey ___ Unsubscribe other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
[boost] Re: Serialization Review Results
From: Peter Petrov [EMAIL PROTECTED] Something that I would really like to see, is the cooperation between the serialization library and the dynamic_any type (developed by Alexander Nasonov, not yet into Boost). More specifically, it must be possible to serialize and deserialize dynamic_any's if, of course, the underlying types support this and the dynamic_any is configured in a suitable way - for example, to support operator and/or operator, and possibly something more if necessary. This would be a very powerful mechanism, allowing serialization of arbitrary data. Why don't you try to implement it with the current library? The package includes serialization of shared_ptr and auto_ptr as examples. Also includes serialization of all stl collection templates. I'm sure you could implement this in short order. Robert Ramey ___ Unsubscribe other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
[boost] Re: Serialization Review Results
My usage of Serialization is: Serialization is a by directional transformation or mapping of an arbitray set of Objects to a sequence of bytes. The following link has a nice definition of Serializer Pattern which is line with my usage. http://www.riehle.org/computer-science-research/1996/plop-1996-serializer.html Of course a Serialization libary can be used to implement persistence, transmission of objects, and perhaps other things that I havn't thought of. But these are separate functions and totally orthogonal to serialization itself. One thing that my usage of serialization does not include is transformation or mapping to any other arbitrary format. Some formats may not be rich enough to capture the meaning of the set of objects in a useful way. For example capturing an arbitrary set of C++ objects in a CSV file might be possible, but it would be no richer than a simple sequence of bytes. Hence, though possible, it would be pointless. A synonym for serialization would be flattening Robert Ramey ___ Unsubscribe other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
[boost] Re:New mailing list: boost-serialize
What is the motivation for having discussion regarding serialization and/or persistence take place on a separate list? Is it inherently different in some way than other boost projects? Has it consumed more space than say shared_ptr ? Is is so disjoint from other boost topics? Is the discussion some how different than other boost discussions?. Are other topics candidates for a separate list? Would this other list be searchable with the WWW interface? I concede that I can't follow every detail of the mailing lists. I really only place close attention to those topics that I already have a personal interest in and/or catch my eye for some reason. But I can't see that the serialization discussion was so much different than others that take place on the list. Rather than commenting on the details of your review - which would only repeat points previously made by myself and others - I would note that it re-enforces my skeptism that a consensus can be achieved on an implementable set of features. Robert Ramey ___ Unsubscribe other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
Re:[boost] Serialization Library: review
part of the same object. I was also burned by the movement of is_polymorphic from the python libray to its appropriate place in the type_traits library. I really see this as belonging somewhere else as its not specific to archives nor serialization. Anybody have any ideas on this? Accordingly directory structure should reflect this: archive.hpp should be the only header under boost. Rest under serialization and serialization/details directories I very much want to break binary_archive, text_archive, etc out of archive.hpp. Aside from dependency issues it would help clarify that these are not really part of the core of the library but rather the most obvious and common instances of its usage. So I'm sort of partial to a directory structure boost /archive archive.hpp iarchvive.hpp oarchive.hpp ... / serialization ... so that users would use #include boost/archive/iarchive.hpp #include boost/archive/oarchive.hpp or maybe #include boost/archive/text_archive.hpp if input/output were not separated. Robert Ramey if input a ___ Unsubscribe other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
[boost] int vs int32_t [was: Serialiization Review ]
Date: Thu, 28 Nov 2002 18:42:06 - From: Iain K.Hanson [EMAIL PROTECTED] Understood. But C 99 standardised the int*_t types and in anticipation that C++ 0x might do the same, boost has them in the integer library ( cstdint.hpp ). I'm am not aware of this. Will these be added to the language as new fundamental types or as a commonly agreed up set of typedefs? If it is the latter (which is what I would expect) then no change is necessary. Two suggestions *I think* have been made. 1) that the serialisation library recommends to users who want portable archives, should use the boost::int*_t types instead of short, int, etc. This would only be applicable to binary archives. The only currently implemented binary archive has absolutely no pretense to portability. Should a portable binary archive be submitted, The comments and or documentation might make this recommendation, but its not clear to me that it would be universally preferred. 2) That the serialisation library could detect the size of integral types being serialised and use the correct int*_t types instead of short, int etc. This would conflict with the basic idea that the archive classes map fundamental types to the storage and/or transmission medium. The mapping is already being done by typedef so no efforts are required in this area. The included native binary archive checks to see that sizes on reading and writing machines are equal and it could check more if someone want's to override the init function and make it more elaborate. Again, that would be up to the person who submits a portable binary archive. Robert Ramey ___ Unsubscribe other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
Re: [boost] Serialization Library Review
Date: 25 Nov 2002 00:03:08 -0500 From: Jeremy Maitin-Shepard [EMAIL PROTECTED] Hmmm - what I don't understand is how this would be different that calling void basic_oarchive::write_array(void* p, size_t count) Incidently, this would work for any kind of archive - not just the binary ones. But then the serialized data becomes platform specific. A platform with a different endian-ness or a different double size or a different double format could not load the data. - Jeremy if one uses ar t where t is a C array, each item is serialized individually. I believe the original question was how would power users override the item by item virtual function call just blast all the data to the output archive. For such a purpose ar.write_binary(t, sizeof(t)) should do nicely. Robert Ramey ___ Unsubscribe other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
[boost] Serialiization Review repost with consistent quoting
. I don't believe there is any way enough flexiblity could be added to deserialize a file serialized by another system. Note: converting legacy files to a new serializaion system is very easy: load the file into memory using the old system save the data into a new file using the new system. forget about the old system. Since these are major changes I would like to see a new review after they are implemented and thus vote NO for now. However I am willing to help Robert with implementing the changes, improving the library, and am willing to discuss further. I very much appreciate your interest in making an portable implementation of an XDR binary archive binary archive and understand you have made great progress in this in a very short time. Please let me know if there is anything you need else you need from me. Many users feel that this is necessary and it would demonstrate the ease of use of the library. I know you have spend a lot of time studying and working with the library and I much appreciate your efforts. Robert Ramey ___ Unsubscribe other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
Re: [boost] Serialization library review
Date: Mon, 25 Nov 2002 10:41:17 +0100 From: Matthias Troyer [EMAIL PROTECTED] I guess this should be changed to: #ifdef BOOST_HAS_MS_INT64 virtual basic_iarchive operator(int64_t _Val) = 0; virtual basic_iarchive operator(uint64_t _Val) = 0; #endif #ifdef BOOST_HAS_LONG_LONG virtual basic_iarchive operator(long long _Val) = 0; #endif This sounds better. Thanks. whoops, that doesn't work either - this will take some work to address. why can't this be handled using basic_oarchive::write_binary(void *p, size_t count) This does not allow type-specific transformation (e.g. change of byte order) to be performed. Thus we neeed one such function for each primitive type. I fail to see the problem. If you want superfast i/o with only once virtual function call use basic_oarchive::write_binary(void *p, size_t count) If you want each array item handled individually use ar t Note that if t is a fundamental type, all serialization machinery is effectively bypassed. So the only overhead is one vtable indirection which should be small compared to doing something like changing bigendian to little endien The current library is however not consistent since [snip] According to the documentation this seems to be the way we have to implement it. Do you want to tell me that if I just want to support conforming compilers, then I can just specialize the serialization class for my template classes? If so, then only a change to the documentation seems needed. Currently there is only one way to specify serialization of template classes. I believe a better way (as hinted at in the manual) will be available to those concerned only with conforming compiler in addition to the current method. Robert Ramey ___ Unsubscribe other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
[boost] Serialization Library Review
to beg users of different compilers to help out. c) The system would be much more efficient in that platform gurus would know about the quirks for thier particular platform while library developer's maintainers can't really be expected to understand all the quirks for all the platforms. Its the M(libraries) * N(platforms) problem in a different form. d) This effort would be applied only after a library is otherwise approved. Given that the number of library actually approved per year is small, being a platform guru wouldn't be an unreasonable burden. d) platform gurus would be making a valuable contribution even though they don't have the time to design, build, and get approved a new library contribution. e) Such a valuable contribution would justify adding these people to that elite group known as boost library contributor along with the web site picture. The boost library contributor is incredibly valuable currency. A little bit spent in this direction would improve boost code alot. Of course, the currency is valuable because it has been wisely spent - so far - so don't go overboard with this idea. 8.O === By the way, I vote for approval of the library. Robert Ramey ___ Unsubscribe other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
RE:[boost] Serialization library review
-- It should be straightforward to factor out the serialization of pointers and to split the library into archives for the serialization of basic types, and an add-on for the serialization of pointers. That way users who do not need the elaborate pointer serialization mechanism can do so and not incur the performance penalty (in terms of memory and speed) of to this feature, while in cases when it is needed we can add it on to he archive. This feature incurrs no performance penalty if it is not used. That is, code for serializaing pointers is not generated nor included if there are no pointers in the classes to be serialized. There is no real reason for separating pointers from the libary just as there is no real reason separating other data types that required a some special handling (enums, C arrays). I believe that the serialization will be split between input and output parts much as the standard i/o library is today. This will result in nicer size modules. and no, I can't say for sure that it would be strait forward. 5.b) Allow overriding of preamble, etc. --- I would like to have more control over some aspects that are currently hardcoded into the library: * writing/reading the preamble I believe the the preamble will be overridable * obtaining the version number of a class * starting/finishing writing an object * a new type is encountered hmmm - new type is encounterd? I don't know what that means. The motivation is very simple: We have hundreds of gigabytes of data lying around in tens of thousands of files that could easily be read by the serialization archive if there were not too small differences: i) I wrote a different preamble ii) I only wrote one version number for all classes in the archive instead of separate version numbers for each class iii) no information was written when a new class was encountered Since otherwise the interface is nearly identical (many classes contain a load and a save function, albeit with a different name for the archive classes), changing all my codes over to a boost::serialization library would be easy if it weren't for the three issues above. I believe you are wrong here. The interfaces might seem similar but there is no reason to believe that the file formats have very much in common. I don't believe there is any way enough flexiblity could be added to deserialize a file serialized by another system. Note: converting legacy files to a new serializaion system is very easy: load the file into memory using the old system save the data into a new file using the new system. forget about the old system. Since these are major changes I would like to see a new review after they are implemented and thus vote NO for now. However I am willing to help Robert with implementing the changes, improving the library, and am willing to discuss further. I very much appreciate your interest in making an portable implementation of an XDR binary archive binary archive and understand you have made great progress in this in a very short time. Please let me know if there is anything you need else you need from me. Many users feel that this is necessary and it would demonstrate the ease of use of the library. I know you have spend a lot of time studying and working with the library and I much appreciate your efforts. Robert Ramey ___ Unsubscribe other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
[boost] RE: Serialization Library Review
Date: Fri, 22 Nov 2002 16:34:46 -0800 (PST) From: Augustus Saunders [EMAIL PROTECTED] Persistence: A transformation-less transfer of application native data to an alternate storage medium. Only useful and only intended to be useful to applications that apriori agree on object type and layout, presumably by sharing headers. May optionally account for differences in architecture or compiler. Must be symmetric--support both store and load. Alternate storage formats would only differ for effeciency reasons, perhaps at the expense of not supporting constructs not needed by a given application. Serialization: A transformation of application native data into a serial intermediate exchange format specified by the application writer. Whether objects can be read back in an order different than they were stored, or if there is any object identification of any kind, is up to each individual format. Because it is not presumed that applications share apriori knowledge, it may be necessary to include meta-data regarding data types. I edited out alot of the message that expanded upon these definitions. I think the distinction is important. But actually I think the definitions should interchanged !!! The goals of the serialization library are clearly stated in the first part of the documentation. Basically it is to be able to convert of any arbitrary structure of data created by a C++ program into a string of bytes and back again. I called this Serialization to distinguish it from other data storage methods that have a diffferent purpose. I didn't try to define Persistence as I see it as a more general notion. Other data storate/recovery methods have different focus. e.g. Database systems support a relational algebra accross multiple platforms and languages. Other things are considered important such as transactional integrity, security, logging and rollback, etc that are not of interest in this context. XML, as I understand its motavation, attempts to package data along with its metadata in a more general way to promote portability across different applications. No logically transparent system can address all of these situtations. They conflict in fundamental ways. Of course, it can be proved that all data is equivalent in some sense, but in the sense that we use it an think about it its not equivalent. We impose a structure to think about it operators to manipulate it in accordance with a larger purpose. The archives used for serialization ar really small bridges to the i/o system. Attempts to make them something they are not can only lead to a system which lacks elegance and simplicity and is hard to use for its intended purpose. I realize that the system as presented can benefit from some improvements. But, as far as I know, there is no system which addresses so comprehensively the question of Serialization (as I have used it). I could find no such system. Certainly none has been submitted has been submitted to boost. That's why I wrote this one. I was strongly influenced by my experience with Microsoft MFC. It was a basically a good and useful system with a fair number of short comings that I sought to remedy. I believe I have largely accomplished that purpose. Thats all for now. I will have much more to say on this topic in the coming days. Robert Ramey ___ Unsubscribe other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
[boost] FW: Serialisation library review
Note: This was sent directly to me by Pavel Vozenilek. I am posting it to
the list.
Robert Ramey
I recommend to accept serialisation library into Boost.
I played with the library for few hours and used Intel C++ 6.0
plugged in Visual C++ 6.0 IDE (and Visual C++ 6.0 STL) to compile
examples and debug the code. I also read the documentation available.
I like the library interface: within C++ limits it is
minimal and natural. I had coded serialisation code few times
in similar way (and didn't like to write the infrastructure over
and over).
The implementation needs a quite lot of effort to understand: probably
more comments on internal structures would help. From pragmatic point
of view I see this as the smallest problem.
The documentation is IMHO the weakest point. Serialisation probably
isn't highest priority for application programmer and lack of easily
understandable documentation can drive him/her of this library
to some ad-hoc quick and dirty code.
I recommend:
- complete redo of the documentation (the text should
be made more clear on many places),
- many more of code examples in documentation
(and these being complete and compilable),
- the code examples should have different background,
e.g. like it is in Spirit library documentation,
it is then much easier to read them,
- reference documentation should be broken into few parts,
More issues are in attached text file. None of them, however
looks to me as showstopper.
/Pavel
_
Help STOP SPAM with the new MSN 8 and get 2 months FREE*
http://join.msn.com/?page=features/junkmail
--
MAJOR ISSUES:
1) Is there any exception for disk-full error? Also exception for
invalid state stream should exists.
2) function text_archive.init() should not call seekp(0).
The stream may contain other data that would be overwritten.
3) This is a bit nitpicking: volatile members/pointers/classes may be
also serialised. Currently it doesn't work.
4) I could not make demo.cpp working: there's some problem with
is_polymorphic. I will be glad to provide more info if asked.
5) function get_guid() isn't mentioned in the documentation.
--
MINOR ISSUES:
1) Is there any possibility to plug-in compression of data (or make
archive subclass for it)? If, can it be documented with small example?
2) Likewise, is there any chance to plug-in OS specific file routines?
(e.g. WIN32 non-buffered reads and/or scattered reads/writes).
This may be important for performance reasons.
3) If exception happens during writing data to file,
shouldn't the stream be rewinded back to original position?
4) set, multiset, hash_set and hash_multiset don't seem to be
included in stl.hpp. Also maybe rope ('big string' from SGI STL)
can be included.
5) The function text_archive::newtoken(): does it really need to be
virtual function? It is pretty big overhead to call it for each
member field.
--
CONSISTENCY CHECK SUPPORT:
(manually written serialisation code can be very dangerous during
maintenance)
1) Can consistency between save() and load() functions supported?
I mean e.g. number of items saved/loaded needs to be the same,
their names/types the same etc.
Maybe if there is macro or runtime parameter to enable/disable
this type of check.
2) Can we have some function returning size of data being written
for an object? It could be used for checking, like:
void AClass::save(...) {
ar field1; ar field2; ...
assert(sizeof(AClass) == ar.writtenBytes()); // check
}
(Such a check is applicable only under circumstances.)
3) Can we have save()/load() functionality merged together not to duplicate
code
and make hand-written code safer?
Something like:
void load_save(..., bool is_saving) {
ar.load_or_save(field1, is_saving);
ar.load_or_save(field2, is_saving);
}
Maybe such a functionality can be provided in addition to current code.
4) Can there be library version (maybe enabled/disabled by macro) that
doesn't require (and doesn't store) version()? This may be of use
for high performance stable systems.
5) Can the call to base class save()/load() from derived save()/load() be
recognized (and assert()ed) in debug version?
(As opposite to valid call base_object().)
--
DOCUMENTATION ISSUES:
1) Can space requirements be documented for ASCII/UNICODE/binary
archiving? Can it be documented on example?
2) First example in tutorial.html includes iostream twice.
Also using namespace std; is missing. ofs.close() should be
replaces by ifs.close(), ar by ia.
Also information what *.cpp file to add to the project
[boost] FW: The results of your email commands
From: Alberto Barbati [EMAIL PROTECTED]
One note: the library, as it is, *does not* support Unicode output, as
stated.
[snip]
Well I quadriple checked and ran your example and of course you are right.
The text archive eliminated the high order byte. I have addressed the problem
with the code below.
Naturally there is an analogous alteration in the output archive.
My reasoning is as follws
text files created/read as part of serialzation are not designed to be
read/interpreted by
humans. The important thing is that the store all the data and recover it without
alteration. Since archives might be passed accross locales, all serlialzation
files
should use the same locale. Of course that locale should correctly support
wide characters and not alter the data. The text should be embeddible in
other texts.So the system is:
Open the archive
save the current locale
create a new locale
classic + override for codecvt facet to supress code conversion
use archive
OnClose restore previous locale.
This seems to me to address the issues of wide characters.
Robert Ramey
/
// class text_iarchive - read serialized objects from a input text stream
templateclass Stream, class Elem
class text_iarchive : public basic_iarchive
{
private:
class codecvt : public std::codecvtElem, char, mbstate_t
{
virtual bool do_always_noconv( ) const{
return true;
}
public:
codecvt() : std::codecvtElem, char, mbstate_t(1) {};
} archive_codecvt;
// locale when stream was passed here
std::locale previous_locale;
// use special local for serialization - no character conversion
std::locale archive_locale;
// stream which contains the archive
Stream is;
public:
virtual void init(){
basic_iarchive::init();
}
text_iarchive(Stream _is) :
is(_is),
archive_locale(
std::locale::classic(),
archive_codecvt
)
{
// archives always use classic locale
#ifndef BOOST_NO_STD_LOCALE
previous_locale = is.imbue(archive_locale);
#endif
init();
}
~text_iarchive(){
#ifndef BOOST_NO_STD_LOCALE
is.imbue(previous_locale);
#endif
}
- Done.
---BeginMessage---
Date: Tue, 12 Nov 2002 23:07:19 +0100
From: Alberto Barbati [EMAIL PROTECTED]
One note: the library, as it is, *does not* support Unicode output, as
stated.
[snip]
Well I quadriple checked and ran your example and of course you are right.
The text archive eliminated the high order byte. I have addressed the problem with
the code below.
Naturally there is an analogous alteration in the output archive.
My reasoning is as follws
text files created/read as part of serialzation are not designed to be
read/interpreted by
humans. The important thing is that the store all the data and recover it without
alteration. Since archives might be passed accross locales, all serlialzation files
should use the same locale. Of course that locale should correctly support
wide characters and not alter the data. The text should be embeddible in
other texts.So the system is:
Open the archive
save the current locale
create a new locale
classic + override for codecvt facet to supress code conversion
use archive
OnClose restore previous locale.
This seems to me to address the issues of wide characters.
Robert Ramey
/
// class text_iarchive - read serialized objects from a input text stream
templateclass Stream, class Elem
class text_iarchive : public basic_iarchive
{
private:
class codecvt : public std::codecvtElem, char, mbstate_t
{
virtual bool do_always_noconv( ) const{
return true;
}
public:
codecvt() : std::codecvtElem, char, mbstate_t(1) {};
} archive_codecvt;
// locale when stream was passed here
std::locale previous_locale;
// use special local for serialization - no character conversion
std::locale archive_locale;
// stream which contains the archive
Stream is;
public:
virtual void init(){
basic_iarchive::init();
}
text_iarchive(Stream _is) :
is(_is),
archive_locale(
std::locale::classic(),
archive_codecvt
)
{
// archives always use classic locale
Re: [boost] Serialization XML (was Serialization Library
From: Beman Dawes [EMAIL PROTECTED] At 10:01 PM 11/18/2002, Robert Ramey wrote: Is there a reason you sent this to me privately? From: David Abrahams [EMAIL PROTECTED] I believe your assessment that some data structures can't be represented using XML is incorrect, and that's easy to prove. A serialization library which makes generation of XML output difficult is severely handicapped in the modern world. Well, I have conceded that it was preliminary. All I know about XML is from a small book containing a concise description of XML. My skeptism is based on the following thought experiment: Suppose on is given a list of polymorphic pointers, some of which correspond to bottom node of a diamond in heritance structure and some of which are repeated in the list and serialized some where else as well. a) How would such a thing be represented in XML? b) Could be loaded back to create an equivalent structure? c) Would it be useful for anything other than this serialization system? If someone can assure me that the answers to all three of the above is yes then it should be possible - otherwise not. Given that its easy to prove these questions should be easy to answer in a convincing way. Robert, I think you may be missing several points with your thought experiment: * The serialization library doesn't have to figure out how all C++ data structures (such as in your thought experiment) would be represented in XML My question is whether XML can capture an arbitrary C++ structure in a meaningful and useful way. So far no one has presented any XML that captures that one proposed example. or any other format. Instead, all that serialization has to supply is a base class with the default hooks for prolog, epilog, separator, data, and similar functions. Well, I don't know that. In general it is extremely difficult to know ahead of time what facilities a serialization library would need to be permit an XML archive to be generated. One would have to take a the library, make changes necessary to provide the desired result and check to see what changes are necessary. It is up to the user to customize for a particular format, beyond a few basic ones supplied by the implementation. * Some approaches, including XML, allow a practically unlimited number of different ways to represent the same data. The user rather than the serialization library should choose the particular design. * Some formats may not be able to support all C++ data structures, and that is okay. For example, the comma separated value (CSV) format used by many desktop tool programs won't extend much beyond arrays of simple structures. That doesn't mean the format is useless and or that it shouldn't be supported. It just means it isn't suitable for all tasks. It does mean that its not suitable for capturing the structure of an arbitrary C++ data structure in a way that it can be restored. It may be useful, but its not serialization. One of the main features of this system that has permitted to cover and even surpass the original set of requirements is the decoupling of different aspects. In the current system the following concepts are orthogonal a) The description of the which data should be saved for each class (save/load/version) b) composition of the above to handle arbitrary C++ data structures (serialization) c) description of how fundamental types should be encoded as a byte stream into a storage medium (archive) Assuming that the questions in my Thought experiment could be answered in the afirmative. What would have to be added to this system to permit it to handle XML. Another concept has to be added - that of reflection. A useful XML representation needs the name of the variable. So some system needs to be designed to hold that information and keep it related to each serializable member. Presumably this would be a orthogonal concept d) Given this, without too much effort and maybe adding some virtual functions to archive one could add begin/end tags to archive. Of course many would object to this on efficiency grounds but it would be possible. But things start to appear. What about versioning? where does that fit into XML? But what about pointers, inheritance, etc. to properly capture this in XML one would have to start altering b) . Its the automatic composition that guarentees that this system can serialize/deserialize any C++ structure. I doubt this would be worth it. At the heart of the matter is that serialization, XML, CSV, Databases, etc. are designed for different and conflicting purposes. After looking at XML and understanding as well as anyone what it takes to make a serialization system, I made a preliminary determination that trying to include XML would result in a system that compromised the other goals. The discussion on the list in reference to this topic confirms my preliminary conclusion. Of course, anyone is free to the the current serialization
[boost] is_polymorphic question
It has been reported to me that is_polymorphicT gives compile time error when T is const type and the compiler is g++. Is there a fix for this? Robert Ramey ___ Unsubscribe other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
[boost] RE: Serialization Library Review - archive exception
From: Jeff Garland [EMAIL PROTECTED] On archive exception: A short recap on the history of this: 1) I made orginally made archive_exception the simplest possible, it wasn't derived from std::exception and and contain an enum of every exception type. It suited my needs and I didn't feel that std::exception added anything. 2) well lots of complaints, some people didn't know about catch(...) syntax and thought that it had to be derived from std::archive to have a catch all. Others thought the embedded string was important. 3) I agreed with none of these things but what the hell, its easier to accomodate than make a big deal on this insignificant point. Its obvious that in retrospect I was wrong. In general, libary code should make no presumptions as to the language of the user. That means not embedded messages. If you want a key in to a message table, use archive_exception::exception_code that's what my intention was. In my view std::exception is a mistake and should be removed from the standard. Robert Ramey ___ Unsubscribe other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
[boost] RE: Serialization Library Review
From: Peter Dimov [EMAIL PROTECTED] I'm really interested in the XDR format, not because I care about the format itself, but because others seem to use it as some sort of litmus test for serialization libraries. Thus knowing that Robert's library handles XDR well is of interest. FWIW, in my experience, XML is a better test for whether a serialization library handles custom formats well. Sequence-based, header-only formats are very similar, but a reasonable XML serializer creates a tree-like representation, with nested tags. After a cursory investigation as to what it would take to make an XML archive, I've concluded that: a) I don't thing XML is rich enough to capture all possible C++ data structures b) This would imply the creation of a system of reflection for C++ i) I not convinced this is a good idea ii) should be dealt with as an indepent project in any case c) XML has a central task to represent data in a platform/programming language independent way. Serialization has the central task of saving/restore the totality of the state of C++ data structures. These tasks are not identical and any course of action based on the presumption that they are is doomed to be frustrating an most likely a failure in my opinion. Robert Ramey ___ Unsubscribe other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
[boost] RE: Serialization Library Review
From: [EMAIL PROTECTED] (Dave Harris)
From the headers...
typedef unsigned char version_type; // upto 255 versions
namespace serialization_detail {
typedef unsigned short class_id_type; // upto 64k kinds
// of objects
typedef int object_id_type; // upto 2G objects
}
It seems to me these limits are arbitrary, and in some cases rather low.
Wouldn't it be better, and more general, to use int or long?
they are arbitrary. but unimaginably high in my view. more than 64K class
definitions in a program? maintaining compatibility with more than
256 previous versions of a particular class? more than 2G objects?
This is a problem?
On a related note, I think variable length integers ought to be supported
as primitive.
[snip]
If we are saving in an ASCII format we wouldn't want to do this because ASCII is
intrinsically
variable length anyway.
you answered your own question.
But it touches upon another issue.
I have gone to great lengths to NOT include ANY non fundemental types as
primitives. The only exception is std::string and std:wstring.
The power of the library is the fact that it can be added used with
any set of classes in a completely orthogonal way. It makes
no presumptions about the data types a user want's to serialize
and no presumptions about the storage medium. Attempts to
make it more efficient that alter these design decisions would
end up make it too complex to use.
Are reference-counted objects supported? If I am using my own
reference-counted class, what do I have to do to get serialisation to work
for it?
[snip]
Serialization of boost::shared_ptr is included as an example. I believe
it address the issues you raise. It also is a good example of how
correct serialization can be implemented non-intrusively for one
of the most subtle examples in the boost library.
I have looked at the posted code but I've not been able to figure it all
out. I see lots of use of (T *), which is wrong.
whats wrong with it?
I ask because in the past I have been bitten by Microsoft's MFC system,
which makes no provision for reference counting.
reference counting is a function of any smart pointer being serialized
not of the serialization system itself - see above cited example.
(1) The 1-step construction issue. In other words, whether:
const MyClass object(ar);
can be supported.
This should not be supported for reasons cited in previous post.
(2) Whether the archive handles reference counts correctly.
This is addressed in the code an tested pretty rigoursly. I have
no evidence that there is any problem in this area. If you have
a counter example please submit it.
(3) That variable length integers be supported. [ as a primitive]
This an extremely bad idea for reasons cited above
(4) That the limits on version numbers etc are too small.
This suggests a lack of shared understanding as to what
the version numbers are used for. Have you a class for which
you need to maintain compatiblity with 255 previous versions?
Robert Ramey
___
Unsubscribe other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
[boost] RE: Serialization Library Review
From: Jeff Garland [EMAIL PROTECTED] If there are technical reasons why the library cannot be extended to do this than I would definitely vote to reject. It sounds like that's what you and Robert are saying, but I don't understand why you think this? I have to admit I have only a cursory knowledge of XML. I bought a book just consider this question. When I tried to envision rendering things like stl containers of polymorphic pointers to objects with diamond inheritance so of which are repeated into XML my imagination failed me. Oh then there is the data name - not readily available in C++. So I rebuffed requests for assurances that this system can be extended to XML. As I do now. This question came up before in the context of a proposed alternative design that purportadly would be extensible to cover XML. I did demonstrate that this system was at least as powerful as the suggested alternative so that wasn't an argument for the alternative. If you need an apriori guarentee that this is extensible to XML in order to vote for the library then your decision is easy. I don't believe anyone can give such a guarentee for this case. Of course, if someone does manage to do this - great. But until someone does, you should vote against it. Robert Ramey ___ Unsubscribe other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
[boost] RE: Serialization Library Review
From: Beman Dawes [EMAIL PROTECTED] Uh... I think one of us must be misunderstanding something about XML. If you can represent serialization of some objects as a character based file, can't you turn it into dumb XML by wrapping it in my_dumb_tag/my_dumb_tag? Perhaps you mean you don't think it is possible to define a set of tags that themselves are rich enough to capture all possible C++ data structures? But so what? You can just supply content directly for any C++ structures that don't represent well in the XML tagged style. And we know from experience that many C++ data structures can be usefully represented in an XML style format. That may be. But a serialization system has to be able to handle ALL (not just many) C++ data structures. That seems a rigid view of XML. Just think of it as a way to markup the output you are already producing to make it a bit easier for other tools to work with. Some of the whitespace or other delimiters become tags. If the output you are already producing captures various C++ data structures, adding XML markup just enhances the already captured data. That's probably a dumb way to use the power of XML, but it may be sufficient for serialization needs. Serialization files contain tables of objects, pointers, classes etc in such a way that even text archives are effectively indeciferable. Of course they are text files so the are readable, producible by any text reading/writing program. But such processing would be useful only another program could exploit the structure of the data. This would be hard to do without the whole serialization code. The attraction of XML is that the structure is regular and accessible to a wide variety of programs thus making data portable. XML may have the escape clauses that could be used to wrap serialization data, but doing so would fail to capture the meaning of the serialized data which would defeat the apeal of XML in the first place. So looked at in this way I might be convinced it was doable, but then I would be unconvinced it was worth doing. I must confess I have zero real world experience with XML so I'm not prepared to stake a hard line on this. It really just mostly intuition plus having first hand knowledge how difficult it is to map an arbitray C++ data structure to a byte stream and back. Robert Ramey ___ Unsubscribe other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
[boost] Re: Reminder: Serialization Library Review
the howl will start again. stay tuned. 5.) This is a point for discussion an no criticism about the library. Instead of polluting the global namespace with a serialization class, I would prefer to implement serialization with free functions save and load instead. Wouldn't that just pollute the global namespace with a large number of save/load functions instead? 6.) Finally, if I am correctly informed, the Java language includes serialization and has a portable archive format. Could this library be made compatible with this Java language standard, i.e. might it be possible to create an archive format which can read such Java serialization files? This is not likely, Java has runtime reflexion which is used to obtain all the information required for serialization. Also, Java has a much more limited usage of pointers so certain problems we are dealing with don't come up. I don't believe that all the data structures can be unambiguously mapped to java. What is your evaluation of the implementation? I would like to see a platform-independent binary archive format (e.g. using XDR), but am also willing to contribute that myself once the interface has been finalized. Thank you. Note that none of the comments made so far have any impact on the interfaces defined by the base classes basic_[i|o]archive, So there is no reason you can't get started now. As you can see from the 3 derivations included in the package, making your own XDRarchive is a pretty simple proposition if you have the xdr-float code. In this case take a copies of biarchive and boarchive and reimplement the functions to read/write XDR instead of binary data and you're done. What is your evaluation of the documentation? As was already remarked by others, I would like to see documentation on exactly which functions a new archive type has to implement. Wouldn't be easier just to look at the basic_[i|o]archive code? Perhaps we might want to break out text_archive an native binary archive into separate headers. That might make it more obvious that these derivations aren't really part of the library but rather more like popular examples. I tried to use the library but could not compile it under MacOS X 10.2 with gcc 3.1 Compiling the file demo.cpp gives me the error: ./../boost/serialization/serialization_imp.hpp:382: sorry, not implemented: ` tree_list' not supported by dump_expr Hmm in my copy that corresponds to a statement: BOOST_STATIC_ASSERT(false); You can just comment this out for now Robert Ramey ___ Unsubscribe other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
[boost] Re Serialization - locale
Gentlemen,
Those interested in locale for text archives:
How does this strike you?
text_iarchive(Stream _is) :
is(_is)
{
// archives always use classic locale
#ifndef BOOST_NO_STD_LOCALE
plocale = is.imbue(std::locale::classic());
#endif
init();
}
~text_iarchive(){
#ifndef BOOST_NO_STD_LOCALE
is.imbue(plocale);
#endif
}
Another observation:
I note that my test.cpp program includes wchar_t member variables initialized to
values in excess of 256.
The system doesn't seem to lose any informaton in storing/loading to a stream with
classic locale.
Robert Ramey
___
Unsubscribe other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
Re: [boost] Serialization Submission version 6
have come to the conclusion that its undoable for fundemental reasons. I havn't given up - it may be possible to craft things such that there is a default overridable system etc. But believe me its much more subtle than first appears. As a matter of fact, I don't think anyone who hasn't done it can appreciate the incredible amount of effort it takes to make a library such as this given the very wide range of users with conficting design goals. Add to this trying to make this work with compilers that have way too much variation and you have a recipe to aggravation almost beyond human capacity. Note that I am posting a response to another user that touches this topic. No doubt you'll be interested. Robert Ramey ___ Unsubscribe other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
[boost] Re: Serialization Library Review
Alberto Barbati wrote 2) A most needed addition to the design is to provide a sort of registry object. This has been a hot topic. It is really not possible to achieve the desired results. I will add a section to th rationale explaining this in detail, Please note that the registry class I described *does not* attempt to solve the broader issue of UUIDs I read about in the discussion between you and Vladimir. My proposal is just a way to separate the registration part from the serialization into two different compontents. Still the classes will be matched according to their registration order, as it happens now. I am ready to discuss the opportunity and/or usefulness of this approach, but I don't see the reason why this could not be done. I considered this approach and found the following problem Suppose I automatically add the class info to a global collection of registered types. This I can easily do. Now when I create an archive I can a) register all the types in the global collection in the archive. bad idea - this would require that the reading program register all the types of the writing program. An intolerable requirement b) register types as needed as the library is written wouldn't work - on loading, we wouldn't know which types to register c) after creating the archive, append a registration file on loading, process the registration file first. In my view this cure is worse than the disease. One note: the library, as it is, *does not* support Unicode output, as stated. The library supports wide streams, yes, but that does not mean Unicode support. So what do I have to do exactly in the warchive specialization to generat Unicode output? As I said in my post, you have to imbue the stream with a locale holding the correct codecvt facet, before using it to serialize. Unfortunately, such a facet is not part of the ANSI standard and is the subject of a separate proposal of mine (see thread codecvt facets for utf8/utf16/utf32). BTW, with the current implementation even doing so is completely useless, as there are lines like this: os.imbue(std::locale::classic()); that reset the locale of the streams to the dumb default. Such lines are IMO both unnecessary and conceptually wrong, and should be removed. I'm still at a loss. Aside from addressing the classic above, what exactly do you recommend I do? How about if I change the wording to specify that the library supports wide streams and leave unicode out of it? On a different topic, I found a portability issue. The current implementation record in the archives the size of the basic types int, long, float and double. This gives you a *false* sense of security that the writing and reading platform agree on the type size. this is for the native binary archive which is explicitly described as being non-portable. It was included because some users felt it would be more efficient. It has no pretensions at all to portability. Knowing that some one will ignore this admonishment and try to move such a binary to another machine architecture, Included free detection so that it would crash in a more graceful manner. Robert Ramey ___ Unsubscribe other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
Re: [boost] Serialization Submission version 6
Alberto Barbati wrote BTW, with the current implementation even doing so is completely useless, as there are lines like this: os.imbue(std::locale::classic()); that reset the locale of the streams to the dumb default. Such lines are IMO both unnecessary and conceptually wrong, and should be removed. Now I remember why I included this. Suppose that an archive is created where the default local is a spanish speaking country where the number 123 thousand is written 123.000 The archive is sent to another country where the default locale is an english speaking country where the string 123.000 means 123 That's why I set the local to classic. Robert Ramey ___ Unsubscribe other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
[boost] Re: Serialization Library Review
From: Alberto Barbati [EMAIL PROTECTED] Hi David, 1) I don't like the non-intrusive way of specifying the version/save/load operation. a non-intrusive method is required to implement serialization for classes that you don't want to change. For example, the library includes serialization for all STL containers with without changing STL itself. This would permit easy and optional addition of serialization to any class that might benefit from it 2) A most needed addition to the design is to provide a sort of registry object. This has been a hot topic. It is really not possible to achieve the desired results. I will add a section to th rationale explaining this in detail, - What is your evaluation of the documentation? One note: the library, as it is, *does not* support Unicode output, as stated. The library supports wide streams, yes, but that does not mean Unicode support. So what do I have to do exactly in the warchive specialization to generat Unicode output? On VC++ 7.0 (.NET) it all went good at the first try. However the restriction, cited in the documentation, of having to add /OPT:NOREF to the linker options is too hard to swallow. I agree that this is exceedingly annoying. I am looking into it. Robert Ramey ___ Unsubscribe other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
