> > On Sun, Jun 14, 2015 at 4:40 PM, David Knezevic < > [email protected]> wrote: > >> In RBEvaluation::read_offline_data_from_files and >> RBEvaluation::write_offline_data_to_files), we read/write from/to a bunch >> of small Xdr files. I've always thought this is pretty messy and I've >> wanted to find a better approach. >> >> I think a better approach would be to use the Cap'n Proto >> <https://capnproto.org/> serialization library. I've made a start on >> implementing this here: >> https://github.com/dknez/libmesh/tree/rb_capnproto >> >> But before I go further with the implementation, I'd like to run this by >> the list to confirm that this change can be incorporated into libMesh. >> > > In my opinion, yes it definitely can be incorporated. >
OK, great. I'll keep working on the branch then, and I'll make a PR when it's ready. > Having used it, are you sufficiently convinced that it is the right way to > go for what you want? I agree that the binary decoding feature is a good > one, but I'm not crazy about the "schema/preprocessor" thing. > I do think Cap'n Proto is the right way to go. It's a widely used tool for serialization, and it's faster than other serialization libraries because it doesn't do any encoding (this is the "infinitely faster" thing that they explain on their website). Regarding the schema/preprocessor: This is the approach that the main serialization libraries use (e.g. protobufs, flatbuffers, Cap'n Proto, Thrift), and I quite like the schema definition because it provides a clear specification of the serialization format that we're using. > Some comments: >> >> - Cap'n Proto requires C++11. My plan would be to make Cap'n Proto an >> optional package, and to keep the current reduced basis read/write code if >> C++11 and Cap'n Proto aren't available. >> > > Sounds good. I regularly build libmesh with -std=c++11 and Moosebuild > currently tests c++98 mode, so both code paths will be tested. > OK, sounds good. > - We define a "schema" for the serialization format, e.g. see here: > >> >> https://github.com/dknez/libmesh/blob/rb_capnproto/include/reduced_basis/rb_data.capnp >> Cap'n Proto then provides a tool to compile the schema into C++ code, >> which yields these files: >> >> https://github.com/dknez/libmesh/blob/rb_capnproto/include/reduced_basis/rb_data.capnp.h >> >> https://github.com/dknez/libmesh/blob/rb_capnproto/src/reduced_basis/rb_data.capnp.C >> So there is a question of what to include in the libMesh source tree >> (just the schema file, or the generated code as well). >> > > My first instinct would be to provide the generated code (as long as it's > not too large in size) the way we do with autotools, if only to avoid > needing to add automake targets/bootstrap code for building it. This is > assuming it won't change much once you get it set up the way you like... > OK. The generated code is about 300KB currently, so I think size isn't an issue. However, one thing worth mentioning here is that the version of Cap'n Proto used to generate the code has to match the version that is linked to at runtime. As a result, the most robust thing to do would be to generate the code using whatever version of Cap'n Proto the user has... could that be done at configure time in libMesh? Basically this just requires us to call "capnp compile -oc++ rb_data.capnp", where capnp is a binary that is in $CAPNP_DIR/bin. If that's too difficult, then it'd be fine with me to just include the generated code like you suggested and to specify the version of Cap'n Proto that needs to be used. > - Cap'n Proto provides a tool for "decoding" binary data: > >> https://capnproto.org/capnp-tool.html >> This is useful for debugging, i.e. we can get an ASCII dump of the >> serialized data by using the "capnp decode" command. >> > > Agreed. > > > - The serialization code that I've written so far uses "auto" a lot, e.g. >> see here: >> >> https://github.com/dknez/libmesh/blob/rb_capnproto/src/reduced_basis/rb_data_serialization.C >> This is convenient since otherwise we would have to write out a lot of >> Cap'n Proto type names in the code. Using auto relies on C++11, but this >> seems OK to me since we have to have C++11 if we're using Cap'n Proto (and >> we would if-def out this code if we didn't configure with Cap'n Proto). >> Does this seem reasonable, or should I avoid "auto" everywhere? >> > > How much to use auto is definitely an area that is open to debate. In > this particular instance, I think it makes sense for you to write CP code > that "looks like" other CP code, so if that's how they write it in > examples, etc. I think it makes sense. Since auto is part of our > base-level test for C++11 compliance, so you are guaranteed to have it if > that configure test passes. > OK, sounds good. I agree that it's good to include idiomatic CP code in libMesh, and as you can see here <https://capnproto.org/cxx.html>, idiomatic CP code uses auto (e.g. see the comment "// Type shown for explanation purposes; normally you'd use auto.") As a result, I'll keep using auto in my branch. If there are any issues with this, then we can review it when I make the PR. Thanks, David
------------------------------------------------------------------------------
_______________________________________________ Libmesh-devel mailing list [email protected] https://lists.sourceforge.net/lists/listinfo/libmesh-devel
