Hi Topher, Unfortunately, Cap'n Proto is not well suited to representing mutable data structures in-memory. Generally, MessageReaders are read-only, and MessageBuilders are write-only. While technically you can modify builders over time, the memory allocation patterns this tends to lead to are not ideal. This is an unavoidable consequence of the memory allocation approach needed to support zero-copy serialization: we need to allocate memory in a contiguous space in order to be able to write it all out at once, but this means memory fragmentation is impossible to avoid when data changes shape over time.
This is covered a bit in the "Best practices" section of the docs here: https://capnproto.org/cxx.html#tips-and-best-practices A feature I've long wanted to add to Cap'n Proto is support for generating appropriate "plain old C++ structs" (POCS) that mirror the Cap'n Proto structs, with the ability to copy from a MesasgeReader into a POCS, and from POCS to a MessageBuailder. Using POCS would mean you have to perform at least one copy (not zero-copy anymore), but may be more convenient in some use cases, especially when the structure will be modified in-memory many times. -Kenton On Mon, Mar 29, 2021 at 6:36 PM Topher Cawlfield <[email protected]> wrote: > I'm new to Cap'nProto, but am trying to use it as a replacement for > structs in an existing project. I'm needing to serialize/deserialize > obviously, but also just access the data within various programs. I'm > having a very hard time keeping a MessageBuilder, Builder, and Reader in a > class. > > It would be nice if I could deserialize a CapnProto struct in one > method/function, modify it in another, read it in a third, and serialize in > a fourth. But I'm stuck turning a PackedFdMessadeReader into a > MessageBuilder (I don't mind any few memcpy's needed), and also creating a > Builder instance as a member variable. > > I'm attaching my closest attempt at this. But it fails to print out the > original data value -- I get a null string -- and it fails to write the > modified value out. > > Any suggestions? > > Topher > > mystruct.capnp: > @0xed859a09d409be91; > > struct MyStruct { > foo @0 :Text; > } > > main.cpp: > #include <string> > #include <iostream> > #include <capnp/message.h> > #include <capnp/serialize-packed.h> > #include "mystruct.capnp.h" > > class MyClass { > private: > capnp::MallocMessageBuilder m_message; // Or capnp::MessageBuilder > *m_message maybe > > public: > MyStruct::Builder m_ms; > > MyClass() : > m_message () > , m_ms (m_message.initRoot<MyStruct>()) // sometimes segfaults here but > absolutely required > { > m_ms = m_message.initRoot<MyStruct>(); // sometimes helps > } > > void deserializeFrom(std::string filename) { > int fd = 0; > ::capnp::PackedFdMessageReader msg(fd); > auto reader = msg.getRoot<MyStruct>(); > m_message.setRoot(reader); // This leads to future problems with m_ms > } > > void modify() { > m_ms.setFoo("oh joy"); > } > > void serializeTo(std::string filename) { > int fd = 1; > writePackedMessageToFd(fd, m_message); > } > }; > > int main() { > MyClass c; > c.deserializeFrom("some.capnp"); > std::cerr << "foo is " << c.m_ms.getFoo().cStr() << std::endl; // read > things occasionally > c.modify(); > c.serializeTo("next.capnp"); > } > > -- > You received this message because you are subscribed to the Google Groups > "Cap'n Proto" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [email protected]. > To view this discussion on the web visit > https://groups.google.com/d/msgid/capnproto/6ce2d01c-43d4-44a6-9d26-0b3147742f55n%40googlegroups.com > <https://groups.google.com/d/msgid/capnproto/6ce2d01c-43d4-44a6-9d26-0b3147742f55n%40googlegroups.com?utm_medium=email&utm_source=footer> > . > -- You received this message because you are subscribed to the Google Groups "Cap'n Proto" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/capnproto/CAJouXQmcwFSOMF%2B4CiJgGbXQaH4zHibSTmPM%2BAn_vwuFxtaT7Q%40mail.gmail.com.
