Hi, Thanks for this! You've saved me much head-scratching -- it didn't occur to me to check if the PackedMessageReader was borrowing or taking ownership of the input stream.
I've create a custom reader class that holds both an ArrayInputStream and a PackedMessageReader and this seems to be working perfectly. Cheers, Jonathan. On Thursday, April 6, 2023 at 9:31:43 PM UTC+1 [email protected] wrote: > Hi Jonathan, > > I think the problem is with this function: > > std::shared_ptr<capnp::MessageReader> uda_capnp_deserialise(const char* > bytes, size_t size) > { > // ArrayPtr requires non-const ptr, but we are only using this to read > from the bytes array > > kj::ArrayPtr<kj::byte> > buffer(reinterpret_cast<kj::byte*>(const_cast<char*>(bytes)), size); > kj::ArrayInputStream in(buffer); > > return std::make_shared<capnp::PackedMessageReader>(in); > } > > Here, you are returning a PackedMessageReader that is wrapping an > ArrayInputStream, but the ArrayInputStream is allocated on the stack, so is > no longer valid once the function returns. The stream needs to remain valid > until the PackedMessageReader is destroyed. Cap'n Proto can actually lazily > read segments of the message from the stream; it doesn't read the entire > thing in the constructor. > > -Kenton > > On Thu, Apr 6, 2023 at 3:26 PM Jonathan Hollocombe <[email protected]> > wrote: > >> Hi, >> >> I am using Cap'n Proto to serialise a data tree which is then sent as a >> blob to a client to be unpacked. I was using something similar to: >> >> kj::VectorOutputStream out; >> capnp::writePackedMessage(out, builder); >> auto arr = out.getArray(); >> char* buffer = (char*)malloc(arr.size()); >> memcpy(buffer, arr.begin(), arr.size()); >> >> and >> >> kj::ArrayPtr<kj::byte> >> buffer(reinterpret_cast<kj::byte*>(const_cast<char*>(bytes)), >> size); >> kj::ArrayInputStream in(buffer); >> capnp::PackedMessageReader reader{in}; >> >> This is working for small tree sizes but segfaults when I get to larger >> number of elements. >> >> I've found a solution that works by switching to messageToFlatArray and >> FlatArrayMessageReader but this dramatically increases the amount of data >> that needs to be sent, especially for small trees. >> >> I've also found that writing the buffer to a file and then using >> PackedFdMessageReader also works but i'd rather avoid creating files >> unnessarily. >> >> Is there a way to use the PackedMessageReader with a byte buffer? >> >> Cheers, >> Jonathan. >> >> p.s. my example code can be found at: >> https://github.com/jholloc/capnp_test >> >> >> >> -- >> You received this message because you are subscribed to the Google Groups >> "Cap'n Proto" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to [email protected]. >> To view this discussion on the web visit >> https://groups.google.com/d/msgid/capnproto/ae6cafe1-f6f8-4eca-bab2-9ea3e73c3c8fn%40googlegroups.com >> >> <https://groups.google.com/d/msgid/capnproto/ae6cafe1-f6f8-4eca-bab2-9ea3e73c3c8fn%40googlegroups.com?utm_medium=email&utm_source=footer> >> . >> > -- You received this message because you are subscribed to the Google Groups "Cap'n Proto" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/capnproto/0c469ce7-aea8-43b8-a5c4-1001aa1b5b4fn%40googlegroups.com.
