Hi All-
I've finally had time this afternoon to work on this and got it working in
a way I find satisfactory.
auto capnpMsgPointer = context.getParams().getMsg();
KJ_REQUIRE(capnpMsgPointer.isStruct(), "Only messages can go in the
'msg' parameter");
auto msgSize = capnpMsgPointer.targetSize();
auto builder = make_unique<MallocMessageBuilder>(msgSize.wordCount + 1,
AllocationStrategy::FIXED_SIZE);
builder->setRoot(capnpMsgPointer);
auto segments = builder->getSegmentsForOutput();
KJ_ASSERT(segments.size() == 1);
auto fstSegmentData = segments[0].asBytes();
and on the way out (this is bidirectional):
auto segment = kj::ArrayPtr<word>((word*)blob->data(), blob->size()
/ 8).asConst();
auto segments = kj::heapArray({segment});
auto msgReader = make_unique<SegmentArrayMessageReader>(segments);
context.getResults().getResp().set(msgReader->getRoot<AnyPointer
>());
wherein 'blob' is a std::vector<uint8_t> which is semantically guarteed to
be 0 mod 8 in length.
This works in a loopback test!
Thanks,
John
On Monday, May 18, 2020 at 8:23:42 AM UTC-7, Kenton Varda wrote:
>
> Hi John,
>
> You have to copy into a new message, as you have done.
>
> The reason is, the message tree pointed to by the AnyPointer is not
> necessarily contiguous in memory. Unlike most other serialization out
> there, Cap'n Proto objects point to their child objects using pointers that
> can cross over other object in between. (Most serializations use nested
> encoding instead, but this doesn't work well with zero-copy.) Generally,
> objects will be ordered in memory in the order in which they were
> allocated. Typically programs writing a message will work on one part of
> the tree at a time, in which case you'll get a depth-first ordering, but
> that's not guaranteed. If a program switches back and forth between
> building different parts of the message, the objects will end up
> interleaved in memory.
>
> So the only way to get an object tree into contiguous memory is to copy it
> into a new arena -- i.e. a new MessageBuilder.
>
> You could, however, avoid the second copy implied by
> `messageToFlatArray()`. Instead, before making the copy, use
> `getMsg().targetSize().wordCount` to see how much memory the message tree
> takes. Add 1 word for the root pointer. Then pass this to
> MallocMessageBuilder's constructor as the first segment size. Now copy the
> message. If you then call messageBuilder.getSegmentsForOutput(), you should
> find there is only one segment, containing the root pointer followed by the
> content. So now you can use that data without making an extra copy...
>
> -Kenton
>
> On Fri, May 15, 2020 at 4:27 AM John Demme <[email protected]
> <javascript:>> wrote:
>
>> I've been playing with this a bit and the closest I've been able to get
>> is constructing a new message with the AnyPointer at the root. That,
>> however, prepends a struct pointer to the whole thing. I can just chop it
>> off, but I'm thinking there must be a better way.
>>
>> auto msg = context.getParams().getMsg(); // AnyPointer
>> MallocMessageBuilder messageBuilder;
>> messageBuilder.setRoot(msg);
>> auto flatWordsArray = messageToFlatArray(messageBuilder);
>> auto byteArray = flatWordsArray.asBytes();
>>
>>
>> ~John
>>
>>
>> On Thursday, May 14, 2020 at 11:30:38 PM UTC-7, John Demme wrote:
>>>
>>> Hi-
>>>
>>> In the C++ API, is there an (easy) way to get the binary encoded message
>>> of an AnyPointer field? Or a generic field? (If I understand, they're
>>> encoded the same under the hood.) I don't think anyPointer->getAs<Data>()
>>> would work, would it?
>>>
>>> The background here is that I'm trying to write a generic proxy for
>>> CapnProto messages using CapnProto RPC. Since I'm not re-compiling the
>>> proxy itself, it cannot know the possible types for AnyPointer. Since
>>> messages don't need any schema data to be traversed (though not
>>> interpreted) correctly, this *should *be possible.
>>>
>>> To be more concrete:
>>> interface ProxyChannel {
>>> send @0 (msg :AnyPointer);
>>> recv @1 () -> (respMsg :AnyPointer);
>>> }
>>>
>>> I'm using :Data instead of :AnyPointer now, but that puts a unnecessary
>>> burden on the clients.
>>>
>>> ~John
>>>
>> --
>> You received this message because you are subscribed to the Google Groups
>> "Cap'n Proto" group.
>> To unsubscribe from this group and stop receiving emails from it, send an
>> email to [email protected] <javascript:>.
>> To view this discussion on the web visit
>> https://groups.google.com/d/msgid/capnproto/7f1b3509-0bac-4854-8256-d85a718031d7%40googlegroups.com
>>
>> <https://groups.google.com/d/msgid/capnproto/7f1b3509-0bac-4854-8256-d85a718031d7%40googlegroups.com?utm_medium=email&utm_source=footer>
>> .
>>
>
--
You received this message because you are subscribed to the Google Groups
"Cap'n Proto" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
To view this discussion on the web visit
https://groups.google.com/d/msgid/capnproto/8077c456-117b-4d8a-95c7-7fbb075c20c7%40googlegroups.com.
