Re: [capnproto] Writing tests for RPC-based code?
Hi Jens, Sorry for the slow reply. Usually when I write tests, I don't actually use the RPC system, but rather have the client call the server directly running in the same thread. Remember that you can implicitly turn an `Own` into a `MyInterface::Client`, and then call it like a client: MyInterface::Client client = kj::heap(); In theory, whether or not you have an actual RPC connection in between should be transparent. -Kenton On Mon, Feb 21, 2022 at 11:47 AM Jens Alfke wrote: > Does anyone have advice on writing C++ automated tests for RPC-based code? > (FWIW, I use the Catch2 unit-test framework, but I don’t restrict it to > true ‘unit’ tests.) Say I’ve got an interface file, a class `Client` built > atop classes generated from that interface, and a class `Server` that > accepts TCP connections from a `Client`. I want to write a test that > creates a Client and Server, connects them to each other, and then invokes > some operations on the Client and asserts the correct results. > > The first step of course is creating a connection. It looks like the > simplest supported mechanism is to open a POSIX pipe and then create kj > AsyncStreams on its two file descriptors. That means I’ll need an > alternative Client constructor that takes a file descriptor or stream > instead of a hostname/port… > > I’d love to take a look at anyone’s test code that does stuff like this. > > —Jens > > -- > You received this message because you are subscribed to the Google Groups > "Cap'n Proto" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to capnproto+unsubscr...@googlegroups.com. > To view this discussion on the web visit > https://groups.google.com/d/msgid/capnproto/CED5FAAE-EC6B-47E7-BDE3-58F3C1DD001F%40mooseyard.com > . > -- You received this message because you are subscribed to the Google Groups "Cap'n Proto" group. To unsubscribe from this group and stop receiving emails from it, send an email to capnproto+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/capnproto/CAJouXQ%3DgXqjV4rV6-eX28aZqL%2BOuvGOu5hnc54vsVTq5_-Z%2BGA%40mail.gmail.com.
Re: [capnproto] Can I do better?
Hi Jitesh,
Yes, this is possible, but a bit harder.
Instead of using MallocMessageBuilder, you would need to define your own
subclass of capnp::MessageBuilder that returns the buffers you want to use.
You will also need to keep track of framing on your own -- that is, keep
track of the number of segments and their sizes. Normally Cap'n Proto will
write a prefix on the message called the "segment table" which specifies
the length of each segment. But, the size of that table depends on the
number of segments, which normally isn't known in advance, so you wouldn't
know how much space to leave at the start of your message for it. That
said, if you are sure your message will always fit in the first segment,
then the segment table is not that useful anyway.
Also, if you're manually managing segments, then on the receiving end
you'll need to use capnp::SegmentArrayMessageReader, or write a custom
MessageReader subclass.
-Kenton
On Sat, Feb 26, 2022 at 8:09 AM Jitesh Khandelwal
wrote:
> Thanks Kenton, this is working well. Now the allocation in
> messageToFlatArray is not happening.
>
> Now, is it possible to build the message in the buffer directly ?
> Currently, I'm building in arena and then calling writeMessage to write
> into the buffer.
>
> --
> auto builder = MallocMessageBuilder(*arena*);
> ...
> ...
> auto message = arrayPtr(reinterpret_cast(*buffer*), N);
>
>
> auto aos = kj::ArrayOutputStream(message);
>
>
> capnp::writeMessage(aos, builder);
>
> --
> On Saturday, 26 February 2022 at 07:07:44 UTC+5:30 ken...@cloudflare.com
> wrote:
>
>> Hi Jitesh,
>>
>> Yes, you can avoid the allocation as follows:
>>
>> 1. Construct a kj::ArrayOutputStream that uses your destination buffer as
>> the output.
>> 2. Use capnp::writeMessage() to write the message to that stream.
>>
>> -Kenton
>>
>> On Fri, Feb 25, 2022 at 12:31 PM Jitesh Khandelwal
>> wrote:
>>
>>> I am trying to implement a middle layer which serialises some data from
>>> a data structure into a buffer, which is owned by and whose contents are
>>> written on the wire by an outer layer.
>>>
>>> Currently, my code looks as follows
>>>
>>>
>>> constexpr size_t N = 2048;
>>> char arena_buffer[N]{0};
>>> auto arena = arrayPtr(reinterpret_cast(arena_buffer), N /
>>> sizeof(word));
>>>
>>> Encode (some_data_structure, arena, buffer)
>>> {
>>> auto builder = MallocMessageBuilder(arena);
>>>
>>> // Step 1. set fields from some_data_structure using builder into
>>> arena
>>>
>>>
>>> // Step 2. convert to flat array (this allocates another array)
>>> auto message = messageToFlatArray(builder);
>>> auto bytes = message.asBytes();
>>>
>>> // Step 3. copy to destination
>>> memcpy(buffer, bytes.begin(), bytes.size());
>>> }
>>>
>>>
>>>
>>> My question is, Can I do better?
>>>
>>> My understanding is that the extra allocation in Step 2, is because of
>>> the segment framing protocol ?
>>>
>>> Can I bypass it somehow, as I am sure that the arena is big enough for
>>> my message and there'll be only 1 segment ?
>>>
>>> And if yes, then may be I can use the buffer as the arena, as in build
>>> directly into the output buffer?
>>>
>>>
>>>
>>> Thanks,
>>> Jitesh
>>>
>>> --
>>> You received this message because you are subscribed to the Google
>>> Groups "Cap'n Proto" group.
>>> To unsubscribe from this group and stop receiving emails from it, send
>>> an email to capnproto+...@googlegroups.com.
>>> To view this discussion on the web visit
>>> https://groups.google.com/d/msgid/capnproto/a6378e3d-fd2e-4009-8cf4-595011170dc9n%40googlegroups.com
>>>
>>> .
>>>
>> --
> You received this message because you are subscribed to the Google Groups
> "Cap'n Proto" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to capnproto+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/capnproto/b09290d5-e2d3-4711-b309-9bcdaaade8e5n%40googlegroups.com
>
> .
>
--
You received this message because you are subscribed to the Google Groups
"Cap'n Proto" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to
Re: [capnproto] Having trouble with Github CI
Hi Jens, Do you get that same error in both Ubuntu and MacOS actions? It would seem pretty surprising if both platforms were getting version-skewed installations. Does your source code include any checked-in generated code (.capnp.h files)? If so then the problem is presumably that the code was generated with the capnp on your machine, which is a different version from what the action machines use. -Kenton On Sat, Feb 26, 2022 at 3:53 PM Jens Alfke wrote: > I’m trying to get my (C++) project to build in Github Actions, but getting > compile errors because apparently there’s a version mismatch between the > Cap’n Proto headers and tool. > > I don’t build Cap’n Proto as part of the project; instead I use a package > manager to install it. My CMakeLists.txt has a step to compile my interface: > find_package(CapnProto CONFIG REQUIRED) > capnp_generate_cpp(myCapnPSources myCapnPHeaders src/interface.capnp) > > This works fine in local builds, both macOS and Ubuntu. > > In my Github Actions yaml file, I added a build step that uses the > get-package action: > - name: Install Capn Proto > uses: mstksg/get-package@v1 > with: > brew: capnp > apt-get: capnproto libcapnp-dev > > Unfortunately the C++ compile then fails with the error: "Version mismatch > between generated code and library headers. You must use the same version > of the Cap'n Proto compiler and library.” > > I assume this means that on the Github runner the `capnp` tool is from a > different version than the headers. I don’t know why that would be, when I > had apt install both packages. (And why would there be a second > installation of Cap’n Proto at all?) > > Has anyone successfully gotten Github Actions to build C++ based Cap’n > Proto projects? > > —Jens > > -- > You received this message because you are subscribed to the Google Groups > "Cap'n Proto" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to capnproto+unsubscr...@googlegroups.com. > To view this discussion on the web visit > https://groups.google.com/d/msgid/capnproto/A1E39982-9835-4E45-B9E1-397CC5B286FF%40mooseyard.com > . > -- You received this message because you are subscribed to the Google Groups "Cap'n Proto" group. To unsubscribe from this group and stop receiving emails from it, send an email to capnproto+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/capnproto/CAJouXQm-XT%2Bsa8Agtgh-32UboA0RSM7%2BfZ9QTB7q3X-U_OLDYA%40mail.gmail.com.
[capnproto] failed: expected ref-kind() == WirePointer::LIST [0 == 1];
I am seeing the following message on Linux upon calling
StreamFdMessageReader on the receiving end of a connection. I did have this
working until I put messages inside a union as a wrapper.
Exception in setting up sender: capnp/layout.c++:2438: failed: expected
ref-kind() == WirePointer::LIST [0 == 1]; Message contains non-list pointer
where text was expected.
I'm guessing something is wrong on the sender as the receiving seems to
only report the problem as soon as it reads from the fd. I am mainly
confused about the WirePointer::LIST [0 == 1] part. Can anyone add
clarification?
struct MyObjA {
fieldA @0 :SomeType;
fieldB @1 :UInt16;
fieldC @2 :Text;
}
struct MyObjB {
fieldA @0 :SomeType;
fieldB @1 :UInt16;
fieldC @2 :Text;
}
struct MyWrapper {
union {
objA @0 :MyObjA;
objB @1 :MyObjB;
}
}
auto w = builder->initRoot();
auto obj = w->initMyObjA();
obj.getMyObjA.setFieldA("a");
obj.getMyObjA.setFieldB("b");
writeMessageToFd(fd, builder);
--
You received this message because you are subscribed to the Google Groups
"Cap'n Proto" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to capnproto+unsubscr...@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/capnproto/8f592f53-8f03-42f2-831a-c8840f5150d1n%40googlegroups.com.
