Yeah, C++ error messages are the worst. :( You kind of have to get used to knowing that "can't convert" errors are usually because you forgot to kj::mv...
-Kenton On Tue, Nov 30, 2021 at 10:08 AM pepijn de vos <[email protected]> wrote: > Ah I solved it... by not storing sim in a variable. Or probably kj::mv > would have done the job too. > It's kinda unfortunate the error you get just tells you it's the wrong > type, rather than.. hey you should move the thing. > Onwards!! > Pepijn > > On Tue, Nov 30, 2021 at 4:50 PM pepijn de vos <[email protected]> > wrote: > >> Hi Kenton, >> >> I got back to this problem and tried implementing the fork idea but ran >> into another silly problem where the C++ templates are smarter than me it >> seems. >> You recommended to not use the EzRPC server, but there is a great lack of >> examples of other ways to go about it. >> >> I found a Rust example that uses the low-level API but of course not >> exactly the C++ one: >> https://github.com/capnproto/capnproto-rust/blob/master/capnp-rpc/examples/calculator/server.rs >> I found some code that's copy-pasted all over using github search that >> seems to use the lower level API: >> https://github.com/Nickan/nickan.github.io/blob/b0eb31c33a3b6720606974f9576e663f3b7852ca/drawio/etc/sandstorm/server.c%2B%2B#L420-L422 >> >> However, I cannot quite get it to work. >> >> kj::AsyncIoContext ctx = kj::setupAsyncIo(); >> auto stream = ctx.lowLevelProvider->wrapSocketFd(clientFd); >> auto network = capnp::TwoPartyVatNetwork(*stream, capnp::rpc::twoparty:: >> Side::CLIENT); >> kj::Own<kj::Filesystem> fs = kj::newDiskFilesystem(); >> const kj::Directory &dir = fs->getCurrent(); >> >> auto sim = kj::heap<SimulatorImpl>(dir); >> auto rpc = capnp::makeRpcServer(network, sim); >> >> So I set up asyncio, wrap my unix socket FD into a stream, use that to >> create a network (client or server??) >> The problem is the call to makeRpcServer. SimulatorImpl is a ::Server >> subclass. >> This seems to match the random copy-pasted example, but gives compile >> errors that it can't convert sim to whatever is expected. >> I tried reading the source code, which tells me it expects a restorer or >> a bootstrap, but the restorer is deprecated, but also what EzRPC seems to >> be using. >> I tried looking at the tests but that appears to do the same thing, just >> pass it an owned instance of the implementation?? >> >> https://github.com/capnproto/capnproto/blob/e5af75ff0513e6d971fa7b48e8108427766e51f9/c%2B%2B/src/capnp/rpc-test.c%2B%2B#L1273-L1279 >> I am at a loss. >> >> Pepijn >> >> >> On Thu, May 6, 2021 at 8:22 PM Kenton Varda <[email protected]> >> wrote: >> >>> I don't recommend using EzRPC. That interface turns out to be too >>> restrictive. >>> >>> To set up a KJ event loop, use kj::setupAsyncIo(). >>> >>> Then to initiate or accept Cap'n Proto RPC connections, use >>> capnp::TwoPartyClient / capnp::TwoPartyServer. >>> >>> For FD passing, create a capnp server object that overrides the virtual >>> method `kj::Maybe<int> Capability::Server::getFd()` to return an FD number. >>> Then on the Client objects pointing to that server, you can call `getFd()` >>> to get that file descriptor -- even from another process, as long as the >>> connection between the two is a unix domain socket. >>> >>> -Kenton >>> >>> On Thu, May 6, 2021 at 1:07 PM pepijn de vos <[email protected]> >>> wrote: >>> >>>> Thanks for the suggestions. >>>> This is still for the simulation server, so each connection is pretty >>>> expensive anyway, and it's unlikely there will be more in flight than the >>>> machine has cores available. It might even make sense to have a pool as you >>>> suggest, and just reject connections if there are no cores available. >>>> Concern is that you need to supervise these pools in case they crash. With >>>> fork you can just launch a new process. Any pointers to useful functions >>>> for this kinds of hacks would be appreciated. I don't have a clue how I'd >>>> go about obtaining and sending file descriptors over RPC and making new >>>> servers out of them. I should probably start by studying EzRPC and >>>> kj::LowLevelAsyncIoProvider. >>>> >>>> Pepijn >>>> >>>> On Thu, May 6, 2021 at 7:48 PM Kenton Varda <[email protected]> >>>> wrote: >>>> >>>>> Ohhh, sorry, I misread. If you haven't started the KJ event loop until >>>>> after the fork, you are probably OK. >>>>> >>>>> You can definitely construct a KJ socket from a raw socket using >>>>> kj::LowLevelAsyncIoProvider. >>>>> >>>>> But if you're OK with the entire connection being handled in the >>>>> isolated process, there are some other options too: >>>>> - Have a pool of processes that are all waiting to accept from the >>>>> same file descriptor. Each connection will only be accepted by one of the >>>>> processes. Make sure that process doesn't accept a new connection until >>>>> the >>>>> old connection is done. >>>>> - Cap'n Proto supports sending file descriptors between processes in >>>>> RPC messages. So you could have one process that is accepting connections, >>>>> but each time it does, it sends the file descriptor to some other >>>>> processes >>>>> via an RPC. Then that process actually handles the connection. >>>>> >>>>> These options are both more complicated than the simple fork() >>>>> approach you mentioned, so aren't necessarily better. But fork() is pretty >>>>> expensive so it might turn out reusing processes has some performance >>>>> benefit. >>>>> >>>>> -Kenton >>>>> >>>>> On Thu, May 6, 2021 at 12:39 PM pepijn de vos <[email protected]> >>>>> wrote: >>>>> >>>>>> Hi Kenton, >>>>>> >>>>>> Agree forking an eventloop will lead to disaster, which is why I >>>>>> suggested having a blocking accept loop, and then fork and start the >>>>>> event >>>>>> loop in the new process. But I'm not sure if it's at all possible to >>>>>> convert a raw unix socket to a jk socket and then give it to an eventloop >>>>>> to handle the connection. >>>>>> >>>>>> RPC to a process certainly requires less hacks. >>>>>> >>>>>> Pepijn >>>>>> >>>>>> On Thu, May 6, 2021 at 7:35 PM Kenton Varda <[email protected]> >>>>>> wrote: >>>>>> >>>>>>> Hi Pepijn, >>>>>>> >>>>>>> Yeah, forking won't work well. I don't recommend that. One problem >>>>>>> is that if you fork() with an event loop running, both the parent and >>>>>>> child >>>>>>> processes will end up with the same epoll FD and so will receive each >>>>>>> other's events, which will likely lead to chaos. >>>>>>> >>>>>>> So I recommend spawning a subprocess and talking to it using RPC >>>>>>> over a unix socket. Yes, it adds some overhead, but really not that >>>>>>> much. >>>>>>> >>>>>>> -Kenton >>>>>>> >>>>>>> On Thu, May 6, 2021 at 2:39 AM [email protected] < >>>>>>> [email protected]> wrote: >>>>>>> >>>>>>>> Hey, >>>>>>>> >>>>>>>> I'm writing an API for some shared libraries that are written with >>>>>>>> a "binary" use in mind. >>>>>>>> They use a ton of global state and are not thread safe. >>>>>>>> Of course as a server, multiple users could try to use the server >>>>>>>> at the same time, these libraries can't deal with that. What's more >>>>>>>> they >>>>>>>> also like to segfault sometimes. >>>>>>>> >>>>>>>> So what I'd like to do is have these shared libraries completely in >>>>>>>> their own address space. >>>>>>>> What would be the best way to do that? >>>>>>>> >>>>>>>> One thing I thought could work is have a blocking accept loop, and >>>>>>>> then fork the handling of the connection. Not sure how this would work >>>>>>>> with >>>>>>>> kj. >>>>>>>> >>>>>>>> The other solution would be that you launch a process in the method >>>>>>>> handler that runs its own capnp server, and then proxy calls to it. >>>>>>>> Seems >>>>>>>> like it adds more overhead. >>>>>>>> >>>>>>>> Regards, >>>>>>>> Pepijn >>>>>>>> >>>>>>>> -- >>>>>>>> You received this message because you are subscribed to the Google >>>>>>>> Groups "Cap'n Proto" group. >>>>>>>> To unsubscribe from this group and stop receiving emails from it, >>>>>>>> send an email to [email protected]. >>>>>>>> To view this discussion on the web visit >>>>>>>> https://groups.google.com/d/msgid/capnproto/99fcf2cf-e6e4-4918-8734-22b7f85cfb85n%40googlegroups.com >>>>>>>> <https://groups.google.com/d/msgid/capnproto/99fcf2cf-e6e4-4918-8734-22b7f85cfb85n%40googlegroups.com?utm_medium=email&utm_source=footer> >>>>>>>> . >>>>>>>> >>>>>>> -- You received this message because you are subscribed to the Google Groups "Cap'n Proto" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/capnproto/CAJouXQmwXf51NRjT2pL0x4do%2B8Ur3C7jFiMURYsLmi8SqARkGA%40mail.gmail.com.
