Re: [capnproto] Clean Disconnect Procedure
Thank you for the info, Kenton. I've been looking deeply at the design
of the RpcSystem, and I have a couple thoughts/questions:
1. I would like to add a new RPC Message in rpc.capnp:
goodbye @14 :Void;
This message indicates to the recipient that the sender has nothing
more to say, and that it should stop read()ing the socket. In other
words, upon receipt of rpc::Message::GOODBYE, messageLoop() ends
gracefully (no exceptions are thrown). The sender then shuts down its
write() side of the connection, causing a TCP FIN to be delivered to
the recipient. Because there is no read() in progress, an exception shall
not be thrown.
The recipient performs whatever cleanup is necessary and sends a
reciprocal GOODBYE, causing the same logic described above to be
invoked on the other end.
Do you think this is a good solution?
2. In your opinion, what is the best way to expose this graceful
disconnect functionality to applications? I considered modifying the
signature of the BootstrapFactory's createFor method in this manner:
Before:
capnp::Capability::Client createFor(VatId::Reader clientId)
After:
capnp::Capability::Client createFor(VatId::Reader clientId,
kj::Own> shutdown)
The PromiseFulfiller can then be passed to the constructor of the Server:
class AdderImpl final: public Adder::Server {
public:
AdderImpl(kj::Own> shutdown) :
shutdown(kj::mv(shutdown)) {}
kj::Promise add(AddContext context) override {
auto params = context.getParams();
context.getResults().setValue(params.getLeft() + params.getRight());
return kj::READY_NOW;
}
kj::Promise cleanupGracefully(CleanupGracefullyContext context)
override {
this->shutdown->fulfill();
}
private:
kj::Own> shutdown;
};
Another approach could be to add a shutdown() method to FooContext.
On the client side, perhaps it is best to simply allow the rpcSystem to
fall out of scope, at which point the destructors can invoke the
necessary machinery to send the GOODBYE and FIN the TCP stream.
What do you think? Are these approaches going to lead to a leaky
abstraction? Do you know of an elegant way to design this?
Regards,
Alex
On Wed, 27 Sep 2023 14:07:07 -0500
"'Kenton Varda' via Cap'n Proto" wrote:
> (Happy to accept a PR. The relevant code is in `messageLoop()` and
> `RpcConnectionState::disconnect()` in `rpc.c++`.)
>
> On Wed, Sep 27, 2023 at 2:05 PM Kenton Varda
> wrote:
>
> > Indeed, there isn't really a clean shutdown mechanism right now. I
> > guess it hasn't come up as a priority because in most use cases we
> > just haven't really cared if there's a TCP RST triggered under the
> > hood... since we're already killing the connection, we ignore that
> > error anyway.
> >
> > I suppose what we should do is, in the case that we receive a clean
> > EOF, inhibit the sending of an abort message back, just send EOF
> > back.
> >
> > -Kenton
> >
> > On Thu, Sep 21, 2023 at 6:09 PM 'Alex' via Cap'n Proto <
> > capnproto@googlegroups.com> wrote:
> >
> >> Hi all,
> >>
> >> I am designing an application (in C++) where, upon invocation of a
> >> particular RPC call, both the server and the client agree to
> >> cleanly disconnect from one another. By "cleanly", I mean that
> >> both the server and the client send a TCP FIN/ACK and nothing more
> >> (e.g. no RSTs). Unfortunately, in the current design the receipt
> >> of a FIN will cause AsyncIoMessageStream::tryReadMessage() to
> >> abort, whereupon it will throw KJ_EXCEPTION(DISCONNECTED, "Peer
> >> disconnected.")[0]. This exception is eventually written to the
> >> client socket, and if the client is already gone, there will be
> >> one or more RSTs in response:
> >>
> >> C -> S: "Goodbye" (RPC call)
> >> C -> S: "I have nothing more to say" (TCP FIN)
> >>
> >> (the client does not expect the server to say anything more and
> >> closes the socket)
> >>
> >> S -> C: "Exception! You disconnected from me" (RPC message)
> >> C -> S: "Error: Connection reset by peer" (TCP RST)
> >>
> >> Given that both the server and client have agreed to shut down the
> >> connection, this is not an exceptional circumstance. Therefore, an
> >> exception should not be thrown.
> >>
> >> Unfortunately, there does not seem to be a way to indicate to the
> >> RpcSystem that the DISCONNECTED exception ought to be suppressed.
> >> Is there something I am missing? I appreciate any assistance.
> >>
> >> Regards,
> >> Alex
> >>
> >> [0]
> >> https://github.com/capnproto/capnproto/blob/761aeb17563a59f43b3fe9bae93df83c6bd57d06/c%2B%2B/src/capnp/rpc.c%2B%2B#L2775
> >>
> >> --
> >> You received this message because you are subscribed to the Google
> >> Groups "Cap'n Proto" group.
> >> To unsubscribe from this group and stop receiving emails from it,
> >> send an email to capnproto+unsubscr...@googlegroups.com.
> >> To view this discussion on the web visit
> >> https://groups.google.com/d/msgid/capnproto/20230921190853.115b911d%40centromere.net
> >> .
> >>
> >
>
--
You
Re: [capnproto] Clean Disconnect Procedure
(Happy to accept a PR. The relevant code is in `messageLoop()` and `RpcConnectionState::disconnect()` in `rpc.c++`.) On Wed, Sep 27, 2023 at 2:05 PM Kenton Varda wrote: > Indeed, there isn't really a clean shutdown mechanism right now. I guess > it hasn't come up as a priority because in most use cases we just haven't > really cared if there's a TCP RST triggered under the hood... since we're > already killing the connection, we ignore that error anyway. > > I suppose what we should do is, in the case that we receive a clean EOF, > inhibit the sending of an abort message back, just send EOF back. > > -Kenton > > On Thu, Sep 21, 2023 at 6:09 PM 'Alex' via Cap'n Proto < > capnproto@googlegroups.com> wrote: > >> Hi all, >> >> I am designing an application (in C++) where, upon invocation of a >> particular RPC call, both the server and the client agree to cleanly >> disconnect from one another. By "cleanly", I mean that both the server >> and the client send a TCP FIN/ACK and nothing more (e.g. no RSTs). >> Unfortunately, in the current design the receipt of a FIN will cause >> AsyncIoMessageStream::tryReadMessage() to abort, whereupon it will >> throw KJ_EXCEPTION(DISCONNECTED, "Peer disconnected.")[0]. This >> exception is eventually written to the client socket, and if the client >> is already gone, there will be one or more RSTs in response: >> >> C -> S: "Goodbye" (RPC call) >> C -> S: "I have nothing more to say" (TCP FIN) >> >> (the client does not expect the server to say anything more and closes >> the socket) >> >> S -> C: "Exception! You disconnected from me" (RPC message) >> C -> S: "Error: Connection reset by peer" (TCP RST) >> >> Given that both the server and client have agreed to shut down the >> connection, this is not an exceptional circumstance. Therefore, an >> exception should not be thrown. >> >> Unfortunately, there does not seem to be a way to indicate to the >> RpcSystem that the DISCONNECTED exception ought to be suppressed. Is >> there something I am missing? I appreciate any assistance. >> >> Regards, >> Alex >> >> [0] >> https://github.com/capnproto/capnproto/blob/761aeb17563a59f43b3fe9bae93df83c6bd57d06/c%2B%2B/src/capnp/rpc.c%2B%2B#L2775 >> >> -- >> You received this message because you are subscribed to the Google Groups >> "Cap'n Proto" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to capnproto+unsubscr...@googlegroups.com. >> To view this discussion on the web visit >> https://groups.google.com/d/msgid/capnproto/20230921190853.115b911d%40centromere.net >> . >> > -- You received this message because you are subscribed to the Google Groups "Cap'n Proto" group. To unsubscribe from this group and stop receiving emails from it, send an email to capnproto+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/capnproto/CAJouXQkvmFfnXVLR6M8itZeog4KrXLFXmaFiQw8ZxSxOfOyFMw%40mail.gmail.com.
Re: [capnproto] Clean Disconnect Procedure
Indeed, there isn't really a clean shutdown mechanism right now. I guess it hasn't come up as a priority because in most use cases we just haven't really cared if there's a TCP RST triggered under the hood... since we're already killing the connection, we ignore that error anyway. I suppose what we should do is, in the case that we receive a clean EOF, inhibit the sending of an abort message back, just send EOF back. -Kenton On Thu, Sep 21, 2023 at 6:09 PM 'Alex' via Cap'n Proto < capnproto@googlegroups.com> wrote: > Hi all, > > I am designing an application (in C++) where, upon invocation of a > particular RPC call, both the server and the client agree to cleanly > disconnect from one another. By "cleanly", I mean that both the server > and the client send a TCP FIN/ACK and nothing more (e.g. no RSTs). > Unfortunately, in the current design the receipt of a FIN will cause > AsyncIoMessageStream::tryReadMessage() to abort, whereupon it will > throw KJ_EXCEPTION(DISCONNECTED, "Peer disconnected.")[0]. This > exception is eventually written to the client socket, and if the client > is already gone, there will be one or more RSTs in response: > > C -> S: "Goodbye" (RPC call) > C -> S: "I have nothing more to say" (TCP FIN) > > (the client does not expect the server to say anything more and closes > the socket) > > S -> C: "Exception! You disconnected from me" (RPC message) > C -> S: "Error: Connection reset by peer" (TCP RST) > > Given that both the server and client have agreed to shut down the > connection, this is not an exceptional circumstance. Therefore, an > exception should not be thrown. > > Unfortunately, there does not seem to be a way to indicate to the > RpcSystem that the DISCONNECTED exception ought to be suppressed. Is > there something I am missing? I appreciate any assistance. > > Regards, > Alex > > [0] > https://github.com/capnproto/capnproto/blob/761aeb17563a59f43b3fe9bae93df83c6bd57d06/c%2B%2B/src/capnp/rpc.c%2B%2B#L2775 > > -- > You received this message because you are subscribed to the Google Groups > "Cap'n Proto" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to capnproto+unsubscr...@googlegroups.com. > To view this discussion on the web visit > https://groups.google.com/d/msgid/capnproto/20230921190853.115b911d%40centromere.net > . > -- You received this message because you are subscribed to the Google Groups "Cap'n Proto" group. To unsubscribe from this group and stop receiving emails from it, send an email to capnproto+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/capnproto/CAJouXQncgKYxvr_LUj4F2dJJRjDLcS%2BPPLYLYKhPkvkF0XOp3g%40mail.gmail.com.
