Awesome. Thanks for taking the time to code that up. I plan on trying to implement this today/tomorrow. I will report back.
On Thu, Feb 8, 2018 at 3:13 PM, Carl Mastrangelo <[email protected]> wrote: > In go (and I may be mistaken), I believe authentication is pulled out of > the headers and put into the Context. When you get the initial metadata > on the server side, you can look in the Context for the auth. > > To issue commands to the particular client, the server would need to > maintain a map of client id to stream. Your server goroutine would select > on a channel for commands from your application, and then call > Stream.SendMsg. A rough sketch of the idea would be: > > var clientmap map[string]chan interface{} > > streamhandler := func(srv interface{}, stream ServerStream) error { > ctxx:= stream.Context() > auth := context.Value(authKey{}) > ch := make(chan interface{}) > clientmap[auth.ID()] = ch > defer delete(clientmap, auth.ID()) > for { > select { > case d := <- ch: > stream.WriteMsg(d) > case <-ctx.Done(): > break > } > } > return nil > } > > > When any goroutine wants to contact a particular client, it attempts to > look up the client by its ID in the clientmap and then send a message to > it. Right now this is very simple and doesn't wait for a response, but > you can add that in. You'll need to fix the threadsafety too, and error > handling, but this is the rough idea. > > > > > > > On Thu, Feb 8, 2018 at 2:40 PM John Pearson <[email protected]> > wrote: > >> I've been checking out the interceptor pattern. Tokens, certificates, >> metadata make sense to me. I'm confused about "the server decodes the >> token and associates the stream with the client". Once a client ID info in >> any way, it's unclear to me how the server associates a stream with a >> client and then when needed invokes a command on that particular stream. >> >> I'm using go-lang. >> >> On Thu, Feb 8, 2018 at 2:35 PM, Carl Mastrangelo <[email protected]> >> wrote: >> >>> The client would need to self identify by setting a header when it >>> initiates the RPC. If you are using auth tokens, you could maybe put the >>> identity in the token. When the client makes an RPC, the server decodes >>> the token and associates the stream with the client. >>> >>> Another option (as mentioned in my first post) would be to use client >>> side certificates. When the server gets the RPC, it can determine the >>> security information from the connection and associate it with the stream. >>> >>> Lastly (and probably easiest) is to add the identity in a custom header >>> fields (also called "metadata"). You can do this directly with some gRPC >>> APIs and indirectly by making a client side RPC interceptor. I am more >>> familiar with Java, which favors the interceptor approach. It will depend >>> on what language of gRPC library you are using. >>> >>> >>> I would personally suggest the latter, since it is the most flexible, >>> and works well with proxies. It also and expandable, so you can put more >>> info in the headers to be identified by. >>> >>> >>> >>> On Thu, Feb 8, 2018 at 2:27 PM John Pearson <[email protected]> >>> wrote: >>> >>>> Makes sense, thanks for the explanation. I see tunneling is being >>>> worked on: https://github.com/grpc/grpc/issues/14101 >>>> >>>> Question: When I have 15 clients all connected in a bidirectional >>>> stream, how does the Server send a message to a particular client? For >>>> example in my case: Server needs to tell a client12 to run a particular >>>> command, how to identify the right connection/channel Server side? >>>> >>>> I've tried searching for this: >>>> - https://groups.google.com/forum/#!searchin/grpc-io/ >>>> identify$20stream/grpc-io/MAjt9cE_uCU/VMAjo_KiAQAJ >>>> - https://groups.google.com/forum/#!searchin/grpc-io/ >>>> identify$20stream/grpc-io/z6aEEiaeopM/AVSZRFPTCAAJ >>>> >>>> >>>> >>>> >>>> On Wed, Feb 7, 2018 at 1:18 PM, Carl Mastrangelo <[email protected]> >>>> wrote: >>>> >>>>> Your flow is correct. Is it RPC? I guess that's a more >>>>> philosophical, but I don't think there is anything wrong with it. >>>>> Streaming RPCs are already kind of weird anyway. >>>>> >>>>> A more pure solution would be something akin to tunneling, where a >>>>> client connects to a server, but then runs a gRPC server _on top of_ the >>>>> client connection allowing the actual server to make client-like requests. >>>>> This has been discussed before, but wasn't implemented due to time >>>>> constraints. gRPC is fairly young so more advanced use cases like this >>>>> don't have solutions yet. >>>>> >>>>> Note that even though the flow is reversed, you still get a lot of the >>>>> benefits of gRPC. The real client still can do intelligent fail over >>>>> across all your servers. You get all the stats and trace capabilities. >>>>> Lastly, gRPC is pretty fast! >>>>> >>>>> >>>>> On Wed, Feb 7, 2018 at 12:46 PM John Pearson <[email protected]> >>>>> wrote: >>>>> >>>>>> Got it. So this isn't considered bad form for RPC? Technically it >>>>>> isn't RPC? In my case: >>>>>> >>>>>> - Server implements rpc Link (stream UpLink) returns (stream >>>>>> DownLink) {} >>>>>> - Client initiates connection >>>>>> - Client waits for messages to come down from the server to be >>>>>> executed >>>>>> - Clients parses the strings/bytes streamed down and uses a case >>>>>> statement to execute commands >>>>>> >>>>>> On Wed, Feb 7, 2018 at 12:04 PM, Carl Mastrangelo <[email protected] >>>>>> > wrote: >>>>>> >>>>>>> Yes, that is correct. (It's bidirectional because the word "stream" >>>>>>> is in both the request (UpLink) and response (DownLink).) >>>>>>> >>>>>>> I mistakenly thought this was another thread (which I replied to >>>>>>> yesterday) which had almost the same issue. Take a look at >>>>>>> https://groups.google.com/d/msg/grpc-io/G4eYs1zNMjE/Yh2WJS7TBwAJ >>>>>>> where I describe how to do what you want. >>>>>>> >>>>>>> >>>>>>> On Wed, Feb 7, 2018 at 11:38 AM John Pearson < >>>>>>> [email protected]> wrote: >>>>>>> >>>>>>>> I'm unclear on what the last part means: "That's why that issue, >>>>>>>> (and my post), suggest treating the message as inverted requests and >>>>>>>> responses." >>>>>>>> >>>>>>>> Do you mean rpc Link (stream UpLink) returns (stream DownLink) {} ? >>>>>>>> With a bidirectional stream? >>>>>>>> >>>>>>>> Thanks >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> On Wed, Feb 7, 2018 at 11:27 AM, 'Carl Mastrangelo' via grpc.io < >>>>>>>> [email protected]> wrote: >>>>>>>> >>>>>>>>> It's unlikely that gRPC will be able to have servers initiate >>>>>>>>> connections / rpcs to clients. The initial headers need to be sent >>>>>>>>> by the >>>>>>>>> client to the server, and only the server can send trailers to >>>>>>>>> indicate the >>>>>>>>> RPC is done. But, after these two, the client and server are peers. >>>>>>>>> That's why that issue, (and my post), suggest treating the message as >>>>>>>>> inverted requests and responses. >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> On Monday, February 5, 2018 at 8:55:46 PM UTC-8, John Pearson >>>>>>>>> wrote: >>>>>>>>>> >>>>>>>>>> Question regarding bi-directional streaming: >>>>>>>>>> >>>>>>>>>> Is this issue(https://github.com/grpc/grpc-go/issues/484# >>>>>>>>>> issuecomment-288880402) solved because of bidirectional rpc? -- >>>>>>>>>> it is almost exactly the same as my use case. >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> On Mon, Feb 5, 2018 at 3:41 PM, 'Carl Mastrangelo' via grpc.io < >>>>>>>>>> [email protected]> wrote: >>>>>>>>>> >>>>>>>>>>> Sounds like you want to use a client streaming rpc. The client >>>>>>>>>>> will start an RPC, and periodically send messages to the server, >>>>>>>>>>> which will >>>>>>>>>>> never respond. This works well if the time between updates is small >>>>>>>>>>> perhaps no more than a minute apart. If it's longer, it would be >>>>>>>>>>> better >>>>>>>>>>> to just make it unary because the connection will be at risk of >>>>>>>>>>> being torn >>>>>>>>>>> down by intermediary things. >>>>>>>>>>> >>>>>>>>>>> If you trust the hardware you are running on, a common way of >>>>>>>>>>> authing would be client side certificates, but we leave the problem >>>>>>>>>>> of >>>>>>>>>>> certificate rotation and revocation up to you. In gRPC, auth is >>>>>>>>>>> more >>>>>>>>>>> commonly (always?) done at the RPC level. Each RPC would need an >>>>>>>>>>> auth >>>>>>>>>>> token, like a JWT. >>>>>>>>>>> >>>>>>>>>>> If you want a command and control style system, you'll need to >>>>>>>>>>> use a bidirectional streaming RPC. This would let the client >>>>>>>>>>> phone home, >>>>>>>>>>> and wait for messages to come down from the server to be executed. >>>>>>>>>>> The >>>>>>>>>>> client must initiate the connection and the RPC, but after that >>>>>>>>>>> each side >>>>>>>>>>> is a peer. >>>>>>>>>>> >>>>>>>>>>> There are examples on how to do each style of these commands in >>>>>>>>>>> the github repos. Look for "Route Guide" examples on how to do it >>>>>>>>>>> in each >>>>>>>>>>> language. >>>>>>>>>>> >>>>>>>>>>> On Sunday, February 4, 2018 at 5:37:40 PM UTC-8, >>>>>>>>>>> [email protected] wrote: >>>>>>>>>>>> >>>>>>>>>>>> I manage a fleet of linux devices and I need a way to send >>>>>>>>>>>> telemetry data(CPU, memory, drives, etc.) from devices to a >>>>>>>>>>>> centralized >>>>>>>>>>>> server and also send commands(linux service restarts for example) >>>>>>>>>>>> from >>>>>>>>>>>> server to devices to execute on the device. The 15 different >>>>>>>>>>>> linux devices are bare metal boxes in 15 different locations >>>>>>>>>>>> access to WAN >>>>>>>>>>>> with a dynamic IP address. In a year the number of devices could >>>>>>>>>>>> jump to >>>>>>>>>>>> 100. >>>>>>>>>>>> >>>>>>>>>>>> I'm looking for suggestions on: >>>>>>>>>>>> - how to gather telemetry data to send to grpc server >>>>>>>>>>>> - how authenticate and identify devices >>>>>>>>>>>> - how to run commands locally on each device and report back >>>>>>>>>>>> >>>>>>>>>>>> Thanks. >>>>>>>>>>>> >>>>>>>>>>> -- >>>>>>>>>>> You received this message because you are subscribed to a topic >>>>>>>>>>> in the Google Groups "grpc.io" group. >>>>>>>>>>> To unsubscribe from this topic, visit >>>>>>>>>>> https://groups.google.com/d/topic/grpc-io/YB0HErGYZPw/ >>>>>>>>>>> unsubscribe. >>>>>>>>>>> To unsubscribe from this group and all its topics, send an email >>>>>>>>>>> to [email protected]. >>>>>>>>>>> To post to this group, send email to [email protected]. >>>>>>>>>>> Visit this group at https://groups.google.com/group/grpc-io. >>>>>>>>>>> To view this discussion on the web visit >>>>>>>>>>> https://groups.google.com/d/msgid/grpc-io/66ed336c-6de9- >>>>>>>>>>> 4f80-b0e0-55b05338e45a%40googlegroups.com >>>>>>>>>>> <https://groups.google.com/d/msgid/grpc-io/66ed336c-6de9-4f80-b0e0-55b05338e45a%40googlegroups.com?utm_medium=email&utm_source=footer> >>>>>>>>>>> . >>>>>>>>>>> >>>>>>>>>>> For more options, visit https://groups.google.com/d/optout. >>>>>>>>>>> >>>>>>>>>> >>>>>>>>>> -- >>>>>>>>> You received this message because you are subscribed to a topic in >>>>>>>>> the Google Groups "grpc.io" group. >>>>>>>>> To unsubscribe from this topic, visit https://groups.google.com/d/ >>>>>>>>> topic/grpc-io/YB0HErGYZPw/unsubscribe. >>>>>>>>> To unsubscribe from this group and all its topics, send an email >>>>>>>>> to [email protected]. >>>>>>>>> To post to this group, send email to [email protected]. >>>>>>>>> Visit this group at https://groups.google.com/group/grpc-io. >>>>>>>>> To view this discussion on the web visit >>>>>>>>> https://groups.google.com/d/msgid/grpc-io/bc74b58e-8ae7- >>>>>>>>> 4538-9c95-e2fad7d66413%40googlegroups.com >>>>>>>>> <https://groups.google.com/d/msgid/grpc-io/bc74b58e-8ae7-4538-9c95-e2fad7d66413%40googlegroups.com?utm_medium=email&utm_source=footer> >>>>>>>>> . >>>>>>>>> >>>>>>>>> For more options, visit https://groups.google.com/d/optout. >>>>>>>>> >>>>>>>> >>>>>>>> >>>>>> >>>> >> -- You received this message because you are subscribed to the Google Groups "grpc.io" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To post to this group, send email to [email protected]. Visit this group at https://groups.google.com/group/grpc-io. To view this discussion on the web visit https://groups.google.com/d/msgid/grpc-io/CAKNtY_xrm-TFyYforCGGhQWmzz8YonpokZWiwy1DPwja%3Dby6%3Dw%40mail.gmail.com. For more options, visit https://groups.google.com/d/optout.
