Just in case somebody get into same issue and found this thread: I was able to connect all discovery requests from client by adding server interceptor where I marked all requests with client socket information and reuse it later in node hashing. Details can be found in this commit [1] in my test repo.
Once again thanks for help. L. [1] https://github.com/LesTR/xds-routing-test/commit/ebb1a00c39b9ab9db7868f1857ca5166d8e3748e On Monday, August 16, 2021 at 8:16:16 PM UTC+2 Lukáš Drbal wrote: > Ach, sorry, my bad. > Thanks. > > L. > > On Monday, August 16, 2021 at 7:57:28 PM UTC+2 Mark D. Roth wrote: > >> I am not familiar with java-control-plane, so I can't answer that. You >> might try asking in their developer community. >> >> On Mon, Aug 16, 2021 at 10:55 AM Lukáš Drbal <[email protected]> wrote: >> >>> Hello Mark, >>> >>> as first thanks a lot for Your replay, it is more clear for me now. >>> >>> Do you have any idea how to distinguish connected clients? I was looking >>> for some information which I can use but I don't see anything usable in >>> NodeGroup interface [1]. It provides access just for `Node` protobuf object >>> but I don't see anything usable there. >>> >>> Any idea or example? >>> >>> Thanks! >>> >>> [1] >>> https://github.com/envoyproxy/java-control-plane/blob/main/cache/src/main/java/io/envoyproxy/controlplane/cache/NodeGroup.java >>> On Monday, August 16, 2021 at 7:15:11 PM UTC+2 Mark D. Roth wrote: >>> >>>> The xDS protocol does not require the node information to be sent by >>>> the client for every request on the stream; the client needs to send it >>>> only on the first request on the stream. Quoting this section of the >>>> xDS spec >>>> <https://www.envoyproxy.io/docs/envoy/latest/api-docs/xds_protocol#basic-protocol-overview> >>>> : >>>> >>>> Only the first request on a stream is guaranteed to carry the node >>>>> identifier. The subsequent discovery requests on the same stream may >>>>> carry >>>>> an empty node identifier. This holds true regardless of the acceptance of >>>>> the discovery responses on the same stream. The node identifier should >>>>> always be identical if present more than once on the stream. It is >>>>> sufficient to only check the first message for the node identifier as a >>>>> result. >>>> >>>> >>>> The Java implementation may currently happen to send the node >>>> information with every request on the stream, but it's not required to do >>>> that, and your xDS server should not expect that behavior. I think you >>>> need to change your xDS server to look at the node information on the >>>> first >>>> request on the stream and store the node.cluster field so that it knows >>>> the >>>> value when it sees subsequent requests on the same stream. >>>> >>>> I hope this information is helpful. >>>> >>>> On Mon, Aug 16, 2021 at 2:30 AM Lukáš Drbal <[email protected]> wrote: >>>> >>>>> Hello everyone, >>>>> >>>>> We are trying to setup routing via XDS to our GRPC services. Routing >>>>> should be based on `node.cluster` information provided from client. >>>>> >>>>> Basically we would like to have 2 groups of GRPC clusters (priority >>>>> and normal) with same endpoints and choose right one by client >>>>> `node.cluster` identification. >>>>> >>>>> I have very minimal setup [1] which works absolutely as we expected >>>>> for java client but doesn't work for C++ (and grpc_cli). Node hashing >>>>> implementation [2]. This is minimal setup to reproducing this behaviour, >>>>> regular routing is more complicated. >>>>> >>>>> From log perspective it looks like from C++ xds server receive >>>>> `node.cluster` information just in first request. >>>>> >>>>> From java I see cluster in all requests: >>>>> grpc-default-executor-0] INFO org.example.xds.routing.XdsServer - >>>>> Routing [priority] to priority group. [grpc-default-executor-1] INFO >>>>> org.example.xds.routing.XdsServer - Routing [priority] to priority group. >>>>> [grpc-default-executor-1] INFO org.example.xds.routing.XdsServer - >>>>> Routing >>>>> [priority] to priority group. [grpc-default-executor-0] INFO >>>>> org.example.xds.routing.XdsServer - Routing [priority] to priority group. >>>>> [grpc-default-executor-1] INFO org.example.xds.routing.XdsServer - >>>>> Routing >>>>> [priority] to priority group. [grpc-default-executor-2] INFO >>>>> org.example.xds.routing.XdsServer - Routing [priority] to priority group. >>>>> [grpc-default-executor-1] INFO org.example.xds.routing.XdsServer - >>>>> Routing >>>>> [priority] to priority group. [grpc-default-executor-2] INFO >>>>> org.example.xds.routing.XdsServer - Routing [priority] to priority group. >>>>> >>>>> But from cli / c++ I see cluster just in first request: >>>>> [grpc-default-executor-0] INFO org.example.xds.routing.XdsServer - >>>>> Routing [priority] to priority group. [grpc-default-executor-0] INFO >>>>> org.example.xds.routing.XdsServer - Routing [] to normal group. >>>>> [grpc-default-executor-0] INFO org.example.xds.routing.XdsServer - >>>>> Routing >>>>> [] to normal group. >>>>> >>>>> This leads to expected error when c++ client is trying to get priority >>>>> listeners and routes from default group. >>>>> >>>>> Can somebody give me any hint what's wrong here? >>>>> >>>>> Thanks a lot! >>>>> >>>>> L. >>>>> >>>>> [1] https://github.com/LesTR/xds-routing-test >>>>> [2] >>>>> https://github.com/LesTR/xds-routing-test/blob/master/src/main/java/org/example/xds/routing/XdsServer.java#L61 >>>>> >>>>> -- >>>>> You received this message because you are subscribed to the Google >>>>> Groups "grpc.io" group. >>>>> To unsubscribe from this group and stop receiving emails from it, send >>>>> an email to [email protected]. >>>>> To view this discussion on the web visit >>>>> https://groups.google.com/d/msgid/grpc-io/1de7e140-862f-414f-b25a-7b1afc4069can%40googlegroups.com >>>>> >>>>> <https://groups.google.com/d/msgid/grpc-io/1de7e140-862f-414f-b25a-7b1afc4069can%40googlegroups.com?utm_medium=email&utm_source=footer> >>>>> . >>>>> >>>> >>>> >>>> -- >>>> Mark D. Roth <[email protected]> >>>> Software Engineer >>>> Google, Inc. >>>> >>> -- >>> You received this message because you are subscribed to the Google >>> Groups "grpc.io" group. >>> To unsubscribe from this group and stop receiving emails from it, send >>> an email to [email protected]. >>> >> To view this discussion on the web visit >>> https://groups.google.com/d/msgid/grpc-io/19380de0-9db6-4095-a120-ae40c453a9e8n%40googlegroups.com >>> >>> <https://groups.google.com/d/msgid/grpc-io/19380de0-9db6-4095-a120-ae40c453a9e8n%40googlegroups.com?utm_medium=email&utm_source=footer> >>> . >>> >> >> >> -- >> Mark D. Roth <[email protected]> >> Software Engineer >> Google, Inc. >> > -- You received this message because you are subscribed to the Google Groups "grpc.io" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/grpc-io/8b674a93-bee9-4135-aeaa-a4f22f500074n%40googlegroups.com.
