Hi Ivan, Sorry for a possibly naive question. As I understand we are talking about order of establishing client-server connections. And I suppose that in some environments (e.g. cloud) servers cannot directly establish connections with clients. But TCP connections are bidirectional and we still can send messages in both directions. Could you please provide an example case in which servers have to initiate new connections to clients?
2020-06-29 13:08 GMT+03:00, Ivan Bessonov <bessonov...@gmail.com>: > Hi igniters, Hi Raymond, > > that was a really good point. I will try to address it as much as I can. > > First of all, this new mode will be configurable for now. As Val suggested, > "TcpCommunicationSpi#forceClientToServerConnections" will be a new > setting to trigger this behavior. Disabled by default. > > About issues with K8S deployments - I'm not an expert, but from what I've > heard, sometimes servers and client nodes are not in the same environments. > For example, there is an Ignite cluster and user tries to start client node > in > isolated K8S pod. In this case clients cannot properly resolve their own > addresses > and send it to servers, making it impossible for servers to connect to such > clients. > Or, in other words, clients are used as if they were thin. > > In your case everything is fine, clients and servers share the same network > and can resolve each other's addresses. > > Now, CQ issue [1]. You can pass a custom event filter when you register a > new > continuous query. But, depending on the setup, the class of this filter may > not > be in the classpath of the server node that holds the data and invokes that > filter. > There are two solutions to the problem: > - server fails to resolve class name and fails to register CQ; > - or server can have p2p deployment enabled. Let's assume that it was a > client > node that requested CQ. In this case the server will try to download > "class" file > directly from the node that sent the filter object in the first place. Due > to a poor > design decision it will be done synchronously while registering the query, > and > query registration is happening in "discovery" thread. In normal > circumstances > the server will load the class and finish query registration, it's just a > little bit slow. > > Second case is not compatible with a new "forceClientToServerConnections" > setting. I'm not sure that I need to go into all technical details, but the > result of > such procedure is a cluster that cannot process any discovery messages > during > TCP connection timeout, we're talking about tens of seconds or maybe even > several minutes depending on the settings and the environment. All this > time the > server will be in a "deadlock" state inside of the "discovery" thread. It > means that > some cluster operations will be unavailable during this period, like new > node joining > or starting a new cache. Node failures will not be processed properly as > well. For > me it's hard to predict real behavior until we reproduce the situation in a > live > environment. I saw this in tests only. > > I hope that my message clarifies the situation, or at least doesn't cause > more > confusion. These changes will not affect your infrastructure or your Ignite > installations, they are aimed at adding more flexibility to other ways of > using Ignite. > > [1] https://issues.apache.org/jira/browse/IGNITE-13156 > > > > сб, 27 июн. 2020 г. в 09:54, Raymond Wilson <raymond_wil...@trimble.com>: > >> I have just caught up with this discussion and wanted to outline a set of >> use >> cases we have that rely on server nodes communicating with client nodes. >> >> Firstly, I'd like to confirm my mental model of server & client nodes >> within >> a grid (ignoring thin clients for now): >> >> A grid contains a set of nodes somewhat arbitrarily labelled 'server' and >> 'client' where the distinction of a 'server' node is that it is >> responsible >> for containing data (in-memory only, or also with persistence). Apart >> from >> that distinction, all nodes are essentially peers in the grid and may use >> the messaging fabric, compute layer and other grid features on an equal >> footing. >> >> In our solution we leverage these capabilities to build and orchestrate >> complex analytics queries that utilise compute functions that are >> initiated >> in three distinct ways: client -> client, client -> server and server -> >> client, and where all three styles of initiation are using within a >> single >> analytics request made to the grid it self. I can go into more detail >> about >> the exact sequencing of these activities if you like, but it may be >> sufficient to know they are used to reason about the problem statement >> and >> proposals outlined here. >> >> Our infrastructure is deployed to Kubernetes using EKS on AWS, and all >> three >> relationships between client and server nodes noted above function well >> (caveat: we do see odd things though such as long pauses on critical >> worker >> threads, and occasional empty topology warnings when locating client >> nodes >> to send requests to). We also use continuous queries in three contexts >> (all >> within server nodes). >> >> If this thread is suggesting changing the functional relationship between >> server and client nodes then this may have impacts on our architecture >> and >> implementation that we will need to consider. >> >> This thread has highlighted issues with K8s deployments and also CQ >> issues. >> The suggestion is that Server to Client just doesn't work on K8s, which >> does >> not agree with our experience of it working. I'd also like to understand >> better the bounds of the issue with CQ: When does it not work and what >> are >> the symptoms we would see if there was an issue with the way we are using >> it, or the K8s infrastructure we deploy to? >> >> Thanks, >> Raymond. >> >> >> >> >> -- >> Sent from: http://apache-ignite-developers.2346864.n4.nabble.com/ >> > > > -- > Sincerely yours, > Ivan Bessonov > -- Best regards, Ivan Pavlukhin
