Alex,
I think you can create a new IEP page and I will fill it with the Cluster
API details.
In short, I’ve introduced several new codes:
Cluster API is pretty straightforward:
OP_CLUSTER_IS_ACTIVE = 5000
OP_CLUSTER_CHANGE_STATE = 5001
OP_CLUSTER_CHANGE_WAL_STATE = 5002
OP_CLUSTER_GET_WAL_STATE = 5003
Cluster group codes:
OP_CLUSTER_GROUP_GET_NODE_IDS = 5100
OP_CLUSTER_GROUP_GET_NODE_INFO = 5101
The underlying implementation is based on the thick client logic.
For every request, we provide a known topology version and if it has
changed,
a client updates it firstly and then re-sends the filtering request.
Alongside the topVer a client sends a serialized nodes projection object
that could be considered as a code to value mapping.
Consider: [{Code = 1, Value= [“DotNet”, “MyAttribute”}, {Code=2, Value=1}]
Where “1” stands for Attribute filtering and “2” – serverNodesOnly flag.
As a result of request processing, a server sends nodeId UUIDs and a
current topVer.
When a client obtains nodeIds, it can perform a NODE_INFO call to get a
serialized ClusterNode object. In addition there should be a different API
method for accessing/updating node metrics.
чт, 21 нояб. 2019 г. в 12:32, Sergey Kozlov <skoz...@gridgain.com>:
> Hi Pavel
>
> On Thu, Nov 21, 2019 at 11:30 AM Pavel Tupitsyn <ptupit...@apache.org>
> wrote:
>
> > 1. I believe that Cluster operations for Thin Client protocol are already
> > in the works
> > by Alexandr Shapkin. Can't find the ticket though.
> > Alexandr, can you please confirm and attach the ticket number?
> >
> > 2. Proposed changes will work only for Java tasks that are already
> deployed
> > on server nodes.
> > This is mostly useless for other thin clients we have (Python, PHP, .NET,
> > C++).
> >
>
> I don't guess so. The task (execution) is a way to implement own layer for
> the thin client application.
>
>
> > We should think of a way to make this useful for all clients.
> > For example, we may allow sending tasks in some scripting language like
> > Javascript.
> > Thoughts?
> >
>
> The arbitrary code execution from a remote client must be protected
> from malicious code.
> I don't know how it could be designed but without that we open the hole to
> kill cluster.
>
>
> >
> > On Thu, Nov 21, 2019 at 11:21 AM Sergey Kozlov <skoz...@gridgain.com>
> > wrote:
> >
> > > Hi Alex
> > >
> > > The idea is great. But I have some concerns that probably should be
> taken
> > > into account for design:
> > >
> > > 1. We need to have the ability to stop a task execution, smth like
> > > OP_COMPUTE_CANCEL_TASK operation (client to server)
> > > 2. What's about task execution timeout? It may help to the cluster
> > > survival for buggy tasks
> > > 3. Ignite doesn't have roles/authorization functionality for now.
> But
> > a
> > > task is the risky operation for cluster (for security reasons).
> Could
> > we
> > > add for Ignite configuration new options:
> > > - Explicit turning on for compute task support for thin protocol
> > > (disabled by default) for whole cluster
> > > - Explicit turning on for compute task support for a node
> > > - The list of task names (classes) allowed to execute by thin
> > client.
> > > 4. Support the labeling for task that may help to investigate issues
> > on
> > > cluster (the idea from IEP-34 [1])
> > >
> > > 1.
> > >
> > >
> >
> https://cwiki.apache.org/confluence/display/IGNITE/IEP-34+Thin+client%3A+transactions+support
> > >
> > >
> > >
> > > On Thu, Nov 21, 2019 at 10:58 AM Alex Plehanov <
> plehanov.a...@gmail.com>
> > > wrote:
> > >
> > > > Hello, Igniters!
> > > >
> > > > I have plans to start implementation of Compute interface for Ignite
> > thin
> > > > client and want to discuss features that should be implemented.
> > > >
> > > > We already have Compute implementation for binary-rest clients
> > > > (GridClientCompute), which have the following functionality:
> > > > - Filtering cluster nodes (projection) for compute
> > > > - Executing task by the name
> > > >
> > > > I think we can implement this functionality in a thin client as well.
> > > >
> > > > First of all, we need some operation types to request a list of all
> > > > available nodes and probably node attributes (by a list of nodes).
> Node
> > > > attributes will be helpful if we will decide to implement analog of
> > > > ClusterGroup#forAttribute or ClusterGroup#forePredicate methods in
> the
> > > thin
> > > > client. Perhaps they can be requested lazily.
> > > >
> > > > From the protocol point of view there will be two new operations:
> > > >
> > > > OP_CLUSTER_GET_NODES
> > > > Request: empty
> > > > Response: long topologyVersion, int minorTopologyVersion, int
> > nodesCount,
> > > > for each node set of node fields (UUID nodeId, Object or String
> > > > consistentId, long order, etc)
> > > >
> > > > OP_CLUSTER_GET_NODE_ATTRIBUTES
> > > > Request: int nodesCount, for each node: UUID nodeId
> > > > Response: int nodesCount, for each node: int attributesCount, for
> each
> > > node
> > > > attribute: String name, Object value
> > > >
> > > > To execute tasks we need something like these methods in the client
> > API:
> > > > Object execute(String task, Object arg)
> > > > Future<Object> executeAsync(String task, Object arg)
> > > > Object affinityExecute(String task, String cache, Object key, Object
> > arg)
> > > > Future<Object> affinityExecuteAsync(String task, String cache, Object
> > > key,
> > > > Object arg)
> > > >
> > > > Which can be mapped to protocol operations:
> > > >
> > > > OP_COMPUTE_EXECUTE_TASK
> > > > Request: UUID nodeId, String taskName, Object arg
> > > > Response: Object result
> > > >
> > > > OP_COMPUTE_EXECUTE_TASK_AFFINITY
> > > > Request: String cacheName, Object key, String taskName, Object arg
> > > > Response: Object result
> > > >
> > > > The second operation is needed because we sometimes can't calculate
> and
> > > > connect to affinity node on the client-side (affinity awareness can
> be
> > > > disabled, custom affinity function can be used or there can be no
> > > > connection between client and affinity node), but we can make best
> > effort
> > > > to send request to target node if affinity awareness is enabled.
> > > >
> > > > Currently, on the server-side requests always processed synchronously
> > and
> > > > responses are sent right after request was processed. To execute long
> > > tasks
> > > > async we should whether change this logic or introduce some kind
> > two-way
> > > > communication between client and server (now only one-way requests
> from
> > > > client to server are allowed).
> > > >
> > > > Two-way communication can also be useful in the future if we will
> send
> > > some
> > > > server-side generated events to clients.
> > > >
> > > > In case of two-way communication there can be new operations
> > introduced:
> > > >
> > > > OP_COMPUTE_EXECUTE_TASK (from client to server)
> > > > Request: UUID nodeId, String taskName, Object arg
> > > > Response: long taskId
> > > >
> > > > OP_COMPUTE_TASK_FINISHED (from server to client)
> > > > Request: taskId, Object result
> > > > Response: empty
> > > >
> > > > The same for affinity requests.
> > > >
> > > > Also, we can implement not only execute task operation, but some
> other
> > > > operations from IgniteCompute (broadcast, run, call), but it will be
> > > useful
> > > > only for java thin client. And even with java thin client we should
> > > whether
> > > > implement peer-class-loading for thin clients (this also requires
> > two-way
> > > > client-server communication) or put classes with executed closures to
> > the
> > > > server locally.
> > > >
> > > > What do you think about proposed protocol changes?
> > > > Do we need two-way requests between client and server?
> > > > Do we need support of compute methods other than "execute task"?
> > > > What do you think about peer-class-loading for thin clients?
> > > >
> > >
> > >
> > > --
> > > Sergey Kozlov
> > > GridGain Systems
> > > www.gridgain.com
> > >
> >
>
>
> --
> Sergey Kozlov
> GridGain Systems
> www.gridgain.com
>
--
Alex.
