Pavel, thanks for the review. I've fixed your comments.
вт, 31 мар. 2020 г. в 13:47, Pavel Tupitsyn :
> Alex, I did a partial review - mostly server-side code. Please see my
> comments on GitHub.
> We will need more eyes on this, especially on Java client side.
>
> Feature masks are a great
Alex, I did a partial review - mostly server-side code. Please see my
comments on GitHub.
We will need more eyes on this, especially on Java client side.
Feature masks are a great addition!
On Tue, Mar 31, 2020 at 1:05 PM Alex Plehanov
wrote:
> Also, I've updated IEP regarding features masks
Also, I've updated IEP regarding features masks (new backward compatibility
mechanism for thin clients) which was discussed earlier in this thread.
вт, 31 мар. 2020 г. в 02:42, Alex Plehanov :
> Guys,
>
> I've finished IEP implementation for server-side and java thin client side
> (ticket: [1],
Guys,
I've finished IEP implementation for server-side and java thin client side
(ticket: [1], PR: [2]). Can you please review the patch?
[1]: https://issues.apache.org/jira/browse/IGNITE-12835
[2]: https://github.com/apache/ignite/pull/7572
пт, 27 мар. 2020 г. в 19:11, Pavel Tupitsyn :
>
Agree with Igor, let's set deployment aside for now, it is out of scope of
this IEP.
On Fri, Mar 27, 2020 at 6:52 PM Igor Sapego wrote:
> Hi guys,
>
> I like the proposal in general.
>
> Now for the task deployment - I think we should have separate API
> for it (and separate IEP I believe).
Hi guys,
I like the proposal in general.
Now for the task deployment - I think we should have separate API
for it (and separate IEP I believe). Also, I'm not sure that this API
should be a part of the API of any thin client as it seems weird to me
to use Python client to deploy Java tasks.
>
> Deployment API definitely needed as one of the next steps. Currently, we
> are talking only about the first step (execution of already deployed
> tasks).
> Also, I'm not sure about automatic redeploy and peer-class-loading for thin
> clients, I think it's better to have more control here and
> will try to provide such guarantees on the server-side
Thanks. I think it is better to do once in the server code than N times in
every client.
On Wed, Mar 25, 2020 at 11:04 PM Alex Plehanov
wrote:
> Pavel,
>
> 1. Actually it can be solved on the client-side (and already solved in PoC
>
Pavel,
1. Actually it can be solved on the client-side (and already solved in PoC
implementation). But I agreed it brings extra complexity for client-side
implementation, will try to provide such guarantees on the server-side.
2. ComputeTask has also "reduce" step which is executed on the
Alex, thanks for preparing the outline.
I'd like us to discuss an approach for compute tasks update with no
downtimes on the servers' end. For instance, let's assume that a
Python/C++/Node.JS developer requested to update a compute task he called
from the app. Should we introduce some system
Hi Alex,
First of all, thank you for preparing this IEP!
Protocol changes look good to me.
However, I have objections regarding race condition behavior and about Java
API.
1. "Due to some races, it's possible that notification for some task will
be delivered to the client before the response
Hello guys.
I've implemented PoC and created IEP [1] for thin client compute grid
functionality. Please have a look.
[1]:
https://cwiki.apache.org/confluence/display/IGNITE/IEP-42+Thin+client%3A+compute+support
пт, 24 янв. 2020 г. в 16:56, Alex Plehanov :
> We've discussed thin client compute
Aleksey Plekhanov created IGNITE-12835:
--
Summary: Thin client: compute support
Key: IGNITE-12835
URL: https://issues.apache.org/jira/browse/IGNITE-12835
Project: Ignite
Issue Type: New
We've discussed thin client compute protocol with Pavel Tupitsyn and Igor
Sapego and come to the conclusion that approach with two-way requests
should be used: client generates taskId and send a request to the server to
execute a task. The server responds that the request has been accepted.
After
Igor, thanks for the reply.
> Approach with taskId will require a lot of changes in protocol and thus
more "heavy" for implementation
Do you mean approach with server notifications mechanism? Yes, it will
require a lot of changes. But in most recent messages we've discussed with
Pavel approach
Huge +1 from me for Feature Masks.
I think this should be our top priority for thin client protocol, since it
simplifies change management a lot.
On Mon, Jan 20, 2020 at 8:21 PM Igor Sapego wrote:
> Sorry for the late reply.
>
> Approach with taskId will require a lot of changes in protocol and
Sorry for the late reply.
Approach with taskId will require a lot of changes in protocol and thus
more "heavy" for implementation, but it definitely looks to me less hacky
than reqId-approach. Moreover, as was mentioned, server notifications
mechanism will be required in a future anyway with high
Looks like we didn't rich consensus here.
Igor, as thin client maintainer, can you please share your opinion?
Everyone else also welcome, please share your thoughts about options to
implement operations for compute.
чт, 28 нояб. 2019 г. в 10:02, Alex Plehanov :
> > Since all thin client
> Since all thin client operations are inherently async, we should be able
to cancel any of them
It's illogical to have such ability. What should do cancel operation of
cancel operation? Moreover, sometimes it's dangerous, for example, create
cache operation should never be canceled. There should
> The request is already processed (task is started), we can't cancel the
request
The request is not "start a task". It is "execute task" (and get result).
Same as "cache get" - you get a result in the end, we don't "start cache
get" then "end cache get".
Since all thin client operations are
> We do cancel a request to perform a task. We may and should use this to
cancel any other request in future.
The request is already processed (task is started), we can't cancel the
request. As you mentioned before, we already do almost the same for queries
(close the cursor, but not cancel the
nterested in the detailed node representation, say with metrics, or only
in a basic format.
This flag should be disabled by default. We could implement a
GetNodeMetrics(nodeId)
method later on if we decide to.
*From: *Pavel Tupitsyn
*Sent: *Tuesday, November 26, 2019 5:44 PM
*To: *dev
*Subject: *Re
> After all, we don't cancel request
We do cancel a request to perform a task. We may and should use this to
cancel any other request in future.
> Client uses some cluster group filtration (for example forServers()
cluster group)
Please see above - Aleksandr Shapkin described how we store
> Anyway, my point stands.
I can't agree. Why you don't want to use task id for this? After all, we
don't cancel request (request is already processed), we cancel the task. So
it's more convenient to use task id here.
> Can you please provide equivalent use case with existing "thick" client?
For
> I can't see any usage of request id in query cursors
You are right, cursor id is a separate thing.
Anyway, my point stands.
> client sends long term tasks to nodes and wants to do it with load
balancing
I still don't get it. Can you please provide equivalent use case with
existing "thick"
> And it is fine to use request ID to identify compute tasks (as we do with
query cursors).
I can't see any usage of request id in query cursors. We send query request
and get cursor id in response. After that, we only use cursor id (to get
next pages and to close the resource). Did I miss
Alex,
> we will mix entities from different layers (transport layer and request
body)
I would not call our message header (which includes the id) "transport
layer".
TCP is our transport layer. And it is fine to use request ID to identify
compute tasks (as we do with query cursors).
> we still
Pavel, in this case, we will mix entities from different layers (transport
layer and request body), it's not very good. The same behavior we can
achieve with generated on client-side task id, but there will be no
inter-layer data intersection and I think it will be easier to implement on
both
Alex,
I have a simpler idea. We already do request id handling in the protocol,
so:
- Client sends a normal request to execute compute task. Request ID is
generated as usual.
- As soon as task is completed, a response is received.
As for cancellation - client can send a new request (with new
Pavel, we need to inform the client when the task is completed, we need the
ability to cancel the task. I see several ways to implement this:
1. Сlient sends a request to the server to start a task, server return task
id in response. Server notifies client when task is completed with a new
Alex,
I think you can create a new IEP page and I will fill it with the Cluster
API details.
In short, I’ve introduced several new codes:
Cluster API is pretty straightforward:
OP_CLUSTER_IS_ACTIVE = 5000
OP_CLUSTER_CHANGE_STATE = 5001
OP_CLUSTER_CHANGE_WAL_STATE = 5002
> Two-way requests (from clients to server and from server to clients)
Alex, can you please clarify the need for server->client requests?
> compute tasks in any language that implemented thin client protocol
(python as a first candidate)
Nikolay, do you have anything in mind here? Why Python?
Hi
On Thu, Nov 21, 2019 at 12:48 PM Denis Garus wrote:
> Hello, Sergey!
>
>
> >> 3. Ignite doesn't have roles/authorization functionality for now.
>
>
> I can't agree with you.
>
My bad, you're right. But the IEP page provided by you is limited and
there's no clear how it works for thin
Hello!
When doing this, I think we should also consider Service invocations to
make their API/implementation as close as possible.
Regards,
--
Ilya Kasnacheev
пт, 22 нояб. 2019 г. в 11:17, Alex Plehanov :
> So, guys:
>
> Cluster API (get nodes and get node attributes) will be implemented in
So, guys:
Cluster API (get nodes and get node attributes) will be implemented in
scope of IGNITE-12385 ticket by Aleksandr.
For compute API as a first step only the ability to execute the task by
task name operation will be implemented, which will require:
- Two-way requests (from clients to
Hello, Igniters.
I think we should support full compute API for the thin clients(at least for
the java clients)
I think our roadmap should be the following:
1. Execution existing compute tasks by id (as Alex suggested).
2. Deploy and execution arbitrary compute tasks from the java thin client
Denis, the primary motivation is to enable execution of deployed to server
java tasks from thin clients (java and other languages).
пт, 22 нояб. 2019 г. в 00:03, Denis Magda :
> Alex, what is the primary motivation for this improvement? Are you looking
> to enable the compute APIs for languages
Alex, what is the primary motivation for this improvement? Are you looking
to enable the compute APIs for languages different from Java?
-
Denis
On Wed, Nov 20, 2019 at 11:58 PM Alex Plehanov
wrote:
> Hello, Igniters!
>
> I have plans to start implementation of Compute interface for Ignite
Igor,
Yes, we can make it work with some additional support on the server:
- Include Platform code along with task name
- Invoke Platform when code is not JAVA
- Add a way to invoke tasks by name in C++ and .NET
On Thu, Nov 21, 2019 at 2:55 PM Igor Sapego wrote:
> Pavel,
>
> The proposed
Pavel,
The proposed solution won't work for PHP, Python, Node.js,
but it will work for C++ and .NET, isn't it? We will just have
to deploy C++/.NET code in closter (just as in Java).
Best Regards,
Igor
On Thu, Nov 21, 2019 at 12:59 PM Aleksandr Shapkin
wrote:
> Folks,
>
>
>
> I started to
Folks,
I started to implement the ClusterGroup API.
There are two tickets at the moment:
- .NET thin client: introduce Cluster API [1]
- .NET Thin Client: introduce ClusterGroup API [2]
The first one introduces the cluster API and is merged to master.
The second ticket is in progress
Guys, thanks for your proposals.
Sergey,
1. Sure we can add OP_COMPUTE_CANCEL_TASK, but it's only possible with a
two-way requests approach.
2. We can also add a timeout to the protocol, also we can add "flags" field
(to pass withNoFailover, withNoResultCache flags).
3. I think having two options
Hello, Sergey!
>> 3. Ignite doesn't have roles/authorization functionality for now.
I can't agree with you.
We already have authorization functionality in Ignite and for a thin client
too [1].
But, compute support for a thin client requires some additional efforts to
get an appropriate
Good points, Sergey.
Maybe you are right, and Java-based compute without peer deployment is a
good first step for thin clients.
On Thu, Nov 21, 2019 at 12:32 PM Sergey Kozlov wrote:
> Hi Pavel
>
> On Thu, Nov 21, 2019 at 11:30 AM Pavel Tupitsyn
> wrote:
>
> > 1. I believe that Cluster
Hi Pavel
On Thu, Nov 21, 2019 at 11:30 AM Pavel Tupitsyn
wrote:
> 1. I believe that Cluster operations for Thin Client protocol are already
> in the works
> by Alexandr Shapkin. Can't find the ticket though.
> Alexandr, can you please confirm and attach the ticket number?
>
> 2. Proposed
1. I believe that Cluster operations for Thin Client protocol are already
in the works
by Alexandr Shapkin. Can't find the ticket though.
Alexandr, can you please confirm and attach the ticket number?
2. Proposed changes will work only for Java tasks that are already deployed
on server nodes.
Hi Alex
The idea is great. But I have some concerns that probably should be taken
into account for design:
1. We need to have the ability to stop a task execution, smth like
OP_COMPUTE_CANCEL_TASK operation (client to server)
2. What's about task execution timeout? It may help to the
Hello, Igniters!
I have plans to start implementation of Compute interface for Ignite thin
client and want to discuss features that should be implemented.
We already have Compute implementation for binary-rest clients
(GridClientCompute), which have the following functionality:
- Filtering
48 matches
Mail list logo
