Re: Enable Native persistence only for one node of the cluster

[Krish]

Background on Use case: We have around 1 tasks that we want to run across
ignite cluster. These tasks will be submitted by multiple independent client
nodes. Each task will have a priority and task key. Two tasks with the same
taskKey should not be running in parallel. Execution of all the tasks should
be based on priority. As far as I know, ignite does not provide locking
between two tasks and it does not provide a way to distribute tasks based on
its priority *across *the cluster.

To meet the above requirements, we have a cluster singleton monitoring
ignite service which will create the compute tasks and distribute them
across the cluster. It will take care of priority and locking. All clients
will create a cache object(based on which service can create actual compute
tasks) and store it in the cache store. Service will query the cache-store
to find the next set of valid tasks based on priority and locking.

We need native persistence because even if the cluster goes down, it should
not lose the tasks that were sent by the clients. And only ignite service
will be dealing with these task objects, we are thinking of enabling it only
for the cluster node on which our ignite service is hosted.

Let me know if this makes sense or if you see any red flags.

Thanks,
Krish







--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Enable Native persistence only for one node of the cluster

[Krish]

Is it possible to have a cluster topology where native persistence is enabled
only for one node and all other nodes use in-memory cache store *without
*native persistence?



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: Execute Ignite Jobs with semaphore

[Krish]

The below implementation may not work for me because if the semaphore is not
available then the task won't do any work. I will have to resubmit the same
task to do the work that it is supposed to do.


 
DonTequila wrote
> public Object execute() {
>  IgniteSemaphore semaphore = this.ignite.semaphore( "123" , 1 , true,
> true);
>  boolean acquired = semaphore.tryAquire();
>  if (acquired) {
>    // run logic
>semaphore.release();
>  }
> }


Thanks,
Krish




--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Execute Ignite Jobs with semaphore

[Krish]

Hi All,

I want to make sure that no two compute jobs with the same key (any
attribute of Compute job) should be executed in parallel by ignite. I am
using semaphore to achieve this.

Execute method of my compute job:

public Object execute() {
IgniteSemaphore semaphore = this.ignite.semaphore( "123" , 1 , true,
true);
semaphore.aquire();
// run logic
semaphore.release();
}

Using the above, I am able to achieve the goal (no two compute jobs with
semaphore key "123" will be executing in parallel). However, consider the
scenario where I have 1000s of Compute jobs with the same key 123, these
tasks will be distributed across the cluster nodes and all cluster nodes
will start executing these jobs in parallel. But because of the semaphore,
only one job will get executed and all other threads(across the cluster)
will be blocked by the semaphore. This will degrade the performance. 

To avoid the blocking, I should not acquire the semaphore inside the job
execution.
But I can not acquire the semaphore on the client-side as that will lead to
synchronous task submission. I want task submission to be asynchronous. What
would be the best place to acquire the semaphore in this scenario?

I am thinking of writing a middle layer using IngiteQueues and
IgniteServices. The client will submit the tasks to IgniteQueue via
IgniteService. IgniteService will try to acquire the semaphore before
submitting the task, if it is not able to acquire that it will push the task
to the end of the queue and pick up the next job. What do you think about
this approach?

Thanks,
Krish






--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: Execute Ignite Callable Jobs with set priorities

[Krish]

ilya.kasnacheev wrote
> This means that Ignite prioritizing is a poor fit for you and you may need
> to roll out your own, perhaps based on IgniteQueue.

Does ignite provide priority queue implementation of IgniteQueue interface?

Thanks,
Krish



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: Execute Ignite Callable Jobs with set priorities

[Krish]

Thanks Stephen and Ilya,

As mentioned in job scheduling documentation, collisionAPI will take care of
job scheduling when jobs arrive at destination node. Lets say I use
PriorityQueueCollisionSpi and send three jobs with priorities 5, 7 and 10 to
one node, then that node will execute job with priority 10 first then job
with priority 7 and then 5.

However, My use case is different than this. Going with above example, in my
use case, ignite client will send three jobs to three different nodes and I
would still want these jobs to be executed based on their priorities.
Basically, no matter how these jobs are distributed across the cluster they
should be executed based on priority. Can this be achieved using
CollisionAPI?

Many Thanks,
K



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/