Re: Problem of datastorage after compute finished

[Michael Jay]

Really appreciate for your reply, Andrei. 
Now I understand why the result didn't store on the same node where it was
calculated. I will try your suggestions.
But there still remain a problem that job b,d,f were calculated on Node_B,
the results were lost. No partitions store these jobs' results.
thanks again.



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: Problem of datastorage after compute finished

[aealexsandrov]

Hi Michael,

As I see from your configuration you are going to have cluster with two
ignite servers and PARTITIONED cache.

You can't choose directly on what node will be stored your cache entities
because ignite will use its own affinity function that will maps keys to
partitions (not nodes).

In this case data from your cache could be stored on both nodes.

What you can to do:

1)Create two caches and setup the node filters for them:

https://ignite.apache.org/releases/latest/javadoc/org/apache/ignite/configuration/CacheConfiguration.html#setNodeFilter-org.apache.ignite.lang.IgnitePredicate-

2)Ignite can try to give you guarantee that entities calculated on Node A
will be stored on one node and entities calculated on node B will be stored
on another. But there is no guarantee that it will be the same nodes where
it was calculated. setup your own AffinityKey. For example something like
that:

public static class EntityKey {
public EntityKey(String id, long nodeId) {
this.id = id;
this.nodeId = nodeId;
}

private String id;

// Subscriber ID which will be used for affinity.
@AffinityKeyMapped
private long nodeId;

public long getNodeId() {
return nodeId;
}

public String getId() {
return id;
}

}

After that use it as follow:

IgniteCache cache =
ignite.getOrCreateCache(cfg);

cache.putIfAbsent( new EntityKey(hid, 0), monthValues ) //from node
A
cache.putIfAbsent( new EntityKey(hid, 1), monthValues ) //from node
B

Thanks,
Andrei








--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Problem of datastorage after compute finished

[Michael Jay]

Hi, all! I am trying to use the computegird of ignite to do some compute job,
but I met a problem.
I started two server nodes(different host) to form a clustergroup. Node_A
was started by eclipse, Node_B was started by ignite.cmd, two nodes had the
same configurations except heap size(Node_A heap_size=25G, Node_B
heap_size=10G).
I totally had 6 jobs(a,b,c,d,e,f) to compute and I expected to store the
computation result where the job was done(for example, job a was executed by
Node_A, then the computation result of job a was stored in Node_A). In my
case, reduce is not necessary, moreover, the computation result doesn't need
to be returned. I just want to split the tasks and store the result after
computation is finished.
Actually, Node_A executed job a,c,e and Node_B executed job b,d,f. However
Node_B didn't store the results of job b,d,f, Node_A stored the result of
job c,e, Node_B stored the result of job a. 
I could't figure out why remote node(Node_B) didn't execut the task of
storing the computation result and Node_A didn't store all the jobs' result
that it dealed with?
Is there something wrong with applying of  computetasksplitadapter is this
case?
Below are my code, config.xml and Node_A's log.(code run in Node_A)
thanks for reply! ComputeTaskSplitExample.java

  
default-config.xml
  
ignite-bcd6a658.log
 
 



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/