[ https://issues.apache.org/jira/browse/IGNITE-2310?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15366024#comment-15366024 ]
Taras Ledkov commented on IGNITE-2310: -------------------------------------- Public API changes at the IgniteCompute: two methods are added: {code:title=IgniteCompute.java|borderStyle=solid} void affinityRun(@Nullable String cacheName, Object affKey, IgniteRunnable job, Collection<String> extraCaches) throws IgniteException; <R> R affinityCall(@Nullable String cacheName, Object affKey, IgniteCallable<R> job, Collection<String> extraCaches) throws IgniteException; {code} These methods overload existing methods: {code:title=IgniteCompute.java|borderStyle=solid} void affinityRun(@Nullable String cacheName, Object affKey, IgniteRunnable job) throws IgniteException; <R> R affinityCall(@Nullable String cacheName, Object affKey, IgniteCallable<R> job) throws IgniteException; {code} The difference in the parameter {{Collection<String> extraCaches}}. The partitions of these caches will be also reserved. The reserved partition similar to partition where affKey is stored. The partition reservation means the data from the caches *cacheName* and *extraCaches* stored at the specified partition will not be migrated from the node while the *job* is executed on the node. > Lock cache partition for affinityRun/affinityCall execution > ----------------------------------------------------------- > > Key: IGNITE-2310 > URL: https://issues.apache.org/jira/browse/IGNITE-2310 > Project: Ignite > Issue Type: New Feature > Components: cache > Reporter: Valentin Kulichenko > Assignee: Taras Ledkov > Priority: Critical > Labels: community > Fix For: 1.7 > > > Partition of a key passed to {{affinityRun}} must be located on the affinity > node when a compute job is being sent to the node. The partition has to be > locked on the cache until the compute job is being executed. This will let to > execute queries safely (Scan or local SQL) over the data that is located > locally in the locked partition. > In addition Ignite Compute API has to be extended by adding {{affinityCall}} > and {{affinityRun}} methods that accept list of caches which partitions have > to be locked at the time a compute task is being executed. > Test cases to validate the functionality: > 1) local SQL query over data located in a concrete partition in multple > caches. > - create cache Organisation cache and create Persons cache. > - collocate Persons by 'organisationID'; > - send {{affinityRun}} using 'organisationID' as an affinity key and passing > Organisation and Persons caches' names to the method to be sure that the > partition will be locked on caches; > - execute local SQL query "SELECT * FROM Persons as p, Organisation as o > WHERE p.orgId=o.id' on a changing topology. The result set must be complete, > the partition over which the query will be executed mustn't be moved to the > other node. Due to affinity collocation the partition number will be the same > for all Persons that belong to particular 'organisationID' > 2) Scan Query over particular partition that is locked when {{affinityCall}} > is executed. > UPD (YZ May, 31) > # If closure arrives to node but partition is not there it should be silently > failed over to current owner. > # I don't think user should provide list of caches. How about reserving only > one partition, but evict partitions after all partitions in all caches (with > same affinity function) on this node are locked for eviction. [~sboikov], can > you please comment? It seems this should work faster for closures and will > hardly affect rebalancing stuff. > # I would add method {{affinityCall(int partId, String cacheName, > IgniteCallable)}} and same for Runnable. This will allow me not to mess with > affinity key in case I know partition before. > UPD (SB, June, 01) > Yakov, I think it is possible to implement this 'locking for evictions' > approach, but personally I better like partitions reservation: > - approach with reservation already implemented and works fine in sql queries > - partition reservation is just CAS operation, if we need do ~10 reservation > I think this will be negligible comparing to job execution time > - now caches are rebalanced completely independently and changing this be > complicated refactoring > - I see some difficulties how to understand that caches have same affinity. > If user uses custom function should he implement 'equals'? For standard > affinity functions user can set backup filter, what do in this case? should > user implement 'equals' for filter? Even if affinity functions are the same > cache configuration can have node filter, so affinity mapping will be > different. -- This message was sent by Atlassian JIRA (v6.3.4#6332)