Re: Understanding Cache Key, Indexes, Partition and Affinity

[anoorag.saxena]

Thank you so much for explaining the concepts, Andrew.
Much appreciated.



--
View this message in context: 
http://apache-ignite-users.70518.x6.nabble.com/Understanding-Cache-Key-Indexes-Partition-and-Affinity-tp11212p11289.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.

Re: Understanding Cache Key, Indexes, Partition and Affinity

[anoorag.saxena]

Thank you for the quick turnaround, Andrew!

I have some follow-up questions:

1.) //AffinityKey should be a part of entry key and if AffinityKey mapping
is configured then AffinityKey will be used instead of entry key for
entry->partition mapping //

What is meant by Entry Key in this context?


3.) //Indexes always resides on same node with the data.//
That is my understanding too. Someone explained to me that if I create
an Index on a column in the Cache, the column (Index Key) is also used as
the partition or affinity key. Is that statement true?




c.) //c.) Lets say I wanted to do a lookup on a key other than Affinity Key
column, can creating an index on the lookup column help? Would I end up
scanning all nodes in that case?

Staging Cache
CustomerID
CustomerEmail
CustomerPhone

Target Cache
Seq_Num
CustomerID
CustomerEmail
CustomerPhone
StartDate
EndDate
//


In the example above where I am using a Staging Cache with 3 columns and a
Target Cache with 6 columns, lets say CustomerID is my Affinity Key and I
have an Index on CustomerID column in both the caches.

Now if I process records from Staging Cache one by one
(atomicityMode='TRANSACTIONAL') and compare whether CustomerEmail exists in
Target Cache or not, will my application scan the Target Cache on all the
nodes?





--
View this message in context: 
http://apache-ignite-users.70518.x6.nabble.com/Understanding-Cache-Key-Indexes-Partition-and-Affinity-tp11212p11253.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.

Understanding Cache Key, Indexes, Partition and Affinity

[anoorag.saxena]

Hello, 

I am new to Apache Ignite and come from a Data Warehousing background. So
pardon if I try to relate to Ignite through DBMS jargon. I already went
through some of the posts on the forum but I am still unclear about some of
the basics. 

1.) CacheMode=PARTITIONED 
 - When I just declare a cache as partitioned, I understand that data is
equally distributed across all 
 nodes. Is there an option to provide a "partition key" based on which
the data would be distributed 
 across the nodes (in which case, the skewness of distribution would
depend on choice of Partition Key? 
  
 - Without an Affinity Key, when I load data (using loadCache()) into a
partitioned cache, will all source 
   rows be sent to all nodes on my cluster? 

2.) Affinity 
 I understand that the concept of affinity is to use a key that
distinctly identifies the node on which the 
 data may reside. 

 - When I partition the cache and define an affinity key, is the data
partitioned based 
 on the Affinity Key itself? If not, how does affinity differ from
partitioning? 
  
 - With an Affinity Key defined, when I load data (using loadCache())
into a partitioned cache, will the 
   source rows be sent to the node they belong to or all the nodes on my
cluster? 

3.) When I create an index for a cache...does it distribute the data
automatically (without defining any 
  Affinity Key or Partition Key) ? 

SCNEARIO DESCRIPTION 
- 
I want to load data from a persistent layer into a Staging Cache (assume
~2B) using loadCache(). 
The cache resides on a 4 node cluster. 
a.) Can I load data in such a way that each node has to process only 0.5B
records? 
 Is that using Partitioned Cache mode and defining an Affinity Key? 

Then I want to read transactions from the Staging Cache in TRANSACTIONAL
atomicity mode, lookup a Target Cache and do some operations. 
b.) When I do the lookup on Target Cache, how can I ensure that the lookup
is happening only on the node where the data resides and not do lookup on
all the nodes on which Target Cache resides? 
Would that be using the Affinity Key? If yes, how? 

c.) Lets say I wanted to do a lookup on a key other than Affinity Key
column, can creating an index on the lookup column help? Would I end up
scanning all nodes in that case? 

Staging Cache 
CustomerID 
CustomerEmail 
CustomerPhone   

Target Cache 
Seq_Num 
CustomerID 
CustomerEmail 
CustomerPhone 
StartDate 
EndDate 




--
View this message in context: 
http://apache-ignite-users.70518.x6.nabble.com/Understanding-Cache-Key-Indexes-Partition-and-Affinity-tp11212.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.