Hi,
I am storing parent child relationship in Ignite cache. Its a runtime
infinite stream of data(From Kafka) and partitioned across many node.
Data is relevant for 30 days only ,so expiry is set to 30 days.
Right now my Input record structure look like following:
/{
id: (String: UUID)
parentId : (String: UUID)
}/
In Ignite I store these records with Id as key and Object as value with both
Id and parentId indexed.
Following are assumption for input record:
- Same input records can come multiple times
- All record are considered as root if Its parentId , do not appear as Id in
another records.
- Records can come any order. ie. For Example leaves Node can come before
root node.
My Use case is, When I searched for an ID, I need to get all parent in
upward direction.( ie. parent node, parent to parent node...till root node).
I am currently making recursive call to get it.
My question is, Hoe can I optimize it. Considering write speed should not be
compromise as its has to keep up with incoming fast data. Right now its
10K-20K/Sec but should have capability to scale further.
I also though about Affinity Collocation, But not sure how to insure all
records of a tree goes to same Ignite Node.
Note: My my previous solution was based on Neo4j but It need something which
can scale horizontally as neo4j expect whole database to be on single
machine.
--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/
