On Mon, Oct 19, 2015 at 7:40 AM, edwardk <[email protected]> wrote:
> Hi Artem, > > Thanks for your response. > > Yes I want to do an initial warmup of the data so that during request time > it would be available soon. > > Moreover, the data would be master data, used for read-only purpose and > which would rarely change. > > In that case, I feel we can go with partitioned mode, having one secondary > node to ensure high availability across the nodes. > Sounds right. > > Also, I looked into the loadCache method in Ignite which can be used bulk > loading. It should be the one I feel I need to use. > But, I am just trying to understand how the bulk load works in ignite. > > Does ignite equally distribute the data across the nodes during load time. > All cache nodes will get a closure which will start the loading process on each node in parallel. Each node can choose to connect to the database and load the required data. Only the data that the node is responsible for will be stored on that node and the rest will be discarded. At the end, each node will end up with the data it is responsible for storing within the partitioned cache. > > Will that help in avoiding the system from hanging with out of memory, > because the data fetched can be more from the database. > I am not sure what you mean by out-of-memory. Are you planning to have enough memory to store the required data set in cache? > > > > > > > > > -- > View this message in context: > http://apache-ignite-users.70518.x6.nabble.com/Bulk-load-data-at-startup-tp1647p1651.html > Sent from the Apache Ignite Users mailing list archive at Nabble.com. >
