DataStreamer is designed to insert data in the fastest way possible, not to save memory. If memory consumption is so critical for your, you can try tuning DataStreamer properties like perNodeBufferSize <https://ignite.apache.org/releases/latest/javadoc/org/apache/ignite/IgniteDataStreamer.html#perNodeBufferSize(int)> and perNodeParallelOperations <https://ignite.apache.org/releases/latest/javadoc/org/apache/ignite/IgniteDataStreamer.html#perNodeParallelOperations(int)>. Decreasing them should save you some heap, but will affect performance for sure.
But if you are not running Ignite on Raspberry PI or something, and have some memory reserve, I would recommend you just to increase available heap space. Denis ср, 3 янв. 2018 г. в 13:30, userx <[email protected]>: > Hi Denis, > > Thanks for the reply, yes I am using Streamer but the whole point of using > the streamer is that its the best api available for optimum memory > utilization rather than a putAll. > > I am currently running on 1G, if I abruptly increase it to 5G then whats > the > importance of a Streamer in such a case ? I can use putAll in that case. > > So the question is how do I decide upon the max memory in such a case, > given > that the volume of the data could not always be judged. I can afford Ignite > to perform slowly in terms of data load but definitely not have a failure > because of OOM. > > > > > > -- > Sent from: http://apache-ignite-users.70518.x6.nabble.com/ >
