OutOfMemoryError while streaming

[smovva]

I'm seeing OutOfMemoryError when I have multiple data streamers pushing data
into the cluster.
My cluster consists of 8 servers running one node each. Each server has 32GB
RAM and 4 Cores.
All the nodes are started like this
=================
bin/ignite.sh -J-Xmx3g config/dev-cluster-config.xml
=================

My cache consists of  
  - key: strings of length 9 chars 
  - Value: POJOs that have some metadata and one large array of integers
(8800 values)

I have 4 data streamers and as they start pushing data, I see some server
nodes throwing OutOfMemoryErrors and the data streamers hangup. 

The streamer code is here.

==========================================
        CacheConfiguration<String, Tile> cacheCfg = new
CacheConfiguration<>(DATA_CACHE_NAME);
        cacheCfg.setAtomicityMode(CacheAtomicityMode.ATOMIC);
        cacheCfg.setCacheMode(CacheMode.PARTITIONED);
        cacheCfg.setBackups(2);
       
cacheCfg.setWriteSynchronizationMode(CacheWriteSynchronizationMode.FULL_SYNC);
        cacheCfg.setIndexedTypes(String.class, Tile.class);

        IgniteCache<String, Tile> viewpointCache =
ignite.getOrCreateCache(cacheCfg);

        try (IgniteDataStreamer<String, Tile> streamer =
ignite.dataStreamer(viewpointCache.getName())) {
            Path sourceFile = Paths.get(hashIndexFile);
            try(BufferedReader reader = Files.newBufferedReader(sourceFile,
Charset.forName("UTF-8")))
            {
                String currentLine = null;
                while((currentLine = reader.readLine()) != null){
                    String hash = currentLine.trim();
                    streamer.addData(hash, getTileFor(hash));
                }
            }
        }
==========================================

Cluster-config.xml

 <bean id="ignite.cfg"
class="org.apache.ignite.configuration.IgniteConfiguration">
        <property name="igniteInstanceName"
value="dev-ignite-cluster-us-east1"/>
       <property name="peerClassLoadingEnabled" value="true"/>

        
        <property name="dataStorageConfiguration">
            <bean
class="org.apache.ignite.configuration.DataStorageConfiguration">
                <property name="defaultDataRegionConfiguration">
                    <bean
class="org.apache.ignite.configuration.DataRegionConfiguration">
                        <property name="persistenceEnabled" value="true"/>
                        <property name="maxSize" value="#{16L * 1024 * 1024
* 1024}"/>
                    </bean>
                </property>
            </bean>
        </property>

         <property name="discoverySpi">
            <bean
class="org.apache.ignite.spi.discovery.tcp.TcpDiscoverySpi">
                <property name="ipFinder">
                    <bean
class="org.apache.ignite.spi.discovery.tcp.ipfinder.s3.TcpDiscoveryS3IpFinder">
                        <!-- Aws config goes here-- >
                    </bean>
                </property>
            </bean>
        </property>
    </bean>

=========================================

Also, I noticed that there is nothing in off-heap memory. Here is the cache
stats for one node.
Total: 513278     
  Heap: 5          
  Off-Heap: 513273
  Off-Heap Memory: 0
and config also shows 
| Non-heap memory initialized | 2mb 
| Non-heap memory used        | 86mb  
| Non-heap memory committed   | 88mb  
| Non-heap memory maximum     | 744mb
How do I make it use and store more in Non-heap memory?

Thanks!



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/