Hi Pavel, Thanks for your reply.
Here's one of our configs: <beans xmlns="http://www.springframework.org/schema/beans"; xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"; xmlns:util="http://www.springframework.org/schema/util"; xsi:schemaLocation="http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans.xsd http://www.springframework.org/schema/util http://www.springframework.org/schema/util/spring-util.xsd";> <bean id="ignite.cfg" class= "org.apache.ignite.configuration.IgniteConfiguration"> <property name="cacheConfiguration"> <list> <bean class="org.apache.ignite.configuration.CacheConfiguration"> <property name="name" value="bot"/> <property name="atomicityMode" value="TRANSACTIONAL"/> <property name="backups" value="2"/> <property name="cacheMode" value="PARTITIONED"/> <property name="statisticsEnabled" value="true"/> </bean> <bean class="org.apache.ignite.configuration.CacheConfiguration"> <property name="name" value="session"/> <property name="atomicityMode" value="TRANSACTIONAL"/> <property name="backups" value="2"/> <property name="cacheMode" value="PARTITIONED"/> <property name="statisticsEnabled" value="true"/> </bean> <bean class="org.apache.ignite.configuration.CacheConfiguration"> <property name="name" value="placements"/> <property name="atomicityMode" value="ATOMIC"/> <property name="backups" value="2"/> <property name="eagerTtl" value="true"/> <property name="cacheMode" value="PARTITIONED"/> <property name="statisticsEnabled" value="true"/> <property name="expiryPolicyFactory"> <bean class="javax.cache.expiry.CreatedExpiryPolicy" factory-method= "factoryOf"> <constructor-arg> <bean class="javax.cache.expiry.Duration"> <constructor-arg value="HOURS"/> <constructor-arg value="48"/> </bean> </constructor-arg> </bean> </property> </bean> </list> </property> <property name="authenticationEnabled" value="true"/> <property name="dataStorageConfiguration"> <bean class="org.apache.ignite.configuration.DataStorageConfiguration"> <property name="defaultDataRegionConfiguration"> <bean class="org.apache.ignite.configuration.DataRegionConfiguration"> <property name="persistenceEnabled" value="true"/> <property name="maxSize" value="#{16L * 1024 * 1024 * 1024}"/> </bean> </property> </bean> </property> <property name="discoverySpi"> <bean class="org.apache.ignite.spi.discovery.tcp.TcpDiscoverySpi"> <property name="ipFinder"> <bean class= "org.apache.ignite.spi.discovery.tcp.ipfinder.multicast.TcpDiscoveryMulticastIpFinder" > <property name="addresses" value="[list of hosts]"/> </bean> </property> </bean> </property> <property name="connectorConfiguration"> <bean class="org.apache.ignite.configuration.ConnectorConfiguration"> <property name="host" value="0.0.0.0"/> <property name="port" value="[port]"/> <property name="idleQueryCursorTimeout" value="1000"/> </bean> </property> <property name="clientConnectorConfiguration"> <bean class="org.apache.ignite.configuration.ClientConnectorConfiguration"> <property name="maxOpenCursorsPerConnection" value="1200"/> </bean> </property> <property name="includeEventTypes"> <util:constant static-field= "org.apache.ignite.events.EventType.EVT_CACHE_OBJECT_EXPIRED"/> </property> </bean> </beans> There are about 20 clients, and I'm not seeing any threading in the client code; however, what's strange is that we're seeing a very large number of connections (thousands), and we can't figure out what's causing the large number of connections. It does seem that performance drops even further when the connections increases over 2,000, but even when connections are in the hundreds, throughput is still below 3,000 / sec at best. One of our dev's said that IgniteDataStreamer didn't support our use case, I think because we're doing a lot of updates. Devin G. Bost On Thu, Nov 12, 2020 at 10:04 PM Pavel Vinokurov <[email protected]> wrote: > Hi Devin, > > MaxSize is set per node for the specified data region. > A few clarifying questions: > What cache configuration are you using? The performance could depend on > the type of a cache and the number of backups. > How many clients and threads are writing to the cluster? Because it is a > possible bottleneck on the client side. > Have you tried IgniteDataStreamer[2]? It's recommended approach to load > the massive amount of data. > > You could refer to persistence tuning documentation[1]. > > [1] https://ignite.apache.org/docs/latest/persistence/persistence-tuning > [2] https://ignite.apache.org/docs/latest/data-streaming#data-streaming > Thanks, > Pavel > > пт, 13 нояб. 2020 г. в 07:26, Devin Bost <[email protected]>: > >> We're trying to figure out how to get more throughput from our Ignite >> cluster. >> We have 11 dedicated Ignite VM nodes, each with 32 GB of RAM. Yet, we're >> only writing at 2k/sec max, even when we parallelize the writes to Ignite. >> We're using native persistence, but it just seems way slower than >> expected. >> >> We have our cache maxSize set to 16 GB. Since the cluster has a total of >> 352 GB of RAM, should our maxSize be set per node, or per cluster? If it's >> per cluster, then I'm thinking we'd want to increase maxSize to at least >> 300 GB. Looking for feedback on this. >> >> We've already turned off swapping on the boxes. >> >> What else can we tune to increase throughput? It seems like we should be >> getting 100x the throughput at least, so I'm hoping something is just >> misconfigured. >> >> Devin G. Bost >> > > > -- > > Regards > > Pavel Vinokurov >
