Hi, Is the problem specific to DataStreamer? Will the problem disappear if insert the same data using IgniteCache.put (or putAll batches)?
вс, 15 сент. 2019 г. в 15:49, KR Kumar <[email protected]>: > > Hi all - Why data streamer take lot of time randomly after 500+ million > writes, it frequently and very consistently takes lot of time to finish the > writes and to the extent of 25 to 45 seconds for write. May be its flushing > the data as i have flush frequency set but why not in the beginning and why > only in the end. And also I see heap size going up after sometime and trend > is consistently upwards. > > Here is the streamer configuration: > > dataStreamer.autoFlushFrequency(10000); > dataStreamer.perNodeBufferSize(32 * 1024); > dataStreamer.perNodeParallelOperations(32); > > Not sure if this is of any use but here is the dataStorageConfiguration > > <property name="dataStorageConfiguration"> > <bean > class="org.apache.ignite.configuration.DataStorageConfiguration"> > <property > name="defaultDataRegionConfiguration"> > <bean > class="org.apache.ignite.configuration.DataRegionConfiguration"> > <property > name="initialSize" value="#{512L * 1024 * 1024}"></property> > <property > name="maxSize" value="#{4L * 1024 * 1024 * 1024}" /> > <property > name="persistenceEnabled" value="true" /> > <property > name="checkpointPageBufferSize" value="#{2L *1024 * 1024 * 1024}" /> > </bean> > </property> > <property name="pageSize" > value="#{4 * 1024}" /> > <property name="storagePath" > value="${grid.data}" /> > <property name="walPath" > value="${grid.wal}" /> > <property > name="walArchivePath" value="${grid.wal}/archive" /> > <property name="walMode" > value="BACKGROUND" /> > <property > name="walFlushFrequency" value="5000"></property> > </bean> > </property> -- Best regards, Ivan Pavlukhin
