Hi Andrey, Yes we are using SSD. Earlier we were using default checkpoint buffer 256 MB , in order to reduce the frequency, we increased the buffer size , but it didn’t have any impact on performance
On Fri, 30 Mar 2018 at 10:49 PM, Andrey Mashenkov < [email protected]> wrote: > Hi, > > Possibly, storage is a bottleneck or checkpoint buffer is too large. > Do you use Provissioned IOPS SSD? > > > On Fri, Mar 30, 2018 at 3:32 PM, rahul aneja <[email protected]> > wrote: > >> Hi , >> >> We are trying to load orc data (around 50 GB) on s3 from spark using >> dataframe API. It starts fast with good write throughput and then after >> sometime throughput just drops and it gets stuck. >> >> We also tried changing multiple configurations , but no luck >> 1. enabling checkpoint write throttling >> 2. disabling throttling and increasing checkpoint buffer >> >> >> Please find below configuration and properties of the cluster >> >> >> 1. 10 node cluster r4.4xl (EMR aws) and shared with spark >> 2. ignite is started with -Xms20g -Xmx30g >> 3. Cache mode is partitioned >> >> 4. persistence is enabled >> 5. DirectIO is enabled >> 6. No backup >> >> <property name=“dataStorageConfiguration”> >> <bean >> class=“org.apache.ignite.configuration.DataStorageConfiguration”> >> <!-- Enable write throttling. --> >> <property name=“writeThrottlingEnabled” value=“false”/> >> <property name=“defaultDataRegionConfiguration”> >> <bean >> class=“org.apache.ignite.configuration.DataRegionConfiguration”> >> <property name=“persistenceEnabled” value=“true”/> >> <property name=“checkpointPageBufferSize” >> value=“#{20L * 1024 * 1024 * 1024}“/> >> <property name=“name” value=“Default_Region”/> >> <property name=“maxSize” value=“#{60L * 1024 * >> 1024 * 1024}“/> >> </bean> >> </property> >> <property name=“walMode” value=“NONE”/> >> </bean> >> </property> >> >> >> Thanks in advance, >> >> Rahul Aneja >> >> >> > > > -- > Best regards, > Andrey V. Mashenkov >
