Re: Ignite Data Streamer Performance is not improving with increase in threads

Hi Andrew, We have observed that following method(segment.access()) blocks ignite data caching using data streamers(For single ignite instance). This limits our resource utilization i.e. CPU, MEM are not fully utilized. How can we avoid this blocking so that we can get maximum performance out

Re: Ignite Data Streamer Performance is not improving with increase in threads

Hi Andrey, Attached is the code we have used for bench marking. Is there any tuning that we can apply to get better performance out of ignite single instance further? Also we have attached logs taken from our tool where we varied datastreamer parallelism from 1 to 16(default). In this

Re: Ignite Data Streamer Performance is not improving with increase in threads

Hi Rish, DataStreamer does not have any threads, DataStreamPool is used for incoming batches processing. DataStreamer split you data into batches which size is equal to streamer perNodeBufferSize parameter. When batch become full, a task is send to a node the buffer belongs to.

Re: Ignite Data Streamer Performance is not improving with increase in threads

I have tried writing while loop which continuously inserts data(same entry) with increamenting cache key(so that there is unique key). Without datastrmr.addData() while loop is generating data at 200K msgs/sec(Data generation rate is much more than caching rate). Is there any blocking done by

Re: Ignite Data Streamer Performance is not improving with increase in threads

Hi Rishi, Is it possible ignite put faster than you preparing data for put? Or may be you change same entry each time and there is a contention on this entry? On Thu, Jun 29, 2017 at 7:44 AM, rishi007bansod wrote: > Hi, > With datastreamer threads = 16(default)

Ignite Data Streamer Performance is not improving with increase in threads

Hi, With datastreamer threads = 16(default) am able to get upto 80 K msgs/sec throughput. As I have 56 core machine(with hyperthreading), I have tried increasing this pernodeparallelprocessing parameter of datastreamer to 56. But still no increase in throughput is observed. Also out of 5600%