Re: Input data is no significant change in multi-threading

Hi Woo, It may be reasonable, if you see, nodes resources utilization is too low and rising per-node-buffer size have no effect (that means you prepare data for nodes too slow). Of course, you should check first if network isn't a bottleneck. On Tue, Apr 25, 2017 at 10:08 AM, woo charles

Re: Input data is no significant change in multi-threading

Same data set mean that I separate original data into 2 parts & input them from 2 separate programs. E.g. a data set with id 1 - 100. Program A input id 1-50. Program B input 51 - 100. 2017-04-21 17:24 GMT+08:00 Andrey Mashenkov : > Hi Woo, > > DataStreamer is

Re: Input data is no significant change in multi-threading

Hi Woo, DataStreamer is designed to fill cache with maximum throughput. By default, streamer will not rewrite cache data, until allowOverwite option is set. Why you need to input same set of data? Why do you expected data input time will change significantly with 2 programs compared to 1 if data

Re: Input data is no significant change in multi-threading

If the data is buffered in client side, the bottleneck should be also in client side. If I use 2 programs to input same set data, it should be a significant change in data input time. Is it right? 2017-04-21 6:46 GMT+08:00 Dmitriy Setrakyan : > > On Wed, Apr 19, 2017 at

Re: Input data is no significant change in multi-threading

On Wed, Apr 19, 2017 at 10:16 PM, woo charles wrote: > When I call addData() in streamer. this data will send & buffer in server > node. is that correct? > If I correct, this data will buffer in random server node or only the one > it directly connected? > addData()

Re: Input data is no significant change in multi-threading

Hi Woo, AddData() add entry to one of local buffer according to key affinity, then buffer is sent to server node that is primary for all buffer keys. On Thu, Apr 20, 2017 at 8:16 AM, woo charles wrote: > When I call addData() in streamer. this data will send & buffer

Re: Input data is no significant change in multi-threading

When I call addData() in streamer. this data will send & buffer in server node. is that correct? If I correct, this data will buffer in random server node or only the one it directly connected? 2017-04-19 18:33 GMT+08:00 Andrey Mashenkov : > It may have effect if you

Re: Input data is no significant change in multi-threading

It may have effect if you prepare data for streamer (call addData) slowly and it is possible to utilize more resources for it. Of course remote nodes should be able to bear pressure of data. Performance can increased, but usually slightly as network will be a bottleneck. On Wed, Apr 19, 2017 at

Re: Input data is no significant change in multi-threading

Is that mean the performance of input data will not affect if I use 2 IgniteDataStreamer(2 client program) to input data as they use the same queue in remote nodes? 2017-04-19 10:02 GMT+08:00 Andrey Mashenkov : > Hi Woo, > > IgniteDataStreamer uses per node buffer to

Re: Input data is no significant change in multi-threading

Hi Woo, IgniteDataStreamer uses per node buffer to make bulk cache updates that shows much better throughput than single updates. Also, IgniteDataStreamer send jobs to remote nodes, to utilize multiple threads on remote nodes. In multi-node grid IgniteDataStreamer usually shows better results

Input data is no significant change in multi-threading

When I try to input data(80 table, each 1 records) to a cluster with 3 server node(each 2 gb), it only has a small change in time if multi thread is performed (ie. max decrease from 8s to 6.5s if using IgniteCache) Is it normal? Also, I found that multi thread do not affect the data input