Hi Woo, It may be reasonable, if you see, nodes resources utilization is too low and rising per-node-buffer size have no effect (that means you prepare data for nodes too slow). Of course, you should check first if network isn't a bottleneck.
On Tue, Apr 25, 2017 at 10:08 AM, woo charles <ignite.charl...@gmail.com> wrote: > Same data set mean that I separate original data into 2 parts & input > them from 2 separate programs. > E.g. a data set with id 1 - 100. Program A input id 1-50. Program B input > 51 - 100. > > 2017-04-21 17:24 GMT+08:00 Andrey Mashenkov <andrey.mashen...@gmail.com>: > >> Hi Woo, >> >> DataStreamer is designed to fill cache with maximum throughput. By >> default, streamer will not rewrite cache data, until allowOverwite option >> is set. >> >> Why you need to input same set of data? Why do you expected data input >> time will change significantly with 2 programs compared to 1 if data set is >> put twice? >> Or I missed smth? >> >> Anyway, if you do not get a speed up but you sure you should, then a >> bottleneck have to be found at first. >> >> On Fri, Apr 21, 2017 at 5:02 AM, woo charles <ignite.charl...@gmail.com> >> wrote: >> >>> If the data is buffered in client side, the bottleneck should be also in >>> client side. >>> If I use 2 programs to input same set data, it should be a significant >>> change in data input time. >>> Is it right? >>> >>> 2017-04-21 6:46 GMT+08:00 Dmitriy Setrakyan <dsetrak...@apache.org>: >>> >>>> >>>> On Wed, Apr 19, 2017 at 10:16 PM, woo charles < >>>> ignite.charl...@gmail.com> wrote: >>>> >>>>> When I call addData() in streamer. this data will send & buffer in >>>>> server node. is that correct? >>>>> If I correct, this data will buffer in random server node or only the >>>>> one it directly connected? >>>>> >>>> >>>> addData() will buffer the data on the client side. As a matter of fact, >>>> there are multiple buffers on the client side, which each buffer associated >>>> with some server node. >>>> >>>> Ignite will never send the data to a random node. The data is always >>>> sent exactly to the node where it will be cached. >>>> >>>> D. >>>> >>> >>> >> >> >> -- >> Best regards, >> Andrey V. Mashenkov >> > > -- Best regards, Andrey V. Mashenkov
