[ https://issues.apache.org/jira/browse/IGNITE-6699?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16462181#comment-16462181 ]
Anton Vinogradov commented on IGNITE-6699: ------------------------------------------ [~ivanan.fed] Looks good to me now. > Optimize client-side data streamer performance > ---------------------------------------------- > > Key: IGNITE-6699 > URL: https://issues.apache.org/jira/browse/IGNITE-6699 > Project: Ignite > Issue Type: Task > Components: streaming > Affects Versions: 2.3 > Reporter: Vladimir Ozerov > Assignee: Ivan Fedotov > Priority: Major > Labels: performance > Fix For: 2.6 > > > Currently if a user has several server nodes and a single client node with > single thread pushing data to streamer, he will not be able to load data at > maximum speed. On the other hand, if he start several data loading threads, > throughput will increase. > One of root causes of this is bad data streamer design. Method > {{IgniteDataStreamer.addData(K, V)}} returns new feature for every operation, > this is too fine grained approach. Also it generates a lot of garbage and > causes contention on streamer internals. > Proposed implementation flow: > 1) Compare performance of {{addData(K, V)}} vs {{addData(Collection)}} > methods from one thread in distributed environment. The latter should show > considerably higher throughput. > 2) Users should receive per-batch features, rather than per-key. > 3) Try caching thread data in some collection until it is large enough to > avoid contention and unnecessary allocations. -- This message was sent by Atlassian JIRA (v7.6.3#76005)