HI Andrew, I doubt setting "perNodeParallelOperations" will help you because it still possible that two keys from the same sender will be mapped to different nodes and arrive out-of-order from user's point of view. So strict ordering is not achievable unless you have only single node in topology, or all updated keys from a single sender are mapped to the same node (e.g. by means of custom affinity function).
Moreover, setting "perNodeParallelOperations" and/or "perNodeBufferSize" to 1 will decrease performance and make DataStreamer behave like regular cache PUTs. Have you tried single put per update from sender? If yes, then how far are results from desired performance numbers? If you give more detailed explanation of your use case and provide configuration you use, probably community will be able to give you some hints on how to improve performance. Vladimir. On Tue, Feb 16, 2016 at 6:21 AM, Andrew <[email protected]> wrote: > Hi igniters, > > My situation is like below. > There are 3000+ message senders outside. > They will send message per 0.1 sec and the sequence of message have to be > kept per each sender. > So I made a cache to buffer all messages then put all message into the > cache > using DataStreamer. > But when I getting message from the cache with continuous query, I found > the > sequence of message are broken. > > I found the DataStreamer cannot keep the sequence of data when it flush at > your document. > > This is my question. > 1. Is there any simple way to make the sequence be right though using > DataStreamer? > - I tried to use putAll() but as the size of message is growing it > performs so poor. > > 2. What problems would be happen if I use DataStreamer like below? > - Limit DataStreamer's perNodeParallelOperations() as 1 to keep > sequence(Affinity key is applied for message sender) > - Create multiple threads to make DataStreamer described above > - Run threads concurrently > > Sincerely, > Andrew > > > > > -- > View this message in context: > http://apache-ignite-users.70518.x6.nabble.com/Need-your-advice-for-using-DataStreamer-tp3015.html > Sent from the Apache Ignite Users mailing list archive at Nabble.com. >
