I don't know how you are doing the indexing . Are you using bulk request or .. ? Bulk insert can greatly increase indexing speed.
You can also check node client. It should have better indexing speed because it will be 1 hop operation, compared to two hop with transport client. (assuming Java AP here) You can hit the limits of the bulk thread pool(can be increased). If you are sending all indexing ops to one server only. One could try to hist all master nodes on round-robin basis. You can monitor IOPs in marvel (or iostat locally on the server) to see if are not hitting IO limit. On my ES cluster i reach 50k indexing ops per second. On Monday, June 9, 2014 5:40:53 PM UTC+2, pranav amin wrote: > > Hi all, > > While doing some prototyping in ES using SSD's we got some good Write TPS. > But the Write TPS saturated after adding some more nodes! > > > Here are the details i used for prototyping - > > Requirement: To read data as soon as possible since the read is followed > by write. > Version of ES:1.0.0 > Document Size:144 KB > Use of SSD for Storage: Yes > Benchmarking Tool: Soap UI or Jmeter > VM: Ubuntu, 64 Bit OS > Total Nodes: 12 > Total Shards: 60 > Threads: 200 > Replica: 2 > Index Shards: 20 > Total Index:1 > Hardware configuration: 4 CPU, 6 GB RAM, 3 GB Heap > > Using the above setup we got Write TPS ~= 500. > > We wanted to know by adding more node if we can increase our Write TPS. > But we couldn't. > * By adding 3 more nodes (i..e Total Nodes = 15) the TPS just increase by > 10 i.e. ~= 510. > * Adding more Hardware like CPU, RAM and increasing Heap didn't help as > well [8 CPU, 12 GB RAM, 5 GB Heap]. > > Can someone help out or point ideas what will be wrong? Conceptually ES > should scale in terms of Write & Read TPS by adding more nodes. However we > aren't able to get that. > > Much appreciated if someone can point us in the right direction. Let me > know if more information is needed. > > Thanks > Pranav. > -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/3470cead-d70a-4dbc-af3c-4b47abce4d40%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
