We have a bunch of different things going into elasticsearch, mostly
network-related telemetry. The latest painful one is IPFIX (netflow). The
aggregation we're working on looks like this:
{
"_index": "ipfix-2014.03.17",
"_type": "logs",
"_id": "oA5uhOc9QpCaH9iGPlY7AQ",
"_score": null,
"_source": {
"peer_ip_src": "207.46.32.122",
"ip_dst": "157.56.106.160",
"as_src": 12670,
"mask_dst": 27,
"as_path": "",
"ip_src": "92.102.0.0",
"bytes": 80,
"port_dst": 3544,
"mask_src": 16,
"stamp_inserted": "2014-03-17 09:18:00",
"stamp_updated": "2014-03-17 10:31:28",
"packets": 1,
"@version": "1",
"@timestamp": "2014-03-17T12:19:27.244Z",
"source": "ipfix"
},
"sort": [
1395058767244
]
}
There's just a really stupid high volume of really small messages that need
little in the way of analysis (which we need to turn off).
On Monday, March 17, 2014 2:45:04 PM UTC-7, Mo wrote:
> It would be helpful to see the document size along with the parameters.
> On Mon, Mar 17, 2014 at 2:27 PM, <[email protected] <javascript:>>wrote:
>
>> I got 25K per second using 3 nodes 32G EC2 system.
>>
>> I am also interested in something like 150k/sec of indexing speed, and
>> need help.
>>
>> --
>> You received this message because you are subscribed to the Google Groups
>> "elasticsearch" group.
>> To unsubscribe from this group and stop receiving emails from it, send an
>> email to [email protected] <javascript:>.
>> To view this discussion on the web visit
>> https://groups.google.com/d/msgid/elasticsearch/ba9799e0-82e8-4337-bd11-e3fbdd9a5f4f%40googlegroups.com<https://groups.google.com/d/msgid/elasticsearch/ba9799e0-82e8-4337-bd11-e3fbdd9a5f4f%40googlegroups.com?utm_medium=email&utm_source=footer>
>> .
>>
>> For more options, visit https://groups.google.com/d/optout.
>>
>
>
--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/ec3c13be-7194-404a-b962-1a3c7612fc07%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
