I just found this great blog about range filter: https://blog.hipchat.com/2014/06/24/elasticsearch-at-hipchat-10x-faster-queries/
from this blog, there are some words about how elasticsearch handle range filter: “For Elasticsearch to answer this query it had to load all the datetime fields from disk to memory for the query, compute the range, and then throw away the data it didn’t need.” I think that's why there is so many disk IO when I facet on a very small time range. And also, someone suggest to use normal filter range with execute option set to datafield. will elasticsearch act exactly the same way as the words above? if that's true, maybe daily index will be my only choice. Alan Sent from Surface From: David Pilato Sent: Friday, August 1, 2014 2:16 PM To: [email protected] There is an issue when using long instead of strings. Cache is not used. Try with something like "from":"2014-06-01","to":"2015-01-01" for example. Or explicitly set cache to true. -- David ;-) Twitter : @dadoonet / @elasticsearchfr / @scrutmydocs Le 1 août 2014 à 06:25, Wang Yong <[email protected]> a écrit : Thank you david 😊 most of my query looks like: { "filtered": { "query": { "match_all": {} }, "filter": { "range": { "timestamp": { "from": 1403567280, "to": 1403567340, "include_lower": true, "include_upper": false } } } }, "facets" : { "val": { "statistical": { "field": "val" } } } } Sent from Surface From: David Pilato Sent: Friday, August 1, 2014 10:43 AM To: [email protected] Well. I guess it depends on your query. What does it look like? -- David ;-) Twitter : @dadoonet / @elasticsearchfr / @scrutmydocs Le 1 août 2014 à 04:14, "Wang Yong" <[email protected]> a écrit : Hi folks, I have an index storing lots of time serial data. The data are put into index by : curl -XPUT 'localhost:9200/testindex/action1/1?pretty' -d ' { "val": 23, "timestamp": 1406822400 }' And the only thing I search in this index is histogram facet in a very short time range, like “recent 5 min”. I found that the performance was pretty good at first. But when the index get bigger, the performance dropped to unacceptable. I found the IO maybe the bottleneck by checking the result of iostat. My question is, even I only facet in a very short time range, why the size of index has so big impact on the performance of such query? Do I have to use daily index, just like logstash? -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/00fd01cfad2e%2457938420%2406ba8c60%24%40gmail.com. For more options, visit https://groups.google.com/d/optout. -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/0D3FCF70-13F5-4E92-8407-47E907736F79%40pilato.fr. For more options, visit https://groups.google.com/d/optout. -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/53db1703.a567440a.7a19.36b5%40mx.google.com. For more options, visit https://groups.google.com/d/optout. -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/93F415A4-AF53-4EAF-999E-07C5CB1BE6C5%40pilato.fr. For more options, visit https://groups.google.com/d/optout. -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/53db6dbc.05f4440a.1bfb.622a%40mx.google.com. For more options, visit https://groups.google.com/d/optout.
