Data that i am storing is a tweet response from twitter which flows in via
a streaming API, so the inserts are quite fast (roughly 50-80 inserts/sec).
query look something like this
{
"query": {
"filtered": {
"filter": {
"and": [{ //i have a range slider in the UI which
selects tweets on rank range
"range": {
"rank": {
"from": "500",
"to": "1000",
"include_upper": true
}
}
},
{ //if there are custom query terms provided by the
quer
"query": {
"terms": {
"entities": ["searchString"],
"minimum_should_match": # of search terms
}
}
},
{
"missing": {
"field": "in_reply_to_status_id",
"existence": true,
"null_value": true
}
},
{ //have a country field query
"query": {
"term": {
"user.country_new": "in"
}
}
}]
}
}
},
"from": "0", //paging
"size": "50",
"sort": [{ //sorting
"created_at": {
"order": "desc"
}
}],
"aggregations": {
"hashtags_freq": [{ //get hashtag freqency aggr in
result set
"terms": {
"field": "entities",
"include": "(#[a-zA-Z][a-zA-Z0-9_-]+)",
"size": 0
},
"aggregations": {
"unique_users": {
"terms": {
"field": "user.screen_name",
"size": 0
}
}
}
}],
"user_tweet_frequency": { //# of tweets by all the users in
resultset
"terms": {
"field": "user.screen_name",
"size": 0
}
}
}
}
this query is fired by all the users that come to our site and it is called
with varying values of rank range
On Tue, Apr 29, 2014 at 7:10 PM, Radu Gheorghe
<[email protected]>wrote:
> Hello Abrar,
>
> The answer to your questions depends a lot on how your data and queries
> look like, how often you run them and how often new data is indexed. You
> could paste those details here, but I don't think anyone could give you a
> definite answer, maybe more of a guesstimate based on experience with
> similar patterns.
>
> The best way to find out is to install some performance monitoring tool
> (there are many out there, you can find one by clicking the link in my
> signature) and start running tests with production-like data and queries.
> And then you'll see how much your machine can handle and where the
> bottlenecks are.
>
> Best regards,
> Radu
> --
> Performance Monitoring * Log Analytics * Search Analytics
> Solr & Elasticsearch Support * http://sematext.com/
>
>
> On Tue, Apr 29, 2014 at 4:02 PM, Abrar Sheikh <[email protected]>wrote:
>
>> Hi,
>>
>> I have a single aws EC2 large instance with 7.5 GB ram and 100 GB
>> harddrive dual core 2.6 GHz. My elastic instance on a average has around
>> 10,000,000 records. I use somewhat complex queries. I am calling elastic
>> apis from my PHP code which is exposed as a rest service(needed to do some
>> post processing of data). my question is how much load can my server
>> handle and at what point do i shift to a multi node architecture. What
>> effect does # of shards and replication have on performance. With my
>> current system configuration how many queries per second(qps) can my
>> elastic search handle?
>>
>> Thanks and Regards,
>> Abrar.
>>
>> --
>> You received this message because you are subscribed to the Google Groups
>> "elasticsearch" group.
>> To unsubscribe from this group and stop receiving emails from it, send an
>> email to [email protected].
>>
>> To view this discussion on the web visit
>> https://groups.google.com/d/msgid/elasticsearch/49f1dbbe-42e6-445e-ba6e-b9d358d8908f%40googlegroups.com<https://groups.google.com/d/msgid/elasticsearch/49f1dbbe-42e6-445e-ba6e-b9d358d8908f%40googlegroups.com?utm_medium=email&utm_source=footer>
>> .
>> For more options, visit https://groups.google.com/d/optout.
>>
>
> --
> You received this message because you are subscribed to a topic in the
> Google Groups "elasticsearch" group.
> To unsubscribe from this topic, visit
> https://groups.google.com/d/topic/elasticsearch/HjuhwaBq_RQ/unsubscribe.
> To unsubscribe from this group and all its topics, send an email to
> [email protected].
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/elasticsearch/CAHXA0_0TiMyRc-9qcPTFWYT6eHYWQc_PWd96RbFy6eNk_aAc_Q%40mail.gmail.com<https://groups.google.com/d/msgid/elasticsearch/CAHXA0_0TiMyRc-9qcPTFWYT6eHYWQc_PWd96RbFy6eNk_aAc_Q%40mail.gmail.com?utm_medium=email&utm_source=footer>
> .
>
> For more options, visit https://groups.google.com/d/optout.
>
--
Abrar Sheikh
--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/CAA2_QZ9RtDt2-xKFC5Gkro4rE%2B8mQk%2Br-Uj2es4EruZ-HyCO2g%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.
