Hey I think I got it!
So in my curl request, I was specifying the two indexes in the rest
endpoint, but I did not specify them in my java search request. My only
other index is a suggest index, and I would think that it would be ignored
when I ran the query, but I guess not.
Thanks a lot for your time. This was definitely one of those bang your
head against the wall issues.
On Thursday, August 21, 2014 11:06:32 AM UTC-4, Elliott Bradshaw wrote:
>
> Hi guys,
>
> Thanks for getting back to me.
>
> Here's the JSON for the query:
>
> { "bool" : {
> "must" : [ {
> "indices" : {
> "indices" : "index1",
> "query" : {
> "filtered" : {
> "query" : {
> "function_score" : {
> "query" : {
> "multi_match" : {
> "query" : "this is my query
> string",
> "fields" : [ "field1^1.2",
> "field2", "field3", "field4", "field5" ],
> "type" : "cross_fields",
> "operator" : "AND",
> "boost" : 1.2,
> "fuzziness" : "AUTO", //does not
> work unfortunately on cross_fields :( ...
> "minimum_should_match" : "75%",
> "tie_breaker" : 0.1
> }
> },
> "gauss" : {
> "fields6" : {
> "origin" : 1,
> "scale" : 25
> }
> }
> }
> },
> "filter" : {
> "and" : {
> "filters" : []
> }
> }
> }
> }
> }
> },
> {
> "indices" : {
> "indices" : "index2",
> "query" : {
> "filtered" : {
> "query" : {
> "multi_match" : {
> "query" : "this is my query string",
> "fields" : [ "field7^1.2", "field6",
> "field9", "field10", "field11" ],
> "type" : "cross_fields",
> "operator" : "AND",
> "boost" : 1.2,
> "fuzziness" : "AUTO", //does not work
> unfortunately on cross_fields :( ...
> "minimum_should_match" : "75%",
> "tie_breaker" : 0.1
> }
> },
> "filter" : {
> "and" : {
> "filters" : []
> }
> }
> }
> }
> }
> }
> ]
> }
>
> This JSON is output to a log by the following Java code:
>
> QueryBuilder query = getQuery();
>
> String json = query.toXContent(jsonBuilder(), null).string();
> logger.debug(json); //Outputs JSON that I copy into my curl body
> (displayed above)
>
> I then execute the query:
>
> logger.debug("Search request beginning");
>
> SearchResponse response = client
> .prepareSearch()
> .setQuery(query)
> .setExplain(false)
> .setSize(10)
> .execute().actionGet();
>
> logger.debug("Search request complete");
>
> The time between the two debug statements is always between 1-2 seconds.
> This is after running many queries via a webservice. The curl request
> always runs in 10-30 ms.
>
> Any other thoughts?
>
> On Wednesday, August 20, 2014 10:23:49 PM UTC-4, vineeth mohan wrote:
>>
>> Hello Elliott ,
>>
>> There could be many other variables here.
>> One is that the first time when you run the query , it would be slow and
>> the second time when you run it , then it is fast.
>> You might have always executed the java code first and then tried the
>> normal curl.
>>
>> Please reproduce the issue in a java code with just the query and paste
>> here.
>> That might help us better debug the issue.
>>
>> Thanks
>> Vineeth
>>
>>
>> On Thu, Aug 21, 2014 at 7:46 AM, Elliott Bradshaw <[email protected]>
>> wrote:
>>
>>> Jorg,
>>>
>>> Thanks for the response. I understand that curl hits the Java API,
>>> that's why the issue is so strange.
>>>
>>> I will try to make some more tweaks tomorrow, and I can send the query
>>> as well. I can't post the java code as it's extensive and spans several
>>> classes (the resultant query is relatively complex). However, I log the
>>> JSON dump of the final QueryBuilder object and use that in my curl request.
>>>
>>> - Elliott
>>>
>>>
>>> On Wednesday, August 20, 2014 2:55:50 PM UTC-4, Jörg Prante wrote:
>>>
>>>> Since curl requests use the Java API, it is impossible to execute this
>>>> faster than Java API.
>>>>
>>>> Can you post the Java code and curl request you use? Have you compared
>>>> the queries sent? What does the query look like? How does the result look
>>>> like?
>>>>
>>>> Without this information it is very hard to discuss any issue, it will
>>>> be just guessing.
>>>>
>>>> Jörg
>>>>
>>>>
>>>> On Wed, Aug 20, 2014 at 8:02 PM, Elliott Bradshaw <[email protected]>
>>>> wrote:
>>>>
>>>>> Hi Vineeth,
>>>>>
>>>>> The system is currently deployed on a Tomcat server and the client is
>>>>> created a single time on application start up. The time I gave of 1-2
>>>>> seconds is for the query alone, as far as I know.
>>>>>
>>>>> Thanks,
>>>>>
>>>>> Elliott
>>>>>
>>>>>
>>>>> On Wednesday, August 20, 2014 12:46:34 PM UTC-4, vineeth mohan wrote:
>>>>>
>>>>>> Hello Eliott ,
>>>>>>
>>>>>> I suspect if this has something to do with client creation.
>>>>>> Can you see the time taken for the query alone and not the client
>>>>>> creation.
>>>>>>
>>>>>> Other possible reasons can be refresh flag ON.
>>>>>> Please check that also.
>>>>>>
>>>>>> Thanks
>>>>>> Vineeth
>>>>>>
>>>>>>
>>>>>>
>>>>>> On Wed, Aug 20, 2014 at 10:11 PM, Elliott Bradshaw <
>>>>>> [email protected]> wrote:
>>>>>>
>>>>>>> Vineeth,
>>>>>>>
>>>>>>> Thanks, but according to the documentation, it looks like sniffing
>>>>>>> is only supported when using the TransportClient. I'm using the
>>>>>>> standard
>>>>>>> Node/Client configuration.
>>>>>>>
>>>>>>>
>>>>>>> On Wednesday, August 20, 2014 12:24:45 PM UTC-4, vineeth mohan wrote:
>>>>>>>
>>>>>>>> Hello Elliot ,
>>>>>>>>
>>>>>>>> Is the sniffing enabled while creating the client.
>>>>>>>> This means that , it will determine which all machines are enabled
>>>>>>>> and does some load balancing.
>>>>>>>> Which means that it needs additional time while creating the client.
>>>>>>>>
>>>>>>>> Thanks
>>>>>>>> Vineeth
>>>>>>>>
>>>>>>>>
>>>>>>>> On Wed, Aug 20, 2014 at 9:39 PM, Elliott Bradshaw <
>>>>>>>> [email protected]> wrote:
>>>>>>>>
>>>>>>>>> I'm joining an existing cluster as a node via the Java API. I
>>>>>>>>> generate a relatively complicated multi-index "indices" query that
>>>>>>>>> combines
>>>>>>>>> the results of two different queries on two modestly sized indexes
>>>>>>>>> (150M
>>>>>>>>> records combined). Execution of actionGet() on this query takes
>>>>>>>>> approximately 1-2 seconds. On the other hand, if I dump the same
>>>>>>>>> query to
>>>>>>>>> JSON via the XContent jsonBuilder() and execute it with curl, it
>>>>>>>>> takes only
>>>>>>>>> 14 ms. Results are identical in both cases.
>>>>>>>>>
>>>>>>>>> Has anyone else had similar problems like this? Any thoughts on
>>>>>>>>> what might be the source of the problem? My next thought was to try
>>>>>>>>> connecting to the cluster via a TransportClient. Not sure if it will
>>>>>>>>> help
>>>>>>>>> or not...
>>>>>>>>>
>>>>>>>>> Thanks!
>>>>>>>>>
>>>>>>>>> - Elliott
>>>>>>>>>
>>>>>>>>> --
>>>>>>>>> You received this message because you are subscribed to the Google
>>>>>>>>> Groups "elasticsearch" group.
>>>>>>>>> To unsubscribe from this group and stop receiving emails from it,
>>>>>>>>> send an email to [email protected].
>>>>>>>>>
>>>>>>>>> To view this discussion on the web visit
>>>>>>>>> https://groups.google.com/d/msgid/elasticsearch/fa4fb57e-3be
>>>>>>>>> b-4e80-8672-1981bcb8c35c%40googlegroups.com
>>>>>>>>> <https://groups.google.com/d/msgid/elasticsearch/fa4fb57e-3beb-4e80-8672-1981bcb8c35c%40googlegroups.com?utm_medium=email&utm_source=footer>
>>>>>>>>> .
>>>>>>>>> For more options, visit https://groups.google.com/d/optout.
>>>>>>>>>
>>>>>>>>
>>>>>>>> --
>>>>>>> You received this message because you are subscribed to the Google
>>>>>>> Groups "elasticsearch" group.
>>>>>>> To unsubscribe from this group and stop receiving emails from it,
>>>>>>> send an email to [email protected].
>>>>>>> To view this discussion on the web visit
>>>>>>> https://groups.google.com/d/msgid/elasticsearch/bfae9628-021
>>>>>>> 2-4b03-8f76-4fb85aab23e9%40googlegroups.com
>>>>>>> <https://groups.google.com/d/msgid/elasticsearch/bfae9628-0212-4b03-8f76-4fb85aab23e9%40googlegroups.com?utm_medium=email&utm_source=footer>
>>>>>>> .
>>>>>>>
>>>>>>> For more options, visit https://groups.google.com/d/optout.
>>>>>>>
>>>>>>
>>>>>> --
>>>>> You received this message because you are subscribed to the Google
>>>>> Groups "elasticsearch" group.
>>>>> To unsubscribe from this group and stop receiving emails from it, send
>>>>> an email to [email protected].
>>>>> To view this discussion on the web visit https://groups.google.com/d/
>>>>> msgid/elasticsearch/16c49c37-4c0c-4b37-85a5-2969aa6c5ee4%
>>>>> 40googlegroups.com
>>>>> <https://groups.google.com/d/msgid/elasticsearch/16c49c37-4c0c-4b37-85a5-2969aa6c5ee4%40googlegroups.com?utm_medium=email&utm_source=footer>
>>>>> .
>>>>>
>>>>> For more options, visit https://groups.google.com/d/optout.
>>>>>
>>>>
>>>> --
>>> You received this message because you are subscribed to the Google
>>> Groups "elasticsearch" group.
>>> To unsubscribe from this group and stop receiving emails from it, send
>>> an email to [email protected].
>>> To view this discussion on the web visit
>>> https://groups.google.com/d/msgid/elasticsearch/cb19a3a8-c8dc-48c8-aa44-ebd718667be3%40googlegroups.com
>>>
>>> <https://groups.google.com/d/msgid/elasticsearch/cb19a3a8-c8dc-48c8-aa44-ebd718667be3%40googlegroups.com?utm_medium=email&utm_source=footer>
>>> .
>>>
>>> For more options, visit https://groups.google.com/d/optout.
>>>
>>
>>
--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/b7b073db-eef5-4680-9d4e-05a261ffd0b3%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
