The reason is that field names are stored in a hash where the order of the
keys depend on JVM implementation. I.e. a Java 7 JVM uses other hashes than
a Java 8 JVM.
ES iterates over the field names in a map to find the first matching field
"name" if the field name is not a full path, and you found a case where two
fields are ending with "name" but their key position in the Java map is
different across JVM implementations.
You have two options:
- never use same field names in ES structured JSON objects
- always use full path to reference the field you want
Full path example:
POST /gdpdm/candidate_gene_list_enrichment/_search
{
"explain": true,
"query": {
"filtered": {
"filter": {
"bool": {
"must": [
{
"term": {
"candidategenelist.id": "20"
}
},
{
"term": {
"status": "Finished"
}
}
]
}
},
"query": {
"multi_match": {
"fields": [
"study_.name^3.5"
],
"query": "Flowering"
}
}
}
}
}
But in all cases, this is a bug in ES, and I suggest to open an issue.
Jörg
On Fri, Oct 3, 2014 at 10:04 PM, [email protected] <
[email protected]> wrote:
> Yes, I can reproduce it now....
>
> ... will investigate. Interesting issue.
>
> Jörg
>
> On Fri, Oct 3, 2014 at 9:48 PM, Ümit Seren <[email protected]> wrote:
>
>> Can you try to do the search on
>> POST /gdpdm/candidate_gene_list_enrichment/_search :
>>
>> POST /gdpdm/candidate_gene_list_enrichment/_search
>> {
>> "query": {
>> "multi_match": {
>> "query": "Flowering",
>> "fields": [
>> "name^3.5"
>> ]
>> }
>> },
>> "post_filter": {
>> "bool": {
>> "must": [
>> {
>> "term": {
>> "candidategenelist.id": "20"
>> }
>> },
>> {
>> "term": {
>> "status": "Finished"
>> }
>> }
>> ]
>> }
>> }
>> }
>>
>> On Fri, Oct 3, 2014 at 9:35 PM, [email protected] <
>> [email protected]> wrote:
>>
>>> Thanks, I could load the data, but can not find anything.
>>>
>>> Your query gave 0 hits but I found this simplified query works:
>>>
>>> POST /_search
>>> {
>>> "query" : {
>>> "multi_match" : {
>>> "query" : "Flowering",
>>> "fields": [
>>> "name"
>>> ]
>>> }
>>> }
>>> }
>>>
>>>
>>> JDK 7 --> 1 hit
>>>
>>> "version": "1.7.0_67",
>>> "vm_name": "Java HotSpot(TM) 64-Bit Server VM",
>>> "vm_version": "24.65-b04",
>>> "vm_vendor": "Oracle Corporation"
>>>
>>> JDK 8 --> 1 hit
>>>
>>> "version": "1.8.0_20",
>>> "vm_name": "Java HotSpot(TM) 64-Bit Server VM",
>>> "vm_version": "25.20-b23",
>>> "vm_vendor": "Oracle Corporation",
>>>
>>>
>>> on Mac OS X.
>>>
>>> So there must be some other fishy thing going on at your cluster...
>>>
>>> Jörg
>>>
>>>
>>>
>>> On Fri, Oct 3, 2014 at 8:19 PM, Ümit Seren <[email protected]>
>>> wrote:
>>>
>>>> Here is the tar.gz folder of the entire elasticsearch data.
>>>> let me know if you need anything elese
>>>>
>>>> https://drive.google.com/file/d/0BwYlrRGOfVARbzkwb2FEUmxlRFU/view?usp=sharing
>>>>
>>>> On Fri, Oct 3, 2014 at 5:00 PM, [email protected] <
>>>> [email protected]> wrote:
>>>>
>>>>> I would love to reproduce it, do you have some test data available for
>>>>> a full reproduction?
>>>>>
>>>>> Thanks,
>>>>>
>>>>> Jörg
>>>>>
>>>>> On Fri, Oct 3, 2014 at 4:21 PM, Ümit Seren <[email protected]>
>>>>> wrote:
>>>>>
>>>>>> I have a weird problem when doing a simple query using different JDK
>>>>>> versions with the same indexed data.
>>>>>>
>>>>>> I have 2 docker images based on the official dockerfiles (
>>>>>> http://dockerfile.github.io/#/elasticsearch).
>>>>>> One image has jdk7u67 installed and one has jdk8u20 installed. Both
>>>>>> use the same index data (mounted as a volume) and have the same
>>>>>> elasticsearch installed (1.3.4)
>>>>>> Of course I only run one instance at a time
>>>>>>
>>>>>> When I issue following search request I get different results (1 hit
>>>>>> with jdk7 and 5 hits with jdk8):
>>>>>>
>>>>>> POST /INDEX/MyType/_search
>>>>>> {
>>>>>> "from": 0,
>>>>>> "size": 50,
>>>>>> "query": {
>>>>>> "multi_match": {
>>>>>> "query": "Flowering",
>>>>>> "fields": [
>>>>>> "name^3.5"
>>>>>> ]
>>>>>> }
>>>>>> },
>>>>>> "post_filter": {
>>>>>> "bool": {
>>>>>> "must": [
>>>>>> {
>>>>>> "term": {
>>>>>> "candidategenelist.id": "20"
>>>>>> }
>>>>>> },
>>>>>> {
>>>>>> "term": {
>>>>>> "status": "Finished"
>>>>>> }
>>>>>> }
>>>>>> ]
>>>>>> }
>>>>>> }
>>>>>> }
>>>>>>
>>>>>> The “MyType” document has a mapping.
>>>>>> candidategenelist is a nested type:
>>>>>>
>>>>>> candidategenelist: {
>>>>>> properties: {
>>>>>> id: {
>>>>>> type: integer
>>>>>> }
>>>>>> name: {
>>>>>> type: string
>>>>>> }
>>>>>>
>>>>>> There are a couple of additional nested field types with a similar
>>>>>> setup (id: integer, name: string)
>>>>>>
>>>>>> With jdk7u67 I get 1 hit:
>>>>>>
>>>>>> Relevant explanation:
>>>>>> description”: “weight(study_.name:flowering in 74274)
>>>>>> [PerFieldSimilarity], result of:”,
>>>>>>
>>>>>> With jdk8u20 I get 5 hits:
>>>>>>
>>>>>> Relevant explanation:
>>>>>> “description”: “weight(candidategenelist.name:flowering in 74274)
>>>>>> [PerFieldSimilarity], result of:”,
>>>>>> “description”: “weight(candidategenelist.name:flowering in 6614)
>>>>>> [PerFieldSimilarity], result of:”,
>>>>>>
>>>>>> “description”: “weight(candidategenelist.name:flowering in 56078)
>>>>>> [PerFieldSimilarity], result of:”,
>>>>>> “description”: “weight(candidategenelist.name:flowering in 6674)
>>>>>> [PerFieldSimilarity], result of:”,
>>>>>> “description”: “weight(candidategenelist.name:flowering in 6646)
>>>>>> [PerFieldSimilarity], result of:”,
>>>>>>
>>>>>> So according to the explanation, the only difference is that with
>>>>>> JDK7u67 it matches “name” in the nested study_ field whereas with JDK8u20
>>>>>> it matches the “name” in candidategenelist.
>>>>>>
>>>>>> I really don’t know why there are different query results when the
>>>>>> only difference is the JDK version?!
>>>>>>
>>>>>>
>>>>>> --
>>>>>> You received this message because you are subscribed to the Google
>>>>>> Groups "elasticsearch" group.
>>>>>> To unsubscribe from this group and stop receiving emails from it,
>>>>>> send an email to [email protected].
>>>>>> To view this discussion on the web visit
>>>>>> https://groups.google.com/d/msgid/elasticsearch/66af6726-d390-4f3f-8620-6ef2cff5397c%40googlegroups.com
>>>>>> <https://groups.google.com/d/msgid/elasticsearch/66af6726-d390-4f3f-8620-6ef2cff5397c%40googlegroups.com?utm_medium=email&utm_source=footer>
>>>>>> .
>>>>>> For more options, visit https://groups.google.com/d/optout.
>>>>>>
>>>>>
>>>>> --
>>>>> You received this message because you are subscribed to a topic in the
>>>>> Google Groups "elasticsearch" group.
>>>>> To unsubscribe from this topic, visit
>>>>> https://groups.google.com/d/topic/elasticsearch/3m9vPPokslw/unsubscribe
>>>>> .
>>>>> To unsubscribe from this group and all its topics, send an email to
>>>>> [email protected].
>>>>> To view this discussion on the web visit
>>>>> https://groups.google.com/d/msgid/elasticsearch/CAKdsXoEkKm-6Z%2B65jWANAACC8nta9JexLjtjaLub5-fS5pSsyg%40mail.gmail.com
>>>>> <https://groups.google.com/d/msgid/elasticsearch/CAKdsXoEkKm-6Z%2B65jWANAACC8nta9JexLjtjaLub5-fS5pSsyg%40mail.gmail.com?utm_medium=email&utm_source=footer>
>>>>> .
>>>>>
>>>>> For more options, visit https://groups.google.com/d/optout.
>>>>>
>>>>
>>>> --
>>>> You received this message because you are subscribed to the Google
>>>> Groups "elasticsearch" group.
>>>> To unsubscribe from this group and stop receiving emails from it, send
>>>> an email to [email protected].
>>>> To view this discussion on the web visit
>>>> https://groups.google.com/d/msgid/elasticsearch/CANBYW4CEmwYd1iy%3DNWDK8iSiOh%3DoeMWGZD4xXw1L9yc1phmf_A%40mail.gmail.com
>>>> <https://groups.google.com/d/msgid/elasticsearch/CANBYW4CEmwYd1iy%3DNWDK8iSiOh%3DoeMWGZD4xXw1L9yc1phmf_A%40mail.gmail.com?utm_medium=email&utm_source=footer>
>>>> .
>>>>
>>>> For more options, visit https://groups.google.com/d/optout.
>>>>
>>>
>>> --
>>> You received this message because you are subscribed to a topic in the
>>> Google Groups "elasticsearch" group.
>>> To unsubscribe from this topic, visit
>>> https://groups.google.com/d/topic/elasticsearch/3m9vPPokslw/unsubscribe.
>>> To unsubscribe from this group and all its topics, send an email to
>>> [email protected].
>>> To view this discussion on the web visit
>>> https://groups.google.com/d/msgid/elasticsearch/CAKdsXoGwmxNUuM%3DW8YF7edDAf64vO%2Bco0dy%2BapLm8bT9RR8LJg%40mail.gmail.com
>>> <https://groups.google.com/d/msgid/elasticsearch/CAKdsXoGwmxNUuM%3DW8YF7edDAf64vO%2Bco0dy%2BapLm8bT9RR8LJg%40mail.gmail.com?utm_medium=email&utm_source=footer>
>>> .
>>>
>>> For more options, visit https://groups.google.com/d/optout.
>>>
>>
>> --
>> You received this message because you are subscribed to the Google Groups
>> "elasticsearch" group.
>> To unsubscribe from this group and stop receiving emails from it, send an
>> email to [email protected].
>> To view this discussion on the web visit
>> https://groups.google.com/d/msgid/elasticsearch/CANBYW4Dfp%2BgvmwJvWLuFE56D0GSg5NaafFERcQhnpJa8W2qo8Q%40mail.gmail.com
>> <https://groups.google.com/d/msgid/elasticsearch/CANBYW4Dfp%2BgvmwJvWLuFE56D0GSg5NaafFERcQhnpJa8W2qo8Q%40mail.gmail.com?utm_medium=email&utm_source=footer>
>> .
>>
>> For more options, visit https://groups.google.com/d/optout.
>>
>
>
--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/CAKdsXoGz%3DCvR7EtyOaBvmrOnStLVCEkz6M8RsgxOn1tz6iZ%3DuA%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.
