It could be really interesting to have a full script on gist.github.com which reproduce your initial error. I think we should not fail with an error such as: `TokenStream contract violation: close() call missing`.
So, if you can create that script, that would be awesome. Thanks! -- David Pilato | Technical Advocate | Elasticsearch.com @dadoonet | @elasticsearchfr Le 17 février 2014 à 08:03:32, Zhihai Xian ([email protected]) a écrit: Hi David I think I solve the problem by separating the full search query into match query (_search) and suggest query (_suggest), The search result and suggest result both looks fine so far. my original full search string is shown below http://searchnode:9200/es_search/article,fullpagepdf,graphics,photographics POST _search { "from" : 0, "size" : 10, "query" : { "filtered" : { "query" : { "multi_match" : { "query" : "raise the capabilities of Singaporeans as well as to calibrate the existing work pass framework to ensure that foreign work", "fields" : [ "headline_en^4", "headline_cn^4", "byline_en", "byline_cn", "caption_en^2", "caption_cn^2", "body_en^3", "body_cn^3" ] , "analyzer": "phrase_analyzer" } }, "filter" : { "and" : { "filters" : [ { "term" : { "wires" : "1" } }, { "term" : { "language" : "en" } }, { "or" : { "filters" : [ { "terms" : { "hideinasiaone" : [ "no", "NO", "No" ] } }, { "and" : { "filters" : [ { "term" : { "language" : "zh" } }, { "range" : { "publicationdate" : { "from" : null, "to" : "2008-12-31", "include_lower" : true, "include_upper" : true } } } ] } }, { "terms" : { "sourcetype" : [ "ad_metadata", "fullpagepdf", "photographics"] } } ] } } ] } } } }, "sort" : [ { "_score" : { "order" : "desc" } } ], "facets" : { "Company" : { "terms" : { "field" : "facet_company", "size" : 20, "exclude" : [ "" ], "order" : "count" } }, "Section" : { "terms" : { "field" : "facet_pagename", "size" : 20, "exclude" : [ "" ], "order" : "count" } }, "Publication" : { "terms" : { "field" : "publication", "size" : 20, "exclude" : [ "" ], "order" : "count" } }, "Advertiser" : { "terms" : { "field" : "facet_advertiser", "size" : 20, "exclude" : [ "" ], "order" : "count" } } }, "highlight" : { "pre_tags" : [ "<span class=\"highlighting_text\">" ], "post_tags" : [ "</span>" ], "fields" : { "headline_en" : { "fragment_size" : 200, "number_of_fragments" : 1 }, "body_en" : { "fragment_size" : 350, "number_of_fragments" : 1 }, "pdfbody_en" : { "fragment_size" : 350, "number_of_fragments" : 1 } } }, "suggest" : { "text" : "raise the capabilities of Singaporeans as well as to calibrate the existing work pass framework to ensure that foreign worker", "search_suggest" : { "phrase" : { "analyzer" : "suggestion_analyzer", "field" : "suggest_phrase_en", "size" : 3, "shard_size" : 3, "real_word_error_likelihood" : 0.05, "confidence" : 0.1, "max_errors" : 0.9, "gram_size" : 5, "direct_generator" : [ { "field" : "suggest_phrase_en", "suggest_mode" : "always", "size" : 3, "max_edits" : 2, "min_word_length" : 3 }, { "field" : "reverse_phrase", "suggest_mode" : "always", "min_word_length" : 3, "pre_filter" : "reverse_analyzer", "post_filter" : "reverse_analyzer" } ] } } } } and now I moved the below phrase suggestion query out of the full query search and do a single phrase suggest search as below http://searchnode:9200/es_search/ POST _suggest { "text" : "raise the capabilities of Singaporeans as well as to calibrate the existing work pass framework to ensure that foreign worker", "search_suggest" : { "phrase" : { "analyzer" : "suggestion_analyzer", "field" : "suggest_phrase_en", "size" : 2, "shard_size" : 2, "real_word_error_likelihood" : 0.05, "confidence" : 0.1, "max_errors" : 0.9, "gram_size" : 5, "direct_generator" : [ { "field" : "suggest_phrase_en", "suggest_mode" : "always", "size" : 3, "max_edits" : 2, "min_word_length" : 3 }, { "field" : "reverse_phrase", "suggest_mode" : "always", "min_word_length" : 3, "pre_filter": "reverse_analyzer", "post_filter" : "reverse_analyzer" } ] } } } But I am not sure if this is a correct approach for my issue. Thanks On Monday, February 17, 2014 1:18:21 PM UTC+8, David Pilato wrote: Could you gist a full curl recreation to reproduce your issue? -- David ;-) Twitter : @dadoonet / @elasticsearchfr / @scrutmydocs Le 17 févr. 2014 à 02:21, Zhihai Xian <[email protected]> a écrit : Hi David, I did not use any plugins The following is my actual suggestion search query : "suggest" : { "text" : "ationel stadiem", "simple_phrase" : { "phrase" : { "analyzer" : "suggestion_analyzer", "field" : "suggest_phrase_en", "size" : 2, "shard_size" : 2, "real_word_error_likelihood" : 0.05, "confidence" : 0.1, "max_errors" : 0.9, "gram_size" : 5, "direct_generator" : [ { "field" : "suggest_phrase_en", "suggest_mode" : "always", "size" : 3, "max_edits" : 2, "min_word_length" : 3 }, { "field" : "reverse_phrase", "suggest_mode" : "always", "min_word_length" : 3, "pre_filter": "reverse_analyzer", "post_filter" : "reverse_analyzer" } ] } } Setting in Analyzer: "suggestion_analyzer": { "type" : "custom", "tokenizer": "standard", "filter":[ "standard", "asciifolding", "word_delimiter", "text_shingle", "lowercase", "stop", "en_GB" ] }, "reverse_analyzer": { "type": "custom", "tokenizer": "standard", "filter": [ "reverse" ] }, Thanks On Sunday, February 16, 2014 9:40:06 PM UTC+8, David Pilato wrote: I have seen that in past but with analysis plugins. Any chance you are using a plugin and did not update it ? -- David ;-) Twitter : @dadoonet / @elasticsearchfr / @scrutmydocs Le 16 févr. 2014 à 14:27, Zhihai Xian <[email protected]> a écrit : Hi After upgrading to elasticsearch 1.0.0, I got a suggestion search error: RemoteTransportException[[data98][inet[/172.28.96.98:9300]][search/phase/query]]; nested: IllegalStateException[TokenStream contract violation: close() call missing]; }] I found that the problem come with the suggestion query: "suggest" : { "text" : "Xor the Got-Jewel", "simple_phrase" : { "phrase" : { "analyzer" : "body", "field" : "bigram", "size" : 4, "real_word_error_likelihood" : 0.95, "confidence" : 2.0, "gram_size" : 2, "direct_generator" : [ { "field" : "body", "suggest_mode" : "always", "min_word_length" : 1 }, { "field" : "reverse", "suggest_mode" : "always", "min_word_length" : 1, "pre_filter" : "reverse", "post_filter" : "reverse" } ] } } } If I remove the last two lines for pre_filter and post_filter, the search will be fine, but the suggestion result will not be expected. "pre_filter" : "reverse", "post_filter" : "reverse" Is there any solution on the above issue? -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/a6ab296e-f49e-4fcc-a7bc-a1ca32f0988e%40googlegroups.com. For more options, visit https://groups.google.com/groups/opt_out. -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/52eec1ed-cf39-42b6-a308-084770d4823d%40googlegroups.com. For more options, visit https://groups.google.com/groups/opt_out. -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/f5d95a4b-e59a-48e1-b3e2-08028d6da802%40googlegroups.com. For more options, visit https://groups.google.com/groups/opt_out. -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/etPan.5301ca4a.3006c83e.f2%40MacBook-Air-de-David.local. For more options, visit https://groups.google.com/groups/opt_out.
