Hi David As requested, I created a gist on gist.github.com* . This is my first time to create somthing on gist. I am not sure if it is correct format. The following is the url :*
https://gist.github.com/donmis/9065851 <http://gist.github.com/> Regards On Mon, Feb 17, 2014 at 4:37 PM, David Pilato <[email protected]> wrote: > It could be really interesting to have a full script on gist.github.com which > reproduce your initial error. > I think we should not fail with an error such as: `TokenStream contract > violation: close() call missing`. > > So, if you can create that script, that would be awesome. > > Thanks! > > -- > *David Pilato* | *Technical Advocate* | *Elasticsearch.com* > @dadoonet <https://twitter.com/dadoonet> | > @elasticsearchfr<https://twitter.com/elasticsearchfr> > > > Le 17 février 2014 à 08:03:32, Zhihai Xian ([email protected]) a écrit: > > Hi David > > I think I solve the problem by separating the full search query into match > query (_search) and suggest query (_suggest), > The search result and suggest result both looks fine so far. > > my original full search string is shown below > > > http://searchnode:9200/es_search/article,fullpagepdf,graphics,photographics > POST _search > { > "from" : 0, > "size" : 10, > "query" : { > "filtered" : { > "query" : { > "multi_match" : { > "query" : "raise the capabilities of Singaporeans as well as to > calibrate the existing work pass framework to ensure that foreign work", > "fields" : [ "headline_en^4", "headline_cn^4", "byline_en", > "byline_cn", "caption_en^2", "caption_cn^2", "body_en^3", "body_cn^3" ] , > "analyzer": "phrase_analyzer" > } > }, > "filter" : { > "and" : { > "filters" : [ { > "term" : { > "wires" : "1" > } > }, { > "term" : { > "language" : "en" > } > }, { > "or" : { > "filters" : [ { > "terms" : { > "hideinasiaone" : [ "no", "NO", "No" ] > } > }, { > "and" : { > "filters" : [ { > "term" : { > "language" : "zh" > } > }, { > "range" : { > "publicationdate" : { > "from" : null, > "to" : "2008-12-31", > "include_lower" : true, > "include_upper" : true > } > } > } ] > } > }, { > "terms" : { > "sourcetype" : [ "ad_metadata", "fullpagepdf", > "photographics"] > } > } ] > } > } ] > } > } > } > }, > "sort" : [ { > "_score" : { > "order" : "desc" > } > } ], > "facets" : { > "Company" : { > "terms" : { > "field" : "facet_company", > "size" : 20, > "exclude" : [ "" ], > "order" : "count" > } > }, > "Section" : { > "terms" : { > "field" : "facet_pagename", > "size" : 20, > "exclude" : [ "" ], > "order" : "count" > } > }, > "Publication" : { > "terms" : { > "field" : "publication", > "size" : 20, > "exclude" : [ "" ], > "order" : "count" > } > }, > "Advertiser" : { > "terms" : { > "field" : "facet_advertiser", > "size" : 20, > "exclude" : [ "" ], > "order" : "count" > } > } > }, > "highlight" : { > "pre_tags" : [ "<span class=\"highlighting_text\">" ], > "post_tags" : [ "</span>" ], > "fields" : { > "headline_en" : { > "fragment_size" : 200, > "number_of_fragments" : 1 > }, > "body_en" : { > "fragment_size" : 350, > "number_of_fragments" : 1 > }, > "pdfbody_en" : { > "fragment_size" : 350, > "number_of_fragments" : 1 > } > } > }, > "suggest" : { > "text" : "raise the capabilities of Singaporeans as well as to > calibrate the existing work pass framework to ensure that foreign worker", > "search_suggest" : { > "phrase" : { > "analyzer" : "suggestion_analyzer", > "field" : "suggest_phrase_en", > "size" : 3, > "shard_size" : 3, > "real_word_error_likelihood" : 0.05, > "confidence" : 0.1, > "max_errors" : 0.9, > "gram_size" : 5, > "direct_generator" : [ { > "field" : "suggest_phrase_en", > "suggest_mode" : "always", > "size" : 3, > "max_edits" : 2, > "min_word_length" : 3 > }, { > "field" : "reverse_phrase", > "suggest_mode" : "always", > "min_word_length" : 3, > "pre_filter" : "reverse_analyzer", > "post_filter" : "reverse_analyzer" > } ] > } > } > } > } > > and now I moved the below phrase suggestion query out of the full query > search and do a single phrase suggest search as below > > http://searchnode:9200/es_search/ > POST _suggest > { > > "text" : "raise the capabilities of Singaporeans as well as to > calibrate the existing work pass framework to ensure that foreign worker", > "search_suggest" : { > "phrase" : { > "analyzer" : "suggestion_analyzer", > "field" : "suggest_phrase_en", > "size" : 2, > "shard_size" : 2, > "real_word_error_likelihood" : 0.05, > "confidence" : 0.1, > "max_errors" : 0.9, > "gram_size" : 5, > "direct_generator" : [ { > "field" : "suggest_phrase_en", > "suggest_mode" : "always", > "size" : 3, > "max_edits" : 2, > "min_word_length" : 3 > }, { > "field" : "reverse_phrase", > "suggest_mode" : "always", > "min_word_length" : 3, > "pre_filter": "reverse_analyzer", > "post_filter" : "reverse_analyzer" > } ] > } > } > } > > But I am not sure if this is a correct approach for my issue. > > Thanks > > On Monday, February 17, 2014 1:18:21 PM UTC+8, David Pilato wrote: >> >> Could you gist a full curl recreation to reproduce your issue? >> >> -- >> David ;-) >> Twitter : @dadoonet / @elasticsearchfr / @scrutmydocs >> >> >> Le 17 févr. 2014 à 02:21, Zhihai Xian <[email protected]> a écrit : >> >> Hi David, >> >> I did not use any plugins >> >> The following is my actual suggestion search query : >> >> "suggest" : { >> "text" : "ationel stadiem", >> "simple_phrase" : { >> "phrase" : { >> "analyzer" : "suggestion_analyzer", >> "field" : "suggest_phrase_en", >> "size" : 2, >> "shard_size" : 2, >> "real_word_error_likelihood" : 0.05, >> "confidence" : 0.1, >> "max_errors" : 0.9, >> "gram_size" : 5, >> "direct_generator" : [ { >> "field" : "suggest_phrase_en", >> "suggest_mode" : "always", >> "size" : 3, >> "max_edits" : 2, >> "min_word_length" : 3 >> }, { >> "field" : "reverse_phrase", >> "suggest_mode" : "always", >> "min_word_length" : 3, >> "pre_filter": "reverse_analyzer", >> "post_filter" : "reverse_analyzer" >> } ] >> } >> } >> >> Setting in Analyzer: >> >> "suggestion_analyzer": { >> "type" : "custom", >> "tokenizer": "standard", >> "filter":[ >> "standard", >> "asciifolding", >> "word_delimiter", >> "text_shingle", >> "lowercase", >> "stop", >> "en_GB" >> >> ] >> }, >> "reverse_analyzer": { >> "type": "custom", >> "tokenizer": "standard", >> "filter": [ >> "reverse" >> ] >> }, >> >> >> Thanks >> >> >> >> On Sunday, February 16, 2014 9:40:06 PM UTC+8, David Pilato wrote: >>> >>> I have seen that in past but with analysis plugins. >>> Any chance you are using a plugin and did not update it ? >>> >>> -- >>> David ;-) >>> Twitter : @dadoonet / @elasticsearchfr / @scrutmydocs >>> >>> >>> Le 16 févr. 2014 à 14:27, Zhihai Xian <[email protected]> a écrit : >>> >>> Hi >>> >>> After upgrading to elasticsearch 1.0.0, I got a suggestion search error: >>> >>> >>> RemoteTransportException[[data98][inet[/172.28.96.98:9300]][search/phase/query]]; >>> nested: IllegalStateException[TokenStream contract violation: close() >>> call missing]; }] >>> >>> I found that the problem come with the suggestion query: >>> >>> "suggest" : { >>> "text" : "Xor the Got-Jewel", >>> "simple_phrase" : { >>> "phrase" : { >>> "analyzer" : "body", >>> "field" : "bigram", >>> "size" : 4, >>> "real_word_error_likelihood" : 0.95, >>> "confidence" : 2.0, >>> "gram_size" : 2, >>> "direct_generator" : [ { >>> "field" : "body", >>> "suggest_mode" : "always", >>> "min_word_length" : 1 >>> }, { >>> "field" : "reverse", >>> "suggest_mode" : "always", >>> "min_word_length" : 1, >>> "pre_filter" : "reverse", >>> "post_filter" : "reverse" >>> } ] >>> } >>> } >>> } >>> >>> If I remove the last two lines for pre_filter and post_filter, the >>> search will be fine, but the suggestion result will not be expected. >>> >>> "pre_filter" : "reverse", >>> "post_filter" : "reverse" >>> >>> Is there any solution on the above issue? >>> >>> >>> >>> -- >>> You received this message because you are subscribed to the Google >>> Groups "elasticsearch" group. >>> To unsubscribe from this group and stop receiving emails from it, send >>> an email to [email protected]. >>> To view this discussion on the web visit https://groups.google.com/d/ >>> msgid/elasticsearch/a6ab296e-f49e-4fcc-a7bc-a1ca32f0988e% >>> 40googlegroups.com. >>> For more options, visit https://groups.google.com/groups/opt_out. >>> >> -- >> You received this message because you are subscribed to the Google Groups >> "elasticsearch" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to [email protected]. >> To view this discussion on the web visit https://groups.google.com/d/ >> msgid/elasticsearch/52eec1ed-cf39-42b6-a308-084770d4823d% >> 40googlegroups.com. >> For more options, visit https://groups.google.com/groups/opt_out. >> > -- > You received this message because you are subscribed to the Google Groups > "elasticsearch" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [email protected]. > > To view this discussion on the web visit > https://groups.google.com/d/msgid/elasticsearch/f5d95a4b-e59a-48e1-b3e2-08028d6da802%40googlegroups.com > . > For more options, visit https://groups.google.com/groups/opt_out. > > -- > You received this message because you are subscribed to a topic in the > Google Groups "elasticsearch" group. > To unsubscribe from this topic, visit > https://groups.google.com/d/topic/elasticsearch/UmLSl9_cTU4/unsubscribe. > To unsubscribe from this group and all its topics, send an email to > [email protected]. > To view this discussion on the web visit > https://groups.google.com/d/msgid/elasticsearch/etPan.5301ca4a.3006c83e.f2%40MacBook-Air-de-David.local > . > > For more options, visit https://groups.google.com/groups/opt_out. > -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAGGrwQFXEb_WJ7%3DvoQk7QqKvzjB4Lv5Q%3DqJ%2BbV-Zm2Q4E7R00Q%40mail.gmail.com. For more options, visit https://groups.google.com/groups/opt_out.
