On Mon, Dec 1, 2014 at 10:42 PM, N Bijalwan <[email protected]> wrote:
> We are using manifolcf to crawl web pages and then index them through > Elastic search. > > Is there way to get only few lines that contain the searched keyword in > response of elastic search query instead of whole content. Like we get in > google search. > Solution we are trying: Reference > http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/docs-termvectors.html > <https://email2010.searshc.com/owa/redir.aspx?C=VTBiULXBnE-XzIuMedjuaGPHLq134dEI2v0GWL91l1pzNGDfDsz11x4ckLumFc5e2EMae1ef3sk.&URL=http%3a%2f%2fwww.elasticsearch.org%2fguide%2fen%2felasticsearch%2freference%2fcurrent%2fdocs-termvectors.html> > > > http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/search-request-highlighting.html > <https://email2010.searshc.com/owa/redir.aspx?C=VTBiULXBnE-XzIuMedjuaGPHLq134dEI2v0GWL91l1pzNGDfDsz11x4ckLumFc5e2EMae1ef3sk.&URL=http%3a%2f%2fwww.elasticsearch.org%2fguide%2fen%2felasticsearch%2freference%2fcurrent%2fsearch-request-highlighting.html> > > We are trying to do mapping like: > > { > "mappings": { > "test": { > "properties": { > "file": { > "type": "attachment", > "path": "full", > "fields": { > "_content_type": { > "type": "string", > "store": true > }, > "_name": { > "type": "string", > "store": true > }, > "content": { > "type": "string", > "term_vector": "with_positions_offsets_payloads", > "store" : true, > "index_analyzer" : "fulltext_analyzer" > > } > }, > "store" : true, > "term_vector" : "with_positions_offsets_payloads" > } > } > } > }, > "settings" : { > "index" : { > "number_of_shards" : 1, > "number_of_replicas" : 0 > }, > "analysis": { > "analyzer": { > "fulltext_analyzer": { > "type": "custom", > "tokenizer": "whitespace", > "filter": [ > "lowercase", > "type_as_payload" > ] > } > } > } > } > } > > > and then query like: > > { > "query": { > "match": { > "file": "CROWLEY" > } > }, > "highlight" : { > "fields" : { > "file" : {"fragment_size" : 150, "number_of_fragments" : 3} > } > } > } > > But we don't get highlight in response instead we get whole content in > response. > > Any help is appreciated. > > > You have to specify the full path of what you want to highlight. Like {"fields": {"file.content"}}. You'll also get the whole content back by default unless you turn it off with "_source": false in the request. You can filter it to only get the parts that you want: http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/search-request-source-filtering.html You'll find that using term vectors costs a ton of extra space and therefor IO on write, merge, and search. You can reduce it by switching to the postings highlighter if your content is prose. The plain highlighter isn't an option if your content might get long. Each of those highlighters require different stored data and produce different shaped results. You could also try the experimental highlighter (if you are using Elasticsearch 1.3.X) - it steals a ton of ideas from the others and is super flexible and quite stable. We wrote it because we couldn't afford the extra space for term vectors but didn't like the posting's highlighter's segmentation rules. Nik -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAPmjWd3St8hp9Mrg7bwB3Ez4ZNy-EuWO3FbGgCQOuu%3DzesB_7g%40mail.gmail.com. For more options, visit https://groups.google.com/d/optout.
