Thanks for the hint that similarity class should be in the ES lib folder. I will try this to see if that enables my plugin code to have per-field custom similarity.
Payloads are a broad subject. For example, in my plugin, payload filters are missing. Let's assume you use UIMA or some NLP tagging. It would be neat to allow filtering for terms that have certain annotations in the payload, e.g. searching only in nouns. Or, named entity recognition - you could filter searches on particular entity classes. Jörg On Fri, Mar 20, 2015 at 8:03 AM, Devaraja Swami <[email protected]> wrote: > Thanks for your help, Jorg. > > I'd already gotten the similarity and analysis part to work, without using > a plugin. > For analysis, I used the builtin delimited_payload_tokenizer. > For similarity, I used a custom similarity (from my Lucene code) and just > stuck it as a jar file into the lib folder of my ES installation, and > restarted ES. > With this approach, I was able to specify per-field similarity and > per-field analyzers, in my index creation settings. > > For the payload query part, I had to create a PayloadTermQueryBuilder > and PayloadTermQueryParser. > I again added the PayloadTermQueryBuilder to the jar file installed into > the ES lib folder. > But I had to install PayloadTermQueryParser as a plugin, because that's > the only way to tell ES to add it to its index module query parser > mappings. > My plugin is just a simplification of your GitHub plugin, containing only > the PayloadTermQueryParser and PayloadPlugin classes, and the > es-plugin.properties file. > > Many thanks again for your detailed help. > > It would be nice to have all this done in ES core, because I feel payloads > are going be increasingly used, and it has been in Lucene long enough. > > Cheers, > devarajaswami > > On Tuesday, March 17, 2015 at 3:57:09 PM UTC-7, Jörg Prante wrote: >> >> I created an example payload plugin >> >> https://github.com/jprante/elasticsearch-payload >> >> but I can't get a custom per-field similarity to work. Setting up a >> field with a prebuilt similarity works flawlessly, but with a custom one, >> it is not even listed in the mapping. >> >> It looks like SimilarityLookupService fails to find custom similarities. >> >> If someone can help in tracking down the issue, I'd be glad. Maybe I do >> something wrong. >> >> Jörg >> >> On Tue, Mar 17, 2015 at 5:02 PM, Nikolas Everett <[email protected]> >> wrote: >> >>> I imagine the right way to do this is with a plugin but I'm not 100% >>> sure. >>> >>> On Tue, Mar 17, 2015 at 11:47 AM, Devaraja Swami <[email protected]> >>> wrote: >>> >>>> I plan to store floats in the payload and boost the score >>>> (multiplicatively) based on the average value of the payloads over the >>>> occurrences of the matching term in the document. ie., exactly as in >>>> AveragePayloadFunction in Lucene. >>>> >>>> On Tue, Mar 17, 2015 at 2:16 AM, [email protected] <[email protected] >>>> > wrote: >>>> >>>>> The concrete implementation depends on what you store in the payload >>>>> (e.g. scores) >>>>> >>>>> Jörg >>>>> >>>>> On Tue, Mar 17, 2015 at 7:01 AM, Devaraja Swami <[email protected]> >>>>> wrote: >>>>> >>>>>> I need to use PayloadTermQuery from Lucene. >>>>>> Does anyone know how I can use this in ElasticSearch? >>>>>> I am using ES 1.4.4, with the Java API. >>>>>> In Lucene, I could use this by directly instantiating >>>>>> PayloadTermQuery, but there are no APIs in ES QueryBuilders for this. >>>>>> I don't need a query parser, because I can build the query directly >>>>>> using the Java API (don't need a JSON representation of the query), >>>>>> so I only need to be able to construct, in Java, a query builder >>>>>> encapsulating a PayloadTermQuery. >>>>>> >>>>>> Thanks in advance! >>>>>> >>>>>> -devarajaswami >>>>>> >>>>>> -- >>>>>> You received this message because you are subscribed to the Google >>>>>> Groups "elasticsearch" group. >>>>>> To unsubscribe from this group and stop receiving emails from it, >>>>>> send an email to [email protected]. >>>>>> To view this discussion on the web visit https://groups.google.com/d/ >>>>>> msgid/elasticsearch/8fc84082-6fc7-42aa-9caf-8ab527bc8a0b% >>>>>> 40googlegroups.com >>>>>> <https://groups.google.com/d/msgid/elasticsearch/8fc84082-6fc7-42aa-9caf-8ab527bc8a0b%40googlegroups.com?utm_medium=email&utm_source=footer> >>>>>> . >>>>>> For more options, visit https://groups.google.com/d/optout. >>>>>> >>>>> >>>>> -- >>>>> You received this message because you are subscribed to the Google >>>>> Groups "elasticsearch" group. >>>>> To unsubscribe from this group and stop receiving emails from it, send >>>>> an email to [email protected]. >>>>> To view this discussion on the web visit https://groups.google.com/d/ >>>>> msgid/elasticsearch/CAKdsXoFwk_Ve_OK9J%2BfsEzGwbtOnaL7%2BeqT% >>>>> 3DR61hCoX8Mzi-fQ%40mail.gmail.com >>>>> <https://groups.google.com/d/msgid/elasticsearch/CAKdsXoFwk_Ve_OK9J%2BfsEzGwbtOnaL7%2BeqT%3DR61hCoX8Mzi-fQ%40mail.gmail.com?utm_medium=email&utm_source=footer> >>>>> . >>>>> >>>>> For more options, visit https://groups.google.com/d/optout. >>>>> >>>> >>>> -- >>>> You received this message because you are subscribed to the Google >>>> Groups "elasticsearch" group. >>>> To unsubscribe from this group and stop receiving emails from it, send >>>> an email to [email protected]. >>>> To view this discussion on the web visit https://groups.google.com/d/ >>>> msgid/elasticsearch/CABMSir6hF%3DuM1jp0jgoBq_ >>>> v30YNVB-8JLF7PLyvFjyXbdqtLvg%40mail.gmail.com >>>> <https://groups.google.com/d/msgid/elasticsearch/CABMSir6hF%3DuM1jp0jgoBq_v30YNVB-8JLF7PLyvFjyXbdqtLvg%40mail.gmail.com?utm_medium=email&utm_source=footer> >>>> . >>>> >>>> For more options, visit https://groups.google.com/d/optout. >>>> >>> >>> -- >>> You received this message because you are subscribed to the Google >>> Groups "elasticsearch" group. >>> To unsubscribe from this group and stop receiving emails from it, send >>> an email to [email protected]. >>> To view this discussion on the web visit https://groups.google.com/d/ >>> msgid/elasticsearch/CAPmjWd2%2B-YFPsRVB0QywSuMFEVXL- >>> UgQyxJRGBjGn4Lw0KWT4A%40mail.gmail.com >>> <https://groups.google.com/d/msgid/elasticsearch/CAPmjWd2%2B-YFPsRVB0QywSuMFEVXL-UgQyxJRGBjGn4Lw0KWT4A%40mail.gmail.com?utm_medium=email&utm_source=footer> >>> . >>> >>> For more options, visit https://groups.google.com/d/optout. >>> >> >> -- > You received this message because you are subscribed to the Google Groups > "elasticsearch" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [email protected]. > To view this discussion on the web visit > https://groups.google.com/d/msgid/elasticsearch/4f540bc9-ac6d-400d-aa40-14ed31f64968%40googlegroups.com > <https://groups.google.com/d/msgid/elasticsearch/4f540bc9-ac6d-400d-aa40-14ed31f64968%40googlegroups.com?utm_medium=email&utm_source=footer> > . > > For more options, visit https://groups.google.com/d/optout. > -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAKdsXoGFCo8mJzoJ%2BGusvyOPyD974S7d_QvNG_tFQGdsWw9zVQ%40mail.gmail.com. For more options, visit https://groups.google.com/d/optout.
