Re: SPLADE implementation

[Michael Wechner]

thank you very much, will try this :-)


Am 15.11.23 um 11:25 schrieb Adrien Grand:

Say your model produces a set of weighted terms:

 - At index time, for each (term, weight) pair, you add a "new FeatureField(fieldName, term, weight)` field to your document.  - At search time, for each (term, weight) pair, you add a "new BooleanClause(FeatureField.newLinearQuery(fieldName, term, weight))" to your BooleanQuery.

On Wed, Nov 15, 2023 at 11:08 AM Michael Wechner <michael.wech...@wyona.com> wrote:

    Hi Adrien

    Ah ok, I did not realize this, thanks for pointing this out!

    I don't quite understand though, how you would implement the
    "SPLADE" approach using FeatureField from the documentation at

    
https://lucene.apache.org/core/9_8_0/core/org/apache/lucene/document/FeatureField.html

    For example when indexing a document or doing a query and I use
    some language model (e.g. BERT) to do the term expansion, how
    do I then make use of FeatureField exactly?

    I tried to find some code examples, but couldn't, do you maybe
    have some pointers?

    Thanks

    Michael


    Am 15.11.23 um 10:34 schrieb Adrien Grand:

    Hi Michael,

    What functionality are you missing? Lucene already supports
    indexing/querying weighted terms using FeatureField.

    On Wed, Nov 15, 2023 at 10:03 AM Michael Wechner
    <michael.wech...@wyona.com> wrote:

        Hi

        I have found the following issue re a possible SPLADE
        implementation

        https://github.com/apache/lucene/issues/11799

        Is somebody still working on this?

        Thanks

        Michael



        ---------------------------------------------------------------------
        To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
        For additional commands, e-mail: dev-h...@lucene.apache.org

-- Adrien

--
Adrien