Re: Other vector similarity metric than provided by VectorSimilarityFunction

2023-01-15 Thread Michael Wechner
Am 15.01.23 um 16:36 schrieb Michael Sokolov: I would suggest building Lucene from source and adding your own similarity function to VectorSimilarity. That is the proper extension point for similarity functions. If you find there is some substantial benefit, it wouldn't be a big lift to add

Re: Other vector similarity metric than provided by VectorSimilarityFunction

2023-01-15 Thread Michael Sokolov
I would suggest building Lucene from source and adding your own similarity function to VectorSimilarity. That is the proper extension point for similarity functions. If you find there is some substantial benefit, it wouldn't be a big lift to add something like that. However I'm dubious about the

Re: Other vector similarity metric than provided by VectorSimilarityFunction

2023-01-14 Thread Michael Wechner
Hi Adrien Thanks for your feedback! Whereas I am not sure I fully understand what you mean At the moment I am using something like: float[] vector = ...; FieldType vectorFieldType = KnnVectorField.createFieldType(vector.length, VectorSimilarityFunction.COSINE); KnnVectorField vectorField

Re: Other vector similarity metric than provided by VectorSimilarityFunction

2023-01-14 Thread Adrien Grand
Hi Michael, You could create a custom KNN vectors format that ignores the vector similarity configured on the field and uses its own. Le sam. 14 janv. 2023, 21:33, Michael Wechner a écrit : > Hi > > IIUC Lucene currently supports > > VectorSimilarityFunction.COSINE >

Other vector similarity metric than provided by VectorSimilarityFunction

2023-01-14 Thread Michael Wechner
Hi IIUC Lucene currently supports VectorSimilarityFunction.COSINE VectorSimilarityFunction.DOT_PRODUCT VectorSimilarityFunction.EUCLIDEAN whereas some embedding models have been trained with other metrics. Also see