Here a concrete example where I combine OpenAI model
"text-similarity-ada-001" with Lucene vector search
INPUT sentence: "What is your age this year?"
Result sentences
1) How old are you this year?
score '0.98860765'
2) What was your age last year?
score '0.97811764'
3) What is your age?
score '0.97094905'
4) How old are you?
score '0.9600177'
Result 1 is great and result 2 looks similar, but is not correct from an
"understanding" point of view and results 3 and 4 are good again.
I understand "similarity" is not the same as "understanding", but I hope
it makes it clearer what I am looking for :-)
Thanks
Michael
Am 12.02.22 um 22:38 schrieb Michael Wechner:
Hi Alessandro
I am mainly interested in detecting similarity, for example whether
the following two sentences are similar resp. likely to mean the same
thing
"How old are you?"
"What is your age?"
and that the following two sentences are not similar, resp. do not
mean the same thing
"How old are you this year?"
"How old have you been last year?"
But also performance or how OpenAI embeddings compare for example with
SBERT (https://sbert.net/docs/usage/semantic_textual_similarity.html)
Thanks
Michael
Am 12.02.22 um 20:41 schrieb Alessandro Benedetti:
Hi Michael, experience to what extent?
We have been exploring the area for a while given we contributed the
first neural search milestone to Apache Solr.
What is your curiosity? Performance? Relevance impact? How to
integrate it?
Regards
On Fri, 11 Feb 2022, 22:38 Michael Wechner,
<michael.wech...@wyona.com> wrote:
Hi
Does anyone have experience using OpenAI embeddings in
combination with Lucene vector search?
https://beta.openai.com/docs/guides/embeddings|
for example comparing performance re vector size
||https://api.openai.com/v1/engines/|||text-similarity-ada-001|/embeddings
and
||https://api.openai.com/v1/engines/||||text-similarity-davinci-001||/embeddings
?
||
|Thanks
Michael
