Re: [PR] FieldInfosFormat translation should be independent of VectorSimilartyFunction enum [lucene]

2024-02-25 Thread via GitHub
benwtrent commented on PR #13119: URL: https://github.com/apache/lucene/pull/13119#issuecomment-1963483170 @msokolov thanks for clarifying. I just wanted to make sure there wasn't an important reason that I missed. -- This is an automated message from the Apache Git Service. To respond

Re: [PR] FieldInfosFormat translation should be independent of VectorSimilartyFunction enum [lucene]

2024-02-25 Thread via GitHub
msokolov commented on PR #13119: URL: https://github.com/apache/lucene/pull/13119#issuecomment-1963094604 > OK, this is really weird to me. For some reason, we are writing the dimension & similarity into the vector metdata but that information is retained in the field info already.

Re: [PR] FieldInfosFormat translation should be independent of VectorSimilartyFunction enum [lucene]

2024-02-22 Thread via GitHub
ChrisHegarty merged PR #13119: URL: https://github.com/apache/lucene/pull/13119 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

Re: [PR] FieldInfosFormat translation should be independent of VectorSimilartyFunction enum [lucene]

2024-02-22 Thread via GitHub
ChrisHegarty commented on PR #13119: URL: https://github.com/apache/lucene/pull/13119#issuecomment-1959890082 Thanks for the reviews. All comments have been addressed. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use

Re: [PR] FieldInfosFormat translation should be independent of VectorSimilartyFunction enum [lucene]

2024-02-22 Thread via GitHub
benwtrent commented on code in PR #13119: URL: https://github.com/apache/lucene/pull/13119#discussion_r1499170841 ## lucene/core/src/java/org/apache/lucene/codecs/lucene99/Lucene99HnswVectorsReader.java: ## @@ -171,15 +172,25 @@ private void validateFieldEntry(FieldInfo info,

Re: [PR] FieldInfosFormat translation should be independent of VectorSimilartyFunction enum [lucene]

2024-02-21 Thread via GitHub
benwtrent commented on code in PR #13119: URL: https://github.com/apache/lucene/pull/13119#discussion_r1497494311 ## lucene/core/src/java/org/apache/lucene/codecs/lucene99/Lucene99HnswVectorsReader.java: ## @@ -171,15 +172,25 @@ private void validateFieldEntry(FieldInfo info,

Re: [PR] FieldInfosFormat translation should be independent of VectorSimilartyFunction enum [lucene]

2024-02-21 Thread via GitHub
ChrisHegarty commented on code in PR #13119: URL: https://github.com/apache/lucene/pull/13119#discussion_r1497177281 ## lucene/core/src/java/org/apache/lucene/codecs/lucene99/Lucene99HnswVectorsReader.java: ## @@ -171,15 +172,25 @@ private void validateFieldEntry(FieldInfo

Re: [PR] FieldInfosFormat translation should be independent of VectorSimilartyFunction enum [lucene]

2024-02-20 Thread via GitHub
ChrisHegarty commented on PR #13119: URL: https://github.com/apache/lucene/pull/13119#issuecomment-1954278599 I see now that we have a similar dependency in `Lucene99HnswVectorsReader`. I'll update in a similar way. -- This is an automated message from the Apache Git Service. To respond

Re: [PR] FieldInfosFormat translation should be independent of VectorSimilartyFunction enum [lucene]

2024-02-20 Thread via GitHub
uschindler commented on PR #13119: URL: https://github.com/apache/lucene/pull/13119#issuecomment-1954160317 > > Hi, as stated in the other issue: I am not really happy to have that enum at all! The similarity/distance functions should be pluggable using `NamedSPILoader`. To implement that,

Re: [PR] FieldInfosFormat translation should be independent of VectorSimilartyFunction enum [lucene]

2024-02-20 Thread via GitHub
ChrisHegarty commented on PR #13119: URL: https://github.com/apache/lucene/pull/13119#issuecomment-1954136856 > > Index format wise, I think the index corruption can occur when reading a Lucene 9.8.0 index with Lucene 9.7.0, as the format would allow that, but I am not sure this is an

Re: [PR] FieldInfosFormat translation should be independent of VectorSimilartyFunction enum [lucene]

2024-02-20 Thread via GitHub
ChrisHegarty commented on PR #13119: URL: https://github.com/apache/lucene/pull/13119#issuecomment-1954132655 > Hi, as stated in the other issue: I am not really happy to have that enum at all! The similarity/distance functions should be pluggable using `NamedSPILoader`. To implement that,

Re: [PR] FieldInfosFormat translation should be independent of VectorSimilartyFunction enum [lucene]

2024-02-20 Thread via GitHub
uschindler commented on PR #13119: URL: https://github.com/apache/lucene/pull/13119#issuecomment-1954105604 > Index format wise, I think the index corruption can occur when reading a Lucene 9.8.0 index with Lucene 9.7.0, as the format would allow that, but I am not sure this is an expected

Re: [PR] FieldInfosFormat translation should be independent of VectorSimilartyFunction enum [lucene]

2024-02-20 Thread via GitHub
tteofili commented on code in PR #13119: URL: https://github.com/apache/lucene/pull/13119#discussion_r1495629501 ## lucene/core/src/java/org/apache/lucene/codecs/lucene94/Lucene94FieldInfosFormat.java: ## @@ -111,6 +112,8 @@ * 0: EUCLIDEAN distance. ({@link

Re: [PR] FieldInfosFormat translation should be independent of VectorSimilartyFunction enum [lucene]

2024-02-20 Thread via GitHub
ChrisHegarty commented on code in PR #13119: URL: https://github.com/apache/lucene/pull/13119#discussion_r1495556754 ## lucene/core/src/java/org/apache/lucene/codecs/lucene94/Lucene94FieldInfosFormat.java: ## @@ -111,6 +112,8 @@ * 0: EUCLIDEAN distance. ({@link

Re: [PR] FieldInfosFormat translation should be independent of VectorSimilartyFunction enum [lucene]

2024-02-20 Thread via GitHub
ChrisHegarty commented on PR #13119: URL: https://github.com/apache/lucene/pull/13119#issuecomment-1953858737 This PR is a prerequisite for future work to make the similarity function symbolic and lookup-able, see https://github.com/apache/lucene/pull/13076#issuecomment-1930363479. --

[PR] FieldInfosFormat translation should be independent of VectorSimilartyFunction enum [lucene]

2024-02-20 Thread via GitHub
ChrisHegarty opened a new pull request, #13119: URL: https://github.com/apache/lucene/pull/13119 This commit updates the FieldInfosFormat translation of vector similarity functions to be independent of the `VectorSimilartyFunction` enum. The `VectorSimilartyFunction` enum lives