[
https://issues.apache.org/jira/browse/LUCENE-10471?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17566868#comment-17566868
]
Mayya Sharipova commented on LUCENE-10471:
------------------------------------------
Sorry, may be I should have provided more explanation.
* First this issue is only about to have max dims up to 2048. We can create a
separate issue to discuss other upper limits if there is a need for them.
* According to our ML experts resnet is an industry standard for images and it
can need up to 2048 dims. It would be good that we can support it in Lucene.
* I can also run a performance test of 1M vectors of 2048 dims to see how much
time and memory it may take to index and search these big vectors.
> Increase the number of dims for KNN vectors to 2048
> ---------------------------------------------------
>
> Key: LUCENE-10471
> URL: https://issues.apache.org/jira/browse/LUCENE-10471
> Project: Lucene - Core
> Issue Type: Wish
> Reporter: Mayya Sharipova
> Priority: Trivial
> Time Spent: 40m
> Remaining Estimate: 0h
>
> The current maximum allowed number of dimensions is equal to 1024. But we see
> in practice a couple well-known models that produce vectors with > 1024
> dimensions (e.g
> [mobilenet_v2|https://tfhub.dev/google/imagenet/mobilenet_v2_035_224/feature_vector/1]
> uses 1280d vectors, OpenAI / GPT-3 Babbage uses 2048d vectors). Increasing
> max dims to `2048` will satisfy these use cases.
> I am wondering if anybody has strong objections against this.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
