Hi Dawid
Can you describe your crash in more detail?
How many millions vectors exactly?
What was the vector dimension?
How much RAM?
etc.
Thanks
Michael
Am 05.04.23 um 17:48 schrieb Dawid Weiss:
Ok, so what should we do then?
I don't know, Alessandro. I just wanted to point out the fact that by
Apache rules a committer's veto to a code change counts as a no-go. It
does not specify any way to "override" such a veto, perhaps counting
on disagreeing parties to resolve conflicting points of views in a
civil manner so that veto can be retracted (or a different solution
suggested).
I think Robert's point is not about a particular limit value but about
the algorithm itself - the current implementation does not scale. I
don't want to be an advocate for either side - I'm all for freedom of
choice but at the same time last time I tried indexing a few million
vectors, I couldn't get far before segment merging blew up with
OOMs...
Did anything similar happen in the past? How was the current limit added?
I honestly don't know, you'd have to git blame or look at the mailing
list archives of the original contribution.
Dawid
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
