Hi Julie,
thank you for working on LUCENE-9322 (I also love the issue).
I think it would be great if we can try some preliminary aknn
implementations (both of clustering-based and graph-based approach) on
LUCENE-9322, to explore working unified API and Codec/Format for vectors;
for now, I still have no good clear image about desirable abstraction we
should have. Sorry for my inactivity on the issues - I wish I could have
more time and expertise on it.

Tomoko


2020年7月23日(木) 3:10 Julie Tibshirani <juliet...@gmail.com>:

> Tomoko -- this fits with my experience as well. I really like the idea of
> treating LUCENE-9322 as setting up a framework for experimentation +
> benchmarking (but not requiring us to commit a particular ANN
> implementation quite yet).
>
> Julie
>
> On 2020/07/17 12:16:18, Tomoko Uchida <t...@gmail.com> wrote:
> > > would it make sense to create a separate Lucene module for ANN search
> ?>
> >
> > From a bit of my experience with LUCENE-9004, it is currently impossible
> to>
> > plug in or opt in custom codecs and indexing chain for aknn search
> without>
> > touching lucene-core module (plz correct that if it's wrong).>
> > I think LUCENE-9322 (unified low-level Codec/Format for dense vectors)>
> > would open the possibility for us to experiment different aknn
> algorithms>
> > on some sandbox modules or even separated jars from Lucene itself.>
> >
> > Tomoko>
> >
> >
> > 2020年7月17日(金) 17:00 Tommaso Teofili <to...@gmail.com>:>
> >
> > > would it make sense to create a separate Lucene module for ANN search
> ?>
> > > we could then experiment with the different approaches and compare
> them>
> > > across the same benchmarks.>
> > >>
> > > On Thu, 16 Jul 2020 at 23:14, Ali Akhtar <al...@ali.actor> wrote:>
> > >>
> > > > I’m a bit of a layman in this area, but if we are talking about
> formats>
> > > for>
> > > > vectors, I vote for the one used by FastAI word vectors. It’s pretty
> easy>
> > > > to work with.>
> > > >>
> > > > If we are talking about the same / similiar things, if not just
> ignore me>
> > > > 😀>
> > > >>
> > > > On Thu, 16 Jul 2020 at 7:06 PM, Michael Sokolov <ms...@gmail.com>>
> > > > wrote:>
> > > >>
> > > > > We have some prototype implementations in the issues you found.
> If>
> > > > > you want to try out the approaches in those issues, you could
> build>
> > > > > Lucene from source and patch it, but there is no release
> containing>
> > > > > KNN/vector support. We're still working to establish consensus on
> what>
> > > > > the best way forward is. I think the most fruitful thing we can do
> at>
> > > > > the moment is establish a format for storing and accessing
> vectors>
> > > > > that will support different approaches since there is such a rich>
> > > > > variety of algorithms and approaches in this area. The last issue
> you>
> > > > > pointed to is focused on the format.>
> > > > >>
> > > > > On Wed, Jul 15, 2020 at 11:20 AM Alex K <ak...@gmail.com> wrote:>
> > > > > >>
> > > > > > Hi Mikhail,>
> > > > > >>
> > > > > > I'm not sure about the state of ANN in lucene proper. Very
> interested>
> > > > to>
> > > > > > see the response from others.>
> > > > > > I've been doing some work on ANN for an Elasticsearch plugin:>
> > > > > > http://elastiknn.klibisz.com/>
> > > > > > I think it's possible to extract my custom queries and modeling
> code>
> > > so>
> > > > > > that it's elasticsearch-agnostic and can be used directly in
> Lucene>
> > > > apps.>
> > > > > > However I'm much more familiar with Elasticsearch's APIs and>
> > > > > usage/testing>
> > > > > > patterns than I am with raw Lucene, so I'd likely need to get
> some>
> > > help>
> > > > > > from the Lucene community.>
> > > > > > Please LMK if that sounds interesting to anyone.>
> > > > > >>
> > > > > > - Alex>
> > > > > >>
> > > > > >>
> > > > > >>
> > > > > > On Wed, Jul 15, 2020 at 11:11 AM Mikhail <wm...@mail.ru.invalid>>
>
> > > > > wrote:>
> > > > > >>
> > > > > > >>
> > > > > > > Hi,>
> > > > > > >>
> > > > > > >            I want to incorporate semantic search in my
> project,>
> > > which>
> > > > > uses>
> > > > > > > Lucene. I want to use sentence embeddings and ANN
> (approximate>
> > > > nearest>
> > > > > > > neighbor) search. I found the related Lucene issues:>
> > > > > > > https://issues.apache.org/jira/browse/LUCENE-9004 ,>
> > > > > > > https://issues.apache.org/jira/browse/LUCENE-9136 ,>
> > > > > > > https://issues.apache.org/jira/browse/LUCENE-9322 . I see
> that>
> > > there>
> > > > > > > are some related work and related PRs. What is the current
> state of>
> > > > > this>
> > > > > > > functionality?>
> > > > > > >>
> > > > > > > -->
> > > > > > > Thanks,>
> > > > > > > Mikhail>
> > > > > > >>
> > > > > > >>
> > > > >>
> > > > >
> --------------------------------------------------------------------->
> > > > > To unsubscribe, e-mail: java-user-unsubscr...@lucene.apache.org>
> > > > > For additional commands, e-mail: java-user-h...@lucene.apache.org>
>
> > > > >>
> > > > >>
> > > >>
> > >>
> >
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-user-unsubscr...@lucene.apache.org
> For additional commands, e-mail: java-user-h...@lucene.apache.org
>
>

Reply via email to