[
https://issues.apache.org/jira/browse/CAMEL-21587?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17910287#comment-17910287
]
Zineb Bendhiba commented on CAMEL-21587:
----------------------------------------
I started working on the idea of adding the similarity search for Langchain4j
as high abstract.
However, while starting coding it, I'm realizing it's not that compatible with
my idea of using our own vector databases components.
So here's a thing, I'm first of all adding the vector search for camel-qdrant.
here's the JIRA: https://issues.apache.org/jira/browse/CAMEL-21596
And once this one is pushed, I'll do some PoC and brainstorm on the next step
for this
> Langchain4j : add similarity search
> -----------------------------------
>
> Key: CAMEL-21587
> URL: https://issues.apache.org/jira/browse/CAMEL-21587
> Project: Camel
> Issue Type: New Feature
> Reporter: Zineb Bendhiba
> Assignee: Zineb Bendhiba
> Priority: Major
>
> +*Problem:*+
> Apache Camel Langchain4j and Vector databases (Qdrant/Pinecone) can't perform
> similarity search for RAG. This limitation hinders our ability to use Apache
> Camel with Vector databases with Langchain4j for robust search capabilities.
> While creating an example of RAG using Apache Camel Langchain4j and Apache
> Camel Vector databases, I realized that the Vector databases components
> cannot handle similarity search for the Langchain4j scenarios.
> Despite trying both Qdrant and Pinecone, I encountered issues when attempting
> to perform RAG (Relevance-based Active Learning) with these databases.
> +*Research*+
> I've analyzed various existing issues related to Search with Qdrant and
> Pinecone. For example, the Camel Qdrant component doesn't provide similarity
> search capabilities. In contrast, Pinecone allows for searching based on
> similarity.
> However, when trying to use Pinecone with Apache Camel, I encountered a
> challenge: we need to convert text to Embeddings using Camel Langchain4j
> Embeddings Component. The resulting list of Embeddings cannot be used
> directly with RAG.
> +*Proposed Solution*+
> To address this limitation, I propose implementing a new feature (or new
> camel AI component) that enables easy similarity search for RAG using Apache
> Camel and Vector databases. This could involve leveraging the abstraction
> from Embedding stores in Langchain4j to provide an easy way to perform
> similarity searches.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
