Hey Aditya, I don't think there is a very well defined priority order. I;ll note that we already have enrichment handlers for Feast/Vertex AI for reading/enriching data with lookups to those systems, so I'd probably say the following prioritization makes sense:
- Sink for Vertex/Feast (finish what we have) Sink and enrichment handlers for the following: - Chroma - Pinecone - Tecton - Sagemaker - Milvus - FAISS That is already more than I'd expect to happen in a single project, but the goal would be to get as far as possible. I also think because the ordering is not very clear, it is fine to prioritize one or two systems which you find particularly interesting if any stand out. Thanks, Danny On Sun, Mar 2, 2025 at 1:10 PM Aditya <adiworkprof...@gmail.com> wrote: > Subject: Clarification on Implementation of Vector Databases and Feature > Stores > > Dear Sir, > > I hope this message finds you well. > > I am seeking clarification on whether it is necessary to implement all the > following vector databases and feature stores in our project: > > *Vector Databases:* > > - Pinecone > - FAISS (Facebook AI Similarity Search) > - Weaviate > - Chroma > - Milvus > > *Feature Stores:* > > - Tecton > - Feast (Open-source feature store) > - Vertex AI Feature Store (Google) > - AWS SageMaker Feature Store > > Could you please advise on which of these technologies we should > prioritize for implementation? > > Thank you for your guidance. > > Best regards, > > Aditya > > On Sun, Mar 2, 2025 at 5:29 PM Aditya <adiworkprof...@gmail.com> wrote: > >> Sir, >> >> I have a question regarding the implementation of the I/O connector for >> Pinecone and Tecton. Should it be developed in Java or Python? >> >> Pinecone provides an official Python client library but does not have one >> for Java. However, most of Apache Beam’s existing I/O connectors are >> written in Java. Given this, would it be better to use Python for >> integration, or should we develop a Java-based solution? >> >> for java, we need to use api >> >> Best regards, >> Aditya >> >> On Sat, 1 Mar, 2025, 09:39 Aditya, <adiworkprof...@gmail.com> wrote: >> >>> Sir, >>> >>> I have a question regarding the implementation of the I/O connector for >>> Pinecone and Tecton. Should it be developed in Java or Python? >>> >>> Pinecone provides an official Python client library but does not have >>> one for Java. However, most of Apache Beam’s existing I/O connectors are >>> written in Java. Given this, would it be better to use Python for >>> integration, or should we develop a Java-based solution? >>> >>> for java, we need to use api >>> >>> Best regards, >>> Aditya >>> >>>>
