mohamedawnallah commented on code in PR #36301: URL: https://github.com/apache/beam/pull/36301#discussion_r2383489459
########## website/www/site/content/en/blog/gsoc-25-ml-connectors.md: ########## @@ -0,0 +1,255 @@ +--- +title: "Google Summer of Code 2025 - Beam ML Vector DB/Feature Store +integrations" +date: 2025-09-26 00:00:00 -0400 +categories: + - blog + - gsoc +aliases: + - /blog/2025/09/26/gsoc-25-ml-connectors.html +authors: + - mohamedawnallah + +--- +<!-- +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + +http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +--> + +## What Will I Cover In This Blog Post? + +I have three objectives in mind when writing this blog post: + +- Documenting the work I've been doing during this GSoC period in collaboration +with the Apache Beam community +- A thoughtful and cumulative thank you to my mentor and the Beam Community +- Writing to an older version of myself before making my first ever contribution +to Beam. This can be helpful for future contributors + +## What Was This GSoC Project About? + +The goal of this project is to enhance Beam's Python SDK by developing +connectors for vector databases like Milvus and feature stores like Tecton. These +integrations will improve support for ML use cases such as Retrieval-Augmented +Generation (RAG) and feature engineering. By bridging Beam with these systems, +this project will attract more users, particularly in the ML community. + +## Why Was This Project Important? + +While Beam's Python SDK supports some vector databases, feature stores and +embedding generators, the current integrations are limited to a few systems as +mentioned in the tables down below. Expanding this ecosystem will provide more +flexibility and richness for ML workflows particularly in feature engineering +and RAG applications, potentially attracting more users, particularly in the ML +community. + +| Vector Database | Feature Store | Embedding Generator | +|----------------|---------------|---------------------| +| BigQuery | Vertex AI | Vertex AI | +| AlloyDB | Feast | Hugging Face | + +## Why Did I Choose Beam As Part of GSoC Among 180+ Orgs? + +I choose to apply to Beam from among 180+ GSoC organizations because it Review Comment: Addressed -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
