GitHub user MisterRaindrop added a comment to the discussion: [DISCUSSION] Making Apache Cloudberry an Agent-Native Analytical Database
## Additional context: Lance for multimodal data It may be useful to clarify what “AI-native / multimodal data” means here. [Lance](https://lance.org/) is an Apache-2.0 open-source file and table format designed for AI datasets. It can keep structured metadata, text, vectors, images, audio, and video in the same versioned dataset. Large media objects can use blob encoding and be loaded lazily, so a system can scan metadata or embeddings first and fetch only the selected media payloads. This is useful because a multimodal dataset is usually more than a collection of vectors. A single video, for example, may have: - business and lineage metadata; - the original video or audio; - frame-, clip-, and text-level embeddings; - captions, transcripts, labels, and model outputs; - several generations of features produced by different models. Lance is designed for workflows where these derived columns evolve frequently. New embeddings, captions, or predictions can be added without rewriting the original media dataset. The same dataset can then support random access for model training, vector and full-text retrieval for serving, and scans for feature engineering or analytics. Typical applications include: - text-to-image, text-to-video, and cross-modal retrieval; - visual similarity search and duplicate detection; - training-data sampling and feature engineering; - media analytics over captions, transcripts, metadata, and embeddings; - multimodal RAG and long-term agent memory. There are already several publicly documented production examples. These are generally described as Lance or LanceDB deployments; [LanceDB is built on the open-source Lance format](https://www.lancedb.com/customers). - [Netflix's Media Data Lake](https://www.lancedb.com/blog/case-study-netflix) uses LanceDB to organize media assets together with metadata, embeddings, and ML-derived features for search, exploration, and training workflows. - [Runway](https://www.lancedb.com/customers) reports using Lance in its generative-video model training pipeline, including a 1.8 TB in-memory video pipeline. - [ByteDance's Volcano Engine](https://www.lancedb.com/blog/volcano-engine-lance-agent-memory) has publicly described Lance as the storage core of an AI data platform covering images, video, embeddings, structured metadata, and point clouds; it also uses LanceDB as the memory backend for its managed agent platform. For Cloudberry, the interesting boundary would not be to move media decoding or model training into the database. Lance could remain the external AI-native dataset layer, while Cloudberry provides distributed SQL, joins with native or Iceberg tables, metadata filtering, Vector Top-K pushdown, and aggregation over the retrieved or derived results. That is the main multimodal opportunity: not only “search similar vectors,” but analyze business data and AI-derived information together through one MPP query layer. GitHub link: https://github.com/apache/cloudberry/discussions/1967#discussioncomment-18323765 ---- This is an automatically sent email for [email protected]. To unsubscribe, please send an email to: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
