GitHub user MisterRaindrop added a comment to the discussion: [DISCUSSION] Making Apache Cloudberry an Agent-Native Analytical Database
Thanks, this is very helpful. I agree that an MCP-first Phase 0 would give us a lower-cost way to validate the Agent-Native use cases before making the Lance integration too deep. I also agree that a read-only Lance FDW is the right starting point for the storage integration. The write-back point maps well to the “Analytical RAG” part of the proposal. My current understanding is: - Lance can provide large-scale vector and multimodal data for retrieval. - Cloudberry can query that data through the FDW and combine it with relational or Iceberg data. - AI-generated results—such as extracted entities, classifications, summaries, scores, and other derived features—can later be written into Cloudberry native tables. - Those results can then be versioned, joined, analyzed, and reused by subsequent RAG workflows without running the same extraction repeatedly. I think this kind of AI-derived data write-back should be distinguished from general Agent Memory. The write-back path stores durable analytical results, while Agent Memory may separately cover task checkpoints, previously established facts, user or business context, and selected execution history. We probably do not need to store the agent’s entire reasoning trace. So the architecture could roughly become: Agent → MCP → Cloudberry → Native Tables / Iceberg / Lance With two different paths: - Read path: retrieve and analyze data from Native Tables, Iceberg, and Lance. - Write-back path: persist selected AI-derived results and agent state into Cloudberry native tables. I’ll revise the roadmap ordering along these lines: Phase 0: MCP and agent-facing tools over existing Cloudberry capabilities Phase 1: Read-only Lance FDW with filter and projection pushdown Phase 2: Vector Top-K pushdown Phase 3: Distributed Top-K and hybrid retrieval Phase 4: Analytical RAG write-back for extracted and classified data Phase 5: Agent memory, task state, and memory lifecycle management Phase 6+: Semantic layer and extreme multi-tenancy exploration This keeps the MCP interface independent of the Lance implementation, while giving Lance a clear role in AI-oriented retrieval and Cloudberry native tables a clear role in durable analytical results and agent state. Thanks again — I think this separation makes the proposal and implementation path much clearer. GitHub link: https://github.com/apache/cloudberry/discussions/1967#discussioncomment-18342460 ---- This is an automatically sent email for [email protected]. To unsubscribe, please send an email to: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
