Hi all, Recording: https://fathom.video/share/Q811Jrv5sBiKsUVBeFbz6y_x2gx_qkFy
*Purpose* Review 2026 roadmap priorities, finalize 0.9 release (largest since open-sourcing), and align on Iceberg V3 integration, AI positioning, Rust client evolution & Security. 0.9 Release (Early Feb Target) Timeline: Code freeze this weekend → RCs next week → final release pre-Lunar New Year. *Key Features* ● Complex types: full array/map/nested row support with schema evolution sync to tiered tables ● Auto-increment columns for global dictionaries and high-performance distinct counts ● Compacted log format for efficient storage of large vectors (row-based, no column projection overhead) ● Cluster rebalance automation during scaling ● Change Data Feed (CDF) system tables for decision tracking ● Spark integration: catalog + batch/streaming reads/writes ● Azure Blob Storage support 2026 Roadmap: Streaming Lakehouse Strategy Iceberg Integration (7 Pillars) - Iceberg V3 Core - Catalog Ecosystem - Performance - Multi-Engine Support - In-Place Lakehouse - Maintenance Operations - Benchmarking *Fluss for AI Positioning* Core Strategy: Unified open storage layer for raw/columnar/vector data; eliminates complex multi-destination pipelines (Kafka + KV + ClickHouse). *Key Capabilities:* ● Feature/context store with external computation engines (e.g., LanceDB for vector search) ● Decision tracking via Virtual tables (trace decisions through changelog) ● Zero-copy schema evolution for model adaptation without pipeline breaks ● Compacted log format optimized for large vector storage ● Engine-agnostic with multi-language SDK support (Rust/Java/Go) *Rust Client Progress* Milestone: 80 PRs closed in 2 months (~10 contributors); 0.1.0 release on track. Recent Additions: ● Column pruning, partitioned KV tables, admin APIs, log poll batching, leader availability handling, C++ bindings ● Pending: partitioned scan/append features for 0.1.0 Impact: Strategic backbone for unified, low-maintenance ecosystem. *Security, PG & Flight* ● PostgreSQL API is the top priority. It unlocks the entire Postgres ecosystem (e.g., psql, pgAdmin), significantly expanding the user base. ● AeroFlight SQL is lower priority. Its user base is unknown, and it can be implemented as an independent gateway to avoid core dependency issues. ● TLS support is confirmed for v0.10. Lorenzo will draft a spec, mirroring Kafka's user experience to leverage existing user familiarity. *Action Items* ● Mehul: Create Iceberg catalog tickets (REST/Glue/Unity/Hive/JDBC); ● Giannis: Sync with Victor on Lake Keeper REST catalog + Flink support ● Community: Email dev list with Rust client progress summary ● Nicholas: Open discussion for DirectIO *Current Objective* Ship cohesive 0.9 with complex types, Changelog, Spark integration, and cluster rebalancing, Fixes on Streaming lakehouse Best Regards, Mehul Batra
