GitHub user MisterRaindrop added a comment to the discussion: [Proposal] Apache Cloudberry as PostgreSQL 19 Extensions
Thank you for sharing this work. I find this a very interesting and potentially important direction for Cloudberry. The most valuable part, in my view, is not only moving Cloudberry to PostgreSQL 19. It is making the boundary between PostgreSQL and Cloudberry’s MPP capabilities much more explicit. If the dispatcher, Motions, distributed transactions and snapshots, ORCA, and the storage engines can live behind a relatively small and well-defined set of hooks, this could significantly reduce the cost of following future PostgreSQL releases. I am currently exploring DataFusion-based vectorized execution in Cloudberry, so this modular boundary is especially interesting to me. In our current direction, Cloudberry still owns planning, slices, Motions, and distributed execution, while supported fragments are executed by DataFusion. Your work suggests a broader possibility: execution engines such as vexec, DataFusion, Velox, or future alternatives could evolve behind clearer interfaces, instead of requiring more changes throughout the PostgreSQL fork. I have a few questions that I think would be valuable for evaluating the long-term direction: 1. Which of the 24 PostgreSQL core hooks are general enough to propose upstream, and which are inherently Cloudberry-specific? 2. How do you plan to keep the adapted or copied files under `pg19/` synchronized with Cloudberry `main` over time? 3. Do you plan to publish an apples-to-apples comparison between the current Cloudberry implementation and the PostgreSQL 19 port on the same multi-host environment, especially for Motion-heavy joins, concurrent workloads, spill, and failure recovery? 4. Would an experimental branch or separate repository be the preferred first step for community collaboration? The current vectorized-execution results are promising, although most of the published measurements appear to be from one physical host. A real multi-host comparison would make it much easier to understand both the performance potential and the remaining distributed-execution overhead. Overall, I think this experiment demonstrates that Cloudberry’s relationship with PostgreSQL does not necessarily have to remain a deeply coupled fork forever. Even if some core hooks are still required, turning the majority of the MPP system into modules looks like a direction worth exploring further. GitHub link: https://github.com/apache/cloudberry/discussions/2065#discussioncomment-18901577 ---- This is an automatically sent email for [email protected]. To unsubscribe, please send an email to: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
