Agree strongly on the points that have been made on building gateway on fluss-rust as well as benefits of monorepo.
I’m very excited to see the possibility of bringing fluss-rust under fluss. The collaborations we have on fluss-rust side and the language itself has been a delight. Best regards Keith Lee On Mon, 9 Mar 2026 at 15:42, Jark Wu <[email protected]> wrote: > Beyond this, I believe a monorepo would be more friendly to coding > agents by providing them with a more complete code context. This > approach would significantly improve efficiency in cross-language > implementations, such as porting features from Java to Rust. I view > this transition as a proactive and necessary step for the Fluss > project to move fast in the era of AI. > > Additionally, if the community considers merging repositories a viable > option, this initiative warrants a dedicated DISCUSS and vote thread. > Such a change requires at least three binding votes from PPMC members, > whereas a FIP only requires three binding votes from committers. > > Best, > Jark > > On Mon, 9 Mar 2026 at 21:33, yuxia <[email protected]> wrote: > > > > Thanks David for start the discussion. > > > > I'm big +1 to use Rust to build the gateway for > > - Performance: Rust avoids GC pauses, which is valuable for high-QPS > gateway workloads > > - Ecosystem: Rust integrates naturally with Arrow/DataFusion/Flight SQL, > and has mature PostgreSQL protocol libraries. > > - Industry momentum: More infrastructure projects are adopting Rust for > gateway layers (e.g., sglang-gateway, a Rust-based AI gateway) > > > > Also, as one the core maintainers of fluss-rust, I'm also +1 to move > fluss-rust to fluss main repo after the first release of fluss-rust for > > - Faster feature delivery: It removes multi-repo release chaining and > shortens time-to-user for fixes and new features. > > - More contributors: A unified repo can attract broader contributor > participation across Java/Rust/gateway areas. > > - Better developer workflow: Developers can change Java + Rust + gateway > code in one PR, reducing coordination overhead and improving review > efficiency. > > > > Best regards, > > Yuxia > > > > ----- 原始邮件 ----- > > 发件人: "Jark Wu" <[email protected]> > > 收件人: "dev" <[email protected]> > > 发送时间: 星期一, 2026年 3 月 09日 下午 8:50:08 > > 主题: Re: [DISCUSS] FIP-32 Multiprotocol Query Gateway for Apache Fluss > > > > Hi David, > > > > I really appreciate this proposal. It represents a significant step > > forward and opens up new possibilities for the Fluss project. > > > > I have uploaded the FIP markdown to the Apache cwiki to improve > > readability and granted you edit permissions. You can access and > > modify the document here: > > > https://cwiki.apache.org/confluence/display/FLUSS/FIP-32%3A+Multiprotocol+Query+Gateway+for+Apache+Fluss > > > > Overall, I think the proposal is well structured. Below are some > > detailed comments for further consideration: > > > > **1. Repository Strategy** > > I am a strong advocate for Rust and fully support leveraging > > DataFusion and fluss-rust to build the gateway. Given that the gateway > > will become a critical component of Fluss, handling lightweight query > > execution tasks, adopting Rust and DataFusion from the outset provides > > a solid foundation for performance and reliability. > > > > However, we need to carefully consider project dependency management. > > The current dependency chain Fluss gateway depends on fluss-rust, > > which in turn depends on fluss repo means that releasing a new feature > > would require three separate repository releases. This could delay > > feature availability to users by nearly a year, which is inefficient. > > I suggest consolidating fluss-rust and fluss-gateway into the Fluss > > main repository (we can do this after the first release of > > fluss-rust). This approach would not only streamline the release > > process but also encourage more contributors to participate in > > Rust-related development and accelerate community iteration across the > > Rust, CLI, and gateway ecosystems. > > > > **2. Separate into multiple FIPs > > I recommend splitting this proposal into multiple FIPs. The primary > > FIP should focus on the Gateway architecture, covering design > > decisions related to dependencies, language choice, repository > > structure, and essential features such as ACL support. Within this > > Gateway FIP, we can reference REST, ADBC, and PostgreSQL protocols in > > a high-level overview without subjecting them to the formal voting > > process. Each protocol should then have its own dedicated FIP to > > address protocol-specific complexities, including supported APIs, > > implementation details, and versioning strategies. This modular > > approach will make the review process more focused and manageable. > > > > **3. Configuration File Format** > > Fluss currently uses YAML for configuration files, which ensures > > consistency across different environments such as Flink SQL, shell > > commands, and Docker Compose. To maintain this uniformity, I suggest > > starting with YAML for the gateway configuration as well. This will > > provide a consistent external API for users. We can consider > > supporting additional formats like TOML or JSON in the future based on > > community feedback. > > > > **4. Connection Parameter** > > We should adopt the standard `bootstrap.servers` parameter for the > > gateway to connect to the Fluss cluster, rather than using > > `coordinator_address`. The coordinator address may change due to > > failover or scaling events, whereas bootstrap servers provide a stable > > entry point for client discovery and connection. > > > > **5. DataFusion as an Optional Component** > > While DataFusion is a powerful query engine, it may not be the optimal > > choice for all gateway use cases. Some scenarios, such as > > high-frequency, lightweight lookup_kv or produce_log operations, could > > experience performance bottlenecks due to DataFusion's planning and > > conversion overhead. Furthermore, certain Fluss-specific APIs are not > > supported by DataFusion. I propose making DataFusion an optional > > library that gateway implementations can choose to integrate. This > > design would also allow bypassing DataFusion entirely and calling the > > fluss-rust client directly when appropriate, providing greater > > flexibility and performance optimization opportunities. > > > > Please let me know your thoughts on these suggestions. I am happy to > > discuss further and help refine the proposal. > > > > Best regards, > > Jark > > > > On Sat, 7 Mar 2026 at 12:08, ForwardXu <[email protected]> wrote: > > > > > > Hi David, > > > > > > > > > Thanks a lot for sharing the FIP-32 draft and George’s prototype work > on the multiprotocol query gateway! > > > > > > > > > Supporting multiple protocols (especially Flight SQL) via Apache > DataFusion is a great direction for Fluss, and I fully support this > initiative. > > > > > > > > > I’ve checked the repo and have one quick question: will this gateway > be designed to be pluggable, so that we can easily add new protocols in the > future? > > > > > > > > > Looking forward to further discussions, and I’m happy to help with any > reviews or testing. > > > > > > > > > Best, > > > Forward > > > > > > 原始邮件 > > > > > > > > > 发件人：David Reger <[email protected]> > > > 发件时间：2026年3月7日 07:39 > > > 收件人：dev <[email protected]> > > > 主题：[DISCUSS] FIP-32 Multiprotocol Query Gateway for Apache Fluss > > > > > > > > > > > > Hey together, > > > > > > > Thanks to George we have already a nice draft of a query gateway using Apache Datafusion du support multiple protocols especially Flight SQL. > > > > > > His Repo is here > https://github.com/gstamatakis95/fluss/tree/feature/query-gateway. > > > > > > > Based on his work I created a FIP to further discuss how the query gateway fits into the project and how we want to integrate and extend it further. > > > > > > You can find the FIP Draft here > https://drive.google.com/file/d/1CC_uSQjtQmlMNGZyyHXr9K162BpKZvZO/view?usp=drivesdk > > > > > > > Thank you and I appreciate every feedback. > > > > > > Kind regards > > > David >
