Yes I think it‘s good for visibility and also motivation to contribute to the „official“ main repo. Other projects also include cross language stuff (e.g. Arrow).
I cannot validate the extension of native Fluss APIs and write a more native gateway with bypassing Datafusion as the work was done completely by George Stamatakis. But I think it’s a reasonable point for the full API support and also for high performance use cases. Kind regards David > On 9. Mar 2026, at 20:40, Keith Lee <[email protected]> wrote: > > Agree strongly on the points that have been made on building gateway on > fluss-rust as well as benefits of monorepo. > > I’m very excited to see the possibility of bringing fluss-rust under fluss. > The collaborations we have on fluss-rust side and the language itself has > been a delight. > > Best regards > Keith Lee > > >> On Mon, 9 Mar 2026 at 15:42, Jark Wu <[email protected]> wrote: >> >> Beyond this, I believe a monorepo would be more friendly to coding >> agents by providing them with a more complete code context. This >> approach would significantly improve efficiency in cross-language >> implementations, such as porting features from Java to Rust. I view >> this transition as a proactive and necessary step for the Fluss >> project to move fast in the era of AI. >> >> Additionally, if the community considers merging repositories a viable >> option, this initiative warrants a dedicated DISCUSS and vote thread. >> Such a change requires at least three binding votes from PPMC members, >> whereas a FIP only requires three binding votes from committers. >> >> Best, >> Jark >> >>> On Mon, 9 Mar 2026 at 21:33, yuxia <[email protected]> wrote: >>> >>> Thanks David for start the discussion. >>> >>> I'm big +1 to use Rust to build the gateway for >>> - Performance: Rust avoids GC pauses, which is valuable for high-QPS >> gateway workloads >>> - Ecosystem: Rust integrates naturally with Arrow/DataFusion/Flight SQL, >> and has mature PostgreSQL protocol libraries. >>> - Industry momentum: More infrastructure projects are adopting Rust for >> gateway layers (e.g., sglang-gateway, a Rust-based AI gateway) >>> >>> Also, as one the core maintainers of fluss-rust, I'm also +1 to move >> fluss-rust to fluss main repo after the first release of fluss-rust for >>> - Faster feature delivery: It removes multi-repo release chaining and >> shortens time-to-user for fixes and new features. >>> - More contributors: A unified repo can attract broader contributor >> participation across Java/Rust/gateway areas. >>> - Better developer workflow: Developers can change Java + Rust + gateway >> code in one PR, reducing coordination overhead and improving review >> efficiency. >>> >>> Best regards, >>> Yuxia >>> >>> ----- 原始邮件 ----- >>> 发件人: "Jark Wu" <[email protected]> >>> 收件人: "dev" <[email protected]> >>> 发送时间: 星期一, 2026年 3 月 09日 下午 8:50:08 >>> 主题: Re: [DISCUSS] FIP-32 Multiprotocol Query Gateway for Apache Fluss >>> >>> Hi David, >>> >>> I really appreciate this proposal. It represents a significant step >>> forward and opens up new possibilities for the Fluss project. >>> >>> I have uploaded the FIP markdown to the Apache cwiki to improve >>> readability and granted you edit permissions. You can access and >>> modify the document here: >>> >> https://cwiki.apache.org/confluence/display/FLUSS/FIP-32%3A+Multiprotocol+Query+Gateway+for+Apache+Fluss >>> >>> Overall, I think the proposal is well structured. Below are some >>> detailed comments for further consideration: >>> >>> **1. Repository Strategy** >>> I am a strong advocate for Rust and fully support leveraging >>> DataFusion and fluss-rust to build the gateway. Given that the gateway >>> will become a critical component of Fluss, handling lightweight query >>> execution tasks, adopting Rust and DataFusion from the outset provides >>> a solid foundation for performance and reliability. >>> >>> However, we need to carefully consider project dependency management. >>> The current dependency chain Fluss gateway depends on fluss-rust, >>> which in turn depends on fluss repo means that releasing a new feature >>> would require three separate repository releases. This could delay >>> feature availability to users by nearly a year, which is inefficient. >>> I suggest consolidating fluss-rust and fluss-gateway into the Fluss >>> main repository (we can do this after the first release of >>> fluss-rust). This approach would not only streamline the release >>> process but also encourage more contributors to participate in >>> Rust-related development and accelerate community iteration across the >>> Rust, CLI, and gateway ecosystems. >>> >>> **2. Separate into multiple FIPs >>> I recommend splitting this proposal into multiple FIPs. The primary >>> FIP should focus on the Gateway architecture, covering design >>> decisions related to dependencies, language choice, repository >>> structure, and essential features such as ACL support. Within this >>> Gateway FIP, we can reference REST, ADBC, and PostgreSQL protocols in >>> a high-level overview without subjecting them to the formal voting >>> process. Each protocol should then have its own dedicated FIP to >>> address protocol-specific complexities, including supported APIs, >>> implementation details, and versioning strategies. This modular >>> approach will make the review process more focused and manageable. >>> >>> **3. Configuration File Format** >>> Fluss currently uses YAML for configuration files, which ensures >>> consistency across different environments such as Flink SQL, shell >>> commands, and Docker Compose. To maintain this uniformity, I suggest >>> starting with YAML for the gateway configuration as well. This will >>> provide a consistent external API for users. We can consider >>> supporting additional formats like TOML or JSON in the future based on >>> community feedback. >>> >>> **4. Connection Parameter** >>> We should adopt the standard `bootstrap.servers` parameter for the >>> gateway to connect to the Fluss cluster, rather than using >>> `coordinator_address`. The coordinator address may change due to >>> failover or scaling events, whereas bootstrap servers provide a stable >>> entry point for client discovery and connection. >>> >>> **5. DataFusion as an Optional Component** >>> While DataFusion is a powerful query engine, it may not be the optimal >>> choice for all gateway use cases. Some scenarios, such as >>> high-frequency, lightweight lookup_kv or produce_log operations, could >>> experience performance bottlenecks due to DataFusion's planning and >>> conversion overhead. Furthermore, certain Fluss-specific APIs are not >>> supported by DataFusion. I propose making DataFusion an optional >>> library that gateway implementations can choose to integrate. This >>> design would also allow bypassing DataFusion entirely and calling the >>> fluss-rust client directly when appropriate, providing greater >>> flexibility and performance optimization opportunities. >>> >>> Please let me know your thoughts on these suggestions. I am happy to >>> discuss further and help refine the proposal. >>> >>> Best regards, >>> Jark >>> >>> On Sat, 7 Mar 2026 at 12:08, ForwardXu <[email protected]> wrote: >>>> >>>> Hi David, >>>> >>>> >>>> Thanks a lot for sharing the FIP-32 draft and George’s prototype work >> on the multiprotocol query gateway! >>>> >>>> >>>> Supporting multiple protocols (especially Flight SQL) via Apache >> DataFusion is a great direction for Fluss, and I fully support this >> initiative. >>>> >>>> >>>> I’ve checked the repo and have one quick question: will this gateway >> be designed to be pluggable, so that we can easily add new protocols in the >> future? >>>> >>>> >>>> Looking forward to further discussions, and I’m happy to help with any >> reviews or testing. >>>> >>>> >>>> Best, >>>> Forward >>>> >>>> 原始邮件 >>>> >>>> >>>> 发件人：David Reger <[email protected]> >>>> 发件时间：2026年3月7日 07:39 >>>> 收件人：dev <[email protected]> >>>> 主题：[DISCUSS] FIP-32 Multiprotocol Query Gateway for Apache Fluss >>>> >>>> >>>> >>>> Hey together, >>>> >>>> >> Thanks to George we have already a nice draft of a query gateway using Apache Datafusion du support multiple protocols especially Flight SQL. >>>> >>>> His Repo is here >> https://github.com/gstamatakis95/fluss/tree/feature/query-gateway. >>>> >>>> >> Based on his work I created a FIP to further discuss how the query gateway fits into the project and how we want to integrate and extend it further. >>>> >>>> You can find the FIP Draft here >> https://drive.google.com/file/d/1CC_uSQjtQmlMNGZyyHXr9K162BpKZvZO/view?usp=drivesdk >>>> >>>> >> Thank you and I appreciate every feedback. >>>> >>>> Kind regards >>>> David >>
