I think this also relates to one of the architectural discussions we've recently had around FIP-32 [1].
My understanding is that there is a useful distinction between *native Fluss operations* and *SQL/query execution*. For example, writes, metadata operations, and potentially lightweight native reads (such as point lookups) could continue to build directly on the fluss-rust foundation, whereas DataFusion would primarily serve the SQL/query path. This would allow the REST gateway to remain lightweight while keeping the query layer reusable for SQL-based protocols in the future. I don't think this necessarily changes the scope of FIP-49, but I do think it's an important architectural distinction as we evolve the gateway beyond the initial REST write MVP. [1] https://lists.apache.org/thread/sf0qfxhr9j2g0dtxvp586p9zknxvpx6x Best regards, David Am Fr., 31. Juli 2026 um 05:02 Uhr schrieb Liting Liu (litiliu) via dev < [email protected]>: > Hi， Junbo > > > Thanks for the detailed proposal. I have a couple of questions about the > technology choice and the future SQL path. > > For the current scope—REST writes and metadata/DDL—have we considered > implementing the Gateway with a Java/Netty REST stack backed by the > existing Fluss Java client? This would be similar in spirit to how Flink > SQL Gateway reuses Flink’s native Java stack and Netty-based REST > infrastructure. The Fluss Java client already supports append, upsert, > delete, partial update, metadata operations, and lookup. It would be > helpful if the FIP could compare this alternative with the proposed > > I also have a question about the future DataFusion integration. Is > DataFusion expected to execute SQL queries inside the same Gateway process? > If so, complex or resource-intensive queries could compete with REST > ingestion and metadata operations for CPU and memory, and a query failure > or OOM could potentially affect the entire Gateway. Do we plan to provide > admission control, memory and concurrency limits, query > timeout/cancellation, spilling, or process-level isolation? > > If the initial read use case is mainly primary-key or prefix lookup, could > we expose a lightweight lookup REST API backed directly by the Fluss > client, without introducing DataFusion? DataFusion seems more valuable when > the intended scope includes general SQL capabilities such as scans, > filters, aggregations, and joins? If that is the long-term goal, would it > be preferable to keep complex SQL execution in a separate optional SQL > service or worker process, so that it does not increase the failure domain > of the REST write Gateway? > > > From: Michael Koepf <[email protected]> > Date: Thursday, July 30, 2026 at 22:15 > To: [email protected] <[email protected]> > Subject: Re: [DISCUSS] FIP-49: Fluss Gateway REST API > > Hi Juno, > > Thanks for the FIP, great initiative. > > A question that comes to my mind: Why not expose metrics via vendor > neutral OTEL instead of limiting it to Prometheus? > -- > Best, > MKO > > > > On 30.07.2026, at 12:23, Forward Xu <[email protected]> wrote: > > > > Thanks junbo, > > > > The proposal looks good to me in general. +1 > > > > Best, > > Foward > > > > Junbo Wang <[email protected]> 于2026年7月30日周四 11:49写道： > > > >> Hi all, I'd like to start a discussion on FIP-49: Fluss Gateway REST > API. > >> > >> The proposal is available here: > >> > https://cwiki.apache.org/confluence/spaces/FLUSS/pages/444334561/FIP-49+Fluss+Gateway+REST+API > >> > >> This FIP is an extension of FIP-32 (Multiprotocol Query Gateway) [1] and > >> does not conflict with it. Following the earlier suggestion to split the > >> gateway work into focused FIPs, FIP-49 defines the REST write path and > DDL > >> / metadata management in detail, while REST read and the SQL path > remain in > >> the scope of FIP-32. > >> > >> In short, it proposes a standalone, stateless fluss-gateway process > (Rust, > >> axum + tokio, built on fluss-rust) providing: > >> > >> HTTP/JSON ingestion into KV and Log tables, at-least-once, with per-row > >> success/failure reporting > >> Full database / table / partition DDL > >> Pluggable client authentication, and a service / user (act-as) identity > >> mode towards Fluss > >> Multi-cluster routing, Prometheus metrics, a unified error model > >> > >> No Fluss server-side changes are required— the gateway accesses existing > >> clusters purely as a fluss-rust client. > >> > >> Looking forward to your feedback and suggestions. > >> > >> > >> > >> [1] https://lists.apache.org/thread/sf0qfxhr9j2g0dtxvp586p9zknxvpx6x > >> > >> > >> Best regards, > >> Junbo Wang > >> > >> >
