Thanks for the MKO's advice — good point! We started with Prometheus pull for the MVP mainly for simplicity — it works out of the box and aligns with how Fluss servers expose metrics today. That said, it doesn't preclude OTel: a Collector can scrape the /metrics endpoint and re-export via OTLP, and we plan to keep the config extensible so a native OTLP exporter can be added later.
Thanks for the great questions, Liting Liu. > Why Rust instead of Java/Netty? Three reasons: (a) the gateway is a proxy-style component where tail latency and footprint matter — Rust gives GC-free predictable latency and a much smaller memory footprint as a single static binary; (b) ecosystem fit — Fluss data is Arrow-native, and the Rust stack (arrow-rs, DataFusion, pgwire) keeps a zero-copy path end to end; (c) the gateway also helps drive the fluss-rust client to production readiness and grow the Rust ecosystem. > a query failure or OOM could potentially affect the entire Gateway. Query execution will run on a dedicated tokio runtime with a bounded thread pool, so it can't starve the write path. For memory, DataFusion's built-in MemoryPool sets per-query budgets — an over-limit query fails alone, not the process. We'll also add concurrency limits and query timeouts. If heavy workloads ever emerge, a separate query worker remains an option. > a lightweight lookup REST API backed directly by the Fluss client, Agreed — the initial read path is a lightweight lookup API backed directly by the Fluss client, no DataFusion. DataFusion comes later during FIP-32 implementation, and only for lightweight queries (filter/projection/limit). -- Best regards, Junbo Wang Anton Borisov <[email protected]> 于2026年8月4日周二 10:30写道： > Hi Junbo, > > Nice proposal, thank you for driving it! > I like the direction overall, +1 > > My questions below: > > 1. Have you seen the new cooperative backpressure? > #3463 [1] went in on 31 July, a day after this thread started. Tablet > servers now > send a per-bucket pressure value on PutKv responses, the client throttles > on > that by up to 3 seconds by default, and the server can reject with a > retriable > StorageBackpressureException when RocksDB is close to stalling. > > Inside an HTTP request that becomes invisible latency, the caller waits and > isn't told why. It also looks like the thing your rate limiting wants to be > driven by. A 429 with Retry-After says "slow down" in a way HTTP clients > already understand, and a fixed requests/sec limit can't track what the > cluster is actually doing. Worth pulling into the FIP? > > 2. What is a "connection" in service mode, and how many are there? > The page says service mode uses "one shared pool per cluster", and > connection.max is a "total connection cap (exceeded -> 429)". But it > doesn't > say what a connection is, or how many service mode opens. > > It matters because a Fluss Connection isn't a socket. It's documented as > heavy-weight to create, thread-safe and meant to be shared, and each one > carries a 64 MB write buffer, a sender task, a writer id and sockets to > every > tablet server. If service mode really is one shared connection, then the > 512 > cap and its 429 never fire, and it'd be clearer to say so. If it's many, > they'll batch worse than one would, since each accumulator sees only a > fraction of the traffic. Kafka REST Proxy shares a single producer across > requests for that reason, and only goes per-principal when identities > differ. > > 3. Is there a request timeout on writes? > The client retries forever by default, and "leader not available" is one of > the errors it retries. So the row in your example wouldn't fail, it would > wait. And if the gateway stops waiting and reports a failure, does the row > stay queued and land later? That would make "failed" sometimes mean > "succeeded, just later". > > 4. Which tables get per-row results? > For Log tables without a bucket key you use append_arrow_batch, which > returns > one result for the whole batch and stops at the first failure. I don't > think > per-row reporting is possible on that path. Is it meant to be > all-or-nothing > there? > > 5. What stops a duplicate when a client retries? > Our idempotence catches the writer resending its own batch. A retried HTTP > request is a different thing - the rows arrive fresh, so nothing links > them to > the first attempt. The writer id also belongs to one process, so a second > gateway can't pick up where the first left off. > > 6. Is per-user authorization on the roadmap? > The page says Fluss-side ACLs see only the gateway account, so every > request > reaches the cluster as gateway_svc and the server can't tell one caller > from > another. Is something planned for later: which principals may write to > which > tables, or is the gateway meant only for deployments where one shared > identity is fine? > > Last one, the type names. Fluss spells types two ways: INT and > TIMESTAMP_LTZ > in SQL, INTEGER and TIMESTAMP_WITH_LOCAL_TIME_ZONE in DataTypeJsonSerde. > The > FIP uses the SQL names with the serde's JSON shape, so I don't think the > examples parse as written. > > Which did you intend? If it's the SQL names, the gateway has to convert > them > before calling CreateTable, and that mapping is worth writing into the FIP. > > Thank you! > > -- Anton > > [1] https://github.com/apache/fluss/pull/3463 > > пн, 3 авг. 2026 г. в 16:02, David Reger <[email protected]>: > > > > I think this also relates to one of the architectural discussions we've > > recently had around FIP-32 [1]. > > > > My understanding is that there is a useful distinction between *native > > Fluss operations* and *SQL/query execution*. > > > > For example, writes, metadata operations, and potentially lightweight > > native reads (such as point lookups) could continue to build directly on > > the fluss-rust foundation, whereas DataFusion would primarily serve the > > SQL/query path. This would allow the REST gateway to remain lightweight > > while keeping the query layer reusable for SQL-based protocols in the > > future. > > > > I don't think this necessarily changes the scope of FIP-49, but I do > think > > it's an important architectural distinction as we evolve the gateway > beyond > > the initial REST write MVP. > > > > [1] https://lists.apache.org/thread/sf0qfxhr9j2g0dtxvp586p9zknxvpx6x > > > > Best regards, > > > > David > > > > Am Fr., 31. Juli 2026 um 05:02 Uhr schrieb Liting Liu (litiliu) via dev < > > [email protected]>: > > > > > Hi， Junbo > > > > > > > > > Thanks for the detailed proposal. I have a couple of questions about > the > > > technology choice and the future SQL path. > > > > > > For the current scope—REST writes and metadata/DDL—have we considered > > > implementing the Gateway with a Java/Netty REST stack backed by the > > > existing Fluss Java client? This would be similar in spirit to how > Flink > > > SQL Gateway reuses Flink’s native Java stack and Netty-based REST > > > infrastructure. The Fluss Java client already supports append, upsert, > > > delete, partial update, metadata operations, and lookup. It would be > > > helpful if the FIP could compare this alternative with the proposed > > > > > > I also have a question about the future DataFusion integration. Is > > > DataFusion expected to execute SQL queries inside the same Gateway > process? > > > If so, complex or resource-intensive queries could compete with REST > > > ingestion and metadata operations for CPU and memory, and a query > failure > > > or OOM could potentially affect the entire Gateway. Do we plan to > provide > > > admission control, memory and concurrency limits, query > > > timeout/cancellation, spilling, or process-level isolation? > > > > > > If the initial read use case is mainly primary-key or prefix lookup, > could > > > we expose a lightweight lookup REST API backed directly by the Fluss > > > client, without introducing DataFusion? DataFusion seems more valuable > when > > > the intended scope includes general SQL capabilities such as scans, > > > filters, aggregations, and joins? If that is the long-term goal, would > it > > > be preferable to keep complex SQL execution in a separate optional SQL > > > service or worker process, so that it does not increase the failure > domain > > > of the REST write Gateway? > > > > > > > > > From: Michael Koepf <[email protected]> > > > Date: Thursday, July 30, 2026 at 22:15 > > > To: [email protected] <[email protected]> > > > Subject: Re: [DISCUSS] FIP-49: Fluss Gateway REST API > > > > > > Hi Juno, > > > > > > Thanks for the FIP, great initiative. > > > > > > A question that comes to my mind: Why not expose metrics via vendor > > > neutral OTEL instead of limiting it to Prometheus? > > > -- > > > Best, > > > MKO > > > > > > > > > > On 30.07.2026, at 12:23, Forward Xu <[email protected]> wrote: > > > > > > > > Thanks junbo, > > > > > > > > The proposal looks good to me in general. +1 > > > > > > > > Best, > > > > Foward > > > > > > > > Junbo Wang <[email protected]> 于2026年7月30日周四 11:49写道： > > > > > > > >> Hi all, I'd like to start a discussion on FIP-49: Fluss Gateway REST > > > API. > > > >> > > > >> The proposal is available here: > > > >> > > > > https://cwiki.apache.org/confluence/spaces/FLUSS/pages/444334561/FIP-49+Fluss+Gateway+REST+API > > > >> > > > >> This FIP is an extension of FIP-32 (Multiprotocol Query Gateway) > [1] and > > > >> does not conflict with it. Following the earlier suggestion to > split the > > > >> gateway work into focused FIPs, FIP-49 defines the REST write path > and > > > DDL > > > >> / metadata management in detail, while REST read and the SQL path > > > remain in > > > >> the scope of FIP-32. > > > >> > > > >> In short, it proposes a standalone, stateless fluss-gateway process > > > (Rust, > > > >> axum + tokio, built on fluss-rust) providing: > > > >> > > > >> HTTP/JSON ingestion into KV and Log tables, at-least-once, with > per-row > > > >> success/failure reporting > > > >> Full database / table / partition DDL > > > >> Pluggable client authentication, and a service / user (act-as) > identity > > > >> mode towards Fluss > > > >> Multi-cluster routing, Prometheus metrics, a unified error model > > > >> > > > >> No Fluss server-side changes are required— the gateway accesses > existing > > > >> clusters purely as a fluss-rust client. > > > >> > > > >> Looking forward to your feedback and suggestions. > > > >> > > > >> > > > >> > > > >> [1] > https://lists.apache.org/thread/sf0qfxhr9j2g0dtxvp586p9zknxvpx6x > > > >> > > > >> > > > >> Best regards, > > > >> Junbo Wang > > > >> > > > >> > > > >
