Hi David,

Great proposal, strong +1 on building the gateway in Rust on top of
fluss-rust and DataFusion.
A few observations from the fluss-rust side.


What fluss-rust supports today vs. what the gateway assumes:

The FIP's FlussBackend trait defines six data operations: lookup,
batch_lookup, prefix_lookup, log_scan, get_schema, and
list_databases/list_tables.
Of these, fluss-rust currently supports single-key KV lookup (via
Lookuper), log scanning (via RecordBatchLogScanner, which returns Arrow
RecordBatch natively), and all implemented admin/metadata APIs.

Three operations don't exist in fluss-rust yet: prefix lookup (the wire
protocol's PrefixLookupRequest and Java's PrefixKeyLookuper exist, Rust has
a TODO), batch lookup (the RPC already accepts multiple keys per bucket,
Rust sends one at a time), and LimitScan (needed for SELECT * FROM kv_table
LIMIT N through DataFusion). None are architectural blockers, but they
should be scoped into the delivery plan explicitly.

Lookuper::lookup() currently takes &mut self because the internal
KeyEncoder reuses a byte buffer across calls for performance. A gateway
caching Lookuper instances per table needs &self for concurrent access. The
required fix - wrap encoders in Mutex, same pattern UpsertWriter already
uses in production. Lock hold time is sub-microsecond (byte encoding only,
no I/O). It can be addressed ahead of the gateway work, ofc.

Reply via email to