Hi Sagar, Thanks for the detailed FIP. Three comments below.
## 1. Public-interface docs need a mapping and a worked example The `lance.*` properties currently stand alone in the FIP — to understand any of them, a reader has to cross-reference the LanceDB docs. I'd like the FIP to add three things to the public-interface section: - An explicit table mapping each Fluss property to the LanceDB API call and parameter it maps to (e.g. `lance.index.type` → `Table.create_index(index_type=...)`). - An explicit mapping of each Fluss default to the corresponding LanceDB default, with rationale for any deliberate divergence. Skimming the FIP, several `lance.fts.*` defaults look like they differ from LanceDB upstream defaults (e.g. `stem`, `remove_stop_words`, `ascii_folding`), and `lance.index.m`'s `(none)` effectively means LanceDB's hardcoded `20`. The reasons aren't stated. - A side-by-side example showing the same index expressed as (a) a Fluss `CREATE TABLE ... WITH (...)` statement, and (b) the equivalent LanceDB Python call. That makes the abstraction concrete for both reviewers and future users. Two specific things worth pinning down while you're in there: - `create_fts_index` has a legacy Tantivy path and a newer native FTS path (`use_tantivy=False`, now the upstream default). Which one is the FIP targeting? The parameter surface and on-disk format both differ. - `Table.create_index` and `Table.optimize` have additional parameters (`replace`, `num_bits`, `delete_unverified`, `retrain`, …) that aren't currently mapped. Either include them or explain why they're deliberately hidden — `replace` in particular matters because the state machine relies on `create_index` being idempotent, which is only true with `replace=True`. ## 2. Who builds the index? Execution model and horizontal scaling The FIP assigns the index lifecycle to the `LanceLakeCommitter`, but the deeper execution-model question is not yet answered: - Where does the committer (and therefore `create_index()` / `optimize()`) physically run? My reading of FIP-5 is that the committer lives inside the Tiering Service workers. Is that the intent here? - If so, the Tiering Service now has to **embed LanceDB**. LanceDB is a Rust core with Python and Node bindings — there is no first-party Java client today. How is it embedded into the JVM-based tiering worker? JNI over the Rust core? A sidecar subprocess? Something else? This is a non-trivial dependency to take on and deserves explicit discussion in the FIP. - How is index work **distributed** across Tiering Service workers? Per-table affinity? Coordinator-assigned? With a single large table whose one heavy index takes hours to build, does the work pin to one worker, or can it be split? If the asynchronous build effectively runs in-process inside the worker that initiated it, then horizontal scaling is per-table at best. ## 3. Configuration drift after the table exists What happens if a user changes `lance.index.type` (or `metric`, `num_partitions`, …) on a table that already has a COMPLETED index? The state machine only models `ABSENT → IN_PROGRESS → COMPLETED`, with no "config changed, rebuild" transition. We need an explicit answer here — silently keep the old index, force a rebuild, or reject the property change at DDL time. Each option has different operational implications and the FIP should commit to one. Looking forward to your thoughts. Best, Jark On Fri, 29 May 2026 at 22:41, Sagar <[email protected]> wrote: > > Hi , > > Bumping this thread. Please take a look. > > Sagar. > > On Sat, 23 May 2026 at 9:53 AM, Sagar <[email protected]> wrote: > > > Hi, > > > > I created FIP-44 > > <https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=429064608> > > to > > enhance the LanceDB integration with Fluss. > > > > Please review. > > > > Sagar. > >
