Hi Anton, Thank you very much for pushing this forward. I can help with it if needed. +1 from my side.On Thu, Apr 23, 2026 at 3:28 PM Anton Borisov < [email protected]> wrote:
Best, Hongshun > Hi Leonard, > > Thanks a lot for the support! Appreciate it > > Giannis has already granted me the Confluence permissions, so I will > migrate > this draft there. > > - Anton > > чт, 23 апр. 2026 г. в 09:14, Leonard Xu <[email protected]>: > > > > +1 from my side, the motivation and the proposed merge plan makes sense > to me. > > > > Btw, could you left your jira account name? Anton, we can add the > Confluence edit permission for you. > > > > Best, > > Leonard > > > > > 2026 4月 23 14:50，Anton Borisov <[email protected]> 写道： > > > > > > Hi all, > > > > > > Following up on the 0.2.0-incubating roadmap thread, where the repo > > > merge came through as the strongest priority signal, I am opening this > > > as its own [DISCUSS] as promised. > > > > > > Per the consensus on that thread, 0.2.0 development continues on > > > fluss-rust in parallel and is not blocked by this FIP. > > > > > > I do not currently have edit rights on the FLUSS Confluence space, so > > > I am posting the full draft inline. Once discussion converges, a > > > committer can paste the agreed text into a FIP-XX page. Please treat > > > the text below as a working draft; I am happy to revise it as feedback > > > comes in. > > > > > > =============================================================== > > > FIP-XX: Consolidate apache/fluss-rust into apache/fluss > > > > > > Discussion thread: this thread > > > Vote thread: TBD > > > Release: Fluss 1.0 (target) > > > Status: Under Discussion > > > > > > Motivation > > > ============= > > > > > > Fluss source code currently lives in two ASF repos. > > > > > > 1.1 Current layout today > > > > > > apache/fluss > > > ├─ fluss-server/ > > > ├─ fluss-client/ > > > ├─ fluss-rpc/ > > > │ └─ src/main/proto/*.proto > > > ├─ fluss-flink/, fluss-spark/, ... > > > ├─ website/ > > > │ └─ fluss.apache.org > > > └─ .github/workflows/ > > > > > > apache/fluss-rust > > > ├─ crates/fluss/ > > > │ └─ src/proto/*.proto > > > │ hand-ported subset > > > ├─ bindings/python/ > > > ├─ bindings/cpp/ > > > ├─ bindings/elixir/ > > > ├─ website/ > > > │ └─ clients.fluss.apache.org > > > └─ .github/workflows/ > > > > > > fluss-rust was incubated in its own repo so the Rust core could move > > > quickly. With 1.0 GA on the horizon, the costs of the split now exceed > > > the benefit. > > > > > > As discussed on the 0.2.0 thread, 1.0 should represent the complete > > > form of the project: all language SDKs and unified release > > > coordination from a single repository, to signal maturity to users. A > > > fragmented multi-repo layout at GA undermines that message. > > > > > > 1.2 Concrete costs today > > > > > > Protobuf schema is duplicated and can diverge. The source of truth is > > > fluss-rpc/src/main/proto/FlussApi.proto (package fluss). > > > fluss-rust carries a hand-maintained subset at > > > crates/fluss/src/proto/fluss_api.proto (package proto). > > > The subset exists only because the two repos cannot share files; > > > nothing enforces consistency. > > > > > > Server version is pinned manually in two files: > > > crates/fluss-test-cluster/test-images.env and > > > bindings/elixir/test/support/cluster.ex, both pointing today to > > > apache/fluss:0.9.0-incubating, with no automation. > > > > > > Docs are split across two subdomains. fluss.apache.org/docs/apis/ > > > *-client.md pages are stubs that link out to > > > clients.fluss.apache.org/user-guide//... This means two > > > Docusaurus builds and two deployment pipelines. > > > > > > Infrastructure is duplicated: 8 CI workflows in each repo, > > > duplicated LICENSE / NOTICE / DISCLAIMER files at repo root, and > > > two release trains. > > > > > > Cross-repo changes cost two PRs, two reviews, and a Docker image cut > > > per wire-protocol change. > > > > > > 1.3 ASF peer evidence > > > > > > ASF peer evidence points the other way: mature ASF projects > > > overwhelmingly colocate language bindings with core. > > > > > > Apache Thrift -> apache/thrift/lib/{java,py,cpp,rs,go,rb,...} > > > > > > Apache Avro -> apache/avro/lang/{java,py,c,rust,ruby,...} > > > > > > Apache Beam -> apache/beam/sdks/{java,python,go,typescript} > > > > > > PyFlink -> apache/flink/flink-python/ > > > > > > PySpark -> apache/spark/python/ > > > > > > Apache Arrow is the visible counter-example: arrow-rs, arrow-go, > > > arrow-java, and arrow-js live in separate repos. Apache Avro also > > > externalized its Go binding (apache/avro-go) on a similar trajectory. > > > In both cases, the split happened after the format stabilized and > > > after the specific language community had grown large enough to > > > self-sustain. > > > > > > Fluss is at neither milestone. The wire protocol is still evolving > > > aggressively pre-1.0, and the Rust, Python, C++, and Elixir bindings > > > are all co-maintained by the same contributors. Splitting now incurs > > > coordination overhead that Arrow and Avro only accepted at maturity, > > > without yet having their enabling conditions. > > > > > > The Flink connector-externalization retrospective is sometimes cited, > > > but does not really apply: connectors are N external-system > > > integrations maintained by independent communities, not N language > > > wrappers over one wire protocol. > > > > > > Public interfaces > > > ==================== > > > > > > None. Published artifacts remain unchanged: > > > > > > Maven Central: org.apache.fluss:fluss-* > > > > > > crates.io: fluss-rs > > > > > > PyPI: pyfluss > > > > > > git: Elixir fluss (planned hex.pm publication) > > > > > > CMake / Bazel: fluss-cpp > > > > > > Proposed changes > > > =================== > > > > > > Jark outlined the merge as five workstreams: code migration, CI > > > unification, documentation restructuring, issue migration, and > > > contributor workflow changes. Sections 3 and 4 address each. > > > > > > 3.1 Code migration - target layout > > > > > > apache/fluss/ > > > ├─ fluss-server/ (Java, unchanged) > > > ├─ fluss-client/ (Java, unchanged) > > > ├─ fluss-rpc/ (single .proto source; see 3.4) > > > │ └─ src/main/proto/FlussApi.proto > > > ├─ fluss-flink/, fluss-spark/, ... (Java modules, unchanged) > > > ├─ fluss-rust/ (new: today's apache/fluss-rust tree) > > > │ ├─ Cargo.toml, Cargo.lock > > > │ ├─ crates/{fluss, examples, fluss-test-cluster}/ > > > │ ├─ bindings/{python, cpp, elixir}/ > > > │ ├─ justfile, MODULE.bazel, rust-toolchain.toml, > > > │ │ deny.toml, rustfmt.toml > > > │ └─ scripts/ > > > ├─ website/ (unified Docusaurus; see 3.3) > > > └─ .github/workflows/ (unified with path filters; see 3.2) > > > > > > fluss-rust mirrors sibling naming such as fluss-server/ and > > > fluss-flink/. Bindings stay nested because they share a Cargo > > > workspace, rust-toolchain.toml, deny.toml, and workspace-level > > > dependency resolution with the Rust core. > > > > > > 3.2 CI unification > > > > > > A single .github/workflows/ directory. Path filters are a hard > > > requirement: > > > > > > Java CI: pom.xml, fluss-*/** (excluding fluss-rust/), tools/** > > > > > > Rust core CI: fluss-rust/Cargo.*, fluss-rust/crates/** > > > OR fluss-rpc/src/main/proto/** > > > > > > Binding CI: fluss-rust/bindings//** > > > OR fluss-rpc/src/main/proto/** > > > > > > Docs build: website/** > > > > > > Existing release workflows move over unchanged. Their triggers are > > > tag-based and already disjoint from each other. > > > > > > 3.3 Documentation restructuring > > > > > > fluss/website/docs/ > > > ├─ intro.md > > > ├─ quickstart/ > > > ├─ concepts/ > > > ├─ clients/ (new section) > > > │ ├─ support-matrix.md > > > │ ├─ java/ > > > │ ├─ rust/ > > > │ ├─ python/ > > > │ └─ cpp/ > > > ├─ engine-flink/, engine-spark/, ... > > > └─ ... > > > > > > clients.fluss.apache.org is retired. Redirects from old paths use > > > @docusaurus/plugin-client-redirects, with mappings generated by > > > walking the two sidebars.ts files. The > > > fluss-rust/.github/workflows/deploy_documentation.yml workflow is > > > removed; deployment converges on the existing fluss/website -> > > > apache/fluss-website dispatch. > > > > > > 3.4 Protobuf single source of truth > > > > > > Today, crates/fluss/src/proto/fluss_api.proto exists as a > > > hand-maintained subset of the Java-side canonical proto solely because > > > the two repos could not share files. The merge removes that > > > constraint, and this FIP removes the subset. > > > > > > After the structural merge: > > > > > > Delete fluss-rust/crates/fluss/src/proto/fluss_api.proto. > > > > > > Point fluss-rust/crates/fluss/build.rs at the canonical proto: > > > > > > config.compile_protos( > > > &["../../../fluss-rpc/src/main/proto/FlussApi.proto"], > > > &["../../../fluss-rpc/src/main/proto"], > > > )?; > > > > > > Update the .bytes() prefixes in build.rs from ".proto." to > > > ".fluss." because the canonical proto declares package fluss. > > > > > > Update lib.rs include! from "/proto.rs" to "/fluss.rs", since > > > prost names the generated file after the proto package. > > > > > > The vendored copy exists only to satisfy crates.io packaging rules; > > > the monorepo build reads the canonical proto directly. For crates.io > > > publishing, vendor the canonical .proto into the crate at > > > cargo-publish time via scripts/release.sh, with build.rs falling > > > back to the in-crate copy when the monorepo path is absent. > > > > > > All existing crate::proto::TypeName call sites compile unchanged, > > > because the pub mod proto { ... } wrapper in lib.rs preserves the > > > import path. > > > > > > After this, adopting a new server RPC becomes a single-PR change: the > > > message lands in the canonical proto and Rust types appear > > > automatically. Nothing remains to drift against, so no CI drift-gate > > > is needed. Section 6 covers why a drift-gate alone would be the wrong > > > answer. > > > > > > 3.5 Release cadence > > > > > > Java server and Rust client keep their current independent SemVer > > > cadences until 1.0 GA. From 1.0 onward, both trees cut a jointly > > > numbered X.Y.Z release from a single tag; patch releases can still > > > ship independently where scope is binding-only or server-only. > > > Binding-specific SemVer continues where public APIs diverge, for > > > example if a pyfluss major bump is driven by PyO3 ABI changes. > > > > > > Open to revision: the post-1.0 release model is listed in section 8 > > > and likely deserves its own [DISCUSS] once this FIP lands. > > > > > > Compatibility, deprecation, and migration plan > > > ================================================= > > > > > > 4.1 Phases > > > > > > [0 Agreement] -> [1 Dry run] -> [2 Merge window] -> [3 Archive] -> [4 > Cleanup] > > > > > > DISCUSS + VOTE > > > > > > fork dry run > > > > > > follow-up PRs > > > > > > redirects > > > > > > INFRA ticket > > > > > > Agreement. This FIP's DISCUSS + VOTE. INFRA ticket filed and > > > resolved before VOTE is called, so the migration plan is confirmed > > > viable rather than assumed. > > > > > > Dry run on a fork: verify CI, path filters, docs build, redirects, > > > and git log --follow across the boundary. > > > > > > Merge window. Short freeze on apache/fluss-rust; structural merge > > > commit into apache/fluss. Protobuf (3.4) and docs (3.3) > > > consolidation land as separate follow-up PRs for reviewability. > > > > > > Archive. INFRA archives apache/fluss-rust read-only; issues, PRs, > > > commits, and tags remain as permalinks. Subdomain redirects go > > > live. A pinned README on the archived repo points to apache/fluss, > > > since GitHub does not forward content-moves between repos > > > automatically. > > > > > > Cleanup. Consolidate root-level duplicates such as LICENSE / > > > NOTICE / DISCLAIMER and contributor docs. > > > > > > 4.2 History preservation > > > > > > On a clone of apache/fluss-rust: > > > > > > git filter-repo --to-subdirectory-filter fluss-rust > > > > > > In apache/fluss: > > > > > > git remote add rust ../fluss-rust-rewritten > > > git fetch rust > > > git merge --allow-unrelated-histories --no-ff rust/main > > > > > > This preserves git log --follow and git blame --follow on every > > > rewritten file. git subtree breaks log --follow on files inside the > > > subtree; git filter-branch is deprecated. > > > > > > 4.3 Issue migration > > > > > > Closed issues and PRs remain at their apache/fluss-rust archive URLs > > > as permalinks. Open issues at merge time are re-filed in apache/fluss > > > with cross-links, scripted via gh CLI. Open PRs are reopened by their > > > authors against the new path; GitHub does not support cross-repo PR > > > transfer. > > > > > > 4.4 Contributor workflow > > > > > > Contributors re-clone and re-open in-flight PRs against the new path. > > > Downstream artifact consumers see no change. Docs visitors follow > > > redirects. > > > > > > 4.5 Timeline > > > > > > There is no dependency on a specific release. Target completion is > > > before 1.0 GA, per the fluss-rust roadmap 0.2.0 DISCUSS consensus. > > > Within that bound, the community can pick the window that minimizes > > > rebase pain, most likely after 0.2.0-incubating is cut, to avoid > > > forcing in-flight roadmap work to rebase across the merge. > > > > > > Test plan > > > ============ > > > > > > In the merged repo, CI runs the full Java + Rust + all-binding > > > matrices on every PR. > > > > > > Path-filter correctness is explicitly tested: a Rust-only PR must > > > not trigger Java jobs, and vice versa. > > > > > > Protobuf: a change to fluss-rpc/src/main/proto/FlussApi.proto alone > > > produces a green Rust build with no Rust-side edit, confirming the > > > single-source claim. > > > > > > Docs: every fluss-rust/website/docs/user-guide/... page renders > > > under fluss.apache.org/docs/clients/...; sampled old URLs redirect. > > > > > > History: git log --follow on a sample migrated file shows full > > > pre-merge history. > > > > > > Rejected alternatives > > > ======================== > > > > > > Status quo. The section 1 costs recur every release cycle and grow > > > with each new binding. > > > > > > Drift-gate instead of full proto consolidation. Keep the Rust > > > subset and add CI to flag divergence on messages present in both > > > files. Rejected because it catches only half the drift problem, > > > preserves a hand-maintained file whose only reason to exist was the > > > cross-repo constraint the merge removes, and adds custom comparison > > > logic to maintain. > > > > > > Polyglot-repo federation, Iceberg-style. This works for Iceberg > > > because bindings have independent maintainer communities and the > > > table-format spec is stable enough that protocol-evolution coupling > > > is low. Neither holds for Fluss pre-1.0: bindings are co-maintained > > > by the same team, and the wire protocol is still evolving > > > aggressively. For Fluss in the run-up to 1.0, locking bindings to > > > server evolution is a feature, not a bug. > > > > > > Apache Arrow precedent (arrow-rs split, 2021). > > > https://arrow.apache.org/blog/2021/05/04/rust-dev-workflow/ > > > Section 1 covers the structural frame: Arrow split after > > > stabilization and after each binding had grown an autonomous > > > community. Arrow's specific 2021 rationale also inverts for Fluss. > > > > > > Git submodules. Standard UX problems and does not solve the docs > > > split. > > > > > > Further splitting into per-binding repos. This multiplies every cost > > > listed in section 1. > > > > > > Flatten bindings to the top level of the merged repo. This breaks > > > the shared Cargo workspace. Keeping them under fluss-rust/ > > > preserves it; rearrangement remains an option later. > > > > > > Risks > > > ======== > > > > > > CI blast radius if path filters are misconfigured. Mitigated by > > > sections 3.2 and 5. > > > > > > In-flight PRs stall during the freeze. Mitigated by maintainers > > > proactively reopening high-value PRs. > > > > > > Open questions > > > ================= > > > > > > Directory name for the Rust tree. Proposal: fluss-rust/ > > > (mirrors fluss-server/ and fluss-flink/ sibling naming). > > > Alternatives: rust/, clients/rust/. > > > > > > Docs section name. Proposal: docs/clients/ to unify client and > > > binding docs under one heading, while keeping docs/engine-*/ for > > > pipeline integrations. Alternative: keep docs/apis/ to reduce link > > > churn during migration. > > > > > > Post-1.0 release model. Section 3.5 proposes synchronized X.Y.Z > > > across Java and Rust with independent patches. Alternative: > > > Arrow-style coordinated-but-independent versioning. This likely > > > deserves its own [DISCUSS] once this FIP lands. > > > > > > Looking forward to feedback. > > > > > > Best, > > > Anton > > >
