Hi Leonard, Thanks a lot for the support! Appreciate it
Giannis has already granted me the Confluence permissions, so I will migrate this draft there. - Anton чт, 23 апр. 2026 г. в 09:14, Leonard Xu <[email protected]>: > > +1 from my side, the motivation and the proposed merge plan makes sense to me. > > Btw, could you left your jira account name? Anton, we can add the Confluence > edit permission for you. > > Best, > Leonard > > > 2026 4月 23 14:50，Anton Borisov <[email protected]> 写道： > > > > Hi all, > > > > Following up on the 0.2.0-incubating roadmap thread, where the repo > > merge came through as the strongest priority signal, I am opening this > > as its own [DISCUSS] as promised. > > > > Per the consensus on that thread, 0.2.0 development continues on > > fluss-rust in parallel and is not blocked by this FIP. > > > > I do not currently have edit rights on the FLUSS Confluence space, so > > I am posting the full draft inline. Once discussion converges, a > > committer can paste the agreed text into a FIP-XX page. Please treat > > the text below as a working draft; I am happy to revise it as feedback > > comes in. > > > > =============================================================== > > FIP-XX: Consolidate apache/fluss-rust into apache/fluss > > > > Discussion thread: this thread > > Vote thread: TBD > > Release: Fluss 1.0 (target) > > Status: Under Discussion > > > > Motivation > > ============= > > > > Fluss source code currently lives in two ASF repos. > > > > 1.1 Current layout today > > > > apache/fluss > > ├─ fluss-server/ > > ├─ fluss-client/ > > ├─ fluss-rpc/ > > │ └─ src/main/proto/*.proto > > ├─ fluss-flink/, fluss-spark/, ... > > ├─ website/ > > │ └─ fluss.apache.org > > └─ .github/workflows/ > > > > apache/fluss-rust > > ├─ crates/fluss/ > > │ └─ src/proto/*.proto > > │ hand-ported subset > > ├─ bindings/python/ > > ├─ bindings/cpp/ > > ├─ bindings/elixir/ > > ├─ website/ > > │ └─ clients.fluss.apache.org > > └─ .github/workflows/ > > > > fluss-rust was incubated in its own repo so the Rust core could move > > quickly. With 1.0 GA on the horizon, the costs of the split now exceed > > the benefit. > > > > As discussed on the 0.2.0 thread, 1.0 should represent the complete > > form of the project: all language SDKs and unified release > > coordination from a single repository, to signal maturity to users. A > > fragmented multi-repo layout at GA undermines that message. > > > > 1.2 Concrete costs today > > > > Protobuf schema is duplicated and can diverge. The source of truth is > > fluss-rpc/src/main/proto/FlussApi.proto (package fluss). > > fluss-rust carries a hand-maintained subset at > > crates/fluss/src/proto/fluss_api.proto (package proto). > > The subset exists only because the two repos cannot share files; > > nothing enforces consistency. > > > > Server version is pinned manually in two files: > > crates/fluss-test-cluster/test-images.env and > > bindings/elixir/test/support/cluster.ex, both pointing today to > > apache/fluss:0.9.0-incubating, with no automation. > > > > Docs are split across two subdomains. fluss.apache.org/docs/apis/ > > *-client.md pages are stubs that link out to > > clients.fluss.apache.org/user-guide//... This means two > > Docusaurus builds and two deployment pipelines. > > > > Infrastructure is duplicated: 8 CI workflows in each repo, > > duplicated LICENSE / NOTICE / DISCLAIMER files at repo root, and > > two release trains. > > > > Cross-repo changes cost two PRs, two reviews, and a Docker image cut > > per wire-protocol change. > > > > 1.3 ASF peer evidence > > > > ASF peer evidence points the other way: mature ASF projects > > overwhelmingly colocate language bindings with core. > > > > Apache Thrift -> apache/thrift/lib/{java,py,cpp,rs,go,rb,...} > > > > Apache Avro -> apache/avro/lang/{java,py,c,rust,ruby,...} > > > > Apache Beam -> apache/beam/sdks/{java,python,go,typescript} > > > > PyFlink -> apache/flink/flink-python/ > > > > PySpark -> apache/spark/python/ > > > > Apache Arrow is the visible counter-example: arrow-rs, arrow-go, > > arrow-java, and arrow-js live in separate repos. Apache Avro also > > externalized its Go binding (apache/avro-go) on a similar trajectory. > > In both cases, the split happened after the format stabilized and > > after the specific language community had grown large enough to > > self-sustain. > > > > Fluss is at neither milestone. The wire protocol is still evolving > > aggressively pre-1.0, and the Rust, Python, C++, and Elixir bindings > > are all co-maintained by the same contributors. Splitting now incurs > > coordination overhead that Arrow and Avro only accepted at maturity, > > without yet having their enabling conditions. > > > > The Flink connector-externalization retrospective is sometimes cited, > > but does not really apply: connectors are N external-system > > integrations maintained by independent communities, not N language > > wrappers over one wire protocol. > > > > Public interfaces > > ==================== > > > > None. Published artifacts remain unchanged: > > > > Maven Central: org.apache.fluss:fluss-* > > > > crates.io: fluss-rs > > > > PyPI: pyfluss > > > > git: Elixir fluss (planned hex.pm publication) > > > > CMake / Bazel: fluss-cpp > > > > Proposed changes > > =================== > > > > Jark outlined the merge as five workstreams: code migration, CI > > unification, documentation restructuring, issue migration, and > > contributor workflow changes. Sections 3 and 4 address each. > > > > 3.1 Code migration - target layout > > > > apache/fluss/ > > ├─ fluss-server/ (Java, unchanged) > > ├─ fluss-client/ (Java, unchanged) > > ├─ fluss-rpc/ (single .proto source; see 3.4) > > │ └─ src/main/proto/FlussApi.proto > > ├─ fluss-flink/, fluss-spark/, ... (Java modules, unchanged) > > ├─ fluss-rust/ (new: today's apache/fluss-rust tree) > > │ ├─ Cargo.toml, Cargo.lock > > │ ├─ crates/{fluss, examples, fluss-test-cluster}/ > > │ ├─ bindings/{python, cpp, elixir}/ > > │ ├─ justfile, MODULE.bazel, rust-toolchain.toml, > > │ │ deny.toml, rustfmt.toml > > │ └─ scripts/ > > ├─ website/ (unified Docusaurus; see 3.3) > > └─ .github/workflows/ (unified with path filters; see 3.2) > > > > fluss-rust mirrors sibling naming such as fluss-server/ and > > fluss-flink/. Bindings stay nested because they share a Cargo > > workspace, rust-toolchain.toml, deny.toml, and workspace-level > > dependency resolution with the Rust core. > > > > 3.2 CI unification > > > > A single .github/workflows/ directory. Path filters are a hard > > requirement: > > > > Java CI: pom.xml, fluss-*/** (excluding fluss-rust/), tools/** > > > > Rust core CI: fluss-rust/Cargo.*, fluss-rust/crates/** > > OR fluss-rpc/src/main/proto/** > > > > Binding CI: fluss-rust/bindings//** > > OR fluss-rpc/src/main/proto/** > > > > Docs build: website/** > > > > Existing release workflows move over unchanged. Their triggers are > > tag-based and already disjoint from each other. > > > > 3.3 Documentation restructuring > > > > fluss/website/docs/ > > ├─ intro.md > > ├─ quickstart/ > > ├─ concepts/ > > ├─ clients/ (new section) > > │ ├─ support-matrix.md > > │ ├─ java/ > > │ ├─ rust/ > > │ ├─ python/ > > │ └─ cpp/ > > ├─ engine-flink/, engine-spark/, ... > > └─ ... > > > > clients.fluss.apache.org is retired. Redirects from old paths use > > @docusaurus/plugin-client-redirects, with mappings generated by > > walking the two sidebars.ts files. The > > fluss-rust/.github/workflows/deploy_documentation.yml workflow is > > removed; deployment converges on the existing fluss/website -> > > apache/fluss-website dispatch. > > > > 3.4 Protobuf single source of truth > > > > Today, crates/fluss/src/proto/fluss_api.proto exists as a > > hand-maintained subset of the Java-side canonical proto solely because > > the two repos could not share files. The merge removes that > > constraint, and this FIP removes the subset. > > > > After the structural merge: > > > > Delete fluss-rust/crates/fluss/src/proto/fluss_api.proto. > > > > Point fluss-rust/crates/fluss/build.rs at the canonical proto: > > > > config.compile_protos( > > &["../../../fluss-rpc/src/main/proto/FlussApi.proto"], > > &["../../../fluss-rpc/src/main/proto"], > > )?; > > > > Update the .bytes() prefixes in build.rs from ".proto." to > > ".fluss." because the canonical proto declares package fluss. > > > > Update lib.rs include! from "/proto.rs" to "/fluss.rs", since > > prost names the generated file after the proto package. > > > > The vendored copy exists only to satisfy crates.io packaging rules; > > the monorepo build reads the canonical proto directly. For crates.io > > publishing, vendor the canonical .proto into the crate at > > cargo-publish time via scripts/release.sh, with build.rs falling > > back to the in-crate copy when the monorepo path is absent. > > > > All existing crate::proto::TypeName call sites compile unchanged, > > because the pub mod proto { ... } wrapper in lib.rs preserves the > > import path. > > > > After this, adopting a new server RPC becomes a single-PR change: the > > message lands in the canonical proto and Rust types appear > > automatically. Nothing remains to drift against, so no CI drift-gate > > is needed. Section 6 covers why a drift-gate alone would be the wrong > > answer. > > > > 3.5 Release cadence > > > > Java server and Rust client keep their current independent SemVer > > cadences until 1.0 GA. From 1.0 onward, both trees cut a jointly > > numbered X.Y.Z release from a single tag; patch releases can still > > ship independently where scope is binding-only or server-only. > > Binding-specific SemVer continues where public APIs diverge, for > > example if a pyfluss major bump is driven by PyO3 ABI changes. > > > > Open to revision: the post-1.0 release model is listed in section 8 > > and likely deserves its own [DISCUSS] once this FIP lands. > > > > Compatibility, deprecation, and migration plan > > ================================================= > > > > 4.1 Phases > > > > [0 Agreement] -> [1 Dry run] -> [2 Merge window] -> [3 Archive] -> [4 > > Cleanup] > > > > DISCUSS + VOTE > > > > fork dry run > > > > follow-up PRs > > > > redirects > > > > INFRA ticket > > > > Agreement. This FIP's DISCUSS + VOTE. INFRA ticket filed and > > resolved before VOTE is called, so the migration plan is confirmed > > viable rather than assumed. > > > > Dry run on a fork: verify CI, path filters, docs build, redirects, > > and git log --follow across the boundary. > > > > Merge window. Short freeze on apache/fluss-rust; structural merge > > commit into apache/fluss. Protobuf (3.4) and docs (3.3) > > consolidation land as separate follow-up PRs for reviewability. > > > > Archive. INFRA archives apache/fluss-rust read-only; issues, PRs, > > commits, and tags remain as permalinks. Subdomain redirects go > > live. A pinned README on the archived repo points to apache/fluss, > > since GitHub does not forward content-moves between repos > > automatically. > > > > Cleanup. Consolidate root-level duplicates such as LICENSE / > > NOTICE / DISCLAIMER and contributor docs. > > > > 4.2 History preservation > > > > On a clone of apache/fluss-rust: > > > > git filter-repo --to-subdirectory-filter fluss-rust > > > > In apache/fluss: > > > > git remote add rust ../fluss-rust-rewritten > > git fetch rust > > git merge --allow-unrelated-histories --no-ff rust/main > > > > This preserves git log --follow and git blame --follow on every > > rewritten file. git subtree breaks log --follow on files inside the > > subtree; git filter-branch is deprecated. > > > > 4.3 Issue migration > > > > Closed issues and PRs remain at their apache/fluss-rust archive URLs > > as permalinks. Open issues at merge time are re-filed in apache/fluss > > with cross-links, scripted via gh CLI. Open PRs are reopened by their > > authors against the new path; GitHub does not support cross-repo PR > > transfer. > > > > 4.4 Contributor workflow > > > > Contributors re-clone and re-open in-flight PRs against the new path. > > Downstream artifact consumers see no change. Docs visitors follow > > redirects. > > > > 4.5 Timeline > > > > There is no dependency on a specific release. Target completion is > > before 1.0 GA, per the fluss-rust roadmap 0.2.0 DISCUSS consensus. > > Within that bound, the community can pick the window that minimizes > > rebase pain, most likely after 0.2.0-incubating is cut, to avoid > > forcing in-flight roadmap work to rebase across the merge. > > > > Test plan > > ============ > > > > In the merged repo, CI runs the full Java + Rust + all-binding > > matrices on every PR. > > > > Path-filter correctness is explicitly tested: a Rust-only PR must > > not trigger Java jobs, and vice versa. > > > > Protobuf: a change to fluss-rpc/src/main/proto/FlussApi.proto alone > > produces a green Rust build with no Rust-side edit, confirming the > > single-source claim. > > > > Docs: every fluss-rust/website/docs/user-guide/... page renders > > under fluss.apache.org/docs/clients/...; sampled old URLs redirect. > > > > History: git log --follow on a sample migrated file shows full > > pre-merge history. > > > > Rejected alternatives > > ======================== > > > > Status quo. The section 1 costs recur every release cycle and grow > > with each new binding. > > > > Drift-gate instead of full proto consolidation. Keep the Rust > > subset and add CI to flag divergence on messages present in both > > files. Rejected because it catches only half the drift problem, > > preserves a hand-maintained file whose only reason to exist was the > > cross-repo constraint the merge removes, and adds custom comparison > > logic to maintain. > > > > Polyglot-repo federation, Iceberg-style. This works for Iceberg > > because bindings have independent maintainer communities and the > > table-format spec is stable enough that protocol-evolution coupling > > is low. Neither holds for Fluss pre-1.0: bindings are co-maintained > > by the same team, and the wire protocol is still evolving > > aggressively. For Fluss in the run-up to 1.0, locking bindings to > > server evolution is a feature, not a bug. > > > > Apache Arrow precedent (arrow-rs split, 2021). > > https://arrow.apache.org/blog/2021/05/04/rust-dev-workflow/ > > Section 1 covers the structural frame: Arrow split after > > stabilization and after each binding had grown an autonomous > > community. Arrow's specific 2021 rationale also inverts for Fluss. > > > > Git submodules. Standard UX problems and does not solve the docs > > split. > > > > Further splitting into per-binding repos. This multiplies every cost > > listed in section 1. > > > > Flatten bindings to the top level of the merged repo. This breaks > > the shared Cargo workspace. Keeping them under fluss-rust/ > > preserves it; rearrangement remains an option later. > > > > Risks > > ======== > > > > CI blast radius if path filters are misconfigured. Mitigated by > > sections 3.2 and 5. > > > > In-flight PRs stall during the freeze. Mitigated by maintainers > > proactively reopening high-value PRs. > > > > Open questions > > ================= > > > > Directory name for the Rust tree. Proposal: fluss-rust/ > > (mirrors fluss-server/ and fluss-flink/ sibling naming). > > Alternatives: rust/, clients/rust/. > > > > Docs section name. Proposal: docs/clients/ to unify client and > > binding docs under one heading, while keeping docs/engine-*/ for > > pipeline integrations. Alternative: keep docs/apis/ to reduce link > > churn during migration. > > > > Post-1.0 release model. Section 3.5 proposes synchronized X.Y.Z > > across Java and Rust with independent patches. Alternative: > > Arrow-style coordinated-but-independent versioning. This likely > > deserves its own [DISCUSS] once this FIP lands. > > > > Looking forward to feedback. > > > > Best, > > Anton >
