JunRuiLee commented on PR #563: URL: https://github.com/apache/paimon-rust/pull/563#issuecomment-5031031403
> Thanks for sharing #568. I've opened #571 as a draft so the scope and implementation are visible. > > My work focuses on the existing non-primary-key/global-index full-text path and DataFusion integration, rather than the primary-key full-text and hybrid search path covered by #568. It migrates the legacy in-repository Tantivy implementation to `paimon-ftindex-core` and covers snapshot index-manifest reads, `fast`/`full`/`detail` modes, deletion vectors, and raw fallback. > > There is some overlap with #563 in the dependency and reader wiring. I'm happy to rebase on or reuse that foundation to avoid duplication. Please let me know which integration order you would prefer. I'll also review #568 separately. Thanks — the scope is clear now, and I think our work is complementary with essentially no functional overlap: you're migrating the append/global-index full-text path (and DataFusion) onto paimon-ftindex-core, and #568 covers the primary-key full-text + hybrid path. The only real intersection is the shared piece: the paimon-ftindex-core dependency + the FullTextArchiveReader wiring (currently in #563). I'd suggest we land that reader foundation first, as its own small PR (#563), and then both of our stacks — your global-index path and my PK path — rebase onto it so the dependency and reader live in one place. That also lets us settle one shared detail: the pinned tag (I'm on v0.1.0-rc4, you're on v0.1.0-rc5) — we should converge on one and switch to the crates.io version before merging, since a git-only dependency blocks publishing. Works for me to have #563 go in first; happy to adjust the reader's API if your global path needs anything from it. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
