merlimat opened a new pull request, #25822: URL: https://github.com/apache/pulsar/pull/25822
## Summary First implementation PR for [PIP-475: Regular-to-Scalable Topic Migration](https://github.com/apache/pulsar/blob/master/pip/pip-475.md). Lays the broker-side foundation: the scalable-topic lookup session now accepts `persistent://` and short-form topic names in addition to `topic://`, and synthesises a layout that wraps an existing regular (partitioned or non-partitioned) topic as one or more **special segments**. V5 clients can then operate against any topic through the scalable surface, even before it has been migrated — the SDK consumption of the synthetic layout will land in a follow-up PR. ### Wire-format additions - `SegmentInfoProto.underlying_topic_name` — when set, the segment wraps the named `persistent://...` topic instead of having its own `segment://...` URI. - `CommandScalableTopicUpdate.resolved_topic_name` — the canonical `topic://t/n/x` identity, set on every successful response so a client that looked up via `persistent://...` or short-form learns the resolved name. ### Java surface - `SegmentInfo.underlyingTopicName` + `activeSpecial(...)` factory + `isSpecial()` check. - `TopicName.toScalableTopic()` — derive the canonical scalable identity from a name in any input domain. ### Broker behaviour - `ServerCnx` rejects `non-persistent://` scalable-topic lookups early. - `DagWatchSession.start()` falls back to a synthetic layout for `persistent://` input when no scalable metadata exists at the canonical path: - Non-partitioned → 1 special segment covering the full hash range. - Partitioned with N partitions → N special segments wrapping each `persistent://...-partition-K` with equal-width contiguous hash ranges. - The metadata-store watch remains active so a later migration that writes scalable metadata to the same path is observed and transparently replaces the synthetic layout. - `pushUpdate` emits the canonical `topic://...` `resolvedTopicName`, plus the new special-segment marker when set. ## Test plan - [x] `DagWatchSessionTest` covers synthetic layouts for both non-partitioned and partitioned regular topics, and verifies the wire response carries `resolved_topic_name` and `underlying_topic_name` (17/17 passing). - [x] `CommandsScalableTopicTest` verifies `resolved_topic_name` round-trips through the new `Commands.newScalableTopicUpdate` signature (8/8 passing). - [x] `ScalableTopicControllerTest` regression check (33/33 passing). - [ ] End-to-end V5 SDK test against synthetic layouts — lands with the SDK consumption PR. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
