Hi Yong, Thanks for driving this proposal.
Could we add an option to the `RawReader` API, such as a `boolean includingMarkers` flag, when creating the `RawReader`? >From a Pulsar perspective, there are two groups of people: - Users who manage applications for publishing and consuming. - Pulsar operators and maintainers. The major issue with broker-level configuration is that operators or maintainers must manage the setting, even though they may not know how users will define their subscriptions. We don't need to change the existing API; we can simply add a new method to create a RawReader with the new option. RawReader is a low-level API that is rarely used in standard scenarios. Regards, Penghui On Thu, Feb 26, 2026 at 7:58 PM Yong Zhang <[email protected]> wrote: > Hi all, > > When using RawReader to read messages for specific use cases like: > - Compaction verification and monitoring > - Custom data analysis pipelines > - Debugging and troubleshooting > > Users want to read **all** messages from the topic, including marker > messages, so they can handle marker types on the reader side. Currently, > the broker filters out these marker messages before delivering them to any > consumer, including RawReader. > > This feature is particularly useful for: > 1. **Compaction monitoring**: Users want to verify compaction results by > reading all marker messages > 2. **Custom processing**: Users need access to all message types for their > specific use cases > 3. **Debugging**: Developers need to inspect marker messages for > troubleshooting > > > This proposal allows us to configure a subscription prefix to allow the > specific readers to read all the data from the topic. > > Here is the proposal: https://github.com/apache/pulsar/pull/25270 > > Please let me know if you have any questions. > > Best, > Yong >
