merlimat opened a new pull request #11287:
URL: https://github.com/apache/pulsar/pull/11287
### Motivation
This change fixes few issues in the compaction mechanism, the
* When a reader is created, reading from "earliest" message, it should read
the compacted data and then continue from the next message.
* When the compaction consumer starts, it shouldn't seek to the beginning.
This causes 2 issues:
* Rescanning of the topic each time the compaction runs
* Keys that are being dropped from the topic are also getting dropped
from the compacted view, while in fact they should be there until explicitly
deleted (with an empty message for a key).
The main source of the problem is that when creating a cursor on "earliest"
message, the cursor gets automatically adjusted on the earliest message
available to read. This confuses the check for the read-compacted because it
may think the reader/consumer is already ahead of the compaction horizon.
### Modifications
Introduced a "isFirstRead" flag to make sure we double check the start
message id and use `MessageId.earliest` instead of the earliest available
message to read on the topic. After the first read, the positioning will be
fine.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
