dat-vikash commented on issue #7030:
URL: https://github.com/apache/druid/issues/7030#issuecomment-1126194538
@devinbost The topic pesistence is set at creation
Persistent Topic: `persistent://property/cluster/namespace/topic`
All messages will be stored on disk
Non persistent: `non-persistent://property/cluster/namespace/topic`
Messages will not be persisted to Disk
The durability refers to the cursor persistence. `Durable` subscriptions
have the cursor persisted
>If a broker restarts from a failure, it can recover the cursor from the
persistent storage (BookKeeper), so that messages can continue to be consumed
from the last consumed position.
While `non-durable` the cursor is lost on bookie restart.
>Once a broker stops, the cursor is lost and can never be recovered, so that
messages can not continue to be consumed from the last consumed position.
Since druid's supervisor creates short lived tasks and we want to support
resumption from any point in the stream, we require a `non-durable`
subscription mode. `durable` subscription cursor position is enforced by the
bookie and cannot be overridden, meaning we could not support resumption from
any point in the stream. So to properly support resiliency and idempotent
resumption, we need to use the `non-durable` subscription.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
