jiafu1115 commented on code in PR #23520: URL: https://github.com/apache/kafka/pull/23520#discussion_r4256643604
########## docs/operations/tiered-storage.md: ########## @@ -167,6 +167,27 @@ $ bin/kafka-topics.sh --delete --topic tieredTopic --bootstrap-server localhost: After topics are deleted, you're safe to set `remote.log.storage.system.enable=false` in the broker configuration. +## Delay Upload + +By default, non-active log segments are uploaded to remote storage as soon as they are eligible, keeping data in remote storage as up to date as possible. As a result, some segments before local retention are stored redundantly in both the local and remote tiers. If you do not need remote storage to always hold the newest data, you can delay upload with `remote.copy.lag.ms` and `remote.copy.lag.bytes` to reduce this redundancy and save remote storage space. The remote log manager uses these time-based and size-based lag parameters to decide when a segment becomes eligible for upload; the same settings are also available at the broker level as `log.remote.copy.lag.ms` and `log.remote.copy.lag.bytes`. + +For example, you can configure both settings for one topic as follows: + +```properties Review Comment: @chia7712 Thanks for your comments！ Would it be possible to avoid adding more examples to this section? Since this is only an extended feature, I’m concerned that dedicating too much space to it might not be appropriate. BTW: Is the description _"keep the last 2 days of data on local disk, so reads of recent data don't hit remote storage"_ somewhat misleading? The goal of delaying uploads is to reduce redundant data in remote storage, not to keep data on local disk for longer. The local retention period is still controlled by the local retention policy. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
