stanislavkozlovski commented on code in PR #586: URL: https://github.com/apache/kafka-site/pull/586#discussion_r1499478729
########## 36/ops.html: ########## @@ -3984,95 +3984,27 @@ <h5 class="anchor-heading"><a id="tiered_storage_config_topic" class="anchor-lin If unset, The value in <code>retention.ms</code> and <code>retention.bytes</code> will be used. </p> -<h4 class="anchor-heading"><a id="tiered_storage_config_ex" class="anchor-link"></a><a href="#tiered_storage_config_ex">Quick Start Example</a></h4> - -<p>Apache Kafka doesn't provide an out-of-the-box RemoteStorageManager implementation. To have a preview of the tiered storage - feature, the <a href="https://github.com/apache/kafka/blob/trunk/storage/src/test/java/org/apache/kafka/server/log/remote/storage/LocalTieredStorage.java";>LocalTieredStorage</a> - implemented for integration test can be used, which will create a temporary directory in local storage to simulate the remote storage. -</p> - -<p>To adopt the `LocalTieredStorage`, the test library needs to be built locally</p> -<pre># please checkout to the specific version tag you're using before building it -# ex: `git checkout 3.6.1` -./gradlew clean :storage:testJar</pre> -<p>After build successfully, there should be a `kafka-storage-x.x.x-test.jar` file under `storage/build/libs`. -Next, setting configurations in the broker side to enable tiered storage feature.</p> +<h4 class="anchor-heading"><a id="tiered_storage_config_ex" class="anchor-link"></a><a href="#tiered_storage_config_ex">Configurations Example</a></h4> +<p>Here is a sample configuration to enable tiered storage feature in broker side: <pre> # Sample Zookeeper/Kraft broker server.properties listening on PLAINTEXT://:9092 remote.log.storage.system.enable=true - -# Setting the listener for the clients in RemoteLogMetadataManager to talk to the brokers. +# Please provide the implementation for remoteStorageManager. This is the mandatory configuration for tiered storage. +# remote.log.storage.manager.class.name=org.apache.kafka.server.log.remote.storage.NoOpRemoteStorageManager +# Using the "PLAINTEXT" listener for the clients in RemoteLogMetadataManager to talk to the brokers. remote.log.metadata.manager.listener.name=PLAINTEXT - -# Please provide the implementation info for remoteStorageManager. -# This is the mandatory configuration for tiered storage. -# Here, we use the `LocalTieredStorage` built above. -remote.log.storage.manager.class.name=org.apache.kafka.server.log.remote.storage.LocalTieredStorage -remote.log.storage.manager.class.path=/PATH/TO/kafka-storage-x.x.x-test.jar - -# These 2 prefix are default values, but customizable -remote.log.storage.manager.impl.prefix=rsm.config. -remote.log.metadata.manager.impl.prefix=rlmm.config. - -# Configure the directory used for `LocalTieredStorage` -# Note, please make sure the brokers need to have access to this directory -rsm.config.dir=/tmp/kafka-remote-storage - -# This needs to be changed if number of brokers in the cluster is more than 1 -rlmm.config.remote.log.metadata.topic.replication.factor=1 - -# Try to speed up the log retention check interval for testing -log.retention.check.interval.ms=1000 </pre> </p> -<p>Following <a href="#quickstart_startserver">quick start guide</a> to start up the kafka environment. - Then, create a topic with tiered storage enabled with configs: - -<pre> -# remote.storage.enable=true -> enables tiered storage on the topic -# local.retention.ms=1000 -> The number of milliseconds to keep the local log segment before it gets deleted. - Note that a local log segment is eligible for deletion only after it gets uploaded to remote. -# retention.ms=3600000 -> when segments exceed this time, the segments in remote storage will be deleted -# segment.bytes=1048576 -> for test only, to speed up the log segment rolling interval -# file.delete.delay.ms=10000 -> for test only, to speed up the local-log segment file delete delay - -bin/kafka-topics.sh --create --topic tieredTopic --bootstrap-server localhost:9092 \ ---config remote.storage.enable=true --config local.retention.ms=1000 --config retention.ms=3600000 \ ---config segment.bytes=1048576 --config file.delete.delay.ms=1000 +<p>After broker is started, creating a topic with tiered storage enabled, and a small log time retention value to try this feature: +<pre>bin/kafka-topics.sh --create --topic tieredTopic --bootstrap-server localhost:9092 --config remote.storage.enable=true --config local.retention.ms=1000 </pre> </p> -<p>Try to send messages to the `tieredTopic` topic to roll the log segment:</p> Review Comment: Need to find the commit that introduced this -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: dev-unsubscr...@kafka.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
