chia7712 commented on code in PR #19371:
URL: https://github.com/apache/kafka/pull/19371#discussion_r2101594593
##########
core/src/main/scala/kafka/raft/KafkaMetadataLog.scala:
##########
@@ -589,8 +589,9 @@ object KafkaMetadataLog extends Logging {
): KafkaMetadataLog = {
val props = new Properties()
props.setProperty(TopicConfig.MAX_MESSAGE_BYTES_CONFIG,
config.maxBatchSizeInBytes.toString)
- props.setProperty(TopicConfig.SEGMENT_BYTES_CONFIG,
config.logSegmentBytes.toString)
- props.setProperty(TopicConfig.SEGMENT_MS_CONFIG,
config.logSegmentMillis.toString)
+ // Since MetadataLogConfig validates the log segment size using a minimum
allowed value,
+ // we can safely use `internal.segment.bytes` to configure it instead of
relying on `segment.bytes`.
+ props.setProperty(LogConfig.INTERNAL_SEGMENT_BYTES_CONFIG,
config.logSegmentBytes.toString)
Review Comment:
@junrao You're right. To support smaller segments for Kraft topics, we could
reintroduce the broker-level `internal.log.segment.bytes` configuration.
However, this duplicates the functionality of the topic-level
`internal.segment.bytes` and would affect all topics on the server.
Perhaps we should revisit `internal.metadata.log.segment.bytes` instead. The
regular topics could utilize topic-level `internal.segment.bytes` for smaller
segments, while Kraft topics would use the broker-level
`internal.metadata.log.segment.bytes`. This approach would require adding
if-else logic when converting Kraft configurations to log configurations.
```scala
props.setProperty(if config.useInternal
LogConfig.INTERNAL_SEGMENT_BYTES_CONFIG else LogConfig.SEGMENT_BYTES_CONFIG,
config.logSegmentBytes.toString)
```
For another, we should move the test-only constructor out of the production
code. This will prevent it from being used accidentally in a production
environment. This could be considered as a follow-up ticket
```java
public MetadataLogConfig(int internalLogSegmentBytes,
long logSegmentMillis,
long retentionMaxBytes,
long retentionMillis,
int maxBatchSizeInBytes,
int maxFetchSizeInBytes,
long deleteDelayMillis) {
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
