Re: [PR] KAFKA-18928: Support dynamically changing configs for a specific dynamic quorum controller [kafka]
chia7712 merged PR #20938: URL: https://github.com/apache/kafka/pull/20938 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
Re: [PR] KAFKA-18928: Support dynamically changing configs for a specific dynamic quorum controller [kafka]
jiafu1115 commented on code in PR #20938:
URL: https://github.com/apache/kafka/pull/20938#discussion_r2660971168
##
server/src/test/java/org/apache/kafka/server/BootstrapControllersIntegrationTest.java:
##
@@ -225,31 +228,93 @@ public void testIncrementalAlterConfigs(ClusterInstance
clusterInstance) throws
testIncrementalAlterConfigs(clusterInstance, false);
}
+@SuppressWarnings("unchecked")
private void testIncrementalAlterConfigs(ClusterInstance clusterInstance,
boolean usingBootstrapControllers) throws Exception {
+Collection nodeIds = usingBootstrapControllers ?
+clusterInstance.controllerIds() :
clusterInstance.brokers().keySet();
try (Admin admin = Admin.create(adminConfig(clusterInstance,
usingBootstrapControllers))) {
-int nodeId = usingBootstrapControllers ?
-
clusterInstance.controllers().values().iterator().next().config().nodeId() :
-
clusterInstance.brokers().values().iterator().next().config().nodeId();
-ConfigResource nodeResource = new ConfigResource(BROKER, "" +
nodeId);
-ConfigResource defaultResource = new ConfigResource(BROKER, "");
-Map> alterations =
Map.of(
-nodeResource, List.of(new AlterConfigOp(new
ConfigEntry("my.custom.config", "foo"), AlterConfigOp.OpType.SET)),
-defaultResource, List.of(new AlterConfigOp(new
ConfigEntry("my.custom.config", "bar"), AlterConfigOp.OpType.SET))
-);
-admin.incrementalAlterConfigs(alterations).all().get(1,
TimeUnit.MINUTES);
-TestUtils.retryOnExceptionWithTimeout(30_000, () -> {
-Config config = admin.describeConfigs(List.of(nodeResource)).
-all().get(1, TimeUnit.MINUTES).get(nodeResource);
-ConfigEntry entry = config.entries().stream().
-filter(e -> e.name().equals("my.custom.config")).
-findFirst().orElseThrow();
-assertEquals(DYNAMIC_BROKER_CONFIG, entry.source(),
-"Expected entry for my.custom.config to come from
DYNAMIC_BROKER_CONFIG. " +
-"Instead, the entry was: " + entry);
-});
+for (int nodeId : nodeIds) {
+ConfigResource nodeResource = new ConfigResource(BROKER, "" +
nodeId);
+ConfigResource defaultResource = new ConfigResource(BROKER,
"");
+String nodeMaxConnectionsValue = String.valueOf(1000 + nodeId);
+String defaultMaxConnectionsValue = String.valueOf(2000 +
nodeId);
+String defaultConnectionRateValue = String.valueOf(2000 +
nodeId);
+
+// Set configs: MAX_CONNECTIONS_CONFIG for per-broker, both
configs for default
+Map> alterations =
Map.of(
+nodeResource, List.of(
+new AlterConfigOp(new
ConfigEntry(SocketServerConfigs.MAX_CONNECTIONS_CONFIG,
nodeMaxConnectionsValue), AlterConfigOp.OpType.SET)
+),
+defaultResource, List.of(
+new AlterConfigOp(new
ConfigEntry(SocketServerConfigs.MAX_CONNECTIONS_CONFIG,
defaultMaxConnectionsValue), AlterConfigOp.OpType.SET),
+new AlterConfigOp(new
ConfigEntry(SocketServerConfigs.MAX_CONNECTION_CREATION_RATE_CONFIG,
defaultConnectionRateValue), AlterConfigOp.OpType.SET)
+)
+);
+admin.incrementalAlterConfigs(alterations).all().get(1,
TimeUnit.MINUTES);
+
+// Verify per-broker configs: MAX_CONNECTIONS_CONFIG and
MAX_CONNECTION_CREATION_RATE_CONFIG
+verifyConfigValue(admin, nodeResource,
SocketServerConfigs.MAX_CONNECTIONS_CONFIG,
+DYNAMIC_BROKER_CONFIG, nodeMaxConnectionsValue);
+verifyConfigValue(admin, nodeResource,
SocketServerConfigs.MAX_CONNECTION_CREATION_RATE_CONFIG,
+DYNAMIC_DEFAULT_BROKER_CONFIG,
defaultConnectionRateValue);
+
+// Verify default broker configs: MAX_CONNECTIONS_CONFIG and
MAX_CONNECTION_CREATION_RATE_CONFIG
+verifyConfigValue(admin, defaultResource,
SocketServerConfigs.MAX_CONNECTIONS_CONFIG,
+DYNAMIC_DEFAULT_BROKER_CONFIG,
defaultMaxConnectionsValue);
+verifyConfigValue(admin, defaultResource,
SocketServerConfigs.MAX_CONNECTION_CREATION_RATE_CONFIG,
+DYNAMIC_DEFAULT_BROKER_CONFIG,
defaultConnectionRateValue);
+
+Object server = (usingBootstrapControllers ?
clusterInstance.controllers() : clusterInstance.brokers()).get(nodeId);
+// Verify that SocketServer has actually updated the max
connection limit in ConnectionQuotas
+
Re: [PR] KAFKA-18928: Support dynamically changing configs for a specific dynamic quorum controller [kafka]
jiafu1115 commented on code in PR #20938:
URL: https://github.com/apache/kafka/pull/20938#discussion_r2660970570
##
server/src/test/java/org/apache/kafka/server/BootstrapControllersIntegrationTest.java:
##
@@ -225,31 +228,93 @@ public void testIncrementalAlterConfigs(ClusterInstance
clusterInstance) throws
testIncrementalAlterConfigs(clusterInstance, false);
}
+@SuppressWarnings("unchecked")
Review Comment:
@chia7712 done. thanks
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
Re: [PR] KAFKA-18928: Support dynamically changing configs for a specific dynamic quorum controller [kafka]
chia7712 commented on code in PR #20938:
URL: https://github.com/apache/kafka/pull/20938#discussion_r2660522820
##
server/src/test/java/org/apache/kafka/server/BootstrapControllersIntegrationTest.java:
##
@@ -225,31 +228,93 @@ public void testIncrementalAlterConfigs(ClusterInstance
clusterInstance) throws
testIncrementalAlterConfigs(clusterInstance, false);
}
+@SuppressWarnings("unchecked")
private void testIncrementalAlterConfigs(ClusterInstance clusterInstance,
boolean usingBootstrapControllers) throws Exception {
+Collection nodeIds = usingBootstrapControllers ?
+clusterInstance.controllerIds() :
clusterInstance.brokers().keySet();
try (Admin admin = Admin.create(adminConfig(clusterInstance,
usingBootstrapControllers))) {
-int nodeId = usingBootstrapControllers ?
-
clusterInstance.controllers().values().iterator().next().config().nodeId() :
-
clusterInstance.brokers().values().iterator().next().config().nodeId();
-ConfigResource nodeResource = new ConfigResource(BROKER, "" +
nodeId);
-ConfigResource defaultResource = new ConfigResource(BROKER, "");
-Map> alterations =
Map.of(
-nodeResource, List.of(new AlterConfigOp(new
ConfigEntry("my.custom.config", "foo"), AlterConfigOp.OpType.SET)),
-defaultResource, List.of(new AlterConfigOp(new
ConfigEntry("my.custom.config", "bar"), AlterConfigOp.OpType.SET))
-);
-admin.incrementalAlterConfigs(alterations).all().get(1,
TimeUnit.MINUTES);
-TestUtils.retryOnExceptionWithTimeout(30_000, () -> {
-Config config = admin.describeConfigs(List.of(nodeResource)).
-all().get(1, TimeUnit.MINUTES).get(nodeResource);
-ConfigEntry entry = config.entries().stream().
-filter(e -> e.name().equals("my.custom.config")).
-findFirst().orElseThrow();
-assertEquals(DYNAMIC_BROKER_CONFIG, entry.source(),
-"Expected entry for my.custom.config to come from
DYNAMIC_BROKER_CONFIG. " +
-"Instead, the entry was: " + entry);
-});
+for (int nodeId : nodeIds) {
+ConfigResource nodeResource = new ConfigResource(BROKER, "" +
nodeId);
+ConfigResource defaultResource = new ConfigResource(BROKER,
"");
+String nodeMaxConnectionsValue = String.valueOf(1000 + nodeId);
+String defaultMaxConnectionsValue = String.valueOf(2000 +
nodeId);
+String defaultConnectionRateValue = String.valueOf(2000 +
nodeId);
+
+// Set configs: MAX_CONNECTIONS_CONFIG for per-broker, both
configs for default
+Map> alterations =
Map.of(
+nodeResource, List.of(
+new AlterConfigOp(new
ConfigEntry(SocketServerConfigs.MAX_CONNECTIONS_CONFIG,
nodeMaxConnectionsValue), AlterConfigOp.OpType.SET)
+),
+defaultResource, List.of(
+new AlterConfigOp(new
ConfigEntry(SocketServerConfigs.MAX_CONNECTIONS_CONFIG,
defaultMaxConnectionsValue), AlterConfigOp.OpType.SET),
+new AlterConfigOp(new
ConfigEntry(SocketServerConfigs.MAX_CONNECTION_CREATION_RATE_CONFIG,
defaultConnectionRateValue), AlterConfigOp.OpType.SET)
+)
+);
+admin.incrementalAlterConfigs(alterations).all().get(1,
TimeUnit.MINUTES);
+
+// Verify per-broker configs: MAX_CONNECTIONS_CONFIG and
MAX_CONNECTION_CREATION_RATE_CONFIG
+verifyConfigValue(admin, nodeResource,
SocketServerConfigs.MAX_CONNECTIONS_CONFIG,
+DYNAMIC_BROKER_CONFIG, nodeMaxConnectionsValue);
+verifyConfigValue(admin, nodeResource,
SocketServerConfigs.MAX_CONNECTION_CREATION_RATE_CONFIG,
+DYNAMIC_DEFAULT_BROKER_CONFIG,
defaultConnectionRateValue);
+
+// Verify default broker configs: MAX_CONNECTIONS_CONFIG and
MAX_CONNECTION_CREATION_RATE_CONFIG
+verifyConfigValue(admin, defaultResource,
SocketServerConfigs.MAX_CONNECTIONS_CONFIG,
+DYNAMIC_DEFAULT_BROKER_CONFIG,
defaultMaxConnectionsValue);
+verifyConfigValue(admin, defaultResource,
SocketServerConfigs.MAX_CONNECTION_CREATION_RATE_CONFIG,
+DYNAMIC_DEFAULT_BROKER_CONFIG,
defaultConnectionRateValue);
+
+Object server = (usingBootstrapControllers ?
clusterInstance.controllers() : clusterInstance.brokers()).get(nodeId);
+// Verify that SocketServer has actually updated the max
connection limit in ConnectionQuotas
+
Re: [PR] KAFKA-18928: Support dynamically changing configs for a specific dynamic quorum controller [kafka]
chia7712 commented on code in PR #20938:
URL: https://github.com/apache/kafka/pull/20938#discussion_r2660520369
##
server/src/test/java/org/apache/kafka/server/BootstrapControllersIntegrationTest.java:
##
@@ -226,30 +227,61 @@ public void testIncrementalAlterConfigs(ClusterInstance
clusterInstance) throws
}
private void testIncrementalAlterConfigs(ClusterInstance clusterInstance,
boolean usingBootstrapControllers) throws Exception {
+Collection nodeIds = usingBootstrapControllers ?
+clusterInstance.controllerIds() :
+clusterInstance.brokers().keySet();
try (Admin admin = Admin.create(adminConfig(clusterInstance,
usingBootstrapControllers))) {
-int nodeId = usingBootstrapControllers ?
-
clusterInstance.controllers().values().iterator().next().config().nodeId() :
-
clusterInstance.brokers().values().iterator().next().config().nodeId();
-ConfigResource nodeResource = new ConfigResource(BROKER, "" +
nodeId);
-ConfigResource defaultResource = new ConfigResource(BROKER, "");
-Map> alterations =
Map.of(
-nodeResource, List.of(new AlterConfigOp(new
ConfigEntry("my.custom.config", "foo"), AlterConfigOp.OpType.SET)),
-defaultResource, List.of(new AlterConfigOp(new
ConfigEntry("my.custom.config", "bar"), AlterConfigOp.OpType.SET))
-);
-admin.incrementalAlterConfigs(alterations).all().get(1,
TimeUnit.MINUTES);
-TestUtils.retryOnExceptionWithTimeout(30_000, () -> {
-Config config = admin.describeConfigs(List.of(nodeResource)).
-all().get(1, TimeUnit.MINUTES).get(nodeResource);
-ConfigEntry entry = config.entries().stream().
-filter(e -> e.name().equals("my.custom.config")).
-findFirst().orElseThrow();
-assertEquals(DYNAMIC_BROKER_CONFIG, entry.source(),
-"Expected entry for my.custom.config to come from
DYNAMIC_BROKER_CONFIG. " +
-"Instead, the entry was: " + entry);
-});
+for (int nodeId : nodeIds) {
+ConfigResource nodeResource = new ConfigResource(BROKER, "" +
nodeId);
+ConfigResource defaultResource = new ConfigResource(BROKER,
"");
+String nodeMaxConnectionsValue = String.valueOf(1000 + nodeId);
+String defaultMaxConnectionsValue = String.valueOf(2000 +
nodeId);
+String defaultConnectionRateValue = String.valueOf(2000 +
nodeId);
+
+// Set configs: MAX_CONNECTIONS_CONFIG for per-broker, both
configs for default
+Map> alterations =
Map.of(
+nodeResource, List.of(
+new AlterConfigOp(new
ConfigEntry(SocketServerConfigs.MAX_CONNECTIONS_CONFIG,
nodeMaxConnectionsValue), AlterConfigOp.OpType.SET)
+),
+defaultResource, List.of(
+new AlterConfigOp(new
ConfigEntry(SocketServerConfigs.MAX_CONNECTIONS_CONFIG,
defaultMaxConnectionsValue), AlterConfigOp.OpType.SET),
+new AlterConfigOp(new
ConfigEntry(SocketServerConfigs.MAX_CONNECTION_CREATION_RATE_CONFIG,
defaultConnectionRateValue), AlterConfigOp.OpType.SET)
+)
+);
+admin.incrementalAlterConfigs(alterations).all().get(1,
TimeUnit.MINUTES);
+
+// Verify per-broker configs: MAX_CONNECTIONS_CONFIG and
MAX_CONNECTION_CREATION_RATE_CONFIG
+verifyConfigValue(admin, nodeResource,
SocketServerConfigs.MAX_CONNECTIONS_CONFIG,
+DYNAMIC_BROKER_CONFIG, nodeMaxConnectionsValue);
+verifyConfigValue(admin, nodeResource,
SocketServerConfigs.MAX_CONNECTION_CREATION_RATE_CONFIG,
+DYNAMIC_DEFAULT_BROKER_CONFIG,
defaultConnectionRateValue);
+
+// Verify default broker configs: MAX_CONNECTIONS_CONFIG and
MAX_CONNECTION_CREATION_RATE_CONFIG
+verifyConfigValue(admin, defaultResource,
SocketServerConfigs.MAX_CONNECTIONS_CONFIG,
+DYNAMIC_DEFAULT_BROKER_CONFIG,
defaultMaxConnectionsValue);
+verifyConfigValue(admin, defaultResource,
SocketServerConfigs.MAX_CONNECTION_CREATION_RATE_CONFIG,
+DYNAMIC_DEFAULT_BROKER_CONFIG,
defaultConnectionRateValue);
Review Comment:
Thanks for the updates. I wonder if the controller actually applies the
latest dynamic configs, rather than just generating the config records
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To
Re: [PR] KAFKA-18928: Support dynamically changing configs for a specific dynamic quorum controller [kafka]
jiafu1115 commented on code in PR #20938:
URL: https://github.com/apache/kafka/pull/20938#discussion_r2659998326
##
server/src/test/java/org/apache/kafka/server/BootstrapControllersIntegrationTest.java:
##
@@ -226,30 +227,61 @@ public void testIncrementalAlterConfigs(ClusterInstance
clusterInstance) throws
}
private void testIncrementalAlterConfigs(ClusterInstance clusterInstance,
boolean usingBootstrapControllers) throws Exception {
+Collection nodeIds = usingBootstrapControllers ?
+clusterInstance.controllerIds() :
+clusterInstance.brokers().keySet();
try (Admin admin = Admin.create(adminConfig(clusterInstance,
usingBootstrapControllers))) {
-int nodeId = usingBootstrapControllers ?
-
clusterInstance.controllers().values().iterator().next().config().nodeId() :
-
clusterInstance.brokers().values().iterator().next().config().nodeId();
-ConfigResource nodeResource = new ConfigResource(BROKER, "" +
nodeId);
-ConfigResource defaultResource = new ConfigResource(BROKER, "");
-Map> alterations =
Map.of(
-nodeResource, List.of(new AlterConfigOp(new
ConfigEntry("my.custom.config", "foo"), AlterConfigOp.OpType.SET)),
-defaultResource, List.of(new AlterConfigOp(new
ConfigEntry("my.custom.config", "bar"), AlterConfigOp.OpType.SET))
-);
-admin.incrementalAlterConfigs(alterations).all().get(1,
TimeUnit.MINUTES);
-TestUtils.retryOnExceptionWithTimeout(30_000, () -> {
-Config config = admin.describeConfigs(List.of(nodeResource)).
-all().get(1, TimeUnit.MINUTES).get(nodeResource);
-ConfigEntry entry = config.entries().stream().
-filter(e -> e.name().equals("my.custom.config")).
-findFirst().orElseThrow();
-assertEquals(DYNAMIC_BROKER_CONFIG, entry.source(),
-"Expected entry for my.custom.config to come from
DYNAMIC_BROKER_CONFIG. " +
-"Instead, the entry was: " + entry);
-});
+for (int nodeId : nodeIds) {
+ConfigResource nodeResource = new ConfigResource(BROKER, "" +
nodeId);
+ConfigResource defaultResource = new ConfigResource(BROKER,
"");
+String nodeMaxConnectionsValue = String.valueOf(1000 + nodeId);
+String defaultMaxConnectionsValue = String.valueOf(2000 +
nodeId);
+String defaultConnectionRateValue = String.valueOf(2000 +
nodeId);
+
+// Set configs: MAX_CONNECTIONS_CONFIG for per-broker, both
configs for default
+Map> alterations =
Map.of(
+nodeResource, List.of(
+new AlterConfigOp(new
ConfigEntry(SocketServerConfigs.MAX_CONNECTIONS_CONFIG,
nodeMaxConnectionsValue), AlterConfigOp.OpType.SET)
+),
+defaultResource, List.of(
+new AlterConfigOp(new
ConfigEntry(SocketServerConfigs.MAX_CONNECTIONS_CONFIG,
defaultMaxConnectionsValue), AlterConfigOp.OpType.SET),
+new AlterConfigOp(new
ConfigEntry(SocketServerConfigs.MAX_CONNECTION_CREATION_RATE_CONFIG,
defaultConnectionRateValue), AlterConfigOp.OpType.SET)
+)
+);
+admin.incrementalAlterConfigs(alterations).all().get(1,
TimeUnit.MINUTES);
+
+// Verify per-broker configs: MAX_CONNECTIONS_CONFIG and
MAX_CONNECTION_CREATION_RATE_CONFIG
+verifyConfigValue(admin, nodeResource,
SocketServerConfigs.MAX_CONNECTIONS_CONFIG,
+DYNAMIC_BROKER_CONFIG, nodeMaxConnectionsValue);
+verifyConfigValue(admin, nodeResource,
SocketServerConfigs.MAX_CONNECTION_CREATION_RATE_CONFIG,
+DYNAMIC_DEFAULT_BROKER_CONFIG,
defaultConnectionRateValue);
+
+// Verify default broker configs: MAX_CONNECTIONS_CONFIG and
MAX_CONNECTION_CREATION_RATE_CONFIG
+verifyConfigValue(admin, defaultResource,
SocketServerConfigs.MAX_CONNECTIONS_CONFIG,
+DYNAMIC_DEFAULT_BROKER_CONFIG,
defaultMaxConnectionsValue);
+verifyConfigValue(admin, defaultResource,
SocketServerConfigs.MAX_CONNECTION_CREATION_RATE_CONFIG,
+DYNAMIC_DEFAULT_BROKER_CONFIG,
defaultConnectionRateValue);
Review Comment:
@chia7712 Thanks for review. Done. PTAL.
I added two extra checkpoints with
https://github.com/apache/kafka/pull/20938/changes#diff-38ac8a8b4162b71f7b0a8244bd9d5f2ddb354b691cbd0b7699f77a5920f331a6R278
in the test.
BTW: https://github.com/apache/kafka/pull/20938#issuecomment-3634864330 is
the real end to
Re: [PR] KAFKA-18928: Support dynamically changing configs for a specific dynamic quorum controller [kafka]
jiafu1115 commented on code in PR #20938:
URL: https://github.com/apache/kafka/pull/20938#discussion_r2659998326
##
server/src/test/java/org/apache/kafka/server/BootstrapControllersIntegrationTest.java:
##
@@ -226,30 +227,61 @@ public void testIncrementalAlterConfigs(ClusterInstance
clusterInstance) throws
}
private void testIncrementalAlterConfigs(ClusterInstance clusterInstance,
boolean usingBootstrapControllers) throws Exception {
+Collection nodeIds = usingBootstrapControllers ?
+clusterInstance.controllerIds() :
+clusterInstance.brokers().keySet();
try (Admin admin = Admin.create(adminConfig(clusterInstance,
usingBootstrapControllers))) {
-int nodeId = usingBootstrapControllers ?
-
clusterInstance.controllers().values().iterator().next().config().nodeId() :
-
clusterInstance.brokers().values().iterator().next().config().nodeId();
-ConfigResource nodeResource = new ConfigResource(BROKER, "" +
nodeId);
-ConfigResource defaultResource = new ConfigResource(BROKER, "");
-Map> alterations =
Map.of(
-nodeResource, List.of(new AlterConfigOp(new
ConfigEntry("my.custom.config", "foo"), AlterConfigOp.OpType.SET)),
-defaultResource, List.of(new AlterConfigOp(new
ConfigEntry("my.custom.config", "bar"), AlterConfigOp.OpType.SET))
-);
-admin.incrementalAlterConfigs(alterations).all().get(1,
TimeUnit.MINUTES);
-TestUtils.retryOnExceptionWithTimeout(30_000, () -> {
-Config config = admin.describeConfigs(List.of(nodeResource)).
-all().get(1, TimeUnit.MINUTES).get(nodeResource);
-ConfigEntry entry = config.entries().stream().
-filter(e -> e.name().equals("my.custom.config")).
-findFirst().orElseThrow();
-assertEquals(DYNAMIC_BROKER_CONFIG, entry.source(),
-"Expected entry for my.custom.config to come from
DYNAMIC_BROKER_CONFIG. " +
-"Instead, the entry was: " + entry);
-});
+for (int nodeId : nodeIds) {
+ConfigResource nodeResource = new ConfigResource(BROKER, "" +
nodeId);
+ConfigResource defaultResource = new ConfigResource(BROKER,
"");
+String nodeMaxConnectionsValue = String.valueOf(1000 + nodeId);
+String defaultMaxConnectionsValue = String.valueOf(2000 +
nodeId);
+String defaultConnectionRateValue = String.valueOf(2000 +
nodeId);
+
+// Set configs: MAX_CONNECTIONS_CONFIG for per-broker, both
configs for default
+Map> alterations =
Map.of(
+nodeResource, List.of(
+new AlterConfigOp(new
ConfigEntry(SocketServerConfigs.MAX_CONNECTIONS_CONFIG,
nodeMaxConnectionsValue), AlterConfigOp.OpType.SET)
+),
+defaultResource, List.of(
+new AlterConfigOp(new
ConfigEntry(SocketServerConfigs.MAX_CONNECTIONS_CONFIG,
defaultMaxConnectionsValue), AlterConfigOp.OpType.SET),
+new AlterConfigOp(new
ConfigEntry(SocketServerConfigs.MAX_CONNECTION_CREATION_RATE_CONFIG,
defaultConnectionRateValue), AlterConfigOp.OpType.SET)
+)
+);
+admin.incrementalAlterConfigs(alterations).all().get(1,
TimeUnit.MINUTES);
+
+// Verify per-broker configs: MAX_CONNECTIONS_CONFIG and
MAX_CONNECTION_CREATION_RATE_CONFIG
+verifyConfigValue(admin, nodeResource,
SocketServerConfigs.MAX_CONNECTIONS_CONFIG,
+DYNAMIC_BROKER_CONFIG, nodeMaxConnectionsValue);
+verifyConfigValue(admin, nodeResource,
SocketServerConfigs.MAX_CONNECTION_CREATION_RATE_CONFIG,
+DYNAMIC_DEFAULT_BROKER_CONFIG,
defaultConnectionRateValue);
+
+// Verify default broker configs: MAX_CONNECTIONS_CONFIG and
MAX_CONNECTION_CREATION_RATE_CONFIG
+verifyConfigValue(admin, defaultResource,
SocketServerConfigs.MAX_CONNECTIONS_CONFIG,
+DYNAMIC_DEFAULT_BROKER_CONFIG,
defaultMaxConnectionsValue);
+verifyConfigValue(admin, defaultResource,
SocketServerConfigs.MAX_CONNECTION_CREATION_RATE_CONFIG,
+DYNAMIC_DEFAULT_BROKER_CONFIG,
defaultConnectionRateValue);
Review Comment:
@chia7712 Thanks for review. Done. PTAL.
I added two extra checkpoints with
https://github.com/apache/kafka/pull/20938/changes#diff-38ac8a8b4162b71f7b0a8244bd9d5f2ddb354b691cbd0b7699f77a5920f331a6R274
in the test.
BTW: https://github.com/apache/kafka/pull/20938#issuecomment-3634864330 is
the real end to
Re: [PR] KAFKA-18928: Support dynamically changing configs for a specific dynamic quorum controller [kafka]
chia7712 commented on code in PR #20938:
URL: https://github.com/apache/kafka/pull/20938#discussion_r2659763247
##
server/src/test/java/org/apache/kafka/server/BootstrapControllersIntegrationTest.java:
##
@@ -226,30 +227,61 @@ public void testIncrementalAlterConfigs(ClusterInstance
clusterInstance) throws
}
private void testIncrementalAlterConfigs(ClusterInstance clusterInstance,
boolean usingBootstrapControllers) throws Exception {
+Collection nodeIds = usingBootstrapControllers ?
+clusterInstance.controllerIds() :
+clusterInstance.brokers().keySet();
try (Admin admin = Admin.create(adminConfig(clusterInstance,
usingBootstrapControllers))) {
-int nodeId = usingBootstrapControllers ?
-
clusterInstance.controllers().values().iterator().next().config().nodeId() :
-
clusterInstance.brokers().values().iterator().next().config().nodeId();
-ConfigResource nodeResource = new ConfigResource(BROKER, "" +
nodeId);
-ConfigResource defaultResource = new ConfigResource(BROKER, "");
-Map> alterations =
Map.of(
-nodeResource, List.of(new AlterConfigOp(new
ConfigEntry("my.custom.config", "foo"), AlterConfigOp.OpType.SET)),
-defaultResource, List.of(new AlterConfigOp(new
ConfigEntry("my.custom.config", "bar"), AlterConfigOp.OpType.SET))
-);
-admin.incrementalAlterConfigs(alterations).all().get(1,
TimeUnit.MINUTES);
-TestUtils.retryOnExceptionWithTimeout(30_000, () -> {
-Config config = admin.describeConfigs(List.of(nodeResource)).
-all().get(1, TimeUnit.MINUTES).get(nodeResource);
-ConfigEntry entry = config.entries().stream().
-filter(e -> e.name().equals("my.custom.config")).
-findFirst().orElseThrow();
-assertEquals(DYNAMIC_BROKER_CONFIG, entry.source(),
-"Expected entry for my.custom.config to come from
DYNAMIC_BROKER_CONFIG. " +
-"Instead, the entry was: " + entry);
-});
+for (int nodeId : nodeIds) {
+ConfigResource nodeResource = new ConfigResource(BROKER, "" +
nodeId);
+ConfigResource defaultResource = new ConfigResource(BROKER,
"");
+String nodeMaxConnectionsValue = String.valueOf(1000 + nodeId);
+String defaultMaxConnectionsValue = String.valueOf(2000 +
nodeId);
+String defaultConnectionRateValue = String.valueOf(2000 +
nodeId);
+
+// Set configs: MAX_CONNECTIONS_CONFIG for per-broker, both
configs for default
+Map> alterations =
Map.of(
+nodeResource, List.of(
+new AlterConfigOp(new
ConfigEntry(SocketServerConfigs.MAX_CONNECTIONS_CONFIG,
nodeMaxConnectionsValue), AlterConfigOp.OpType.SET)
+),
+defaultResource, List.of(
+new AlterConfigOp(new
ConfigEntry(SocketServerConfigs.MAX_CONNECTIONS_CONFIG,
defaultMaxConnectionsValue), AlterConfigOp.OpType.SET),
+new AlterConfigOp(new
ConfigEntry(SocketServerConfigs.MAX_CONNECTION_CREATION_RATE_CONFIG,
defaultConnectionRateValue), AlterConfigOp.OpType.SET)
+)
+);
+admin.incrementalAlterConfigs(alterations).all().get(1,
TimeUnit.MINUTES);
+
+// Verify per-broker configs: MAX_CONNECTIONS_CONFIG and
MAX_CONNECTION_CREATION_RATE_CONFIG
+verifyConfigValue(admin, nodeResource,
SocketServerConfigs.MAX_CONNECTIONS_CONFIG,
+DYNAMIC_BROKER_CONFIG, nodeMaxConnectionsValue);
+verifyConfigValue(admin, nodeResource,
SocketServerConfigs.MAX_CONNECTION_CREATION_RATE_CONFIG,
+DYNAMIC_DEFAULT_BROKER_CONFIG,
defaultConnectionRateValue);
+
+// Verify default broker configs: MAX_CONNECTIONS_CONFIG and
MAX_CONNECTION_CREATION_RATE_CONFIG
+verifyConfigValue(admin, defaultResource,
SocketServerConfigs.MAX_CONNECTIONS_CONFIG,
+DYNAMIC_DEFAULT_BROKER_CONFIG,
defaultMaxConnectionsValue);
+verifyConfigValue(admin, defaultResource,
SocketServerConfigs.MAX_CONNECTION_CREATION_RATE_CONFIG,
+DYNAMIC_DEFAULT_BROKER_CONFIG,
defaultConnectionRateValue);
Review Comment:
It also addresses @ahuang98's comment
(https://github.com/apache/kafka/pull/20938#pullrequestreview-3546706656)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail:
Re: [PR] KAFKA-18928: Support dynamically changing configs for a specific dynamic quorum controller [kafka]
chia7712 commented on code in PR #20938:
URL: https://github.com/apache/kafka/pull/20938#discussion_r2659762066
##
server/src/test/java/org/apache/kafka/server/BootstrapControllersIntegrationTest.java:
##
@@ -226,30 +227,61 @@ public void testIncrementalAlterConfigs(ClusterInstance
clusterInstance) throws
}
private void testIncrementalAlterConfigs(ClusterInstance clusterInstance,
boolean usingBootstrapControllers) throws Exception {
+Collection nodeIds = usingBootstrapControllers ?
+clusterInstance.controllerIds() :
+clusterInstance.brokers().keySet();
try (Admin admin = Admin.create(adminConfig(clusterInstance,
usingBootstrapControllers))) {
-int nodeId = usingBootstrapControllers ?
-
clusterInstance.controllers().values().iterator().next().config().nodeId() :
-
clusterInstance.brokers().values().iterator().next().config().nodeId();
-ConfigResource nodeResource = new ConfigResource(BROKER, "" +
nodeId);
-ConfigResource defaultResource = new ConfigResource(BROKER, "");
-Map> alterations =
Map.of(
-nodeResource, List.of(new AlterConfigOp(new
ConfigEntry("my.custom.config", "foo"), AlterConfigOp.OpType.SET)),
-defaultResource, List.of(new AlterConfigOp(new
ConfigEntry("my.custom.config", "bar"), AlterConfigOp.OpType.SET))
-);
-admin.incrementalAlterConfigs(alterations).all().get(1,
TimeUnit.MINUTES);
-TestUtils.retryOnExceptionWithTimeout(30_000, () -> {
-Config config = admin.describeConfigs(List.of(nodeResource)).
-all().get(1, TimeUnit.MINUTES).get(nodeResource);
-ConfigEntry entry = config.entries().stream().
-filter(e -> e.name().equals("my.custom.config")).
-findFirst().orElseThrow();
-assertEquals(DYNAMIC_BROKER_CONFIG, entry.source(),
-"Expected entry for my.custom.config to come from
DYNAMIC_BROKER_CONFIG. " +
-"Instead, the entry was: " + entry);
-});
+for (int nodeId : nodeIds) {
+ConfigResource nodeResource = new ConfigResource(BROKER, "" +
nodeId);
+ConfigResource defaultResource = new ConfigResource(BROKER,
"");
+String nodeMaxConnectionsValue = String.valueOf(1000 + nodeId);
+String defaultMaxConnectionsValue = String.valueOf(2000 +
nodeId);
+String defaultConnectionRateValue = String.valueOf(2000 +
nodeId);
+
+// Set configs: MAX_CONNECTIONS_CONFIG for per-broker, both
configs for default
+Map> alterations =
Map.of(
+nodeResource, List.of(
+new AlterConfigOp(new
ConfigEntry(SocketServerConfigs.MAX_CONNECTIONS_CONFIG,
nodeMaxConnectionsValue), AlterConfigOp.OpType.SET)
+),
+defaultResource, List.of(
+new AlterConfigOp(new
ConfigEntry(SocketServerConfigs.MAX_CONNECTIONS_CONFIG,
defaultMaxConnectionsValue), AlterConfigOp.OpType.SET),
+new AlterConfigOp(new
ConfigEntry(SocketServerConfigs.MAX_CONNECTION_CREATION_RATE_CONFIG,
defaultConnectionRateValue), AlterConfigOp.OpType.SET)
+)
+);
+admin.incrementalAlterConfigs(alterations).all().get(1,
TimeUnit.MINUTES);
+
+// Verify per-broker configs: MAX_CONNECTIONS_CONFIG and
MAX_CONNECTION_CREATION_RATE_CONFIG
+verifyConfigValue(admin, nodeResource,
SocketServerConfigs.MAX_CONNECTIONS_CONFIG,
+DYNAMIC_BROKER_CONFIG, nodeMaxConnectionsValue);
+verifyConfigValue(admin, nodeResource,
SocketServerConfigs.MAX_CONNECTION_CREATION_RATE_CONFIG,
+DYNAMIC_DEFAULT_BROKER_CONFIG,
defaultConnectionRateValue);
+
+// Verify default broker configs: MAX_CONNECTIONS_CONFIG and
MAX_CONNECTION_CREATION_RATE_CONFIG
+verifyConfigValue(admin, defaultResource,
SocketServerConfigs.MAX_CONNECTIONS_CONFIG,
+DYNAMIC_DEFAULT_BROKER_CONFIG,
defaultMaxConnectionsValue);
+verifyConfigValue(admin, defaultResource,
SocketServerConfigs.MAX_CONNECTION_CREATION_RATE_CONFIG,
+DYNAMIC_DEFAULT_BROKER_CONFIG,
defaultConnectionRateValue);
Review Comment:
Would you mind verifying `SocketServer` to ensure the max connectio limit
gets updated?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For
Re: [PR] KAFKA-18928: Support dynamically changing configs for a specific dynamic quorum controller [kafka]
jiafu1115 commented on code in PR #20938:
URL: https://github.com/apache/kafka/pull/20938#discussion_r2657861535
##
server/src/test/java/org/apache/kafka/server/BootstrapControllersIntegrationTest.java:
##
@@ -226,27 +226,29 @@ public void testIncrementalAlterConfigs(ClusterInstance
clusterInstance) throws
}
private void testIncrementalAlterConfigs(ClusterInstance clusterInstance,
boolean usingBootstrapControllers) throws Exception {
+Collection nodeIds = usingBootstrapControllers ?
+clusterInstance.controllerIds() :
+
List.of(clusterInstance.brokers().values().iterator().next().config().nodeId());
try (Admin admin = Admin.create(adminConfig(clusterInstance,
usingBootstrapControllers))) {
-int nodeId = usingBootstrapControllers ?
-
clusterInstance.controllers().values().iterator().next().config().nodeId() :
-
clusterInstance.brokers().values().iterator().next().config().nodeId();
-ConfigResource nodeResource = new ConfigResource(BROKER, "" +
nodeId);
-ConfigResource defaultResource = new ConfigResource(BROKER, "");
-Map> alterations =
Map.of(
-nodeResource, List.of(new AlterConfigOp(new
ConfigEntry("my.custom.config", "foo"), AlterConfigOp.OpType.SET)),
-defaultResource, List.of(new AlterConfigOp(new
ConfigEntry("my.custom.config", "bar"), AlterConfigOp.OpType.SET))
-);
-admin.incrementalAlterConfigs(alterations).all().get(1,
TimeUnit.MINUTES);
-TestUtils.retryOnExceptionWithTimeout(30_000, () -> {
-Config config = admin.describeConfigs(List.of(nodeResource)).
-all().get(1, TimeUnit.MINUTES).get(nodeResource);
-ConfigEntry entry = config.entries().stream().
-filter(e -> e.name().equals("my.custom.config")).
-findFirst().orElseThrow();
-assertEquals(DYNAMIC_BROKER_CONFIG, entry.source(),
-"Expected entry for my.custom.config to come from
DYNAMIC_BROKER_CONFIG. " +
-"Instead, the entry was: " + entry);
-});
+for (int nodeId : nodeIds) {
+ConfigResource nodeResource = new ConfigResource(BROKER, "" +
nodeId);
+ConfigResource defaultResource = new ConfigResource(BROKER,
"");
+Map> alterations =
Map.of(
+nodeResource, List.of(new AlterConfigOp(new
ConfigEntry("my.custom.config", "foo"), AlterConfigOp.OpType.SET)),
+defaultResource, List.of(new AlterConfigOp(new
ConfigEntry("my.custom.config", "bar"), AlterConfigOp.OpType.SET))
Review Comment:
@chia7712 Done. I add two configure items for test instead of only one.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
Re: [PR] KAFKA-18928: Support dynamically changing configs for a specific dynamic quorum controller [kafka]
jiafu1115 commented on code in PR #20938:
URL: https://github.com/apache/kafka/pull/20938#discussion_r2657845028
##
test-common/test-common-runtime/src/main/java/org/apache/kafka/common/test/junit/RaftClusterInvocationContext.java:
##
@@ -324,6 +324,9 @@ public void format() throws Exception {
KafkaClusterTestKit.Builder builder = new
KafkaClusterTestKit.Builder(nodes);
// Copy properties into the TestKit builder
clusterConfig.serverProperties().forEach(builder::setConfigProp);
+if (clusterConfig.standalone()) {
+builder.setStandalone(true);
+}
Review Comment:
@TaiJuWu Done. good catch. thanks!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
Re: [PR] KAFKA-18928: Support dynamically changing configs for a specific dynamic quorum controller [kafka]
jiafu1115 commented on code in PR #20938:
URL: https://github.com/apache/kafka/pull/20938#discussion_r2657835465
##
server/src/test/java/org/apache/kafka/server/BootstrapControllersIntegrationTest.java:
##
@@ -226,27 +226,29 @@ public void testIncrementalAlterConfigs(ClusterInstance
clusterInstance) throws
}
private void testIncrementalAlterConfigs(ClusterInstance clusterInstance,
boolean usingBootstrapControllers) throws Exception {
+Collection nodeIds = usingBootstrapControllers ?
+clusterInstance.controllerIds() :
+
List.of(clusterInstance.brokers().values().iterator().next().config().nodeId());
try (Admin admin = Admin.create(adminConfig(clusterInstance,
usingBootstrapControllers))) {
-int nodeId = usingBootstrapControllers ?
-
clusterInstance.controllers().values().iterator().next().config().nodeId() :
-
clusterInstance.brokers().values().iterator().next().config().nodeId();
-ConfigResource nodeResource = new ConfigResource(BROKER, "" +
nodeId);
-ConfigResource defaultResource = new ConfigResource(BROKER, "");
-Map> alterations =
Map.of(
-nodeResource, List.of(new AlterConfigOp(new
ConfigEntry("my.custom.config", "foo"), AlterConfigOp.OpType.SET)),
-defaultResource, List.of(new AlterConfigOp(new
ConfigEntry("my.custom.config", "bar"), AlterConfigOp.OpType.SET))
-);
-admin.incrementalAlterConfigs(alterations).all().get(1,
TimeUnit.MINUTES);
-TestUtils.retryOnExceptionWithTimeout(30_000, () -> {
-Config config = admin.describeConfigs(List.of(nodeResource)).
-all().get(1, TimeUnit.MINUTES).get(nodeResource);
-ConfigEntry entry = config.entries().stream().
-filter(e -> e.name().equals("my.custom.config")).
-findFirst().orElseThrow();
-assertEquals(DYNAMIC_BROKER_CONFIG, entry.source(),
-"Expected entry for my.custom.config to come from
DYNAMIC_BROKER_CONFIG. " +
-"Instead, the entry was: " + entry);
-});
+for (int nodeId : nodeIds) {
+ConfigResource nodeResource = new ConfigResource(BROKER, "" +
nodeId);
+ConfigResource defaultResource = new ConfigResource(BROKER,
"");
+Map> alterations =
Map.of(
+nodeResource, List.of(new AlterConfigOp(new
ConfigEntry("my.custom.config", "foo"), AlterConfigOp.OpType.SET)),
+defaultResource, List.of(new AlterConfigOp(new
ConfigEntry("my.custom.config", "bar"), AlterConfigOp.OpType.SET))
+);
+admin.incrementalAlterConfigs(alterations).all().get(1,
TimeUnit.MINUTES);
+TestUtils.retryOnExceptionWithTimeout(30_000, () -> {
+Config config =
admin.describeConfigs(List.of(nodeResource)).
+all().get(1, TimeUnit.MINUTES).get(nodeResource);
+ConfigEntry entry = config.entries().stream().
+filter(e -> e.name().equals("my.custom.config")).
+findFirst().orElseThrow();
+assertEquals(DYNAMIC_BROKER_CONFIG, entry.source(),
Review Comment:
@chia7712 Done. I’ve already updated the test to use max.connections instead
of my.custom.config, since the latter is treated as sensitive and its value
cannot be queried. That's the reason for why existed tests don't check the
value.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
Re: [PR] KAFKA-18928: Support dynamically changing configs for a specific dynamic quorum controller [kafka]
jiafu1115 commented on code in PR #20938:
URL: https://github.com/apache/kafka/pull/20938#discussion_r2657829788
##
server/src/test/java/org/apache/kafka/server/BootstrapControllersIntegrationTest.java:
##
@@ -226,27 +226,29 @@ public void testIncrementalAlterConfigs(ClusterInstance
clusterInstance) throws
}
private void testIncrementalAlterConfigs(ClusterInstance clusterInstance,
boolean usingBootstrapControllers) throws Exception {
+Collection nodeIds = usingBootstrapControllers ?
+clusterInstance.controllerIds() :
+
List.of(clusterInstance.brokers().values().iterator().next().config().nodeId());
Review Comment:
@chia7712 done. thanks
https://github.com/user-attachments/assets/ef44a76d-9518-4379-a929-e08a50e92f17";
/>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
Re: [PR] KAFKA-18928: Support dynamically changing configs for a specific dynamic quorum controller [kafka]
TaiJuWu commented on code in PR #20938:
URL: https://github.com/apache/kafka/pull/20938#discussion_r2657828783
##
test-common/test-common-runtime/src/main/java/org/apache/kafka/common/test/junit/RaftClusterInvocationContext.java:
##
@@ -324,6 +324,9 @@ public void format() throws Exception {
KafkaClusterTestKit.Builder builder = new
KafkaClusterTestKit.Builder(nodes);
// Copy properties into the TestKit builder
clusterConfig.serverProperties().forEach(builder::setConfigProp);
+if (clusterConfig.standalone()) {
+builder.setStandalone(true);
+}
Review Comment:
How about `builder.setStandalone(clusterConfig.standalone())` ?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
Re: [PR] KAFKA-18928: Support dynamically changing configs for a specific dynamic quorum controller [kafka]
chia7712 commented on code in PR #20938:
URL: https://github.com/apache/kafka/pull/20938#discussion_r2657710627
##
server/src/test/java/org/apache/kafka/server/BootstrapControllersIntegrationTest.java:
##
@@ -226,27 +226,29 @@ public void testIncrementalAlterConfigs(ClusterInstance
clusterInstance) throws
}
private void testIncrementalAlterConfigs(ClusterInstance clusterInstance,
boolean usingBootstrapControllers) throws Exception {
+Collection nodeIds = usingBootstrapControllers ?
+clusterInstance.controllerIds() :
+
List.of(clusterInstance.brokers().values().iterator().next().config().nodeId());
Review Comment:
```java
Collection nodeIds = usingBootstrapControllers ?
clusterInstance.controllerIds() :
clusterInstance.brokers().keySet();
```
##
server/src/test/java/org/apache/kafka/server/BootstrapControllersIntegrationTest.java:
##
@@ -226,27 +226,29 @@ public void testIncrementalAlterConfigs(ClusterInstance
clusterInstance) throws
}
private void testIncrementalAlterConfigs(ClusterInstance clusterInstance,
boolean usingBootstrapControllers) throws Exception {
+Collection nodeIds = usingBootstrapControllers ?
+clusterInstance.controllerIds() :
+
List.of(clusterInstance.brokers().values().iterator().next().config().nodeId());
try (Admin admin = Admin.create(adminConfig(clusterInstance,
usingBootstrapControllers))) {
-int nodeId = usingBootstrapControllers ?
-
clusterInstance.controllers().values().iterator().next().config().nodeId() :
-
clusterInstance.brokers().values().iterator().next().config().nodeId();
-ConfigResource nodeResource = new ConfigResource(BROKER, "" +
nodeId);
-ConfigResource defaultResource = new ConfigResource(BROKER, "");
-Map> alterations =
Map.of(
-nodeResource, List.of(new AlterConfigOp(new
ConfigEntry("my.custom.config", "foo"), AlterConfigOp.OpType.SET)),
-defaultResource, List.of(new AlterConfigOp(new
ConfigEntry("my.custom.config", "bar"), AlterConfigOp.OpType.SET))
-);
-admin.incrementalAlterConfigs(alterations).all().get(1,
TimeUnit.MINUTES);
-TestUtils.retryOnExceptionWithTimeout(30_000, () -> {
-Config config = admin.describeConfigs(List.of(nodeResource)).
-all().get(1, TimeUnit.MINUTES).get(nodeResource);
-ConfigEntry entry = config.entries().stream().
-filter(e -> e.name().equals("my.custom.config")).
-findFirst().orElseThrow();
-assertEquals(DYNAMIC_BROKER_CONFIG, entry.source(),
-"Expected entry for my.custom.config to come from
DYNAMIC_BROKER_CONFIG. " +
-"Instead, the entry was: " + entry);
-});
+for (int nodeId : nodeIds) {
+ConfigResource nodeResource = new ConfigResource(BROKER, "" +
nodeId);
+ConfigResource defaultResource = new ConfigResource(BROKER,
"");
+Map> alterations =
Map.of(
+nodeResource, List.of(new AlterConfigOp(new
ConfigEntry("my.custom.config", "foo"), AlterConfigOp.OpType.SET)),
+defaultResource, List.of(new AlterConfigOp(new
ConfigEntry("my.custom.config", "bar"), AlterConfigOp.OpType.SET))
+);
+admin.incrementalAlterConfigs(alterations).all().get(1,
TimeUnit.MINUTES);
+TestUtils.retryOnExceptionWithTimeout(30_000, () -> {
+Config config =
admin.describeConfigs(List.of(nodeResource)).
+all().get(1, TimeUnit.MINUTES).get(nodeResource);
+ConfigEntry entry = config.entries().stream().
+filter(e -> e.name().equals("my.custom.config")).
+findFirst().orElseThrow();
+assertEquals(DYNAMIC_BROKER_CONFIG, entry.source(),
Review Comment:
why we don't verify the value?
##
server/src/test/java/org/apache/kafka/server/BootstrapControllersIntegrationTest.java:
##
@@ -226,27 +226,29 @@ public void testIncrementalAlterConfigs(ClusterInstance
clusterInstance) throws
}
private void testIncrementalAlterConfigs(ClusterInstance clusterInstance,
boolean usingBootstrapControllers) throws Exception {
+Collection nodeIds = usingBootstrapControllers ?
+clusterInstance.controllerIds() :
+
List.of(clusterInstance.brokers().values().iterator().next().config().nodeId());
try (Admin admin = Admin.create(adminConfig(clusterInstance,
usingBootstrapControllers))) {
-int nodeId = usingBootstrapControllers ?
-
Re: [PR] KAFKA-18928: Support dynamically changing configs for a specific dynamic quorum controller [kafka]
jiafu1115 commented on code in PR #20938:
URL: https://github.com/apache/kafka/pull/20938#discussion_r2639338880
##
server/src/test/java/org/apache/kafka/server/BootstrapControllersIntegrationTest.java:
##
@@ -357,4 +357,9 @@ public void testDescribeConfigs(ClusterInstance
clusterInstance) throws Exceptio
assertEquals("2", configEntry.value());
}
}
+
+@ClusterTest(controllers = 1, standalone = true)
+public void
testIncrementalAlterConfigsByControllersWithDynamicQuorum(ClusterInstance
clusterInstance) throws Exception {
+testIncrementalAlterConfigs(clusterInstance, true);
Review Comment:
@chia7712 done. add one test: create one 3 controller cluster and one admin
client. then change 3 controller.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
Re: [PR] KAFKA-18928: Support dynamically changing configs for a specific dynamic quorum controller [kafka]
chia7712 commented on code in PR #20938:
URL: https://github.com/apache/kafka/pull/20938#discussion_r2638800463
##
server/src/test/java/org/apache/kafka/server/BootstrapControllersIntegrationTest.java:
##
@@ -357,4 +357,9 @@ public void testDescribeConfigs(ClusterInstance
clusterInstance) throws Exceptio
assertEquals("2", configEntry.value());
}
}
+
+@ClusterTest(controllers = 1, standalone = true)
+public void
testIncrementalAlterConfigsByControllersWithDynamicQuorum(ClusterInstance
clusterInstance) throws Exception {
+testIncrementalAlterConfigs(clusterInstance, true);
Review Comment:
Could you add a test to ensure all controllers can accept dynamic
configurations? The current logic only verifies the single initial controller
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
Re: [PR] KAFKA-18928: Support dynamically changing configs for a specific dynamic quorum controller [kafka]
jiafu1115 commented on PR #20938: URL: https://github.com/apache/kafka/pull/20938#issuecomment-3677793147 > @jiafu1115 please fix the docs @chia7712 done. thanks -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
Re: [PR] KAFKA-18928: Support dynamically changing configs for a specific dynamic quorum controller [kafka]
chia7712 commented on PR #20938: URL: https://github.com/apache/kafka/pull/20938#issuecomment-361487 @jiafu1115 please fix the docs -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
Re: [PR] KAFKA-18928: Support dynamically changing configs for a specific dynamic quorum controller [kafka]
jiafu1115 commented on code in PR #20938:
URL: https://github.com/apache/kafka/pull/20938#discussion_r2636629968
##
docs/documentation/zk2kraft.html:
##
@@ -210,7 +210,7 @@ Dynamic Controller
Configurations
--alter \
--add-config max.connections=1
-It is not currently possible to apply a dynamic configuration
on only a single controller.
+Prior to version 4.3, it was not possible to apply a dynamic
configuration on only a single dynamic controller.
Review Comment:
@chia7712 based on follow tests' result, I think your propose make sense.
There is no way to update all controllers at once. --entity-default only
changes the default value and does not override values that have already been
set.
Highlighting “only a single dynamic controller” may be misleading, since
users might assume there is a way to update all controllers, which is not the
case.
1 alter command should contain entity-name or --entity-default in alter
command
root@ip-10-20-1-107:/usr/local/kafka/bin# ./kafka-configs.sh
--bootstrap-controller --bootstrap-controller 10.20.1.107:9889 --entity-type
brokers --command-config /etc/kafka/admin_config.properties --alter
--add-config max.connections=10086
**An entity-name or default entity must be specified with --alter of users,
clients, brokers or ips**
2 --entity-default is to update the default value:
DYNAMIC_DEFAULT_BROKER_CONFIG instead of the all controller.
root@ip-10-20-1-107:/usr/local/kafka/bin# ./kafka-configs.sh
--bootstrap-controller 10.20.1.107:9889 --entity-type brokers
**--entity-default**--command-config /etc/kafka/admin_config.properties
--alter --add-config **max.connections=10086**
Completed updating default config for brokers in the cluster.
root@ip-10-20-1-107:/usr/local/kafka/bin# ./kafka-configs.sh
--bootstrap-controller 10.20.1.107:9889 --entity-type brokers --entity-name
1001 --command-config /etc/kafka/admin_config.properties --describe --all
|grep -E 'max.connections|node'
max.connections=10086 sensitive=false
synonyms={**DYNAMIC_DEFAULT_BROKER_CONFIG:max.connections=10086**,
DEFAULT_CONFIG:max.connections=2147483647}
3 alert with --entity-name will override the default value. (note: after
changed. if alter with --entity-default. it won't change the final value. it
only change the default value)
root@ip-10-20-1-107:/usr/local/kafka/bin# ./kafka-configs.sh
--bootstrap-controller 10.20.1.107:9889 --entity-type brokers **--entity-name
1001**--command-config /etc/kafka/admin_config.properties --alter
--add-config max.connections=**1**
Completed updating config for broker 1001.
root@ip-10-20-1-107:/usr/local/kafka/bin# ./kafka-configs.sh
--bootstrap-controller 10.20.1.107:9889 --entity-type brokers --entity-name
1001 --command-config /etc/kafka/admin_config.properties --describe --all
|grep -E 'max.connections|node'
max.connections=1 sensitive=false
synonyms={**DYNAMIC_BROKER_CONFIG:max.connections=1**,
**DYNAMIC_DEFAULT_BROKER_CONFIG:max.connections=10086**,
DEFAULT_CONFIG:max.connections=2147483647}
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
Re: [PR] KAFKA-18928: Support dynamically changing configs for a specific dynamic quorum controller [kafka]
jiafu1115 commented on code in PR #20938:
URL: https://github.com/apache/kafka/pull/20938#discussion_r2636629968
##
docs/documentation/zk2kraft.html:
##
@@ -210,7 +210,7 @@ Dynamic Controller
Configurations
--alter \
--add-config max.connections=1
-It is not currently possible to apply a dynamic configuration
on only a single controller.
+Prior to version 4.3, it was not possible to apply a dynamic
configuration on only a single dynamic controller.
Review Comment:
@chia7712 based on follow tests' result, I think your propose make sense.
There is no any method to change all controllers in only one time. change
with --entity-default only change the default value. it won't change the final
value if the value had been set.
So there is no need to highlight "only a single dynamic controller".if so.
it will hint the user "you can change all the controller instead of only one".
In fact. --entity-default is to change the default value.
1 alter command should contain entity-name or --entity-default in alter
command
root@ip-10-20-1-107:/usr/local/kafka/bin# ./kafka-configs.sh
--bootstrap-controller --bootstrap-controller 10.20.1.107:9889 --entity-type
brokers --command-config /etc/kafka/admin_config.properties --alter
--add-config max.connections=10086
**An entity-name or default entity must be specified with --alter of users,
clients, brokers or ips**
2 --entity-default is to update the default value:
DYNAMIC_DEFAULT_BROKER_CONFIG instead of the all controller.
root@ip-10-20-1-107:/usr/local/kafka/bin# ./kafka-configs.sh
--bootstrap-controller 10.20.1.107:9889 --entity-type brokers
**--entity-default**--command-config /etc/kafka/admin_config.properties
--alter --add-config **max.connections=10086**
Completed updating default config for brokers in the cluster.
root@ip-10-20-1-107:/usr/local/kafka/bin# ./kafka-configs.sh
--bootstrap-controller 10.20.1.107:9889 --entity-type brokers --entity-name
1001 --command-config /etc/kafka/admin_config.properties --describe --all
|grep -E 'max.connections|node'
max.connections=10086 sensitive=false
synonyms={**DYNAMIC_DEFAULT_BROKER_CONFIG:max.connections=10086**,
DEFAULT_CONFIG:max.connections=2147483647}
3 alert with --entity-name will override the default value. (note: after
changed. if alter with --entity-default. it won't change the final value. it
only change the default value)
root@ip-10-20-1-107:/usr/local/kafka/bin# ./kafka-configs.sh
--bootstrap-controller 10.20.1.107:9889 --entity-type brokers **--entity-name
1001**--command-config /etc/kafka/admin_config.properties --alter
--add-config max.connections=**1**
Completed updating config for broker 1001.
root@ip-10-20-1-107:/usr/local/kafka/bin# ./kafka-configs.sh
--bootstrap-controller 10.20.1.107:9889 --entity-type brokers --entity-name
1001 --command-config /etc/kafka/admin_config.properties --describe --all
|grep -E 'max.connections|node'
max.connections=1 sensitive=false
synonyms={**DYNAMIC_BROKER_CONFIG:max.connections=1**,
**DYNAMIC_DEFAULT_BROKER_CONFIG:max.connections=10086**,
DEFAULT_CONFIG:max.connections=2147483647}
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
Re: [PR] KAFKA-18928: Support dynamically changing configs for a specific dynamic quorum controller [kafka]
jiafu1115 commented on PR #20938:
URL: https://github.com/apache/kafka/pull/20938#issuecomment-3677084064
Attach additional test results: (not directly related to this change;
recorded here to avoid forgetting this logic.)
There is no any method to change all controllers in only one time. change
with --entity-default only change the default value. it won't change the final
value if the value had been set.
1 alter command should contain entity-name or --entity-default in alter
command
root@ip-10-20-1-107:/usr/local/kafka/bin# ./kafka-configs.sh
--bootstrap-controller --bootstrap-controller 10.20.1.107:9889 --entity-type
brokers --command-config /etc/kafka/admin_config.properties --alter
--add-config max.connections=10086
**An entity-name or default entity must be specified with --alter of users,
clients, brokers or ips**
2 --entity-default is to update the default value:
DYNAMIC_DEFAULT_BROKER_CONFIG instead of the all controller.
root@ip-10-20-1-107:/usr/local/kafka/bin# ./kafka-configs.sh
--bootstrap-controller 10.20.1.107:9889 --entity-type brokers
**--entity-default**--command-config /etc/kafka/admin_config.properties
--alter --add-config **max.connections=10086**
Completed updating default config for brokers in the cluster.
root@ip-10-20-1-107:/usr/local/kafka/bin# ./kafka-configs.sh
--bootstrap-controller 10.20.1.107:9889 --entity-type brokers --entity-name
1001 --command-config /etc/kafka/admin_config.properties --describe --all
|grep -E 'max.connections|node'
max.connections=10086 sensitive=false
synonyms={**DYNAMIC_DEFAULT_BROKER_CONFIG:max.connections=10086**,
DEFAULT_CONFIG:max.connections=2147483647}
3 alert with --entity-name will override the default value. (note: after
changed. if alter with --entity-default. it won't change the final value. it
only change the default value)
root@ip-10-20-1-107:/usr/local/kafka/bin# ./kafka-configs.sh
--bootstrap-controller 10.20.1.107:9889 --entity-type brokers **--entity-name
1001**--command-config /etc/kafka/admin_config.properties --alter
--add-config max.connections=**1**
Completed updating config for broker 1001.
root@ip-10-20-1-107:/usr/local/kafka/bin# ./kafka-configs.sh
--bootstrap-controller 10.20.1.107:9889 --entity-type brokers --entity-name
1001 --command-config /etc/kafka/admin_config.properties --describe --all
|grep -E 'max.connections|node'
max.connections=1 sensitive=false
synonyms={**DYNAMIC_BROKER_CONFIG:max.connections=1**,
**DYNAMIC_DEFAULT_BROKER_CONFIG:max.connections=10086**,
DEFAULT_CONFIG:max.connections=2147483647}
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
Re: [PR] KAFKA-18928: Support dynamically changing configs for a specific dynamic quorum controller [kafka]
jiafu1115 commented on code in PR #20938: URL: https://github.com/apache/kafka/pull/20938#discussion_r2636634343 ## docs/documentation/zk2kraft.html: ## @@ -210,7 +210,7 @@ Dynamic Controller Configurations --alter \ --add-config max.connections=1 -It is not currently possible to apply a dynamic configuration on only a single controller. +Prior to version 4.3, it was not possible to apply a dynamic configuration on only a single dynamic controller. Review Comment: @chia7712 done with the change. thanks for your review! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
Re: [PR] KAFKA-18928: Support dynamically changing configs for a specific dynamic quorum controller [kafka]
jiafu1115 commented on code in PR #20938:
URL: https://github.com/apache/kafka/pull/20938#discussion_r2636629968
##
docs/documentation/zk2kraft.html:
##
@@ -210,7 +210,7 @@ Dynamic Controller
Configurations
--alter \
--add-config max.connections=1
-It is not currently possible to apply a dynamic configuration
on only a single controller.
+Prior to version 4.3, it was not possible to apply a dynamic
configuration on only a single dynamic controller.
Review Comment:
@chia7712 based on follow tests' result, I think your propose make sense.
There is no any method to change all controllers in only one time. change
with --entity-default only change the default value. it won't change the final
value if the value had been set.
So there is no need to highlight "only a single dynamic controller"
1 alter command should contain entity-name or --entity-default in alter
command
root@ip-10-20-1-107:/usr/local/kafka/bin# ./kafka-configs.sh
--bootstrap-controller --bootstrap-controller 10.20.1.107:9889 --entity-type
brokers --command-config /etc/kafka/admin_config.properties --alter
--add-config max.connections=10086
**An entity-name or default entity must be specified with --alter of users,
clients, brokers or ips**
2 --entity-default is to update the default value:
DYNAMIC_DEFAULT_BROKER_CONFIG instead of the all controller.
root@ip-10-20-1-107:/usr/local/kafka/bin# ./kafka-configs.sh
--bootstrap-controller 10.20.1.107:9889 --entity-type brokers
**--entity-default**--command-config /etc/kafka/admin_config.properties
--alter --add-config **max.connections=10086**
Completed updating default config for brokers in the cluster.
root@ip-10-20-1-107:/usr/local/kafka/bin# ./kafka-configs.sh
--bootstrap-controller 10.20.1.107:9889 --entity-type brokers --entity-name
1001 --command-config /etc/kafka/admin_config.properties --describe --all
|grep -E 'max.connections|node'
max.connections=10086 sensitive=false
synonyms={**DYNAMIC_DEFAULT_BROKER_CONFIG:max.connections=10086**,
DEFAULT_CONFIG:max.connections=2147483647}
3 alert with --entity-name will override the default value. (note: after
changed. if alter with --entity-default. it won't change the final value. it
only change the default value)
root@ip-10-20-1-107:/usr/local/kafka/bin# ./kafka-configs.sh
--bootstrap-controller 10.20.1.107:9889 --entity-type brokers **--entity-name
1001**--command-config /etc/kafka/admin_config.properties --alter
--add-config max.connections=**1**
Completed updating config for broker 1001.
root@ip-10-20-1-107:/usr/local/kafka/bin# ./kafka-configs.sh
--bootstrap-controller 10.20.1.107:9889 --entity-type brokers --entity-name
1001 --command-config /etc/kafka/admin_config.properties --describe --all
|grep -E 'max.connections|node'
max.connections=1 sensitive=false
synonyms={**DYNAMIC_BROKER_CONFIG:max.connections=1**,
**DYNAMIC_DEFAULT_BROKER_CONFIG:max.connections=10086**,
DEFAULT_CONFIG:max.connections=2147483647}
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
Re: [PR] KAFKA-18928: Support dynamically changing configs for a specific dynamic quorum controller [kafka]
jiafu1115 commented on code in PR #20938:
URL: https://github.com/apache/kafka/pull/20938#discussion_r2636629968
##
docs/documentation/zk2kraft.html:
##
@@ -210,7 +210,7 @@ Dynamic Controller
Configurations
--alter \
--add-config max.connections=1
-It is not currently possible to apply a dynamic configuration
on only a single controller.
+Prior to version 4.3, it was not possible to apply a dynamic
configuration on only a single dynamic controller.
Review Comment:
@chia7712 based on follow tests' result, I think your propose make sense.
There is no method to change all controllers in only one time. change with
--entity-default only change the default value. it won't change the final value
if the value had been set.
1 alter command should contain entity-name or --entity-default in alter
command
root@ip-10-20-1-107:/usr/local/kafka/bin# ./kafka-configs.sh
--bootstrap-controller --bootstrap-controller 10.20.1.107:9889 --entity-type
brokers --command-config /etc/kafka/admin_config.properties --alter
--add-config max.connections=10086
**An entity-name or default entity must be specified with --alter of users,
clients, brokers or ips**
2 --entity-default is to update the default value:
DYNAMIC_DEFAULT_BROKER_CONFIG instead of the all controller.
root@ip-10-20-1-107:/usr/local/kafka/bin# ./kafka-configs.sh
--bootstrap-controller 10.20.1.107:9889 --entity-type brokers
**--entity-default**--command-config /etc/kafka/admin_config.properties
--alter --add-config **max.connections=10086**
Completed updating default config for brokers in the cluster.
root@ip-10-20-1-107:/usr/local/kafka/bin# ./kafka-configs.sh
--bootstrap-controller 10.20.1.107:9889 --entity-type brokers --entity-name
1001 --command-config /etc/kafka/admin_config.properties --describe --all
|grep -E 'max.connections|node'
max.connections=10086 sensitive=false
synonyms={**DYNAMIC_DEFAULT_BROKER_CONFIG:max.connections=10086**,
DEFAULT_CONFIG:max.connections=2147483647}
3 alert with --entity-name will override the default value. (note: after
changed. if alter with --entity-default. it won't change the final value. it
only change the default value)
root@ip-10-20-1-107:/usr/local/kafka/bin# ./kafka-configs.sh
--bootstrap-controller 10.20.1.107:9889 --entity-type brokers **--entity-name
1001**--command-config /etc/kafka/admin_config.properties --alter
--add-config max.connections=**1**
Completed updating config for broker 1001.
root@ip-10-20-1-107:/usr/local/kafka/bin# ./kafka-configs.sh
--bootstrap-controller 10.20.1.107:9889 --entity-type brokers --entity-name
1001 --command-config /etc/kafka/admin_config.properties --describe --all
|grep -E 'max.connections|node'
max.connections=1 sensitive=false
synonyms={**DYNAMIC_BROKER_CONFIG:max.connections=1**,
**DYNAMIC_DEFAULT_BROKER_CONFIG:max.connections=10086**,
DEFAULT_CONFIG:max.connections=2147483647}
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
Re: [PR] KAFKA-18928: Support dynamically changing configs for a specific dynamic quorum controller [kafka]
jiafu1115 commented on code in PR #20938:
URL: https://github.com/apache/kafka/pull/20938#discussion_r2636629968
##
docs/documentation/zk2kraft.html:
##
@@ -210,7 +210,7 @@ Dynamic Controller
Configurations
--alter \
--add-config max.connections=1
-It is not currently possible to apply a dynamic configuration
on only a single controller.
+Prior to version 4.3, it was not possible to apply a dynamic
configuration on only a single dynamic controller.
Review Comment:
@chia7712 based on follow tests' result, I think the change is make sense.
There is no method to change all controllers in only one time. change with
--entity-default only change the default value. it won't change the final value
if the value had been set.
1 alter command should contain entity-name or --entity-default in alter
command
root@ip-10-20-1-107:/usr/local/kafka/bin# ./kafka-configs.sh
--bootstrap-controller --bootstrap-controller 10.20.1.107:9889 --entity-type
brokers --command-config /etc/kafka/admin_config.properties --alter
--add-config max.connections=10086
**An entity-name or default entity must be specified with --alter of users,
clients, brokers or ips**
2 --entity-default is to update the default value:
DYNAMIC_DEFAULT_BROKER_CONFIG instead of the all controller.
root@ip-10-20-1-107:/usr/local/kafka/bin# ./kafka-configs.sh
--bootstrap-controller 10.20.1.107:9889 --entity-type brokers
**--entity-default**--command-config /etc/kafka/admin_config.properties
--alter --add-config **max.connections=10086**
Completed updating default config for brokers in the cluster.
root@ip-10-20-1-107:/usr/local/kafka/bin# ./kafka-configs.sh
--bootstrap-controller 10.20.1.107:9889 --entity-type brokers --entity-name
1001 --command-config /etc/kafka/admin_config.properties --describe --all
|grep -E 'max.connections|node'
max.connections=10086 sensitive=false
synonyms={**DYNAMIC_DEFAULT_BROKER_CONFIG:max.connections=10086**,
DEFAULT_CONFIG:max.connections=2147483647}
3 alert with --entity-name will override the default value. (note: after
changed. if alter with --entity-default. it won't change the final value. it
only change the default value)
root@ip-10-20-1-107:/usr/local/kafka/bin# ./kafka-configs.sh
--bootstrap-controller 10.20.1.107:9889 --entity-type brokers **--entity-name
1001**--command-config /etc/kafka/admin_config.properties --alter
--add-config max.connections=**1**
Completed updating config for broker 1001.
root@ip-10-20-1-107:/usr/local/kafka/bin# ./kafka-configs.sh
--bootstrap-controller 10.20.1.107:9889 --entity-type brokers --entity-name
1001 --command-config /etc/kafka/admin_config.properties --describe --all
|grep -E 'max.connections|node'
max.connections=1 sensitive=false
synonyms={**DYNAMIC_BROKER_CONFIG:max.connections=1**,
**DYNAMIC_DEFAULT_BROKER_CONFIG:max.connections=10086**,
DEFAULT_CONFIG:max.connections=2147483647}
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
Re: [PR] KAFKA-18928: Support dynamically changing configs for a specific dynamic quorum controller [kafka]
chia7712 commented on code in PR #20938: URL: https://github.com/apache/kafka/pull/20938#discussion_r2635524597 ## docs/documentation/zk2kraft.html: ## @@ -210,7 +210,7 @@ Dynamic Controller Configurations --alter \ --add-config max.connections=1 -It is not currently possible to apply a dynamic configuration on only a single controller. +Prior to version 4.3, it was not possible to apply a dynamic configuration on only a single dynamic controller. Review Comment: how about "Prior to version 4.3, dynamic configuration updates were not supported unless a static quorum was used"? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
Re: [PR] KAFKA-18928: Support dynamically changing configs for a specific dynamic quorum controller [kafka]
jiafu1115 commented on PR #20938: URL: https://github.com/apache/kafka/pull/20938#issuecomment-3649929018 @apalan60 Thank you! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
Re: [PR] KAFKA-18928: Support dynamically changing configs for a specific dynamic quorum controller [kafka]
apalan60 commented on PR #20938:
URL: https://github.com/apache/kafka/pull/20938#issuecomment-3649449338
Thanks for the PR. I tested it locally with a dynamic voter setup, and it
successfully updates a single controller’s config without any exceptions.
```bash
./bin/kafka-configs.sh --bootstrap-controller localhost:9093 --entity-type
brokers --entity-name 2 --describe --all | grep log.cleaner.threads
./bin/kafka-configs.sh --bootstrap-controller localhost:9093 --entity-type
brokers --entity-name 2 --alter --add-config log.cleaner.threads=6
./bin/kafka-configs.sh --bootstrap-controller localhost:9093 --entity-type
brokers --entity-name 2 --describe --all | grep log.cleaner.threads
log.cleaner.threads=1 sensitive=false
synonyms={DEFAULT_CONFIG:log.cleaner.threads=1}
Completed updating config for broker 2.
log.cleaner.threads=6 sensitive=false
synonyms={DYNAMIC_BROKER_CONFIG:log.cleaner.threads=6,
DEFAULT_CONFIG:log.cleaner.threads=1}
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
Re: [PR] KAFKA-18928: Support dynamically changing configs for a specific dynamic quorum controller [kafka]
jiafu1115 commented on PR #20938: URL: https://github.com/apache/kafka/pull/20938#issuecomment-3646963278 > @jiafu1115 could you please fix the conflicts? @chia7712 done. thanks -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
Re: [PR] KAFKA-18928: Support dynamically changing configs for a specific dynamic quorum controller [kafka]
chia7712 commented on PR #20938: URL: https://github.com/apache/kafka/pull/20938#issuecomment-3646805080 @jiafu1115 could you please fix the conflicts? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
Re: [PR] KAFKA-18928: Support dynamically changing configs for a specific dynamic quorum controller [kafka]
jiafu1115 commented on PR #20938: URL: https://github.com/apache/kafka/pull/20938#issuecomment-3637372041 @chia7712 @ahuang98 add the related core test. thanks ! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
Re: [PR] KAFKA-18928: Support dynamically changing configs for a specific dynamic quorum controller [kafka]
chia7712 commented on PR #20938: URL: https://github.com/apache/kafka/pull/20938#issuecomment-3637249404 > And, we probably need additional tests in the code for the dynamic config update (for instance, check out DynamicConfigChangeTest) I agree. We should create a helper test to validate all dynamic configurations on the controller. We must also cover both static quorum and dynamic quorum. `ClusterTest` supports static quorum. The dynamic quorum tests could be implemented by `KafkaClusterTestKit` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
Re: [PR] KAFKA-18928: Support dynamically changing configs for a specific dynamic quorum controller [kafka]
jiafu1115 commented on PR #20938: URL: https://github.com/apache/kafka/pull/20938#issuecomment-3637046974 org.apache.kafka.server.BootstrapControllersIntegrationTest#testIncrementalAlterConfigsByControllers the existed test to prove static quorum can work. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
Re: [PR] KAFKA-18928: Support dynamically changing configs for a specific dynamic quorum controller [kafka]
jiafu1115 commented on PR #20938: URL: https://github.com/apache/kafka/pull/20938#issuecomment-3634864330 @ahuang98 thanks for reviewing again. you can check this code lines and compare with the log I pasted in last comment (I copied it here) https://github.com/user-attachments/assets/2b51c9f0-0977-4e59-82f0-72059de114a6"; /> [2025-12-05 23:52:39,776] INFO [SocketServer listenerType=CONTROLLER, nodeId=1001] Updating broker-wide maxConnections: 123456789 (kafka.network.SocketServer) For the unit test. I can take some time to check if we can add the extra one for it. thanks -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
Re: [PR] KAFKA-18928: Support dynamically changing configs for a specific dynamic quorum controller [kafka]
ahuang98 commented on PR #20938: URL: https://github.com/apache/kafka/pull/20938#issuecomment-3634844227 @jiafu1115 thanks for the response and for testing it out manually! Good to see that the config isn't picked up by the other controllers. I wanted to clarify that my bigger concern (maybe uninformed concern since I'm not very familiar with config code) is that we are able to set the config for individual controllers, but they might not actually take effect. For instance, maybe kafka correctly creates a config record in the metadata log that broker id 1001 has this new config, but is controller 1001 actually able to apply this config (would it actually limit max connections at the new value). And, we probably need additional tests in the code for the dynamic config update (for instance, check out DynamicConfigChangeTest) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
Re: [PR] KAFKA-18928: Support dynamically changing configs for a specific dynamic quorum controller [kafka]
jiafu1115 commented on PR #20938: URL: https://github.com/apache/kafka/pull/20938#issuecomment-3618998411 https://github.com/user-attachments/assets/1eeeb88d-02a1-4bc4-a0d4-51a9d434308c"; /> @ahuang98 Thanks for your review. I attach this test according to your suggestion. Help to check again. Thanks -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
Re: [PR] KAFKA-18928: Support dynamically changing configs for a specific dynamic quorum controller [kafka]
github-actions[bot] commented on PR #20938: URL: https://github.com/apache/kafka/pull/20938#issuecomment-3587798592 A label of 'needs-attention' was automatically added to this PR in order to raise the attention of the committers. Once this issue has been triaged, the `triage` label should be removed to prevent this automation from happening again. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
