[jira] [Commented] (KAFKA-10633) Constant probing rebalances in Streams 2.6
[ https://issues.apache.org/jira/browse/KAFKA-10633?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17224555#comment-17224555 ] Bruno Cadonna commented on KAFKA-10633: --- [~thebearmayor] [~eran-levy] FYI: The 2.6.1 release is currently ongoing. Here the plan: [https://cwiki.apache.org/confluence/display/KAFKA/Release+Plan+2.6.1] It is planned to be released this month. > Constant probing rebalances in Streams 2.6 > -- > > Key: KAFKA-10633 > URL: https://issues.apache.org/jira/browse/KAFKA-10633 > Project: Kafka > Issue Type: Bug > Components: streams >Affects Versions: 2.6.0 >Reporter: Bradley Peterson >Priority: Major > Attachments: Discover 2020-10-21T23 34 03.867Z - 2020-10-21T23 44 > 46.409Z.csv > > > We are seeing a few issues with the new rebalancing behavior in Streams 2.6. > This ticket is for constant probing rebalances on one StreamThread, but I'll > mention the other issues, as they may be related. > First, when we redeploy the application we see tasks being moved, even though > the task assignment was stable before redeploying. We would expect to see > tasks assigned back to the same instances and no movement. The application is > in EC2, with persistent EBS volumes, and we use static group membership to > avoid rebalancing. To redeploy the app we terminate all EC2 instances. The > new instances will reattach the EBS volumes and use the same group member id. > After redeploying, we sometimes see the group leader go into a tight probing > rebalance loop. This doesn't happen immediately, it could be several hours > later. Because the redeploy caused task movement, we see expected probing > rebalances every 10 minutes. But, then one thread will go into a tight loop > logging messages like "Triggering the followup rebalance scheduled for > 1603323868771 ms.", handling the partition assignment (which doesn't change), > then "Requested to schedule probing rebalance for 1603323868771 ms." This > repeats several times a second until the app is restarted again. I'll attach > a log export from one such incident. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (KAFKA-10666) Kafka doesn't use keystore / key / truststore passwords for named SSL connections
[ https://issues.apache.org/jira/browse/KAFKA-10666?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17224605#comment-17224605 ] lqjacklee commented on KAFKA-10666: --- [~pfjason] checked the code. we just load the configuration from the static key. > Kafka doesn't use keystore / key / truststore passwords for named SSL > connections > - > > Key: KAFKA-10666 > URL: https://issues.apache.org/jira/browse/KAFKA-10666 > Project: Kafka > Issue Type: Bug > Components: admin >Affects Versions: 2.5.0, 2.6.0 > Environment: kafka in an openjdk-11 docker container, the client java > application is in an alpine container. zookeeper in a separate container. >Reporter: Jason >Priority: Minor > > When configuring named listener SSL connections with ssl key and keystore > with passwords including listener.name.ourname.ssl.key.password, > listener.name.ourname.ssl.keystore.password, and > listener.name.ourname.ssl.truststore.password via via the AdminClient the > settings are not used and the setting is not accepted if the default > ssl.key.password or ssl.keystore.password are not set. We configure all > keystore and truststore values for the named listener in a single batch using > incrementalAlterConfigs. Additionally, when ssl.keystore.password is set to > the value of our keystore password the keystore is loaded for SSL > communication without issue, however if ssl.keystore.password is incorrect > and listener.name.ourname.keystore.password is correct, we are unable to load > the keystore with bad password errors. It appears that only the default > ssl.xxx.password settings are used. This setting is immutable as when we > attempt to set it we get an error indicating that the listener.name. setting > can be set. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Assigned] (KAFKA-9892) Producer state snapshot needs to be forced to disk
[ https://issues.apache.org/jira/browse/KAFKA-9892?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brajesh Kumar reassigned KAFKA-9892: - Assignee: (was: Brajesh Kumar) > Producer state snapshot needs to be forced to disk > -- > > Key: KAFKA-9892 > URL: https://issues.apache.org/jira/browse/KAFKA-9892 > Project: Kafka > Issue Type: Bug > Components: core >Affects Versions: 2.6.0 >Reporter: Jun Rao >Priority: Major > > Currently, ProducerStateManager.writeSnapshot() only calls > fileChannel.close(), but not explicitly fileChannel.force(). It seems force() > is not guaranteed to be called on close(). -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Assigned] (KAFKA-9892) Producer state snapshot needs to be forced to disk
[ https://issues.apache.org/jira/browse/KAFKA-9892?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brajesh Kumar reassigned KAFKA-9892: - Assignee: Brajesh Kumar > Producer state snapshot needs to be forced to disk > -- > > Key: KAFKA-9892 > URL: https://issues.apache.org/jira/browse/KAFKA-9892 > Project: Kafka > Issue Type: Bug > Components: core >Affects Versions: 2.6.0 >Reporter: Jun Rao >Assignee: Brajesh Kumar >Priority: Major > > Currently, ProducerStateManager.writeSnapshot() only calls > fileChannel.close(), but not explicitly fileChannel.force(). It seems force() > is not guaranteed to be called on close(). -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Assigned] (KAFKA-9892) Producer state snapshot needs to be forced to disk
[ https://issues.apache.org/jira/browse/KAFKA-9892?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brajesh Kumar reassigned KAFKA-9892: - Assignee: Brajesh Kumar > Producer state snapshot needs to be forced to disk > -- > > Key: KAFKA-9892 > URL: https://issues.apache.org/jira/browse/KAFKA-9892 > Project: Kafka > Issue Type: Bug > Components: core >Affects Versions: 2.6.0 >Reporter: Jun Rao >Assignee: Brajesh Kumar >Priority: Major > > Currently, ProducerStateManager.writeSnapshot() only calls > fileChannel.close(), but not explicitly fileChannel.force(). It seems force() > is not guaranteed to be called on close(). -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (KAFKA-10633) Constant probing rebalances in Streams 2.6
[ https://issues.apache.org/jira/browse/KAFKA-10633?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17224645#comment-17224645 ] Eran Levy commented on KAFKA-10633: --- thanks [~cadonna] [~thebearmayor] > Constant probing rebalances in Streams 2.6 > -- > > Key: KAFKA-10633 > URL: https://issues.apache.org/jira/browse/KAFKA-10633 > Project: Kafka > Issue Type: Bug > Components: streams >Affects Versions: 2.6.0 >Reporter: Bradley Peterson >Priority: Major > Attachments: Discover 2020-10-21T23 34 03.867Z - 2020-10-21T23 44 > 46.409Z.csv > > > We are seeing a few issues with the new rebalancing behavior in Streams 2.6. > This ticket is for constant probing rebalances on one StreamThread, but I'll > mention the other issues, as they may be related. > First, when we redeploy the application we see tasks being moved, even though > the task assignment was stable before redeploying. We would expect to see > tasks assigned back to the same instances and no movement. The application is > in EC2, with persistent EBS volumes, and we use static group membership to > avoid rebalancing. To redeploy the app we terminate all EC2 instances. The > new instances will reattach the EBS volumes and use the same group member id. > After redeploying, we sometimes see the group leader go into a tight probing > rebalance loop. This doesn't happen immediately, it could be several hours > later. Because the redeploy caused task movement, we see expected probing > rebalances every 10 minutes. But, then one thread will go into a tight loop > logging messages like "Triggering the followup rebalance scheduled for > 1603323868771 ms.", handling the partition assignment (which doesn't change), > then "Requested to schedule probing rebalance for 1603323868771 ms." This > repeats several times a second until the app is restarted again. I'll attach > a log export from one such incident. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (KAFKA-7500) MirrorMaker 2.0 (KIP-382)
[
https://issues.apache.org/jira/browse/KAFKA-7500?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17224651#comment-17224651
]
Sarita commented on KAFKA-7500:
---
Hi [~ryannedolan]
We are trying to set up MM2 with connection distributed setup.
Worker configs are as follows
```
bootstrap.servers=abc-broker:9093
security.protocol=SASL_SSL
sasl.mechanism=PLAIN
sasl.jaas.config=org.apache.kafka.common.security.plain.PlainLoginModule
required username="superuser" password="superuser_password";
group.id=connect-tails
key.converter=org.apache.kafka.connect.json.JsonConverter
value.converter=org.apache.kafka.connect.json.JsonConverter
key.converter.schemas.enable=false
value.converter.schemas.enable=false
internal.key.converter=org.apache.kafka.connect.json.JsonConverter
internal.value.converter=org.apache.kafka.connect.json.JsonConverter
internal.key.converter.schemas.enable=false
internal.value.converter.schemas.enable=false
offset.storage.topic=connect-offsets-test
config.storage.topic=connect-configs-test
status.storage.topic=connect-status-test
offset.flush.interval.ms=30
producer.buffer.memory=1234
producer.ssl.truststore.password=truststore_password
producer.ssl.truststore.location=/opt/projects/confluent/wildcard.kafka.iggroup.local.jks
producer.ssl.keystore.password=keystore_password
producer.ssl.keystore.location=/opt/projects/confluent/wildcard.kafka.iggroup.local.jks
```
When i run below command, I can see worker connector created:
```nohup sh connect-distributed ../etc/kafka/connect-distributed-1.properties
&```
When I try to create a connector using POST call, I start seeing message as
```WARN [Producer clientId=connector-producer-MM9-0] Bootstrap broker
abc-broker:9093 (id: -1 rack: null) disconnected
(org.apache.kafka.clients.NetworkClient:1037)```
json content for POST call is as below
```
{{ "name": "MM9",
"config":
{ "connector.class": "org.apache.kafka.connect.mirror.MirrorSourceConnector",
"tasks.max": 3,
"topics": "messaging_ops_mm8",
"errors.log.enable": true,
"errors.log.include.messages": true,
"key.converter": "org.apache.kafka.connect.converters.ByteArrayConverter",
"value.converter": "org.apache.kafka.connect.converters.ByteArrayConverter",
"clusters": "xyz-broker, abc-broker",
"source.cluster.alias": "xyz-broker",
"target.cluster.alias": "abc-broker",
"source.cluster.bootstrap.servers": "xyz-broker:9093",
"source.cluster.security.protocol": "SASL_SSL",
"source.cluster.sasl.mechanism": "PLAIN",
"source.cluster.sasl.jaas.config":
"org.apache.kafka.common.security.plain.PlainLoginModule required
username='superuser' password='superuser_password';",
"source.cluster.ssl.truststore.password" : "truststore_password",
"source.cluster.ssl.truststore.location" :
"/opt/projects/confluent/wildcard.kafka.iggroup.local.jks",
"source.cluster.ssl.keystore.password" : "keystore_password",
"source.cluster.ssl.keystore.location" :
"/opt/projects/confluent/ssl/wildcard.kafka.iggroup.local.jks",
"target.cluster.bootstrap.servers": "abc-broker:9093",
"target.cluster.security.protocol": "SASL_SSL",
"target.cluster.sasl.mechanism": "PLAIN",
"target.cluster.sasl.jaas.config":
"org.apache.kafka.common.security.plain.PlainLoginModule required
username='superuser' password='superuser_password';",
"target.cluster.ssl.truststore.password" : "truststore_password",
"target.cluster.ssl.truststore.location" :
"/opt/projects/confluent/ssl/wildcard.kafka.iggroup.local.jks",
"target.cluster.ssl.keystore.password" : "keystore_password",
"target.cluster.ssl.keystore.location" :
"/opt/projects/confluent/ssl/wildcard.kafka.iggroup.local.jks"
}
}
```
Note the bootstrap-server added in connect-distributed.properties file and the
target cluster bootstrap-server are same(abc-broker).
I have added all the SASL credentials, did a swap of source and destination
cluster in the json but I continue to get this broker disconnected WARNING.
We are on kafka version 0.11.0.3.
What is it that we are missing?
> MirrorMaker 2.0 (KIP-382)
> -
>
> Key: KAFKA-7500
> URL: https://issues.apache.org/jira/browse/KAFKA-7500
> Project: Kafka
> Issue Type: New Feature
> Components: KafkaConnect, mirrormaker
>Affects Versions: 2.4.0
>Reporter: Ryanne Dolan
>Assignee: Ryanne Dolan
>Priority: Major
> Labels: pull-request-available, ready-to-commit
> Fix For: 2.4.0
>
> Attachments: Active-Active XDCR setup.png
>
>
> Implement a drop-in replacement for MirrorMaker leveraging the Connect
> framework.
> [https://cwiki.apache.org/confluence/display/KAFKA/KIP-382%3A+MirrorMaker+2.0]
> [https://github.com/apache/kafka/pull/6295]
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Comment Edited] (KAFKA-7500) MirrorMaker 2.0 (KIP-382)
[
https://issues.apache.org/jira/browse/KAFKA-7500?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17224651#comment-17224651
]
Sarita edited comment on KAFKA-7500 at 11/2/20, 1:07 PM:
-
Hi [~ryannedolan]
We are trying to set up MM2 with connection distributed setup.
Worker configs are as follows
```
bootstrap.servers=abc-broker:9093
security.protocol=SASL_SSL
sasl.mechanism=PLAIN
sasl.jaas.config=org.apache.kafka.common.security.plain.PlainLoginModule
required username="superuser" password="superuser_password";
group.id=connect-tails
key.converter=org.apache.kafka.connect.json.JsonConverter
value.converter=org.apache.kafka.connect.json.JsonConverter
key.converter.schemas.enable=false
value.converter.schemas.enable=false
internal.key.converter=org.apache.kafka.connect.json.JsonConverter
internal.value.converter=org.apache.kafka.connect.json.JsonConverter
internal.key.converter.schemas.enable=false
internal.value.converter.schemas.enable=false
offset.storage.topic=connect-offsets-test
config.storage.topic=connect-configs-test
status.storage.topic=connect-status-test
offset.flush.interval.ms=30
producer.buffer.memory=1234
producer.ssl.truststore.password=truststore_password
producer.ssl.truststore.location=/opt/projects/confluent/wildcard.kafka.iggroup.local.jks
producer.ssl.keystore.password=keystore_password
producer.ssl.keystore.location=/opt/projects/confluent/wildcard.kafka.iggroup.local.jks
```
When i run below command, I can see worker connector created:
```nohup sh connect-distributed ../etc/kafka/connect-distributed-1.properties
&```
When I try to create a connector using POST call, I start seeing message as
```WARN [Producer clientId=connector-producer-MM9-0] Bootstrap broker
abc-broker:9093 (id: -1 rack: null) disconnected
(org.apache.kafka.clients.NetworkClient:1037)```
json content for POST call is as below
```
{{ "name": "MM9",
"config":
{
"connector.class": "org.apache.kafka.connect.mirror.MirrorSourceConnector",
"tasks.max": 3,
"topics": "messaging_ops_mm8",
"errors.log.enable": true,
"errors.log.include.messages": true,
"key.converter": "org.apache.kafka.connect.converters.ByteArrayConverter",
"value.converter": "org.apache.kafka.connect.converters.ByteArrayConverter",
"clusters": "xyz-broker, abc-broker",
"source.cluster.alias": "xyz-broker",
"target.cluster.alias": "abc-broker",
"source.cluster.bootstrap.servers": "xyz-broker:9093",
"source.cluster.security.protocol": "SASL_SSL",
"source.cluster.sasl.mechanism": "PLAIN",
"source.cluster.sasl.jaas.config":
"org.apache.kafka.common.security.plain.PlainLoginModule required
username='superuser' password='superuser_password';",
"source.cluster.ssl.truststore.password" : "truststore_password",
"source.cluster.ssl.truststore.location" :
"/opt/projects/confluent/wildcard.kafka.iggroup.local.jks",
"source.cluster.ssl.keystore.password" : "keystore_password",
"source.cluster.ssl.keystore.location" :
"/opt/projects/confluent/ssl/wildcard.kafka.iggroup.local.jks",
"target.cluster.bootstrap.servers": "abc-broker:9093",
"target.cluster.security.protocol": "SASL_SSL",
"target.cluster.sasl.mechanism": "PLAIN",
"target.cluster.sasl.jaas.config":
"org.apache.kafka.common.security.plain.PlainLoginModule required
username='superuser' password='superuser_password';",
"target.cluster.ssl.truststore.password" : "truststore_password",
"target.cluster.ssl.truststore.location" :
"/opt/projects/confluent/ssl/wildcard.kafka.iggroup.local.jks",
"target.cluster.ssl.keystore.password" : "keystore_password",
"target.cluster.ssl.keystore.location" :
"/opt/projects/confluent/ssl/wildcard.kafka.iggroup.local.jks" }
}
```
Note the bootstrap-server added in connect-distributed.properties file and the
target cluster bootstrap-server are same(abc-broker).
I have added all the SASL credentials, did a swap of source and destination
cluster in the json but I continue to get this broker disconnected WARNING.
We are on kafka version 0.11.0.3.
What is it that we are missing?
was (Author: saritago):
Hi [~ryannedolan]
We are trying to set up MM2 with connection distributed setup.
Worker configs are as follows
```
bootstrap.servers=abc-broker:9093
security.protocol=SASL_SSL
sasl.mechanism=PLAIN
sasl.jaas.config=org.apache.kafka.common.security.plain.PlainLoginModule
required username="superuser" password="superuser_password";
group.id=connect-tails
key.converter=org.apache.kafka.connect.json.JsonConverter
value.converter=org.apache.kafka.connect.json.JsonConverter
key.converter.schemas.enable=false
value.converter.schemas.enable=false
internal.key.converter=org.apache.kafka.connect.json.JsonConverter
internal.value.converter=org.apache.kafka.connect.json.JsonConverter
internal.key.converter.schemas.enable=false
internal.value.converter.schema
[jira] [Comment Edited] (KAFKA-7500) MirrorMaker 2.0 (KIP-382)
[
https://issues.apache.org/jira/browse/KAFKA-7500?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17224651#comment-17224651
]
Sarita edited comment on KAFKA-7500 at 11/2/20, 1:07 PM:
-
Hi [~ryannedolan]
We are trying to set up MM2 with connection distributed setup.
Worker configs are as follows
```
bootstrap.servers=abc-broker:9093
security.protocol=SASL_SSL
sasl.mechanism=PLAIN
sasl.jaas.config=org.apache.kafka.common.security.plain.PlainLoginModule
required username="superuser" password="superuser_password";
group.id=connect-tails
key.converter=org.apache.kafka.connect.json.JsonConverter
value.converter=org.apache.kafka.connect.json.JsonConverter
key.converter.schemas.enable=false
value.converter.schemas.enable=false
internal.key.converter=org.apache.kafka.connect.json.JsonConverter
internal.value.converter=org.apache.kafka.connect.json.JsonConverter
internal.key.converter.schemas.enable=false
internal.value.converter.schemas.enable=false
offset.storage.topic=connect-offsets-test
config.storage.topic=connect-configs-test
status.storage.topic=connect-status-test
offset.flush.interval.ms=30
producer.buffer.memory=1234
producer.ssl.truststore.password=truststore_password
producer.ssl.truststore.location=/opt/projects/confluent/wildcard.kafka.iggroup.local.jks
producer.ssl.keystore.password=keystore_password
producer.ssl.keystore.location=/opt/projects/confluent/wildcard.kafka.iggroup.local.jks
```
When i run below command, I can see worker connector created:
```nohup sh connect-distributed ../etc/kafka/connect-distributed-1.properties
&```
When I try to create a connector using POST call, I start seeing message as
```WARN [Producer clientId=connector-producer-MM9-0] Bootstrap broker
abc-broker:9093 (id: -1 rack: null) disconnected
(org.apache.kafka.clients.NetworkClient:1037)```
json content for POST call is as below
```
{{ "name": "MM9",
"config":
{
"connector.class": "org.apache.kafka.connect.mirror.MirrorSourceConnector",
"tasks.max": 3,
"topics": "messaging_ops_mm8", "errors.log.enable": true,
"errors.log.include.messages": true, "key.converter":
"org.apache.kafka.connect.converters.ByteArrayConverter", "value.converter":
"org.apache.kafka.connect.converters.ByteArrayConverter", "clusters":
"xyz-broker, abc-broker", "source.cluster.alias": "xyz-broker",
"target.cluster.alias": "abc-broker", "source.cluster.bootstrap.servers":
"xyz-broker:9093", "source.cluster.security.protocol": "SASL_SSL",
"source.cluster.sasl.mechanism": "PLAIN", "source.cluster.sasl.jaas.config":
"org.apache.kafka.common.security.plain.PlainLoginModule required
username='superuser' password='superuser_password';",
"source.cluster.ssl.truststore.password" : "truststore_password",
"source.cluster.ssl.truststore.location" :
"/opt/projects/confluent/wildcard.kafka.iggroup.local.jks",
"source.cluster.ssl.keystore.password" : "keystore_password",
"source.cluster.ssl.keystore.location" :
"/opt/projects/confluent/ssl/wildcard.kafka.iggroup.local.jks",
"target.cluster.bootstrap.servers": "abc-broker:9093",
"target.cluster.security.protocol": "SASL_SSL",
"target.cluster.sasl.mechanism": "PLAIN", "target.cluster.sasl.jaas.config":
"org.apache.kafka.common.security.plain.PlainLoginModule required
username='superuser' password='superuser_password';",
"target.cluster.ssl.truststore.password" : "truststore_password",
"target.cluster.ssl.truststore.location" :
"/opt/projects/confluent/ssl/wildcard.kafka.iggroup.local.jks",
"target.cluster.ssl.keystore.password" : "keystore_password",
"target.cluster.ssl.keystore.location" :
"/opt/projects/confluent/ssl/wildcard.kafka.iggroup.local.jks" }
}
```
Note the bootstrap-server added in connect-distributed.properties file and the
target cluster bootstrap-server are same(abc-broker).
I have added all the SASL credentials, did a swap of source and destination
cluster in the json but I continue to get this broker disconnected WARNING.
We are on kafka version 0.11.0.3.
What is it that we are missing?
was (Author: saritago):
Hi [~ryannedolan]
We are trying to set up MM2 with connection distributed setup.
Worker configs are as follows
```
bootstrap.servers=abc-broker:9093
security.protocol=SASL_SSL
sasl.mechanism=PLAIN
sasl.jaas.config=org.apache.kafka.common.security.plain.PlainLoginModule
required username="superuser" password="superuser_password";
group.id=connect-tails
key.converter=org.apache.kafka.connect.json.JsonConverter
value.converter=org.apache.kafka.connect.json.JsonConverter
key.converter.schemas.enable=false
value.converter.schemas.enable=false
internal.key.converter=org.apache.kafka.connect.json.JsonConverter
internal.value.converter=org.apache.kafka.connect.json.JsonConverter
internal.key.converter.schemas.enable=false
internal.value.convert
[jira] [Comment Edited] (KAFKA-7500) MirrorMaker 2.0 (KIP-382)
[
https://issues.apache.org/jira/browse/KAFKA-7500?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17224651#comment-17224651
]
Sarita edited comment on KAFKA-7500 at 11/2/20, 1:09 PM:
-
Hi [~ryannedolan]
We are trying to set up MM2 with connection distributed setup.
Worker configs are as follows
```
bootstrap.servers=abc-broker:9093
security.protocol=SASL_SSL
sasl.mechanism=PLAIN
sasl.jaas.config=org.apache.kafka.common.security.plain.PlainLoginModule
required username="superuser" password="superuser_password";
group.id=connect-tails
key.converter=org.apache.kafka.connect.json.JsonConverter
value.converter=org.apache.kafka.connect.json.JsonConverter
key.converter.schemas.enable=false
value.converter.schemas.enable=false
internal.key.converter=org.apache.kafka.connect.json.JsonConverter
internal.value.converter=org.apache.kafka.connect.json.JsonConverter
internal.key.converter.schemas.enable=false
internal.value.converter.schemas.enable=false
offset.storage.topic=connect-offsets-test
config.storage.topic=connect-configs-test
status.storage.topic=connect-status-test
offset.flush.interval.ms=30
producer.buffer.memory=1234
producer.ssl.truststore.password=truststore_password
producer.ssl.truststore.location=/opt/projects/confluent/wildcard.kafka.iggroup.local.jks
producer.ssl.keystore.password=keystore_password
producer.ssl.keystore.location=/opt/projects/confluent/wildcard.kafka.iggroup.local.jks
```
When i run below command, I can see worker connector created:
```nohup sh connect-distributed ../etc/kafka/connect-distributed-1.properties
&```
When I try to create a connector using POST call, I start seeing message as
```WARN [Producer clientId=connector-producer-MM9-0] Bootstrap broker
abc-broker:9093 (id: -1 rack: null) disconnected
(org.apache.kafka.clients.NetworkClient:1037)```
json content for POST call is as below
```
{{ "name": "MM9",
"config":
{ "connector.class": "org.apache.kafka.connect.mirror.MirrorSourceConnector",
"tasks.max": 3,
"topics": "messaging_ops_mm8",
"errors.log.enable": true,
"errors.log.include.messages": true,
"key.converter": "org.apache.kafka.connect.converters.ByteArrayConverter",
"value.converter": "org.apache.kafka.connect.converters.ByteArrayConverter",
"clusters": "xyz-broker, abc-broker",
"source.cluster.alias": "xyz-broker",
"target.cluster.alias": "abc-broker",
"source.cluster.bootstrap.servers": "xyz-broker:9093",
"source.cluster.security.protocol": "SASL_SSL",
"source.cluster.sasl.mechanism": "PLAIN",
"source.cluster.sasl.jaas.config":
"org.apache.kafka.common.security.plain.PlainLoginModule required
username='superuser' password='superuser_password';",
"source.cluster.ssl.truststore.password" : "truststore_password",
"source.cluster.ssl.truststore.location" :
"/opt/projects/confluent/wildcard.kafka.iggroup.local.jks",
"source.cluster.ssl.keystore.password" : "keystore_password",
"source.cluster.ssl.keystore.location" :
"/opt/projects/confluent/ssl/wildcard.kafka.iggroup.local.jks",
"target.cluster.bootstrap.servers": "abc-broker:9093",
"target.cluster.security.protocol": "SASL_SSL",
"target.cluster.sasl.mechanism": "PLAIN",
"target.cluster.sasl.jaas.config":
"org.apache.kafka.common.security.plain.PlainLoginModule required
username='superuser' password='superuser_password';",
"target.cluster.ssl.truststore.password" : "truststore_password",
"target.cluster.ssl.truststore.location" :
"/opt/projects/confluent/ssl/wildcard.kafka.iggroup.local.jks",
"target.cluster.ssl.keystore.password" : "keystore_password",
"target.cluster.ssl.keystore.location" :
"/opt/projects/confluent/ssl/wildcard.kafka.iggroup.local.jks"
}
}
```
Note the bootstrap-server added in connect-distributed.properties file and the
target cluster bootstrap-server are same(abc-broker).
I have added all the SASL credentials, did a swap of source and destination
cluster in the json but I continue to get this broker disconnected WARNING.
We are on kafka version 0.11.0.3.
What is it that we are missing?
was (Author: saritago):
Hi [~ryannedolan]
We are trying to set up MM2 with connection distributed setup.
Worker configs are as follows
```
bootstrap.servers=abc-broker:9093
security.protocol=SASL_SSL
sasl.mechanism=PLAIN
sasl.jaas.config=org.apache.kafka.common.security.plain.PlainLoginModule
required username="superuser" password="superuser_password";
group.id=connect-tails
key.converter=org.apache.kafka.connect.json.JsonConverter
value.converter=org.apache.kafka.connect.json.JsonConverter
key.converter.schemas.enable=false
value.converter.schemas.enable=false
internal.key.converter=org.apache.kafka.connect.json.JsonConverter
internal.value.converter=org.apache.kafka.connect.json.JsonConverter
internal.key.converter.schemas.enable=false
inter
[jira] [Commented] (KAFKA-10666) Kafka doesn't use keystore / key / truststore passwords for named SSL connections
[ https://issues.apache.org/jira/browse/KAFKA-10666?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17224720#comment-17224720 ] Jason commented on KAFKA-10666: --- Is the intended functionality that only one keystore, truststore, or key password are valid per broker node? > Kafka doesn't use keystore / key / truststore passwords for named SSL > connections > - > > Key: KAFKA-10666 > URL: https://issues.apache.org/jira/browse/KAFKA-10666 > Project: Kafka > Issue Type: Bug > Components: admin >Affects Versions: 2.5.0, 2.6.0 > Environment: kafka in an openjdk-11 docker container, the client java > application is in an alpine container. zookeeper in a separate container. >Reporter: Jason >Priority: Minor > > When configuring named listener SSL connections with ssl key and keystore > with passwords including listener.name.ourname.ssl.key.password, > listener.name.ourname.ssl.keystore.password, and > listener.name.ourname.ssl.truststore.password via via the AdminClient the > settings are not used and the setting is not accepted if the default > ssl.key.password or ssl.keystore.password are not set. We configure all > keystore and truststore values for the named listener in a single batch using > incrementalAlterConfigs. Additionally, when ssl.keystore.password is set to > the value of our keystore password the keystore is loaded for SSL > communication without issue, however if ssl.keystore.password is incorrect > and listener.name.ourname.keystore.password is correct, we are unable to load > the keystore with bad password errors. It appears that only the default > ssl.xxx.password settings are used. This setting is immutable as when we > attempt to set it we get an error indicating that the listener.name. setting > can be set. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (KAFKA-10629) TopologyTestDriver should not require a Properties arg
[ https://issues.apache.org/jira/browse/KAFKA-10629?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17224839#comment-17224839 ] Matthias J. Sax commented on KAFKA-10629: - Absolutely [~rohitdeshaws] – note, that this ticket requires a KIP: [https://cwiki.apache.org/confluence/display/KAFKA/Kafka+Improvement+Proposals] – Let us know if you have any question about the KIP process. > TopologyTestDriver should not require a Properties arg > -- > > Key: KAFKA-10629 > URL: https://issues.apache.org/jira/browse/KAFKA-10629 > Project: Kafka > Issue Type: Task > Components: streams, streams-test-utils >Reporter: John Roesler >Assignee: Rohit Deshpande >Priority: Minor > Labels: needs-kip, newbie > > As of [https://github.com/apache/kafka/pull/9477,] many TopologyTestDriver > usages will have no configurations at all to specify, so we should provide a > constructor that doesn't take a Properties argument. Right now, such > configuration-free usages have to provide an empty Properties object. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (KAFKA-4628) Support KTable/GlobalKTable Joins
[ https://issues.apache.org/jira/browse/KAFKA-4628?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17224844#comment-17224844 ] Matthias J. Sax commented on KAFKA-4628: There was a discussion [https://lists.apache.org/list.html?d...@kafka.apache.org:2018-6:kip-314] But eventually, the discussion died and the KIP is inactive for a long time. > Support KTable/GlobalKTable Joins > - > > Key: KAFKA-4628 > URL: https://issues.apache.org/jira/browse/KAFKA-4628 > Project: Kafka > Issue Type: Improvement > Components: streams >Affects Versions: 0.10.2.0 >Reporter: Damian Guy >Priority: Major > Labels: needs-kip > > In KIP-99 we have added support for GlobalKTables, however we don't currently > support KTable/GlobalKTable joins as they require materializing a state store > for the join. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (KAFKA-10669) ListOffsetRequest: make CurrentLeaderEpoch field ignorable and set MaxNumOffsets field to 1
[ https://issues.apache.org/jira/browse/KAFKA-10669?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Manikumar resolved KAFKA-10669. --- Resolution: Fixed Issue resolved by pull request 9540 [https://github.com/apache/kafka/pull/9540] > ListOffsetRequest: make CurrentLeaderEpoch field ignorable and set > MaxNumOffsets field to 1 > --- > > Key: KAFKA-10669 > URL: https://issues.apache.org/jira/browse/KAFKA-10669 > Project: Kafka > Issue Type: Task >Affects Versions: 2.7.0 >Reporter: Manikumar >Priority: Blocker > Fix For: 2.7.0 > > > Couple of failures observed after KAFKA-9627: Replace ListOffset > request/response with automated protocol > ([https://github.com/apache/kafka/pull/8295]) > 1. Latest consumer fails to consume from 0.10.0.1 brokers. Below system tests > are failing > > kafkatest.tests.client.client_compatibility_features_test.ClientCompatibilityFeaturesTest > > kafkatest.tests.client.client_compatibility_produce_consume_test.ClientCompatibilityProduceConsumeTest > 2. In some scenarios, latest consumer fails with below error when connecting > to a Kafka cluster which consists of newer and older (<=2.0) Kafka brokers > org.apache.kafka.common.errors.UnsupportedVersionException: Attempted to > write a non-default currentLeaderEpoch at version 3 -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Assigned] (KAFKA-10669) ListOffsetRequest: make CurrentLeaderEpoch field ignorable and set MaxNumOffsets field to 1
[ https://issues.apache.org/jira/browse/KAFKA-10669?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Manikumar reassigned KAFKA-10669: - Assignee: Manikumar > ListOffsetRequest: make CurrentLeaderEpoch field ignorable and set > MaxNumOffsets field to 1 > --- > > Key: KAFKA-10669 > URL: https://issues.apache.org/jira/browse/KAFKA-10669 > Project: Kafka > Issue Type: Task >Affects Versions: 2.7.0 >Reporter: Manikumar >Assignee: Manikumar >Priority: Blocker > Fix For: 2.7.0 > > > Couple of failures observed after KAFKA-9627: Replace ListOffset > request/response with automated protocol > ([https://github.com/apache/kafka/pull/8295]) > 1. Latest consumer fails to consume from 0.10.0.1 brokers. Below system tests > are failing > > kafkatest.tests.client.client_compatibility_features_test.ClientCompatibilityFeaturesTest > > kafkatest.tests.client.client_compatibility_produce_consume_test.ClientCompatibilityProduceConsumeTest > 2. In some scenarios, latest consumer fails with below error when connecting > to a Kafka cluster which consists of newer and older (<=2.0) Kafka brokers > org.apache.kafka.common.errors.UnsupportedVersionException: Attempted to > write a non-default currentLeaderEpoch at version 3 -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (KAFKA-10673) ConnectionQuotas should cache interbroker listener name
David Mao created KAFKA-10673: - Summary: ConnectionQuotas should cache interbroker listener name Key: KAFKA-10673 URL: https://issues.apache.org/jira/browse/KAFKA-10673 Project: Kafka Issue Type: Improvement Components: core Affects Versions: 2.7.0 Reporter: David Mao Assignee: David Mao Fix For: 2.8.0 ConnectionQuotas.protectedListener calls `config.interBrokerListenerName`. This is a surprisingly expensive call that creates a copy of all properties set on the config. Given that this method is called multiple times per connection created, this is not really ideal. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (KAFKA-10674) Brokers should know the active controller ApiVersion after enabling KIP-590 forwarding
Boyang Chen created KAFKA-10674: --- Summary: Brokers should know the active controller ApiVersion after enabling KIP-590 forwarding Key: KAFKA-10674 URL: https://issues.apache.org/jira/browse/KAFKA-10674 Project: Kafka Issue Type: Improvement Components: clients, core Reporter: Boyang Chen Admin clients send ApiVersions to the broker upon the first connection establishes. The tricky thing after forwarding is enabled is that for forwardable APIs, admin client needs to know a commonly-agreed rang of ApiVersions among handling broker, active controller and itself. Right now the inter-broker APIs are guaranteed by IBP constraints, but not for forwardable APIs. A compromised solution would be to put all forwardable APIs under IBP, which is brittle and hard to maintain consistency. Instead, any broker connecting to the active controller should send an ApiVersion request from beginning, so it is easy to compute that information and send back to the admin clients upon ApiVersion request from admin. Any rolling of the active controller will trigger reconnection between broker and controller, which guarantees a refreshed ApiVersions between the two. This approach avoids the tight bond with IBP and broker could just close the connection between admin client to trigger retry logic and refreshing of the ApiVersions. Since this failure should be rare, two round-trips and timeout delays are well compensated by the less engineering work. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (KAFKA-10614) Group coordinator onElection/onResignation should guard against leader epoch
[
https://issues.apache.org/jira/browse/KAFKA-10614?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17224992#comment-17224992
]
Bruno Cadonna commented on KAFKA-10614:
---
FYI: This bug makes system test
{{StreamsBrokerBounceTest.test_broker_type_bounce}} fail once in a while when
the group coordinator (i.e., leader of partition 2 of the {{__consumer_offset}}
topic) is bounced. Unfortunately, I haven't been able to reproduce the failure
locally.
> Group coordinator onElection/onResignation should guard against leader epoch
>
>
> Key: KAFKA-10614
> URL: https://issues.apache.org/jira/browse/KAFKA-10614
> Project: Kafka
> Issue Type: Bug
> Components: core
>Reporter: Guozhang Wang
>Assignee: Tom Bentley
>Priority: Major
>
> When there are a sequence of LeaderAndISR or StopReplica requests sent from
> different controllers causing the group coordinator to elect / resign, we may
> re-order the events due to race condition. For example:
> 1) First LeaderAndISR request received from old controller to resign as the
> group coordinator.
> 2) Second LeaderAndISR request received from new controller to elect as the
> group coordinator.
> 3) Although threads handling the 1/2) requests are synchronized on the
> replica manager, their callback {{onLeadershipChange}} would trigger
> {{onElection/onResignation}} which would schedule the loading / unloading on
> background threads, and are not synchronized.
> 4) As a result, the {{onElection}} maybe triggered by the thread first, and
> then {{onResignation}}. As a result, the coordinator would not recognize it
> self as the coordinator and hence would respond any coordinator request with
> {{NOT_COORDINATOR}}.
> Here are two proposals on top of my head:
> 1) Let the scheduled load / unload function to keep the passed in leader
> epoch, and also materialize the epoch in memory. Then when execute the
> unloading check against the leader epoch.
> 2) This may be a bit simpler: using a single background thread working on a
> FIFO queue of loading / unloading jobs, since the caller are actually
> synchronized on replica manager and order preserved, the enqueued loading /
> unloading job would be correctly ordered as well. In that case we would avoid
> the reordering.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Created] (KAFKA-10675) Error message from ConnectSchema.validateValue() should include the name of the schema.
Alexander Iskuskov created KAFKA-10675:
--
Summary: Error message from ConnectSchema.validateValue() should
include the name of the schema.
Key: KAFKA-10675
URL: https://issues.apache.org/jira/browse/KAFKA-10675
Project: Kafka
Issue Type: Improvement
Components: KafkaConnect
Reporter: Alexander Iskuskov
The following error message
{code:java}
org.apache.kafka.connect.errors.DataException: Invalid Java object for schema
type INT64: class java.lang.Long for field: "moderate_time"
{code}
can be confusing because {{java.lang.Long}} is acceptable type for schema
{{INT64}}. In fact, in this case {{org.apache.kafka.connect.data.Timestamp}} is
used but this info is not logged.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (KAFKA-10675) Error message from ConnectSchema.validateValue() should include the name of the schema.
[
https://issues.apache.org/jira/browse/KAFKA-10675?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alexander Iskuskov updated KAFKA-10675:
---
External issue URL: https://github.com/apache/kafka/pull/9541
> Error message from ConnectSchema.validateValue() should include the name of
> the schema.
> ---
>
> Key: KAFKA-10675
> URL: https://issues.apache.org/jira/browse/KAFKA-10675
> Project: Kafka
> Issue Type: Improvement
> Components: KafkaConnect
>Reporter: Alexander Iskuskov
>Priority: Minor
>
> The following error message
> {code:java}
> org.apache.kafka.connect.errors.DataException: Invalid Java object for schema
> type INT64: class java.lang.Long for field: "moderate_time"
> {code}
> can be confusing because {{java.lang.Long}} is acceptable type for schema
> {{INT64}}. In fact, in this case {{org.apache.kafka.connect.data.Timestamp}}
> is used but this info is not logged.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Resolved] (KAFKA-10632) Raft client should push all committed data to listeners
[ https://issues.apache.org/jira/browse/KAFKA-10632?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jason Gustafson resolved KAFKA-10632. - Resolution: Fixed > Raft client should push all committed data to listeners > --- > > Key: KAFKA-10632 > URL: https://issues.apache.org/jira/browse/KAFKA-10632 > Project: Kafka > Issue Type: Sub-task >Reporter: Jason Gustafson >Assignee: Jason Gustafson >Priority: Major > > We would like to move to a push model for sending committed data to the state > machine. This simplifies the state machine a bit since it does not need to > track its own position and poll for new data. It also allows the raft layer > to ensure that all committed data up to the state of a leader epoch has been > sent before allowing the state machine to begin sending writes. Finally, it > allows us to take advantage of optimizations. For example, we can save the > need to re-read writes that have been sent to the leader; instead, we can > retain the data in memory and push it to the state machine after it becomes > committed. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (KAFKA-10676) Decide whether Raft listener callback errors are fatal
Jason Gustafson created KAFKA-10676: --- Summary: Decide whether Raft listener callback errors are fatal Key: KAFKA-10676 URL: https://issues.apache.org/jira/browse/KAFKA-10676 Project: Kafka Issue Type: Sub-task Reporter: Jason Gustafson If a `RaftClient.Listener` callback fails, we need to decide how to handle it. The current code assumes that these errors are fatal and exceptions will get propagated. This might be what we want long term. With KIP-631, there will be one listener for the broker and one listener for the controller. If one of them fails, probably we should shutdown the server rather than remaining in a half-fenced state. However, we should reconsider this once we get closer to integration. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (KAFKA-10677) Complete fetches in purgatory immediately after raft leader resigns
Jason Gustafson created KAFKA-10677: --- Summary: Complete fetches in purgatory immediately after raft leader resigns Key: KAFKA-10677 URL: https://issues.apache.org/jira/browse/KAFKA-10677 Project: Kafka Issue Type: Sub-task Reporter: Jason Gustafson The current logic does not complete fetches in purgatory immediately after the leader has resigned. The idea was that there was no point in doing so until the election had completed because clients would just have to retry. However, the fetches in purgatory might correspond to requests from other voters, so the concern is that this might delay a leader election. For example, the voter might be trying to send a Vote request on the same socket that is blocking on a pending Fetch. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Comment Edited] (KAFKA-10343) Add IBP based ApiVersion constraint tests
[ https://issues.apache.org/jira/browse/KAFKA-10343?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17225029#comment-17225029 ] Boyang Chen edited comment on KAFKA-10343 at 11/3/20, 12:32 AM: We are planning to work on a more long-term fix for this issue, see https://issues.apache.org/jira/browse/KAFKA-10674 was (Author: bchen225242): We are planning to work on a more long-term fix for this issue, see > Add IBP based ApiVersion constraint tests > - > > Key: KAFKA-10343 > URL: https://issues.apache.org/jira/browse/KAFKA-10343 > Project: Kafka > Issue Type: Sub-task >Reporter: Boyang Chen >Assignee: Boyang Chen >Priority: Major > Fix For: 2.8.0 > > > We need to add ApiVersion constraints test based on IBP to remind future > developer bump it when new RPC version is developed. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (KAFKA-10343) Add IBP based ApiVersion constraint tests
[ https://issues.apache.org/jira/browse/KAFKA-10343?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Boyang Chen resolved KAFKA-10343. - Resolution: Won't Fix We are planning to work on a more long-term fix for this issue, see > Add IBP based ApiVersion constraint tests > - > > Key: KAFKA-10343 > URL: https://issues.apache.org/jira/browse/KAFKA-10343 > Project: Kafka > Issue Type: Sub-task >Reporter: Boyang Chen >Assignee: Boyang Chen >Priority: Major > Fix For: 2.8.0 > > > We need to add ApiVersion constraints test based on IBP to remind future > developer bump it when new RPC version is developed. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (KAFKA-10342) Redirect remaining RPCs to the controller
[ https://issues.apache.org/jira/browse/KAFKA-10342?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Boyang Chen updated KAFKA-10342: Summary: Redirect remaining RPCs to the controller (was: Redirect Create/DeleteAcls to the controller) > Redirect remaining RPCs to the controller > - > > Key: KAFKA-10342 > URL: https://issues.apache.org/jira/browse/KAFKA-10342 > Project: Kafka > Issue Type: Sub-task >Reporter: Boyang Chen >Priority: Major > > In the bridge release broker,Create/DeleteAcls should be redirected to the > active controller. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Assigned] (KAFKA-10342) Migrate remaining RPCs to forward to the controller
[ https://issues.apache.org/jira/browse/KAFKA-10342?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Boyang Chen reassigned KAFKA-10342: --- Assignee: Boyang Chen > Migrate remaining RPCs to forward to the controller > --- > > Key: KAFKA-10342 > URL: https://issues.apache.org/jira/browse/KAFKA-10342 > Project: Kafka > Issue Type: Sub-task >Reporter: Boyang Chen >Assignee: Boyang Chen >Priority: Major > > This ticket tracks the progress along migrating the rest of RPCs to the > controller access only through forwarding. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (KAFKA-10342) Migrate remaining RPCs to forward to the controller
[ https://issues.apache.org/jira/browse/KAFKA-10342?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Boyang Chen updated KAFKA-10342: Summary: Migrate remaining RPCs to forward to the controller (was: Redirect remaining RPCs to the controller) > Migrate remaining RPCs to forward to the controller > --- > > Key: KAFKA-10342 > URL: https://issues.apache.org/jira/browse/KAFKA-10342 > Project: Kafka > Issue Type: Sub-task >Reporter: Boyang Chen >Priority: Major > > This ticket tracks the progress along migrating the rest of RPCs to the > controller access only through forwarding. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (KAFKA-10342) Redirect remaining RPCs to the controller
[ https://issues.apache.org/jira/browse/KAFKA-10342?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Boyang Chen updated KAFKA-10342: Description: This ticket tracks the progress along migrating the rest of RPCs to the controller access only through forwarding. (was: In the bridge release broker,Create/DeleteAcls should be redirected to the active controller.) > Redirect remaining RPCs to the controller > - > > Key: KAFKA-10342 > URL: https://issues.apache.org/jira/browse/KAFKA-10342 > Project: Kafka > Issue Type: Sub-task >Reporter: Boyang Chen >Priority: Major > > This ticket tracks the progress along migrating the rest of RPCs to the > controller access only through forwarding. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (KAFKA-10643) Static membership - repetitive PreparingRebalance with updating metadata for member reason
[
https://issues.apache.org/jira/browse/KAFKA-10643?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17225073#comment-17225073
]
A. Sophie Blee-Goldman commented on KAFKA-10643:
Can you be a little more specific about the actual problem you're experiencing?
Is it just that the group seems to be rebalancing more often than you expected,
or are you concerned about the stuck rebalance and think that the "Preparing to
rebalance group..." log message is related to that? Either way, please upload
the broker and client side logs so we can better understand what's happening
I doubt that you hit KAFKA-9752 since that should be resolved in 2.6.0, unless
you saw that stuck rebalance on an older version. But there could be something
similar that hasn't been found yet. If you have the logs (both client and
broker) from that occurrence, that might help uncover the root cause.
Re: the DisconnectException, I suspect that it's unrelated especially if the
occurrence is not correlated with anything else you're seeing. It might suggest
an unstable network connection that should be looked into, but probably it's
not causing any harm
> Static membership - repetitive PreparingRebalance with updating metadata for
> member reason
> --
>
> Key: KAFKA-10643
> URL: https://issues.apache.org/jira/browse/KAFKA-10643
> Project: Kafka
> Issue Type: Bug
> Components: streams
>Affects Versions: 2.6.0
>Reporter: Eran Levy
>Priority: Major
>
> Kafka streams 2.6.0, brokers version 2.6.0. Kafka nodes are healthy, kafka
> streams app is healthy.
> Configured with static membership.
> Every 10 minutes (I assume cause of topic.metadata.refresh.interval.ms), I
> see the following group coordinator log for different stream consumers:
> INFO [GroupCoordinator 2]: Preparing to rebalance group **--**-stream in
> state PreparingRebalance with old generation 12244 (__consumer_offsets-45)
> (reason: Updating metadata for member
> -stream-11-1-013edd56-ed93-4370-b07c-1c29fbe72c9a)
> (kafka.coordinator.group.GroupCoordinator)
> and right after that the following log:
> INFO [GroupCoordinator 2]: Assignment received from leader for group
> **-**-stream for generation 12246 (kafka.coordinator.group.GroupCoordinator)
>
> Looked a bit on the kafka code and Im not sure that I get why such a thing
> happening - is this line described the situation that happens here re the
> "reason:"?[https://github.com/apache/kafka/blob/7ca299b8c0f2f3256c40b694078e422350c20d19/core/src/main/scala/kafka/coordinator/group/GroupCoordinator.scala#L311]
> I also dont see it happening too often in other kafka streams applications
> that we have.
> The only thing suspicious that I see around every hour that different pods of
> that kafka streams application throw this exception:
> {"timestamp":"2020-10-25T06:44:20.414Z","level":"INFO","thread":"**-**-stream-94561945-4191-4a07-ac1b-07b27e044402-StreamThread-1","logger":"org.apache.kafka.clients.FetchSessionHandler","message":"[Consumer
>
> clientId=**-**-stream-94561945-4191-4a07-ac1b-07b27e044402-StreamThread-1-restore-consumer,
> groupId=null] Error sending fetch request (sessionId=34683236, epoch=2872)
> to node
> 3:","context":"default","exception":"org.apache.kafka.common.errors.DisconnectException:
> null\n"}
> I came across this strange behaviour after stated to investigate a strange
> stuck rebalancing state after one of the members left the group and caused
> the rebalance to stuck - the only thing that I found is that maybe because
> that too often preparing to rebalance states, the app might affected of this
> bug - KAFKA-9752 ?
> I dont understand why it happens, it wasn't before I applied static
> membership to that kafka streams application (since around 2 weeks ago).
> Will be happy if you can help me
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (KAFKA-10629) TopologyTestDriver should not require a Properties arg
[ https://issues.apache.org/jira/browse/KAFKA-10629?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17225189#comment-17225189 ] Rohit Deshpande commented on KAFKA-10629: - Thanks [~mjsax] Created [#KIP-680]|https://cwiki.apache.org/confluence/display/KAFKA/KIP-680%3A+TopologyTestDriver+should+not+require+a+Properties+argument] > TopologyTestDriver should not require a Properties arg > -- > > Key: KAFKA-10629 > URL: https://issues.apache.org/jira/browse/KAFKA-10629 > Project: Kafka > Issue Type: Task > Components: streams, streams-test-utils >Reporter: John Roesler >Assignee: Rohit Deshpande >Priority: Minor > Labels: needs-kip, newbie > > As of [https://github.com/apache/kafka/pull/9477,] many TopologyTestDriver > usages will have no configurations at all to specify, so we should provide a > constructor that doesn't take a Properties argument. Right now, such > configuration-free usages have to provide an empty Properties object. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Comment Edited] (KAFKA-10629) TopologyTestDriver should not require a Properties arg
[ https://issues.apache.org/jira/browse/KAFKA-10629?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17225189#comment-17225189 ] Rohit Deshpande edited comment on KAFKA-10629 at 11/3/20, 7:20 AM: --- Thanks [~mjsax] Created [#KIP-680]] was (Author: rohitdeshaws): Thanks [~mjsax] Created [#KIP-680]|https://cwiki.apache.org/confluence/display/KAFKA/KIP-680%3A+TopologyTestDriver+should+not+require+a+Properties+argument] > TopologyTestDriver should not require a Properties arg > -- > > Key: KAFKA-10629 > URL: https://issues.apache.org/jira/browse/KAFKA-10629 > Project: Kafka > Issue Type: Task > Components: streams, streams-test-utils >Reporter: John Roesler >Assignee: Rohit Deshpande >Priority: Minor > Labels: needs-kip, newbie > > As of [https://github.com/apache/kafka/pull/9477,] many TopologyTestDriver > usages will have no configurations at all to specify, so we should provide a > constructor that doesn't take a Properties argument. Right now, such > configuration-free usages have to provide an empty Properties object. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Comment Edited] (KAFKA-10629) TopologyTestDriver should not require a Properties arg
[ https://issues.apache.org/jira/browse/KAFKA-10629?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17225189#comment-17225189 ] Rohit Deshpande edited comment on KAFKA-10629 at 11/3/20, 7:21 AM: --- Thanks [~mjsax] Created https://cwiki.apache.org/confluence/display/KAFKA/KIP-680%3A+TopologyTestDriver+should+not+require+a+Properties+argument was (Author: rohitdeshaws): Thanks [~mjsax] Created [#KIP-680]] > TopologyTestDriver should not require a Properties arg > -- > > Key: KAFKA-10629 > URL: https://issues.apache.org/jira/browse/KAFKA-10629 > Project: Kafka > Issue Type: Task > Components: streams, streams-test-utils >Reporter: John Roesler >Assignee: Rohit Deshpande >Priority: Minor > Labels: needs-kip, newbie > > As of [https://github.com/apache/kafka/pull/9477,] many TopologyTestDriver > usages will have no configurations at all to specify, so we should provide a > constructor that doesn't take a Properties argument. Right now, such > configuration-free usages have to provide an empty Properties object. -- This message was sent by Atlassian Jira (v8.3.4#803005)
