Is kafka support dynamic ACL rule

Hi As I learned that kafka can use '--resource-pattern-type prefixed' to add rule for prefixed topic. For example an user 'kafkaclient', we could define a rule let the user can access all topics start with that user name, i.e., 'kafkaclient--', such as 'kafkaclient--topic1',

Re: Kafka tests on a remote cluster

Thanks fair points. Probably best if I simplify the question: How does Kafka community run tests besides using mocked local Kafka components? Surely there are tests to confirm different failure scenarios such as losing a broker in a real clustered environment (multi node cluster with Ip, port,

SIGSEGV (0xb) on TransactionCoordinator

Dear all, We got a coredump with the following info last night, on this environment, we enable the transaction. Please kindly advice what would be the problem here. # > # A fatal error has been detected by the Java Runtime Environment: > # > # SIGSEGV (0xb) at pc=0x7f546a857d0d, pid=13288,

Re: Knowing when to grow a Kafka cluster

I really recommend the book "Kafka, the definitive guide" it's really useful for people running clusters, lots of good advice on tuning, metrics etc. Basically, you scale your cluster when you're hitting limits of your most important resources (to Kafka) on the broker nodes - CPU, network or

Re: Kafka tests on a remote cluster

Without looking into how the integration tests work my best guess is within the context they were written to run in, it doesn't make sense to run them against a remote cluster. The "internal" cluster is running the same code, so why require having to coordinate with an external dependency? For

Kafka tests on a remote cluster

Hi, I'm looking to see who has done this before and get some guidance. On frequent basis I like to run basic tests on a remote Kafka cluster while some random chaos/faults are being performed. In other words I like to run chaos engineering tasks (network outage, disk outage, etc) and see how

Knowing when to grow a Kafka cluster

Hi, Many articles exist about running Kafka at scale, but there are fewer resources for learning when to grow your cluster (e.g. adding a new broker or upgrading the computer it's running on). At first, the answer to that seems straightforward - you add a broker if you want to reduce the amount

Re: Cannot read messages using a specific group

Hi all, I set the log level to INFO and here are the logs. I’m not sure why those logs say that the Group coordinator is unavailable or invalid because the brokers are all up (including the ones with the ip specified in the log) and just changing the consumer group makes it work. If anybody had

kafka-acls.sh --list failed when zookeeper SASL/PLAIN authentication is enabled

I have a kakfa/zookeeper(embedded zookeeper) cluster with SASL/PLAIN + ACL enabled. It worked fine with version kafka_2.12-1.0.0. But recently, I need to upgrade to kafka_2.12-2.1.0. Unfortunately, the ACL function cannot work normally. kafka-acls.sh command failed, for example: > # echo

?????? KTable.suppress(Suppressed.untilWindowCloses) does not suppresssome non-final results when the kafka streams process is restarted

I'm not talking about orderliness, but that the same consumer group, the same partition, is consumed by multiple consumers. I use kafka-consumer-groups.sh and org.apache.kafka.clients.admin.AdminClient to validate the results. Because the same consumer group subscribes to a topic, the same

Re: KTable.suppress(Suppressed.untilWindowCloses) does not suppress some non-final results when the kafka streams process is restarted

On 12/21/18 3:16 PM, Peter Levart wrote: I also see some results that are actual non-final window aggregations that precede the final aggregations. These non-final results are never emitted out of order (for example, no such non-final result would ever come after the final result for a

The problem of kafka consumer group

Dear Kafka Team : I encountered some problems in using Kafka2.0.0, In the same consumer group of Kafka cluster, Two consumers consume the same partition, Perhaps the picture in the attachment is more illustrative. Everything works well with Range Assignor, and this problem may