Re: Magic v1 Does not Support Record Headers

2019-11-25 Thread Shalom Sagges
Luckily, I was able to find that needle in a haystack. :-D Thanks a lot for your guidance Matthias, it helped me a lot to understand the issue. On Mon, Nov 25, 2019 at 9:01 PM Matthias J. Sax wrote: > Fankly, I am not entirely sure... > > I would _assume_ that you could still change the message

Re:Re: The question of kafka

2019-11-25 Thread 房延鹏
hello, like this. The replicas of partition 4 and 10 not contains their isr. sh /u01/sqcar/kafka_2.11-0.11.0.0/bin/kafka-topics.sh --describe --zookeeper 10.0.7.65:2181,10.0.7.66:2181,10.0.7.67:2181 --topic sqyc_kid_bigdata Topic:sqyc_kid_bigdata PartitionCount:15 ReplicationFactor:2 Configs: To

New topic gets stuck in bad state after Zookeeper error

2019-11-25 Thread James Brown
We have an 8-node Kafka cluster that's been running for about four years and is currently on Kafka 2.3.1. It's supported by a 3-node Zookeeper cluster running ZK 3.5.5. Last week, a topic got stuck in a weird state after it was auto-created: kafka-topics.sh didn't show any leader at all (not the u

Re: Preliminary blog post about the Apache Kafka 2.4.0 release

2019-11-25 Thread Ismael Juma
Manikumar, One thing I had previously missed, should we mention the upgrade to ZK 3.5? Ismael On Thu, Nov 14, 2019 at 10:41 AM Manikumar wrote: > Hi all, > > I've prepared a preliminary blog post about the upcoming Apache Kafka 2.4.0 > release. > Please take a look and let me know if you want

Re: Magic v1 Does not Support Record Headers

2019-11-25 Thread Matthias J. Sax
Fankly, I am not entirely sure... I would _assume_ that you could still change the message format but I would highly recommend to try it out first in a non-production environment first. -Matthias On 11/25/19 4:51 AM, Shalom Sagges wrote: > Thanks a lot Matthias! > > This problematic topic is ac

FW: kafka connect incoming port

2019-11-25 Thread Johnson, Paul
Hello, I am working on setting up a Kafka Connect JMS Source to pull data from an external IBM MQ and push to an internal Kafka topic. We will have a firewall between the external MQ and the internal Connect so I need to do a firewall exception. The network people are asking for the incoming

kafka connect incoming port

2019-11-25 Thread Johnson, Paul
Hello, I am working on setting up a Kafka Connect JMS Source to pull data from an external IBM MQ and push to an internal Kafka topic. We will have a firewall between the external MQ and the internal Connect so I need to do a firewall exception. The network people are asking for the incoming

rename.topics setting missing from MirrorMaker 2 ?

2019-11-25 Thread Alan
Hi, I've just downloaded and built 2.4-rc1 to test the MirrorMaker 2 feature and I cannot disable the name.topics setting as detailed in the KIP https://cwiki.apache.org/confluence/display/KAFKA/KIP-382%3A+MirrorMaker+2.0#KIP-382:MirrorMaker2.0-ConnectorConfigurationProperties I did a q

Re: The question of kafka

2019-11-25 Thread Jonathan Santilli
Hello, do you mean why replicas are not in sync? I think you need to be more specific, please. Cheers! -- Jonathan On Mon, Nov 25, 2019 at 2:54 PM 房延鹏 <13120336...@163.com> wrote: > Hi, > >I want to ask a question. Why replicas not contain the isr. The version > is kafka_2.11-0.11.00. Lo

Re: Transactional Producer

2019-11-25 Thread Jonathan Santilli
Hello Sergi, I think we need to step back a little bit and understand what is what you are trying to achieve, please, will be beneficial to give you an accurate answer. What order can I expect for these published messages? - This depends on different factors, like linger, batch size, buffers, etc

The question of kafka

2019-11-25 Thread 房延鹏
Hi, I want to ask a question. Why replicas not contain the isr. The version is kafka_2.11-0.11.00. Look forward to your reply. Thanks.

Custom auth broken in confluent-5.3/kafka-2.3 ?

2019-11-25 Thread Joris Peeters
Hello, We use custom Kafka authentication and authorisation, in a manner very similar to https://github.com/navikt/kafka-plain-saslserver-2-ad, i.e. by providing an implementation of org.apache.kafka.common.security.auth.AuthenticateCallBackHandler and kafka.security.auth.Authorizer - for plain us

Re: Magic v1 Does not Support Record Headers

2019-11-25 Thread Shalom Sagges
Thanks a lot Matthias! This problematic topic is actually a topic that's been mirrored from an older 0.8 version (using kafka-mirror). I guess it's not possible to upgrade the message format in this case? Thanks again for your help! On Fri, Nov 22, 2019 at 7:32 AM Matthias J. Sax wrote: > It's