electing leader failed and result in 0 latest offset

Hi folks, Recently we run into an odd issue that some partition's latest offset becomes 0. Here's the snapshot of the Kafka Manager. As you can see partition 2 and 3 becomes zero. *Partition* *Latest Offset* *Leader* *Replicas* *In Sync Replicas* *Preferred Leader?* *Under Replicated?* 0

Re: Fallout from upgrading to kafka 0.9.0.0 from 0.8.2.1

:org.apache.kafka.common.errors.NotLeaderForPartitionException: This server is not the leader for that topic-partition. (kafka.server.ReplicaFetcherThread) And after restart that controller, everything works again. On Tue, Mar 22, 2016 at 6:14 PM, Qi Xu <shkir...@gmail.com> wrote: > Hi folks, Rajiv, J

Fallout from upgrading to kafka 0.9.0.0 from 0.8.2.1

Hi folks, Rajiv, Jun, I'd like to bring up this thread again from Rajiv Kurian 3 months ago. Basically we did the same thing as Rajiv did. I upgraded two machines (out of 10) from 0.8.2.1 to 0.9. SO after the upgrade, there will be 2 machines in 0.9 and 8 machines in 0.8.2.1. And initially it all

Help: Producer with SSL did not work after upgrading the kafka 0.8.2 to Kafka 0.9

Hi folks, We just finished the upgrade from 0.8.2 to 0.9 with the instructions in Kafka web site (that set the protocol version as 0.8.2.x in Kafka server 0.9). After the upgrade, we want to try the producer with SSL endpoint, but never worked. Here's the error: ~/kafka_2.11-0.9.0.0$

Re: All brokers are running but some partitions' leader is -1

> 2. I'm not aware of issues with 0.9.0 and SparkStreaming. However, > definitely do your own testing to make sure. > > On Wed, Nov 25, 2015 at 11:25 AM, Qi Xu <shkir...@gmail.com> wrote: > > > Hi Gwen, > > Yes, we're going to upgrade the 0.9.0 version. Regarding the upgr

Re: All brokers are running but some partitions' leader is -1

many many bugs since August. Since we are about to release 0.9.0 > > (with SSL!), maybe wait a day and go with a released and tested version. > > > > On Mon, Nov 23, 2015 at 3:01 PM, Qi Xu <shkir...@gmail.com> wrote: > > > > > Forgot to mention is that the Kafka versi

Re: All brokers are running but some partitions' leader is -1

Loop another guy from our team. On Mon, Nov 23, 2015 at 2:26 PM, Qi Xu <shkir...@gmail.com> wrote: > Hi folks, > We have a 10 node cluster and have several topics. Each topic has about > 256 partitions with 3 replica factor. Now we run into an issue that in some > topic, a few

All brokers are running but some partitions' leader is -1

Hi folks, We have a 10 node cluster and have several topics. Each topic has about 256 partitions with 3 replica factor. Now we run into an issue that in some topic, a few partition (< 10)'s leader is -1 and all of them has only one synced partition. >From the Kafka manager, here's the snapshot:

Re: All brokers are running but some partitions' leader is -1

Forgot to mention is that the Kafka version we're using is from Aug's Trunk branch---which has the SSL support. Thanks again, Qi On Mon, Nov 23, 2015 at 2:29 PM, Qi Xu <shkir...@gmail.com> wrote: > Loop another guy from our team. > > On Mon, Nov 23, 2015 at 2:26 PM, Qi Xu <

Re: Kafka cluster cannot start anymore after unexpected shutdown

,023] ERROR Processor got uncaught exception. (kafka.network.Processor) java.lang.OutOfMemoryError: Java heap space [2015-09-12 01:01:35,676] ERROR Uncaught exception in scheduled task 'kafka-recovery-point-checkpoint' (kafka.utils.KafkaScheduler) And the broker does not appear in zookeeper. Any idea? Thanks, Qi On

Kafka cluster cannot start anymore after unexpected shutdown

Hi, We're running the Trunk version of Kafka (for its SSL feature) and recently I'm trying to enable the kafka manager with it. After enabling that, I find out some machine's Kafka Server is dead. Looking at the server.log, it has the following logs. java.lang.OutOfMemoryError: Java heap space

Huge Kafka Message size ( 386073344 ) in response

Hi all, I'm using the Kafka.Net library for implementing the Kafka Producer. One issue I find out is that sometimes it reads the response from kafka server, which indicates a huge message size 386073344. Apparently something must be wrong. But I'm not sure if it's a special flag that Kafka.net

Re: is SSL support feature ready to use in kafka-truck branch

that helps Ben On 21 Aug 2015, at 07:10, Qi Xu shkir...@gmail.com wrote: Hi folks, I tried to clone the latest version of kafka truck and try to enable the SSL. The server.properties seems not having any security related settings, and it seems there's no other config file relevant to SSL

Re: is SSL support feature ready to use in kafka-truck branch

/consumer pair. You’ll need the latest cut from trunk (from today) to get a console consumer that works. Hope that helps Ben On 21 Aug 2015, at 07:10, Qi Xu shkir...@gmail.com wrote: Hi folks, I tried to clone the latest version of kafka truck and try to enable

is SSL support feature ready to use in kafka-truck branch

Hi folks, I tried to clone the latest version of kafka truck and try to enable the SSL. The server.properties seems not having any security related settings, and it seems there's no other config file relevant to SSL either. So may I know is this feature ready to use now in truck branch? BTW,

Re: request forwarding in Kafka?

ping~ On Tue, Jul 7, 2015 at 6:27 PM, Qi Xu shkir...@gmail.com wrote: Hi Kafka folks, I'm investigating about using Kafka as a data ingress endpoint for our analytic service. In our scenario, Kafka cluster is behind NLB which has a public IP, and the data source service is under another

request forwarding in Kafka?

Hi Kafka folks, I'm investigating about using Kafka as a data ingress endpoint for our analytic service. In our scenario, Kafka cluster is behind NLB which has a public IP, and the data source service is under another public IP. Basically the data source service cannot access Kafka cluster nodes