Re: BufferOverflowException on rolling new segment after upgrading Kafka from 1.1.0 to 2.3.1

2019-11-27 Thread Daniyar Kulakhmetov
bumping this up with new update: I've investigated another occurrence of this exception. For analyzes, I used: 1) a memory dump that was taken from the broker 2) kafka log file 3) kafka state-change log 4) log, index and time-index files of a failed segment 5) Kafka source code, version 2.3.1

Re: BufferOverflowException on rolling new segment after upgrading Kafka from 1.1.0 to 2.3.1

2019-11-19 Thread Daniyar Kulakhmetov
Filed JIRA bug: https://issues.apache.org/jira/browse/KAFKA-9213 On Tue, Nov 19, 2019 at 2:58 PM Ismael Juma wrote: > Can you please file a JIRA? > > Ismael > > On Tue, Nov 19, 2019 at 11:52 AM Daniyar Kulakhmetov < > dkulakhme...@liftoff.io> wrote: > > > Hi Kafka users, > > > > We updated

Re: BufferOverflowException on rolling new segment after upgrading Kafka from 1.1.0 to 2.3.1

2019-11-19 Thread Ismael Juma
Can you please file a JIRA? Ismael On Tue, Nov 19, 2019 at 11:52 AM Daniyar Kulakhmetov < dkulakhme...@liftoff.io> wrote: > Hi Kafka users, > > We updated our Kafka cluster from 1.1.0 version to 2.3.1. > Message format and inter-broker protocol versions left the same: > >

Re: BufferOverflowException on rolling new segment after upgrading Kafka from 1.1.0 to 2.3.1

2019-11-19 Thread Daniyar Kulakhmetov
Hi, We followed the upgrade instruction ( https://kafka.apache.org/documentation/#upgrade) up to step 2, and as it is said in step 3 "Once the cluster's behavior and performance has been verified, bump the protocol version by editing" we were verifying cluster's behavior. Thanks, On Tue, Nov

Re: BufferOverflowException on rolling new segment after upgrading Kafka from 1.1.0 to 2.3.1

2019-11-19 Thread M. Manna
Hi, Is there any reason why you haven’t performed the upgrade based on official docs ? Or, is this something you’re planning to do now? Thanks, On Tue, 19 Nov 2019 at 19:52, Daniyar Kulakhmetov wrote: > Hi Kafka users, > > We updated our Kafka cluster from 1.1.0 version to 2.3.1. > Message

BufferOverflowException on rolling new segment after upgrading Kafka from 1.1.0 to 2.3.1

2019-11-19 Thread Daniyar Kulakhmetov
Hi Kafka users, We updated our Kafka cluster from 1.1.0 version to 2.3.1. Message format and inter-broker protocol versions left the same: inter.broker.protocol.version=1.1 log.message.format.version=1.1 After upgrading, we started to get some occasional exceptions: 2019/11/19 05:30:53 INFO