consumer fetch multiple topic partitions committed offset

2018-11-01 Thread hacker win7
Hi, After reviewing the KafkaConsumer source about API of *committed():* I found that old consumer support committed(mutipleTopicPartitions) to return multiple committed offset, while in new consumer, there is only committed(singleTopicPartition) and return only one committed offset. It is a

Re: [VOTE] 2.0.1 RC0

2018-11-01 Thread Jakub Scholz
+1 (non-binding) ... I used the staged binaries and run tests with different clients. On Fri, Oct 26, 2018 at 4:29 AM Manikumar wrote: > Hello Kafka users, developers and client-developers, > > This is the first candidate for release of Apache Kafka 2.0.1. > > This is a bug fix release closing

Re: [VOTE] 2.1.0 RC0

2018-11-01 Thread Jakub Scholz
+1 (non-binding) ... I used the staged binaries and checked it with different clients. On Wed, Oct 24, 2018 at 10:17 AM Dong Lin wrote: > Hello Kafka users, developers and client-developers, > > This is the first candidate for feature release of Apache Kafka 2.1.0. > > This is a major version

Re: [VOTE] 2.0.1 RC0

2018-11-01 Thread Harsha Chintalapani
+1. Ran a 3 node cluster with few simple tests. Thanks, Harsha On Nov 1, 2018, 9:50 AM -0700, Eno Thereska , wrote: > Anything else holding this up? > > Thanks > Eno > > On Thu, Nov 1, 2018 at 10:27 AM Jakub Scholz wrote: > > > +1 (non-binding) ... I used the staged binaries and run tests with >

Re: [VOTE] 2.0.1 RC0

2018-11-01 Thread Eno Thereska
Anything else holding this up? Thanks Eno On Thu, Nov 1, 2018 at 10:27 AM Jakub Scholz wrote: > +1 (non-binding) ... I used the staged binaries and run tests with > different clients. > > On Fri, Oct 26, 2018 at 4:29 AM Manikumar > wrote: > > > Hello Kafka users, developers and

Re: [VOTE] 2.0.1 RC0

2018-11-01 Thread Manikumar
We were waiting for the system test results. There were few failures: KAFKA-7579, KAFKA-7559, KAFKA-7561 they are not blockers for 2.0.1 release. We need more votes from PMC/committers :) Thanks Stanislav! for the system test results. Thanks, Manikumar On Thu, Nov 1, 2018 at 10:20 PM Eno

Re: consumer fetch multiple topic partitions committed offset

2018-11-01 Thread Matthias J. Sax
You need to call `committed()` multiple times. -Matthias On 11/1/18 12:28 AM, hacker win7 wrote: > Hi, > > After reviewing the KafkaConsumer source about API of *committed():* > I found that old consumer support committed(mutipleTopicPartitions) to > return multiple committed offset, while in

Re: Despite of log.retention.hours=168 the log files of messages sente yesterday are not present

2018-11-01 Thread Matthias J. Sax
/tmp/kafka-logs is just a convenient default, but not a reliable folder to store data. /tmp/ might be cleared by the operation system. Note that the quickstart is not designed to give "production ready" configuration etc. It's just to play with the system. You should change the config

Re: Bemchmarks for KTable Joins and Queries

2018-11-01 Thread Matthias J. Sax
I am not aware if benchmarks, but want to point out, that KTables work somewhat different to relational database system. Thus, you might want to evaluate not base on performance, but on the semantics KTable provide. Recall, that Kafka Streams is a stream processing library while a database system

Deduplicating a topic in the face of producer crashes over a time window?

2018-11-01 Thread Andrew Wilcox
Suppose I have a producer which is ingesting a data stream with unique keys from an external service and sending it to a Kafka topic. In my producer I can set enable.idempotence and get exactly-once delivery in the presence of broker crashes. However my producer might crash after it delivers a