Re: [DISCUSS] KIP-250 Add Support for Quorum-based Producer Acknowledgment

Hey Colin, Yeah I think you have a good point that controller is currently a scalability bottleneck in Kafka and it will be great if we can re-design it in distributed way using some consensus protocol. I have not gone through the available options here but I imagine this will require a lot of

Re: [VOTE] KIP-232: Detect outdated metadata using leaderEpoch and partitionEpoch

Hey Jun, Sure, I will come up with a KIP this week. I think there is a way to allow partition expansion to arbitrary number without introducing new concepts such as read-only partition or repartition epoch. Thanks, Dong On Wed, Feb 7, 2018 at 5:28 PM, Jun Rao wrote: > Hi,

[jira] [Created] (KAFKA-6541) StackOverflow exceptions in thread 'kafka-coordinator-heartbeat-thread

Anh Le created KAFKA-6541: - Summary: StackOverflow exceptions in thread 'kafka-coordinator-heartbeat-thread Key: KAFKA-6541 URL: https://issues.apache.org/jira/browse/KAFKA-6541 Project: Kafka

Re: Eclipse or Intellij workspace setup

Here's what I did recently with OS X/IntelliJ.  I hadn't quite fleshed out all the instructions to put up an updated Wiki version yet: - Use Homebrew to install gradle/scala@2.11 (use --with-docs option)/sbt - git clone Kafka - Run gradle/gradlew commands as documented in

Eclipse or Intellij workspace setup

Hi Team, I am newbie to Kafka dev forum, I would like to be part of kafka development. I tried to setup Eclipse and Intellij workspace setup by follow below link( https://cwiki.apache.org/confluence/display/KAFKA/Developer+Setup). But, I am not to setup compile successfully, after importing

Build failed in Jenkins: kafka-trunk-jdk9 #379

See Changes: [wangguoz] MINOR: adding system tests for how streams functions with broker faiures [wangguoz] MINOR: Update TupleForwarder comment (#4414) [wangguoz] KAFKA-6405: Fix incorrect comment in

Ability to raise a KIP

Hi there I'd like the ability to raise a KIP. I intend to raise on to suggest adding KTable.selectKey method which differs in behaviour from KStream.selectKey in that it knows when the selected key changes for a given record so can react appropriately. Thanks Andy Bryant

Build failed in Jenkins: kafka-trunk-jdk8 #2398

See Changes: [github] KAFKA-6519; Reduce log level for normal replica fetch errors (#4501) [wangguoz] MINOR: adding system tests for how streams functions with broker faiures [wangguoz] MINOR: Update

Build failed in Jenkins: kafka-1.1-jdk7 #22

See Changes: [jason] KAFKA-6519; Reduce log level for normal replica fetch errors (#4501) -- [...truncated 411.36 KB...]

Build failed in Jenkins: kafka-trunk-jdk9 #378

See Changes: [github] KAFKA-6519; Reduce log level for normal replica fetch errors (#4501) -- [...truncated 1.47 MB...] kafka.tools.ConsoleConsumerTest >

Jenkins build is back to normal : kafka-1.1-jdk7 #21

See

Re: [VOTE] KIP-232: Detect outdated metadata using leaderEpoch and partitionEpoch

Hi, Dong, Thanks for the reply. The general idea that you had for adding partitions is similar to what we had in mind. It would be useful to make this more general, allowing adding an arbitrary number of partitions (instead of just doubling) and potentially removing partitions as well. The

[jira] [Created] (KAFKA-6540) Consumer lag metric is not updated when a partition is paused

Jason Gustafson created KAFKA-6540: -- Summary: Consumer lag metric is not updated when a partition is paused Key: KAFKA-6540 URL: https://issues.apache.org/jira/browse/KAFKA-6540 Project: Kafka

Build failed in Jenkins: kafka-1.1-jdk7 #20

See Changes: [matthias] MINOR: update upgrade notes for Streams API; message format 0.10 [matthias] MINOR: Add attributes `processedKeys` and `processedValues` to --

Re: [DISCUSS] KIP-228 Negative record timestamp support

TL.DR: I'd agree with Colin here. I admit that define -1 == unknown is not a good idea, and if we design the system from scratch we should not do so. But Colin's point is valid, that it has been treated like this in past versions, and I have seen producer client users doing so indeed. So

Re: [DISCUSS] KIP-251: Allow timestamp manipulation in Processor API

I intend to suggest we piggyback this change, just for connivence. Guozhang On Tue, Feb 6, 2018 at 7:25 PM, Matthias J. Sax wrote: > I am open to remove the index API and also open to piggyback this change > on this KIP... Not sure if this was the intention of your

[jira] [Resolved] (KAFKA-4641) Improve test coverage of StreamsThread

[ https://issues.apache.org/jira/browse/KAFKA-4641?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guozhang Wang resolved KAFKA-4641. -- Resolution: Fixed Assignee: Guozhang Wang Fix Version/s: 1.2.0 > Improve test

Build failed in Jenkins: kafka-1.1-jdk7 #19

See Changes: [matthias] KAFKA-6367: StateRestoreListener use actual last restored offset for -- [...truncated 411.82 KB...] kafka.tools.ConsoleConsumerTest >

Bay Area Apache Kafka Meetup - Morning of Feb 20

Kafka users and developers, The next *Bay Area Apache Kafka Meetup* is on the *morning of Feb 20* and is hosted by *Index Developer Conference* at Moscone West in San Francisco . Meetup Info: https://www.meetup.com/KafkaBayArea/events/247433783/ Registration Link: https://ibm.co/2n742Jn

Re: [VOTE] 1.0.1 RC0

Hi Ewen, +1 Building from source and running the quickstart were successful on Ubuntu and Windows 10. Thanks for running the release. --Vahid From: Ewen Cheslack-Postava To: dev@kafka.apache.org, us...@kafka.apache.org, kafka-clie...@googlegroups.com Date:

Re: [DISCUSS] KIP-248 Create New ConfigCommand That Uses The New AdminClient

Hi Viktor, Thanks for the updates. Looks good, just a few minor comments: 1. AdminOperation - could be AlterOperation since it is only applied to 'Alter'? 2. Don't think we need `Unknown` type to process old requests. We can use `Set` as the default for alter requests with version 0.

Problem in use of Kafka connect to import and export data.

Hello Team, I am replicating the same steps mentioned in your quickstart. I have written some message to test.txt file through the below commands mentioned. > echo foo> test.txt > echo bar>> test.txt But when I am running below command to instantiate connectors

Re: [DISCUSS] KIP-248 Create New ConfigCommand That Uses The New AdminClient

Hi Rajini, I think it makes sense absolutely and even we could do it for AlterQuotas as we will have the same problem there. Updated my KIP to reflect these changes: - proposed

[jira] [Resolved] (KAFKA-6532) Delegation token internals should not impact public interfaces

[ https://issues.apache.org/jira/browse/KAFKA-6532?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajini Sivaram resolved KAFKA-6532. --- Resolution: Fixed Fix Version/s: 1.1.0 > Delegation token internals should not impact

Jenkins build is back to normal : kafka-trunk-jdk8 #2395

See

[jira] [Resolved] (KAFKA-6524) kafka mirror can't producer internal topic

[ https://issues.apache.org/jira/browse/KAFKA-6524?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ahmed Madkour resolved KAFKA-6524. -- Resolution: Information Provided > kafka mirror can't producer internal topic >

Jenkins build is back to normal : kafka-trunk-jdk7 #3161

See

Re: Documentation build system

Hi, I'm in favor of moving away from HTML as well, while I have not had direct exposure to the pain points that Ewen mentions it is in general not pleasant to write docs in I think and bears a lot of risk of parsing failures etc. I can't vote for using rst, not having used it, but I am happy to