Re: [DISCUSS] KIP-68 Add a consumed log retention before log retention

Hi David, Thanks for the explanation. Could you update the KIP-68 wiki to include the changes that need to be made? I have a few more comments below: 1. We already have an internal topic __consumer_offsets to store all the committed offsets. So the brokers can probably just consume from that to

Re: client use high cpu which caused by delayedFetch operation immediately return

Can you check if you have KAFKA-3003 when you run the code? On Sat, Oct 8, 2016 at 12:52 AM, Kafka wrote: > Hi all, > we found our consumer have high cpu load in our product > enviroment，as we know，fetch.min.bytes and fetch.wait.ma < > http://fetch.wait.ma/>x.ms will affect the frequency

[jira] [Created] (KAFKA-4280) Add REST resource for showing available connector plugin configs

Gwen Shapira created KAFKA-4280: --- Summary: Add REST resource for showing available connector plugin configs Key: KAFKA-4280 URL: https://issues.apache.org/jira/browse/KAFKA-4280 Project: Kafka

[jira] [Created] (KAFKA-4279) REST endpoint to list converter plugins

Gwen Shapira created KAFKA-4279: --- Summary: REST endpoint to list converter plugins Key: KAFKA-4279 URL: https://issues.apache.org/jira/browse/KAFKA-4279 Project: Kafka Issue Type: Bug

[jira] [Created] (KAFKA-4278) Undocumented REST resources

Gwen Shapira created KAFKA-4278: --- Summary: Undocumented REST resources Key: KAFKA-4278 URL: https://issues.apache.org/jira/browse/KAFKA-4278 Project: Kafka Issue Type: Bug Components:

Build failed in Jenkins: kafka-trunk-jdk7 #1613

See Changes: [ismael] KAFKA-4265; Run replication quotas test with producer acks=1 -- [...truncated 7821 lines...] kafka.coordinator.GroupMetadataManagerTest > testExpireGroup STARTED kafka.co

Build failed in Jenkins: kafka-0.10.1-jdk7 #57

See Changes: [ismael] KAFKA-4265; Run replication quotas test with producer acks=1 -- [...truncated 3812 lines...] kafka.integration.SslTopicMetadataTest > testBasicTopicMetadata STARTED kafka.

Jenkins build is back to normal : kafka-trunk-jdk8 #958

See

[jira] [Updated] (KAFKA-4222) Transient failure in QueryableStateIntegrationTest.queryOnRebalance

[ https://issues.apache.org/jira/browse/KAFKA-4222?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ismael Juma updated KAFKA-4222: --- Fix Version/s: (was: 0.10.1.1) 0.10.1.0 > Transient failure in QueryableStateIn

[jira] [Updated] (KAFKA-4253) Fix Kafka Stream thread shutting down process ordering

[ https://issues.apache.org/jira/browse/KAFKA-4253?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ismael Juma updated KAFKA-4253: --- Fix Version/s: (was: 0.10.1.1) 0.10.1.0 > Fix Kafka Stream thread shutting down

[jira] [Updated] (KAFKA-4176) Node stopped receiving heartbeat responses once another node started within the same group

[ https://issues.apache.org/jira/browse/KAFKA-4176?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ismael Juma updated KAFKA-4176: --- Fix Version/s: (was: 0.10.1.1) 0.10.1.0 > Node stopped receiving heartbeat resp

[jira] [Updated] (KAFKA-4267) Quota initialization for uses incorrect ZK path

[ https://issues.apache.org/jira/browse/KAFKA-4267?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ismael Juma updated KAFKA-4267: --- Fix Version/s: (was: 0.10.1.1) 0.10.1.0 > Quota initialization for uses incorr

[jira] [Updated] (KAFKA-4262) Intermittent unit test failure ReassignPartitionsClusterTest.shouldExecuteThrottledReassignment

[ https://issues.apache.org/jira/browse/KAFKA-4262?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ismael Juma updated KAFKA-4262: --- Fix Version/s: (was: 0.10.1.1) 0.10.1.0 > Intermittent unit test failure > Rea

[jira] [Resolved] (KAFKA-4265) Intermittent test failure ReplicationQuotasTest.shouldBootstrapTwoBrokersWithFollowerThrottle

[ https://issues.apache.org/jira/browse/KAFKA-4265?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ismael Juma resolved KAFKA-4265. Resolution: Fixed Fix Version/s: 0.10.1.0 Issue resolved by pull request 1991 [https://github

[jira] [Commented] (KAFKA-4265) Intermittent test failure ReplicationQuotasTest.shouldBootstrapTwoBrokersWithFollowerThrottle

[ https://issues.apache.org/jira/browse/KAFKA-4265?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15558134#comment-15558134 ] ASF GitHub Bot commented on KAFKA-4265: --- Github user asfgit closed the pull request

[GitHub] kafka pull request #1991: KAFKA-4265: Run replication quotas test with produ...

Github user asfgit closed the pull request at: https://github.com/apache/kafka/pull/1991 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enab

[jira] [Commented] (KAFKA-4266) Replication Quota Tests: Ensure ZK updated before tests start

[ https://issues.apache.org/jira/browse/KAFKA-4266?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15557872#comment-15557872 ] ASF GitHub Bot commented on KAFKA-4266: --- GitHub user benstopford opened a pull reque

[GitHub] kafka pull request #1997: KAFKA-4266: ReassignPartitionsClusterTest - Ensure...

GitHub user benstopford opened a pull request: https://github.com/apache/kafka/pull/1997 KAFKA-4266: ReassignPartitionsClusterTest - Ensure ZK publication completed before start Increase the reliability of the one temporal comparison in ReassignPartitionsClusterTest by imposing a d

[jira] [Updated] (KAFKA-4247) kafka-run-class has potential to add a leading colon to classpath

[ https://issues.apache.org/jira/browse/KAFKA-4247?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ryan P updated KAFKA-4247: -- Description: https://github.com/confluentinc/kafka/blob/trunk/bin/kafka-run-class.sh#L128-L133 In the event CLA

[jira] [Updated] (KAFKA-4247) kafka-run-class has potential to add a leading colon to classpath

[ https://issues.apache.org/jira/browse/KAFKA-4247?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ryan P updated KAFKA-4247: -- Description: https://github.com/confluentinc/kafka/blob/trunk/bin/kafka-run-class.sh#L128-L133 In the event CLA

[jira] [Updated] (KAFKA-4266) Replication Quota Tests: Ensure ZK updated before tests start

[ https://issues.apache.org/jira/browse/KAFKA-4266?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ben Stopford updated KAFKA-4266: Description: Alter temporal test in ReassignPartitionsClusterTest so that it doesn't fail if ZK i

Running kafka in distributed file system and storage

Hi, Can Kafka be configured (disable data distribution and replicas) in a HPC environment where the parallel file system and distributed storage perform the parallelism, replicas, and fault tolerant, the log and topic / partitions can be viewed by multiple brokers in the VMs without needing Kafka

Re: [DISCUSS] KIP-68 Add a consumed log retention before log retention

Hi Becket, Thanks for the feedback: 1. We use the simple consumer api to query the commit offset, so we don't need to specify the consumer group. 2. Every broker using the simple consumer api(OffsetFetchKey) to query the commit offset in the log retention process. The client can commit offs

[GitHub] kafka pull request #1996: Fixed introduction doc - wrong streams api link

GitHub user JakubDziworski opened a pull request: https://github.com/apache/kafka/pull/1996 Fixed introduction doc - wrong streams api link You can merge this pull request into a Git repository by running: $ git pull https://github.com/JakubDziworski/kafka doc_fix_streams_link

[jira] [Updated] (KAFKA-4277) creating ephemeral node already exist

[ https://issues.apache.org/jira/browse/KAFKA-4277?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Feixiang Yan updated KAFKA-4277: Description: I use zookeeper 3.4.6. Zookeeper session time out, zkClient try reconnect failed. Then

[jira] [Created] (KAFKA-4277) creating ephemeral node already exist

Feixiang Yan created KAFKA-4277: --- Summary: creating ephemeral node already exist Key: KAFKA-4277 URL: https://issues.apache.org/jira/browse/KAFKA-4277 Project: Kafka Issue Type: Bug Affects

[jira] [Commented] (KAFKA-4254) Questionable handling of unknown partitions in KafkaProducer

[ https://issues.apache.org/jira/browse/KAFKA-4254?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15557526#comment-15557526 ] Konstantine Karantasis commented on KAFKA-4254: --- A patch has been submitted,

[jira] [Commented] (KAFKA-4254) Questionable handling of unknown partitions in KafkaProducer

[ https://issues.apache.org/jira/browse/KAFKA-4254?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15557508#comment-15557508 ] ASF GitHub Bot commented on KAFKA-4254: --- GitHub user kkonstantine opened a pull requ

[GitHub] kafka pull request #1995: KAFKA-4254: Update producers metadata before faili...

GitHub user kkonstantine opened a pull request: https://github.com/apache/kafka/pull/1995 KAFKA-4254: Update producers metadata before failing on non-existent … …partition You can merge this pull request into a Git repository by running: $ git pull https://github.com/kkonst

client use high cpu which caused by delayedFetch operation immediately return

Hi all, we found our consumer have high cpu load in our product enviroment，as we know，fetch.min.bytes and fetch.wait.ma x.ms will affect the frequency of consumer’s return， so we adjust them to very big so that broker is very hard to satisfy it. then we foun

[GitHub] kafka pull request #1994: Introduction Doc: Fixed incomplete sentence

GitHub user JakubDziworski opened a pull request: https://github.com/apache/kafka/pull/1994 Introduction Doc: Fixed incomplete sentence You can merge this pull request into a Git repository by running: $ git pull https://github.com/JakubDziworski/kafka doc_fix Alternatively y