Re: delete topic causing spikes in fetch/metadata requests

2016-10-16 Thread sunil kalva
Hi Can you guys help me with this issue On Oct 12, 2016 10:35 PM, "sunil kalva" wrote: > > We are using kafka 0.8.2.2 (client and server), when ever we delete a > topic we see lot of errors in broker logs like below, and there is also a > spike in fetch/metadata requests.

?????? [DISCUSS] KIP-68 Add a consumed log retention before log retention

2016-10-16 Thread ????????
Hi Renu: Sorry for the delay, here are the comments: 1. You mean the config for the topic ? We also support the per topic's consumed retention configuration. 2. The consumer group's commit offset timeout is support in the 0.9, the consumed retention only concern about the current commit

?????? [DISCUSS] KIP-68 Add a consumed log retention before log retention

2016-10-16 Thread ????????
Hi Dong, Thanks for the comments: 1. Say some user starts a consumer to consume topic A with enable.auto.commit = true. Later they change the group name in the config. Then the proposed solution will never execute consumed log retention for the topic A, right? I think group name change is

?????? [DISCUSS] KIP-68 Add a consumed log retention before log retention

2016-10-16 Thread ????????
Hi Dong, The KIP is used to solve both these 2 cases, we specify a small consumed log retention time to deleted the consumed data and avoid losing un-consumed data. And the specify a large force log retention time used as higher bound for the data. I will update the KIP for this info.

[jira] [Updated] (KAFKA-2167) ZkUtils updateEphemeralPath JavaDoc (spelling and correctness)

2016-10-16 Thread Neelesh Srinivas Salian (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2167?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Neelesh Srinivas Salian updated KAFKA-2167: --- Assignee: (was: Neelesh Srinivas Salian) > ZkUtils updateEphemeralPath

[jira] [Issue Comment Deleted] (KAFKA-2167) ZkUtils updateEphemeralPath JavaDoc (spelling and correctness)

2016-10-16 Thread Neelesh Srinivas Salian (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2167?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Neelesh Srinivas Salian updated KAFKA-2167: --- Comment: was deleted (was: [~nehanarkhede], could you please help review this

Re: [DISCUSS] KIP-80: Kafka REST Server

2016-10-16 Thread Jay Kreps
Hey Nacho, Yeah, I think it is definitely a call we have to make case by case. We have some experience with this: originally we attempted to maintain things like non-java clients, a hadoop connector, etc all in the main project. The difficulty of that lead us to the current federated approach. In

Re: [DISCUSS] KIP-80: Kafka REST Server

2016-10-16 Thread Jungtaek Lim
I guess no one doubts its power on REST server or even UI. I understand the difficulty to add a module to project, but it's maximized when there is less support expected hence maintenance issue is likely to rise, and IMHO this seems to be not the case. There're also pain points when project

Re: [DISCUSS] KIP-82 - Add Record Headers

2016-10-16 Thread Jun Rao
Hi, Michael, We do have online KIP discussion meetings from time to time. How about we discuss this KIP Wed (Oct 19) at 11:00am PST? I will send out an invite (we typically do the meeting through Zoom and will post the video recording to Kafka wiki). Thanks, Jun On Wed, Oct 12, 2016 at 1:22