[jira] [Updated] (KAFKA-4042) DistributedHerder thread can die because of connector & task lifecycle exceptions

[ https://issues.apache.org/jira/browse/KAFKA-4042?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shikhar Bhushan updated KAFKA-4042: --- Description: As one example, there isn't exception handling in

[jira] [Updated] (KAFKA-4042) DistributedHerder thread can die because of connector & task lifecycle exceptions

[ https://issues.apache.org/jira/browse/KAFKA-4042?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shikhar Bhushan updated KAFKA-4042: --- Summary: DistributedHerder thread can die because of connector & task lifecycle exceptions

[jira] [Updated] (KAFKA-3117) Fail test at: PlaintextConsumerTest. testAutoCommitOnRebalance

[ https://issues.apache.org/jira/browse/KAFKA-3117?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jason Gustafson updated KAFKA-3117: --- Resolution: Resolved Status: Resolved (was: Patch Available) Marking this resolved

[jira] [Updated] (KAFKA-4044) log actual socket send/receive buffer size after connecting in Selector

[ https://issues.apache.org/jira/browse/KAFKA-4044?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sriharsha Chintalapani updated KAFKA-4044: -- Assignee: Manikumar Reddy > log actual socket send/receive buffer size after

Re: [VOTE] KIP-75 - Add per-connector Converters

+1. On Mon, Aug 15, 2016 at 2:55 PM, Jason Gustafson wrote: > +1 (non-binding) > > On Mon, Aug 15, 2016 at 2:53 PM, Gwen Shapira wrote: > > > +1 (binding) > > > > On Mon, Aug 15, 2016 at 11:21 AM, Ewen Cheslack-Postava > > wrote: > > >

Re: [VOTE] KIP:71 Enable log compaction and deletion to co-exist

+1 On Mon, Aug 15, 2016 at 4:58 PM, Ewen Cheslack-Postava wrote: > +1 (binding) > > Thanks, > -Ewen > > On Mon, Aug 15, 2016 at 4:26 PM, Jun Rao wrote: > > > Thanks for the proposal. +1 > > > > Jun > > > > On Mon, Aug 15, 2016 at 6:20 AM, Damian Guy

Re: [DISCUSS] KIP-71 Enable log compaction and deletion to co-exist

Thanks for the proposal. I'm +1 overall with a thought somewhat related to Jun's comments. While there may not yet be a sensible use case for it, it should be (in theory) legal to have compact_and_delete with size based retention as well. I'm just wondering if it makes sense to allow specifying

Re: [VOTE] KIP:71 Enable log compaction and deletion to co-exist

+1 (binding) Thanks, -Ewen On Mon, Aug 15, 2016 at 4:26 PM, Jun Rao wrote: > Thanks for the proposal. +1 > > Jun > > On Mon, Aug 15, 2016 at 6:20 AM, Damian Guy wrote: > > > I would like to initiate the voting process for KIP-71 ( > >

[jira] [Created] (KAFKA-4044) log actual socket send/receive buffer size after connecting in Selector

Jun Rao created KAFKA-4044: -- Summary: log actual socket send/receive buffer size after connecting in Selector Key: KAFKA-4044 URL: https://issues.apache.org/jira/browse/KAFKA-4044 Project: Kafka

[jira] [Created] (KAFKA-4043) User-defined handler for topology restart

Greg Fodor created KAFKA-4043: - Summary: User-defined handler for topology restart Key: KAFKA-4043 URL: https://issues.apache.org/jira/browse/KAFKA-4043 Project: Kafka Issue Type: Improvement

Re: [VOTE] KIP:71 Enable log compaction and deletion to co-exist

Thanks for the proposal. +1 Jun On Mon, Aug 15, 2016 at 6:20 AM, Damian Guy wrote: > I would like to initiate the voting process for KIP-71 ( > https://cwiki.apache.org/confluence/display/KAFKA/KIP- > 71%3A+Enable+log+compaction+and+deletion+to+co-exist > ). > > This

Re: [DISCUSS] KIP-54 Sticky Partition Assignment Strategy

Thanks Andrew for your feedback and interest on this feature. If there is no further feedback on this KIP (and no objection) I'll start the voting process soon. Thanks. --Vahid From: Andrew Coates To: dev@kafka.apache.org Date: 08/10/2016 12:38 AM

Re: [VOTE] KIP-75 - Add per-connector Converters

+1 (non-binding) On Mon, Aug 15, 2016 at 2:53 PM, Gwen Shapira wrote: > +1 (binding) > > On Mon, Aug 15, 2016 at 11:21 AM, Ewen Cheslack-Postava > wrote: > > I would like to initiate the voting process for KIP-75: > >

Re: [DISCUSS] KIP-62: Allow consumer to send heartbeats from a background thread

Hi All, I wanted to give an update on KIP-62. The patch has gone through a number of iterations and is getting closer to merging, but there have been a couple changes which were not part of the initial KIP that I wanted to call out. 1. To make room for the rebalance timeout in the group metadata

Re: [VOTE] KIP-75 - Add per-connector Converters

+1 (binding) On Mon, Aug 15, 2016 at 11:21 AM, Ewen Cheslack-Postava wrote: > I would like to initiate the voting process for KIP-75: > https://cwiki.apache.org/confluence/display/KAFKA/KIP-75+-+Add+per-connector+Converters > > I'll kick things off with a +1 (binding). > > --

Re: [VOTE] KIP-75 - Add per-connector Converters

+1 (binding) On Mon, Aug 15, 2016 at 2:10 PM Shikhar Bhushan wrote: > +1 (non-binding) > > On Mon, Aug 15, 2016 at 1:20 PM Ismael Juma wrote: > > > +1 (binding) > > > > On 15 Aug 2016 7:21 pm, "Ewen Cheslack-Postava" > wrote: > > > >

[jira] [Work started] (KAFKA-3054) Connect Herder fail forever if sent a wrong connector config or task config

[ https://issues.apache.org/jira/browse/KAFKA-3054?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on KAFKA-3054 started by Shikhar Bhushan. -- > Connect Herder fail forever if sent a wrong connector config or task

[jira] [Assigned] (KAFKA-3054) Connect Herder fail forever if sent a wrong connector config or task config

[ https://issues.apache.org/jira/browse/KAFKA-3054?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shikhar Bhushan reassigned KAFKA-3054: -- Assignee: Shikhar Bhushan (was: jin xing) > Connect Herder fail forever if sent a

Re: [VOTE] KIP:71 Enable log compaction and deletion to co-exist

Oops, it appears I was a bit trigger happy on closing the vote. Sorry, it will remain open for the requisite 72 hours. Sorry again. Damian On Mon, 15 Aug 2016 at 19:56, Damian Guy wrote: > And that concludes the vote. KIP-71 has been accepted. > Thanks everyone. > > Damian

Re: [VOTE] KIP-75 - Add per-connector Converters

+1 (non-binding) On Mon, Aug 15, 2016 at 1:20 PM Ismael Juma wrote: > +1 (binding) > > On 15 Aug 2016 7:21 pm, "Ewen Cheslack-Postava" wrote: > > > I would like to initiate the voting process for KIP-75: > >

Re: [DISCUSS] Time-based releases for Apache Kafka

Exactly :) The goal is to have stable releases. We are hoping that time-based will help achieve this goal as well as introduce some stability to the planning process. On Mon, Aug 15, 2016 at 12:55 PM, Nacho Solis wrote: > To clear up, I'm not against time-based

Re: [VOTE] KIP-75 - Add per-connector Converters

+1 (binding) On 15 Aug 2016 7:21 pm, "Ewen Cheslack-Postava" wrote: > I would like to initiate the voting process for KIP-75: > https://cwiki.apache.org/confluence/display/KAFKA/KIP- > 75+-+Add+per-connector+Converters > > I'll kick things off with a +1 (binding). > > -- >

[jira] [Comment Edited] (KAFKA-3892) Clients retain metadata for non-subscribed topics

[ https://issues.apache.org/jira/browse/KAFKA-3892?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15421601#comment-15421601 ] Noah Sloan edited comment on KAFKA-3892 at 8/15/16 8:18 PM: Unpatched 0.10.0.0

[jira] [Updated] (KAFKA-3892) Clients retain metadata for non-subscribed topics

[ https://issues.apache.org/jira/browse/KAFKA-3892?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Noah Sloan updated KAFKA-3892: -- Affects Version/s: 0.10.0.0 > Clients retain metadata for non-subscribed topics >

[jira] [Commented] (KAFKA-3892) Clients retain metadata for non-subscribed topics

[ https://issues.apache.org/jira/browse/KAFKA-3892?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15421601#comment-15421601 ] Noah Sloan commented on KAFKA-3892: --- Unpatched 0.10.0.0 client exhibits the name behavior. > Clients

Re: [DISCUSS] Time-based releases for Apache Kafka

To clear up, I'm not against time-based releases, I just think that the goals that were stated are not intrinsic to time-based releases but the release process (whether it's time-based or not). The goal of "when will my code get into a release" and the goal of getting features faster in a release

[jira] [Work started] (KAFKA-4042) Missing error handling in Worker.startConnector() can cause Herder thread to die

[ https://issues.apache.org/jira/browse/KAFKA-4042?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on KAFKA-4042 started by Shikhar Bhushan. -- > Missing error handling in Worker.startConnector() can cause Herder thread

Re: [VOTE] KIP-75 - Add per-connector Converters

+1 (binding) On Mon, Aug 15, 2016 at 11:21 AM, Ewen Cheslack-Postava wrote: > I would like to initiate the voting process for KIP-75: > https://cwiki.apache.org/confluence/display/KAFKA/KIP- > 75+-+Add+per-connector+Converters > > I'll kick things off with a +1 (binding). > >

[jira] [Created] (KAFKA-4042) Missing error handling in Worker.startConnector() can cause Herder thread to die

Shikhar Bhushan created KAFKA-4042: -- Summary: Missing error handling in Worker.startConnector() can cause Herder thread to die Key: KAFKA-4042 URL: https://issues.apache.org/jira/browse/KAFKA-4042

Re: [VOTE] KIP:71 Enable log compaction and deletion to co-exist

And that concludes the vote. KIP-71 has been accepted. Thanks everyone. Damian On Mon, 15 Aug 2016 at 19:37 Guozhang Wang wrote: > +1 (binding). Thanks for putting this KIP together Damian! > > On Mon, Aug 15, 2016 at 10:43 AM, Michael Noll > wrote: >

Re: [VOTE] KIP:71 Enable log compaction and deletion to co-exist

+1 (binding). Thanks for putting this KIP together Damian! On Mon, Aug 15, 2016 at 10:43 AM, Michael Noll wrote: > +1 (non-binding) > > On Mon, Aug 15, 2016 at 7:34 PM, Sriram Subramanian > wrote: > > > +1 (binding) > > > > On Mon, Aug 15, 2016 at 10:27

[VOTE] KIP-75 - Add per-connector Converters

I would like to initiate the voting process for KIP-75: https://cwiki.apache.org/confluence/display/KAFKA/KIP-75+-+Add+per-connector+Converters I'll kick things off with a +1 (binding). -- Thanks, Ewen

Re: Review request for KAFKA-3600

I am open to any forum for design discussions. I had run this by Jason and gotten some feedback, four months back. I have addressed those feedbacks, and I believe not much have changed since then in the design. I would love to have more feedback. - do we want to modify the state machine?

[GitHub] kafka pull request #1741: Add more verbose logging for offset map building

GitHub user eliasdorneles opened a pull request: https://github.com/apache/kafka/pull/1741 Add more verbose logging for offset map building I propose to make the offset map building a bit more verbose to make log cleaning messages a bit easier to follow, for people not much

[jira] [Created] (KAFKA-4041) kafka unable to reconnect to zookeeper behind an ELB

prabhakar created KAFKA-4041: Summary: kafka unable to reconnect to zookeeper behind an ELB Key: KAFKA-4041 URL: https://issues.apache.org/jira/browse/KAFKA-4041 Project: Kafka Issue Type: Bug

Re: [VOTE] KIP:71 Enable log compaction and deletion to co-exist

+1 (non-binding) On Mon, Aug 15, 2016 at 7:34 PM, Sriram Subramanian wrote: > +1 (binding) > > On Mon, Aug 15, 2016 at 10:27 AM, Ismael Juma wrote: > > > Thanks for the KIP, +1 (binding) > > > > Ismael > > > > On Mon, Aug 15, 2016 at 2:20 PM, Damian Guy

Re: [VOTE] KIP:71 Enable log compaction and deletion to co-exist

+1 (binding) On Mon, Aug 15, 2016 at 10:27 AM, Ismael Juma wrote: > Thanks for the KIP, +1 (binding) > > Ismael > > On Mon, Aug 15, 2016 at 2:20 PM, Damian Guy wrote: > > > I would like to initiate the voting process for KIP-71 ( > >

Re: [VOTE] KIP:71 Enable log compaction and deletion to co-exist

Thanks for the KIP, +1 (binding) Ismael On Mon, Aug 15, 2016 at 2:20 PM, Damian Guy wrote: > I would like to initiate the voting process for KIP-71 ( > https://cwiki.apache.org/confluence/display/KAFKA/KIP- > 71%3A+Enable+log+compaction+and+deletion+to+co-exist > ). > >

Change Apache Kafka properties dynamically without restarting servers using Archaius

Hello, I have some questions related to properties of Apache Kafka. We have a four node Kafka Cluster which is on Amazon EMR. Currently, in order to change any properties we do restart the servers to pick the new changes made. For the following: 1. Can we change the topic name that a producer is

Re: [DISCUSS] KIP-74: Add Fetch Response Size Limit in Bytes

Hi, Andrey, Thanks for the update to the wiki. Just a few more minor comments. 1. "If *response_max_bytes* parameter is zero ("no limit"), the request is processed *exactly* as before." Instead of using 0, it seems it's more natural to use Int.MAX_INT to preserve the old behavior. 2. "For the

[jira] [Commented] (KAFKA-4040) Round Robin Assignment does not create balanced assignments

[ https://issues.apache.org/jira/browse/KAFKA-4040?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15421243#comment-15421243 ] Mathias Herberts commented on KAFKA-4040: - The drawback could be overcome by sorting the consumers

[jira] [Updated] (KAFKA-4037) Transient failure in ConnectRestApiTest

[ https://issues.apache.org/jira/browse/KAFKA-4037?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ewen Cheslack-Postava updated KAFKA-4037: - Status: Patch Available (was: Open) > Transient failure in ConnectRestApiTest >

Re: [VOTE] KIP:71 Enable log compaction and deletion to co-exist

+1 (binding) On Mon, Aug 15, 2016 at 9:26 AM Matthias J. Sax wrote: > +1 > > On 08/15/2016 06:00 PM, Eno Thereska wrote: > > +1 (non-binding) > > > >> On 15 Aug 2016, at 14:20, Damian Guy wrote: > >> > >> I would like to initiate the voting process

Re: [VOTE] KIP:71 Enable log compaction and deletion to co-exist

+1 On 08/15/2016 06:00 PM, Eno Thereska wrote: > +1 (non-binding) > >> On 15 Aug 2016, at 14:20, Damian Guy wrote: >> >> I would like to initiate the voting process for KIP-71 ( >>

Re: Review request for KAFKA-3600

In general, I believe it is good to have design discussions in the mailing list (do we want to modify the state machine? do we need to maintain version per connection? which layer is responsible for the API check)? PR works well with tactical issues (class / object / method level) but less so for

[GitHub] kafka pull request #1740: MINOR: Remove # from .bat start script

GitHub user p-thorpe opened a pull request: https://github.com/apache/kafka/pull/1740 MINOR: Remove # from .bat start script On Windows, the following output is seen when starting Zookeeper and Kafka servers: ``` '#' is not recognized as an internal or external command,

Re: [VOTE] KIP:71 Enable log compaction and deletion to co-exist

+1 (non-binding) > On 15 Aug 2016, at 14:20, Damian Guy wrote: > > I would like to initiate the voting process for KIP-71 ( > https://cwiki.apache.org/confluence/display/KAFKA/KIP-71%3A+Enable+log+compaction+and+deletion+to+co-exist > ). > > This change will add a new

[jira] [Updated] (KAFKA-4035) AclCommand should allow Describe operation on groups

[ https://issues.apache.org/jira/browse/KAFKA-4035?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Manikumar Reddy updated KAFKA-4035: --- Resolution: Fixed Fix Version/s: 0.10.1.0 Status: Resolved (was: Patch

Re: Review request for KAFKA-3600

Hello Dana, Thanks for the feedback. Kafka-3600 is not intended to address backwards compatibility of clients. Backwards compatibility of clients is a lengthier discussion and it will be better to take it on a separate KIP. We initiated that discussion on KIP-35's discuss thread and it was hard

[jira] [Commented] (KAFKA-4040) Round Robin Assignment does not create balanced assignments

[ https://issues.apache.org/jira/browse/KAFKA-4040?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15421131#comment-15421131 ] Vahid Hashemian commented on KAFKA-4040: [~herberts] Please see

[jira] [Created] (KAFKA-4040) Round Robin Assignment does not create balanced assignments

Mathias Herberts created KAFKA-4040: --- Summary: Round Robin Assignment does not create balanced assignments Key: KAFKA-4040 URL: https://issues.apache.org/jira/browse/KAFKA-4040 Project: Kafka

[VOTE] KIP:71 Enable log compaction and deletion to co-exist

I would like to initiate the voting process for KIP-71 ( https://cwiki.apache.org/confluence/display/KAFKA/KIP-71%3A+Enable+log+compaction+and+deletion+to+co-exist ). This change will add a new cleanup.policy, compact_and_delete, that when enabled will run both compaction and deletion. Thanks,

[jira] [Commented] (KAFKA-3979) Optimize memory used by replication process by using adaptive fetch message size

[ https://issues.apache.org/jira/browse/KAFKA-3979?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15420917#comment-15420917 ] ASF GitHub Bot commented on KAFKA-3979: --- Github user nepal closed the pull request at:

[GitHub] kafka pull request #1642: [KAFKA-3979] Optimize memory used by replication p...

Github user nepal closed the pull request at: https://github.com/apache/kafka/pull/1642 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[jira] [Updated] (KAFKA-3777) Extract the existing LRU cache out of RocksDBStore

[ https://issues.apache.org/jira/browse/KAFKA-3777?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eno Thereska updated KAFKA-3777: Assignee: (was: Eno Thereska) > Extract the existing LRU cache out of RocksDBStore >

[jira] [Assigned] (KAFKA-3974) LRU cache should store bytes/object and not records

[ https://issues.apache.org/jira/browse/KAFKA-3974?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eno Thereska reassigned KAFKA-3974: --- Assignee: Eno Thereska (was: Bill Bejeck) > LRU cache should store bytes/object and not

[jira] [Assigned] (KAFKA-3777) Extract the existing LRU cache out of RocksDBStore

[ https://issues.apache.org/jira/browse/KAFKA-3777?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eno Thereska reassigned KAFKA-3777: --- Assignee: Eno Thereska (was: Anna Povzner) > Extract the existing LRU cache out of

[jira] [Assigned] (KAFKA-3776) Unify store and downstream caching in streams

[ https://issues.apache.org/jira/browse/KAFKA-3776?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eno Thereska reassigned KAFKA-3776: --- Assignee: Eno Thereska > Unify store and downstream caching in streams >

Re: [DISCUSS] KIP-74: Add Fetch Response Size Limit in Bytes

Hi all! KIP-74 is updated to sync up with mail list discussion. https://cwiki.apache.org/confluence/display/KAFKA/KIP-74%3A+Add+Fetch+Response+Size+Limit+in+Bytes Your feedback is highly appreciated. Thanks, Andrey. > On 12 Aug 2016, at 20:49, Andrey L. Neporada

Re: [DISCUSS] KIP-71 Enable log compaction and deletion to co-exist

Thanks Jason. The log retention.ms will be set to a value that greater than the window retention time. So as windows expire, they eventually get cleaned up by the broker. It doesn't matter if old windows are around for sometime beyond their usefulness, more that they do eventually get removed and