Re: KIP-121 [Discuss]: Add KStream peek method

Many thanks for the KIP and the PR, Steven! My opinion, too, is that we should consider including this. One thing that I would like to see clarified is the difference between the proposed peek() and existing functions map() and foreach(), for instance. My understanding (see also the Java 8 links

Re: KIP-121 [Discuss]: Add KStream peek method

Hi, I like the proposal, thank you. I have found it frustrating myself not to be able to understand simple things, like how many records have been currently processed. The peek method would allow those kinds of diagnostics and debugging. Gwen, it is possible to do this with the existing

Re: [DISCUSS] KIP-112: Handle disk failure for JBOD

Hi Dong, To simplify the discussion today, on my part I'll zoom into one thing only: - I'll discuss the options called below : "one-broker-per-disk" or "one-broker-per-few-disks". - I completely buy the JBOD vs RAID arguments so there is no need to discuss that part for me. I buy it that

Re: KIP-121 [Discuss]: Add KStream peek method

Hi Steven, Thanks for the KIP. I think this is a worthy addition to the API. Thanks, Damian On Tue, 7 Feb 2017 at 09:30 Eno Thereska wrote: > Hi, > > I like the proposal, thank you. I have found it frustrating myself not to > be able to understand simple things, like

[jira] [Commented] (KAFKA-4567) Connect Producer and Consumer ignore ssl parameters configured for worker

[ https://issues.apache.org/jira/browse/KAFKA-4567?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15855932#comment-15855932 ] Sönke Liebau commented on KAFKA-4567: - Alright. I have added a small paragraph to the docs about this

[jira] [Commented] (KAFKA-4461) When using ProcessorTopologyTestDriver, the combination of map and .groupByKey does not produce any result

[ https://issues.apache.org/jira/browse/KAFKA-4461?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15855792#comment-15855792 ] Adrian McCague commented on KAFKA-4461: --- [~guozhang] Thanks! > When using

[jira] [Commented] (KAFKA-4567) Connect Producer and Consumer ignore ssl parameters configured for worker

[ https://issues.apache.org/jira/browse/KAFKA-4567?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15855931#comment-15855931 ] ASF GitHub Bot commented on KAFKA-4567: --- GitHub user soenkeliebau opened a pull request:

[GitHub] kafka pull request #2511: KAFKA-4567 - Connect Producer and Consumer ignore ...

GitHub user soenkeliebau opened a pull request: https://github.com/apache/kafka/pull/2511 KAFKA-4567 - Connect Producer and Consumer ignore ssl parameters… KAFKA-4567 - Connect Producer and Consumer ignore ssl parameters configured for worker Added brief explanation to

Correct prefetching of data to KTable-like structure on application startup

Hi all, I have a question how to do a correct caching in KTable-like structure on application startup. I'm not sure if this belongs to user or dev maillist, so sorry if I've chosen the bad one. What is my observation so far: - if I don't send any data to a kafka partition for a period

[GitHub] kafka pull request #2183: Removed obsolete parameter form example config in ...

Github user soenkeliebau closed the pull request at: https://github.com/apache/kafka/pull/2183 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature

[GitHub] kafka pull request #2512: Removed obsolete parameter form example config in ...

GitHub user soenkeliebau opened a pull request: https://github.com/apache/kafka/pull/2512 Removed obsolete parameter form example config in docs. Parameter controller.message.queue.size was removed in 0.9 (KAFKA-2122) but is still listed in an example broker configuration in the

[jira] [Commented] (KAFKA-2122) Remove controller.message.queue.size Config

[ https://issues.apache.org/jira/browse/KAFKA-2122?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15856134#comment-15856134 ] ASF GitHub Bot commented on KAFKA-2122: --- GitHub user soenkeliebau opened a pull request:

Re: [DISCUSS] KIP-120: Cleanup Kafka Streams builder API

On Mon, Feb 6, 2017 at 2:35 PM, Matthias J. Sax wrote: > - adding KStreamBuilder#topologyBuilder() seems like be a good idea to > address any concern with limited access to TopologyBuilder and DSL/PAPI > mix-and-match approach. However, we should try to cover as much as >

[jira] [Commented] (KAFKA-2122) Remove controller.message.queue.size Config

[ https://issues.apache.org/jira/browse/KAFKA-2122?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15856115#comment-15856115 ] ASF GitHub Bot commented on KAFKA-2122: --- Github user soenkeliebau closed the pull request at:

Re: [DISCUSS] KIP-117: Add a public AdministrativeClient API for Kafka admin operations

Hi all, I think it's good that we have discussed a number of API that would make sense in the AdminClient. Having done that, I think we should now narrow the focus of this KIP to a small set of methods to get us started. Once we have an AdminClient in the codebase, we can propose subsequent KIPs

[jira] [Updated] (KAFKA-4734) timeindex on old segments not trimmed to actual size

[ https://issues.apache.org/jira/browse/KAFKA-4734?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jun Rao updated KAFKA-4734: --- Resolution: Fixed Fix Version/s: 0.10.2.0 Status: Resolved (was: Patch Available) Issue

[jira] [Commented] (KAFKA-4734) timeindex on old segments not trimmed to actual size

[ https://issues.apache.org/jira/browse/KAFKA-4734?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15856309#comment-15856309 ] ASF GitHub Bot commented on KAFKA-4734: --- Github user asfgit closed the pull request at:

[GitHub] kafka pull request #2501: KAFKA-4734; Trim the time index on old segments

Github user asfgit closed the pull request at: https://github.com/apache/kafka/pull/2501 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[jira] [Commented] (KAFKA-4686) Null Message payload is shutting down broker

[ https://issues.apache.org/jira/browse/KAFKA-4686?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15856272#comment-15856272 ] Rodrigo Queiroz Saramago commented on KAFKA-4686: - Hi [~hachikuji], no, we are using the

[jira] [Comment Edited] (KAFKA-4686) Null Message payload is shutting down broker

[ https://issues.apache.org/jira/browse/KAFKA-4686?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15856272#comment-15856272 ] Rodrigo Queiroz Saramago edited comment on KAFKA-4686 at 2/7/17 4:29 PM:

[jira] [Resolved] (KAFKA-4727) A Production server configuration needs to be updated

[ https://issues.apache.org/jira/browse/KAFKA-4727?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jun Rao resolved KAFKA-4727. Resolution: Fixed Fix Version/s: 0.10.2.0 Issue resolved by pull request 2490

[GitHub] kafka pull request #2490: kafka-4727: A Production server configuration need...

Github user asfgit closed the pull request at: https://github.com/apache/kafka/pull/2490 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

Re: [DISCUSS] KIP-117: Add a public AdministrativeClient API for Kafka admin operations

+1 I think its important to focus this KIP discussion on the "patterns" we would like to see in the client and a few key methods in order to make progress and then iterate from there. I think we should let Colin drive the APIs he thinks are important since he is volunteering to do the work. And

[jira] [Commented] (KAFKA-2122) Remove controller.message.queue.size Config

[ https://issues.apache.org/jira/browse/KAFKA-2122?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15856324#comment-15856324 ] ASF GitHub Bot commented on KAFKA-2122: --- Github user soenkeliebau closed the pull request at:

[GitHub] kafka pull request #2512: Removed obsolete parameter form example config in ...

Github user soenkeliebau closed the pull request at: https://github.com/apache/kafka/pull/2512 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature

[GitHub] kafka pull request #2513: WIP: Remove Struct from Request/Response classes

GitHub user ijuma opened a pull request: https://github.com/apache/kafka/pull/2513 WIP: Remove Struct from Request/Response classes You can merge this pull request into a Git repository by running: $ git pull https://github.com/ijuma/kafka separate-struct Alternatively you

[jira] [Commented] (KAFKA-4686) Null Message payload is shutting down broker

[ https://issues.apache.org/jira/browse/KAFKA-4686?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15856379#comment-15856379 ] Jason Gustafson commented on KAFKA-4686: [~rodrigo.saramago] Are there no files with the ".log"

[jira] [Commented] (KAFKA-4686) Null Message payload is shutting down broker

[ https://issues.apache.org/jira/browse/KAFKA-4686?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15856412#comment-15856412 ] Jason Gustafson commented on KAFKA-4686: One additional question: which clients and which versions

Re: [DISCUSS] KIP-117: Add a public AdministrativeClient API for Kafka admin operations

+1. under ismael's "facet" approach we could always start with AdminClient.topics() and then pile on more of them later. On Tue, Feb 7, 2017 at 8:57 AM, Grant Henke wrote: > +1 I think its important to focus this KIP discussion on the "patterns" we > would like to see in

[jira] [Comment Edited] (KAFKA-4686) Null Message payload is shutting down broker

[ https://issues.apache.org/jira/browse/KAFKA-4686?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15856272#comment-15856272 ] Rodrigo Queiroz Saramago edited comment on KAFKA-4686 at 2/7/17 5:31 PM:

Re: [DISCUSS] KIP-117: Add a public AdministrativeClient API for Kafka admin operations

+1 > On Feb 7, 2017, at 9:17 AM, radai wrote: > > +1. > > under ismael's "facet" approach we could always start with > AdminClient.topics() and then pile on more of them later. > >> On Tue, Feb 7, 2017 at 8:57 AM, Grant Henke wrote: >> >> +1

[jira] [Commented] (KAFKA-4686) Null Message payload is shutting down broker

[ https://issues.apache.org/jira/browse/KAFKA-4686?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15856518#comment-15856518 ] Rodrigo Queiroz Saramago commented on KAFKA-4686: - Running the Dumplog without

[jira] [Comment Edited] (KAFKA-4686) Null Message payload is shutting down broker

[ https://issues.apache.org/jira/browse/KAFKA-4686?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15856500#comment-15856500 ] Rodrigo Queiroz Saramago edited comment on KAFKA-4686 at 2/7/17 6:42 PM:

[jira] [Commented] (KAFKA-4686) Null Message payload is shutting down broker

[ https://issues.apache.org/jira/browse/KAFKA-4686?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15856522#comment-15856522 ] Jason Gustafson commented on KAFKA-4686: Can you try using DumpLogSegments on the log files

[jira] [Commented] (KAFKA-4686) Null Message payload is shutting down broker

[ https://issues.apache.org/jira/browse/KAFKA-4686?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15856534#comment-15856534 ] Rodrigo Queiroz Saramago commented on KAFKA-4686: - Sure, the results are below:

[jira] [Commented] (KAFKA-1194) The kafka broker cannot delete the old log files after the configured time

[ https://issues.apache.org/jira/browse/KAFKA-1194?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15856748#comment-15856748 ] Brady Vidovic commented on KAFKA-1194: -- I'm trying to determine if/when this will be in a Kafka

Re: Kafka KIP meeting Feb 7 at 11:00am PST

The following are the notes from today's KIP discussion. - KIP-112 - Handle disk failure for JBOD: We discussed whether we need to support JBOD directly in Kafka or just rely on the 1 disk per broker model. The general consensus is that direct JBOD support in Kafka is needed. There

[jira] [Commented] (KAFKA-4738) Remove generic type of class ClientState

[ https://issues.apache.org/jira/browse/KAFKA-4738?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15856683#comment-15856683 ] Matthias J. Sax commented on KAFKA-4738: Maybe. But we should have a new JIRA for this case. It's

[jira] [Updated] (KAFKA-4721) KafkaStreams (and possibly others) should inherit Closeable

[ https://issues.apache.org/jira/browse/KAFKA-4721?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthias J. Sax updated KAFKA-4721: --- Labels: needs-kip (was: ) > KafkaStreams (and possibly others) should inherit Closeable >

Re: [VOTE] KIP-111 Kafka should preserve the Principal generated by the PrincipalBuilder while processing the request received on socket channel, on the broker.

Bumping up this thread. Thanks, Mayuresh On Fri, Feb 3, 2017 at 5:09 PM, radai wrote: > +1 > > On Fri, Feb 3, 2017 at 11:25 AM, Mayuresh Gharat < > gharatmayures...@gmail.com > > wrote: > > > Hi All, > > > > It seems that there is no further concern with the

Re: [DISCUSS] KIP-117: Add a public AdministrativeClient API for Kafka admin operations

Hey Colin, This is great, thought I'd throw out a couple of opinions to the mix, feel free to ignore: - I think it would be good to not use "get" as the prefix for things making remote calls. We've tried to avoid the java getter convention entirely (see code style guide), but for remote

[jira] [Created] (KAFKA-4742) ResetTool does not commit offsets correclty

Matthias J. Sax created KAFKA-4742: -- Summary: ResetTool does not commit offsets correclty Key: KAFKA-4742 URL: https://issues.apache.org/jira/browse/KAFKA-4742 Project: Kafka Issue Type:

Re: [DISCUSS] KIP-117: Add a public AdministrativeClient API for Kafka admin operations

Hi Jay, Thanks for the feedback. Comments inline. On Tue, Feb 7, 2017 at 8:18 PM, Jay Kreps wrote: > >- I think it would be good to not use "get" as the prefix for things >making remote calls. We've tried to avoid the java getter convention >entirely (see code

Reg: Kafka Kerberos

Hi, I am using Kafka 0.10.1.0 and kerberozied cluster. Kafka_jaas.conf file: Client { com.sun.security.auth.module.Krb5LoginModule required useKeyTab=true keyTab="/etc/security/keytabs/kafka.service.keytab" storeKey=true useTicketCache=false serviceName="zookeeper"

Re: [VOTE] KIP-48 Support for delegation tokens as an authentication mechanism

Looks like a great proposal! I noticed that key rotation is not included. That may be reasonable for the initial work, but it might be nice to share some thoughts on how that might work in the future. For example, I could imagine delegation.token.master.key could be a list, which would allow users

Build failed in Jenkins: kafka-trunk-jdk8 #1255

See Changes: [junrao] KAFKA-4727; A Production server configuration needs to be updated [junrao] KAFKA-4734; Trim the time index on old segments -- [...truncated 37436 lines...]

Re: [VOTE] 0.10.2.0 RC0

Hi, Ewen, KAFKA-4725 has been committed to 0.10.2. Thanks, Jun On Mon, Feb 6, 2017 at 11:18 PM, Ewen Cheslack-Postava wrote: > All, > > It sounds like https://issues.apache.org/jira/browse/KAFKA-4725 is a > regression in an earlier release and worth fixing. I've marked it

[subscribe Request]

Hi Team, I would like to subscribe to Kafka dev group Thanks, Ashish Singh

Re: [VOTE] KIP-48 Support for delegation tokens as an authentication mechanism

Hi, Mani, If a token expires, then every broker will potentially try to delete it around the same time, but only one will succeed. So, we will have to deal with failures in that case? Another way is to let just one broker (say, the controller) deletes expired tokens. It would also be helpful for

[jira] [Updated] (KAFKA-4697) Simplify Streams Reset Tool

[ https://issues.apache.org/jira/browse/KAFKA-4697?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthias J. Sax updated KAFKA-4697: --- Affects Version/s: 0.10.2.0 > Simplify Streams Reset Tool > --- > >

[jira] [Commented] (KAFKA-4725) Kafka broker fails due to OOM when producer exceeds throttling quota for extended periods of time

[ https://issues.apache.org/jira/browse/KAFKA-4725?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15856990#comment-15856990 ] ASF GitHub Bot commented on KAFKA-4725: --- Github user asfgit closed the pull request at:

[GitHub] kafka pull request #2496: KAFKA-4725: Stop leaking messages in produce reque...

Github user asfgit closed the pull request at: https://github.com/apache/kafka/pull/2496 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[jira] [Commented] (KAFKA-4564) When the destination brokers are down or misconfigured in config, Streams should fail fast

[ https://issues.apache.org/jira/browse/KAFKA-4564?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15856964#comment-15856964 ] Gwen Shapira commented on KAFKA-4564: - OMG, this is great. I got this: [2017-02-07 14:48:10,250]

[jira] [Comment Edited] (KAFKA-4564) When the destination brokers are down or misconfigured in config, Streams should fail fast

[ https://issues.apache.org/jira/browse/KAFKA-4564?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15856964#comment-15856964 ] Gwen Shapira edited comment on KAFKA-4564 at 2/7/17 10:51 PM: -- OMG, this is

[jira] [Commented] (KAFKA-4725) Kafka broker fails due to OOM when producer exceeds throttling quota for extended periods of time

[ https://issues.apache.org/jira/browse/KAFKA-4725?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15856965#comment-15856965 ] Tim Carey-Smith commented on KAFKA-4725: We have run stress tests on builds which include this

Re: [DISCUSS] KIP-98: Exactly Once Delivery and Transactional Messaging

A couple follow-ups from Ismael's previous comments. 1. We have removed the individual message CRC field from the message format. Because this field can already change on the broker in some situations, we feel it is probably not wise to let clients depend on it, and removing it saves some space

[jira] [Updated] (KAFKA-4725) Kafka broker fails due to OOM when producer exceeds throttling quota for extended periods of time

[ https://issues.apache.org/jira/browse/KAFKA-4725?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jun Rao updated KAFKA-4725: --- Resolution: Fixed Fix Version/s: (was: 0.10.3.0) Status: Resolved (was: Patch

Re: [DISCUSS] KIP-117: Add a public AdministrativeClient API for Kafka admin operations

>>- Personally I don't think splitting the admin methods up actually makes >>things more usable. It just makes you have to dig through our >> hierarchy. I >>think a flat class with a bunch of operations (like the consumer api) is >>probably the easiest for people to grok and find

Re: [DISCUSS] KIP-112: Handle disk failure for JBOD

Hey Jun, Thanks for the all the comments. I should have written the summary earlier but got delayed. I think Grant has summarized pretty much every major issues we discussed in the KIP meeting. I have provided answer to each issue. Let me try to address your questions here. I will update the KIP

Re: [DISCUSS] KIP-112: Handle disk failure for JBOD

Hi Dong, Thanks for proposing the KIP and all the hard work on it! In order to help summarize the discussion from the KIP call today I wanted to list the things I heard as the main discussion points that people would like to be considered or discussed. However, this is strictly from memory so

Re: [DISCUSS] KIP-112: Handle disk failure for JBOD

Hi, Dong, Thanks for the discussion in the KIP meeting today. A few comments inlined below. On Mon, Feb 6, 2017 at 7:22 PM, Dong Lin wrote: > Hey Jun, > > Thanks for the review! Please see reply inline. > > On Mon, Feb 6, 2017 at 6:21 PM, Jun Rao wrote:

Re: [DISCUSS] KIP-112: Handle disk failure for JBOD

Hey Grant, Thanks much for the detailed summary! Yes, this is pretty much my understanding of the KIP meeting. I also think everyone agreed on the point you outlined in the email. Here is my reply to the five issues you mentioned. 1) Automatic vs Manual Recovery In the case where a disk is

Re: [VOTE] KIP-48 Support for delegation tokens as an authentication mechanism

Read the KIP again and I think it looks good. +1 from me. On Tue, Feb 7, 2017 at 3:05 PM, Jun Rao wrote: > Hi, Mani, > > If a token expires, then every broker will potentially try to delete it > around the same time, but only one will succeed. So, we will have to deal > with

Build failed in Jenkins: kafka-trunk-jdk7 #1919

See Changes: [junrao] KAFKA-4725; Stop leaking messages in produce request body when requests -- [...truncated 15912 lines...] org.apache.kafka.clients.producer.internals.RecordAccumulatorTest

[jira] [Updated] (KAFKA-4743) Add a tool to Reset Consumer Group Offsets

[ https://issues.apache.org/jira/browse/KAFKA-4743?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jorge Quilcate updated KAFKA-4743: -- Description: Add an external tool to reset Consumer Group offsets, and achieve rewind over the

Re: [DISCUSS] KIP-117: Add a public AdministrativeClient API for Kafka admin operations

+1 for placing the AdminClient in a separate package. I have several dozen applications that use kafka-clients.jar to produce/consume, and those applications don't have a need for the AdminClient. And, it's probably better not to have the AdminClient methods available in those applications from a

Re: KIP-122: Add a tool to Reset Consumer Group Offsets

Jorge, thanks for you KIP. I like it a lot and think it will be a nice addition! -Matthias On 2/7/17 7:04 PM, Dong Lin wrote: > Hey Jorge, > > Thanks for the KIP. I have some quick comments: > > - Should we allow user to use wildcard to reset offset of all groups for a > given topic as

Re: [DISCUSS] KIP-116 - Add State Store Checkpoint Interval Configuration

Damian, I am not strict about it either. However, if there is no advantage in disabling it, we might not want to allow it. This would have the advantage to guard users to accidentally switch it off. -Matthias On 2/3/17 2:03 AM, Damian Guy wrote: > Hi Matthias, > > It possibly doesn't make

Re: KIP-122: Add a tool to Reset Consumer Group Offsets

I've been meaning to suggest something very similar to this KIP. Something lacking in the KIP is under what scenarios the offset reset tool will run. Are all members of the group expected to be offline or can we override offsets while members of the group are live? This matters when factoring in

Re: KIP-122: Add a tool to Reset Consumer Group Offsets

Jorge, can you please add your KIP to this table: https://cwiki.apache.org/confluence/display/KAFKA/Kafka+Improvement+Proposals#KafkaImprovementProposals-KIPsunderdiscussion Thanks! -Matthias On 2/7/17 9:29 PM, Matthias J. Sax wrote: > Jorge, > > thanks for you KIP. I like it a lot and

Re: KIP-122: Add a tool to Reset Consumer Group Offsets

I think it makes sense to just add the feature to kafka-consumer-groups.sh On Tue, Feb 7, 2017 at 10:24 PM, Gwen Shapira wrote: > Thanks for the KIP. I'm super happy about adding the capability. > > I hate the interface, though. It looks exactly like the replica > assignment

Re: KIP-122: Add a tool to Reset Consumer Group Offsets

As long as the CLI is a bit consistent? Like, not just adding 3 arguments and a JSON parser to the existing tool, right? On Tue, Feb 7, 2017 at 10:29 PM, Onur Karaman wrote: > I think it makes sense to just add the feature to kafka-consumer-groups.sh > > On Tue, Feb

[jira] [Assigned] (KAFKA-4743) Add a tool to Reset Consumer Group Offsets

[ https://issues.apache.org/jira/browse/KAFKA-4743?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthias J. Sax reassigned KAFKA-4743: -- Assignee: Matthias J. Sax > Add a tool to Reset Consumer Group Offsets >

[jira] [Updated] (KAFKA-4743) Add a tool to Reset Consumer Group Offsets

[ https://issues.apache.org/jira/browse/KAFKA-4743?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthias J. Sax updated KAFKA-4743: --- Labels: kip (was: ) > Add a tool to Reset Consumer Group Offsets >

[jira] [Assigned] (KAFKA-4743) Add a tool to Reset Consumer Group Offsets

[ https://issues.apache.org/jira/browse/KAFKA-4743?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthias J. Sax reassigned KAFKA-4743: -- Assignee: (was: Matthias J. Sax) > Add a tool to Reset Consumer Group Offsets >

Re: KIP-122: Add a tool to Reset Consumer Group Offsets

Thanks for the KIP. I'm super happy about adding the capability. I hate the interface, though. It looks exactly like the replica assignment tool. A tool everyone loves so much that there are multiple projects, open and closed, that try to fix it. Can we swap it with something that looks a bit

Build failed in Jenkins: kafka-trunk-jdk8 #1256

See Changes: [junrao] KAFKA-4725; Stop leaking messages in produce request body when requests -- [...truncated 8409 lines...] kafka.security.auth.SimpleAclAuthorizerTest > testLoadCache

[jira] [Created] (KAFKA-4743) Add a tool to Reset Consumers Offsets

Jorge Quilcate created KAFKA-4743: - Summary: Add a tool to Reset Consumers Offsets Key: KAFKA-4743 URL: https://issues.apache.org/jira/browse/KAFKA-4743 Project: Kafka Issue Type: New

[jira] [Updated] (KAFKA-4743) Add a tool to Reset Consumer Group Offsets

[ https://issues.apache.org/jira/browse/KAFKA-4743?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jorge Quilcate updated KAFKA-4743: -- Summary: Add a tool to Reset Consumer Group Offsets (was: Add a tool to Reset Consumers

Re: [DISCUSS] KIP-117: Add a public AdministrativeClient API for Kafka admin operations

Totally agree on CompletableFuture. Also agree with some of the rough edges on the Consumer. I don't have much of a leg to stand on with the splitting vs not splitting thing, really hard to argue one or the other is better. I guess the one observation in watching us try to make good public apis

Re: [DISCUSS] KIP-117: Add a public AdministrativeClient API for Kafka admin operations

Ismael raised the concern (offline) that even if we mark the API as unstable, people will use it right away because it is so valuable. So we'll have trouble changing later if we prefer a different API. I agree, but since I still prefer to experiment a bit "in the wild", I suggested placing the

[jira] [Commented] (KAFKA-4564) When the destination brokers are down or misconfigured in config, Streams should fail fast

[ https://issues.apache.org/jira/browse/KAFKA-4564?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15857198#comment-15857198 ] Matthias J. Sax commented on KAFKA-4564: [~guozhang] Can we close this ? > When the destination

Re: [VOTE] KIP-48 Support for delegation tokens as an authentication mechanism

Hi, Roger, Just to clarify. This KIP already allows you to do impersonation at the connection level. Are you talking about impersonation at the request level? Thanks, Jun On Tue, Feb 7, 2017 at 5:53 PM, Roger Hoover wrote: > Just wondering...how difficult would be it

Re: [VOTE] KIP-48 Support for delegation tokens as an authentication mechanism

Just wondering...how difficult would be it be to later add impersonation ( https://issues.apache.org/jira/browse/KAFKA-3712)? One use case would be a Kafka admin UI that would take action on the cluster on behalf different users.I suppose we could later add an "effectiveUserId" (in Unix

Re: KIP-122: Add a tool to Reset Consumer Group Offsets

Hey Jorge, Thanks for the KIP. I have some quick comments: - Should we allow user to use wildcard to reset offset of all groups for a given topic as well? - Should we allow user to specify timestamp per topic partition in the json file as well? - Should the script take some credential file to

Re: [DISCUSS] KIP-112: Handle disk failure for JBOD

Hey Eno, Thanks much for the comment! I still think the complexity added to Kafka is justified by its benefit. Let me provide my reasons below. 1) The additional logic is easy to understand and thus its complexity should be reasonable. On the broker side, it needs to catch exception when

Re: [VOTE] KIP-48 Support for delegation tokens as an authentication mechanism

+1 from me as well. On Tue, Feb 7, 2017 at 7:10 PM, Jason Gustafson wrote: > Looks like a great proposal! I noticed that key rotation is not included. > That may be reasonable for the initial work, but it might be nice to share > some thoughts on how that might work in the

[jira] [Commented] (KAFKA-4564) When the destination brokers are down or misconfigured in config, Streams should fail fast

[ https://issues.apache.org/jira/browse/KAFKA-4564?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15857193#comment-15857193 ] Matthias J. Sax commented on KAFKA-4564: [~gwenshap] You can configure it via `request.timeout.ms`

Re: [VOTE] KIP-48 Support for delegation tokens as an authentication mechanism

Hi Jun, How does it allow impersonation at the connection level? Looking at the KIP, the DelegationTokenRequest does not have an "Owner" field that can be set. The owner field of the DelegationTokenResponse says it's the "Kakfa Principal which requested the delegation token". For

[GitHub] kafka pull request #2514: MINOR: Add logging when commitSync fails in Stream...

GitHub user jmoney8080 opened a pull request: https://github.com/apache/kafka/pull/2514 MINOR: Add logging when commitSync fails in StreamTask When `consumer.commitSync` fails in `StreamTask`, the `CommitFailedException` bubbles up to

KIP-122: Add a tool to Reset Consumer Group Offsets

Hi all, I would like to propose a KIP to Add a tool to Reset Consumer Group Offsets. https://cwiki.apache.org/confluence/display/KAFKA/KIP-122%3A+Add+a+tool+to+Reset+Consumer+Group+Offsets Please, take a look at the proposal and share your feedback. Thanks, Jorge.

[jira] [Comment Edited] (KAFKA-4686) Null Message payload is shutting down broker

[ https://issues.apache.org/jira/browse/KAFKA-4686?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15856534#comment-15856534 ] Rodrigo Queiroz Saramago edited comment on KAFKA-4686 at 2/7/17 6:54 PM:

[jira] [Commented] (KAFKA-4738) Remove generic type of class ClientState

[ https://issues.apache.org/jira/browse/KAFKA-4738?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15856547#comment-15856547 ] Sharad commented on KAFKA-4738: --- [~mjsax] TaskAssignor takes two generic arguments.

[jira] [Comment Edited] (KAFKA-4738) Remove generic type of class ClientState

[ https://issues.apache.org/jira/browse/KAFKA-4738?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15856547#comment-15856547 ] Sharad edited comment on KAFKA-4738 at 2/7/17 6:58 PM: --- [~mjsax] TaskAssignor

Re: KIP-121 [Discuss]: Add KStream peek method

Far better! Thank you! On Tue, Feb 7, 2017 at 10:19 AM, Steven Schlansker wrote: > Thanks for the feedback. I improved the javadoc a bit, do you like it better? > > /** > * Perform an action on each record of {@code KStream}. > * This is a stateless

[jira] [Commented] (KAFKA-4686) Null Message payload is shutting down broker

[ https://issues.apache.org/jira/browse/KAFKA-4686?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15856592#comment-15856592 ] Jason Gustafson commented on KAFKA-4686: [~rodrigo.saramago] Thanks for the updates! One more

[jira] [Commented] (KAFKA-4686) Null Message payload is shutting down broker

[ https://issues.apache.org/jira/browse/KAFKA-4686?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15856602#comment-15856602 ] Rodrigo Queiroz Saramago commented on KAFKA-4686: - {noformat:title=broker001}

Re: [DISCUSS] KIP-117: Add a public AdministrativeClient API for Kafka admin operations

On Mon, Feb 6, 2017, at 13:54, Guozhang Wang wrote: > Some follow-up on 2) / 3) below. > > On Mon, Feb 6, 2017 at 11:21 AM, Colin McCabe wrote: > > > On Fri, Feb 3, 2017, at 16:25, Guozhang Wang wrote: > > > Thanks for the proposal Colin. A few comments below: > > > > Thanks

Re: [DISCUSS] KIP-117: Add a public AdministrativeClient API for Kafka admin operations

On Mon, Feb 6, 2017, at 14:05, Jianbin Wei wrote: > In the specify group information, can we also return information like > partition assignment for each member, the lag/offset of each > member/partition? It would be useful for Ops/Admin regarding the health > of the consumer group. This

Re: [DISCUSS] KIP-117: Add a public AdministrativeClient API for Kafka admin operations

Hi Hu Xi, On Mon, Feb 6, 2017, at 19:37, Hu Xi wrote: > Two things I want to confirm. Please advise. > > > 1. Seems the KIP only cares about topic management things. Is there any > plan for this KIP to merge the feature of what `GetOffsetShell` script > offers? Since a lot of people really want

  1   2   >