[ANNOUNCE] New committer: Ismael Juma

The PMC for Apache Kafka has invited Ismael Juma to join as a committer and we are pleased to announce that he has accepted! Ismael has contributed 121 commits to a wide range of areas, notably within the security and the network layer. His

Re: Can't reprocess events

Please check your retention policy. I think you are expecting data beyond your retention period. Regards, Ashiq > On 25-Apr-2016, at 9:26 PM, Ole Kozaczenko > wrote: > > Hey guys, > > I have the following system: Kafka (0.8.2.1) ist getting messages per HTTP

Re: [VOTE] KIP-57: Interoperable LZ4 Framing

+1 On Mon, Apr 25, 2016 at 2:26 PM, Dana Powers wrote: > Hi all, > > Initiating a vote thread because the KIP-57 proposal is specific to > the 0.10 release. > > KIP-57 can be accessed here: >

[jira] [Updated] (KAFKA-3621) Add tests for ApiVersionRequest/Response

[ https://issues.apache.org/jira/browse/KAFKA-3621?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashish K Singh updated KAFKA-3621: -- Priority: Critical (was: Major) > Add tests for ApiVersionRequest/Response >

[jira] [Updated] (KAFKA-3621) Add tests for ApiVersionRequest/Response

[ https://issues.apache.org/jira/browse/KAFKA-3621?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashish K Singh updated KAFKA-3621: -- Fix Version/s: 0.10.0.0 > Add tests for ApiVersionRequest/Response >

[jira] [Updated] (KAFKA-3621) Add tests for ApiVersionRequest/Response

[ https://issues.apache.org/jira/browse/KAFKA-3621?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashish K Singh updated KAFKA-3621: -- Affects Version/s: 0.10.0.0 > Add tests for ApiVersionRequest/Response >

[jira] [Created] (KAFKA-3621) Add tests for ApiVersionRequest/Response

Ashish K Singh created KAFKA-3621: - Summary: Add tests for ApiVersionRequest/Response Key: KAFKA-3621 URL: https://issues.apache.org/jira/browse/KAFKA-3621 Project: Kafka Issue Type: Task

[GitHub] kafka pull request: KAFKA-2825, KAFKA-2851: Controller failover te...

Github user apovzner closed the pull request at: https://github.com/apache/kafka/pull/570 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[jira] [Commented] (KAFKA-2825) Add controller failover to existing replication tests

[ https://issues.apache.org/jira/browse/KAFKA-2825?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15257237#comment-15257237 ] ASF GitHub Bot commented on KAFKA-2825: --- Github user apovzner closed the pull request at:

[GitHub] kafka pull request: KAFKA-3614: Consolidate duplicate code in KGro...

Github user vahidhashemian closed the pull request at: https://github.com/apache/kafka/pull/1268 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the

[jira] [Commented] (KAFKA-3614) Consolidate duplicate code in KGroupedTableImpl

[ https://issues.apache.org/jira/browse/KAFKA-3614?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15257190#comment-15257190 ] ASF GitHub Bot commented on KAFKA-3614: --- Github user vahidhashemian closed the pull request at:

Build failed in Jenkins: kafka-trunk-jdk8 #547

See Changes: [wangguoz] KAFKA-3619: File handles are leaked on .lock files of -- [...truncated 4875 lines...] kafka.zk.ZKPathTest > testCreateEphemeralPathExists PASSED kafka.zk.ZKPathTest >

[jira] [Commented] (KAFKA-3614) Consolidate duplicate code in KGroupedTableImpl

[ https://issues.apache.org/jira/browse/KAFKA-3614?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15257125#comment-15257125 ] ASF GitHub Bot commented on KAFKA-3614: --- GitHub user vahidhashemian opened a pull request:

[GitHub] kafka pull request: KAFKA-3614: Consolidate duplicate code in KGro...

GitHub user vahidhashemian opened a pull request: https://github.com/apache/kafka/pull/1268 KAFKA-3614: Consolidate duplicate code in KGroupedTableImpl Two methods aggregate() and reduce() share common code that is consolidated in this patch. You can merge this pull request into a

[jira] [Assigned] (KAFKA-3614) Consolidate duplicate code in KGroupedTableImpl

[ https://issues.apache.org/jira/browse/KAFKA-3614?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vahid Hashemian reassigned KAFKA-3614: -- Assignee: Vahid Hashemian > Consolidate duplicate code in KGroupedTableImpl >

Build failed in Jenkins: kafka-trunk-jdk7 #1216

See Changes: [wangguoz] KAFKA-3619: File handles are leaked on .lock files of -- [...truncated 454 lines...] kafka.log.BrokerCompressionTest > testBrokerSideCompression[19] PASSED

Re: [VOTE] KIP-57: Interoperable LZ4 Framing

+1 (non-binding) 2016-04-25 14:26 GMT-07:00 Dana Powers : > Hi all, > > Initiating a vote thread because the KIP-57 proposal is specific to > the 0.10 release. > > KIP-57 can be accessed here: > < >

[jira] [Updated] (KAFKA-3620) Clean up Protocol class.

[ https://issues.apache.org/jira/browse/KAFKA-3620?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashish K Singh updated KAFKA-3620: -- Description: This came up on PR of KAFKA-3307. Below is excerpt. {quote} With the versioning

[jira] [Created] (KAFKA-3620) Clean up Protocol class.

Ashish K Singh created KAFKA-3620: - Summary: Clean up Protocol class. Key: KAFKA-3620 URL: https://issues.apache.org/jira/browse/KAFKA-3620 Project: Kafka Issue Type: Improvement

[VOTE] KIP-57: Interoperable LZ4 Framing

Hi all, Initiating a vote thread because the KIP-57 proposal is specific to the 0.10 release. KIP-57 can be accessed here: . The related JIRA is https://issues.apache.org/jira/browse/KAFKA-3160 and working

Re: No Heartbeat request on commit

Hi Vinay, This was fixed via https://issues.apache.org/jira/browse/KAFKA-3470 (will be part of 0.10.0.0). Ismael On Mon, Apr 25, 2016 at 1:52 PM, vinay sharma wrote: > Hello, > > I am using client API 0.9.0.1 and facing an issue. As per my logs it seems > that on

Re: KIP-57: Interoperable LZ4 Framing

Hi Dana, Thank you for proposing this fix. It looks great to me (and LinkedIn, who are running trunk confirmed that they did not use LZ4 yet). Since for 0.10.0 time is of essence, do you mind starting a voting thread in parallel? Gwen On Mon, Apr 25, 2016 at 9:30 AM, Dana Powers

No Heartbeat request on commit

Hello, I am using client API 0.9.0.1 and facing an issue. As per my logs it seems that on each commitSync(Offsets) a heartbeat request is sent but after a metada refresh request till next poll(), commits do not send any hearbeat request. KafkaConsumers i create sometimes get session time out due

[GitHub] kafka pull request: KAFKA-3619 - File handles are leaked on .lock ...

Github user asfgit closed the pull request at: https://github.com/apache/kafka/pull/1267 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[jira] [Updated] (KAFKA-3619) State lock file handle leaks

[ https://issues.apache.org/jira/browse/KAFKA-3619?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guozhang Wang updated KAFKA-3619: - Assignee: Greg Fodor (was: Guozhang Wang) > State lock file handle leaks >

[jira] [Updated] (KAFKA-3619) State lock file handle leaks

[ https://issues.apache.org/jira/browse/KAFKA-3619?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guozhang Wang updated KAFKA-3619: - Reviewer: Guozhang Wang > State lock file handle leaks > > >

[jira] [Updated] (KAFKA-3567) Add --security-protocol option to console consumer and producer

[ https://issues.apache.org/jira/browse/KAFKA-3567?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bharat Viswanadham updated KAFKA-3567: -- Summary: Add --security-protocol option to console consumer and producer (was: Add

[jira] [Assigned] (KAFKA-3567) Add --security-protocol option console tools

[ https://issues.apache.org/jira/browse/KAFKA-3567?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bharat Viswanadham reassigned KAFKA-3567: - Assignee: Bharat Viswanadham (was: Sriharsha Chintalapani) > Add

Jenkins build is back to normal : kafka-trunk-jdk7 #1215

See

Build failed in Jenkins: kafka-trunk-jdk8 #546

See Changes: [wangguoz] MINOR: add unit test for KGroupedTable.count -- [...truncated 2798 lines...] kafka.admin.AdminRackAwareTest > testAssignmentWithRackAware PASSED

[jira] [Updated] (KAFKA-3619) State lock file handle leaks

[ https://issues.apache.org/jira/browse/KAFKA-3619?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Greg Fodor updated KAFKA-3619: -- Description: The .lock files in the state store directories do not seem to be having their file

Re: Kafka Streams file handle leak

Great, thanks: https://issues.apache.org/jira/browse/KAFKA-3619 On Mon, Apr 25, 2016 at 11:13 AM, Guozhang Wang wrote: > Thanks for reporting that Greg. > > I have just added you as a Kafka contributor so you should be able to > create JIRAs now. Could you create a new JIRA

[jira] [Created] (KAFKA-3619) State lock file handle leaks

Greg Fodor created KAFKA-3619: - Summary: State lock file handle leaks Key: KAFKA-3619 URL: https://issues.apache.org/jira/browse/KAFKA-3619 Project: Kafka Issue Type: Bug Components:

Re: Kafka missing from ASF Jira?

Yep, that worked. I was able to create jiras a few weeks ago, not sure if something changed. Thanks! On Mon, Apr 25, 2016 at 11:43 AM, BigData dev wrote: > Hi Wang, > Me too facing the same issue. > Can you provide access to me too. > > > Regards, > Bharat > > > On Mon,

[jira] [Commented] (KAFKA-3565) Producer's throughput lower with compressed data after KIP-31/32

[ https://issues.apache.org/jira/browse/KAFKA-3565?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15256788#comment-15256788 ] Ismael Juma commented on KAFKA-3565: Thanks for your work on this [~becket_qin]. I finally have a bit

Re: Kafka missing from ASF Jira?

Hi Wang, Me too facing the same issue. Can you provide access to me too. Regards, Bharat On Mon, Apr 25, 2016 at 11:21 AM, Guozhang Wang wrote: > Greg, > > Could you try again now? I think even non-contributor should be able to > create Kafka JIRAs, but they may not be

Re: Kafka missing from ASF Jira?

Greg, Could you try again now? I think even non-contributor should be able to create Kafka JIRAs, but they may not be able to assign JIRAs to themselves. Anyways, have just added you to the contributor list. Guozhang On Mon, Apr 25, 2016 at 9:14 AM, Greg Fodor wrote: > If

[GitHub] kafka pull request: MINOR: add unit test for KGroupedTable.count

Github user asfgit closed the pull request at: https://github.com/apache/kafka/pull/1255 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

Re: Kafka Streams file handle leak

Thanks for reporting that Greg. I have just added you as a Kafka contributor so you should be able to create JIRAs now. Could you create a new JIRA for this and change the RB title with the JIRA number? Thanks! Guozhang On Mon, Apr 25, 2016 at 12:13 AM, Greg Fodor wrote: >

Re: wiki privs

Dana, I have grant you the permission. You should be good to go now. Guozhang On Sun, Apr 24, 2016 at 11:19 PM, Dana Powers wrote: > I'd like to create a new KIP page on the wiki, but I can't figure out > how to create a new page or edit an existing page. Do I need

[jira] [Commented] (KAFKA-2693) Run relevant ducktape tests with SASL/PLAIN and multiple mechanisms

[ https://issues.apache.org/jira/browse/KAFKA-2693?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15256679#comment-15256679 ] Rajini Sivaram commented on KAFKA-2693: --- Include tests for - SASL/PLAIN - Multiple mechanisms -

Re: [VOTE] KIP-50: Move Authorizer to o.a.k.common package - Round 2

+1 (non-binding) On Mon, Apr 25, 2016 at 12:30 PM, Gwen Shapira wrote: > +1 > > On Mon, Apr 25, 2016 at 10:04 AM, Ashish Singh > wrote: > > Hey Guys, > > > > I would like to re-initiate the voting process for KIP-50: Move > Authorizer > > to o.a.k.common

Re: [VOTE] KIP-35: Retrieve protocol version - Round 2

Ismael, thanks for the suggestions, added a bit more text to be more informative. Note that changes are made only to text and examples, the proposal remains as it was before voting was started. Magnus has already taken care of Rajini's comment. KIP-35 has passed with +3 (binding) and +6

Re: [VOTE] KIP-50: Move Authorizer to o.a.k.common package - Round 2

+1 On Mon, Apr 25, 2016 at 10:04 AM, Ashish Singh wrote: > Hey Guys, > > I would like to re-initiate the voting process for KIP-50: Move Authorizer > to o.a.k.common package. KIP-50 is available here >

Re: [VOTE] KIP-50: Move Authorizer to o.a.k.common package - Round 2

+1 -Jay On Mon, Apr 25, 2016 at 10:04 AM, Ashish Singh wrote: > Hey Guys, > > I would like to re-initiate the voting process for KIP-50: Move Authorizer > to o.a.k.common package. KIP-50 is available here > < >

[jira] [Created] (KAFKA-3618) Handle ApiVersionRequest before SASL handshake

Rajini Sivaram created KAFKA-3618: - Summary: Handle ApiVersionRequest before SASL handshake Key: KAFKA-3618 URL: https://issues.apache.org/jira/browse/KAFKA-3618 Project: Kafka Issue Type:

[jira] [Created] (KAFKA-3617) Add unit tests for SASL authentication

Rajini Sivaram created KAFKA-3617: - Summary: Add unit tests for SASL authentication Key: KAFKA-3617 URL: https://issues.apache.org/jira/browse/KAFKA-3617 Project: Kafka Issue Type: Test

[jira] [Updated] (KAFKA-2693) Run relevant ducktape tests with SASL/PLAIN and multiple mechanisms

[ https://issues.apache.org/jira/browse/KAFKA-2693?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajini Sivaram updated KAFKA-2693: -- Fix Version/s: 0.10.0.0 Summary: Run relevant ducktape tests with SASL/PLAIN and

[VOTE] KIP-50: Move Authorizer to o.a.k.common package - Round 2

Hey Guys, I would like to re-initiate the voting process for KIP-50: Move Authorizer to o.a.k.common package. KIP-50 is available here . Associated PR is available here

[GitHub] kafka pull request: KAFKA-2658: Add PLAIN mechanism to SASL implem...

Github user rajinisivaram closed the pull request at: https://github.com/apache/kafka/pull/341 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature

Re: KIP-57: Interoperable LZ4 Framing

Thanks for submitting the KIP Dana. I think it would make a lot of sense to include this change as part of the bump to message format 1 (although it's a bit tight given the current release plan). Ismael On Mon, Apr 25, 2016 at 9:30 AM, Dana Powers wrote: > Hi all, > >

KIP-57: Interoperable LZ4 Framing

Hi all, I've written up a new KIP based on KAFKA-3160 / fixing LZ4 framing. The write-up is here: https://cwiki.apache.org/confluence/display/KAFKA/KIP-57+-+Interoperable+LZ4+Framing Please take a look if you are using LZ4 compression or are interested in framing specs. One question: does

Re: Kafka missing from ASF Jira?

If I go there, and hit "Create", the Project List at the top has lots of apache projects does not contain any items containing 'kafka' :( On Mon, Apr 25, 2016 at 12:35 AM, Liquan Pei wrote: > Hi Greg, > > Can you try this link? >

Can't reprocess events

Hey guys, I have the following system: Kafka (0.8.2.1) ist getting messages per HTTP and a Spark parser is consuming them. Now on the weekend the parser broke, and some events were not processed. How do I reprocess them? I already tried spawning a 2nd parser, with auto-offset "smallest" and

Re: [DISCUSS] KIP-43: Kafka SASL enhancements

Magnus, I have updated KIP-43 to include a section with the handshake request/response format. Have also added some more text to distinguish the actual authentication flow from the Kafka handshake/request flow. Thank you, Rajini On Mon, Apr 25, 2016 at 3:41 AM, Magnus Edenhill

[jira] [Commented] (KAFKA-3616) Make kafka producers/consumers injectable for KafkaStreams

[ https://issues.apache.org/jira/browse/KAFKA-3616?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15256190#comment-15256190 ] Yuto Kawamura commented on KAFKA-3616: -- > About "reliable message delivery", out of curiosity could

[jira] [Issue Comment Deleted] (KAFKA-3616) Make kafka producers/consumers injectable for KafkaStreams

[ https://issues.apache.org/jira/browse/KAFKA-3616?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuto Kawamura updated KAFKA-3616: - Comment: was deleted (was: > About "reliable message delivery", out of curiosity could you

[jira] [Commented] (KAFKA-3616) Make kafka producers/consumers injectable for KafkaStreams

[ https://issues.apache.org/jira/browse/KAFKA-3616?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15256187#comment-15256187 ] Yuto Kawamura commented on KAFKA-3616: -- > About "reliable message delivery", out of curiosity could

Re: Kafka missing from ASF Jira?

Hi Greg, Can you try this link? https://issues.apache.org/jira/browse/KAFKA/?selectedTab=com.atlassian.jira.jira-projects-plugin:summary-panel Thanks, Liquan On Mon, Apr 25, 2016 at 12:04 AM, Greg Fodor wrote: > I am trying to file a bug, but when I go to create a ticket on

Kafka Streams file handle leak

Apologies for a second posting to the list, but the ASF JIRA is preventing me from creating KAFKA- tickets so I wanted to report this issue. This seems to be a critical bug with KStreams, the .lock files in the state store directories do not seem to be having their file handles freed (despite the

[GitHub] kafka pull request: File handles are leaked on .lock files

GitHub user gfodor opened a pull request: https://github.com/apache/kafka/pull/1267 File handles are leaked on .lock files Kafka Streams seems to hold file handles on the `.lock` files for the state dirs, resulting in an explosion of filehandles over time. Running `lsof` shows the

Kafka missing from ASF Jira?

I am trying to file a bug, but when I go to create a ticket on the ASF Jira, Kafka is not visible in the list of projects in the first field of the ticket. I see the hundred+ other Apache projects, but no Kafka :(

wiki privs

I'd like to create a new KIP page on the wiki, but I can't figure out how to create a new page or edit an existing page. Do I need certain privileges on my user account to be able to do this? Thanks, -Dana