[GitHub] kafka pull request: KAFKA-2366 [WIP]; Copycat

GitHub user ewencp opened a pull request: https://github.com/apache/kafka/pull/99 KAFKA-2366 [WIP]; Copycat This is an initial patch implementing the basics of Copycat for KIP-26. The intent here is to start a review of the key pieces of the core API and get a reasonably fu

[jira] [Commented] (KAFKA-2366) Initial patch for Copycat

[ https://issues.apache.org/jira/browse/KAFKA-2366?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14642362#comment-14642362 ] ASF GitHub Bot commented on KAFKA-2366: --- GitHub user ewencp opened a pull request:

Re: [DISCUSS] KIP-28 - Add a transform client for data processing

Hi, Jay and all, Thanks for all your quick responses. I tried to summarize my thoughts here: - ConsumerRecord as stream processor API: * This KafkaProcessor API is targeted to receive the message from Kafka. So, to Yasuhiro's join/transformation example, any join/transformation results that a

[jira] [Commented] (KAFKA-2377) Add copycat system tests

[ https://issues.apache.org/jira/browse/KAFKA-2377?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14642336#comment-14642336 ] Gwen Shapira commented on KAFKA-2377: - Two connectors that are not useful for anything

[jira] [Created] (KAFKA-2379) Add Copycat documentation

Ewen Cheslack-Postava created KAFKA-2379: Summary: Add Copycat documentation Key: KAFKA-2379 URL: https://issues.apache.org/jira/browse/KAFKA-2379 Project: Kafka Issue Type: Sub-task

[jira] [Issue Comment Deleted] (KAFKA-2365) Copycat checklist

[ https://issues.apache.org/jira/browse/KAFKA-2365?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gwen Shapira updated KAFKA-2365: Comment: was deleted (was: I keep waiting for [~ewencp] to create the "log connector" jira so I can

[jira] [Commented] (KAFKA-2374) Implement Copycat log/file connector

[ https://issues.apache.org/jira/browse/KAFKA-2374?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14642333#comment-14642333 ] Gwen Shapira commented on KAFKA-2374: - Top log features missing from Flume (more or le

[jira] [Created] (KAFKA-2378) Add Copycat embedded API

Ewen Cheslack-Postava created KAFKA-2378: Summary: Add Copycat embedded API Key: KAFKA-2378 URL: https://issues.apache.org/jira/browse/KAFKA-2378 Project: Kafka Issue Type: Sub-task

[jira] [Created] (KAFKA-2377) Add copycat system tests

Ewen Cheslack-Postava created KAFKA-2377: Summary: Add copycat system tests Key: KAFKA-2377 URL: https://issues.apache.org/jira/browse/KAFKA-2377 Project: Kafka Issue Type: Sub-task

[jira] [Created] (KAFKA-2376) Add Copycat metrics

Ewen Cheslack-Postava created KAFKA-2376: Summary: Add Copycat metrics Key: KAFKA-2376 URL: https://issues.apache.org/jira/browse/KAFKA-2376 Project: Kafka Issue Type: Sub-task

[jira] [Created] (KAFKA-2375) Implement elasticsearch Copycat sink connector

Ewen Cheslack-Postava created KAFKA-2375: Summary: Implement elasticsearch Copycat sink connector Key: KAFKA-2375 URL: https://issues.apache.org/jira/browse/KAFKA-2375 Project: Kafka

[jira] [Updated] (KAFKA-2374) Implement Copycat log/file connector

[ https://issues.apache.org/jira/browse/KAFKA-2374?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ewen Cheslack-Postava updated KAFKA-2374: - Description: This is a good baseline connector that has zero dependencies and work

[jira] [Commented] (KAFKA-2365) Copycat checklist

[ https://issues.apache.org/jira/browse/KAFKA-2365?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14642323#comment-14642323 ] Gwen Shapira commented on KAFKA-2365: - I keep waiting for [~ewencp] to create the "log

[jira] [Created] (KAFKA-2374) Implement Copycat log/file connector

Ewen Cheslack-Postava created KAFKA-2374: Summary: Implement Copycat log/file connector Key: KAFKA-2374 URL: https://issues.apache.org/jira/browse/KAFKA-2374 Project: Kafka Issue Type

[jira] [Created] (KAFKA-2373) Copycat distributed offset storage

Ewen Cheslack-Postava created KAFKA-2373: Summary: Copycat distributed offset storage Key: KAFKA-2373 URL: https://issues.apache.org/jira/browse/KAFKA-2373 Project: Kafka Issue Type:

[jira] [Created] (KAFKA-2372) Copycat distributed config storage

Ewen Cheslack-Postava created KAFKA-2372: Summary: Copycat distributed config storage Key: KAFKA-2372 URL: https://issues.apache.org/jira/browse/KAFKA-2372 Project: Kafka Issue Type:

[jira] [Created] (KAFKA-2371) Add distributed coordinator implementation for Copycat

Ewen Cheslack-Postava created KAFKA-2371: Summary: Add distributed coordinator implementation for Copycat Key: KAFKA-2371 URL: https://issues.apache.org/jira/browse/KAFKA-2371 Project: Kafka

[jira] [Commented] (KAFKA-2365) Copycat checklist

[ https://issues.apache.org/jira/browse/KAFKA-2365?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14642318#comment-14642318 ] Gwen Shapira commented on KAFKA-2365: - I see the release is 0.8.3 - is the intent to g

[jira] [Created] (KAFKA-2370) Add pause/unpause connector support

Ewen Cheslack-Postava created KAFKA-2370: Summary: Add pause/unpause connector support Key: KAFKA-2370 URL: https://issues.apache.org/jira/browse/KAFKA-2370 Project: Kafka Issue Type:

[jira] [Created] (KAFKA-2369) Add Copycat REST API

Ewen Cheslack-Postava created KAFKA-2369: Summary: Add Copycat REST API Key: KAFKA-2369 URL: https://issues.apache.org/jira/browse/KAFKA-2369 Project: Kafka Issue Type: Sub-task

[jira] [Created] (KAFKA-2368) Add Copycat standalone CLI

Ewen Cheslack-Postava created KAFKA-2368: Summary: Add Copycat standalone CLI Key: KAFKA-2368 URL: https://issues.apache.org/jira/browse/KAFKA-2368 Project: Kafka Issue Type: Sub-task

[jira] [Created] (KAFKA-2367) Add Copycat runtime API

Ewen Cheslack-Postava created KAFKA-2367: Summary: Add Copycat runtime API Key: KAFKA-2367 URL: https://issues.apache.org/jira/browse/KAFKA-2367 Project: Kafka Issue Type: Sub-task

[jira] [Created] (KAFKA-2366) Initial patch for Copycat

Ewen Cheslack-Postava created KAFKA-2366: Summary: Initial patch for Copycat Key: KAFKA-2366 URL: https://issues.apache.org/jira/browse/KAFKA-2366 Project: Kafka Issue Type: Sub-task

[jira] [Updated] (KAFKA-2365) Copycat checklist

[ https://issues.apache.org/jira/browse/KAFKA-2365?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ewen Cheslack-Postava updated KAFKA-2365: - Labels: feature (was: ) Description: This covers the development plan fo

[jira] [Updated] (KAFKA-2100) Client Error doesn't preserve or display original server error code when it is an unknown code

[ https://issues.apache.org/jira/browse/KAFKA-2100?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Jacot updated KAFKA-2100: --- Status: Patch Available (was: Open) > Client Error doesn't preserve or display original server error

[jira] [Commented] (KAFKA-2100) Client Error doesn't preserve or display original server error code when it is an unknown code

[ https://issues.apache.org/jira/browse/KAFKA-2100?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14642296#comment-14642296 ] David Jacot commented on KAFKA-2100: Thank you for your feedback. I just uploaded a pa

[jira] [Updated] (KAFKA-2365) Copycat checklist

[ https://issues.apache.org/jira/browse/KAFKA-2365?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ewen Cheslack-Postava updated KAFKA-2365: - Issue Type: New Feature (was: Bug) > Copycat checklist > - > >

[jira] [Created] (KAFKA-2365) Copycat checklist

Ewen Cheslack-Postava created KAFKA-2365: Summary: Copycat checklist Key: KAFKA-2365 URL: https://issues.apache.org/jira/browse/KAFKA-2365 Project: Kafka Issue Type: Bug R

[jira] [Updated] (KAFKA-2100) Client Error doesn't preserve or display original server error code when it is an unknown code

[ https://issues.apache.org/jira/browse/KAFKA-2100?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Jacot updated KAFKA-2100: --- Attachment: KAFKA-2100-2.patch > Client Error doesn't preserve or display original server error code w

[jira] [Commented] (KAFKA-1690) new java producer needs ssl support as a client

[ https://issues.apache.org/jira/browse/KAFKA-1690?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14642237#comment-14642237 ] Dong Lin commented on KAFKA-1690: - [~harsha_ch] Thanks much for letting us know. > new ja

[jira] [Comment Edited] (KAFKA-2350) Add KafkaConsumer pause capability

[ https://issues.apache.org/jira/browse/KAFKA-2350?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14642229#comment-14642229 ] Neha Narkhede edited comment on KAFKA-2350 at 7/27/15 3:55 AM: -

[jira] [Commented] (KAFKA-2350) Add KafkaConsumer pause capability

[ https://issues.apache.org/jira/browse/KAFKA-2350?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14642229#comment-14642229 ] Neha Narkhede commented on KAFKA-2350: -- Few thoughts: bq. 1. Add pause/unpause bq. 2.

Re: Listen interface

solved it by removing host.name setting in server.properties.. now kafka is listening on all interfaces, as i expected... thank you all On Mon, Jul 27, 2015 at 8:03 AM, madhavan kumar wrote: > hi all, >On my ubuntu 14.04 machine, i have a virt-manager/libvirtd private > network running. And

Listen interface

hi all, On my ubuntu 14.04 machine, i have a virt-manager/libvirtd private network running. And that has made a virtual network interface, virbr0 with ip 192.168.122.1. in my kafka broker settings, i have set the host.name property to the host name of the machine. When i do netstat -lntp |

[jira] [Commented] (KAFKA-2100) Client Error doesn't preserve or display original server error code when it is an unknown code

[ https://issues.apache.org/jira/browse/KAFKA-2100?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14642179#comment-14642179 ] Ewen Cheslack-Postava commented on KAFKA-2100: -- Option #2 looks good to me as

Re: New Producer and "acks" configuration

Aha! Yes, I was missing the part with the dummy response. Thank you! Gwen On Sun, Jul 26, 2015 at 2:17 PM, Ewen Cheslack-Postava wrote: > It's different because it changes whether the client waits for the response > from the broker at all. Take a look at NetworkClient.handleCompletedSends, > wh

Re: New Producer and "acks" configuration

It's different because it changes whether the client waits for the response from the broker at all. Take a look at NetworkClient.handleCompletedSends, which fills in dummy responses when a response is not expected (and that flag gets set via Sender.produceRequest using acks != 0 as a flag to Client

Re: New Producer and "acks" configuration

What bugs me is that even with acks = 0, the broker will append to local log before responding (unless I'm misreading the code), so I don't see why a client with acks = 0 will be any faster. Unless the client chooses to not wait for response, which is orthogonal to acks parameter. On Mon, Jul 20,

[jira] [Resolved] (KAFKA-2362) kafka-consumer-perf-test.sh got a negative result

[ https://issues.apache.org/jira/browse/KAFKA-2362?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ewen Cheslack-Postava resolved KAFKA-2362. -- Resolution: Duplicate Assignee: Ewen Cheslack-Postava Fix Version

Re: [DISCUSS] KIP-27 - Conditional Publish

So I had another look at the 'Idempotent Producer' proposal this afternoon, and made a few notes on how I think they compare; if I've made any mistakes, I'd be delighted if someone with more context on the idempotent producer design would correct me. As a first intuition, you can think of the 'con