[jira] [Commented] (KAFKA-2804) Create / Update changelog topics upon state store initialization

2015-11-24 Thread Yi Pan (Data Infrastructure) (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2804?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15025815#comment-15025815 ] Yi Pan (Data Infrastructure) commented on KAFKA-2804: - [~guozhang], as for item 2

[jira] [Commented] (KAFKA-2887) TopicMetadataRequest creates topic if it does not exist

2015-11-24 Thread Yi Pan (Data Infrastructure) (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2887?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15025804#comment-15025804 ] Yi Pan (Data Infrastructure) commented on KAFKA-2887: - Agree with [~AWinterman

[jira] [Commented] (KAFKA-2887) TopicMetadataRequest creates topic if it does not exist

2015-11-24 Thread Yi Pan (Data Infrastructure) (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2887?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15025854#comment-15025854 ] Yi Pan (Data Infrastructure) commented on KAFKA-2887: - HI, [~harsha_ch], thanks

Re: [DISCUSS] KIP-28 - Add a transform client for data processing

2015-07-28 Thread Yi Pan
Hi, Aditya, {quote} - The KIP states that cmd line tools will be provided to deploy as a separate service. Is the proposed scope limited to providing a library with which makes it possible build stream-processing-as- a-service or provide such a service within Kafka itself? {quote} There has

Re: [DISCUSS] KIP-28 - Add a transform client for data processing

2015-07-28 Thread Yi Pan
Hi, Neha, {quote} We do hope to include a DSL since that is the most natural way of expressing stream processing operations on top of the processor client. The DSL layer should be equivalent to that provided by Spark streaming or Flink in terms of expressiveness though there will be differences

Re: [DISCUSS] KIP-28 - Add a transform client for data processing

2015-07-28 Thread Yi Pan
, 2015 at 11:31 PM, Yi Pan nickpa...@gmail.com wrote: Hi, Jay and all, Thanks for all your quick responses. I tried to summarize my thoughts here: - ConsumerRecord as stream processor API: * This KafkaProcessor API is targeted to receive the message from Kafka. So, to Yasuhiro's

Re: [DISCUSS] KIP-28 - Add a transform client for data processing

2015-07-27 Thread Yi Pan
/hard it is to use copycat with the transformer api in the same program. I think this is something we should work out as part of the prototyping. -Jay On Fri, Jul 24, 2015 at 12:57 AM, Yi Pan nickpa...@gmail.com wrote: Hi, Guozhang, Thanks for starting this. I took a quick look and had

Re: [DISCUSS] KIP-28 - Add a transform client for data processing

2015-07-24 Thread Yi Pan
Hi, Guozhang, Thanks for starting this. I took a quick look and had the following thoughts to share: - In the proposed KafkaProcessor API, there is no interface like Collector that allows users to send messages to. Why is that? Is the idea to initialize the producer once and re-use it in the

Re: Review Request 34607: Fix SAMZA-686: adding test for max changelog partition computation

2015-05-23 Thread Yi Pan (Data Infrastructure)
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/34607/#review85083 --- Ship it! Ship It! - Yi Pan (Data Infrastructure) On May 22