Build failed in Jenkins: kafka-trunk-jdk8 #301

See Changes: [me] KAFKA-2071: Replace Producer Request/Response with their -- Started by an SCM change [EnvInject] - Loading node environment variables. Building remotely on H11 (Ubuntu ubuntu)

Re: Pluggable Log Compaction Policy

Hello Bill, Just to clarify your use case, is your "log compaction" executed manually, or it is triggered periodically like the current log cleaning by-key does? If it is the latter case, how will you advance the "threshold transaction_id" each time when it executes? Guozhang On Wed, Jan 20,

Re: Pluggable Log Compaction Policy

Compaction would be performed in the same manner as it is currently. There is a predicate applied in the "shouldRetainMessage" function in LogCleaner; ultimately we just want to be able to swap a custom implementation of that particular method in. Nothing else in the compaction codepath would

[jira] [Commented] (KAFKA-3066) Add Demo Examples for Kafka Streams

[ https://issues.apache.org/jira/browse/KAFKA-3066?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15110168#comment-15110168 ] ASF GitHub Bot commented on KAFKA-3066: --- GitHub user guozhangwang opened a pull request:

[GitHub] kafka pull request: KAFKA-3066: Demo Examples for Kafka Streams

GitHub user guozhangwang opened a pull request: https://github.com/apache/kafka/pull/797 KAFKA-3066: Demo Examples for Kafka Streams You can merge this pull request into a Git repository by running: $ git pull https://github.com/guozhangwang/kafka K3066 Alternatively you can

Re: Pluggable Log Compaction Policy

So do you need to periodically update the key-value pairs to "advance the threshold for each topic"? Guozhang On Wed, Jan 20, 2016 at 5:51 PM, Bill Warshaw wrote: > Compaction would be performed in the same manner as it is currently. There > is a predicate applied in

[GitHub] kafka pull request: 0.8.2

GitHub user chetansomani opened a pull request: https://github.com/apache/kafka/pull/796 0.8.2 You can merge this pull request into a Git repository by running: $ git pull https://github.com/apache/kafka 0.8.2 Alternatively you can review and apply these changes as the patch

Re: Pluggable Log Compaction Policy

For our particular use case, we would need to. This proposal is really two separate pieces: custom log compaction policy, and the ability to set arbitrary key-value pairs in a Topic configuration. I believe that Kafka's current behavior of throwing errors when it encounters configuration keys

[jira] [Created] (KAFKA-3127) Output unix time when use KafkaCSVMetricsReporter

Sasaki Toru created KAFKA-3127: -- Summary: Output unix time when use KafkaCSVMetricsReporter Key: KAFKA-3127 URL: https://issues.apache.org/jira/browse/KAFKA-3127 Project: Kafka Issue Type:

[jira] [Updated] (KAFKA-3127) Output unix time when use KafkaCSVMetricsReporter

[ https://issues.apache.org/jira/browse/KAFKA-3127?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sasaki Toru updated KAFKA-3127: --- Description: When we use KafkaCSVMetricsReporter, it outputs the time from started up Kafka process.

[jira] [Commented] (KAFKA-3127) Output unix time when use KafkaCSVMetricsReporter

[ https://issues.apache.org/jira/browse/KAFKA-3127?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15110220#comment-15110220 ] Sasaki Toru commented on KAFKA-3127: Current Kafka uses Yammer Metrics for reporting metrics, and it

[GitHub] kafka pull request: Minor: "-daemon" option is not mentioned in us...

Github user sasakitoa closed the pull request at: https://github.com/apache/kafka/pull/634 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[jira] [Commented] (KAFKA-3083) a soft failure in controller may leave a topic partition in an inconsistent state

[ https://issues.apache.org/jira/browse/KAFKA-3083?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15109838#comment-15109838 ] Jun Rao commented on KAFKA-3083: [~mgharat], in step 4) ii), if the session expires, you won't get

[jira] [Commented] (KAFKA-1397) delete topic is not working

[ https://issues.apache.org/jira/browse/KAFKA-1397?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15109040#comment-15109040 ] Mayuresh Gharat commented on KAFKA-1397: Can you paste the server side logs? > delete topic is

[jira] [Created] (KAFKA-3124) Update protocol wiki page to reflect latest request/response formats

Joel Koshy created KAFKA-3124: - Summary: Update protocol wiki page to reflect latest request/response formats Key: KAFKA-3124 URL: https://issues.apache.org/jira/browse/KAFKA-3124 Project: Kafka

Pluggable Log Compaction Policy

Hello, I'm working on a team that is starting to use Kafka as a distributed transaction log for a set of in-memory databases which can be replicated across nodes. We decided to use Kafka instead of Bookkeeper for a variety of reasons, but there are a couple spots where Kafka is not a perfect

Re: Pluggable Log Compaction Policy

Hi Bill, Have you looked at: http://kafka.apache.org/documentation.html#compaction It supports deletes so if you keyed by transaction_id you could, in theory, delete them. Cheers, Damian On 20 January 2016 at 18:35, Bill Warshaw wrote: > Hello, > > I'm working on a

Re: [jira] [Updated] (KAFKA-2071) Replace Produce Request/Response with their org.apache.kafka.common.requests equivalents

unsubscribe On Wed, Jan 20, 2016 at 2:09 PM, David Jacot (JIRA) wrote: > > [ > https://issues.apache.org/jira/browse/KAFKA-2071?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel > ] > > David Jacot updated KAFKA-2071: > --- >

[jira] [Commented] (KAFKA-3006) Make collection default container type for sequences in the consumer API

[ https://issues.apache.org/jira/browse/KAFKA-3006?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15109230#comment-15109230 ] Grant Henke commented on KAFKA-3006: Is this ready to be committed? Did we make a final decision? do

Re: [jira] [Commented] (KAFKA-1397) delete topic is not working

unsubscribe On Wed, Jan 20, 2016 at 1:02 PM, Mayuresh Gharat (JIRA) wrote: > > [ > https://issues.apache.org/jira/browse/KAFKA-1397?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15109040#comment-15109040 > ] > > Mayuresh Gharat commented on

[jira] [Updated] (KAFKA-2071) Replace Produce Request/Response with their org.apache.kafka.common.requests equivalents

[ https://issues.apache.org/jira/browse/KAFKA-2071?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Jacot updated KAFKA-2071: --- Status: In Progress (was: Patch Available) > Replace Produce Request/Response with their

[jira] [Updated] (KAFKA-2071) Replace Produce Request/Response with their org.apache.kafka.common.requests equivalents

[ https://issues.apache.org/jira/browse/KAFKA-2071?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Jacot updated KAFKA-2071: --- Status: Patch Available (was: In Progress) > Replace Produce Request/Response with their

Re: [GitHub] kafka pull request: WIP: KStream,Type and Serialization

unsubscribe On Wed, Jan 20, 2016 at 3:28 PM, ymatsuda wrote: > GitHub user ymatsuda opened a pull request: > > https://github.com/apache/kafka/pull/794 > > WIP: KStream,Type and Serialization > > @guozhangwang > > > You can merge this pull request into a Git

[GitHub] kafka pull request: WIP: KStream,Type and Serialization

GitHub user ymatsuda opened a pull request: https://github.com/apache/kafka/pull/794 WIP: KStream,Type and Serialization @guozhangwang You can merge this pull request into a Git repository by running: $ git pull https://github.com/ymatsuda/kafka type_and_serdes

[jira] [Commented] (KAFKA-3121) KStream DSL API Improvement

[ https://issues.apache.org/jira/browse/KAFKA-3121?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15109381#comment-15109381 ] ASF GitHub Bot commented on KAFKA-3121: --- GitHub user guozhangwang opened a pull request:

Re: Pluggable Log Compaction Policy

Damian, I appreciate your quick response. Our transaction_id is incrementing for each transaction, so we will only ever have one message in Kafka with a given transaction_id. We thought about using a rolling counter that is incremented on each checkpoint as the key, and manually triggering

Re: Feature request - Ability to use a network interface name instead of hostname/IP for Kafka server to use advertised hosts

Hi Jaikiran, maybe this would help. I use Kafka inside VirtualBox, with no DNS and potentially changing IP when guest is started. So, in my Kafka startup script that runs on boot, I just replace the advertised.host.name with the current external IP (it runs after network is up): sed -i

[jira] [Created] (KAFKA-3125) Exception Hierarchy for Streams

Guozhang Wang created KAFKA-3125: Summary: Exception Hierarchy for Streams Key: KAFKA-3125 URL: https://issues.apache.org/jira/browse/KAFKA-3125 Project: Kafka Issue Type: Sub-task

Re: [DISCUSS] KIP-36 - Rack aware replica assignment

Hi Arun, This is about making replica assignment rack aware. It is not about making replica assignment algorithm pluggable. I think plug-ability should be discussed separately from this KIP. Thanks, Allen On Tue, Jan 19, 2016 at 11:16 PM, Arun Mahadevan wrote: > Nice

[jira] [Created] (KAFKA-3126) Weird behavior in kafkaController on Controlled shutdowns. The leaderAndIsr in zookeeper is not updated during controlled shutdown.

Mayuresh Gharat created KAFKA-3126: -- Summary: Weird behavior in kafkaController on Controlled shutdowns. The leaderAndIsr in zookeeper is not updated during controlled shutdown. Key: KAFKA-3126 URL:

Build failed in Jenkins: kafka-trunk-jdk8 #300

See Changes: [wangguoz] KAFKA-3121: Remove aggregatorSupplier and add Reduce functions -- Started by an SCM change [EnvInject] - Loading node environment variables. Building remotely on H11

[GitHub] kafka pull request: KAFKA-2071: Replace Producer Request/Response ...

Github user asfgit closed the pull request at: https://github.com/apache/kafka/pull/110 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[jira] [Updated] (KAFKA-2071) Replace Produce Request/Response with their org.apache.kafka.common.requests equivalents

[ https://issues.apache.org/jira/browse/KAFKA-2071?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ewen Cheslack-Postava updated KAFKA-2071: - Resolution: Fixed Fix Version/s: 0.9.1.0 Status: Resolved (was:

[jira] [Commented] (KAFKA-2071) Replace Produce Request/Response with their org.apache.kafka.common.requests equivalents

[ https://issues.apache.org/jira/browse/KAFKA-2071?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15109813#comment-15109813 ] ASF GitHub Bot commented on KAFKA-2071: --- Github user asfgit closed the pull request at:

Build failed in Jenkins: kafka-trunk-jdk7 #973

See Changes: [wangguoz] KAFKA-3121: Remove aggregatorSupplier and add Reduce functions [me] KAFKA-2071: Replace Producer Request/Response with their -- Started by an SCM change [EnvInject] -

[jira] [Updated] (KAFKA-3126) Weird behavior in kafkaController on Controlled shutdowns. The leaderAndIsr in zookeeper is not updated during controlled shutdown.

[ https://issues.apache.org/jira/browse/KAFKA-3126?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mayuresh Gharat updated KAFKA-3126: --- Description: Consider Broker B is controller, broker A is undergoing shutdown. 2016/01/14

[GitHub] kafka pull request: KAFKA-3121: Remove aggregatorSupplier and add ...

Github user asfgit closed the pull request at: https://github.com/apache/kafka/pull/795 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[jira] [Commented] (KAFKA-3121) KStream DSL API Improvement

[ https://issues.apache.org/jira/browse/KAFKA-3121?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15109790#comment-15109790 ] ASF GitHub Bot commented on KAFKA-3121: --- Github user asfgit closed the pull request at:

Jenkins build is back to normal : kafka-trunk-jdk8 #299

See

Jenkins build is back to normal : kafka-trunk-jdk7 #972

See

[GitHub] kafka pull request: Remove the very misleading comment lines

GitHub user gaob13 opened a pull request: https://github.com/apache/kafka/pull/793 Remove the very misleading comment lines It is not true in practice. Maybe the implied feature is not yet implemented or removed. These lines can be super misleading. Please merge.

Feature request - Ability to use a network interface name instead of hostname/IP for Kafka server to use advertised hosts

We are using Kafka 0.9.0 on our dev setups. We see that the Kafka configurations currently expose a way to configure the host.name and/or advertised.host.name which state: # Hostname the broker will bind to. If not set, the server will bind to all interfaces #host.name= # Hostname the

[jira] [Commented] (KAFKA-1397) delete topic is not working

[ https://issues.apache.org/jira/browse/KAFKA-1397?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15108568#comment-15108568 ] Vivek Meghanathan commented on KAFKA-1397: -- I have 0.8.2.2 version kafka and the delete topic was