[jira] [Commented] (KAFKA-3775) Throttle maximum number of tasks assigned to a single KafkaStreams

2016-06-02 Thread Yuto Kawamura (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-3775?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15313577#comment-15313577 ] Yuto Kawamura commented on KAFKA-3775: -- Thanks for feedback [~mjsax] . > 1) a KStreams application

[jira] [Resolved] (KAFKA-2305) Cluster Monitoring and Management UI

2016-06-02 Thread Soumyajit Sahu (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2305?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Soumyajit Sahu resolved KAFKA-2305. --- Resolution: Invalid > Cluster Monitoring and Management UI >

[jira] [Assigned] (KAFKA-2305) Cluster Monitoring and Management UI

2016-06-02 Thread Soumyajit Sahu (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2305?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Soumyajit Sahu reassigned KAFKA-2305: - Assignee: Soumyajit Sahu (was: Neha Narkhede) > Cluster Monitoring and Management UI >

[jira] [Commented] (KAFKA-3786) Avoid unused property from parent configs causing WARN entries

2016-06-02 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-3786?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15313547#comment-15313547 ] ASF GitHub Bot commented on KAFKA-3786: --- GitHub user guozhangwang opened a pull request:

[GitHub] kafka pull request #1465: KAFKA-3786: Let ConfigDef filter property key valu...

2016-06-02 Thread guozhangwang
GitHub user guozhangwang opened a pull request: https://github.com/apache/kafka/pull/1465 KAFKA-3786: Let ConfigDef filter property key value pairs You can merge this pull request into a Git repository by running: $ git pull https://github.com/guozhangwang/kafka

[jira] [Updated] (KAFKA-3786) Avoid unused property from parent configs causing WARN entries

2016-06-02 Thread Guozhang Wang (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-3786?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guozhang Wang updated KAFKA-3786: - Summary: Avoid unused property from parent configs causing WARN entries (was: AbstractConfig to

[jira] [Updated] (KAFKA-3786) AbstractConfig to only retain defined ConfigDef when parsing property map

2016-06-02 Thread Guozhang Wang (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-3786?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guozhang Wang updated KAFKA-3786: - Description: Currently the {{AbstractConfig}}'s constructor accepts the passed property map as

[jira] [Updated] (KAFKA-3786) AbstractConfig to only retain defined ConfigDef when parsing property map

2016-06-02 Thread Guozhang Wang (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-3786?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guozhang Wang updated KAFKA-3786: - Description: Currently the {{AbstractConfig}}'s constructor accepts the passed property map as

[jira] [Created] (KAFKA-3786) AbstractConfig to only retain defined ConfigDef when parsing property map

2016-06-02 Thread Guozhang Wang (JIRA)
Guozhang Wang created KAFKA-3786: Summary: AbstractConfig to only retain defined ConfigDef when parsing property map Key: KAFKA-3786 URL: https://issues.apache.org/jira/browse/KAFKA-3786 Project:

Re: [DISCUSS] KIP-62: Allow consumer to send heartbeats from a background thread

2016-06-02 Thread Guozhang Wang
Hi Ewen, I think you are right, the rebalance process could potentially involve all the delayed compute / IO. More specifically, this is what I think of the rebalance process: 1. Coordinator decides to rebalance, start ticking based on rebalance time out. 2. Consumer realize rebalance needed

Jenkins build is back to normal : kafka-trunk-jdk7 #1335

2016-06-02 Thread Apache Jenkins Server
See

[jira] [Commented] (KAFKA-3785) Fetcher spending unnecessary time during metrics recording

2016-06-02 Thread Greg Fodor (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-3785?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15313461#comment-15313461 ] Greg Fodor commented on KAFKA-3785: --- https://github.com/apache/kafka/pull/1464 > Fetcher spending

[GitHub] kafka pull request #1464: KAFKA-3785; Fetcher spending unnecessary time duri...

2016-06-02 Thread gfodor
GitHub user gfodor opened a pull request: https://github.com/apache/kafka/pull/1464 KAFKA-3785; Fetcher spending unnecessary time during metrics recording You can merge this pull request into a Git repository by running: $ git pull https://github.com/AltspaceVR/kafka

[jira] [Commented] (KAFKA-3785) Fetcher spending unnecessary time during metrics recording

2016-06-02 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-3785?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15313460#comment-15313460 ] ASF GitHub Bot commented on KAFKA-3785: --- GitHub user gfodor opened a pull request:

[jira] [Updated] (KAFKA-3785) Fetcher spending unnecessary time during metrics recording

2016-06-02 Thread Greg Fodor (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-3785?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Greg Fodor updated KAFKA-3785: -- Component/s: consumer > Fetcher spending unnecessary time during metrics recording >

[jira] [Created] (KAFKA-3785) Fetcher spending unnecessary time during metrics recording

2016-06-02 Thread Greg Fodor (JIRA)
Greg Fodor created KAFKA-3785: - Summary: Fetcher spending unnecessary time during metrics recording Key: KAFKA-3785 URL: https://issues.apache.org/jira/browse/KAFKA-3785 Project: Kafka Issue

[jira] [Commented] (KAFKA-3693) Race condition between highwatermark-checkpoint thread and handleLeaderAndIsrRequest at broker start-up

2016-06-02 Thread Jun Rao (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-3693?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15313441#comment-15313441 ] Jun Rao commented on KAFKA-3693: [~maysamyabandeh], yes, I agree it would be better if the broker can be

Build failed in Jenkins: kafka-0.10.0-jdk7 #112

2016-06-02 Thread Apache Jenkins Server
See Changes: [ismael] MINOR: Fix setting of ACLs and ZK shutdown in test harnesses -- Started by an SCM change [EnvInject] - Loading node environment variables. Building remotely on H10 (docker

[jira] [Commented] (KAFKA-3770) KStream job should be able to specify linger.ms

2016-06-02 Thread Greg Fodor (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-3770?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15313434#comment-15313434 ] Greg Fodor commented on KAFKA-3770: --- Cut a new PR since I screwed up the rebase on the previous one.

[jira] [Commented] (KAFKA-3770) KStream job should be able to specify linger.ms

2016-06-02 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-3770?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15313432#comment-15313432 ] ASF GitHub Bot commented on KAFKA-3770: --- GitHub user gfodor opened a pull request:

[GitHub] kafka pull request #1463: KAFKA-3770: KStream job should be able to specify ...

2016-06-02 Thread gfodor
GitHub user gfodor opened a pull request: https://github.com/apache/kafka/pull/1463 KAFKA-3770: KStream job should be able to specify linger.ms You can merge this pull request into a Git repository by running: $ git pull https://github.com/AltspaceVR/kafka gfodor/kafka-3770-2

Build failed in Jenkins: kafka-trunk-jdk8 #671

2016-06-02 Thread Apache Jenkins Server
See Changes: [ismael] MINOR: Fix setting of ACLs and ZK shutdown in test harnesses -- Started by an SCM change [EnvInject] - Loading node environment variables. Building remotely on H11 (docker

[jira] [Commented] (KAFKA-3770) KStream job should be able to specify linger.ms

2016-06-02 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-3770?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15313424#comment-15313424 ] ASF GitHub Bot commented on KAFKA-3770: --- Github user gfodor closed the pull request at:

[GitHub] kafka pull request #1448: KAFKA-3770: KStream job should be able to specify ...

2016-06-02 Thread gfodor
Github user gfodor closed the pull request at: https://github.com/apache/kafka/pull/1448 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

Re: [DISCUSS] KIP-62: Allow consumer to send heartbeats from a background thread

2016-06-02 Thread Guozhang Wang
Okay, now I understand that the usefulness of a third rebalance timeout is only when users do not care about process timeout (i.e. they just make it infinity), but still want to let the rebalance to finish in a reasonable amount of time even some consumers are delayed in processing before their

Re: [DISCUSS] KIP-62: Allow consumer to send heartbeats from a background thread

2016-06-02 Thread Ewen Cheslack-Postava
Jason, I've been thinking about this more in terms of something like Connect. I think the rebalance timeout may be a bit different from the process timeout, and even the process timeout is a bit of a misnomer. We sort of talk about the process timeout as if it can be an indicator of maximum

[jira] [Commented] (KAFKA-3752) Provide a way for KStreams to recover from unclean shutdown

2016-06-02 Thread Guozhang Wang (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-3752?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15313385#comment-15313385 ] Guozhang Wang commented on KAFKA-3752: -- [~theduderog] I looked through the file locking issue. After

[jira] [Commented] (KAFKA-3777) Extract the LRU cache out of RocksDBStore

2016-06-02 Thread Jay Kreps (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-3777?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15313382#comment-15313382 ] Jay Kreps commented on KAFKA-3777: -- Probably everyone already knows but just want to capture a couple of

[GitHub] kafka pull request #1455: MINOR: Fix setting of ACLs and ZK shutdown in test...

2016-06-02 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/kafka/pull/1455 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[jira] [Commented] (KAFKA-3693) Race condition between highwatermark-checkpoint thread and handleLeaderAndIsrRequest at broker start-up

2016-06-02 Thread Maysam Yabandeh (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-3693?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15313379#comment-15313379 ] Maysam Yabandeh commented on KAFKA-3693: Thanks for looking into this [~junrao] I agree that the

[jira] [Commented] (KAFKA-3784) TimeWindows#windowsFor misidentifies some windows if TimeWindows#advanceBy is used

2016-06-02 Thread Tom Rybak (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-3784?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15313374#comment-15313374 ] Tom Rybak commented on KAFKA-3784: -- I made a pull request with the fix, plus a unit test that can be used

[jira] [Commented] (KAFKA-3784) TimeWindows#windowsFor misidentifies some windows if TimeWindows#advanceBy is used

2016-06-02 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-3784?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15313368#comment-15313368 ] ASF GitHub Bot commented on KAFKA-3784: --- GitHub user trybak opened a pull request:

[jira] [Commented] (KAFKA-3769) KStream job spending 60% of time writing metrics

2016-06-02 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-3769?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15313369#comment-15313369 ] ASF GitHub Bot commented on KAFKA-3769: --- Github user gfodor closed the pull request at:

[GitHub] kafka pull request #1447: KAFKA-3769 - KStream job spending 60% of time writ...

2016-06-02 Thread gfodor
Github user gfodor closed the pull request at: https://github.com/apache/kafka/pull/1447 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] kafka pull request #1462: KAFKA-3784 TimeWindows#windowsFor misidentifies so...

2016-06-02 Thread trybak
GitHub user trybak opened a pull request: https://github.com/apache/kafka/pull/1462 KAFKA-3784 TimeWindows#windowsFor misidentifies some windows if TimeWindows#advanceBy is used - Fixed the logic calculating the windows that are affected by a new …event in the case of hopping

Build failed in Jenkins: kafka-trunk-jdk7 #1334

2016-06-02 Thread Apache Jenkins Server
See Changes: [wangguoz] HOTFIX: updated JavaDoc example for 0.9 tech-prev to 0.10 -- Started by an SCM change [EnvInject] - Loading node environment variables. Building remotely on H11 (docker

[jira] [Created] (KAFKA-3784) TimeWindows#windowsFor misidentifies some windows if TimeWindows#advanceBy is used

2016-06-02 Thread Tom Rybak (JIRA)
Tom Rybak created KAFKA-3784: Summary: TimeWindows#windowsFor misidentifies some windows if TimeWindows#advanceBy is used Key: KAFKA-3784 URL: https://issues.apache.org/jira/browse/KAFKA-3784 Project:

[jira] [Commented] (KAFKA-3769) KStream job spending 60% of time writing metrics

2016-06-02 Thread Guozhang Wang (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-3769?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15313295#comment-15313295 ] Guozhang Wang commented on KAFKA-3769: -- Hello Greg, I will continue the discussion in your PR. >

Build failed in Jenkins: kafka-trunk-jdk8 #670

2016-06-02 Thread Apache Jenkins Server
See Changes: [wangguoz] HOTFIX: updated JavaDoc example for 0.9 tech-prev to 0.10 -- Started by an SCM change [EnvInject] - Loading node environment variables. Building remotely on H11 (docker

[jira] [Commented] (KAFKA-3335) Kafka Connect hangs in shutdown hook

2016-06-02 Thread Ben Kirwin (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-3335?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15313267#comment-15313267 ] Ben Kirwin commented on KAFKA-3335: --- Excellent -- thanks for following up on this! > Kafka Connect

[jira] [Commented] (KAFKA-2260) Allow specifying expected offset on produce

2016-06-02 Thread Ben Kirwin (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2260?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15313258#comment-15313258 ] Ben Kirwin commented on KAFKA-2260: --- Hi! I haven't had time to push on this at all in the last few

[jira] [Updated] (KAFKA-3777) Extract the LRU cache out of RocksDBStore

2016-06-02 Thread Guozhang Wang (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-3777?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guozhang Wang updated KAFKA-3777: - Description: The LRU cache that is currently inside the RocksDbStore class. As part of

[jira] [Updated] (KAFKA-3777) Extract the LRU cache out of RocksDBStore

2016-06-02 Thread Guozhang Wang (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-3777?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guozhang Wang updated KAFKA-3777: - Description: The LRU cache that is currently inside the RocksDbStore class. As part of

[jira] [Updated] (KAFKA-3779) Add the LRU cache for KTable.to() operator

2016-06-02 Thread Guozhang Wang (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-3779?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guozhang Wang updated KAFKA-3779: - Description: The KTable.to operator currently does not use a cache. We can add a cache to this

[jira] [Updated] (KAFKA-3778) Avoidin using range queries of RocksDBWindowStore on KStream windowed aggregations

2016-06-02 Thread Guozhang Wang (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-3778?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guozhang Wang updated KAFKA-3778: - Summary: Avoidin using range queries of RocksDBWindowStore on KStream windowed aggregations

[jira] [Updated] (KAFKA-3778) Change the RocksDB implementation for KStream windowed aggregations, to not use “range queries” but multiple gets

2016-06-02 Thread Guozhang Wang (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-3778?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guozhang Wang updated KAFKA-3778: - Description: RocksDbWindowStore currently does not use caches, but its window segments

[GitHub] kafka pull request #1450: HOTFIX: updated JavaDoc example for 0.9 tech-prev ...

2016-06-02 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/kafka/pull/1450 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[jira] [Updated] (KAFKA-3689) ERROR Processor got uncaught exception. (kafka.network.Processor)

2016-06-02 Thread Jun Rao (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-3689?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jun Rao updated KAFKA-3689: --- Attachment: kafka-3689-instrumentation.patch [~buvana.rama...@nokia.com], our current logging doesn't tell

Re: [DISCUSS] KIP-62: Allow consumer to send heartbeats from a background thread

2016-06-02 Thread Guozhang Wang
Hi Jason, With the current usage pattern of: while(..) { consumer.poll(/* where rebalance happens */) // process messages } -- And since rebalance is till on the caller thread, not the background thread, if coordinator decides to rebalance while user thread is still on processing

[GitHub] kafka pull request #1461: KAFKA-3783: Catch proper exception on path delete

2016-06-02 Thread slaunay
GitHub user slaunay opened a pull request: https://github.com/apache/kafka/pull/1461 KAFKA-3783: Catch proper exception on path delete - ZkClient is used for conditional path deletion and wraps `KeeperException.BadVersionException` into `ZkBadVersionException` - add unit test

[jira] [Commented] (KAFKA-3783) Race condition on last ACL removal for a resource fails with a ZkBadVersionException

2016-06-02 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-3783?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15313063#comment-15313063 ] ASF GitHub Bot commented on KAFKA-3783: --- GitHub user slaunay opened a pull request:

[jira] [Created] (KAFKA-3783) Race condition on last ACL removal for a resource fails with a ZkBadVersionException

2016-06-02 Thread JIRA
Sébastien Launay created KAFKA-3783: --- Summary: Race condition on last ACL removal for a resource fails with a ZkBadVersionException Key: KAFKA-3783 URL: https://issues.apache.org/jira/browse/KAFKA-3783

[jira] [Commented] (KAFKA-3755) tightening the offset check in ReplicaFetcherThread

2016-06-02 Thread Jun Rao (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-3755?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15313028#comment-15313028 ] Jun Rao commented on KAFKA-3755: [~imandhan], we can just add a unit test that calls log.append() with

Kafka take too long to update the client with metadata when a broker is gone

2016-06-02 Thread safique ahemad
Hi All, We are using Kafka broker cluster in our data center. Recently, It is realized that when a Kafka broker goes down then client try to refresh the metadata but it get stale metadata upto near 30 seconds. After near 30-35 seconds, updated metadata is obtained by client. This is really a

[jira] [Commented] (KAFKA-3689) ERROR Processor got uncaught exception. (kafka.network.Processor)

2016-06-02 Thread Buvaneswari Ramanan (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-3689?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15312948#comment-15312948 ] Buvaneswari Ramanan commented on KAFKA-3689: One more scenario under which this occurs

[jira] [Comment Edited] (KAFKA-3689) ERROR Processor got uncaught exception. (kafka.network.Processor)

2016-06-02 Thread Buvaneswari Ramanan (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-3689?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15312945#comment-15312945 ] Buvaneswari Ramanan edited comment on KAFKA-3689 at 6/2/16 7:53 PM:

[jira] [Commented] (KAFKA-3689) ERROR Processor got uncaught exception. (kafka.network.Processor)

2016-06-02 Thread Buvaneswari Ramanan (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-3689?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15312945#comment-15312945 ] Buvaneswari Ramanan commented on KAFKA-3689: All connections are shown to be in ESTABLISHED

[jira] [Updated] (KAFKA-3782) Transient failure with kafkatest.tests.connect.connect_distributed_test.ConnectDistributedTest.test_bounce.clean=True

2016-06-02 Thread Liquan Pei (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-3782?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Liquan Pei updated KAFKA-3782: -- Description: For commit 946ae60 max() arg is an empty sequence Traceback (most recent call last):

[jira] [Assigned] (KAFKA-3764) Error processing append operation on partition

2016-06-02 Thread Grant Henke (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-3764?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Grant Henke reassigned KAFKA-3764: -- Assignee: Grant Henke > Error processing append operation on partition >

[jira] [Commented] (KAFKA-3764) Error processing append operation on partition

2016-06-02 Thread Grant Henke (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-3764?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15312916#comment-15312916 ] Grant Henke commented on KAFKA-3764: It looks like this is likely caused by

[jira] [Commented] (KAFKA-3758) KStream job fails to recover after Kafka broker stopped

2016-06-02 Thread Greg Fodor (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-3758?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15312913#comment-15312913 ] Greg Fodor commented on KAFKA-3758: --- also, if we did not run at an elevated number of threads, we were

[jira] [Commented] (KAFKA-3758) KStream job fails to recover after Kafka broker stopped

2016-06-02 Thread Greg Fodor (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-3758?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15312912#comment-15312912 ] Greg Fodor commented on KAFKA-3758: --- Hey, we're running 16 threads -- for this job we have 25 topics,

[jira] [Commented] (KAFKA-3758) KStream job fails to recover after Kafka broker stopped

2016-06-02 Thread Guozhang Wang (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-3758?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15312888#comment-15312888 ] Guozhang Wang commented on KAFKA-3758: -- Hi Greg, I have a few more questions that will help me

[jira] [Created] (KAFKA-3782) Transient failure with kafkatest.tests.connect.connect_distributed_test.ConnectDistributedTest.test_bounce.clean=True

2016-06-02 Thread Liquan Pei (JIRA)
Liquan Pei created KAFKA-3782: - Summary: Transient failure with kafkatest.tests.connect.connect_distributed_test.ConnectDistributedTest.test_bounce.clean=True Key: KAFKA-3782 URL:

[jira] [Commented] (KAFKA-3764) Error processing append operation on partition

2016-06-02 Thread Taro L. Saito (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-3764?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15312782#comment-15312782 ] Taro L. Saito commented on KAFKA-3764: -- Released snappy-java-1.1.2.6, which would resolve this issue.

Re: [DISCUSS] KIP-62: Allow consumer to send heartbeats from a background thread

2016-06-02 Thread Jason Gustafson
Hey Guozhang, I think the problem is that users may not want to sacrifice rebalance latency because of uncertainty around processing time. As soon as a rebalance begins, there are basically two choices: 1. Block the rebalance until all consumers have finished their current processing. 2. Let all

Changing default logger to RollingFileAppender (KAFKA-2394)

2016-06-02 Thread Dustin Cote
Hi all, I'm looking at changing the Kafka default logging setup to use the RollingFileAppender instead of the DailyRollingFileAppender in an effort to accomplish two goals: 1) Avoid filling up users' disks if the log files grow unexpectedly 2) Move off the admittedly unreliable

[jira] [Assigned] (KAFKA-2394) Use RollingFileAppender by default in log4j.properties

2016-06-02 Thread Dustin Cote (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2394?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dustin Cote reassigned KAFKA-2394: -- Assignee: Dustin Cote (was: jin xing) > Use RollingFileAppender by default in

[jira] [Commented] (KAFKA-2394) Use RollingFileAppender by default in log4j.properties

2016-06-02 Thread Dustin Cote (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2394?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15312409#comment-15312409 ] Dustin Cote commented on KAFKA-2394: Thanks [~ewencp]. I went ahead and sent out to users@ and dev@

[jira] [Comment Edited] (KAFKA-3764) Error processing append operation on partition

2016-06-02 Thread Martin Nowak (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-3764?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15312402#comment-15312402 ] Martin Nowak edited comment on KAFKA-3764 at 6/2/16 2:34 PM: - Good guess,

[jira] [Commented] (KAFKA-3764) Error processing append operation on partition

2016-06-02 Thread Martin Nowak (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-3764?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15312402#comment-15312402 ] Martin Nowak commented on KAFKA-3764: - Good guess, there is a bug report for snappy-java w/ a similar

[jira] [Commented] (KAFKA-3725) Update documentation with regards to XFS

2016-06-02 Thread Todd Palino (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-3725?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15312288#comment-15312288 ] Todd Palino commented on KAFKA-3725: I'll take a look at this and put together a PR with some updates.

[jira] [Assigned] (KAFKA-3725) Update documentation with regards to XFS

2016-06-02 Thread Todd Palino (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-3725?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Todd Palino reassigned KAFKA-3725: -- Assignee: Todd Palino > Update documentation with regards to XFS >

Re: KStreams Rewind Offset

2016-06-02 Thread Matthias J. Sax
Hi Mike, currently, this is not possible. We are already discussing some changes with regard to reprocess. However, I doubt that going back to a specific offset of a specific partition will be supported as it would be too difficult to reset the internal data structures and intermediate results

[jira] [Commented] (KAFKA-3775) Throttle maximum number of tasks assigned to a single KafkaStreams

2016-06-02 Thread Matthias J. Sax (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-3775?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15312250#comment-15312250 ] Matthias J. Sax commented on KAFKA-3775: I have some concerns about this: 1) a KStreams

[jira] [Commented] (KAFKA-3560) Kafka is not working on Solaris

2016-06-02 Thread JIRA
[ https://issues.apache.org/jira/browse/KAFKA-3560?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15312230#comment-15312230 ] Marcus Gründler commented on KAFKA-3560: With Kafka 0.10.0.0 everything works fine on Solaris with

[jira] [Updated] (KAFKA-3101) Optimize Aggregation Outputs

2016-06-02 Thread Eno Thereska (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-3101?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eno Thereska updated KAFKA-3101: Assignee: (was: Eno Thereska) > Optimize Aggregation Outputs > > >

[jira] [Commented] (KAFKA-3101) Optimize Aggregation Outputs

2016-06-02 Thread Eno Thereska (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-3101?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15312133#comment-15312133 ] Eno Thereska commented on KAFKA-3101: - [~h...@pinterest.com], [~bbejeck], I just added a KIP around

[jira] [Created] (KAFKA-3780) Add new config cache.max.bytes.buffering to the streams configuration

2016-06-02 Thread Eno Thereska (JIRA)
Eno Thereska created KAFKA-3780: --- Summary: Add new config cache.max.bytes.buffering to the streams configuration Key: KAFKA-3780 URL: https://issues.apache.org/jira/browse/KAFKA-3780 Project: Kafka

[jira] [Created] (KAFKA-3779) Add the LRU cache for KTable.to() operator

2016-06-02 Thread Eno Thereska (JIRA)
Eno Thereska created KAFKA-3779: --- Summary: Add the LRU cache for KTable.to() operator Key: KAFKA-3779 URL: https://issues.apache.org/jira/browse/KAFKA-3779 Project: Kafka Issue Type: Sub-task

[jira] [Created] (KAFKA-3778) Change the RocksDB implementation for KStream windowed aggregations, to not use “range queries” but multiple gets

2016-06-02 Thread Eno Thereska (JIRA)
Eno Thereska created KAFKA-3778: --- Summary: Change the RocksDB implementation for KStream windowed aggregations, to not use “range queries” but multiple gets Key: KAFKA-3778 URL:

[jira] [Updated] (KAFKA-3776) Unify store and downstream caching in streams

2016-06-02 Thread Eno Thereska (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-3776?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eno Thereska updated KAFKA-3776: Assignee: (was: Guozhang Wang) > Unify store and downstream caching in streams >

[jira] [Updated] (KAFKA-3777) Extract the LRU cache out of RocksDBStore

2016-06-02 Thread Eno Thereska (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-3777?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eno Thereska updated KAFKA-3777: Assignee: (was: Guozhang Wang) > Extract the LRU cache out of RocksDBStore >

[jira] [Updated] (KAFKA-3776) Unify store and downstream caching in streams

2016-06-02 Thread Eno Thereska (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-3776?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eno Thereska updated KAFKA-3776: Reviewer: Eno Thereska > Unify store and downstream caching in streams >

[jira] [Created] (KAFKA-3777) Extract the LRU cache out of RocksDBStore

2016-06-02 Thread Eno Thereska (JIRA)
Eno Thereska created KAFKA-3777: --- Summary: Extract the LRU cache out of RocksDBStore Key: KAFKA-3777 URL: https://issues.apache.org/jira/browse/KAFKA-3777 Project: Kafka Issue Type: Sub-task

[jira] [Created] (KAFKA-3776) Unify store and downstream caching in streams

2016-06-02 Thread Eno Thereska (JIRA)
Eno Thereska created KAFKA-3776: --- Summary: Unify store and downstream caching in streams Key: KAFKA-3776 URL: https://issues.apache.org/jira/browse/KAFKA-3776 Project: Kafka Issue Type: New

[DISCUSS] KIP-63: Unify store and downstream caching in streams

2016-06-02 Thread Eno Thereska
Hi there, I have created KIP-63: Unify store and downstream caching in streams https://cwiki.apache.org/confluence/display/KAFKA/KIP-63%3A+Unify+store+and+downstream+caching+in+streams

[jira] [Commented] (KAFKA-3775) Throttle maximum number of tasks assigned to a single KafkaStreams

2016-06-02 Thread Yuto Kawamura (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-3775?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15311838#comment-15311838 ] Yuto Kawamura commented on KAFKA-3775: -- [~guozhang] What do you think? > Throttle maximum number of

[jira] [Updated] (KAFKA-3775) Throttle maximum number of tasks assigned to a single KafkaStreams

2016-06-02 Thread Yuto Kawamura (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-3775?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuto Kawamura updated KAFKA-3775: - Status: Patch Available (was: In Progress) > Throttle maximum number of tasks assigned to a

[jira] [Commented] (KAFKA-3775) Throttle maximum number of tasks assigned to a single KafkaStreams

2016-06-02 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-3775?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15311836#comment-15311836 ] ASF GitHub Bot commented on KAFKA-3775: --- GitHub user kawamuray opened a pull request:

[GitHub] kafka pull request #1460: KAFKA-3775: Throttle maximum number of tasks assig...

2016-06-02 Thread kawamuray
GitHub user kawamuray opened a pull request: https://github.com/apache/kafka/pull/1460 KAFKA-3775: Throttle maximum number of tasks assigned to a single KafkaStreams Issue: https://issues.apache.org/jira/browse/KAFKA-3775 POC. Discussion in progress. You can merge this

[jira] [Work started] (KAFKA-3775) Throttle maximum number of tasks assigned to a single KafkaStreams

2016-06-02 Thread Yuto Kawamura (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-3775?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on KAFKA-3775 started by Yuto Kawamura. > Throttle maximum number of tasks assigned to a single KafkaStreams >

[jira] [Created] (KAFKA-3775) Throttle maximum number of tasks assigned to a single KafkaStreams

2016-06-02 Thread Yuto Kawamura (JIRA)
Yuto Kawamura created KAFKA-3775: Summary: Throttle maximum number of tasks assigned to a single KafkaStreams Key: KAFKA-3775 URL: https://issues.apache.org/jira/browse/KAFKA-3775 Project: Kafka

Build failed in Jenkins: kafka-trunk-jdk7 #1333

2016-06-02 Thread Apache Jenkins Server
See Changes: [me] KAFKA-3774: Make 'time' an optional argument of GetOffsetShell -- Started by an SCM change [EnvInject] - Loading node environment variables. Building remotely on H11 (docker

Build failed in Jenkins: kafka-trunk-jdk8 #669

2016-06-02 Thread Apache Jenkins Server
See Changes: [me] KAFKA-3774: Make 'time' an optional argument of GetOffsetShell -- Started by an SCM change [EnvInject] - Loading node environment variables. Building remotely on ubuntu-us1