Re: [DISCUSS] KIP-62: Allow consumer to send heartbeats from a background thread

Hi Jason, Thanks for this very useful KIP. In general I am with Guozhang on the purpose of of the three timeouts. 1) session timeout for consumer liveness, 2) process timeout (or maybe we should rename it to max.poll.interval.ms) for application liveness, 3) rebalance timeout for faster

[jira] [Updated] (KAFKA-2651) Remove deprecated config alteration from TopicCommand in 0.9.1.0

[ https://issues.apache.org/jira/browse/KAFKA-2651?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Manikumar Reddy updated KAFKA-2651: --- Fix Version/s: 0.11.0.0 > Remove deprecated config alteration from TopicCommand in 0.9.1.0 >

Jenkins build is back to normal : kafka-trunk-jdk8 #680

See

Build failed in Jenkins: kafka-trunk-jdk7 #1343

See Changes: [harsha] MINOR: Fix producer leak in `PlaintextProducerSendTest` [harsha] MINOR: Move `ConsoleProducerTest` to the unit test directory -- Started by an SCM change [EnvInject] -

[GitHub] kafka pull request #1470: MINOR: Move `ConsoleProducerTest` to the unit test...

Github user asfgit closed the pull request at: https://github.com/apache/kafka/pull/1470 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] kafka pull request #1471: MINOR: Fix producer leak in `PlaintextProducerSend...

Github user asfgit closed the pull request at: https://github.com/apache/kafka/pull/1471 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[jira] [Created] (KAFKA-3793) Kafka Python Consumer library messages gets truncated

Rahul created KAFKA-3793: Summary: Kafka Python Consumer library messages gets truncated Key: KAFKA-3793 URL: https://issues.apache.org/jira/browse/KAFKA-3793 Project: Kafka Issue Type: Bug

Re: [DISCUSS] KIP-63: Unify store and downstream caching in streams

So you were listing the difficulties in order to provide context for the upcoming design change and discussion? Its all good then :) On Sun, Jun 5, 2016 at 11:26 PM, Guozhang Wang wrote: > Don't get me wrong Gwen :) I'm definitely for removing as less burden as > possible

Re: [DISCUSS] KIP-63: Unify store and downstream caching in streams

Don't get me wrong Gwen :) I'm definitely for removing as less burden as possible from users. All I'm saying it is not straight-forward to do so, and we'd better at least have a concrete implementation design on the KIP page rather than just a one-line change of the config semantics. On Sun, Jun

Re: [DISCUSS] KIP-63: Unify store and downstream caching in streams

If it is hard for you, just imagine how much fun the users will have. Machines have X GB available RAM. Someone has to figure out how to divide it for processors and rocksDB. The user doesn't have any special knowledge that you don't in this case, so there's no point in pushing the decision to

Re: [DISCUSS] KIP-63: Unify store and downstream caching in streams

There are some details needed to be figured out if we go global: A KafkaStreams instance could have M threads, and each thread could various number (let's say N, but in practice it may be different from thread to thread) tasks, and each task contains a sub-topology with P caches (again in

[jira] [Created] (KAFKA-3792) Fix log spam in clients when topic doesn't exist

Rajini Sivaram created KAFKA-3792: - Summary: Fix log spam in clients when topic doesn't exist Key: KAFKA-3792 URL: https://issues.apache.org/jira/browse/KAFKA-3792 Project: Kafka Issue Type:

[jira] [Commented] (KAFKA-3561) Auto create through topic for KStream aggregation and join

[ https://issues.apache.org/jira/browse/KAFKA-3561?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15315908#comment-15315908 ] ASF GitHub Bot commented on KAFKA-3561: --- GitHub user dguy opened a pull request:

[GitHub] kafka pull request #1472: KAFKA-3561: Auto create through topic for KStream ...

GitHub user dguy opened a pull request: https://github.com/apache/kafka/pull/1472 KAFKA-3561: Auto create through topic for KStream aggregation and join [WIP] @guozhangwang @enothereska @mjsax @miguno If you get a chance can you please take a look at this. I've done the

[GitHub] kafka pull request #1471: MINOR: Fix producer leak in `PlaintextProducerSend...

GitHub user ijuma opened a pull request: https://github.com/apache/kafka/pull/1471 MINOR: Fix producer leak in `PlaintextProducerSendTest` You can merge this pull request into a Git repository by running: $ git pull https://github.com/ijuma/kafka

Build failed in Jenkins: kafka-trunk-jdk7 #1342

See Changes: [ismael] KAFKA-3787; Preserve the message timestamp in mirror maker -- Started by an SCM change [EnvInject] - Loading node environment variables. Building remotely on H11 (docker

Build failed in Jenkins: kafka-0.10.0-jdk7 #117

See Changes: [ismael] KAFKA-3787; Preserve the message timestamp in mirror maker -- Started by an SCM change [EnvInject] - Loading node environment variables. Building remotely on H10 (docker

Build failed in Jenkins: kafka-trunk-jdk8 #679

See Changes: [ismael] KAFKA-3787; Preserve the message timestamp in mirror maker -- Started by an SCM change [EnvInject] - Loading node environment variables. Building remotely on ubuntu-us1

[jira] [Updated] (KAFKA-3787) Preserve message timestamp in mirror mkaer

[ https://issues.apache.org/jira/browse/KAFKA-3787?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ismael Juma updated KAFKA-3787: --- Resolution: Fixed Status: Resolved (was: Patch Available) Issue resolved by pull request

[GitHub] kafka pull request #1466: KAFKA-3787: Preserve the message timestamp in mirr...

Github user asfgit closed the pull request at: https://github.com/apache/kafka/pull/1466 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[jira] [Commented] (KAFKA-3787) Preserve message timestamp in mirror mkaer

[ https://issues.apache.org/jira/browse/KAFKA-3787?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15315792#comment-15315792 ] ASF GitHub Bot commented on KAFKA-3787: --- Github user asfgit closed the pull request at:

[GitHub] kafka pull request #1470: MINOR: Move `ConsoleProducerTest` to the unit test...

GitHub user ijuma opened a pull request: https://github.com/apache/kafka/pull/1470 MINOR: Move `ConsoleProducerTest` to the unit test directory Included a couple of clean-ups: removed unused variable and close the instantiated `KafkaProducer`. You can merge this pull request into

[jira] [Commented] (KAFKA-724) Allow automatic socket.send.buffer from operating system

[ https://issues.apache.org/jira/browse/KAFKA-724?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15315780#comment-15315780 ] Ismael Juma commented on KAFKA-724: --- This was fixed on the new producer and consumer in 0.9.0.0. The PR

[jira] [Updated] (KAFKA-724) Allow automatic socket.send.buffer from operating system

[ https://issues.apache.org/jira/browse/KAFKA-724?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ismael Juma updated KAFKA-724: -- Fix Version/s: 0.10.1.0 > Allow automatic socket.send.buffer from operating system >

Re: [DISCUSS] KIP-63: Unify store and downstream caching in streams

Hi Jay, We can make it global instead of per-processor, sounds good. Thanks Eno > On 3 Jun 2016, at 23:15, Jay Kreps wrote: > > Hey Eno, > > Should the config be the global memory use rather than the per-processor? > That is, let’s say I know I have fixed a 1GB heap