Re: Kafka Streams for Remote Server

Thanks Guozhang, Yes, I rely on auto-create, and it works locally. Maybe I need to tweak some timeout conf for that? Also I identified, that even I manually create a topic, it lists but I can not produce messages to this topic with the same exception. Producing to other topics works well, so it

[GitHub] kafka pull request #1656: KAFKA-3977: Defer fetch parsing for space efficien...

Github user asfgit closed the pull request at: https://github.com/apache/kafka/pull/1656 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enab

[jira] [Commented] (KAFKA-3977) KafkaConsumer swallows exceptions raised from message deserializers

[ https://issues.apache.org/jira/browse/KAFKA-3977?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15397364#comment-15397364 ] ASF GitHub Bot commented on KAFKA-3977: --- Github user asfgit closed the pull request

[jira] [Updated] (KAFKA-3977) KafkaConsumer swallows exceptions raised from message deserializers

[ https://issues.apache.org/jira/browse/KAFKA-3977?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ismael Juma updated KAFKA-3977: --- Resolution: Fixed Reviewer: Ismael Juma Status: Resolved (was: Patch Available) > Kafka

Jenkins build is back to normal : kafka-trunk-jdk7 #1446

See

[GitHub] kafka pull request #1672: MINOR: More graceful handling of buffers that are ...

GitHub user ijuma opened a pull request: https://github.com/apache/kafka/pull/1672 MINOR: More graceful handling of buffers that are too small in Record's `isValid` and `ensureValid` Also add tests and make `Crc32.update` perform the same argument checks as `java.util.zip.CRC32`

Build failed in Jenkins: kafka-trunk-jdk8 #782

See Changes: [ismael] KAFKA-3977; Defer fetch parsing for space efficiency and to ensure -- [...truncated 6665 lines...] kafka.utils.ByteBoundedBlockingQueueTest > testByteBoundedBlockingQueue P

Build failed in Jenkins: kafka-0.10.0-jdk7 #167

See Changes: [ismael] KAFKA-3977; Defer fetch parsing for space efficiency and to ensure -- [...truncated 2836 lines...] kafka.api.PlaintextConsumerTest > testInterceptors PASSED kafka.api.Pla

[jira] [Work started] (KAFKA-3929) Add prefix for underlying clients configs in StreamConfig

[ https://issues.apache.org/jira/browse/KAFKA-3929?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on KAFKA-3929 started by Damian Guy. - > Add prefix for underlying clients configs in StreamConfig > -

[jira] [Updated] (KAFKA-3929) Add prefix for underlying clients configs in StreamConfig

[ https://issues.apache.org/jira/browse/KAFKA-3929?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Damian Guy updated KAFKA-3929: -- Status: Patch Available (was: In Progress) > Add prefix for underlying clients configs in StreamConfig

Build failed in Jenkins: kafka-trunk-jdk8 #783

See Changes: [ismael] HOTFIX: Add license information to release_notes.py -- [...truncated 11734 lines...] org.apache.kafka.streams.processor.internals.StateDirectoryTest > shouldLockMulitpleTas

Build failed in Jenkins: kafka-0.10.0-jdk7 #168

See Changes: [ismael] HOTFIX: Add license information to release_notes.py -- [...truncated 5729 lines...] org.apache.kafka.streams.kstream.internals.KTableMapValuesTest > testNotSendingOldValue

LEADER_NOT_AVAILABLE error

Hello, I'm receiving an error while publishing messages to a kafka topic, the steps I took were:1. Starting zookeeper server2. Starting kafka server3. Sending messages to kafka topic with a Kafka Producer 8105 [kafka-producer-network-thread | producer-2] DEBUG org.apache.kafka.clients.NetworkCli

[GitHub] kafka pull request #1673: Hotfix: fixed instable Streams application reset i...

GitHub user mjsax opened a pull request: https://github.com/apache/kafka/pull/1673 Hotfix: fixed instable Streams application reset integration test You can merge this pull request into a Git repository by running: $ git pull https://github.com/mjsax/kafka hotfix Alternativel

[GitHub] kafka pull request #1674: HOTFIX: Fixes to javadoc and to state store name f...

GitHub user enothereska opened a pull request: https://github.com/apache/kafka/pull/1674 HOTFIX: Fixes to javadoc and to state store name for link joins You can merge this pull request into a Git repository by running: $ git pull https://github.com/enothereska/kafka hotfix-mis

[GitHub] kafka pull request #1675: MINOR: Fixed a few warnings in clients

GitHub user mimaison opened a pull request: https://github.com/apache/kafka/pull/1675 MINOR: Fixed a few warnings in clients - Removed an unnecessary annotation - Parameterized a couple of raw types You can merge this pull request into a Git repository by running: $ git pul

[jira] [Created] (KAFKA-4003) Followers are dropped for being too far behind, yet are less than 30 messages / 10k bytes behind.

Eric Abbott created KAFKA-4003: -- Summary: Followers are dropped for being too far behind, yet are less than 30 messages / 10k bytes behind. Key: KAFKA-4003 URL: https://issues.apache.org/jira/browse/KAFKA-4003

[GitHub] kafka pull request #1676: MINOR: Consumer should throw KafkaException on inv...

GitHub user hachikuji opened a pull request: https://github.com/apache/kafka/pull/1676 MINOR: Consumer should throw KafkaException on invalid checksum You can merge this pull request into a Git repository by running: $ git pull https://github.com/hachikuji/kafka raise-kafka-e

[jira] [Created] (KAFKA-4004) NetworkReceive.complete() should not throw NullPointerException after partially reading the size

Onur Karaman created KAFKA-4004: --- Summary: NetworkReceive.complete() should not throw NullPointerException after partially reading the size Key: KAFKA-4004 URL: https://issues.apache.org/jira/browse/KAFKA-4004

[GitHub] kafka pull request #1677: KAFKA-4004: NetworkReceive.complete() should not t...

GitHub user onurkaraman opened a pull request: https://github.com/apache/kafka/pull/1677 KAFKA-4004: NetworkReceive.complete() should not throw NullPointerException after partially reading the size NetworkReceive.readFrom can partially read the size and not allocate a buffer. A fol

[jira] [Commented] (KAFKA-4004) NetworkReceive.complete() should not throw NullPointerException after partially reading the size

[ https://issues.apache.org/jira/browse/KAFKA-4004?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15397957#comment-15397957 ] ASF GitHub Bot commented on KAFKA-4004: --- GitHub user onurkaraman opened a pull reque

[jira] [Commented] (KAFKA-4004) NetworkReceive.complete() should not throw NullPointerException after partially reading the size

[ https://issues.apache.org/jira/browse/KAFKA-4004?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15398011#comment-15398011 ] ASF GitHub Bot commented on KAFKA-4004: --- Github user onurkaraman closed the pull req

[GitHub] kafka pull request #1677: KAFKA-4004: NetworkReceive.complete() should not t...

Github user onurkaraman closed the pull request at: https://github.com/apache/kafka/pull/1677 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] kafka pull request #1678: KAFKA-3852: Clarify how to handle message format u...

GitHub user gwenshap opened a pull request: https://github.com/apache/kafka/pull/1678 KAFKA-3852: Clarify how to handle message format upgrade without kill… …ing performance You can merge this pull request into a Git repository by running: $ git pull https://github.com/gwen

[jira] [Resolved] (KAFKA-4004) NetworkReceive.complete() should not throw NullPointerException after partially reading the size

[ https://issues.apache.org/jira/browse/KAFKA-4004?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Onur Karaman resolved KAFKA-4004. - Resolution: Not A Bug Okay yeah the code is doing the right thing. I missed an earlier OutOfMemor

[jira] [Commented] (KAFKA-3852) Clarify how to handle message format upgrade without killing performance

[ https://issues.apache.org/jira/browse/KAFKA-3852?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15398012#comment-15398012 ] ASF GitHub Bot commented on KAFKA-3852: --- GitHub user gwenshap opened a pull request:

[jira] [Commented] (KAFKA-3990) Kafka New Producer may raise an OutOfMemoryError

[ https://issues.apache.org/jira/browse/KAFKA-3990?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15398115#comment-15398115 ] Jason Gustafson commented on KAFKA-3990: [~bric3] This kind of error is typically

Re: [DISCUSS] KIP-66 Kafka Connect Transformers for messages

Some thoughts on the KIP and single-message transforms in general. * When does transformation take place? In the KIP, it seems like the connector-implemented task is responsible for calling into the transformation logic. I'd propose that, - for source connectors, the transformer chain operates o

[GitHub] kafka pull request #1676: MINOR: Consumer should throw KafkaException on inv...

Github user asfgit closed the pull request at: https://github.com/apache/kafka/pull/1676 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enab

[jira] [Created] (KAFKA-4005) Add per topic compression ratio in broker and consumer.

Jiangjie Qin created KAFKA-4005: --- Summary: Add per topic compression ratio in broker and consumer. Key: KAFKA-4005 URL: https://issues.apache.org/jira/browse/KAFKA-4005 Project: Kafka Issue Typ

[jira] [Assigned] (KAFKA-4005) Add per topic compression ratio in broker and consumer.

[ https://issues.apache.org/jira/browse/KAFKA-4005?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jiangjie Qin reassigned KAFKA-4005: --- Assignee: Jiangjie Qin > Add per topic compression ratio in broker and consumer. > --

[GitHub] kafka pull request #1678: KAFKA-3852: Clarify how to handle message format u...

Github user asfgit closed the pull request at: https://github.com/apache/kafka/pull/1678 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enab

[jira] [Resolved] (KAFKA-3852) Clarify how to handle message format upgrade without killing performance

[ https://issues.apache.org/jira/browse/KAFKA-3852?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ewen Cheslack-Postava resolved KAFKA-3852. -- Resolution: Fixed Fix Version/s: 0.10.1.0 Issue resolved by pull request

[jira] [Commented] (KAFKA-3852) Clarify how to handle message format upgrade without killing performance

[ https://issues.apache.org/jira/browse/KAFKA-3852?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15398167#comment-15398167 ] ASF GitHub Bot commented on KAFKA-3852: --- Github user asfgit closed the pull request

[jira] [Commented] (KAFKA-4000) Consumer per-topic metrics do not aggregate partitions from the same topic

[ https://issues.apache.org/jira/browse/KAFKA-4000?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15398270#comment-15398270 ] Vahid Hashemian commented on KAFKA-4000: [~hachikuji] Are you referring to [this

[jira] [Commented] (KAFKA-3990) Kafka New Producer may raise an OutOfMemoryError

[ https://issues.apache.org/jira/browse/KAFKA-3990?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15398283#comment-15398283 ] Jay Kreps commented on KAFKA-3990: -- [~hachikuji] But the response format is dictated by t

[jira] [Commented] (KAFKA-3990) Kafka New Producer may raise an OutOfMemoryError

[ https://issues.apache.org/jira/browse/KAFKA-3990?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15398290#comment-15398290 ] Ismael Juma commented on KAFKA-3990: [~jkreps], that's the idea, but bugs have been re

[jira] [Comment Edited] (KAFKA-3990) Kafka New Producer may raise an OutOfMemoryError

[ https://issues.apache.org/jira/browse/KAFKA-3990?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15398290#comment-15398290 ] Ismael Juma edited comment on KAFKA-3990 at 7/28/16 10:02 PM: --

Build failed in Jenkins: kafka-trunk-jdk7 #1448

See Changes: [ismael] MINOR: Consumer should throw KafkaException on invalid checksum -- [...truncated 6250 lines...] kafka.log.LogTest > testThatGarbageCollectingSegmentsDoesntChangeOffset STA

[jira] [Commented] (KAFKA-4000) Consumer per-topic metrics do not aggregate partitions from the same topic

[ https://issues.apache.org/jira/browse/KAFKA-4000?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15398364#comment-15398364 ] Jason Gustafson commented on KAFKA-4000: [~vahid] Yes, that's the line I was refer

[jira] [Commented] (KAFKA-4000) Consumer per-topic metrics do not aggregate partitions from the same topic

[ https://issues.apache.org/jira/browse/KAFKA-4000?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15398373#comment-15398373 ] Vahid Hashemian commented on KAFKA-4000: [~hachikuji] Yes, thanks for confirming.

[jira] [Assigned] (KAFKA-4000) Consumer per-topic metrics do not aggregate partitions from the same topic

[ https://issues.apache.org/jira/browse/KAFKA-4000?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vahid Hashemian reassigned KAFKA-4000: -- Assignee: Vahid Hashemian > Consumer per-topic metrics do not aggregate partitions from

Build failed in Jenkins: kafka-trunk-jdk8 #784

See Changes: [ismael] MINOR: Consumer should throw KafkaException on invalid checksum [me] KAFKA-3852: Clarify how to handle message format upgrade without killing -- [...truncated 1406 lines...

Build failed in Jenkins: kafka-0.10.0-jdk7 #169

See Changes: [ismael] MINOR: Consumer should throw KafkaException on invalid checksum [me] KAFKA-3852: Clarify how to handle message format upgrade without -- [...truncated 5762 lines...] org.a

[GitHub] kafka pull request #1673: Hotfix: fixed instable Streams application reset i...

Github user mjsax closed the pull request at: https://github.com/apache/kafka/pull/1673 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabl

Re: [DISCUSS] KIP-66 Kafka Connect Transformers for messages

On Thu, Jul 28, 2016 at 1:13 PM, Shikhar Bhushan wrote: > Some thoughts on the KIP and single-message transforms in general. > > * When does transformation take place? In the KIP, it seems like the > connector-implemented task is responsible for calling into the > transformation logic. I'd propos

Build failed in Jenkins: kafka-trunk-jdk7 #1449

See Changes: [me] KAFKA-3852: Clarify how to handle message format upgrade without killing -- [...truncated 5408 lines...] kafka.api.test.ProducerCompressionTest > testCompression[2] STARTED k

[jira] [Updated] (KAFKA-3665) Default ssl.endpoint.identification.algorithm should be https

[ https://issues.apache.org/jira/browse/KAFKA-3665?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ismael Juma updated KAFKA-3665: --- Affects Version/s: 0.10.0.0 Status: In Progress (was: Patch Available) > Default ssl.e

Jenkins build is back to normal : kafka-0.10.0-jdk7 #170

See

Jenkins build is back to normal : kafka-trunk-jdk7 #1450

See

Re: [DISCUSS] KIP-4 ACL Admin Schema

Grant, Thanks for the writeup. Is there any benefit for sending the AlterAcls request to the controller? The controller is currently only designed for sending topic level metadata. Jun On Thu, Jul 14, 2016 at 9:09 AM, Grant Henke wrote: > The KIP-4 Delete Topic Schema vote has passed and the p

Build failed in Jenkins: kafka-trunk-jdk8 #785

See Changes: [ismael] HOTFIX: Fix unstable Streams application reset integration test -- [...truncated 8363 lines...] org.apache.kafka.common.utils.UtilsTest > testGetHost STARTED org.apache.ka

Re: [DISCUSS] KIP-4 ACL Admin Schema

In the database world (Resource, ACL) pair is typically called a "grant". (sorry) You "grant" permission on a resource to a user. http://dev.mysql.com/doc/refman/5.7/en/show-grants.html Gwen On Fri, Jul 22, 2016 at 4:13 AM, Jim Jagielski wrote: > >> On Jul 21, 2016, at 10:57 PM, Ismael Juma

Re: [DISCUSS] KIP-66 Kafka Connect Transformers for messages

> > > Hmm, operating on ConnectRecords probably doesn't work since you need to > emit the right type of record, which might mean instantiating a new one. I > think that means we either need 2 methods, one for SourceRecord, one for > SinkRecord, or we'd need to limit what parts of the message you ca