Re: Migrating from 07.1 .100

There has been a lot of changes, and it can also be quit a challenge to get the SSL working. And with only a null-pointer, there is little to go on. I would first focus on getting it to work with 0.10 (if possible also on a clustered test setup), and if it all works, try to configure the ssl. On M

[jira] [Commented] (KAFKA-2857) ConsumerGroupCommand throws GroupCoordinatorNotAvailableException when describing a non-existent group before the offset topic is created

[ https://issues.apache.org/jira/browse/KAFKA-2857?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15328866#comment-15328866 ] Jason Gustafson commented on KAFKA-2857: [~vahid] That's a good question. Maybe we

[jira] [Commented] (KAFKA-3686) Kafka producer is not fault tolerant

[ https://issues.apache.org/jira/browse/KAFKA-3686?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15328781#comment-15328781 ] radha commented on KAFKA-3686: -- We have a similar issue and even with a very very large retry

Re: [DISCUSS] KIP-4 Create Topic Schema

Thanks for the review Jun. You probably want to make it clearer if timeout > 0, what waiting for topic > metadata is "complete" means. In the first implementation, it really means > that the topic metadata is propagated to the controller's metadata cache. I updated the wiki to be more descriptiv

Re: [DISCUSS] KIP-4 Create Topic Schema

Thanks for the review Gwen. 1. The replica assignment protocol takes [replicas], there is the > implicit assumption that the first replica is the leader. This matches > current behavior elsewhere, but lets document it explicitly. I added this to the wiki and will update the protocol doc string i

Re: [DISCUSS] KIP-63: Unify store and downstream caching in streams

Although this KIP is not mainly for memory management of Kafka Streams, since it touches on quite some part of it I think it is good to first think of what we would REALLY want as an end goal for memory usage in order to make sure that whatever we proposed in this KIP aligns with that long-term pla

Re: [DISCUSS] KIP-4 Create Topic Schema

Hi Jay, Good point one of the main benefits of the create topic api is removing the server side auto create. The work is noted in the Follow Up Changes

Re: [DISCUSS] KIP-62: Allow consumer to send heartbeats from a background thread

Hey Becket, Sorry for the late response. I agree there's a little more to think through on the implementation. The offset commit is the tricky one since we could execute a user-provided callback. I'm thinking if there is an inflight request to the coordinator, we may simply skip the heartbeat and

[jira] [Commented] (KAFKA-2857) ConsumerGroupCommand throws GroupCoordinatorNotAvailableException when describing a non-existent group before the offset topic is created

[ https://issues.apache.org/jira/browse/KAFKA-2857?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15328708#comment-15328708 ] Vahid Hashemian commented on KAFKA-2857: [~hachikuji] Could you also clarify what

[GitHub] kafka pull request #1502: MINOR: Clarify the at least once delivery with aut...

GitHub user Ishiihara opened a pull request: https://github.com/apache/kafka/pull/1502 MINOR: Clarify the at least once delivery with auto commit enabled. You can merge this pull request into a Git repository by running: $ git pull https://github.com/Ishiihara/kafka fetcher-co

[jira] [Commented] (KAFKA-3209) Support single message transforms in Kafka Connect

[ https://issues.apache.org/jira/browse/KAFKA-3209?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15328647#comment-15328647 ] Ewen Cheslack-Postava commented on KAFKA-3209: -- [~pablocasares] I don't think

Jenkins build is back to normal : kafka-trunk-jdk8 #693

See

[GitHub] kafka pull request #1501: MINOR: Expose window store sequence number

GitHub user theduderog opened a pull request: https://github.com/apache/kafka/pull/1501 MINOR: Expose window store sequence number @guozhangwang @mjsax @enothereska Currently, Kafka Streams does not have a util to get access to the sequence number added to the key of windo

Re: [DISCUSS] KIP-63: Unify store and downstream caching in streams

I am just catching up on this thread. From my point of view, easy tuning for the user is the most important thing, because Kafka Streams is a library. Thus, a global cache size parameter should be the best. About dividing the memory vs a single global cache. I would argue that in the first place

Re: [VOTE] KIP-33 - Add a time based log index

Thanks Jiangjie, I see the need for sensitive data purging, the above proposed change LGTM. One minor concern is that a wrongly marked timestamp on the first record could cause the segment to roll much later / earlier, though it may be rare. Guozhang On Fri, Jun 10, 2016 at 10:07 AM, Becket Qin

[jira] [Commented] (KAFKA-3786) Avoid unused property from parent configs causing WARN entries

[ https://issues.apache.org/jira/browse/KAFKA-3786?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15328287#comment-15328287 ] ASF GitHub Bot commented on KAFKA-3786: --- Github user asfgit closed the pull request

[GitHub] kafka pull request #1465: KAFKA-3786: Let ConfigDef filter property key valu...

Github user asfgit closed the pull request at: https://github.com/apache/kafka/pull/1465 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enab

[jira] [Resolved] (KAFKA-3786) Avoid unused property from parent configs causing WARN entries

[ https://issues.apache.org/jira/browse/KAFKA-3786?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guozhang Wang resolved KAFKA-3786. -- Resolution: Fixed Fix Version/s: 0.10.1.0 Issue resolved by pull request 1465 [https://gi

[jira] [Commented] (KAFKA-3576) Unify KStream and KTable API

[ https://issues.apache.org/jira/browse/KAFKA-3576?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15328242#comment-15328242 ] Matthias J. Sax commented on KAFKA-3576: I see two points: (1) this is similar to

[GitHub] kafka pull request #1500: MINOR: Fix javadoc typos in ConsumerRebalanceListe...

GitHub user vahidhashemian opened a pull request: https://github.com/apache/kafka/pull/1500 MINOR: Fix javadoc typos in ConsumerRebalanceListener You can merge this pull request into a Git repository by running: $ git pull https://github.com/vahidhashemian/kafka typo07/fix_ja

RE: Migrating from 07.1 .100

Hi, I currently embed kafka 0.7.1 in my java process. To support SSL, we have decided to upgrade Kafka to 0.10.0. After upgrade, I am seeing following error during kafka startup. java.lang.NullPointerException at kafka.utils.Throttler.(Throttler.scala:45) at kafka.

[jira] [Updated] (KAFKA-3831) Preparation for updating the default partition assignment strategy of Mirror Maker to round robin

[ https://issues.apache.org/jira/browse/KAFKA-3831?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vahid Hashemian updated KAFKA-3831: --- Status: Patch Available (was: Open) > Preparation for updating the default partition assignme

[jira] [Commented] (KAFKA-3831) Preparation for updating the default partition assignment strategy of Mirror Maker to round robin

[ https://issues.apache.org/jira/browse/KAFKA-3831?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15327770#comment-15327770 ] ASF GitHub Bot commented on KAFKA-3831: --- GitHub user vahidhashemian opened a pull re

[GitHub] kafka pull request #1499: KAFKA-3831: Prepare for updating Mirror Maker's de...

GitHub user vahidhashemian opened a pull request: https://github.com/apache/kafka/pull/1499 KAFKA-3831: Prepare for updating Mirror Maker's default partition assignment strategy to round robin This patch adds proper warning message and necessary doc updates for updating the default

[jira] [Created] (KAFKA-3834) Consumer should not block in poll on coordinator discovery

Jason Gustafson created KAFKA-3834: -- Summary: Consumer should not block in poll on coordinator discovery Key: KAFKA-3834 URL: https://issues.apache.org/jira/browse/KAFKA-3834 Project: Kafka

[jira] [Updated] (KAFKA-3833) Feature Request: allow Kafka clients to retrieve Kafka broker versions

[ https://issues.apache.org/jira/browse/KAFKA-3833?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hari Sekhon updated KAFKA-3833: --- Description: Feature request for the ability of Kafka clients to query the Kafka brokers for their ve

[jira] [Created] (KAFKA-3833) Feature Request: allow Kafka clients to retrieve Kafka broker versions

Hari Sekhon created KAFKA-3833: -- Summary: Feature Request: allow Kafka clients to retrieve Kafka broker versions Key: KAFKA-3833 URL: https://issues.apache.org/jira/browse/KAFKA-3833 Project: Kafka

[jira] [Created] (KAFKA-3832) Kafka Connect's JSON Converter never outputs a null value

Randall Hauch created KAFKA-3832: Summary: Kafka Connect's JSON Converter never outputs a null value Key: KAFKA-3832 URL: https://issues.apache.org/jira/browse/KAFKA-3832 Project: Kafka Issue

[jira] [Created] (KAFKA-3831) Preparation for updating the default partition assignment strategy of Mirror Maker to round robin

Vahid Hashemian created KAFKA-3831: -- Summary: Preparation for updating the default partition assignment strategy of Mirror Maker to round robin Key: KAFKA-3831 URL: https://issues.apache.org/jira/browse/KAFKA-383

[jira] [Commented] (KAFKA-3818) Change Mirror Maker default assignment strategy to round robin

[ https://issues.apache.org/jira/browse/KAFKA-3818?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15327629#comment-15327629 ] Vahid Hashemian commented on KAFKA-3818: [~ewencp] [~hachikuji] I suggest we keep

[jira] [Commented] (KAFKA-1464) Add a throttling option to the Kafka replication tool

[ https://issues.apache.org/jira/browse/KAFKA-1464?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15327561#comment-15327561 ] Ralph Weires commented on KAFKA-1464: - We have similar problems as described by Jason

[jira] [Updated] (KAFKA-3818) Change Mirror Maker default assignment strategy to round robin

[ https://issues.apache.org/jira/browse/KAFKA-3818?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vahid Hashemian updated KAFKA-3818: --- Status: Open (was: Patch Available) > Change Mirror Maker default assignment strategy to roun

[jira] [Commented] (KAFKA-3828) Consumer thread stalls after consumer re balance for some partition

[ https://issues.apache.org/jira/browse/KAFKA-3828?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15327513#comment-15327513 ] Joseph Aliase commented on KAFKA-3828: -- Sure will test and let you know > Consumer t

[jira] [Commented] (KAFKA-3828) Consumer thread stalls after consumer re balance for some partition

[ https://issues.apache.org/jira/browse/KAFKA-3828?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15327509#comment-15327509 ] Ismael Juma commented on KAFKA-3828: Yes, upgrading the consumer to 0.9.0.1 or 0.10.0.

[jira] [Commented] (KAFKA-3828) Consumer thread stalls after consumer re balance for some partition

[ https://issues.apache.org/jira/browse/KAFKA-3828?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15327488#comment-15327488 ] Joseph Aliase commented on KAFKA-3828: -- I assume you are asking to upgrade to Consume

[jira] [Commented] (KAFKA-3828) Consumer thread stalls after consumer re balance for some partition

[ https://issues.apache.org/jira/browse/KAFKA-3828?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15327472#comment-15327472 ] Ismael Juma commented on KAFKA-3828: Thanks. There are known consumer bugs in 0.9.0.0

[jira] [Commented] (KAFKA-3828) Consumer thread stalls after consumer re balance for some partition

[ https://issues.apache.org/jira/browse/KAFKA-3828?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15327462#comment-15327462 ] Joseph Aliase commented on KAFKA-3828: -- Broker version: 0.10.0.0 and 0.9.0.1 Consume

[jira] [Commented] (KAFKA-2394) Use RollingFileAppender by default in log4j.properties

[ https://issues.apache.org/jira/browse/KAFKA-2394?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15327390#comment-15327390 ] Dustin Cote commented on KAFKA-2394: [~ewencp] yeah I think that's for the best. I wo

[jira] [Updated] (KAFKA-3830) getTGT() debug logging exposes confidential information

[ https://issues.apache.org/jira/browse/KAFKA-3830?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ismael Juma updated KAFKA-3830: --- Status: Patch Available (was: Open) > getTGT() debug logging exposes confidential information > -

[jira] [Commented] (KAFKA-3830) getTGT() debug logging exposes confidential information

[ https://issues.apache.org/jira/browse/KAFKA-3830?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15327353#comment-15327353 ] ASF GitHub Bot commented on KAFKA-3830: --- GitHub user ijuma opened a pull request:

[GitHub] kafka pull request #1498: KAFKA-3830; getTGT() debug logging exposes confide...

GitHub user ijuma opened a pull request: https://github.com/apache/kafka/pull/1498 KAFKA-3830; getTGT() debug logging exposes confidential information Only log the client and server principals, which is what ZooKeeper does after ZOOKEEPER-2405. You can merge this pull request into

[jira] [Commented] (KAFKA-3576) Unify KStream and KTable API

[ https://issues.apache.org/jira/browse/KAFKA-3576?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15327335#comment-15327335 ] Eno Thereska commented on KAFKA-3576: - I understand the goal, but why is the first opt

[jira] [Commented] (KAFKA-3775) Throttle maximum number of tasks assigned to a single KafkaStreams

[ https://issues.apache.org/jira/browse/KAFKA-3775?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15327326#comment-15327326 ] Matthias J. Sax commented on KAFKA-3775: [~tenggyut] [~wushujames] Kafka Streams a

[jira] [Created] (KAFKA-3830) getTGT() debug logging exposes confidential information

Ismael Juma created KAFKA-3830: -- Summary: getTGT() debug logging exposes confidential information Key: KAFKA-3830 URL: https://issues.apache.org/jira/browse/KAFKA-3830 Project: Kafka Issue Type:

Re: [VOTE] KIP-55: Secure quotas for authenticated users

I have updated KIP-55 to reflect the changes from the discussions in the voting thread ( https://www.mail-archive.com/dev@kafka.apache.org/msg51610.html). Jun/Gwen, Existing client-id quotas will be used as default client-id quotas for users when no user quotas are configured - i.e., default user

[jira] [Commented] (KAFKA-3209) Support single message transforms in Kafka Connect

[ https://issues.apache.org/jira/browse/KAFKA-3209?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15327225#comment-15327225 ] Pablo Casares commented on KAFKA-3209: -- Hey, I'm interested in produce keys based on

Re: [VOTE] KIP-55: Secure quotas for authenticated users

Thank you all for the feedback. Closing this voting thread to continue discussions on the updated KIP in the discuss thread. On Sat, Jun 11, 2016 at 7:45 AM, Gwen Shapira wrote: > I'd also like to see clarification regarding the ZK structures. > Currently they appear as if user-quotas and client

Re: Kafka Streaming - Window expiration

Many Thanks Eno. I will try .until method. Cheers Pari On 13 June 2016 at 14:10, Eno Thereska wrote: > Hi Pari, > > Try the .until method like this: > > > (TimeWindows) TimeWindows.of("tumbling-window-example", > windowSizeMs).until(60 * 1000L) > > Thanks > Eno > > > > On 13 Jun 2016, at 08:31

[jira] [Commented] (KAFKA-3828) Consumer thread stalls after consumer re balance for some partition

[ https://issues.apache.org/jira/browse/KAFKA-3828?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15327066#comment-15327066 ] Ismael Juma commented on KAFKA-3828: Can you please include the consumer and broker ve

[jira] [Created] (KAFKA-3829) Warn that kafka-connect group.id must not conflice with connector names

Barry Kaplan created KAFKA-3829: --- Summary: Warn that kafka-connect group.id must not conflice with connector names Key: KAFKA-3829 URL: https://issues.apache.org/jira/browse/KAFKA-3829 Project: Kafka

Re: Kafka Streaming - Window expiration

Hi Pari, Try the .until method like this: > (TimeWindows) TimeWindows.of("tumbling-window-example", > windowSizeMs).until(60 * 1000L) Thanks Eno > On 13 Jun 2016, at 08:31, Pariksheet Barapatre wrote: > > Hello Experts, > > As per documentation in kafka docs - > *Windowing* is a common pre

[jira] [Commented] (KAFKA-3775) Throttle maximum number of tasks assigned to a single KafkaStreams

[ https://issues.apache.org/jira/browse/KAFKA-3775?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15326937#comment-15326937 ] James Cheng commented on KAFKA-3775: [~kawamuray], for the network traffic, could you

Kafka Streaming - Window expiration

Hello Experts, As per documentation in kafka docs - *Windowing* is a common prerequisite for stateful transformations which group records in a stream, for example, by their timestamps. A local state store is usually needed for a windowing operation to store recently received records based on the w