[GitHub] kafka-site issue #64: MINOR: Add release dates to downloads page

Github user ewencp commented on the issue: https://github.com/apache/kafka-site/pull/64 I can dig up the release dates, I just grabbed the ones that were readily available via the git history for this file (which, admittedly, might be slightly off). I'm happy to round them out via

Jenkins build is back to normal : kafka-trunk-jdk8 #1785

See

[GitHub] kafka pull request #3488: KAFKA-5557: Using a logPrefix inside the StreamPar...

Github user asfgit closed the pull request at: https://github.com/apache/kafka/pull/3488 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

Re: [DISCUSS] KIP-167: Add interface for the state store restoration process

Bill, Thanks for the updated KIP. From your WIP PR I think one piece of information that still need to be included in the KIP is that the library will check the type of the restoreCallback in runtime, and if it is extending StateRestoreListener it will also execute its functions at the runtime.

[GitHub] kafka-site issue #64: MINOR: Add release dates to downloads page

Github user guozhangwang commented on the issue: https://github.com/apache/kafka-site/pull/64 What about pre-0.8.2 releases? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] kafka-site pull request #64: MINOR: Add release dates to downloads page

GitHub user ewencp opened a pull request: https://github.com/apache/kafka-site/pull/64 MINOR: Add release dates to downloads page You can merge this pull request into a Git repository by running: $ git pull https://github.com/ewencp/kafka-site release-dates Alternatively you

[GitHub] kafka-site issue #64: MINOR: Add release dates to downloads page

Github user ewencp commented on the issue: https://github.com/apache/kafka-site/pull/64 @ijuma @guozhangwang @junrao I think it'd be good to start including these to help indicate to users how old the releases they are downloading are. --- If your project is set up for it, you can

[GitHub] kafka pull request #3449: KAFKA-5167: Release state locks in case of failure

Github user asfgit closed the pull request at: https://github.com/apache/kafka/pull/3449 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] kafka pull request #3311: MINOR: fix local Docker setup

Github user mjsax closed the pull request at: https://github.com/apache/kafka/pull/3311 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

Re: [ANNOUNCE] New Kafka PMC member Ismael Juma

Congrats! On 7/5/17 3:42 PM, Roger Hoover wrote: > Well deserved, indeed! Congrats, Ismael. > > On Wed, Jul 5, 2017 at 3:24 PM, Damian Guy wrote: > >> Congratulations Ismael! Very well deserved. >> Cheers, >> Damian >> On Wed, 5 Jul 2017 at 22:54, Dong Lin

[jira] [Created] (KAFKA-5559) Metrics should throw if two client registers with same ID

Matthias J. Sax created KAFKA-5559: -- Summary: Metrics should throw if two client registers with same ID Key: KAFKA-5559 URL: https://issues.apache.org/jira/browse/KAFKA-5559 Project: Kafka

Build failed in Jenkins: kafka-trunk-jdk8 #1784

See Changes: [wangguoz] MINOR: add IQ docs to streams documentation [wangguoz] KAFKA-5525: Streams reset tool should have same console output with or -- [...truncated

Re: Transactional Consistency beyond KIP-98 (transactional consumer)

Hello Werner, 1) Regarding the long transaction, we did discuss about this in the design doc ( https://docs.google.com/document/d/11Jqy_GjUGtdXJK94XGsEIK7CP1SnQGdp2eF0wSw9ra8/edit#bookmark=id.z9b0a6k2a3zm), and as stated the main reason we chose to go with the current design is to preserve offset

Jenkins build is back to normal : kafka-trunk-jdk7 #2491

See

Mirroring multiple clusters into one

The literature suggests running the MM on the target cluster when possible (with the exception of when encryption is required for transferred data). I am wondering if this is still the recommended approach when mirroring from multiple clusters to a single cluster (i.e. multiple MM instances). Is

Re: [ANNOUNCE] New Kafka PMC member Ismael Juma

Well deserved, indeed! Congrats, Ismael. On Wed, Jul 5, 2017 at 3:24 PM, Damian Guy wrote: > Congratulations Ismael! Very well deserved. > Cheers, > Damian > On Wed, 5 Jul 2017 at 22:54, Dong Lin wrote: > > > Congratulations Ismael! > > > > On Wed,

Re: [ANNOUNCE] New Kafka PMC member Ismael Juma

Congratulations Ismael! Very well deserved. Cheers, Damian On Wed, 5 Jul 2017 at 22:54, Dong Lin wrote: > Congratulations Ismael! > > On Wed, Jul 5, 2017 at 1:55 PM, Jun Rao wrote: > > > Hi, Everyone, > > > > Ismael Juma has been active in the Kafka

Re: RocksDB flushing issue on 0.10.2 streams

Thanks Greg. I'll look into it more tomorrow. Just finding it difficult to reproduce in a test. Thanks for providing the sequence, gives me something to try and repo. Appreciated. Thanks, Damian On Wed, 5 Jul 2017 at 19:57, Greg Fodor wrote: > Also, the sequence of events is:

[GitHub] kafka pull request #3443: KAFKA-5525: Streams reset tool should have same co...

Github user asfgit closed the pull request at: https://github.com/apache/kafka/pull/3443 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

Build failed in Jenkins: kafka-trunk-jdk7 #2490

See Changes: [rajinisivaram] MINOR: Wait for tasks to terminate to avoid exception in test teardown -- [...truncated 967.74 KB...] kafka.producer.AsyncProducerTest >

Re: [ANNOUNCE] New Kafka PMC member Ismael Juma

Congratulations Ismael! On Wed, Jul 5, 2017 at 1:55 PM, Jun Rao wrote: > Hi, Everyone, > > Ismael Juma has been active in the Kafka community since he became > a Kafka committer about a year ago. I am glad to announce that Ismael is > now a member of Kafka PMC. > >

Re: [ANNOUNCE] New Kafka PMC member Ismael Juma

Congratulations Ismael! --Vahid From: Rajini Sivaram To: dev Date: 07/05/2017 02:06 PM Subject:Re: [ANNOUNCE] New Kafka PMC member Ismael Juma Congratulations, Ismael! On Wed, Jul 5, 2017 at 9:55 PM, Jun Rao

Re: [ANNOUNCE] New Kafka PMC member Ismael Juma

Congrats, Ismael !! Guozhang On Wed, Jul 5, 2017 at 2:06 PM, Rajini Sivaram wrote: > Congratulations, Ismael! > > On Wed, Jul 5, 2017 at 9:55 PM, Jun Rao wrote: > > > Hi, Everyone, > > > > Ismael Juma has been active in the Kafka community since he

[GitHub] kafka pull request #3484: MINOR: add IQ docs to streams documentation

Github user asfgit closed the pull request at: https://github.com/apache/kafka/pull/3484 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

Re: [ANNOUNCE] New Kafka PMC member Ismael Juma

Congratulations Ismael! On Wed, Jul 5, 2017 at 2:06 PM, Rajini Sivaram wrote: > Congratulations, Ismael! > > On Wed, Jul 5, 2017 at 9:55 PM, Jun Rao wrote: > > > Hi, Everyone, > > > > Ismael Juma has been active in the Kafka community since he became

[GitHub] kafka pull request #3483: MINOR: Wait for tasks to terminate to avoid except...

Github user asfgit closed the pull request at: https://github.com/apache/kafka/pull/3483 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

Re: [ANNOUNCE] New Kafka PMC member Ismael Juma

Congratulations, Ismael! On Wed, Jul 5, 2017 at 9:55 PM, Jun Rao wrote: > Hi, Everyone, > > Ismael Juma has been active in the Kafka community since he became > a Kafka committer about a year ago. I am glad to announce that Ismael is > now a member of Kafka PMC. > >

[ANNOUNCE] New Kafka PMC member Ismael Juma

Hi, Everyone, Ismael Juma has been active in the Kafka community since he became a Kafka committer about a year ago. I am glad to announce that Ismael is now a member of Kafka PMC. Congratulations, Ismael! Jun

Re: RocksDB flushing issue on 0.10.2 streams

Also, the sequence of events is: - Job starts, rebalance happens, things run along smoothly. - After 10 minutes (retrospectively) the cleanup task kicks on and removes some directories - Tasks immediately start failing when trying to flush their state stores On Wed, Jul 5, 2017 at 11:55 AM,

Re: RocksDB flushing issue on 0.10.2 streams

The issue I am hitting is not the directory locking issues we've seen in the past. The issue seems to be, as you mentioned, that the state dir is getting deleted by the store cleanup process, but there are still tasks running that are trying to flush the state store. It seems more than a little

[jira] [Resolved] (KAFKA-5528) Error while reading topic, offset, partition info from process method

[ https://issues.apache.org/jira/browse/KAFKA-5528?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthias J. Sax resolved KAFKA-5528. Resolution: Not A Bug > Error while reading topic, offset, partition info from process

[jira] [Reopened] (KAFKA-5528) Error while reading topic, offset, partition info from process method

[ https://issues.apache.org/jira/browse/KAFKA-5528?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthias J. Sax reopened KAFKA-5528: > Error while reading topic, offset, partition info from process method >

Re: [VOTE]: KIP-149: Enabling key access in ValueTransformer, ValueMapper, and ValueJoiner

+1 On 6/27/17 1:41 PM, Jeyhun Karimov wrote: > Dear all, > > I would like to start the vote on KIP-149 [1]. > > > Cheers, > Jeyhun > > > [1] > https://cwiki.apache.org/confluence/display/KAFKA/KIP-149%3A+Enabling+key+access+in+ValueTransformer%2C+ValueMapper%2C+and+ValueJoiner >

Transactional Consistency beyond KIP-98 (transactional consumer)

While the new transactional consistency feature is definitely a move into the right direction, I find the current design limitations quite severe. Understandable but severe. First is the offset being the add-time, not the commit time. The consequence of that is, if one transactional producer

[jira] [Resolved] (KAFKA-3823) QuickStart documentation is still showing MirrorMakers supports more than one consumer.config

[ https://issues.apache.org/jira/browse/KAFKA-3823?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Evgeny Veretennikov resolved KAFKA-3823. Resolution: Fixed Closed issue, as [~tombentley] suggested. > QuickStart

Re: RocksDB flushing issue on 0.10.2 streams

BTW - i'm trying to reproduce it, but not having much luck so far... On Wed, 5 Jul 2017 at 09:27 Damian Guy wrote: > Thans for the updates Greg. There were some minor changes around this in > 0.11.0 to make it less likely to happen, but we've only ever seen the > locking

Re: kafka connect questions

Thanks Per for the feedback. My comments below. 1) There is a load balancer in front of the cluster. 2) Kafka does run in a cluster mode. I was referring to the producer that doesn't. Furthermore, the data sources are not heterogeneous and we cannot install anything on them. All we can do have

[GitHub] kafka pull request #3490: MINOR: Pass over DSL documentation up to stateful ...

GitHub user enothereska opened a pull request: https://github.com/apache/kafka/pull/3490 MINOR: Pass over DSL documentation up to stateful transformations You can merge this pull request into a Git repository by running: $ git pull https://github.com/enothereska/kafka

[GitHub] kafka pull request #3489: KAFKA-5556 : KafkaConsumer.commitSync throws Illeg...

GitHub user umesh9794 opened a pull request: https://github.com/apache/kafka/pull/3489 KAFKA-5556 : KafkaConsumer.commitSync throws IllegalStateException: A… This PR makes `commitOffsetsSync` method check whether future is completed after client's poll or not. Tests:

[jira] [Resolved] (KAFKA-5558) can not connect to the unsecure port after config SASL/PLAIN

[ https://issues.apache.org/jira/browse/KAFKA-5558?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] zhu fangbo resolved KAFKA-5558. --- Resolution: Fixed this is not a bug > can not connect to the unsecure port after config SASL/PLAIN >

[jira] [Created] (KAFKA-5558) can not connect to the unsecure port after config SASL/PLAIN

zhu fangbo created KAFKA-5558: - Summary: can not connect to the unsecure port after config SASL/PLAIN Key: KAFKA-5558 URL: https://issues.apache.org/jira/browse/KAFKA-5558 Project: Kafka Issue

[GitHub] kafka pull request #3488: KAFKA-5557: Using a logPrefix inside the StreamPar...

GitHub user ppatierno opened a pull request: https://github.com/apache/kafka/pull/3488 KAFKA-5557: Using a logPrefix inside the StreamPartitionAssignor Added logPrefix for avoiding stream thread name formatting replicated more times You can merge this pull request into a Git

[GitHub] kafka pull request #3482: MINOR: Make topic config a section, with TOC entry...

Github user tombentley closed the pull request at: https://github.com/apache/kafka/pull/3482 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

Re: [DISCUSS]: KIP-159: Introducing Rich functions to Streams

HI Jeyhun, Is the intention that these methods are new overloads on the KStream, KTable, etc? It is worth noting that a ProcessorContext is not a RecordContext. A RecordContext, as it stands, only exists during the processing of a single record. Whereas the ProcessorContext exists for the

[jira] [Created] (KAFKA-5557) Using a logPrefix inside the StreamPartitionAssignor

Paolo Patierno created KAFKA-5557: - Summary: Using a logPrefix inside the StreamPartitionAssignor Key: KAFKA-5557 URL: https://issues.apache.org/jira/browse/KAFKA-5557 Project: Kafka Issue

Re: RocksDB flushing issue on 0.10.2 streams

Thans for the updates Greg. There were some minor changes around this in 0.11.0 to make it less likely to happen, but we've only ever seen the locking fail in the event of a rebalance. When everything is running state dirs shouldn't be deleted if they are being used as the lock will fail. On

Re: kafka connect questions

Hi Clay, I agree with Gwen in thinking that you might want to take a second look at streaming protobuf data to Kafka and then having connectors read that from Kafka. To address the issues in order: 1. You say you have hundreds of machines sending data, but if you run a connector that is not tied

Re: kafka connect questions

Well I guess there is one: https://github.com/dhanuka84/kafka-connect-tcp Maybe you can use or build on top of that. On 05/07/17 05:45, Gwen Shapira wrote: I don't remember seeing one. There is no reason not to write one (let us know if you do, so we can put it on the connector hub!). Few

Re: [DISCUSS] KAFKA-4930 & KAFKA 4938 - Treatment of name parameter in create connector requests

Hi, regarding "breaking existing functionality" .. yes...that was me getting confused about intended and existing functionality :) You are right, this won't break anything that is currently working. I'll leave placement of "name" parameter as is and open a new issue to clarify this later on.

Re: RocksDB flushing issue on 0.10.2 streams

I can report that setting state.cleanup.delay.ms to a very large value (effectively disabling it) works around the issue. It seems that the state store cleanup process can somehow get out ahead of another task that still thinks it should be writing to the state store/flushing it. In my test runs,