[jira] [Commented] (KAFKA-4311) Multi layer cache eviction causes forwarding to incorrect ProcessorNode

[ https://issues.apache.org/jira/browse/KAFKA-4311?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15595593#comment-15595593 ] Frank Lyaruu commented on KAFKA-4311: - The ClassCast exception is gone, but I still see the Illegal

Re: [DISCUSS] KIP-80: Kafka REST Server

Here is Ignite’s - essentially idea is having different owners/maintainers per module/area that keep a check on that area. https://cwiki.apache.org/confluence/display/IGNITE/How+to+Contribute On 10/21/16, 3:44 PM, "isma...@gmail.com on behalf of Ismael Juma"

[GitHub] kafka pull request #2051: KAFKA-4311: Multi layer cache eviction causes forw...

GitHub user dguy opened a pull request: https://github.com/apache/kafka/pull/2051 KAFKA-4311: Multi layer cache eviction causes forwarding to incorrect ProcessorNode Given a topology like the one below. If a record arriving in `tableOne` causes a cache eviction, it will trigger

[jira] [Commented] (KAFKA-4311) Multi layer cache eviction causes forwarding to incorrect ProcessorNode

[ https://issues.apache.org/jira/browse/KAFKA-4311?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15595401#comment-15595401 ] ASF GitHub Bot commented on KAFKA-4311: --- GitHub user dguy opened a pull request:

Re: [DISCUSS] KIP-80: Kafka REST Server

Harsha, You seem to be saying that your only two options are to fork or duplicate the existing REST project or add it to Kafka to be able to contribute. I don't think those are the only two options. The other option is to contribute to the existing successful project--which is Apache licensed and

[jira] [Updated] (KAFKA-4311) Multi layer cache eviction causes forwarding to incorrect Processor Node

[ https://issues.apache.org/jira/browse/KAFKA-4311?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Damian Guy updated KAFKA-4311: -- Description: The two exceptions below were reported by Frank on the dev mailing list. After

Re: Kafka KIP meeting Oct 19 at 11:00am PST

I think this will be a good idea. It will separate an issue with concrete use-case and a major pain-point from a wider discussion about architectures and who is responsible for data formats. On Fri, Oct 21, 2016 at 12:57 AM, Michael Pearce wrote: > I had noted that what

Re: [DISCUSS] KIP-80: Kafka REST Server

Are you saying Kafka REST is subjective but Kafka Streams and Kafka Connect are not subjective? > "there are likely places that can live without a rest proxy" There are also places that can live without Kafka Streams and Kafka Connect. Nacho On Fri, Oct 21, 2016 at 11:17 AM, Jun Rao

Re: [jira] [Commented] (KAFKA-4113) Allow KTable bootstrap

I managed to track down one case where we were seeing issues with missing data when transitioning to a new node to being a retention policy on the topic. There is an additional case but have not been able to repro at this time. We recently fixed a problem where we were failing to properly

[jira] [Commented] (KAFKA-4099) Change the time based log rolling to only based on the message timestamp.

[ https://issues.apache.org/jira/browse/KAFKA-4099?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15596037#comment-15596037 ] Jun Rao commented on KAFKA-4099: I had two use cases of time-based rolling in mind. The first one is for

Re: [DISCUSS] KIP-80: Kafka REST Server

At the high level, I think ideally it makes sense to add a component to Apache Kafka if (1) it's widely needed and (2) it needs tight integration with Kafka core. For Kafka Stream, we do expect stream processing will be used widely in the future. Implementation wise, Kafka Stream only supports

Re: Kafka KIP meeting Oct 19 at 11:00am PST

I think a separate KIP is a good idea as well. Note however that potential decisions in this KIP could affect the other KIP. Nacho On Fri, Oct 21, 2016 at 10:23 AM, Jun Rao wrote: > Michael, > > Yes, doing a separate KIP to address the null payload issue for compacted >

Re: Kafka KIP meeting Oct 19 at 11:00am PST

Michael, Yes, doing a separate KIP to address the null payload issue for compacted topics is a good idea. Thanks, Jun On Fri, Oct 21, 2016 at 12:57 AM, Michael Pearce wrote: > I had noted that what ever the solution having compaction based on null > payload was agreed

[jira] [Created] (KAFKA-4331) Kafka Streams resetter is slow because it joins the same group for each topic

Roger Hoover created KAFKA-4331: --- Summary: Kafka Streams resetter is slow because it joins the same group for each topic Key: KAFKA-4331 URL: https://issues.apache.org/jira/browse/KAFKA-4331 Project:

[jira] [Commented] (KAFKA-3144) report members with no assigned partitions in ConsumerGroupCommand

[ https://issues.apache.org/jira/browse/KAFKA-3144?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15596540#comment-15596540 ] Vahid Hashemian commented on KAFKA-3144: [~hachikuji] suggested that we report the coordinator id

Re: [VOTE] KIP-73 - Replication Quotas

Thanks for catching that and the fix as well. Makes sense to me. We should consider adding an "amendments" section to KIPs - perhaps just a link to KAFKA-4313 would suffice in this case. Thanks, Joel On Wed, Oct 19, 2016 at 7:12 PM, Jun Rao wrote: > Hi, > > While testing

[jira] [Comment Edited] (KAFKA-3144) report members with no assigned partitions in ConsumerGroupCommand

[ https://issues.apache.org/jira/browse/KAFKA-3144?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15596540#comment-15596540 ] Vahid Hashemian edited comment on KAFKA-3144 at 10/21/16 10:31 PM: ---

[jira] [Created] (KAFKA-4332) kafka.api.UserQuotaTest.testThrottledProducerConsumer transient unit test failure

Jun Rao created KAFKA-4332: -- Summary: kafka.api.UserQuotaTest.testThrottledProducerConsumer transient unit test failure Key: KAFKA-4332 URL: https://issues.apache.org/jira/browse/KAFKA-4332 Project: Kafka

[jira] [Commented] (KAFKA-4296) LogCleaner CleanerStats swap logic seems incorrect

[ https://issues.apache.org/jira/browse/KAFKA-4296?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15596460#comment-15596460 ] ASF GitHub Bot commented on KAFKA-4296: --- Github user asfgit closed the pull request at:

[GitHub] kafka pull request #2052: MINOR: add list_topics command to help debug tests

Github user asfgit closed the pull request at: https://github.com/apache/kafka/pull/2052 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[jira] [Created] (KAFKA-4333) Report consumer group coordinator id when '--list' option is used

Vahid Hashemian created KAFKA-4333: -- Summary: Report consumer group coordinator id when '--list' option is used Key: KAFKA-4333 URL: https://issues.apache.org/jira/browse/KAFKA-4333 Project: Kafka

[GitHub] kafka pull request #2053: KAFKA-4326: Refactor LogCleaner for better reuse o...

GitHub user hachikuji opened a pull request: https://github.com/apache/kafka/pull/2053 KAFKA-4326: Refactor LogCleaner for better reuse of common copy/compress logic You can merge this pull request into a Git repository by running: $ git pull

[jira] [Commented] (KAFKA-4326) Refactor LogCleaner to remove duplicate log copying logic

[ https://issues.apache.org/jira/browse/KAFKA-4326?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15596601#comment-15596601 ] ASF GitHub Bot commented on KAFKA-4326: --- GitHub user hachikuji opened a pull request:

[jira] [Reopened] (KAFKA-2089) MetadataTest transient failure

[ https://issues.apache.org/jira/browse/KAFKA-2089?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jun Rao reopened KAFKA-2089: Reopening this issue. Saw the following transient failure in trunk. org.apache.kafka.clients.MetadataTest >

Re: [DISCUSS] KIP-80: Kafka REST Server

FWIW, Apache Kafka has evolved a lot from where it started. It did start as a messaging system. Over time we realized that that the vision for Kafka is to build a streaming platform and not just a messaging system. You can take a look at the site for more description about what comprises the

Re: [jira] [Commented] (KAFKA-4113) Allow KTable bootstrap

Great to know! Thanks Greg. Please keep us posted with any new finding you have. Guozhang On Fri, Oct 21, 2016 at 12:35 PM, Greg Fodor wrote: > I managed to track down one case where we were seeing issues with missing > data when transitioning to a new node to being a

[jira] [Resolved] (KAFKA-4296) LogCleaner CleanerStats swap logic seems incorrect

[ https://issues.apache.org/jira/browse/KAFKA-4296?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jason Gustafson resolved KAFKA-4296. Resolution: Fixed Fix Version/s: (was: 0.10.1.1) 0.10.2.0

[GitHub] kafka pull request #2016: KAFKA-4296: Fix LogCleaner statistics rolling

Github user asfgit closed the pull request at: https://github.com/apache/kafka/pull/2016 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] kafka pull request #2052: MINOR: add list_topics command to help debug tests

GitHub user xvrl opened a pull request: https://github.com/apache/kafka/pull/2052 MINOR: add list_topics command to help debug tests You can merge this pull request into a Git repository by running: $ git pull https://github.com/xvrl/kafka test-add-list-topics Alternatively

Jenkins build is back to normal : kafka-trunk-jdk8 #997

See

[jira] [Updated] (KAFKA-3144) report members with no assigned partitions in ConsumerGroupCommand

[ https://issues.apache.org/jira/browse/KAFKA-3144?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jason Gustafson updated KAFKA-3144: --- Resolution: Fixed Fix Version/s: 0.10.2.0 Status: Resolved (was: Patch

[jira] [Commented] (KAFKA-3144) report members with no assigned partitions in ConsumerGroupCommand

[ https://issues.apache.org/jira/browse/KAFKA-3144?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15596984#comment-15596984 ] ASF GitHub Bot commented on KAFKA-3144: --- Github user asfgit closed the pull request at:

[GitHub] kafka pull request #1336: KAFKA-3144: Report members with no assigned partit...

Github user asfgit closed the pull request at: https://github.com/apache/kafka/pull/1336 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] kafka pull request #2045: Fix for kafka-4295: kafka-console-consumer.sh does...

Github user amethystic closed the pull request at: https://github.com/apache/kafka/pull/2045 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[jira] [Commented] (KAFKA-4099) Change the time based log rolling to only based on the message timestamp.

[ https://issues.apache.org/jira/browse/KAFKA-4099?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15596965#comment-15596965 ] Jiangjie Qin commented on KAFKA-4099: - [~junrao] Thanks for the explanation. I agree that it is

[jira] [Updated] (KAFKA-4311) Multi layer cache eviction causes forwarding to incorrect ProcessorNode

[ https://issues.apache.org/jira/browse/KAFKA-4311?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Damian Guy updated KAFKA-4311: -- Summary: Multi layer cache eviction causes forwarding to incorrect ProcessorNode (was: Multi layer

Re: Kafka KIP meeting Oct 19 at 11:00am PST

++1 > On Oct 21, 2016, at 11:47 AM, Gwen Shapira wrote: > > I think this will be a good idea. It will separate an issue with concrete > use-case and a major pain-point from a wider discussion about architectures > and who is responsible for data formats. > > > On Fri, Oct

Re: [DISCUSS] KIP-80: Kafka REST Server

Sriram, "Can the streaming platform exist without stream processing? - No. Processing stream data again is a core part of streaming platform." Yes, it can. There are no.of Stream processing frameworks out there, and they all have integration into Kafka. It doesn't need to be developed within

Build failed in Jenkins: kafka-trunk-jdk7 #1649

See Changes: [jason] KAFKA-3144; Report members with no assigned partitions in -- [...truncated 14356 lines...] org.apache.kafka.streams.processor.internals.StreamPartitionAssignorTest >

[jira] [Commented] (KAFKA-4099) Change the time based log rolling to only based on the message timestamp.

[ https://issues.apache.org/jira/browse/KAFKA-4099?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15594273#comment-15594273 ] Jiangjie Qin commented on KAFKA-4099: - [~junrao] Is the purpose of log.rolling.ms to avoid the case

Re: [DISCUSS] KIP-80: Kafka REST Server

So from my reading essentially the first question needs to answered/and voted on is: Is Apache Kafka Community only about the Core or does the apache community also support some subprojects (and just we need some better way to manage this) If vote for Core only wins, then the following should

[GitHub] kafka pull request #2045: Fix for kafka-4295: kafka-console-consumer.sh does...

GitHub user amethystic reopened a pull request: https://github.com/apache/kafka/pull/2045 Fix for kafka-4295: kafka-console-consumer.sh does not delete the temporary group in zookeeper Since consumer stop logic and zk node removal code are in separate threads, so when two threads

Build failed in Jenkins: kafka-trunk-jdk8 #996

See Changes: [ismael] KAFKA-4309; Allow "pluggable" properties in KafkaService in System Tests -- [...truncated 1066 lines...] at

[jira] [Created] (KAFKA-4330) Leader/Follower Quota Rates in JMX have ambiguous names

Ben Stopford created KAFKA-4330: --- Summary: Leader/Follower Quota Rates in JMX have ambiguous names Key: KAFKA-4330 URL: https://issues.apache.org/jira/browse/KAFKA-4330 Project: Kafka Issue

?????? [DISCUSS] KIP-68 Add a consumed log retention before log retention

Hi Becket, It seems a good idea to have the topic to some specified groups to using the consumed log retention, default is all the consumed groups, it is more flexible. I have some comments for this : 1. What scenario is used to this configuration? 2. I think we can only support this

Jenkins build is back to normal : kafka-trunk-jdk7 #1647

See

?????? [DISCUSS] KIP-68 Add a consumed log retention before log retention

Hi Mayuresh, Thanks for the reply: 1. In the log retention check schedule, the broker first find the all the consumed group which are consuming this topic, and query the commit offset of this consumed group for the topic using the OffsetFetch API. And the min commit offset is the minimal

[jira] [Updated] (KAFKA-4311) Mutli layer cache eviction causes forwarding to incorrect Processor Node

[ https://issues.apache.org/jira/browse/KAFKA-4311?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Damian Guy updated KAFKA-4311: -- Summary: Mutli layer cache eviction causes forwarding to incorrect Processor Node (was: Exception in

[jira] [Updated] (KAFKA-4311) Multi layer cache eviction causes forwarding to incorrect Processor Node

[ https://issues.apache.org/jira/browse/KAFKA-4311?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Damian Guy updated KAFKA-4311: -- Summary: Multi layer cache eviction causes forwarding to incorrect Processor Node (was: Mutli layer

Re: [DISCUSS] KIP-80: Kafka REST Server

Hi Michael, Can you please share which Apache projects have a MMC? I couldn't find anything after a quick google. Ismael On Fri, Oct 21, 2016 at 7:28 AM, Michael Pearce wrote: > So from my reading essentially the first question needs to answered/and > voted on is: > >

[jira] [Updated] (KAFKA-4311) Multi layer cache eviction causes forwarding to incorrect Processor Node

[ https://issues.apache.org/jira/browse/KAFKA-4311?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Damian Guy updated KAFKA-4311: -- Description: The two exceptions below were reported by Frank on the dev mailing list. After

[jira] [Commented] (KAFKA-4309) Allow "pluggable" properties in KafkaService in System Tests

[ https://issues.apache.org/jira/browse/KAFKA-4309?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15594582#comment-15594582 ] ASF GitHub Bot commented on KAFKA-4309: --- Github user asfgit closed the pull request at:

[jira] [Resolved] (KAFKA-4309) Allow "pluggable" properties in KafkaService in System Tests

[ https://issues.apache.org/jira/browse/KAFKA-4309?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ismael Juma resolved KAFKA-4309. Resolution: Fixed Fix Version/s: 0.10.1.1 0.10.2.0 Issue resolved by pull

[jira] [Created] (KAFKA-4328) The parameters for creating the ZkUtils object is reverse

Matt Wang created KAFKA-4328: Summary: The parameters for creating the ZkUtils object is reverse Key: KAFKA-4328 URL: https://issues.apache.org/jira/browse/KAFKA-4328 Project: Kafka Issue Type:

Re: Kafka KIP meeting Oct 19 at 11:00am PST

I had noted that what ever the solution having compaction based on null payload was agreed isn't elegant. Shall we raise another kip to : as discussed propose using a attribute bit for delete/compaction flag as well/or instead of null value and updating compaction logic to look at that

Re: [DISCUSS] KIP-80: Kafka REST Server

Harsha Ch wrote on 20/10/2016 22:26:53: > The Streams and Connect projects that were pushed into Kafka could > have been left in their own Github projects based on your arguments. What > about the REST API is so different that such that it should stay out of the >

Re: [ANNOUNCE] Apache Kafka 0.10.1.0 Released

Great job ! Thanks Jason and to everyone who contributed. On Fri, Oct 21, 2016 at 6:09 AM, Manikumar wrote: > Thanks Jason. Great job. > > On Fri, Oct 21, 2016 at 10:29 AM, Becket Qin wrote: > >> Congratulations! Great job, Jason. >> >> On Thu,

[jira] [Created] (KAFKA-4329) The parameters for creating the ZkUtils object is reverse

Matt Wang created KAFKA-4329: Summary: The parameters for creating the ZkUtils object is reverse Key: KAFKA-4329 URL: https://issues.apache.org/jira/browse/KAFKA-4329 Project: Kafka Issue Type:

[jira] [Updated] (KAFKA-4329) The parameters for creating the ZkUtils object is reverse

[ https://issues.apache.org/jira/browse/KAFKA-4329?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matt Wang updated KAFKA-4329: - Description: When creating the ZkUtils object, the parameters of zkSessionTimeOutMs and

[jira] [Updated] (KAFKA-4329) The order of the parameters for creating the ZkUtils object is reversed

[ https://issues.apache.org/jira/browse/KAFKA-4329?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matt Wang updated KAFKA-4329: - Summary: The order of the parameters for creating the ZkUtils object is reversed (was: The parameters

[GitHub] kafka pull request #2034: KAFKA-4309: Allow "pluggable" properties in KafkaS...

Github user asfgit closed the pull request at: https://github.com/apache/kafka/pull/2034 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] kafka pull request #2050: Replaced unnecessary isDefined and get on option v...

GitHub user himani1 opened a pull request: https://github.com/apache/kafka/pull/2050 Replaced unnecessary isDefined and get on option values with fold You can merge this pull request into a Git repository by running: $ git pull https://github.com/himani1/kafka refactored_code

[jira] [Commented] (KAFKA-4311) Exception in NamedCache.flush - Key found in dirty key set, but entry is null

[ https://issues.apache.org/jira/browse/KAFKA-4311?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15594852#comment-15594852 ] Damian Guy commented on KAFKA-4311: --- Hi Frank, if you have the time, would you mind trying to run your