[jira] [Comment Edited] (KAFKA-3772) MirrorMaker crashes on Corrupted Message

[ https://issues.apache.org/jira/browse/KAFKA-3772?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15616151#comment-15616151 ] TUSHAR SHARAD MHASKAR edited comment on KAFKA-3772 at 10/28/16 6:20 PM:

[jira] [Commented] (KAFKA-3772) MirrorMaker crashes on Corrupted Message

[ https://issues.apache.org/jira/browse/KAFKA-3772?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15616151#comment-15616151 ] TUSHAR SHARAD MHASKAR commented on KAFKA-3772: -- I have seen this same issue with 0.9.0.0

[jira] [Created] (KAFKA-4356) o.a.k.common.utils.SystemTime.sleep() swallows InterruptedException

Shikhar Bhushan created KAFKA-4356: -- Summary: o.a.k.common.utils.SystemTime.sleep() swallows InterruptedException Key: KAFKA-4356 URL: https://issues.apache.org/jira/browse/KAFKA-4356 Project: Kafka

Kafka client circuit breaker

Hi Kafka developers :) We use Kafka for around half a year as a main backend system for storing events from big variety of internal microservices. In general It works quite well and we experience issues only in some rare 'corner cases' where we either run against a bug or do not understand why

[jira] [Commented] (KAFKA-4355) StreamThread intermittently dies with "Topic not found during partition assignment" when broker restarted

[ https://issues.apache.org/jira/browse/KAFKA-4355?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15616118#comment-15616118 ] Michal Borowiecki commented on KAFKA-4355: -- Perhaps the DefultPartitionGrouper here:

[jira] [Commented] (KAFKA-4355) StreamThread intermittently dies with "Topic not found during partition assignment" when broker restarted

[ https://issues.apache.org/jira/browse/KAFKA-4355?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15616185#comment-15616185 ] Guozhang Wang commented on KAFKA-4355: -- [~mihbor] Thanks for reporting this. A first note is that in

Jenkins build is back to normal : kafka-trunk-jdk8 #1009

See

Re: [DISCUSS] KIP-68 Add a consumed log retention before log retention

I do agree with Guozhang on having applications request an external service(admin) that talks to kafka, for trimming, in which case this external service(admin) can check if its OK to send the trim request to kafka brokers based on a certain conditions. On broker side we can have authorization by

[jira] [Updated] (KAFKA-4355) StreamThread intermittently dies with "Topic not found during partition assignment" when broker restarted

[ https://issues.apache.org/jira/browse/KAFKA-4355?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michal Borowiecki updated KAFKA-4355: - Affects Version/s: 0.10.1.0 0.10.0.0 > StreamThread intermittently

[jira] [Updated] (KAFKA-4355) StreamThread intermittently dies with "Topic not found during partition assignment" when broker restarted

[ https://issues.apache.org/jira/browse/KAFKA-4355?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michal Borowiecki updated KAFKA-4355: - Description: When (a) starting kafka streams app before the broker or (b) restarting the

[jira] [Updated] (KAFKA-4355) StreamThread intermittently dies with "Topic not found during partition assignment" when broker restarted

[ https://issues.apache.org/jira/browse/KAFKA-4355?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michal Borowiecki updated KAFKA-4355: - Environment: kafka 0.10.0.0 kafka 0.10.1.0 uname -a Linux lp02485 4.4.0-34-generic

[jira] [Created] (KAFKA-4355) StreamThread intermittently dies with "Topic not found during partition assignment" when broker restarted

Michal Borowiecki created KAFKA-4355: Summary: StreamThread intermittently dies with "Topic not found during partition assignment" when broker restarted Key: KAFKA-4355 URL:

Re: [DISCUSS] KIP-68 Add a consumed log retention before log retention

Yes trim() should be an admin API and, if security is concerned, it should be under admin authorization as well. For applications that needs this feature, it then boils down to the problem that they should request the authorization token from who operates Kafka before starting their app to use in

Re: [DISCUSS] KIP-68 Add a consumed log retention before log retention

Hi Guozhang, I agree that pushing out the complexity of coordination to the client application makes it more simple for the broker in the sense that it does not have to be the decision maker regarding when to trim and till what offset. An I agree that if we go in this direction, providing an

[jira] [Commented] (KAFKA-4352) Failure in org.apache.kafka.streams.integration.ResetIntegrationTest.testReprocessingFromScratchAfterReset

[ https://issues.apache.org/jira/browse/KAFKA-4352?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15616052#comment-15616052 ] Matthias J. Sax commented on KAFKA-4352: Thanks for the update [~vahid]. > Failure in >

Re: [DISCUSS] KIP-87 - Add Compaction Tombstone Flag

> > I kind of agree with James that it is a bit questionable how valuable any > data in a delete marker can be since it will be deleted somewhat > nondeterministically. > One could argue that even in normal topics, assuming a time-based log retention policy is in place, any message will be

[VOTE] KIP-47 - Add timestamp-based log deletion policy

> > - It seems that the consumer will need to write log.retention.min.timestamp > periodically to zookeeper as dynamic configuration of the topic, so that > broker can pick up log.retention.min.timestamp. However, this introduces > dependency of consumer on zookeeper which is undesirable. Note

[jira] [Commented] (KAFKA-4355) StreamThread intermittently dies with "Topic not found during partition assignment" when broker restarted

[ https://issues.apache.org/jira/browse/KAFKA-4355?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15615967#comment-15615967 ] Michal Borowiecki commented on KAFKA-4355: -- My first suspect so far is the ConsumerCoordinator.

[jira] [Created] (KAFKA-4358) Following a hung broker, newly elected leader is unnecessarily slow assuming leadership because of ReplicaFetcherThread

Nelson Elhage created KAFKA-4358: Summary: Following a hung broker, newly elected leader is unnecessarily slow assuming leadership because of ReplicaFetcherThread Key: KAFKA-4358 URL:

[jira] [Updated] (KAFKA-4357) Consumer group describe exception when there is no active member (old consumer)

[ https://issues.apache.org/jira/browse/KAFKA-4357?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vahid Hashemian updated KAFKA-4357: --- Status: Patch Available (was: In Progress) > Consumer group describe exception when there is

[jira] [Work started] (KAFKA-4357) Consumer group describe exception when there is no active member (old consumer)

[ https://issues.apache.org/jira/browse/KAFKA-4357?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on KAFKA-4357 started by Vahid Hashemian. -- > Consumer group describe exception when there is no active member (old >

[jira] [Commented] (KAFKA-3853) Report offsets for empty groups in ConsumerGroupCommand

[ https://issues.apache.org/jira/browse/KAFKA-3853?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15616482#comment-15616482 ] ASF GitHub Bot commented on KAFKA-3853: --- GitHub user vahidhashemian opened a pull request:

[GitHub] kafka pull request #2058: kafka-4295: ConsoleConsumer does not delete the te...

Github user amethystic closed the pull request at: https://github.com/apache/kafka/pull/2058 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[jira] [Commented] (KAFKA-4348) On Mac OS, KafkaConsumer.poll returns 0 when there are still messages on Kafka server

[ https://issues.apache.org/jira/browse/KAFKA-4348?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15614582#comment-15614582 ] Yiquan Zhou commented on KAFKA-4348: The issue occurs when there are large numbers of records on the

[jira] [Reopened] (KAFKA-4351) Topic regex behavioral change with MirrorMaker new consumer

[ https://issues.apache.org/jira/browse/KAFKA-4351?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dustin Cote reopened KAFKA-4351: Looks like this got marked resolved accidentally when the PR was made. Reopening but cc'ing

[jira] [Commented] (KAFKA-4351) Topic regex behavioral change with MirrorMaker new consumer

[ https://issues.apache.org/jira/browse/KAFKA-4351?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15614635#comment-15614635 ] huxi commented on KAFKA-4351: - thanks for helping reopen the JIRA. yes, I incorrectly set the wrong status. >

[GitHub] kafka pull request #2072: Kafka-4351: Topic regex behavioral change with Mir...

GitHub user amethystic opened a pull request: https://github.com/apache/kafka/pull/2072 Kafka-4351: Topic regex behavioral change with MirrorMaker new consumer Support CSV values in the regexp for MirrorMaker new consumer as OldConsumer does. You can merge this pull request into a

[jira] [Resolved] (KAFKA-4351) Topic regex behavioral change with MirrorMaker new consumer

[ https://issues.apache.org/jira/browse/KAFKA-4351?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] huxi resolved KAFKA-4351. - Resolution: Resolved > Topic regex behavioral change with MirrorMaker new consumer >

Build failed in Jenkins: kafka-trunk-jdk7 #1659

See Changes: [jason] KAFKA-4349; Handle 'PreparingRebalance' and 'AwaitingSync' states in -- [...truncated 14345 lines...]

Re: [DISCUSS] KIP-87 - Add Compaction Tombstone Flag

Michael, What information would you want to include on a delete tombstone, and what would you use it for? -James Sent from my iPhone > On Oct 27, 2016, at 9:17 PM, Michael Pearce wrote: > > Hi Jay, > > I think use case that is the issue that Konstantin mentioned in

[GitHub] kafka pull request #2073: MINOR: Fix issue in `AsyncProducerTest` where it e...

GitHub user ijuma opened a pull request: https://github.com/apache/kafka/pull/2073 MINOR: Fix issue in `AsyncProducerTest` where it expects the `port` config to be set This test fails locally when I run it, but somehow Jenkins builds are passed. Not clear how. You can merge this

Re: [DISCUSS] KIP-68 Add a consumed log retention before log retention

Hey Guozhang, I think the trim() interface is generally useful. What I was wondering is the following: if the user has multiple applications to coordinate, it seems simpler for the broker to coordinate instead of asking the applications to coordinate among themselves. If we let the broker do the

[jira] [Commented] (KAFKA-4326) Refactor LogCleaner to remove duplicate log copying logic

[ https://issues.apache.org/jira/browse/KAFKA-4326?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15615052#comment-15615052 ] ASF GitHub Bot commented on KAFKA-4326: --- Github user asfgit closed the pull request at:

[jira] [Resolved] (KAFKA-4326) Refactor LogCleaner to remove duplicate log copying logic

[ https://issues.apache.org/jira/browse/KAFKA-4326?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ismael Juma resolved KAFKA-4326. Resolution: Fixed Fix Version/s: 0.10.2.0 Issue resolved by pull request 2053

[GitHub] kafka pull request #2053: KAFKA-4326: Refactor LogCleaner for better reuse o...

Github user asfgit closed the pull request at: https://github.com/apache/kafka/pull/2053 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] kafka pull request #2074: KAFKA-3853: Report offsets for empty groups in con...

GitHub user vahidhashemian opened a pull request: https://github.com/apache/kafka/pull/2074 KAFKA-3853: Report offsets for empty groups in consumer group describe command (new consumer) When there are no active new consumers in a consumer group report the offsets within the group

[jira] [Created] (KAFKA-4357) Consumer group describe exception when there is no active member (old consumer)

Vahid Hashemian created KAFKA-4357: -- Summary: Consumer group describe exception when there is no active member (old consumer) Key: KAFKA-4357 URL: https://issues.apache.org/jira/browse/KAFKA-4357

[jira] [Commented] (KAFKA-4357) Consumer group describe exception when there is no active member (old consumer)

[ https://issues.apache.org/jira/browse/KAFKA-4357?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15616669#comment-15616669 ] ASF GitHub Bot commented on KAFKA-4357: --- GitHub user vahidhashemian opened a pull request:

[GitHub] kafka pull request #2075: KAFKA-4357: Fix consumer group describe output whe...

GitHub user vahidhashemian opened a pull request: https://github.com/apache/kafka/pull/2075 KAFKA-4357: Fix consumer group describe output when there is no active member (old consumer) You can merge this pull request into a Git repository by running: $ git pull

[GitHub] kafka pull request #2076: HOTFIX: improve error message on invalid input rec...

GitHub user mjsax opened a pull request: https://github.com/apache/kafka/pull/2076 HOTFIX: improve error message on invalid input record timestamp You can merge this pull request into a Git repository by running: $ git pull https://github.com/mjsax/kafka hotfixTSExtractor

[jira] [Commented] (KAFKA-3893) Kafka Borker ID disappears from /borkers/ids

[ https://issues.apache.org/jira/browse/KAFKA-3893?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15616914#comment-15616914 ] Rekha Joshi commented on KAFKA-3893: We saw it too in Kafka 0.8.x and upgraded to Kafka 0.9.x and

[GitHub] kafka pull request #2032: KAFKA-3559: Recycle old tasks when possible

GitHub user enothereska reopened a pull request: https://github.com/apache/kafka/pull/2032 KAFKA-3559: Recycle old tasks when possible You can merge this pull request into a Git repository by running: $ git pull https://github.com/enothereska/kafka

[jira] [Updated] (KAFKA-4358) Following a hung broker, newly elected leader is unnecessarily slow assuming leadership because of ReplicaFetcherThread

[ https://issues.apache.org/jira/browse/KAFKA-4358?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nelson Elhage updated KAFKA-4358: - Status: Patch Available (was: Open) This patch resolves the hang, but I'm not sure whether or

[jira] [Issue Comment Deleted] (KAFKA-4358) Following a hung broker, newly elected leader is unnecessarily slow assuming leadership because of ReplicaFetcherThread

[ https://issues.apache.org/jira/browse/KAFKA-4358?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nelson Elhage updated KAFKA-4358: - Comment: was deleted (was: This patch resolves the hang, but I'm not sure whether or not it's

[jira] [Updated] (KAFKA-4358) Following a hung broker, newly elected leader is unnecessarily slow assuming leadership because of ReplicaFetcherThread

[ https://issues.apache.org/jira/browse/KAFKA-4358?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nelson Elhage updated KAFKA-4358: - Status: Open (was: Patch Available) > Following a hung broker, newly elected leader is

[jira] [Updated] (KAFKA-4358) Following a hung broker, newly elected leader is unnecessarily slow assuming leadership because of ReplicaFetcherThread

[ https://issues.apache.org/jira/browse/KAFKA-4358?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nelson Elhage updated KAFKA-4358: - Attachment: shutdown.patch This patch resolves the hang, but I'm not sure whether or not it's

[jira] [Commented] (KAFKA-3559) Task creation time taking too long in rebalance callback

[ https://issues.apache.org/jira/browse/KAFKA-3559?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15616983#comment-15616983 ] ASF GitHub Bot commented on KAFKA-3559: --- GitHub user enothereska reopened a pull request:

[jira] [Commented] (KAFKA-3559) Task creation time taking too long in rebalance callback

[ https://issues.apache.org/jira/browse/KAFKA-3559?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15616982#comment-15616982 ] ASF GitHub Bot commented on KAFKA-3559: --- Github user enothereska closed the pull request at:

[GitHub] kafka pull request #2032: KAFKA-3559: Recycle old tasks when possible

Github user enothereska closed the pull request at: https://github.com/apache/kafka/pull/2032 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature

Kafka cannot shutdown

Hi all, We have a kafka cluster with 11 nodes, and we found there are some partition’s replica num is not equal to isr’s num，because our data traffic is small，we think it should isr’s num should equal to replica’s num at last， but it can not recovery to normal，so we try to shutdown a

Re: [DISCUSS] KIP-68 Add a consumed log retention before log retention

Here are my thoughts: If there are indeed multiple consumer groups on the same topic that needs to coordinate, it is equally complex if the coordination is on the broker or among the applications themselves: for the latter case, you would imagine some coordination services used (like ZK) to

Build failed in Jenkins: kafka-trunk-jdk7 #1660

See Changes: [ismael] KAFKA-4326; Refactor LogCleaner for better reuse of common copy/compress -- [...truncated 3907 lines...] kafka.api.AuthorizerIntegrationTest >

[jira] [Updated] (KAFKA-4351) Topic regex behavioral change with MirrorMaker new consumer

[ https://issues.apache.org/jira/browse/KAFKA-4351?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ismael Juma updated KAFKA-4351: --- Status: Patch Available (was: Reopened) > Topic regex behavioral change with MirrorMaker new

[jira] [Updated] (KAFKA-4351) Topic regex behavioral change with MirrorMaker new consumer

[ https://issues.apache.org/jira/browse/KAFKA-4351?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ismael Juma updated KAFKA-4351: --- Fix Version/s: 0.10.2.0 > Topic regex behavioral change with MirrorMaker new consumer >

[jira] [Created] (KAFKA-4354) System test failure: test_rolling_upgrade_sasl_mechanism_phase_two.new_sasl_mechanism=PLAIN

Ismael Juma created KAFKA-4354: -- Summary: System test failure: test_rolling_upgrade_sasl_mechanism_phase_two.new_sasl_mechanism=PLAIN Key: KAFKA-4354 URL: https://issues.apache.org/jira/browse/KAFKA-4354