[GitHub] kafka pull request: MINOR: Fix broken docs link

Github user asfgit closed the pull request at: https://github.com/apache/kafka/pull/676 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[jira] [Resolved] (KAFKA-2990) NoSuchMethodError when Kafka is compiled with 1.8 and run on 1.7

[ https://issues.apache.org/jira/browse/KAFKA-2990?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ewen Cheslack-Postava resolved KAFKA-2990. -- Resolution: Fixed Fix Version/s: 0.9.1.0 0.9.0.1

[jira] [Commented] (KAFKA-2990) NoSuchMethodError when Kafka is compiled with 1.8 and run on 1.7

[ https://issues.apache.org/jira/browse/KAFKA-2990?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15056988#comment-15056988 ] ASF GitHub Bot commented on KAFKA-2990: --- GitHub user hachikuji opened a pull request:

Re: [DISCUSS] KIP-32 - Add CreateTime and LogAppendTime to Kafka message

1. I was thinking more about saving the decompression overhead in the follower. Currently, the follower doesn't decompress the messages. To keep it that way, the outer message needs to include the timestamp of the latest inner message to build the time index in the follower. The simplest thing to

Build failed in Jenkins: kafka-trunk-jdk7 #904

See Changes: [cshapi] MINOR: Fix broken docs link [me] KAFKA-2990; Fix NoSuchMethodError in Pool with cast to ConcurrentMap -- [...truncated 2824 lines...] kafka.log.LogTest >

Re: Setup for debugging Kafka in eclipse

Never mind. I was able to config my IDE so I can run/debug commands in there. Thanks. --Vahid From: Vahid S Hashemian/Silicon Valley/IBM@IBMUS To: dev@kafka.apache.org Date: 12/14/2015 01:12 PM Subject:Re: Setup for debugging Kafka in eclipse Thanks Gwen. Would you mind

[GitHub] kafka pull request: KAFKA-2837 Follow-up: Default max block to 60 ...

Github user asfgit closed the pull request at: https://github.com/apache/kafka/pull/674 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

Build failed in Jenkins: kafka-trunk-jdk8 #234

See Changes: [wangguoz] KAFKA-2978: consumer stops fetching when consumed and fetch positions -- [...truncated 1415 lines...] kafka.log.LogTest > testTruncateTo PASSED kafka.log.LogTest >

[jira] [Commented] (KAFKA-2985) Consumer group stuck in rebalancing state

[ https://issues.apache.org/jira/browse/KAFKA-2985?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15057055#comment-15057055 ] Ismael Juma commented on KAFKA-2985: Thanks for the report. Would you be able to test the 0.9.0

Jenkins build is back to normal : kafka_0.9.0_jdk7 #69

See

[jira] [Commented] (KAFKA-2987) error of producing messages after topic was marked for deleteion

[ https://issues.apache.org/jira/browse/KAFKA-2987?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15057219#comment-15057219 ] Glen Cao commented on KAFKA-2987: - Could anyone help? > error of producing messages after topic was

[GitHub] kafka pull request: Documentation updates for reserved.broker.max....

Github user asfgit closed the pull request at: https://github.com/apache/kafka/pull/670 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] kafka pull request: KAFKA-2990: fix NoSuchMethodError in Pool with...

GitHub user hachikuji opened a pull request: https://github.com/apache/kafka/pull/675 KAFKA-2990: fix NoSuchMethodError in Pool with cast to ConcurrentMap You can merge this pull request into a Git repository by running: $ git pull https://github.com/hachikuji/kafka

[jira] [Commented] (KAFKA-2854) Make KerberosName implement PrincipalToLocal plugin so authorizer and authenticator can share this.

[ https://issues.apache.org/jira/browse/KAFKA-2854?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15057015#comment-15057015 ] Ismael Juma commented on KAFKA-2854: I can take a look tomorrow Parth > Make KerberosName implement

[jira] [Commented] (KAFKA-2987) error of producing messages after topic was marked for deleteion

[ https://issues.apache.org/jira/browse/KAFKA-2987?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15057228#comment-15057228 ] Jiangjie Qin commented on KAFKA-2987: - Have you set delete.topic.enabled to true? > error of

[jira] [Commented] (KAFKA-1374) LogCleaner (compaction) does not support compressed topics

[ https://issues.apache.org/jira/browse/KAFKA-1374?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15057052#comment-15057052 ] Manikumar Reddy commented on KAFKA-1374: [~granthenke] Yes, I will update the docs. > LogCleaner

[jira] [Commented] (KAFKA-2987) error of producing messages after topic was marked for deleteion

[ https://issues.apache.org/jira/browse/KAFKA-2987?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15057242#comment-15057242 ] Jiangjie Qin commented on KAFKA-2987: - If I remember correctly, delete topic in 0.8.2.1 is not that

[jira] [Commented] (KAFKA-2990) NoSuchMethodError when Kafka is compiled with 1.8 and run on 1.7

[ https://issues.apache.org/jira/browse/KAFKA-2990?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15057007#comment-15057007 ] ASF GitHub Bot commented on KAFKA-2990: --- Github user asfgit closed the pull request at:

[GitHub] kafka pull request: KAFKA-2990: fix NoSuchMethodError in Pool with...

Github user asfgit closed the pull request at: https://github.com/apache/kafka/pull/675 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[jira] [Commented] (KAFKA-2985) Consumer group stuck in rebalancing state

[ https://issues.apache.org/jira/browse/KAFKA-2985?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15057081#comment-15057081 ] Jason Gustafson commented on KAFKA-2985: [~ztyx] Could you attach the source for your consumer?

Build failed in Jenkins: kafka_0.9.0_jdk7 #70

See Changes: [me] KAFKA-2990; Fix NoSuchMethodError in Pool with cast to ConcurrentMap -- [...truncated 3797 lines...] org.apache.kafka.common.record.RecordTest > testChecksum[46] PASSED

[jira] [Commented] (KAFKA-2143) Replicas get ahead of leader and fail

[ https://issues.apache.org/jira/browse/KAFKA-2143?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15056999#comment-15056999 ] Jiangjie Qin commented on KAFKA-2143: - [~ottomata], KAFKA-2477 is a more likely cause. You may want to

[jira] [Commented] (KAFKA-2837) FAILING TEST: kafka.api.ProducerBounceTest > testBrokerFailure

[ https://issues.apache.org/jira/browse/KAFKA-2837?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15057164#comment-15057164 ] ASF GitHub Bot commented on KAFKA-2837: --- Github user asfgit closed the pull request at:

[GitHub] kafka pull request: modify config specification of topic level

GitHub user EamonZhang reopened a pull request: https://github.com/apache/kafka/pull/612 modify config specification of topic level topic level config delete config options use --delete-config instead of --deleteConfig You can merge this pull request into a Git repository

[jira] [Commented] (KAFKA-2987) error of producing messages after topic was marked for deleteion

[ https://issues.apache.org/jira/browse/KAFKA-2987?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15057230#comment-15057230 ] Glen Cao commented on KAFKA-2987: - Yes > error of producing messages after topic was marked for deleteion

[jira] [Issue Comment Deleted] (KAFKA-2987) error of producing messages after topic was marked for deleteion

[ https://issues.apache.org/jira/browse/KAFKA-2987?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Glen Cao updated KAFKA-2987: Comment: was deleted (was: Yes) > error of producing messages after topic was marked for deleteion >

[jira] [Commented] (KAFKA-2987) error of producing messages after topic was marked for deleteion

[ https://issues.apache.org/jira/browse/KAFKA-2987?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15057231#comment-15057231 ] Glen Cao commented on KAFKA-2987: - Yes > error of producing messages after topic was marked for deleteion

[jira] [Created] (KAFKA-2987) error of producing messages after topic was marked for deleteion

Glen Cao created KAFKA-2987: --- Summary: error of producing messages after topic was marked for deleteion Key: KAFKA-2987 URL: https://issues.apache.org/jira/browse/KAFKA-2987 Project: Kafka Issue

Jenkins build is back to normal : kafka-trunk-jdk8 #236

See

Build failed in Jenkins: kafka-trunk-jdk7 #905

See Changes: [wangguoz] KAFKA-2837 Follow-up: Default max block to 60 seconds -- [...truncated 1422 lines...] kafka.log.LogTest > testTruncateTo PASSED kafka.log.LogTest >

[jira] [Commented] (KAFKA-2987) error of producing messages after topic was marked for deleteion

[ https://issues.apache.org/jira/browse/KAFKA-2987?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15057279#comment-15057279 ] Glen Cao commented on KAFKA-2987: - Could you tell the specific steps in more details? Step1, remove ZK

[jira] [Resolved] (KAFKA-2987) error of producing messages after topic was marked for deleteion

[ https://issues.apache.org/jira/browse/KAFKA-2987?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Grant Henke resolved KAFKA-2987. Resolution: Not A Problem > error of producing messages after topic was marked for deleteion >

[jira] [Resolved] (KAFKA-2975) The newtorkClient should request a metadata update after it gets an error in the handleResponse()

[ https://issues.apache.org/jira/browse/KAFKA-2975?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mayuresh Gharat resolved KAFKA-2975. Resolution: Not A Problem > The newtorkClient should request a metadata update after it

[jira] [Commented] (KAFKA-2987) error of producing messages after topic was marked for deleteion

[ https://issues.apache.org/jira/browse/KAFKA-2987?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15057344#comment-15057344 ] Glen Cao commented on KAFKA-2987: - Got it. Thank you very much. > error of producing messages after topic

Build failed in Jenkins: kafka-trunk-jdk7 #906

See Changes: [me] MINOR: Documentation updates for reserved.broker.max.id -- [...truncated 4712 lines...] org.apache.kafka.connect.runtime.rest.resources.ConnectorsResourceTest >

Build failed in Jenkins: kafka-trunk-jdk8 #237

See Changes: [me] MINOR: Documentation updates for reserved.broker.max.id -- [...truncated 3363 lines...] org.apache.kafka.clients.NetworkClientTest > testClose PASSED

[jira] [Commented] (KAFKA-2987) error of producing messages after topic was marked for deleteion

[ https://issues.apache.org/jira/browse/KAFKA-2987?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15057305#comment-15057305 ] Glen Cao commented on KAFKA-2987: - Could anyone please tell the detailed steps on deleting a topic

[jira] [Commented] (KAFKA-2987) error of producing messages after topic was marked for deleteion

[ https://issues.apache.org/jira/browse/KAFKA-2987?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15057341#comment-15057341 ] Grant Henke commented on KAFKA-2987: Hi [~microwish], I would suggest posting questions like this to

Re: [DISCUSS] KIP-32 - Add CreateTime and LogAppendTime to Kafka message

1. Hmm, it's more intuitive if the consumer sees the same timestamp whether the messages are compressed or not. When message.timestamp.type=LogAppendTime, we will need to set timestamp in each message if messages are not compressed, so that the follower can get the same timestamp. So, it seems

retrieve commit time for messages

Hi Is there any way to get the commit timestamp of the messages which are retrieved using kafka consumer API. t SunilKalva

Re: [DISCUSS] KIP-36 - Rack aware replica assignment

Allen, Thanks for the proposal. A few comments. 1. Since this KIP changes the inter broker communication protocol (UpdateMetadataRequest), we will need to document the upgrade path (similar to what's described in http://kafka.apache.org/090/documentation.html#upgrade). 2. It might be useful to

[jira] [Commented] (KAFKA-2920) Storage module

[ https://issues.apache.org/jira/browse/KAFKA-2920?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15056235#comment-15056235 ] Leonid Kyrpychenko commented on KAFKA-2920: --- [~abiletskyi], could you please elaborate a bit

Consumer Offsets Compaction

Since 0.9.0 the internal "__consumer_offsets" topic is being used more heavily. Because this is a compacted topic does "log.cleaner.enable" need to be "true" in order for it to be compacted? Or is there special handling for internal topics? If log.cleaner.enable=true is required, should we make

[jira] [Commented] (KAFKA-2920) Storage module

[ https://issues.apache.org/jira/browse/KAFKA-2920?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15056278#comment-15056278 ] Andrii Biletskyi commented on KAFKA-2920: - The idea is to implement plug-gable interface of the

[jira] [Created] (KAFKA-2989) Verify all partitions consumed after rebalancing in system tests

Jason Gustafson created KAFKA-2989: -- Summary: Verify all partitions consumed after rebalancing in system tests Key: KAFKA-2989 URL: https://issues.apache.org/jira/browse/KAFKA-2989 Project: Kafka

[GitHub] kafka pull request: MINOR: Fix typos in code comments

GitHub user vahidhashemian opened a pull request: https://github.com/apache/kafka/pull/673 MINOR: Fix typos in code comments You can merge this pull request into a Git repository by running: $ git pull https://github.com/vahidhashemian/kafka typo02/fix_typos_in_code_comments

Re: Consumer Offsets Compaction

That's a good point. It doesn't look like there's any special handling for the offsets topic, so enabling the cleaner by default makes sense to me. If compaction is not enabled, it would grow without bound, so I wonder if we should even deprecate that setting. Are there any use cases where it

[jira] [Created] (KAFKA-2988) Change default value of log.cleaner.enable

Grant Henke created KAFKA-2988: -- Summary: Change default value of log.cleaner.enable Key: KAFKA-2988 URL: https://issues.apache.org/jira/browse/KAFKA-2988 Project: Kafka Issue Type:

Re: Consumer Offsets Compaction

The reason for disabling it by default was (1) general paranoia about log compaction when we released it, (2) avoid allocating the compaction buffer. The first concern is now definitely obsolete, but the second concern is maybe valid. Basically that compaction buffer is a preallocated chunk of

Re: Consumer Offsets Compaction

Thanks for the responses and confirmation. I have created https://issues.apache.org/jira/browse/KAFKA-2988 to track the work/changes. We can continue discussion here on what changes to make. On Mon, Dec 14, 2015 at 1:51 PM, Jason Gustafson wrote: > That's a good point. It

Re: retrieve commit time for messages

At the moment, there is no direct way to do this, but you could use the commit API to include metadata with each committed offset: public void commitSync(final Map offsets); public OffsetAndMetadata committed(TopicPartition partition); The OffsetAndMetadata

[GitHub] kafka pull request: KAFKA-2984: ktable sends old values when requi...

GitHub user ymatsuda opened a pull request: https://github.com/apache/kafka/pull/672 KAFKA-2984: ktable sends old values when required @guozhangwang At DAG level, `KTable` sends (key, (new value, old value)) to down stream. This is done by wrapping the new value and

[jira] [Commented] (KAFKA-2980) ZookeeperConsumerConnector may enter deadlock if a rebalance occurs during a stream creation.

[ https://issues.apache.org/jira/browse/KAFKA-2980?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15056529#comment-15056529 ] Jiangjie Qin commented on KAFKA-2980: - [~guozhang] Can you take a looks that the patch when got a

[GitHub] kafka pull request: MINOR: Fix typos in code comments

Github user vahidhashemian closed the pull request at: https://github.com/apache/kafka/pull/662 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature

Re: Consumer Offsets Compaction

This makes sense to me. Copycat also works better if topics are compacted. Just to clarify: log.cleaner.enable = true just makes the compaction thread run, but doesn't force compaction on any specific topic. You still need to set delete.policy=compact, and we should not change defaults here. On

[jira] [Commented] (KAFKA-2984) KTable should send old values along with new values to downstreams

[ https://issues.apache.org/jira/browse/KAFKA-2984?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15056429#comment-15056429 ] ASF GitHub Bot commented on KAFKA-2984: --- GitHub user ymatsuda opened a pull request:

Re: [DISCUSS] KIP-32 - Add CreateTime and LogAppendTime to Kafka message

Hi, Becket, Thanks for the proposal. Looks good overall. A few comments below. 1. KIP-32 didn't say what timestamp should be set in a compressed message. We probably should set it to the timestamp of the latest messages included in the compressed one. This way, during indexing, we don't have to

[jira] [Updated] (KAFKA-2978) Topic partition is not sometimes consumed after rebalancing of consumer group

[ https://issues.apache.org/jira/browse/KAFKA-2978?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ismael Juma updated KAFKA-2978: --- Reviewer: Guozhang Wang Status: Patch Available (was: Open) > Topic partition is not sometimes

Re: Setup for debugging Kafka in eclipse

Thanks Gwen. Would you mind sharing how you configure your "Run Configuration" in Intellij to run/debug the code? I also looked at this page but couldn't figure out the missing details: http://markmail.org/message/6fgdqfu6k7upswoz Thanks. --Vahid

[jira] [Commented] (KAFKA-2143) Replicas get ahead of leader and fail

[ https://issues.apache.org/jira/browse/KAFKA-2143?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15056672#comment-15056672 ] Andrew Otto commented on KAFKA-2143: I'd like to add that Wikimedia is experiencing this as well. It

[jira] [Commented] (KAFKA-2978) Topic partition is not sometimes consumed after rebalancing of consumer group

[ https://issues.apache.org/jira/browse/KAFKA-2978?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15056886#comment-15056886 ] ASF GitHub Bot commented on KAFKA-2978: --- Github user asfgit closed the pull request at:

[GitHub] kafka pull request: KAFKA-2978: consumer stops fetching when consu...

Github user asfgit closed the pull request at: https://github.com/apache/kafka/pull/666 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[jira] [Updated] (KAFKA-2978) Topic partition is not sometimes consumed after rebalancing of consumer group

[ https://issues.apache.org/jira/browse/KAFKA-2978?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guozhang Wang updated KAFKA-2978: - Resolution: Fixed Status: Resolved (was: Patch Available) Issue resolved by pull request

Re: [DISCUSS] KIP-32 - Add CreateTime and LogAppendTime to Kafka message

Hi Jun, Thanks a lot for the comments. Please see inline replies. Thanks, Jiangjie (Becket) Qin On Mon, Dec 14, 2015 at 10:19 AM, Jun Rao wrote: > Hi, Becket, > > Thanks for the proposal. Looks good overall. A few comments below. > > 1. KIP-32 didn't say what timestamp

[jira] [Commented] (KAFKA-2794) Add group support for authorizer acls

[ https://issues.apache.org/jira/browse/KAFKA-2794?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15056934#comment-15056934 ] Parth Brahmbhatt commented on KAFKA-2794: - https://github.com/apache/kafka/pull/483 > Add group

[GitHub] kafka pull request: KAFKA-2837 Follow-up: Default max block to 60 ...

GitHub user guozhangwang opened a pull request: https://github.com/apache/kafka/pull/674 KAFKA-2837 Follow-up: Default max block to 60 seconds You can merge this pull request into a Git repository by running: $ git pull https://github.com/guozhangwang/kafka K2837

[jira] [Commented] (KAFKA-2854) Make KerberosName implement PrincipalToLocal plugin so authorizer and authenticator can share this.

[ https://issues.apache.org/jira/browse/KAFKA-2854?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15056930#comment-15056930 ] Parth Brahmbhatt commented on KAFKA-2854: - [~ijuma][~harsha_ch] Can you review

[jira] [Work started] (KAFKA-2854) Make KerberosName implement PrincipalToLocal plugin so authorizer and authenticator can share this.

[ https://issues.apache.org/jira/browse/KAFKA-2854?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on KAFKA-2854 started by Parth Brahmbhatt. --- > Make KerberosName implement PrincipalToLocal plugin so authorizer and >

[jira] [Work started] (KAFKA-2794) Add group support for authorizer acls

[ https://issues.apache.org/jira/browse/KAFKA-2794?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on KAFKA-2794 started by Parth Brahmbhatt. --- > Add group support for authorizer acls >

[jira] [Commented] (KAFKA-2837) FAILING TEST: kafka.api.ProducerBounceTest > testBrokerFailure

[ https://issues.apache.org/jira/browse/KAFKA-2837?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15056936#comment-15056936 ] ASF GitHub Bot commented on KAFKA-2837: --- GitHub user guozhangwang opened a pull request:

[jira] [Created] (KAFKA-2990) NoSuchMethodError when Kafka is compiled with 1.8 and run on 1.7

Jason Gustafson created KAFKA-2990: -- Summary: NoSuchMethodError when Kafka is compiled with 1.8 and run on 1.7 Key: KAFKA-2990 URL: https://issues.apache.org/jira/browse/KAFKA-2990 Project: Kafka