[jira] [Assigned] (KAFKA-3160) Kafka LZ4 framing code miscalculates header checksum

2016-01-28 Thread Magnus Edenhill (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-3160?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Magnus Edenhill reassigned KAFKA-3160: -- Assignee: Magnus Edenhill > Kafka LZ4 framing code miscalculates header checksum >

[GitHub] kafka pull request: KAFKA-3068: Keep track of bootstrap nodes inst...

2016-01-28 Thread enothereska
Github user enothereska closed the pull request at: https://github.com/apache/kafka/pull/804 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

Re: JIRA access

2016-01-28 Thread Guozhang Wang
Done. Guozhang On Thu, Jan 28, 2016 at 4:16 AM, Magnus Edenhill wrote: > Hey, > > could I have access to assign myself to issues in the Apache JIRA? > Username is edenhill. > Looking to provide some patches for KAFKA-3160. > > Regards, > Magnus > -- -- Guozhang

[jira] [Updated] (KAFKA-3064) Improve resync method to waste less time and data transfer

2016-01-28 Thread Michael Graff (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-3064?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Graff updated KAFKA-3064: - Affects Version/s: 0.8.2.1 0.9.0.0 > Improve resync method to waste less

[jira] [Reopened] (KAFKA-3155) Transient Failure in kafka.integration.PlaintextTopicMetadataTest.testIsrAfterBrokerShutDownAndJoinsBack

2016-01-28 Thread Ismael Juma (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-3155?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ismael Juma reopened KAFKA-3155: > Transient Failure in >

[jira] [Commented] (KAFKA-3155) Transient Failure in kafka.integration.PlaintextTopicMetadataTest.testIsrAfterBrokerShutDownAndJoinsBack

2016-01-28 Thread Ismael Juma (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-3155?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15121877#comment-15121877 ] Ismael Juma commented on KAFKA-3155: Hmm, that's true. It was stated in the PR for KAFKA-3103 that the

Re: [DISCUSS] KIP-42: Add Producer and Consumer Interceptors

2016-01-28 Thread Becket Qin
I am +1 on #1.2 and #3. #2: Regarding CRC, I am not sure if users care about CRC. is there any specific use case? Currently we validate messages by calling ensureValid() to verify the checksum and throw exception if it does not match. Message size would be useful. We can add that to

[jira] [Commented] (KAFKA-3068) NetworkClient may connect to a different Kafka cluster than originally configured

2016-01-28 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-3068?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15121671#comment-15121671 ] ASF GitHub Bot commented on KAFKA-3068: --- Github user enothereska closed the pull request at:

[jira] [Commented] (KAFKA-1464) Add a throttling option to the Kafka replication tool

2016-01-28 Thread Jay Kreps (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1464?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15121706#comment-15121706 ] Jay Kreps commented on KAFKA-1464: -- Another issue this raises is that a partition might have a natural

[jira] [Commented] (KAFKA-3064) Improve resync method to waste less time and data transfer

2016-01-28 Thread Michael Graff (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-3064?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15121886#comment-15121886 ] Michael Graff commented on KAFKA-3064: -- Any thought on this issue? If all we had to use were two

Apache Kafka Contributor list

2016-01-28 Thread Denise Fernandez
Hi, I have just made my first Kafka PR and would like to be added to the contributor list so that I can assign myself to Jira issues and make changes to documentation. How do I go about getting this access? Thanks, Denise

Kafka Consumer stops receiving messages.

2016-01-28 Thread Nikhil Bansal
Hi All, My kafka consumer suddenly stops receiving messages and lags keep on increasing. Stack trace shows me the following for all the consumer threads. *"pool-43-thread-20" prio=10 tid=0x7f9a1d9d5000 nid=0x17888 waiting on condition [0x7f95d8e0d000]* * java.lang.Thread.State:

[jira] [Created] (KAFKA-3164) Document client and mirrormaker upgrade procedure/requirements

2016-01-28 Thread Grant Henke (JIRA)
Grant Henke created KAFKA-3164: -- Summary: Document client and mirrormaker upgrade procedure/requirements Key: KAFKA-3164 URL: https://issues.apache.org/jira/browse/KAFKA-3164 Project: Kafka

[GitHub] kafka pull request: KAFKA-3068: Remove retry with nodesEverSeen

2016-01-28 Thread enothereska
GitHub user enothereska opened a pull request: https://github.com/apache/kafka/pull/823 KAFKA-3068: Remove retry with nodesEverSeen @ewencp @ijuma if this looks good please merge when you can. Thanks. You can merge this pull request into a Git repository by running: $ git pull

[jira] [Commented] (KAFKA-3161) Refactor Java client's use of the Properties class

2016-01-28 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-3161?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15121798#comment-15121798 ] ASF GitHub Bot commented on KAFKA-3161: --- GitHub user crhyne opened a pull request:

Re: Apache Kafka Contributor list

2016-01-28 Thread Guozhang Wang
Hi Denise, What is your apache id? Guozhang On Thu, Jan 28, 2016 at 12:07 AM, Denise Fernandez wrote: > Hi, > > I have just made my first Kafka PR and would like to be added to the > contributor list so that I can assign myself to Jira issues and make > changes to

[jira] [Commented] (KAFKA-3068) NetworkClient may connect to a different Kafka cluster than originally configured

2016-01-28 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-3068?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15121772#comment-15121772 ] ASF GitHub Bot commented on KAFKA-3068: --- GitHub user enothereska opened a pull request:

Build failed in Jenkins: kafka-trunk-jdk7 #1000

2016-01-28 Thread Apache Jenkins Server
See Changes: [wangguoz] MINOR: Add Rolling Upgrade Notes to Security Docs -- Started by an SCM change [EnvInject] - Loading node environment variables. Building remotely on ubuntu3 (Ubuntu

Build failed in Jenkins: kafka-trunk-jdk7 #999

2016-01-28 Thread Apache Jenkins Server
See Changes: [wangguoz] KAFKA-2143: fix replica offset truncate to beginning during leader [wangguoz] KAFKA-2221: Log the real exception which triggered a reconnect -- [...truncated 7056

Does kstreams have a key-value store that is backed by Kafka?

2016-01-28 Thread James Cheng
Hi, I think I've heard that the upcoming Kafka Streams has something like Samza's local state key-value database. That is, something that looks like a key-value store that saves all writes to Kafka, and that can then later be reinitialized from Kafka. Does such a thing exist? Can someone

Build failed in Jenkins: kafka-trunk-jdk8 #326

2016-01-28 Thread Apache Jenkins Server
See Changes: [wangguoz] KAFKA-2143: fix replica offset truncate to beginning during leader [wangguoz] KAFKA-2221: Log the real exception which triggered a reconnect [wangguoz] MINOR: Add Rolling Upgrade Notes to Security Docs

Re: JIRA access

2016-01-28 Thread Magnus Edenhill
Thanks! 2016-01-28 17:36 GMT+01:00 Guozhang Wang : > Done. > > Guozhang > > On Thu, Jan 28, 2016 at 4:16 AM, Magnus Edenhill > wrote: > > > Hey, > > > > could I have access to assign myself to issues in the Apache JIRA? > > Username is edenhill. > >

[GitHub] kafka pull request: MINOR: Fix spelling and grammar issues in Repl...

2016-01-28 Thread ijuma
GitHub user ijuma opened a pull request: https://github.com/apache/kafka/pull/829 MINOR: Fix spelling and grammar issues in ReplicaFetcherThread detailed comment I noticed them while looking at the recent commit:

[GitHub] kafka pull request: KAFKA-3147 : Memory records is not writable in...

2016-01-28 Thread MayureshGharat
GitHub user MayureshGharat opened a pull request: https://github.com/apache/kafka/pull/825 KAFKA-3147 : Memory records is not writable in MirrorMaker Remove the batch from the RecordAccumulator once its closed while aborting batches. Make sure we don't accept new batch appends to

[jira] [Commented] (KAFKA-3147) Memory records is not writable in MirrorMaker

2016-01-28 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-3147?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15122084#comment-15122084 ] ASF GitHub Bot commented on KAFKA-3147: --- GitHub user MayureshGharat opened a pull request:

[jira] [Commented] (KAFKA-1791) Corrupt index after safe shutdown and restart

2016-01-28 Thread sandeep reddy (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1791?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15122134#comment-15122134 ] sandeep reddy commented on KAFKA-1791: -- There was an error in one of the threads during logs loading:

[jira] [Commented] (KAFKA-3060) Refactor MeteredXXStore

2016-01-28 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-3060?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15122147#comment-15122147 ] ASF GitHub Bot commented on KAFKA-3060: --- GitHub user guozhangwang opened a pull request:

Re: [DISCUSS] KIP-42: Add Producer and Consumer Interceptors

2016-01-28 Thread Anna Povzner
Hi Becket, The use-case for CRC is end-to-end audit, rather than checking whether a single message is corrupt or not. Regarding record size, I was thinking to extract record size from Record. That will include header overhead as well. I think total record size will tell users how much bandwidth

[GitHub] kafka pull request: KAFKA-3060: Refactor MeteredStore and RockDBSt...

2016-01-28 Thread guozhangwang
GitHub user guozhangwang opened a pull request: https://github.com/apache/kafka/pull/826 KAFKA-3060: Refactor MeteredStore and RockDBStore Impl Changes include: 1) Move logging logic from MeteredXXXStore to internal stores, and leave WindowedStore API clean by removed all

Re: [DISCUSS] KIP-42: Add Producer and Consumer Interceptors

2016-01-28 Thread Becket Qin
Anna, Using CRC to do end2end auditing might be very costly because you will need to collect all the CRC from both producer and consumer. And it is based on the assumption that broker does not modify the record. Can you shed some idea on how end to end auditing will be using the CRC before we

Re: [DISCUSS] KIP-42: Add Producer and Consumer Interceptors

2016-01-28 Thread Anna Povzner
Regarding record size as bytes sent over the wire, my concern is that it is almost impossible to calculate per-record. We could do as: 1) compressed bytes / number of records in a compressed message, as Todd mentioned; or 2) or same as #1 but take it proportional to uncompressed record size vs.

Re: [DISCUSS] KIP-42: Add Producer and Consumer Interceptors

2016-01-28 Thread Todd Palino
It may be difficult (or nearly impossible) to get actual compressed bytes for a message from a compressed batch, but I do think it’s useful information to have available for the very reason noted, bandwidth consumed. Does it make sense to have an interceptor at the batch level that can provide

Build failed in Jenkins: kafka-trunk-jdk7 #997

2016-01-28 Thread Apache Jenkins Server
See Changes: [wangguoz] KAFKA-3103; Fix transient Failure in -- [...truncated 4741 lines...] org.apache.kafka.connect.runtime.rest.resources.ConnectorsResourceTest > testCreateConnectorExists

Re: kafka-consumer-groups.sh doesn't work when consumers are off

2016-01-28 Thread Guozhang Wang
Once the offset is written to the log it is persistent and hence should survive broker failures. And its retention policy is configurable. It may be a bit misleading in saying "in-memory cache" in my previous email: the brokers just keep the in-memory map of [group, partition] -> latest_offset,

[jira] [Commented] (KAFKA-3159) Kafka consumer 0.9.0.0 client poll is very CPU intensive under certain conditions

2016-01-28 Thread Jason Gustafson (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-3159?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15122507#comment-15122507 ] Jason Gustafson commented on KAFKA-3159: However, I do see significantly more exceptions when the

[jira] [Commented] (KAFKA-3159) Kafka consumer 0.9.0.0 client poll is very CPU intensive under certain conditions

2016-01-28 Thread Jason Gustafson (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-3159?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15122467#comment-15122467 ] Jason Gustafson commented on KAFKA-3159: [~ra...@signalfx.com] Looks like these EOFExceptions are

Re: kafka-consumer-groups.sh doesn't work when consumers are off

2016-01-28 Thread Cliff Rhyne
Just following up on this concern. Is there a constraint that prevents ConsumerGroupCommand from reporting offsets on a group if no members are connected, or is this just the current implementation? Thanks, Cliff On Mon, Jan 25, 2016 at 3:50 PM, Cliff Rhyne wrote: > I'm

[GitHub] kafka pull request: KAFKA-3166; Disable SSL client authentication ...

2016-01-28 Thread ijuma
GitHub user ijuma opened a pull request: https://github.com/apache/kafka/pull/827 KAFKA-3166; Disable SSL client authentication for SASL_SSL security protocol Also: * Fixed a bug in `createSslConfig` where we were always generating a keystore even if `useClientCert` was

[jira] [Commented] (KAFKA-3166) Disable SSL client authentication for SASL_SSL security protocol

2016-01-28 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-3166?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15122259#comment-15122259 ] ASF GitHub Bot commented on KAFKA-3166: --- GitHub user ijuma opened a pull request:

[jira] [Updated] (KAFKA-3166) Disable SSL client authentication for SASL_SSL security protocol

2016-01-28 Thread Ismael Juma (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-3166?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ismael Juma updated KAFKA-3166: --- Reviewer: Jun Rao Status: Patch Available (was: Open) > Disable SSL client authentication for

[jira] [Commented] (KAFKA-3166) Disable SSL client authentication for SASL_SSL security protocol

2016-01-28 Thread Ismael Juma (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-3166?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15122266#comment-15122266 ] Ismael Juma commented on KAFKA-3166: Thoughts [~rsivaram] [~harsha_ch]? > Disable SSL client

[jira] [Created] (KAFKA-3166) Disable SSL client authentication for SASL_SSL security protocol

2016-01-28 Thread Ismael Juma (JIRA)
Ismael Juma created KAFKA-3166: -- Summary: Disable SSL client authentication for SASL_SSL security protocol Key: KAFKA-3166 URL: https://issues.apache.org/jira/browse/KAFKA-3166 Project: Kafka

[jira] [Assigned] (KAFKA-3166) Disable SSL client authentication for SASL_SSL security protocol

2016-01-28 Thread Ismael Juma (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-3166?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ismael Juma reassigned KAFKA-3166: -- Assignee: Ismael Juma > Disable SSL client authentication for SASL_SSL security protocol >

[jira] [Updated] (KAFKA-3103) Transient Failure in kafka.integration.PlaintextTopicMetadataTest.testIsrAfterBrokerShutDownAndJoinsBack

2016-01-28 Thread Guozhang Wang (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-3103?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guozhang Wang updated KAFKA-3103: - Assignee: Denise Fernandez (was: Guozhang Wang) > Transient Failure in >

[jira] [Assigned] (KAFKA-3103) Transient Failure in kafka.integration.PlaintextTopicMetadataTest.testIsrAfterBrokerShutDownAndJoinsBack

2016-01-28 Thread Guozhang Wang (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-3103?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guozhang Wang reassigned KAFKA-3103: Assignee: Guozhang Wang > Transient Failure in >

[jira] [Created] (KAFKA-3167) Use local to the workspace cache and recreate it on every build

2016-01-28 Thread Ismael Juma (JIRA)
Ismael Juma created KAFKA-3167: -- Summary: Use local to the workspace cache and recreate it on every build Key: KAFKA-3167 URL: https://issues.apache.org/jira/browse/KAFKA-3167 Project: Kafka

[GitHub] kafka pull request: KAFKA-3060: Refactor MeteredStore and RockDBSt...

2016-01-28 Thread guozhangwang
GitHub user guozhangwang reopened a pull request: https://github.com/apache/kafka/pull/826 KAFKA-3060: Refactor MeteredStore and RockDBStore Impl Changes include: 1) Move logging logic from MeteredXXXStore to internal stores, and leave WindowedStore API clean by removed

[GitHub] kafka pull request: KAFKA-3060: Refactor MeteredStore and RockDBSt...

2016-01-28 Thread guozhangwang
Github user guozhangwang closed the pull request at: https://github.com/apache/kafka/pull/826 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature

Jenkins build is back to normal : kafka_0.9.0_jdk7 #97

2016-01-28 Thread Apache Jenkins Server
See

[jira] [Updated] (KAFKA-3103) Transient Failure in kafka.integration.PlaintextTopicMetadataTest.testIsrAfterBrokerShutDownAndJoinsBack

2016-01-28 Thread Guozhang Wang (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-3103?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guozhang Wang updated KAFKA-3103: - Resolution: Fixed Fix Version/s: 0.9.0.1 Status: Resolved (was: Patch

[jira] [Created] (KAFKA-3168) Failure in kafka.integration.PrimitiveApiTest.testPipelinedProduceRequests

2016-01-28 Thread Guozhang Wang (JIRA)
Guozhang Wang created KAFKA-3168: Summary: Failure in kafka.integration.PrimitiveApiTest.testPipelinedProduceRequests Key: KAFKA-3168 URL: https://issues.apache.org/jira/browse/KAFKA-3168 Project:

[jira] [Commented] (KAFKA-3103) Transient Failure in kafka.integration.PlaintextTopicMetadataTest.testIsrAfterBrokerShutDownAndJoinsBack

2016-01-28 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-3103?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15122270#comment-15122270 ] ASF GitHub Bot commented on KAFKA-3103: --- Github user asfgit closed the pull request at:

[GitHub] kafka pull request: KAFKA-3103; Transient Failure in testIsrAfterB...

2016-01-28 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/kafka/pull/822 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[jira] [Commented] (KAFKA-3103) Transient Failure in kafka.integration.PlaintextTopicMetadataTest.testIsrAfterBrokerShutDownAndJoinsBack

2016-01-28 Thread Guozhang Wang (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-3103?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15122276#comment-15122276 ] Guozhang Wang commented on KAFKA-3103: -- Thanks [~rowdyrabbit]. Just added you to the contributor list

Build failed in Jenkins: kafka-trunk-jdk8 #324

2016-01-28 Thread Apache Jenkins Server
See Changes: [wangguoz] KAFKA-3103; Fix transient Failure in -- Started by an SCM change [EnvInject] - Loading node environment variables. Building remotely on ubuntu-2 (docker Ubuntu ubuntu

Build failed in Jenkins: kafka_0.9.0_jdk7 #98

2016-01-28 Thread Apache Jenkins Server
See Changes: [wangguoz] KAFKA-3103; Fix transient Failure in -- [...truncated 585 lines...] kafka.server.ServerShutdownTest > testCleanShutdown PASSED kafka.server.ServerShutdownTest >

[jira] [Commented] (KAFKA-2676) AclCommandTest has wrong package name

2016-01-28 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2676?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15122331#comment-15122331 ] ASF GitHub Bot commented on KAFKA-2676: --- GitHub user kichristensen opened a pull request:

[GitHub] kafka pull request: KAFKA-2676: AclCommandTest has wrong package n...

2016-01-28 Thread kichristensen
GitHub user kichristensen opened a pull request: https://github.com/apache/kafka/pull/828 KAFKA-2676: AclCommandTest has wrong package name Also fixed a couple of other tests with the same issue. This is my original work and I license the work to the project under the

Re: kafka-consumer-groups.sh doesn't work when consumers are off

2016-01-28 Thread Guozhang Wang
Hi Cliff, Short answer to your question is it is just the current implementation. The coordinator stores the offsets as messages in an internal topic and also keeps the latest offset values in in-memory. It answers ConsumerGroupRequest using its cached offset, and upon the consumer group being

[jira] [Commented] (KAFKA-3060) Refactor MeteredXXStore

2016-01-28 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-3060?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15122348#comment-15122348 ] ASF GitHub Bot commented on KAFKA-3060: --- GitHub user guozhangwang reopened a pull request:

Re: kafka-consumer-groups.sh doesn't work when consumers are off

2016-01-28 Thread Cliff Rhyne
Hi Guozhang, That looks like it might help but feels like there might be some gaps. Would it be able to survive restarts of the kafka broker? How long would it stay in the cache (and is that configurable)? If it expires from the cache, what's the cache-miss operation look like? (yes, a lot of

Re: [DISCUSS] KIP-42: Add Producer and Consumer Interceptors

2016-01-28 Thread Anna Povzner
Hi Becket, It will be up to the interceptor to implement their audit or monitoring strategy. I would also imagine there is more than one good way to do audit. So, I agree that some of the interceptors may not use CRC, and we will not require it. The question is now whether intercepting CRCs is

[GitHub] kafka pull request: Fix mis-spell in ConsumerConfig.scala

2016-01-28 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/kafka/pull/4 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

Build failed in Jenkins: kafka_0.9.0_jdk7 #99

2016-01-28 Thread Apache Jenkins Server
See Changes: [junrao] KAFKA-3152; kafka-acl doesn't allow space in principal name (backport [wangguoz] MINOR: Fix mis-spell in ConsumerConfig.scala -- Started by an SCM change [EnvInject] -

[jira] [Commented] (KAFKA-3159) Kafka consumer 0.9.0.0 client poll is very CPU intensive under certain conditions

2016-01-28 Thread Jason Gustafson (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-3159?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15122640#comment-15122640 ] Jason Gustafson commented on KAFKA-3159: [~ra...@signalfx.com] Here's a patch I've been messing

[jira] [Commented] (KAFKA-3159) Kafka consumer 0.9.0.0 client poll is very CPU intensive under certain conditions

2016-01-28 Thread Jason Gustafson (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-3159?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15122607#comment-15122607 ] Jason Gustafson commented on KAFKA-3159: You can use the DumpLogSegmentsTool. The output should

[jira] [Commented] (KAFKA-3159) Kafka consumer 0.9.0.0 client poll is very CPU intensive under certain conditions

2016-01-28 Thread Rajiv Kurian (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-3159?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15122614#comment-15122614 ] Rajiv Kurian commented on KAFKA-3159: - Actually I managed to dig through the logs and find the

[jira] [Updated] (KAFKA-3152) kafka-acl doesn't allow space in principal name

2016-01-28 Thread Jun Rao (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-3152?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jun Rao updated KAFKA-3152: --- Resolution: Fixed Fix Version/s: (was: 0.9.1.0) Status: Resolved (was: Patch Available)

[jira] [Commented] (KAFKA-3159) Kafka consumer 0.9.0.0 client poll is very CPU intensive under certain conditions

2016-01-28 Thread Rajiv Kurian (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-3159?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15122659#comment-15122659 ] Rajiv Kurian commented on KAFKA-3159: - Thanks Jason. I can try to do that early next week. Have a lot

Build failed in Jenkins: kafka-trunk-jdk7 #998

2016-01-28 Thread Apache Jenkins Server
See Changes: [wangguoz] MINOR: Fix mis-spell in ConsumerConfig.scala -- Started by an SCM change [EnvInject] - Loading node environment variables. Building remotely on H11 (docker Ubuntu ubuntu

[jira] [Commented] (KAFKA-3159) Kafka consumer 0.9.0.0 client poll is very CPU intensive under certain conditions

2016-01-28 Thread Rajiv Kurian (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-3159?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15122594#comment-15122594 ] Rajiv Kurian commented on KAFKA-3159: - I don't enable compression on the topic. However the producer

[jira] [Updated] (KAFKA-3103) Transient Failure in kafka.integration.PlaintextTopicMetadataTest.testIsrAfterBrokerShutDownAndJoinsBack

2016-01-28 Thread Ismael Juma (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-3103?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ismael Juma updated KAFKA-3103: --- Reviewer: Guozhang Wang Status: Patch Available (was: Open) > Transient Failure in >

[jira] [Resolved] (KAFKA-2836) FAILING TEST: SaslSslTopicMetadataTest

2016-01-28 Thread Ismael Juma (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2836?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ismael Juma resolved KAFKA-2836. Resolution: Duplicate Marking as a duplicate of KAFKA-3103, which has a PR. > FAILING TEST:

[jira] [Resolved] (KAFKA-3155) Transient Failure in kafka.integration.PlaintextTopicMetadataTest.testIsrAfterBrokerShutDownAndJoinsBack

2016-01-28 Thread Ismael Juma (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-3155?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ismael Juma resolved KAFKA-3155. Resolution: Duplicate Duplicate of KAFKA-3103. > Transient Failure in >

JIRA access

2016-01-28 Thread Magnus Edenhill
Hey, could I have access to assign myself to issues in the Apache JIRA? Username is edenhill. Looking to provide some patches for KAFKA-3160. Regards, Magnus

Re: [DISCUSS] KIP-43: Kafka SASL enhancements

2016-01-28 Thread Rajini Sivaram
Tao, We currently add the security provider in a static initializer in our login module. This ensures that the security provider is always installed before Kafka creates SaslServer/SaslClient. As you say, it is also possible to insert code into your application to add security provider before

[jira] [Created] (KAFKA-3165) Fix ignored parameters in the gradle "All" tasks

2016-01-28 Thread Grant Henke (JIRA)
Grant Henke created KAFKA-3165: -- Summary: Fix ignored parameters in the gradle "All" tasks Key: KAFKA-3165 URL: https://issues.apache.org/jira/browse/KAFKA-3165 Project: Kafka Issue Type:

Re: kafka-consumer-groups.sh doesn't work when consumers are off

2016-01-28 Thread tao xiao
Guozhang, The old ConsumerOffsetChecker works for new consumer too with offset stored in Kafka. I tested it with mirror maker with new consumer enabled. it is able to show offset during mirror maker running and after its shutdown. On Fri, 29 Jan 2016 at 06:34 Guozhang Wang

[GitHub] kafka pull request: KAFKA-2143: fix replica offset truncate to beg...

2016-01-28 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/kafka/pull/129 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[jira] [Updated] (KAFKA-2143) Replicas get ahead of leader and fail

2016-01-28 Thread Guozhang Wang (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2143?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guozhang Wang updated KAFKA-2143: - Resolution: Fixed Fix Version/s: 0.9.0.1 Status: Resolved (was: Patch

[jira] [Commented] (KAFKA-2143) Replicas get ahead of leader and fail

2016-01-28 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2143?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15122919#comment-15122919 ] ASF GitHub Bot commented on KAFKA-2143: --- Github user asfgit closed the pull request at:

[GitHub] kafka pull request: Log the real exception which triggered a recon...

2016-01-28 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/kafka/pull/138 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[jira] [Updated] (KAFKA-2221) Log the entire cause which caused a reconnect in the SimpleConsumer

2016-01-28 Thread Guozhang Wang (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2221?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guozhang Wang updated KAFKA-2221: - Resolution: Fixed Fix Version/s: 0.9.0.1 Status: Resolved (was: Patch

[jira] [Commented] (KAFKA-2221) Log the entire cause which caused a reconnect in the SimpleConsumer

2016-01-28 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2221?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15122923#comment-15122923 ] ASF GitHub Bot commented on KAFKA-2221: --- Github user asfgit closed the pull request at:

[jira] [Updated] (KAFKA-2435) More optimally balanced partition assignment strategy

2016-01-28 Thread Guozhang Wang (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2435?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guozhang Wang updated KAFKA-2435: - Reviewer: Jason Gustafson Assigning to [~hachikuji] to review. > More optimally balanced

[jira] [Updated] (KAFKA-2522) ConsumerGroupCommand sends all output to STDOUT

2016-01-28 Thread Guozhang Wang (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2522?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guozhang Wang updated KAFKA-2522: - Reviewer: Ismael Juma > ConsumerGroupCommand sends all output to STDOUT >

[GitHub] kafka pull request: Add Rolling Upgrade Notes to Security Docs

2016-01-28 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/kafka/pull/625 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[jira] [Commented] (KAFKA-2653) Stateful operations in the KStream DSL layer

2016-01-28 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-2653?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15122951#comment-15122951 ] ASF GitHub Bot commented on KAFKA-2653: --- Github user guozhangwang closed the pull request at:

[GitHub] kafka pull request: KAFKA-2653: Kafka Streams Stateful API Design

2016-01-28 Thread guozhangwang
Github user guozhangwang closed the pull request at: https://github.com/apache/kafka/pull/691 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature

Re: [DISCUSS] KIP-43: Kafka SASL enhancements

2016-01-28 Thread tao xiao
Sounds like a good approach to add provider in login module. Would love to see updates in the PR to reflect the changes in Login and AuthCallbackHandler. On Thu, 28 Jan 2016 at 19:31 Rajini Sivaram wrote: > Tao, > > We currently add the security provider in a

[jira] [Commented] (KAFKA-1545) java.net.InetAddress.getLocalHost in KafkaHealthcheck.register may fail on some irregular hostnames

2016-01-28 Thread Guozhang Wang (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1545?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15122719#comment-15122719 ] Guozhang Wang commented on KAFKA-1545: -- Left some comments in the PR, closing this ticket as not

[jira] [Commented] (KAFKA-948) ISR list in LeaderAndISR path not updated for partitions when Broker (which is not leader) is down

2016-01-28 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-948?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15122731#comment-15122731 ] ASF GitHub Bot commented on KAFKA-948: -- Github user dibbhatt closed the pull request at:

[GitHub] kafka pull request: KAFKA-948 : Update ReplicaStateMachine.scala

2016-01-28 Thread dibbhatt
Github user dibbhatt closed the pull request at: https://github.com/apache/kafka/pull/5 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

Re: [DISCUSS] KIP-42: Add Producer and Consumer Interceptors

2016-01-28 Thread Anna Povzner
On a second thought, yes, I think we should expose record size that represents application bytes. This is Becket's option #1. I updated the KIP wiki with new fields in RecordMetadata and ConsumerRecord. I would like to start a voting thread tomorrow if there are no objections or more things to

[jira] [Resolved] (KAFKA-1545) java.net.InetAddress.getLocalHost in KafkaHealthcheck.register may fail on some irregular hostnames

2016-01-28 Thread Guozhang Wang (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1545?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guozhang Wang resolved KAFKA-1545. -- Resolution: Not A Problem > java.net.InetAddress.getLocalHost in KafkaHealthcheck.register may

Build failed in Jenkins: kafka-trunk-jdk8 #325

2016-01-28 Thread Apache Jenkins Server
See Changes: [wangguoz] MINOR: Fix mis-spell in ConsumerConfig.scala -- [...truncated 82 lines...]

[jira] [Commented] (KAFKA-1464) Add a throttling option to the Kafka replication tool

2016-01-28 Thread Jay Kreps (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-1464?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15121691#comment-15121691 ] Jay Kreps commented on KAFKA-1464: -- I agree that the key difference is in-sync vs out-of-sync replicas.

[GitHub] kafka pull request: KAFKA-3161: Fixed ProducerConfig/ConsumerConfi...

2016-01-28 Thread crhyne
GitHub user crhyne opened a pull request: https://github.com/apache/kafka/pull/824 KAFKA-3161: Fixed ProducerConfig/ConsumerConfig so that defaults are used in java.util.Properties This impacts the ProducerConfig and ConsumerConfig. I added a unit test to reflect the new case.

[jira] [Commented] (KAFKA-3155) Transient Failure in kafka.integration.PlaintextTopicMetadataTest.testIsrAfterBrokerShutDownAndJoinsBack

2016-01-28 Thread Guozhang Wang (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-3155?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15121853#comment-15121853 ] Guozhang Wang commented on KAFKA-3155: -- [~ijuma] Not sure if it the same error since the error stack

Re: kafka-consumer-groups.sh doesn't work when consumers are off

2016-01-28 Thread Guozhang Wang
Tao, Hmm that is a bit wired since ConsumerOffsetChecker itself does not talk to brokers at all, but only through ZK. Guozhang On Thu, Jan 28, 2016 at 6:07 PM, tao xiao wrote: > Guozhang, > > The old ConsumerOffsetChecker works for new consumer too with offset stored >

  1   2   >