[jira] [Assigned] (KAFKA-2460) Fix capitalization in SSL classes

[Ismael Juma (JIRA)]

 [ 
https://issues.apache.org/jira/browse/KAFKA-2460?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ismael Juma reassigned KAFKA-2460:
--

Assignee: Ismael Juma  (was: sriharsha chintalapani)

> Fix capitalization in SSL classes
> -
>
> Key: KAFKA-2460
> URL: https://issues.apache.org/jira/browse/KAFKA-2460
> Project: Kafka
>  Issue Type: Improvement
>Affects Versions: 0.9.0.0
>Reporter: Jay Kreps
>Assignee: Ismael Juma
>Priority: Minor
>
> I notice that all the SSL classes are using the convention SSLChannelBuilder, 
> SSLConfigs, etc. Kafka has always used the convention SslChannelBuilder, 
> SslConfigs, etc. See e.g. KafkaApis, ApiUtils, LeaderAndIsrRequest, 
> ClientIdAndTopic, etc.
> We should fix this.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (KAFKA-2460) Fix capitalization in SSL classes

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/KAFKA-2460?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14972694#comment-14972694
 ] 

ASF GitHub Bot commented on KAFKA-2460:
---

Github user asfgit closed the pull request at:

https://github.com/apache/kafka/pull/355


> Fix capitalization in SSL classes
> -
>
> Key: KAFKA-2460
> URL: https://issues.apache.org/jira/browse/KAFKA-2460
> Project: Kafka
>  Issue Type: Improvement
>Affects Versions: 0.9.0.0
>Reporter: Jay Kreps
>Assignee: Ismael Juma
>Priority: Minor
>
> I notice that all the SSL classes are using the convention SSLChannelBuilder, 
> SSLConfigs, etc. Kafka has always used the convention SslChannelBuilder, 
> SslConfigs, etc. See e.g. KafkaApis, ApiUtils, LeaderAndIsrRequest, 
> ClientIdAndTopic, etc.
> We should fix this.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] kafka pull request: KAFKA-2460; Fix capitalisation in SSL classes

[asfgit]

Github user asfgit closed the pull request at:

https://github.com/apache/kafka/pull/355


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Updated] (KAFKA-2449) Update mirror maker (MirrorMaker) docs

[Ismael Juma (JIRA)]

 [ 
https://issues.apache.org/jira/browse/KAFKA-2449?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ismael Juma updated KAFKA-2449:
---
Status: Patch Available  (was: Open)

> Update mirror maker (MirrorMaker) docs
> --
>
> Key: KAFKA-2449
> URL: https://issues.apache.org/jira/browse/KAFKA-2449
> Project: Kafka
>  Issue Type: Bug
>Reporter: Geoff Anderson
>Assignee: Gwen Shapira
>Priority: Blocker
> Fix For: 0.9.0.0
>
>
> The Kafka docs on Mirror Maker state that it mirrors from N source clusters 
> to 1 destination, but this is no longer the case. Docs should be updated to 
> reflect that it mirrors from single source cluster to single target cluster.
> Docs I've found where this should be updated:
> http://kafka.apache.org/documentation.html#basic_ops_mirror_maker
> https://cwiki.apache.org/confluence/display/KAFKA/Kafka+mirroring+(MirrorMaker)



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (KAFKA-2472) Fix kafka ssl configs to not throw warnings

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/KAFKA-2472?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14972684#comment-14972684
 ] 

ASF GitHub Bot commented on KAFKA-2472:
---

GitHub user ijuma opened a pull request:

https://github.com/apache/kafka/pull/355

KAFKA-2472; Fix capitalisation in SSL classes



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/ijuma/kafka 
kafka-2460-fix-capitalisation-in-ssl-classes

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/kafka/pull/355.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #355


commit 9bbf8b701f13f0b4ea8df842b46cab29002033be
Author: Ismael Juma 
Date:   2015-10-24T16:24:18Z

Fix capitalisation in SSL classes




> Fix kafka ssl configs to not throw warnings
> ---
>
> Key: KAFKA-2472
> URL: https://issues.apache.org/jira/browse/KAFKA-2472
> Project: Kafka
>  Issue Type: Bug
>Reporter: Sriharsha Chintalapani
>Assignee: Ismael Juma
> Fix For: 0.9.0.0
>
>
> This is a follow-up fix on kafka-1690.
> [2015-08-25 18:20:48,236] WARN The configuration ssl.truststore.password = 
> striker was supplied but isn't a known config. 
> (org.apache.kafka.clients.producer.ProducerConfig)
> [2015-08-25 18:20:48,236] WARN The configuration security.protocol = SSL was 
> supplied but isn't a known config. 
> (org.apache.kafka.clients.producer.ProducerConfig)



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Jenkins build is back to normal : kafka_system_tests #119

[ewen]

See 

[jira] [Updated] (KAFKA-2449) Update mirror maker (MirrorMaker) docs

[Gwen Shapira (JIRA)]

 [ 
https://issues.apache.org/jira/browse/KAFKA-2449?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Gwen Shapira updated KAFKA-2449:

Assignee: Gwen Shapira  (was: Geoff Anderson)

> Update mirror maker (MirrorMaker) docs
> --
>
> Key: KAFKA-2449
> URL: https://issues.apache.org/jira/browse/KAFKA-2449
> Project: Kafka
>  Issue Type: Bug
>Reporter: Geoff Anderson
>Assignee: Gwen Shapira
>Priority: Blocker
> Fix For: 0.9.0.0
>
>
> The Kafka docs on Mirror Maker state that it mirrors from N source clusters 
> to 1 destination, but this is no longer the case. Docs should be updated to 
> reflect that it mirrors from single source cluster to single target cluster.
> Docs I've found where this should be updated:
> http://kafka.apache.org/documentation.html#basic_ops_mirror_maker
> https://cwiki.apache.org/confluence/display/KAFKA/Kafka+mirroring+(MirrorMaker)



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (KAFKA-2449) Update mirror maker (MirrorMaker) docs

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/KAFKA-2449?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14972690#comment-14972690
 ] 

ASF GitHub Bot commented on KAFKA-2449:
---

GitHub user gwenshap opened a pull request:

https://github.com/apache/kafka/pull/356

KAFKA-2449: Update mirror maker (MirrorMaker) docs - remove reference…

…s to multiple source clusters

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/gwenshap/kafka KAFKA-2449

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/kafka/pull/356.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #356


commit bf8fc4d075c565e6b25bf7d63dff363d2b8b782c
Author: Gwen Shapira 
Date:   2015-10-24T16:37:38Z

KAFKA-2449: Update mirror maker (MirrorMaker) docs - remove references to 
multiple source clusters




> Update mirror maker (MirrorMaker) docs
> --
>
> Key: KAFKA-2449
> URL: https://issues.apache.org/jira/browse/KAFKA-2449
> Project: Kafka
>  Issue Type: Bug
>Reporter: Geoff Anderson
>Assignee: Geoff Anderson
>Priority: Blocker
> Fix For: 0.9.0.0
>
>
> The Kafka docs on Mirror Maker state that it mirrors from N source clusters 
> to 1 destination, but this is no longer the case. Docs should be updated to 
> reflect that it mirrors from single source cluster to single target cluster.
> Docs I've found where this should be updated:
> http://kafka.apache.org/documentation.html#basic_ops_mirror_maker
> https://cwiki.apache.org/confluence/display/KAFKA/Kafka+mirroring+(MirrorMaker)



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] kafka pull request: KAFKA-2449: Update mirror maker (MirrorMaker) ...

[gwenshap]

GitHub user gwenshap opened a pull request:

https://github.com/apache/kafka/pull/356

KAFKA-2449: Update mirror maker (MirrorMaker) docs - remove reference…

…s to multiple source clusters

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/gwenshap/kafka KAFKA-2449

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/kafka/pull/356.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #356


commit bf8fc4d075c565e6b25bf7d63dff363d2b8b782c
Author: Gwen Shapira 
Date:   2015-10-24T16:37:38Z

KAFKA-2449: Update mirror maker (MirrorMaker) docs - remove references to 
multiple source clusters




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Updated] (KAFKA-2460) Fix capitalization in SSL classes

[Ismael Juma (JIRA)]

 [ 
https://issues.apache.org/jira/browse/KAFKA-2460?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ismael Juma updated KAFKA-2460:
---
Reviewer: Jun Rao
  Status: Patch Available  (was: Open)

Harsha, I hope you don't mind; I submitted a PR for this:

https://github.com/apache/kafka/pull/355

Jun, we had discussed that we could do this after 0.9.0.0, but there are two 
reasons why I think we should do it before:

1. We claim that the variable names for config classes cannot be changed, I 
assume that means that we don't want to change the class name of `SSLConfigs` 
after the release.

2. This will make it easier to backport changes from trunk to the 0.9 branch 
after it's created.

> Fix capitalization in SSL classes
> -
>
> Key: KAFKA-2460
> URL: https://issues.apache.org/jira/browse/KAFKA-2460
> Project: Kafka
>  Issue Type: Improvement
>Affects Versions: 0.9.0.0
>Reporter: Jay Kreps
>Assignee: Ismael Juma
>Priority: Minor
>
> I notice that all the SSL classes are using the convention SSLChannelBuilder, 
> SSLConfigs, etc. Kafka has always used the convention SslChannelBuilder, 
> SslConfigs, etc. See e.g. KafkaApis, ApiUtils, LeaderAndIsrRequest, 
> ClientIdAndTopic, etc.
> We should fix this.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (KAFKA-2588) ReplicaManager partitionCount metric should actually be replicaCount

[Gwen Shapira (JIRA)]

[ 
https://issues.apache.org/jira/browse/KAFKA-2588?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14972752#comment-14972752
 ] 

Gwen Shapira commented on KAFKA-2588:
-

I'm trying to follow your logic and failing...

You mentioned "The metric (aggregated) would show..."

But this is a per-broker metric, and I'm not sure it makes sense to aggregate. 

PartitionCount shows number of partitions served by the broker. This number is 
fixed (unless someone  reassigns) and since each partition takes certain number 
of resources (mostly memory and file handles), and since each partition can 
potentially become leader or follower, it is pretty important to have this 
number. 

LeaderCount on a broker can change and is a good proxy for how loaded a broker 
is at the moment.

So, long way of saying - I think you are a bit confused because you are trying 
to aggregate cluster-wide two metrics that are built for monitoring load on 
specific brokers and were not written to make sense when aggregated at cluster 
level.




> ReplicaManager partitionCount metric should actually be replicaCount
> 
>
> Key: KAFKA-2588
> URL: https://issues.apache.org/jira/browse/KAFKA-2588
> Project: Kafka
>  Issue Type: Bug
>  Components: core
>Affects Versions: 0.8.2.1
>Reporter: Grant Henke
>Assignee: Grant Henke
>
> The metrics "partitionCount" in the ReplicaManager actually represents the 
> count of replicas. 
> As an example if I have a cluster with 1 topic with 1 partitions and a 
> replication factor of 3. The metric (aggregated) would show a value of 3. 
> There is a metric called "LeaderCount" that actually represents the 
> "partitionCount". In my example above the metric (aggregated) would show a 
> value of 1. 
> We do need to consider compatibility of consuming systems. I think the most 
> simple change would be to:
> - Adjust the "partitionCount" metric to be the same value as "LeaderCount"
> - Add a "replicaCount" metric which contains the values "partitionCount" does 
> today
> - Leave "LeaderCount" in for compatibility
> Documentation will need to be updated as well. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (KAFKA-2609) SSL renegotiation code paths need more tests

[Ismael Juma (JIRA)]

[ 
https://issues.apache.org/jira/browse/KAFKA-2609?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14972794#comment-14972794
 ] 

Ismael Juma commented on KAFKA-2609:


[~rsivaram], one thing to keep in mind is that it looks like renegotiation has 
been removed in the TLS 1.3 draft spec:

{quote}
1.2. Major Differences from TLS 1.2

draft-10

Remove ClientCertificateTypes field from CertificateRequest and add extensions.
Merge client and server key shares into a single extension.
draft-09

Change to RSA-PSS signatures for handshake messages.
Remove support for DSA.
Update key schedule per suggestions by Hugo, Hoeteck, and Bjoern Tackmann.
Add support for per-record padding.
Switch to encrypted record ContentType.
Change HKDF labeling to include protocol version and value lengths.
Shift the final decision to abort a handshake due to incompatible certificates 
to the client rather than having servers abort early.
Deprecate SHA-1 with signatures.
Add MTI algorithms.
draft-08

Remove support for weak and lesser used named curves.
Remove support for MD5 and SHA-224 hashes with signatures.
Update lists of available AEAD cipher suites and error alerts.
Reduce maximum permitted record expansion for AEAD from 2048 to 256 octets.
Require digital signatures even when a previous configuration is used.
Merge EarlyDataIndication and KnownConfiguration.
Change code point for server_configuration to avoid collision with 
server_hello_done.
Relax certificate_list ordering requirement to match current practice.
draft-07

Integration of semi-ephemeral DH proposal.
Add initial 0-RTT support.
Remove resumption and replace with PSK + tickets.
Move ClientKeyShare into an extension.
Move to HKDF.
draft-06

Prohibit RC4 negotiation for backwards compatibility.
Freeze & deprecate record layer version field.
Update format of signatures with context.
Remove explicit IV.
draft-05

Prohibit SSL negotiation for backwards compatibility.
Fix which MS is used for exporters.
draft-04

Modify key computations to include session hash.
Remove ChangeCipherSpec.
Renumber the new handshake messages to be somewhat more consistent with 
existing convention and to remove a duplicate registration.
Remove renegotiation.
Remove point format negotiation.
draft-03

Remove GMT time.
Merge in support for ECC from RFC 4492 but without explicit curves.
Remove the unnecessary length field from the AD input to AEAD ciphers.
Rename {Client,Server}KeyExchange to {Client,Server}KeyShare.
Add an explicit HelloRetryRequest to reject the client’s.
draft-02

Increment version number.
Rework handshake to provide 1-RTT mode.
Remove custom DHE groups.
Remove support for compression.
Remove support for static RSA and DH key exchange.
Remove support for non-AEAD ciphers.
{quote}
https://tlswg.github.io/tls13-spec/

> SSL renegotiation code paths need more tests
> 
>
> Key: KAFKA-2609
> URL: https://issues.apache.org/jira/browse/KAFKA-2609
> Project: Kafka
>  Issue Type: Test
>Affects Versions: 0.9.0.0
>Reporter: Rajini Sivaram
>Assignee: Rajini Sivaram
> Fix For: 0.10.0.0
>
>
> If renegotiation is triggered when read interest is off, at the moment it 
> looks like read interest is never turned back on. More unit tests are 
> required to test different renegotiation scenarios since these are much 
> harder to exercise in system tests.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Re: (User) Document/guide on new Kafka features on security in upcoming 0.9.0?

[Ismael Juma]

Hi Jaikan,

Testing before the release is very much appreciated. As Harsha said,
user-focused documentation is still in-progress. In addition to the SSL
wiki page that Harsha mentioned, Parth wrote a wiki page for the
kafka-acls.sh tool:

https://cwiki.apache.org/confluence/display/KAFKA/Kafka+Authorization+Command+Line+Interface

There is also a KIP for ZooKeeper authentication:

https://cwiki.apache.org/confluence/display/KAFKA/KIP-38%3A+ZooKeeper+Authentication

In terms of the various security features that will be in 0.9 (SSL
authentication and encryption, SASL/Kerberos authentication, ACL
authorization, ZooKeeper authentication), which are of interest to you?

Ismael

On Sat, Oct 24, 2015 at 4:35 AM, Harsha  wrote:

> Jaikiran,
>  Here is the SSL setup doc
>
> https://cwiki.apache.org/confluence/display/KAFKA/Deploying+SSL+for+Kafka
>  .
> I am going to write one for SASL ( kerberos ) soon and Parth is working
> on pushing one for Authorizer as well.
>
> Thanks,
> Harsha
>
> On Fri, Oct 23, 2015, at 04:42 PM, Jaikiran Pai wrote:
> > Given that 0.9.0 is being planned to released sometime soon in November,
> > our team has started looking into what's coming in the newer version and
> > what we need to do to start using the features. We are very much and
> > mainly interested in the security features of it for the upcoming
> > releases. We have been watching some of the KIP discussion threads and
> > the KIP articles around security[1][2]. However, those are very much
> > like design discussions/rationale. Is there some document which explains
> > to end users of Kafka to understand what security features have been
> > implemented for this release and what needs to be done to start using
> > those?
> >
> > The reason I ask for these now, before the release, is so that some of
> > us can start testing these out before the release and raise any issues
> > that we might run into.
> >
> > [1]
> >
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-11+-+Authorization+Interface
> > [2]
> >
> https://cwiki.apache.org/confluence/display/KAFKA/Security#Security-Authorization
> >
> > -Jaikiran
>

[GitHub] kafka pull request: KAFKA-2472; Fix capitalisation in SSL classes

[ijuma]

GitHub user ijuma opened a pull request:

https://github.com/apache/kafka/pull/355

KAFKA-2472; Fix capitalisation in SSL classes



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/ijuma/kafka 
kafka-2460-fix-capitalisation-in-ssl-classes

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/kafka/pull/355.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #355


commit 9bbf8b701f13f0b4ea8df842b46cab29002033be
Author: Ismael Juma 
Date:   2015-10-24T16:24:18Z

Fix capitalisation in SSL classes




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Commented] (KAFKA-2644) Run relevant ducktape tests with SASL_PLAINTEXT and SASL_SSL

[Ewen Cheslack-Postava (JIRA)]

[ 
https://issues.apache.org/jira/browse/KAFKA-2644?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14972724#comment-14972724
 ] 

Ewen Cheslack-Postava commented on KAFKA-2644:
--

It shouldn't be any different. Generally, ducktape just splits getting a new 
service running into installing it (which can be done via the scripts we use 
for Vagrant provisioning) and running it as a Service (just requires writing a 
new Service class for Kerberos, which might need to be a little more flexible 
than what [~sriharsha] has, but not significantly more complex).

Given that there are already scripts for setting up Kerberos, both are probably 
about the same level of effort to get working.

> Run relevant ducktape tests with SASL_PLAINTEXT and SASL_SSL
> 
>
> Key: KAFKA-2644
> URL: https://issues.apache.org/jira/browse/KAFKA-2644
> Project: Kafka
>  Issue Type: Sub-task
>  Components: security
>Reporter: Ismael Juma
>Assignee: Rajini Sivaram
>Priority: Critical
> Fix For: 0.9.0.0
>
>
> We need to define which of the existing ducktape tests are relevant. cc 
> [~rsivaram]



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Build failed in Jenkins: kafka-trunk-jdk8 #55

[Apache Jenkins Server]

See 

Changes:

[harsha] KAFKA-2460; Fix capitalisation in SSL classes

--
[...truncated 1801 lines...]

kafka.log.CleanerTest > testCleaningWithUnkeyedMessages PASSED

kafka.log.OffsetMapTest > testClear PASSED

kafka.log.OffsetMapTest > testBasicValidation PASSED

kafka.coordinator.GroupCoordinatorResponseTest > testHeartbeatWrongCoordinator 
PASSED

kafka.coordinator.GroupCoordinatorResponseTest > testHeartbeatIllegalGeneration 
PASSED

kafka.coordinator.GroupCoordinatorResponseTest > testLeaderFailureInSyncGroup 
PASSED

kafka.coordinator.GroupCoordinatorResponseTest > 
testGenerationIdIncrementsOnRebalance PASSED

kafka.coordinator.GroupCoordinatorResponseTest > 
testSyncGroupFromIllegalGeneration PASSED

kafka.coordinator.GroupCoordinatorResponseTest > testHeartbeatUnknownGroup 
PASSED

kafka.coordinator.GroupCoordinatorResponseTest > 
testHeartbeatDuringRebalanceCausesRebalanceInProgress PASSED

kafka.coordinator.GroupCoordinatorResponseTest > 
testJoinGroupInconsistentGroupProtocol PASSED

kafka.coordinator.GroupCoordinatorResponseTest > 
testJoinGroupSessionTimeoutTooLarge PASSED

kafka.coordinator.GroupCoordinatorResponseTest > 
testJoinGroupSessionTimeoutTooSmall PASSED

kafka.coordinator.GroupCoordinatorResponseTest > 
testCommitOffsetWithDefaultGeneration PASSED

kafka.coordinator.GroupCoordinatorResponseTest > 
testJoinGroupFromUnchangedLeaderShouldRebalance PASSED

kafka.coordinator.GroupCoordinatorResponseTest > 
testHeartbeatRebalanceInProgress PASSED

kafka.coordinator.GroupCoordinatorResponseTest > testLeaveGroupUnknownGroup 
PASSED

kafka.coordinator.GroupCoordinatorResponseTest > 
testSyncGroupFollowerAfterLeader PASSED

kafka.coordinator.GroupCoordinatorResponseTest > testCommitOffsetInAwaitingSync 
PASSED

kafka.coordinator.GroupCoordinatorResponseTest > testJoinGroupWrongCoordinator 
PASSED

kafka.coordinator.GroupCoordinatorResponseTest > 
testJoinGroupUnknownConsumerExistingGroup PASSED

kafka.coordinator.GroupCoordinatorResponseTest > testSyncGroupFromUnknownGroup 
PASSED

kafka.coordinator.GroupCoordinatorResponseTest > 
testJoinGroupInconsistentProtocolType PASSED

kafka.coordinator.GroupCoordinatorResponseTest > 
testCommitOffsetFromUnknownGroup PASSED

kafka.coordinator.GroupCoordinatorResponseTest > testLeaveGroupWrongCoordinator 
PASSED

kafka.coordinator.GroupCoordinatorResponseTest > 
testLeaveGroupUnknownConsumerExistingGroup PASSED

kafka.coordinator.GroupCoordinatorResponseTest > 
testJoinGroupUnknownConsumerNewGroup PASSED

kafka.coordinator.GroupCoordinatorResponseTest > 
testJoinGroupFromUnchangedFollowerDoesNotRebalance PASSED

kafka.coordinator.GroupCoordinatorResponseTest > testValidJoinGroup PASSED

kafka.coordinator.GroupCoordinatorResponseTest > 
testSyncGroupLeaderAfterFollower PASSED

kafka.coordinator.GroupCoordinatorResponseTest > testSyncGroupFromUnknownMember 
PASSED

kafka.coordinator.GroupCoordinatorResponseTest > testValidLeaveGroup PASSED

kafka.coordinator.GroupCoordinatorResponseTest > testSyncGroupNotCoordinator 
PASSED

kafka.coordinator.GroupCoordinatorResponseTest > 
testHeartbeatUnknownConsumerExistingGroup PASSED

kafka.coordinator.GroupCoordinatorResponseTest > testValidHeartbeat PASSED

kafka.coordinator.MemberMetadataTest > testMatchesSupportedProtocols PASSED

kafka.coordinator.MemberMetadataTest > testMetadata PASSED

kafka.coordinator.MemberMetadataTest > testMetadataRaisesOnUnsupportedProtocol 
PASSED

kafka.coordinator.MemberMetadataTest > testVoteForPreferredProtocol PASSED

kafka.coordinator.MemberMetadataTest > testVoteRaisesOnNoSupportedProtocols 
PASSED

kafka.coordinator.GroupMetadataTest > testDeadToAwaitingSyncIllegalTransition 
PASSED

kafka.coordinator.GroupMetadataTest > 
testPreparingRebalanceToStableIllegalTransition PASSED

kafka.coordinator.GroupMetadataTest > testCannotRebalanceWhenDead PASSED

kafka.coordinator.GroupMetadataTest > testStableToDeadIllegalTransition PASSED

kafka.coordinator.GroupMetadataTest > testSelectProtocol PASSED

kafka.coordinator.GroupMetadataTest > testCannotRebalanceWhenPreparingRebalance 
PASSED

kafka.coordinator.GroupMetadataTest > 
testDeadToPreparingRebalanceIllegalTransition PASSED

kafka.coordinator.GroupMetadataTest > testCanRebalanceWhenAwaitingSync PASSED

kafka.coordinator.GroupMetadataTest > 
testAwaitingSyncToPreparingRebalanceTransition PASSED

kafka.coordinator.GroupMetadataTest > testStableToAwaitingSyncIllegalTransition 
PASSED

kafka.coordinator.GroupMetadataTest > testDeadToDeadIllegalTransition PASSED

kafka.coordinator.GroupMetadataTest > testSelectProtocolRaisesIfNoMembers PASSED

kafka.coordinator.GroupMetadataTest > testStableToPreparingRebalanceTransition 
PASSED

kafka.coordinator.GroupMetadataTest > testPreparingRebalanceToDeadTransition 
PASSED

kafka.coordinator.GroupMetadataTest > testStableToStableIllegalTransition PASSED

kafka.coordinator.GroupMetadataTest > 

[jira] [Updated] (KAFKA-2460) Fix capitalization in SSL classes

[Ismael Juma (JIRA)]

 [ 
https://issues.apache.org/jira/browse/KAFKA-2460?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ismael Juma updated KAFKA-2460:
---
Resolution: Fixed
  Reviewer: sriharsha chintalapani  (was: Jun Rao)
Status: Resolved  (was: Patch Available)

> Fix capitalization in SSL classes
> -
>
> Key: KAFKA-2460
> URL: https://issues.apache.org/jira/browse/KAFKA-2460
> Project: Kafka
>  Issue Type: Improvement
>Affects Versions: 0.9.0.0
>Reporter: Jay Kreps
>Assignee: Ismael Juma
>Priority: Minor
>
> I notice that all the SSL classes are using the convention SSLChannelBuilder, 
> SSLConfigs, etc. Kafka has always used the convention SslChannelBuilder, 
> SslConfigs, etc. See e.g. KafkaApis, ApiUtils, LeaderAndIsrRequest, 
> ClientIdAndTopic, etc.
> We should fix this.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Jenkins build is back to normal : kafka-trunk-jdk7 #718

[Apache Jenkins Server]

See 

[jira] [Commented] (KAFKA-2689) Expose select gauges and metrics programmatically (not just through JMX)

[Ismael Juma (JIRA)]

[ 
https://issues.apache.org/jira/browse/KAFKA-2689?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14972824#comment-14972824
 ] 

Ismael Juma commented on KAFKA-2689:


It should be a val.

> Expose select gauges and metrics programmatically (not just through JMX)
> 
>
> Key: KAFKA-2689
> URL: https://issues.apache.org/jira/browse/KAFKA-2689
> Project: Kafka
>  Issue Type: Improvement
>  Components: core
>Affects Versions: 0.9.0.0
>Reporter: Eno Thereska
>Assignee: Eno Thereska
>  Labels: newbie
> Fix For: 0.9.0.0
>
>
> There are several gauges in core that are registered but cannot be accessed 
> programmatically. For example, gauges "LeaderCount", "PartitionCount", 
> "UnderReplicatedParittions" are all registered in ReplicaManager.scala but 
> there is no way to access them programmatically if one has access to the 
> kafka.server object. Other metrics,  such as isrExpandRate (also in 
> ReplicaManager.scala) can be accessed. The solution here is trivial, add a 
> var  in front of newGauge, as shown below
> var partitionCount newGauge(
> "PartitionCount",
> new Gauge[Int] {
>   def value = allPartitions.size
> }
> )



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (KAFKA-2689) Expose select gauges and metrics programmatically (not just through JMX)

[Eno Thereska (JIRA)]

[ 
https://issues.apache.org/jira/browse/KAFKA-2689?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14972836#comment-14972836
 ] 

Eno Thereska commented on KAFKA-2689:
-

I meant val, sorry typo. Thanks.

> Expose select gauges and metrics programmatically (not just through JMX)
> 
>
> Key: KAFKA-2689
> URL: https://issues.apache.org/jira/browse/KAFKA-2689
> Project: Kafka
>  Issue Type: Improvement
>  Components: core
>Affects Versions: 0.9.0.0
>Reporter: Eno Thereska
>Assignee: Eno Thereska
>  Labels: newbie
> Fix For: 0.9.0.0
>
>
> There are several gauges in core that are registered but cannot be accessed 
> programmatically. For example, gauges "LeaderCount", "PartitionCount", 
> "UnderReplicatedParittions" are all registered in ReplicaManager.scala but 
> there is no way to access them programmatically if one has access to the 
> kafka.server object. Other metrics,  such as isrExpandRate (also in 
> ReplicaManager.scala) can be accessed. The solution here is trivial, add a 
> var  in front of newGauge, as shown below
> var partitionCount newGauge(
> "PartitionCount",
> new Gauge[Int] {
>   def value = allPartitions.size
> }
> )



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (KAFKA-2689) Expose select gauges and metrics programmatically (not just through JMX)

[Eno Thereska (JIRA)]

Eno Thereska created KAFKA-2689:
---

 Summary: Expose select gauges and metrics programmatically (not 
just through JMX)
 Key: KAFKA-2689
 URL: https://issues.apache.org/jira/browse/KAFKA-2689
 Project: Kafka
  Issue Type: Improvement
  Components: core
Affects Versions: 0.9.0.0
Reporter: Eno Thereska
Assignee: Eno Thereska
 Fix For: 0.9.0.0


There are several gauges in core that are registered but cannot be accessed 
programmatically. For example, gauges "LeaderCount", "PartitionCount", 
"UnderReplicatedParittions" are all registered in ReplicaManager.scala but 
there is no way to access them programmatically if one has access to the 
kafka.server object. Other metrics,  such as isrExpandRate (also in 
ReplicaManager.scala) can be accessed. The solution here is trivial, add a var 
 in front of newGauge, as shown below
var partitionCount newGauge(
"PartitionCount",
new Gauge[Int] {
  def value = allPartitions.size
}
)




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (KAFKA-2645) Document potentially breaking changes in the release notes for 0.9.0

[Gwen Shapira (JIRA)]

 [ 
https://issues.apache.org/jira/browse/KAFKA-2645?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Gwen Shapira updated KAFKA-2645:

Priority: Blocker  (was: Major)

> Document potentially breaking changes in the release notes for 0.9.0
> 
>
> Key: KAFKA-2645
> URL: https://issues.apache.org/jira/browse/KAFKA-2645
> Project: Kafka
>  Issue Type: Improvement
>Reporter: Grant Henke
>Assignee: Grant Henke
>Priority: Blocker
> Fix For: 0.9.0.0
>
>




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Build failed in Jenkins: kafka_system_tests #118

[ewen]

See 

--
[...truncated 1671 lines...]


test_id:
2015-10-24--001.kafkatest.tests.benchmark_test.Benchmark.test_producer_throughput.topic=topic-replication-factor-three.num_producers=3.acks=1

status: PASS

run time:   1 minute 51.788 seconds

{"records_per_sec": 151533.177535, "mb_per_sec": 14.45}



test_id:
2015-10-24--001.kafkatest.tests.benchmark_test.Benchmark.test_producer_throughput.topic=topic-replication-factor-three.security_protocol=PLAINTEXT.acks=1.message_size=10

status: PASS

run time:   2 minutes 19.184 seconds

{"records_per_sec": 225932.936067, "mb_per_sec": 2.15}



test_id:
2015-10-24--001.kafkatest.tests.benchmark_test.Benchmark.test_producer_throughput.topic=topic-replication-factor-three.security_protocol=SSL.acks=1.message_size=10

status: PASS

run time:   3 minutes 6.232 seconds

{"records_per_sec": 134161.372224, "mb_per_sec": 1.28}



test_id:
2015-10-24--001.kafkatest.tests.benchmark_test.Benchmark.test_producer_throughput.topic=topic-replication-factor-three.security_protocol=PLAINTEXT.acks=1.message_size=100

status: PASS

run time:   1 minute 42.102 seconds

{"records_per_sec": 65500.805232, "mb_per_sec": 6.25}



test_id:
2015-10-24--001.kafkatest.tests.benchmark_test.Benchmark.test_producer_throughput.topic=topic-replication-factor-three.security_protocol=SSL.acks=1.message_size=100

status: PASS

run time:   1 minute 59.064 seconds

{"records_per_sec": 41133.220962, "mb_per_sec": 3.92}



test_id:
2015-10-24--001.kafkatest.tests.benchmark_test.Benchmark.test_producer_throughput.topic=topic-replication-factor-three.security_protocol=PLAINTEXT.acks=1.message_size=1000

status: PASS

run time:   1 minute 35.656 seconds

{"records_per_sec": 8417.497648, "mb_per_sec": 8.03}



test_id:
2015-10-24--001.kafkatest.tests.benchmark_test.Benchmark.test_producer_throughput.topic=topic-replication-factor-three.security_protocol=SSL.acks=1.message_size=1000

status: PASS

run time:   1 minute 53.173 seconds

{"records_per_sec": 5226.315175, "mb_per_sec": 4.98}



test_id:
2015-10-24--001.kafkatest.tests.benchmark_test.Benchmark.test_producer_throughput.topic=topic-replication-factor-three.security_protocol=PLAINTEXT.acks=1.message_size=1

status: PASS

run time:   1 minute 34.831 seconds

{"records_per_sec": 1010.465291, "mb_per_sec": 9.64}



test_id:
2015-10-24--001.kafkatest.tests.benchmark_test.Benchmark.test_producer_throughput.topic=topic-replication-factor-three.security_protocol=SSL.acks=1.message_size=1

status: PASS

run time:   1 minute 51.873 seconds

{"records_per_sec": 584.614714, "mb_per_sec": 5.58}



test_id:
2015-10-24--001.kafkatest.tests.benchmark_test.Benchmark.test_producer_throughput.topic=topic-replication-factor-three.security_protocol=PLAINTEXT.acks=1.message_size=10

status: PASS

run time:   1 minute 26.553 seconds

{"records_per_sec": 276.986584, "mb_per_sec": 26.42}



test_id:
2015-10-24--001.kafkatest.tests.benchmark_test.Benchmark.test_producer_throughput.topic=topic-replication-factor-three.security_protocol=SSL.acks=1.message_size=10

status: PASS

run time:   1 minute 45.867 seconds

{"records_per_sec": 88.950752, "mb_per_sec": 8.48}



test_id:
2015-10-24--001.kafkatest.sanity_checks.test_console_consumer.ConsoleConsumerTest.test_lifecycle.security_protocol=SSL.new_consumer=True

status: PASS

run time:   1 minute 0.396 seconds



test_id:
2015-10-24--001.kafkatest.sanity_checks.test_console_consumer.ConsoleConsumerTest.test_lifecycle.security_protocol=PLAINTEXT.new_consumer=False

status: PASS

run time:   55.964 seconds



test_id: