[jira] [Commented] (KAFKA-3728) EndToEndAuthorizationTest offsets_topic misconfigured

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/KAFKA-3728?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15314748#comment-15314748
 ] 

ASF GitHub Bot commented on KAFKA-3728:
---

Github user asfgit closed the pull request at:

https://github.com/apache/kafka/pull/1425


> EndToEndAuthorizationTest offsets_topic misconfigured
> -
>
> Key: KAFKA-3728
> URL: https://issues.apache.org/jira/browse/KAFKA-3728
> Project: Kafka
>  Issue Type: Bug
>Reporter: Edoardo Comar
>Assignee: Edoardo Comar
>
> A consumer that is manually assigned a topic-partition is able to consume 
> messages that a consumer that subscribes to the topic can not.
> To reproduce : take the test 
> EndToEndAuthorizationTest.testProduceConsume 
> (eg the SaslSslEndToEndAuthorizationTest implementation)
>  
> it passes ( = messages are consumed) 
> if the consumer is assigned the single topic-partition
>   consumers.head.assign(List(tp).asJava)
> but fails 
> if the consumer subscribes to the topic - changing the line to :
>   consumers.head.subscribe(List(topic).asJava)
> The failure when subscribed shows this error about synchronization:
>  org.apache.kafka.common.KafkaException: Unexpected error from SyncGroup: 
> Messages are rejected since there are fewer in-sync replicas than required.
>   at 
> org.apache.kafka.clients.consumer.internals.AbstractCoordinator$SyncGroupResponseHandler.handle(AbstractCoordinator.java:455)
> The test passes in both cases (subscribe and assign) with the setting
>   this.serverConfig.setProperty(KafkaConfig.MinInSyncReplicasProp, "1")



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (KAFKA-3728) EndToEndAuthorizationTest offsets_topic misconfigured

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/KAFKA-3728?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15298907#comment-15298907
 ] 

ASF GitHub Bot commented on KAFKA-3728:
---

GitHub user edoardocomar opened a pull request:

https://github.com/apache/kafka/pull/1425

KAFKA-3728 EndToEndAuthorizationTest offsets_topic misconfigured

Set OffsetsTopicReplicationFactorProp to 3 like MinInSyncReplicasProp
Earlier set acl for admin to read all
OffsetCommitRequiredAcksProp to 1 

Rationales:
Moving the set acl for admin avoided errors like :

ERROR [ReplicaFetcherThread-0-0], Error for partition
[__consumer_offsets,0] to broker
0:org.apache.kafka.common.errors.TopicAuthorizationException: Not
authorized to access topics: [Topic authorization failed.]
(kafka.server.ReplicaFetcherThread:97)

that were happening before the read to all was set

OffsetCommitRequiredAcksProp to 1 should not be needed, but without it,
testProduceConsumeViaSubscribe fails if it's not the first one to be
executed. Which suggests a cleanup problem, the test passes on its own.

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/edoardocomar/kafka KAFKA-3728

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/kafka/pull/1425.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #1425


commit 82a1d1bb0ce3065d7008f18f9d1bcd84bcb24d94
Author: Edoardo Comar 
Date:   2016-05-19T15:52:15Z

KAFKA-3728 EndToEndAuthorizationTest offsets_topic misconfigured

Set OffsetsTopicReplicationFactorProp to 3 like MinInSyncReplicasProp
Earlier set acl for admin to read all
OffsetCommitRequiredAcksProp to 1 

Rationales:
Moving the set acl for admin avoided errors like :

ERROR [ReplicaFetcherThread-0-0], Error for partition
[__consumer_offsets,0] to broker
0:org.apache.kafka.common.errors.TopicAuthorizationException: Not
authorized to access topics: [Topic authorization failed.]
(kafka.server.ReplicaFetcherThread:97)

that were happening before the read to all was set

OffsetCommitRequiredAcksProp to 1 should not be needed, but without it,
testProduceConsumeViaSubscribe fails if it's not the first one to be
executed. Which suggests a cleanup problem, the test passes on its own.




> EndToEndAuthorizationTest offsets_topic misconfigured
> -
>
> Key: KAFKA-3728
> URL: https://issues.apache.org/jira/browse/KAFKA-3728
> Project: Kafka
>  Issue Type: Bug
>Reporter: Edoardo Comar
>
> A consumer that is manually assigned a topic-partition is able to consume 
> messages that a consumer that subscribes to the topic can not.
> To reproduce : take the test 
> EndToEndAuthorizationTest.testProduceConsume 
> (eg the SaslSslEndToEndAuthorizationTest implementation)
>  
> it passes ( = messages are consumed) 
> if the consumer is assigned the single topic-partition
>   consumers.head.assign(List(tp).asJava)
> but fails 
> if the consumer subscribes to the topic - changing the line to :
>   consumers.head.subscribe(List(topic).asJava)
> The failure when subscribed shows this error about synchronization:
>  org.apache.kafka.common.KafkaException: Unexpected error from SyncGroup: 
> Messages are rejected since there are fewer in-sync replicas than required.
>   at 
> org.apache.kafka.clients.consumer.internals.AbstractCoordinator$SyncGroupResponseHandler.handle(AbstractCoordinator.java:455)
> The test passes in both cases (subscribe and assign) with the setting
>   this.serverConfig.setProperty(KafkaConfig.MinInSyncReplicasProp, "1")



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (KAFKA-3728) EndToEndAuthorizationTest offsets_topic misconfigured

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/KAFKA-3728?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15298724#comment-15298724
 ] 

ASF GitHub Bot commented on KAFKA-3728:
---

Github user edoardocomar closed the pull request at:

https://github.com/apache/kafka/pull/1414


> EndToEndAuthorizationTest offsets_topic misconfigured
> -
>
> Key: KAFKA-3728
> URL: https://issues.apache.org/jira/browse/KAFKA-3728
> Project: Kafka
>  Issue Type: Bug
>Reporter: Edoardo Comar
>
> A consumer that is manually assigned a topic-partition is able to consume 
> messages that a consumer that subscribes to the topic can not.
> To reproduce : take the test 
> EndToEndAuthorizationTest.testProduceConsume 
> (eg the SaslSslEndToEndAuthorizationTest implementation)
>  
> it passes ( = messages are consumed) 
> if the consumer is assigned the single topic-partition
>   consumers.head.assign(List(tp).asJava)
> but fails 
> if the consumer subscribes to the topic - changing the line to :
>   consumers.head.subscribe(List(topic).asJava)
> The failure when subscribed shows this error about synchronization:
>  org.apache.kafka.common.KafkaException: Unexpected error from SyncGroup: 
> Messages are rejected since there are fewer in-sync replicas than required.
>   at 
> org.apache.kafka.clients.consumer.internals.AbstractCoordinator$SyncGroupResponseHandler.handle(AbstractCoordinator.java:455)
> The test passes in both cases (subscribe and assign) with the setting
>   this.serverConfig.setProperty(KafkaConfig.MinInSyncReplicasProp, "1")



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (KAFKA-3728) EndToEndAuthorizationTest offsets_topic misconfigured

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/KAFKA-3728?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15298722#comment-15298722
 ] 

ASF GitHub Bot commented on KAFKA-3728:
---

Github user edoardocomar closed the pull request at:

https://github.com/apache/kafka/pull/1414


> EndToEndAuthorizationTest offsets_topic misconfigured
> -
>
> Key: KAFKA-3728
> URL: https://issues.apache.org/jira/browse/KAFKA-3728
> Project: Kafka
>  Issue Type: Bug
>Reporter: Edoardo Comar
>
> A consumer that is manually assigned a topic-partition is able to consume 
> messages that a consumer that subscribes to the topic can not.
> To reproduce : take the test 
> EndToEndAuthorizationTest.testProduceConsume 
> (eg the SaslSslEndToEndAuthorizationTest implementation)
>  
> it passes ( = messages are consumed) 
> if the consumer is assigned the single topic-partition
>   consumers.head.assign(List(tp).asJava)
> but fails 
> if the consumer subscribes to the topic - changing the line to :
>   consumers.head.subscribe(List(topic).asJava)
> The failure when subscribed shows this error about synchronization:
>  org.apache.kafka.common.KafkaException: Unexpected error from SyncGroup: 
> Messages are rejected since there are fewer in-sync replicas than required.
>   at 
> org.apache.kafka.clients.consumer.internals.AbstractCoordinator$SyncGroupResponseHandler.handle(AbstractCoordinator.java:455)
> The test passes in both cases (subscribe and assign) with the setting
>   this.serverConfig.setProperty(KafkaConfig.MinInSyncReplicasProp, "1")



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (KAFKA-3728) EndToEndAuthorizationTest offsets_topic misconfigured

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/KAFKA-3728?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15298723#comment-15298723
 ] 

ASF GitHub Bot commented on KAFKA-3728:
---

GitHub user edoardocomar reopened a pull request:

https://github.com/apache/kafka/pull/1414

KAFKA-3728 EndToEndAuthorizationTest offsets_topic misconfigured

Set OffsetsTopicReplicationFactorProp to 3 like MinInSyncReplicasProp
and OffsetCommitRequiredAcksProp to 1 to avoid timeouts

unit test for consumer that subscribes added 

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/edoardocomar/kafka KAFKA-3728

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/kafka/pull/1414.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #1414


commit 82a1d1bb0ce3065d7008f18f9d1bcd84bcb24d94
Author: Edoardo Comar 
Date:   2016-05-19T15:52:15Z

KAFKA-3728 EndToEndAuthorizationTest offsets_topic misconfigured

Set OffsetsTopicReplicationFactorProp to 3 like MinInSyncReplicasProp
Earlier set acl for admin to read all
OffsetCommitRequiredAcksProp to 1 

Rationales:
Moving the set acl for admin avoided errors like :

ERROR [ReplicaFetcherThread-0-0], Error for partition
[__consumer_offsets,0] to broker
0:org.apache.kafka.common.errors.TopicAuthorizationException: Not
authorized to access topics: [Topic authorization failed.]
(kafka.server.ReplicaFetcherThread:97)

that were happening before the read to all was set

OffsetCommitRequiredAcksProp to 1 should not be needed, but without it,
testProduceConsumeViaSubscribe fails if it's not the first one to be
executed. Which suggests a cleanup problem, the test passes on its own.




> EndToEndAuthorizationTest offsets_topic misconfigured
> -
>
> Key: KAFKA-3728
> URL: https://issues.apache.org/jira/browse/KAFKA-3728
> Project: Kafka
>  Issue Type: Bug
>Reporter: Edoardo Comar
>
> A consumer that is manually assigned a topic-partition is able to consume 
> messages that a consumer that subscribes to the topic can not.
> To reproduce : take the test 
> EndToEndAuthorizationTest.testProduceConsume 
> (eg the SaslSslEndToEndAuthorizationTest implementation)
>  
> it passes ( = messages are consumed) 
> if the consumer is assigned the single topic-partition
>   consumers.head.assign(List(tp).asJava)
> but fails 
> if the consumer subscribes to the topic - changing the line to :
>   consumers.head.subscribe(List(topic).asJava)
> The failure when subscribed shows this error about synchronization:
>  org.apache.kafka.common.KafkaException: Unexpected error from SyncGroup: 
> Messages are rejected since there are fewer in-sync replicas than required.
>   at 
> org.apache.kafka.clients.consumer.internals.AbstractCoordinator$SyncGroupResponseHandler.handle(AbstractCoordinator.java:455)
> The test passes in both cases (subscribe and assign) with the setting
>   this.serverConfig.setProperty(KafkaConfig.MinInSyncReplicasProp, "1")



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (KAFKA-3728) EndToEndAuthorizationTest offsets_topic misconfigured

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/KAFKA-3728?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15293278#comment-15293278
 ] 

ASF GitHub Bot commented on KAFKA-3728:
---

GitHub user edoardocomar opened a pull request:

https://github.com/apache/kafka/pull/1414

KAFKA-3728 EndToEndAuthorizationTest offsets_topic misconfigured

Set OffsetsTopicReplicationFactorProp to 3 like MinInSyncReplicasProp
and OffsetCommitRequiredAcksProp to 1 to avoid timeouts

unit test for consumer that subscribes added 

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/edoardocomar/kafka KAFKA-3728

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/kafka/pull/1414.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #1414


commit eab6d3f7aa1facc873332228fa32f35e98c02b2d
Author: Edoardo Comar 
Date:   2016-05-19T15:52:15Z

KAFKA-3728 EndToEndAuthorizationTest offsets_topic misconfigured

Set OffsetsTopicReplicationFactorProp to 3 like MinInSyncReplicasProp
and OffsetCommitRequiredAcksProp to 1 to avoid timeouts




> EndToEndAuthorizationTest offsets_topic misconfigured
> -
>
> Key: KAFKA-3728
> URL: https://issues.apache.org/jira/browse/KAFKA-3728
> Project: Kafka
>  Issue Type: Bug
>Reporter: Edoardo Comar
>
> A consumer that is manually assigned a topic-partition is able to consume 
> messages that a consumer that subscribes to the topic can not.
> To reproduce : take the test 
> EndToEndAuthorizationTest.testProduceConsume 
> (eg the SaslSslEndToEndAuthorizationTest implementation)
>  
> it passes ( = messages are consumed) 
> if the consumer is assigned the single topic-partition
>   consumers.head.assign(List(tp).asJava)
> but fails 
> if the consumer subscribes to the topic - changing the line to :
>   consumers.head.subscribe(List(topic).asJava)
> The failure when subscribed shows this error about synchronization:
>  org.apache.kafka.common.KafkaException: Unexpected error from SyncGroup: 
> Messages are rejected since there are fewer in-sync replicas than required.
>   at 
> org.apache.kafka.clients.consumer.internals.AbstractCoordinator$SyncGroupResponseHandler.handle(AbstractCoordinator.java:455)
> The test passes in both cases (subscribe and assign) with the setting
>   this.serverConfig.setProperty(KafkaConfig.MinInSyncReplicasProp, "1")



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (KAFKA-3728) EndToEndAuthorizationTest offsets_topic misconfigured

[Edoardo Comar (JIRA)]

[ 
https://issues.apache.org/jira/browse/KAFKA-3728?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15293272#comment-15293272
 ] 

Edoardo Comar commented on KAFKA-3728:
--

Thanks [~rsivaram] 
it wasn't obvious that ack=-1 requires a number of replicas equal to 
min.insync.replicas, even when the topic has fewer replicas than the ISR.

Possibly due to timeouts, even setting OffsetsTopicReplicationFactorProp to  
min.insync.replicas (=3) doesn't make the test pass reliably.
so I also set the OffsetCommitRequiredAcksProp to 1.

PR coming

> EndToEndAuthorizationTest offsets_topic misconfigured
> -
>
> Key: KAFKA-3728
> URL: https://issues.apache.org/jira/browse/KAFKA-3728
> Project: Kafka
>  Issue Type: Bug
>Reporter: Edoardo Comar
>
> A consumer that is manually assigned a topic-partition is able to consume 
> messages that a consumer that subscribes to the topic can not.
> To reproduce : take the test 
> EndToEndAuthorizationTest.testProduceConsume 
> (eg the SaslSslEndToEndAuthorizationTest implementation)
>  
> it passes ( = messages are consumed) 
> if the consumer is assigned the single topic-partition
>   consumers.head.assign(List(tp).asJava)
> but fails 
> if the consumer subscribes to the topic - changing the line to :
>   consumers.head.subscribe(List(topic).asJava)
> The failure when subscribed shows this error about synchronization:
>  org.apache.kafka.common.KafkaException: Unexpected error from SyncGroup: 
> Messages are rejected since there are fewer in-sync replicas than required.
>   at 
> org.apache.kafka.clients.consumer.internals.AbstractCoordinator$SyncGroupResponseHandler.handle(AbstractCoordinator.java:455)
> The test passes in both cases (subscribe and assign) with the setting
>   this.serverConfig.setProperty(KafkaConfig.MinInSyncReplicasProp, "1")



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)