[jira] [Commented] (KAFKA-2700) delete topic should remove the corresponding ACL and configs
[ https://issues.apache.org/jira/browse/KAFKA-2700?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15493958#comment-15493958 ] Parth Brahmbhatt commented on KAFKA-2700: - all yours. > delete topic should remove the corresponding ACL and configs > > > Key: KAFKA-2700 > URL: https://issues.apache.org/jira/browse/KAFKA-2700 > Project: Kafka > Issue Type: Bug > Components: core >Affects Versions: 0.9.0.0 >Reporter: Jun Rao >Assignee: Parth Brahmbhatt > > After a topic is successfully deleted, we should also remove any ACL, configs > and perhaps committed offsets associated with topic. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (KAFKA-3294) Kafka REST API
[ https://issues.apache.org/jira/browse/KAFKA-3294?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15261562#comment-15261562 ] Parth Brahmbhatt commented on KAFKA-3294: - Here is a version we are working on https://github.com/Parth-Brahmbhatt/kafka-rest. Its still work in progress but Its better to start the discussion around getting more clarity on requirements, how we want to package and distribute it and what does the community need from this feature. > Kafka REST API > -- > > Key: KAFKA-3294 > URL: https://issues.apache.org/jira/browse/KAFKA-3294 > Project: Kafka > Issue Type: New Feature >Reporter: Sriharsha Chintalapani >Assignee: Parth Brahmbhatt > > This JIRA is to build Kafka REST API for producer, consumer and also any > administrative tasks such as create topic, delete topic. We do have lot of > kafka client api support in different languages but having REST API for > producer and consumer will make it easier for users to read or write Kafka. > Also having administrative API will help in managing a cluster or building > administrative dashboards. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (KAFKA-3291) DumpLogSegment tool should also provide an option to only verify index sanity.
Parth Brahmbhatt created KAFKA-3291: --- Summary: DumpLogSegment tool should also provide an option to only verify index sanity. Key: KAFKA-3291 URL: https://issues.apache.org/jira/browse/KAFKA-3291 Project: Kafka Issue Type: Improvement Components: tools Reporter: Parth Brahmbhatt Assignee: Parth Brahmbhatt DumpLogSegment tool should call index.sanityCheck function as part of index sanity check as that function determines if an index will be rebuilt on restart or not. This is a cheap check as it only checks the file size and can help in scenarios where customer is trying to figure out which index files will be rebuilt on startup which directly affects the broker bootstrap time. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (KAFKA-1696) Kafka should be able to generate Hadoop delegation tokens
[ https://issues.apache.org/jira/browse/KAFKA-1696?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15168010#comment-15168010 ] Parth Brahmbhatt commented on KAFKA-1696: - [~gwenshap] I think that discussion is happening as part of another KIP and no matter what we chose I don't think it affects the delegation token design. > Kafka should be able to generate Hadoop delegation tokens > - > > Key: KAFKA-1696 > URL: https://issues.apache.org/jira/browse/KAFKA-1696 > Project: Kafka > Issue Type: Sub-task > Components: security >Reporter: Jay Kreps >Assignee: Parth Brahmbhatt > > For access from MapReduce/etc jobs run on behalf of a user. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (KAFKA-1696) Kafka should be able to generate Hadoop delegation tokens
[ https://issues.apache.org/jira/browse/KAFKA-1696?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15167928#comment-15167928 ] Parth Brahmbhatt commented on KAFKA-1696: - [~gwenshap] [~sriharsha] Please share your preference here so I can update the KIP accordingly and start a discussion thread. I wanted to try SASL over MD5 to ensure that this design will work as is but I don't see myself writing that prototype soon so might as well start the KIP discussion. > Kafka should be able to generate Hadoop delegation tokens > - > > Key: KAFKA-1696 > URL: https://issues.apache.org/jira/browse/KAFKA-1696 > Project: Kafka > Issue Type: Sub-task > Components: security >Reporter: Jay Kreps >Assignee: Parth Brahmbhatt > > For access from MapReduce/etc jobs run on behalf of a user. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Comment Edited] (KAFKA-1696) Kafka should be able to generate Hadoop delegation tokens
[ https://issues.apache.org/jira/browse/KAFKA-1696?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15167800#comment-15167800 ] Parth Brahmbhatt edited comment on KAFKA-1696 at 2/25/16 8:23 PM: -- So here is how that request path would work in my mind: * Client sends request for token acquisition to any broker. * Broker forwards the request to the controller. * Controller generates the token and pushes the tokens to all brokers. (Will need a new API) * Controller responds back to original broker with the token. * Broker responds back to client with the token. Renewal is pretty much the same. The race condition you are describing can still happen in the above case during renewal because controller may have pushed the renewal information to a subset of broker and die. The clients depending on which broker it connects to may get an exception or success. I do agree though that given controller would not have responded back with success the original renew request should be retried and most likely the scenario can be avoided. If the above steps seems right , here are the advantages of this approach: Advantage: * Token generation/renewal will not involve zookeeper. I am not too worried about the load on zookeeper added due to this but it definitely seems more secure and follows the Hadoop model more closely. However zookeeper needs to be secure for lot of other things in kafka so not sure if this should really be a concern. * Clients will get better consistency. Disadvantage: * We will have to add new APIs to support controller pushing tokens to brokers on top of the minimal APIs that are currently proposed. I like the publicly available APIs to be minimal and I like them to be something that we expect clients to use + this adds more development complexity. Overall this seems like a more philosophical thing so depending on who you ask they may see this as disadvantage or not. * We will also have to add APIs to support the bootstrapping case. What I mean is , when a new broker comes up it will have to get all delegation tokens from the controller so we will again need to add new APIs like getAllTokens. Again some of us may see that as disadvantage and some may not. * In catastrophic failures where all brokers go down, the tokens will be lost even if servers are restarted as tokens are not persisted anywhere. Granted if something like this happens customer has bigger things to worry about but if they don't have to regenerate/redistribute tokens that is one less thing. I don't see strong reasons to go one way or another so I would still like to go with zookeeper but don't really feel strongly about it. If you think I have mischaracterized what you were proposing feel free to add more details or list and other advantages/disadvantages. was (Author: parth.brahmbhatt): So here is how that request path would work in my mind: * Client sends request for token acquisition to any broker. * Broker forwards the request to the controller. * Controller generates the token and pushes the tokens to all brokers. (Will need a new API) * Controller responds back to original broker with the token. * Broker responds back to client with the token. Renewal is pretty much the same. The race condition you are describing can still happen in the above case during renewal because controller may have pushed the renewal information to a subset of broker and die. The clients depending on which broker it connects to may get an exception or success. I do agree though that given controller would not have responded back with success the original renew request should be retried and most likely the scenario can be avoided. If the above steps seems right , here are the advantages of this approach: Advantage: * Token generation/renewal will not involve zookeeper. I am not too worried about the load on zookeeper added due to this but it definitely seems more secure and follows the Hadoop model more closely. However zookeeper needs to be secure for lot of other things in kafka so not sure if this should really be a concern. Disadvantage: * We will have to add new APIs to support controller pushing tokens to brokers on top of the minimal APIs that are currently proposed. I like the publicly available APIs to be minimal and I like them to be something that we expect clients to use + this adds more development complexity. Overall this seems like a more philosophical thing so depending on who you ask they may see this as disadvantage or not. * We will also have to add APIs to support the bootstrapping case. What I mean is , when a new broker comes up it will have to get all delegation tokens from the controller so we will again need to add new APIs like getAllTokens. Again some of us may see that as disadvantage and some may not. * In catastrophic failures where all brokers go down, the tokens will be
[jira] [Commented] (KAFKA-1696) Kafka should be able to generate Hadoop delegation tokens
[ https://issues.apache.org/jira/browse/KAFKA-1696?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15167800#comment-15167800 ] Parth Brahmbhatt commented on KAFKA-1696: - So here is how that request path would work in my mind: * Client sends request for token acquisition to any broker. * Broker forwards the request to the controller. * Controller generates the token and pushes the tokens to all brokers. (Will need a new API) * Controller responds back to original broker with the token. * Broker responds back to client with the token. Renewal is pretty much the same. The race condition you are describing can still happen in the above case during renewal because controller may have pushed the renewal information to a subset of broker and die. The clients depending on which broker it connects to may get an exception or success. I do agree though that given controller would not have responded back with success the original renew request should be retried and most likely the scenario can be avoided. If the above steps seems right , here are the advantages of this approach: Advantage: * Token generation/renewal will not involve zookeeper. I am not too worried about the load on zookeeper added due to this but it definitely seems more secure and follows the Hadoop model more closely. However zookeeper needs to be secure for lot of other things in kafka so not sure if this should really be a concern. Disadvantage: * We will have to add new APIs to support controller pushing tokens to brokers on top of the minimal APIs that are currently proposed. I like the publicly available APIs to be minimal and I like them to be something that we expect clients to use + this adds more development complexity. Overall this seems like a more philosophical thing so depending on who you ask they may see this as disadvantage or not. * We will also have to add APIs to support the bootstrapping case. What I mean is , when a new broker comes up it will have to get all delegation tokens from the controller so we will again need to add new APIs like getAllTokens. Again some of us may see that as disadvantage and some may not. * In catastrophic failures where all brokers go down, the tokens will be lost even if servers are restarted as tokens are not persisted anywhere. Granted if something like this happens customer has bigger things to worry about but if they don't have to regenerate/redistribute tokens that is one less thing. I don't see strong reasons to go one way or another so I would still like to go with zookeeper but don't really feel strongly about it. If you think I have mischaracterized what you were proposing feel free to add more details or list and other advantages/disadvantages. > Kafka should be able to generate Hadoop delegation tokens > - > > Key: KAFKA-1696 > URL: https://issues.apache.org/jira/browse/KAFKA-1696 > Project: Kafka > Issue Type: Sub-task > Components: security >Reporter: Jay Kreps >Assignee: Parth Brahmbhatt > > For access from MapReduce/etc jobs run on behalf of a user. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (KAFKA-1696) Kafka should be able to generate Hadoop delegation tokens
[ https://issues.apache.org/jira/browse/KAFKA-1696?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15167696#comment-15167696 ] Parth Brahmbhatt commented on KAFKA-1696: - [~singhashish] Thanks for taking the time to review. I will add the errorcode/exception to the KIP and sample CLI. For point 2, can you elaborate why it might be a better idea to communicate via coordinator? When I was thinking about distributing the tokens I felt the coordinator will just add extra complexity and wont buy us much. We already have multiple things ACLs/Configs for which we rely on zookeeper watchers so it seemed like a natural choice. > Kafka should be able to generate Hadoop delegation tokens > - > > Key: KAFKA-1696 > URL: https://issues.apache.org/jira/browse/KAFKA-1696 > Project: Kafka > Issue Type: Sub-task > Components: security >Reporter: Jay Kreps >Assignee: Parth Brahmbhatt > > For access from MapReduce/etc jobs run on behalf of a user. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (KAFKA-1696) Kafka should be able to generate Hadoop delegation tokens
[ https://issues.apache.org/jira/browse/KAFKA-1696?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15149659#comment-15149659 ] Parth Brahmbhatt commented on KAFKA-1696: - [~gwenshap] [~harsha_ch] [~singhashish] I posted an initial KIP Draft https://cwiki.apache.org/confluence/display/KAFKA/KIP-48+Delegation+token+support+for+Kafka. I haven't yet opened a Discuss thread as I need to verify some assumptions I have made. > Kafka should be able to generate Hadoop delegation tokens > - > > Key: KAFKA-1696 > URL: https://issues.apache.org/jira/browse/KAFKA-1696 > Project: Kafka > Issue Type: Sub-task > Components: security >Reporter: Jay Kreps >Assignee: Parth Brahmbhatt > > For access from MapReduce/etc jobs run on behalf of a user. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (KAFKA-1696) Kafka should be able to generate Hadoop delegation tokens
[ https://issues.apache.org/jira/browse/KAFKA-1696?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15113295#comment-15113295 ] Parth Brahmbhatt commented on KAFKA-1696: - I will assign it to my self and file a KIP > Kafka should be able to generate Hadoop delegation tokens > - > > Key: KAFKA-1696 > URL: https://issues.apache.org/jira/browse/KAFKA-1696 > Project: Kafka > Issue Type: Sub-task > Components: security >Reporter: Jay Kreps >Assignee: Sriharsha Chintalapani > > For access from MapReduce/etc jobs run on behalf of a user. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Assigned] (KAFKA-1696) Kafka should be able to generate Hadoop delegation tokens
[ https://issues.apache.org/jira/browse/KAFKA-1696?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Parth Brahmbhatt reassigned KAFKA-1696: --- Assignee: Parth Brahmbhatt (was: Sriharsha Chintalapani) > Kafka should be able to generate Hadoop delegation tokens > - > > Key: KAFKA-1696 > URL: https://issues.apache.org/jira/browse/KAFKA-1696 > Project: Kafka > Issue Type: Sub-task > Components: security >Reporter: Jay Kreps >Assignee: Parth Brahmbhatt > > For access from MapReduce/etc jobs run on behalf of a user. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (KAFKA-1696) Kafka should be able to generate Hadoop delegation tokens
[ https://issues.apache.org/jira/browse/KAFKA-1696?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15106023#comment-15106023 ] Parth Brahmbhatt commented on KAFKA-1696: - [~gwenshap] Are you still working on this? We have some customers that needs this feature and if you have not started the design work I would like to take this over. > Kafka should be able to generate Hadoop delegation tokens > - > > Key: KAFKA-1696 > URL: https://issues.apache.org/jira/browse/KAFKA-1696 > Project: Kafka > Issue Type: Sub-task > Components: security >Reporter: Jay Kreps >Assignee: Sriharsha Chintalapani > > For access from MapReduce/etc jobs run on behalf of a user. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (KAFKA-2000) Delete consumer offsets from kafka once the topic is deleted
[ https://issues.apache.org/jira/browse/KAFKA-2000?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15064564#comment-15064564 ] Parth Brahmbhatt commented on KAFKA-2000: - Talked to [~harsha_ch] , I will submit an updated patch for this one today. > Delete consumer offsets from kafka once the topic is deleted > > > Key: KAFKA-2000 > URL: https://issues.apache.org/jira/browse/KAFKA-2000 > Project: Kafka > Issue Type: Bug >Reporter: Sriharsha Chintalapani >Assignee: Sriharsha Chintalapani > Labels: newbie++ > Fix For: 0.9.1.0 > > Attachments: KAFKA-2000.patch, KAFKA-2000_2015-05-03_10:39:11.patch > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Work started] (KAFKA-2547) Make DynamicConfigManager to use the ZkNodeChangeNotificationListener introduced as part of KAFKA-2211
[ https://issues.apache.org/jira/browse/KAFKA-2547?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on KAFKA-2547 started by Parth Brahmbhatt. --- > Make DynamicConfigManager to use the ZkNodeChangeNotificationListener > introduced as part of KAFKA-2211 > -- > > Key: KAFKA-2547 > URL: https://issues.apache.org/jira/browse/KAFKA-2547 > Project: Kafka > Issue Type: Improvement >Reporter: Parth Brahmbhatt >Assignee: Parth Brahmbhatt > > As part of KAFKA-2211 (https://github.com/apache/kafka/pull/195/files) we > introduced a reusable ZkNodeChangeNotificationListener to ensure node changes > can be processed in a loss less way. This was pretty much the same code in > DynamicConfigManager with little bit of refactoring so it can be reused. We > now need to make DynamicConfigManager itself to use this new class once > KAFKA-2211 is committed to avoid code duplication. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (KAFKA-2794) Add group support for authorizer acls
[ https://issues.apache.org/jira/browse/KAFKA-2794?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15056934#comment-15056934 ] Parth Brahmbhatt commented on KAFKA-2794: - https://github.com/apache/kafka/pull/483 > Add group support for authorizer acls > - > > Key: KAFKA-2794 > URL: https://issues.apache.org/jira/browse/KAFKA-2794 > Project: Kafka > Issue Type: Sub-task > Components: security >Affects Versions: 0.9.0.0 >Reporter: Parth Brahmbhatt >Assignee: Parth Brahmbhatt > > Currently out of box kafka authorizer and ACLs only support single > principal/users. This is kind of a two part jira where we add support to > convert kerberos principal to unix user names and then from unix user name to > group. > With this feature , authorizer will be able to support Group level acls so > admins wont have to add support for individual users. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (KAFKA-2854) Make KerberosName implement PrincipalToLocal plugin so authorizer and authenticator can share this.
[ https://issues.apache.org/jira/browse/KAFKA-2854?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15056930#comment-15056930 ] Parth Brahmbhatt commented on KAFKA-2854: - [~ijuma][~harsha_ch] Can you review https://github.com/apache/kafka/pull/547 > Make KerberosName implement PrincipalToLocal plugin so authorizer and > authenticator can share this. > --- > > Key: KAFKA-2854 > URL: https://issues.apache.org/jira/browse/KAFKA-2854 > Project: Kafka > Issue Type: Improvement >Affects Versions: 0.9.0.0 >Reporter: Parth Brahmbhatt >Assignee: Parth Brahmbhatt > > Both Authorizer and SASL Authenticator needs a way to convert kerberos > principal into a local identity. This jira proposes to add an interface and a > config for this conversion so users can inject their own implementation. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Work started] (KAFKA-2854) Make KerberosName implement PrincipalToLocal plugin so authorizer and authenticator can share this.
[ https://issues.apache.org/jira/browse/KAFKA-2854?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on KAFKA-2854 started by Parth Brahmbhatt. --- > Make KerberosName implement PrincipalToLocal plugin so authorizer and > authenticator can share this. > --- > > Key: KAFKA-2854 > URL: https://issues.apache.org/jira/browse/KAFKA-2854 > Project: Kafka > Issue Type: Improvement >Affects Versions: 0.9.0.0 >Reporter: Parth Brahmbhatt >Assignee: Parth Brahmbhatt > > Both Authorizer and SASL Authenticator needs a way to convert kerberos > principal into a local identity. This jira proposes to add an interface and a > config for this conversion so users can inject their own implementation. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Work started] (KAFKA-2794) Add group support for authorizer acls
[ https://issues.apache.org/jira/browse/KAFKA-2794?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on KAFKA-2794 started by Parth Brahmbhatt. --- > Add group support for authorizer acls > - > > Key: KAFKA-2794 > URL: https://issues.apache.org/jira/browse/KAFKA-2794 > Project: Kafka > Issue Type: Sub-task > Components: security >Affects Versions: 0.9.0.0 >Reporter: Parth Brahmbhatt >Assignee: Parth Brahmbhatt > > Currently out of box kafka authorizer and ACLs only support single > principal/users. This is kind of a two part jira where we add support to > convert kerberos principal to unix user names and then from unix user name to > group. > With this feature , authorizer will be able to support Group level acls so > admins wont have to add support for individual users. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (KAFKA-2852) Kafka Authroizer CLI should use consistent way to specify multiple values for all config options.
Parth Brahmbhatt created KAFKA-2852: --- Summary: Kafka Authroizer CLI should use consistent way to specify multiple values for all config options. Key: KAFKA-2852 URL: https://issues.apache.org/jira/browse/KAFKA-2852 Project: Kafka Issue Type: Bug Reporter: Parth Brahmbhatt Currently, AclCommand uses inconsistent conventions. For principals, we take --allow-principal User:Bob --allow-principal User:Alice For other options like hosts, we take --allow-hosts Host1,Host2 We should have a consistent way to specify this. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (KAFKA-2854) Make KerberosName implement PrincipalToLocal plugin so authorizer and authenticator can share this.
Parth Brahmbhatt created KAFKA-2854: --- Summary: Make KerberosName implement PrincipalToLocal plugin so authorizer and authenticator can share this. Key: KAFKA-2854 URL: https://issues.apache.org/jira/browse/KAFKA-2854 Project: Kafka Issue Type: Improvement Affects Versions: 0.9.0.0 Reporter: Parth Brahmbhatt Assignee: Parth Brahmbhatt Fix For: 0.9.0.0 Both Authorizer and SASL Authenticator needs a way to convert kerberos principal into a local identity. This jira proposes to add an interface and a config for this conversion so users can inject their own implementation. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (KAFKA-2808) Support auto.create.topics.enable with automatic WRITE permissions for creator
[ https://issues.apache.org/jira/browse/KAFKA-2808?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15000985#comment-15000985 ] Parth Brahmbhatt commented on KAFKA-2808: - [~tgraves] your last comment is what I had in mind. Basically any time a topic is created , using CLI, AdminUtils or through auto create, in secure mode we should be able to derive the identity of the user who is creating the topic (from JAAS Login or if creation is through auto create using the caller's session on server side) and assign him as the owner. Namespace can solve the problem and I believe https://cwiki.apache.org/confluence/display/KAFKA/KIP-37+-+Add+Namespaces+to+Kafka is addressing it. I am assuming by WildCardTopics you mean something that supports regex, which wont be that different from namespacing it self. > Support auto.create.topics.enable with automatic WRITE permissions for > creator > --- > > Key: KAFKA-2808 > URL: https://issues.apache.org/jira/browse/KAFKA-2808 > Project: Kafka > Issue Type: Improvement > Components: core >Affects Versions: 0.9.0.0 >Reporter: Thomas Graves > > we have a user that wants to use the topic auto create functionality and > automatically have it give WRITE permissions so that they don't have to > explicitly create and grant acls ahead of time or make explicit call. > it seems like if you have auto.create.topics.enable enabled and the user has > CREATE acls we could automatically just give WRITE acls to the user who > creates the topic. Without that the auto create topics with acls doesn't add > much benefit. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (KAFKA-2808) Support auto.create.topics.enable with automatic WRITE permissions for creator
[ https://issues.apache.org/jira/browse/KAFKA-2808?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15000865#comment-15000865 ] Parth Brahmbhatt commented on KAFKA-2808: - Ideally we would always give admin access to a topic to the creator of the topic, problem is kafka does not have the concept of topic owner. I filed a jira to introduce the concept of topic owner https://issues.apache.org/jira/browse/KAFKA-2145 which should set the ground to support your request. However the PR is kind of stuck in review for more than 3 months now so I don't expect this request to be fulfilled in the upcoming release. > Support auto.create.topics.enable with automatic WRITE permissions for > creator > --- > > Key: KAFKA-2808 > URL: https://issues.apache.org/jira/browse/KAFKA-2808 > Project: Kafka > Issue Type: Improvement > Components: core >Affects Versions: 0.9.0.0 >Reporter: Thomas Graves > > we have a user that wants to use the topic auto create functionality and > automatically have it give WRITE permissions so that they don't have to > explicitly create and grant acls ahead of time or make explicit call. > it seems like if you have auto.create.topics.enable enabled and the user has > CREATE acls we could automatically just give WRITE acls to the user who > creates the topic. Without that the auto create topics with acls doesn't add > much benefit. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (KAFKA-2788) allow comma when specifying principals in AclCommand
[ https://issues.apache.org/jira/browse/KAFKA-2788?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14998905#comment-14998905 ] Parth Brahmbhatt commented on KAFKA-2788: - I have started working on it, will send a PR sometime today. > allow comma when specifying principals in AclCommand > > > Key: KAFKA-2788 > URL: https://issues.apache.org/jira/browse/KAFKA-2788 > Project: Kafka > Issue Type: Bug > Components: core >Affects Versions: 0.9.0.0 >Reporter: Jun Rao >Assignee: Parth Brahmbhatt >Priority: Blocker > Fix For: 0.9.0.0 > > > Currently, comma doesn't seem to be allowed in AclCommand when specifying > allow-principals and deny-principals. However, when using ssl authentication, > by default, the client will look like the following and one can't pass that > in through AclCommand. > "CN=writeuser,OU=Unknown,O=Unknown,L=Unknown,ST=Unknown,C=Unknown" -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (KAFKA-2788) allow comma when specifying principals in AclCommand
[ https://issues.apache.org/jira/browse/KAFKA-2788?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14998035#comment-14998035 ] Parth Brahmbhatt commented on KAFKA-2788: - Looks like we already have a valid use case. I will add the support for it which will mean we will either have to change the separator(My Preferred way but inconsistent with all the other CLIs) or just stop supporting more than one principal in CLI(Users will probably hate this given they will have to script it up). > allow comma when specifying principals in AclCommand > > > Key: KAFKA-2788 > URL: https://issues.apache.org/jira/browse/KAFKA-2788 > Project: Kafka > Issue Type: Bug > Components: core >Affects Versions: 0.9.0.0 >Reporter: Jun Rao >Assignee: Parth Brahmbhatt > Fix For: 0.9.0.0 > > > Currently, comma doesn't seem to be allowed in AclCommand when specifying > allow-principals and deny-principals. However, when using ssl authentication, > by default, the client will look like the following and one can't pass that > in through AclCommand. > "CN=writeuser,OU=Unknown,O=Unknown,L=Unknown,ST=Unknown,C=Unknown" -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (KAFKA-2794) Add group support for authorizer acls
Parth Brahmbhatt created KAFKA-2794: --- Summary: Add group support for authorizer acls Key: KAFKA-2794 URL: https://issues.apache.org/jira/browse/KAFKA-2794 Project: Kafka Issue Type: Sub-task Components: security Affects Versions: 0.9.0.0 Reporter: Parth Brahmbhatt Assignee: Parth Brahmbhatt Currently out of box kafka authorizer and ACLs only support single principal/users. This is kind of a two part jira where we add support to convert kerberos principal to unix user names and then from unix user name to group. With this feature , authorizer will be able to support Group level acls so admins wont have to add support for individual users. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Assigned] (KAFKA-2754) Support defining ACLs at topic creation time
[ https://issues.apache.org/jira/browse/KAFKA-2754?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Parth Brahmbhatt reassigned KAFKA-2754: --- Assignee: Parth Brahmbhatt > Support defining ACLs at topic creation time > > > Key: KAFKA-2754 > URL: https://issues.apache.org/jira/browse/KAFKA-2754 > Project: Kafka > Issue Type: Improvement >Affects Versions: 0.9.0.0 >Reporter: Thomas Graves >Assignee: Parth Brahmbhatt > > With a secured kafka cluster we want to be able have certain users create > topics and set the acls at the same time. We have a use case where topics > will be dynamically created and we don't know the names ahead of time so it > would be really nice to create the topic and set the acls at the same time. > We want to be able to do this programmatically. > Ideally it would be nice to have a way to set the acls with the auto create > topics enabled also but that might be a separate jira. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Assigned] (KAFKA-2700) delete topic should remove the corresponding ACL and configs
[ https://issues.apache.org/jira/browse/KAFKA-2700?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Parth Brahmbhatt reassigned KAFKA-2700: --- Assignee: Parth Brahmbhatt > delete topic should remove the corresponding ACL and configs > > > Key: KAFKA-2700 > URL: https://issues.apache.org/jira/browse/KAFKA-2700 > Project: Kafka > Issue Type: Bug > Components: core >Affects Versions: 0.9.0.0 >Reporter: Jun Rao >Assignee: Parth Brahmbhatt > > After a topic is successfully deleted, we should also remove any ACL, configs > and perhaps committed offsets associated with topic. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Assigned] (KAFKA-2682) Authorization section in official docs
[ https://issues.apache.org/jira/browse/KAFKA-2682?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Parth Brahmbhatt reassigned KAFKA-2682: --- Assignee: Parth Brahmbhatt > Authorization section in official docs > -- > > Key: KAFKA-2682 > URL: https://issues.apache.org/jira/browse/KAFKA-2682 > Project: Kafka > Issue Type: Sub-task > Components: security >Reporter: Ismael Juma >Assignee: Parth Brahmbhatt > Fix For: 0.9.0.0 > > > We need to add a section in the official documentation regarding > authorization: > http://kafka.apache.org/documentation.html -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (KAFKA-2212) KafkaAuthorizer: Add CLI for Acl management.
[ https://issues.apache.org/jira/browse/KAFKA-2212?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14959503#comment-14959503 ] Parth Brahmbhatt commented on KAFKA-2212: - [~junrao] [~ijuma] # we already have a PR for KAFKA-2598 https://github.com/apache/kafka/pull/300 # Here is the [https://cwiki.apache.org/confluence/display/KAFKA/Kafka+Authorization+Command+Line+Interface wiki], please review and you can update the wiki or comment on this jira if you want me to add more details. # As part of the PR for KAKFA-2598 , I have removed the unused import. > KafkaAuthorizer: Add CLI for Acl management. > - > > Key: KAFKA-2212 > URL: https://issues.apache.org/jira/browse/KAFKA-2212 > Project: Kafka > Issue Type: Sub-task > Components: security >Reporter: Parth Brahmbhatt >Assignee: Parth Brahmbhatt >Priority: Blocker > Fix For: 0.9.0.0 > > Attachments: KAFKA-2212.patch > > > This is subtask-3 for Kafka-1688. > Please see KIP-11 for details on CLI for Authorizer. > https://cwiki.apache.org/confluence/display/KAFKA/KIP-11+-+Authorization+Interface -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (KAFKA-2598) Add Test with authorizer for producer and consumer
[ https://issues.apache.org/jira/browse/KAFKA-2598?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14947417#comment-14947417 ] Parth Brahmbhatt commented on KAFKA-2598: - I just started yesterday, should have an initial patch by EOD with producer and consumer. It is hard to write a test when we don't have miniKDC dependency as currently there is no way to establish identity and without multiple identities some test cases can not be written. I am trying to write the basic test cases for producer and consumer by granting access to "ANNONYMOUS" user to Write and Read which is the principal we get when no identity is established. > Add Test with authorizer for producer and consumer > -- > > Key: KAFKA-2598 > URL: https://issues.apache.org/jira/browse/KAFKA-2598 > Project: Kafka > Issue Type: Sub-task > Components: security, unit tests >Affects Versions: 0.8.2.2 >Reporter: Parth Brahmbhatt >Assignee: Parth Brahmbhatt >Priority: Blocker > Fix For: 0.9.0.0 > > > Now that we have all the authorizer code merged into trunk we should add a > test that enables authorizer and tests that only authorized users can > produce/consume from topics or issue cluster actions. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (KAFKA-2587) Transient test failure: `SimpleAclAuthorizerTest`
[ https://issues.apache.org/jira/browse/KAFKA-2587?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14943792#comment-14943792 ] Parth Brahmbhatt commented on KAFKA-2587: - https://github.com/apache/kafka/pull/277/files > Transient test failure: `SimpleAclAuthorizerTest` > - > > Key: KAFKA-2587 > URL: https://issues.apache.org/jira/browse/KAFKA-2587 > Project: Kafka > Issue Type: Sub-task >Reporter: Ismael Juma >Assignee: Parth Brahmbhatt > Fix For: 0.9.0.0 > > > I've seen `SimpleAclAuthorizerTest ` fail a couple of times since its recent > introduction. Here's one such build: > https://builds.apache.org/job/kafka-trunk-git-pr/576/console > [~parth.brahmbhatt], can you please take a look and see if it's an easy fix? -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (KAFKA-2587) Transient test failure: `SimpleAclAuthorizerTest`
[ https://issues.apache.org/jira/browse/KAFKA-2587?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14941474#comment-14941474 ] Parth Brahmbhatt commented on KAFKA-2587: - I looked at the code to reason around why this can happen. The state reported is indeed one of the valid states during our test https://github.com/apache/kafka/blob/5764e54de147af81aac85acc00687c23e9646a5c/core/src/test/scala/unit/kafka/security/auth/SimpleAclAuthorizerTest.scala#L217 After that line we actually remove all acls for that resource, add one acl back to it and remove that one acl. All those steps pass verification. https://github.com/apache/kafka/blob/5764e54de147af81aac85acc00687c23e9646a5c/core/src/test/scala/unit/kafka/security/auth/SimpleAclAuthorizerTest.scala#L225 and https://github.com/apache/kafka/blob/5764e54de147af81aac85acc00687c23e9646a5c/core/src/test/scala/unit/kafka/security/auth/SimpleAclAuthorizerTest.scala#L226 Given we are using the same instance of the authorizer the cache of that instance is immediately updated for both add and remove. https://github.com/apache/kafka/blob/5764e54de147af81aac85acc00687c23e9646a5c/core/src/main/scala/kafka/security/auth/SimpleAclAuthorizer.scala#L171 https://github.com/apache/kafka/blob/5764e54de147af81aac85acc00687c23e9646a5c/core/src/main/scala/kafka/security/auth/SimpleAclAuthorizer.scala#L189 The only other place that can update the cache is notification handler as part of handling acl-changed notification. https://github.com/apache/kafka/blob/5764e54de147af81aac85acc00687c23e9646a5c/core/src/main/scala/kafka/security/auth/SimpleAclAuthorizer.scala#L269 However in that case we read the data from zookeeper and then update the cache. If the notifications processing was delayed for some reason, it should still read the acls from zk and then update the cache. There are pathological cases that can lead to this failure , for example: 1) Notification handler starts, reads acls from zk and a thread switch happens before it can update the cache 2) All the other cache updates go through (remove resource, add the acl, remove the acl). 3) Before verification finishes for the last "remove one acl" a thread switch happens and notification handler update the cache with stale acls that it read before. Even with this case there should be follow up notifications about adding an acl and removing an acl which should again cause the notification process to read state from zookeeper and update the cache to correct state. Plus this seems unlikely enough that it would not happen every other day. I will continue to look into this. In the meantime if this is a continuous dev pain, we can remove the last 3 lines of test that removes the last acl and tries to verify that the zookeeper path is deleted. > Transient test failure: `SimpleAclAuthorizerTest` > - > > Key: KAFKA-2587 > URL: https://issues.apache.org/jira/browse/KAFKA-2587 > Project: Kafka > Issue Type: Sub-task >Reporter: Ismael Juma >Assignee: Parth Brahmbhatt > Fix For: 0.9.0.0 > > > I've seen `SimpleAclAuthorizerTest ` fail a couple of times since its recent > introduction. Here's one such build: > https://builds.apache.org/job/kafka-trunk-git-pr/576/console > [~parth.brahmbhatt], can you please take a look and see if it's an easy fix? -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (KAFKA-2587) Transient test failure: `SimpleAclAuthorizerTest`
[ https://issues.apache.org/jira/browse/KAFKA-2587?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14941533#comment-14941533 ] Parth Brahmbhatt commented on KAFKA-2587: - Hey Ismael, As mentioned above, in production given all notifications will be processed sequentially and as part of each notification we read the acls from zookeeper (source of truth) this issue should not happen in production as long as zookeeper guarantees read after write consistency. We may run into this issue in the case where we observe inconsistency due to zookeeper not supporting "Simultaneously Consistent Cross-Client Views". Please see Notes section in http://zookeeper.apache.org/doc/trunk/zookeeperProgrammers.html#ch_zkGuarantees. I looked at the zkClient code to see if we can call sync before read to avoid any consistency concerns but zkClient does not support sync. > Transient test failure: `SimpleAclAuthorizerTest` > - > > Key: KAFKA-2587 > URL: https://issues.apache.org/jira/browse/KAFKA-2587 > Project: Kafka > Issue Type: Sub-task >Reporter: Ismael Juma >Assignee: Parth Brahmbhatt > Fix For: 0.9.0.0 > > > I've seen `SimpleAclAuthorizerTest ` fail a couple of times since its recent > introduction. Here's one such build: > https://builds.apache.org/job/kafka-trunk-git-pr/576/console > [~parth.brahmbhatt], can you please take a look and see if it's an easy fix? -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (KAFKA-2598) Add Test with authorizer for producer and consumer
[ https://issues.apache.org/jira/browse/KAFKA-2598?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14935589#comment-14935589 ] Parth Brahmbhatt commented on KAFKA-2598: - I should be able to pick this one up next week. > Add Test with authorizer for producer and consumer > -- > > Key: KAFKA-2598 > URL: https://issues.apache.org/jira/browse/KAFKA-2598 > Project: Kafka > Issue Type: Sub-task > Components: security, unit tests >Affects Versions: 0.8.2.2 >Reporter: Parth Brahmbhatt >Assignee: Parth Brahmbhatt > Fix For: 0.9.0.0 > > > Now that we have all the authorizer code merged into trunk we should add a > test that enables authorizer and tests that only authorized users can > produce/consume from topics or issue cluster actions. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (KAFKA-2598) Add Test with authorizer for producer and consumer
Parth Brahmbhatt created KAFKA-2598: --- Summary: Add Test with authorizer for producer and consumer Key: KAFKA-2598 URL: https://issues.apache.org/jira/browse/KAFKA-2598 Project: Kafka Issue Type: Bug Components: security, unit tests Affects Versions: 0.8.2.2 Reporter: Parth Brahmbhatt Assignee: Parth Brahmbhatt Fix For: 0.8.2.2 Now that we have all the authorizer code merged into trunk we should add a test that enables authorizer and tests that only authorized users can produce/consume from topics or issue cluster actions. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (KAFKA-2587) Transient test failure: `SimpleAclAuthorizerTest`
[
https://issues.apache.org/jira/browse/KAFKA-2587?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14934498#comment-14934498
]
Parth Brahmbhatt commented on KAFKA-2587:
-
[~ijuma] I tried to reproduce this locally by running the test in loop for 50
times and it never failed.
{code}
for i in `seq 1 50`; do ./gradlew clean core:test
-Dtest.single=SimpleAclAuthorizerTest; done
{code}
Just analyzing the failure shows that the transient failure is reported during
removal of the acl on this line.
https://github.com/apache/kafka/blob/0990b6ba6a28276f79a1a8bfaf48455c6eddfa1f/core/src/test/scala/unit/kafka/security/auth/SimpleAclAuthorizerTest.scala#L190
Looking at the code given we update the local authorizer's cache immediately I
am not entirely sure why would we see transient failures. I have modified the
failure message to indicate a little bit more debug info and I have bumped up
the timeout from 5 seconds to 10 seconds in case notification processing delay
interleaved with the changes made to cache is somehow resulting in this failure.
{code}
kafka.security.auth.SimpleAclAuthorizerTest > testAclManagementAPIs FAILED
java.lang.AssertionError: changes not propagated in timeout period.
at org.junit.Assert.fail(Assert.java:88)
at kafka.utils.TestUtils$.waitUntilTrue(TestUtils.scala:655)
at
kafka.security.auth.SimpleAclAuthorizerTest.changeAclAndVerify(SimpleAclAuthorizerTest.scala:229)
at
kafka.security.auth.SimpleAclAuthorizerTest.testAclManagementAPIs(SimpleAclAuthorizerTest.scala:190)
{code}
> Transient test failure: `SimpleAclAuthorizerTest`
> -
>
> Key: KAFKA-2587
> URL: https://issues.apache.org/jira/browse/KAFKA-2587
> Project: Kafka
> Issue Type: Sub-task
>Reporter: Ismael Juma
>Assignee: Parth Brahmbhatt
>
> I've seen `SimpleAclAuthorizerTest ` fail a couple of times since its recent
> introduction. Here's one such build:
> https://builds.apache.org/job/kafka-trunk-git-pr/576/console
> [~parth.brahmbhatt], can you please take a look and see if it's an easy fix?
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Work started] (KAFKA-2587) Transient test failure: `SimpleAclAuthorizerTest`
[ https://issues.apache.org/jira/browse/KAFKA-2587?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on KAFKA-2587 started by Parth Brahmbhatt. --- > Transient test failure: `SimpleAclAuthorizerTest` > - > > Key: KAFKA-2587 > URL: https://issues.apache.org/jira/browse/KAFKA-2587 > Project: Kafka > Issue Type: Sub-task >Reporter: Ismael Juma >Assignee: Parth Brahmbhatt > > I've seen `SimpleAclAuthorizerTest ` fail a couple of times since its recent > introduction. Here's one such build: > https://builds.apache.org/job/kafka-trunk-git-pr/576/console > [~parth.brahmbhatt], can you please take a look and see if it's an easy fix? -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (KAFKA-2212) KafkaAuthorizer: Add CLI for Acl management.
[ https://issues.apache.org/jira/browse/KAFKA-2212?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Parth Brahmbhatt updated KAFKA-2212: Status: In Progress (was: Patch Available) > KafkaAuthorizer: Add CLI for Acl management. > - > > Key: KAFKA-2212 > URL: https://issues.apache.org/jira/browse/KAFKA-2212 > Project: Kafka > Issue Type: Sub-task > Components: security >Reporter: Parth Brahmbhatt >Assignee: Parth Brahmbhatt >Priority: Blocker > Fix For: 0.9.0.0 > > Attachments: KAFKA-2212.patch > > > This is subtask-3 for Kafka-1688. > Please see KIP-11 for details on CLI for Authorizer. > https://cwiki.apache.org/confluence/display/KAFKA/KIP-11+-+Authorization+Interface -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (KAFKA-2549) Checkstyle reporting failure in trunk due to unused imports in Selector.java
[ https://issues.apache.org/jira/browse/KAFKA-2549?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14744368#comment-14744368 ] Parth Brahmbhatt commented on KAFKA-2549: - [~ijuma] You are right, my bad, edited the description. > Checkstyle reporting failure in trunk due to unused imports in Selector.java > > > Key: KAFKA-2549 > URL: https://issues.apache.org/jira/browse/KAFKA-2549 > Project: Kafka > Issue Type: Bug >Reporter: Parth Brahmbhatt >Assignee: Parth Brahmbhatt >Priority: Blocker > > Introduced in > https://github.com/apache/kafka/commit/d02ca36ca1cccdb6962191b97f54ce96b9d75abc#diff-db8f8be6ef2f1c81515d1ed83b3ab107 > in which the Selector.java was modified with some unused imports so the > trunk can not execute test targets as it fails in client section during > checkstyle stage. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (KAFKA-2549) Checkstyle reporting failure in trunk due to unused imports in Selector.java
[ https://issues.apache.org/jira/browse/KAFKA-2549?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Parth Brahmbhatt updated KAFKA-2549: Description: Introduced in https://github.com/apache/kafka/commit/d02ca36ca1cccdb6962191b97f54ce96b9d75abc#diff-db8f8be6ef2f1c81515d1ed83b3ab107 in which the Selector.java was modified with some unused imports so the trunk can not execute test targets as it fails in client section during checkstyle stage. (was: Again introduced in https://github.com/apache/kafka/commit/d02ca36ca1cccdb6962191b97f54ce96b9d75abc#diff-db8f8be6ef2f1c81515d1ed83b3ab107 in which the Selector.java was modified with some unused imports so the trunk can not execute test targets as it fails in client section during checkstyle stage.) > Checkstyle reporting failure in trunk due to unused imports in Selector.java > > > Key: KAFKA-2549 > URL: https://issues.apache.org/jira/browse/KAFKA-2549 > Project: Kafka > Issue Type: Bug >Reporter: Parth Brahmbhatt >Assignee: Parth Brahmbhatt >Priority: Blocker > > Introduced in > https://github.com/apache/kafka/commit/d02ca36ca1cccdb6962191b97f54ce96b9d75abc#diff-db8f8be6ef2f1c81515d1ed83b3ab107 > in which the Selector.java was modified with some unused imports so the > trunk can not execute test targets as it fails in client section during > checkstyle stage. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (KAFKA-2547) Make DynamicConfigManager to use the ZkNodeChangeNotificationListener introduced as part of KAFKA-2211
Parth Brahmbhatt created KAFKA-2547: --- Summary: Make DynamicConfigManager to use the ZkNodeChangeNotificationListener introduced as part of KAFKA-2211 Key: KAFKA-2547 URL: https://issues.apache.org/jira/browse/KAFKA-2547 Project: Kafka Issue Type: Improvement Reporter: Parth Brahmbhatt Assignee: Parth Brahmbhatt As part of KAFKA-2211 (https://github.com/apache/kafka/pull/195/files) we introduced a reusable ZkNodeChangeNotificationListener to ensure node changes can be processed in a loss less way. This was pretty much the same code in DynamicConfigManager with little bit of refactoring so it can be reused. We now need to make DynamicConfigManager itself to use this new class once KAFKA-2211 is committed to avoid code duplication. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Assigned] (KAFKA-2538) Compilation in trunk is failing due to https://github.com/apache/kafka/commit/845514d62329be8382e6d02b8041fc858718d534
[ https://issues.apache.org/jira/browse/KAFKA-2538?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Parth Brahmbhatt reassigned KAFKA-2538: --- Assignee: Parth Brahmbhatt > Compilation in trunk is failing due to > https://github.com/apache/kafka/commit/845514d62329be8382e6d02b8041fc858718d534 > -- > > Key: KAFKA-2538 > URL: https://issues.apache.org/jira/browse/KAFKA-2538 > Project: Kafka > Issue Type: Bug >Reporter: Parth Brahmbhatt >Assignee: Parth Brahmbhatt >Priority: Blocker > > Getting > /Users/pbrahmbhatt/repo/kafka/core/src/main/scala/kafka/tools/EndToEndLatency.scala:82: > value commit is not a member of > org.apache.kafka.clients.consumer.KafkaConsumer[Array[Byte],Array[Byte]] > consumer.commit(CommitType.SYNC) >^ > Which I believe was missed when committing KAFKA-2389 which replaces all > occurrences of commit(mode) with commit(Sync/Async). This is resulting in > other PRS reporting as bad by jenkins like > https://github.com/apache/kafka/pull/195 where 2 failures were reported by > jenkins https://builds.apache.org/job/kafka-trunk-git-pr/410/ and > https://builds.apache.org/job/kafka-trunk-git-pr/411/ -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (KAFKA-2538) Compilation in trunk is failing due to https://github.com/apache/kafka/commit/845514d62329be8382e6d02b8041fc858718d534
Parth Brahmbhatt created KAFKA-2538: --- Summary: Compilation in trunk is failing due to https://github.com/apache/kafka/commit/845514d62329be8382e6d02b8041fc858718d534 Key: KAFKA-2538 URL: https://issues.apache.org/jira/browse/KAFKA-2538 Project: Kafka Issue Type: Bug Reporter: Parth Brahmbhatt Priority: Blocker Getting /Users/pbrahmbhatt/repo/kafka/core/src/main/scala/kafka/tools/EndToEndLatency.scala:82: value commit is not a member of org.apache.kafka.clients.consumer.KafkaConsumer[Array[Byte],Array[Byte]] consumer.commit(CommitType.SYNC) ^ Which I believe was missed when committing KAFKA-2389 which replaces all occurrences of commit(mode) with commit(Sync/Async). This is resulting in other PRS reporting as bad by jenkins like https://github.com/apache/kafka/pull/195 where 2 failures were reported by jenkins https://builds.apache.org/job/kafka-trunk-git-pr/410/ and https://builds.apache.org/job/kafka-trunk-git-pr/411/ -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Work started] (KAFKA-2211) KafkaAuthorizer: Add simpleACLAuthorizer implementation.
[ https://issues.apache.org/jira/browse/KAFKA-2211?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on KAFKA-2211 started by Parth Brahmbhatt. --- > KafkaAuthorizer: Add simpleACLAuthorizer implementation. > > > Key: KAFKA-2211 > URL: https://issues.apache.org/jira/browse/KAFKA-2211 > Project: Kafka > Issue Type: Sub-task > Components: security >Reporter: Parth Brahmbhatt >Assignee: Parth Brahmbhatt >Priority: Blocker > Fix For: 0.8.3 > > Attachments: KAFKA-2211.patch > > > Subtask-2 for Kafka-1688. > Please see KIP-11 to get details on out of box SimpleACLAuthorizer > implementation > https://cwiki.apache.org/confluence/display/KAFKA/KIP-11+-+Authorization+Interface. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (KAFKA-2211) KafkaAuthorizer: Add simpleACLAuthorizer implementation.
[ https://issues.apache.org/jira/browse/KAFKA-2211?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Parth Brahmbhatt updated KAFKA-2211: Status: Patch Available (was: In Progress) > KafkaAuthorizer: Add simpleACLAuthorizer implementation. > > > Key: KAFKA-2211 > URL: https://issues.apache.org/jira/browse/KAFKA-2211 > Project: Kafka > Issue Type: Sub-task > Components: security >Reporter: Parth Brahmbhatt >Assignee: Parth Brahmbhatt >Priority: Blocker > Fix For: 0.8.3 > > Attachments: KAFKA-2211.patch > > > Subtask-2 for Kafka-1688. > Please see KIP-11 to get details on out of box SimpleACLAuthorizer > implementation > https://cwiki.apache.org/confluence/display/KAFKA/KIP-11+-+Authorization+Interface. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (KAFKA-2210) KafkaAuthorizer: Add all public entities, config changes and changes to KafkaAPI and kafkaServer to allow pluggable authorizer implementation.
[ https://issues.apache.org/jira/browse/KAFKA-2210?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Parth Brahmbhatt updated KAFKA-2210: Attachment: KAFKA-2210_2015-09-02_17:32:06.patch > KafkaAuthorizer: Add all public entities, config changes and changes to > KafkaAPI and kafkaServer to allow pluggable authorizer implementation. > -- > > Key: KAFKA-2210 > URL: https://issues.apache.org/jira/browse/KAFKA-2210 > Project: Kafka > Issue Type: Sub-task > Components: security >Reporter: Parth Brahmbhatt >Assignee: Parth Brahmbhatt >Priority: Blocker > Fix For: 0.8.3 > > Attachments: KAFKA-2210.patch, KAFKA-2210_2015-06-03_16:36:11.patch, > KAFKA-2210_2015-06-04_16:07:39.patch, KAFKA-2210_2015-07-09_18:00:34.patch, > KAFKA-2210_2015-07-14_10:02:19.patch, KAFKA-2210_2015-07-14_14:13:19.patch, > KAFKA-2210_2015-07-20_16:42:18.patch, KAFKA-2210_2015-07-21_17:08:21.patch, > KAFKA-2210_2015-08-10_18:31:54.patch, KAFKA-2210_2015-08-20_11:27:18.patch, > KAFKA-2210_2015-08-25_17:59:22.patch, KAFKA-2210_2015-08-26_14:29:02.patch, > KAFKA-2210_2015-09-01_15:36:02.patch, KAFKA-2210_2015-09-02_14:50:29.patch, > KAFKA-2210_2015-09-02_17:32:06.patch > > > This is the first subtask for Kafka-1688. As Part of this jira we intend to > agree on all the public entities, configs and changes to existing kafka > classes to allow pluggable authorizer implementation. > Please see KIP-11 > https://cwiki.apache.org/confluence/display/KAFKA/KIP-11+-+Authorization+Interface > for detailed design. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (KAFKA-2210) KafkaAuthorizer: Add all public entities, config changes and changes to KafkaAPI and kafkaServer to allow pluggable authorizer implementation.
[ https://issues.apache.org/jira/browse/KAFKA-2210?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14728293#comment-14728293 ] Parth Brahmbhatt commented on KAFKA-2210: - Updated reviewboard https://reviews.apache.org/r/34492/diff/ against branch origin/trunk > KafkaAuthorizer: Add all public entities, config changes and changes to > KafkaAPI and kafkaServer to allow pluggable authorizer implementation. > -- > > Key: KAFKA-2210 > URL: https://issues.apache.org/jira/browse/KAFKA-2210 > Project: Kafka > Issue Type: Sub-task > Components: security >Reporter: Parth Brahmbhatt >Assignee: Parth Brahmbhatt >Priority: Blocker > Fix For: 0.8.3 > > Attachments: KAFKA-2210.patch, KAFKA-2210_2015-06-03_16:36:11.patch, > KAFKA-2210_2015-06-04_16:07:39.patch, KAFKA-2210_2015-07-09_18:00:34.patch, > KAFKA-2210_2015-07-14_10:02:19.patch, KAFKA-2210_2015-07-14_14:13:19.patch, > KAFKA-2210_2015-07-20_16:42:18.patch, KAFKA-2210_2015-07-21_17:08:21.patch, > KAFKA-2210_2015-08-10_18:31:54.patch, KAFKA-2210_2015-08-20_11:27:18.patch, > KAFKA-2210_2015-08-25_17:59:22.patch, KAFKA-2210_2015-08-26_14:29:02.patch, > KAFKA-2210_2015-09-01_15:36:02.patch, KAFKA-2210_2015-09-02_14:50:29.patch, > KAFKA-2210_2015-09-02_17:32:06.patch > > > This is the first subtask for Kafka-1688. As Part of this jira we intend to > agree on all the public entities, configs and changes to existing kafka > classes to allow pluggable authorizer implementation. > Please see KIP-11 > https://cwiki.apache.org/confluence/display/KAFKA/KIP-11+-+Authorization+Interface > for detailed design. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (KAFKA-2210) KafkaAuthorizer: Add all public entities, config changes and changes to KafkaAPI and kafkaServer to allow pluggable authorizer implementation.
[ https://issues.apache.org/jira/browse/KAFKA-2210?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14728299#comment-14728299 ] Parth Brahmbhatt commented on KAFKA-2210: - Updated reviewboard https://reviews.apache.org/r/34492/diff/ against branch origin/trunk > KafkaAuthorizer: Add all public entities, config changes and changes to > KafkaAPI and kafkaServer to allow pluggable authorizer implementation. > -- > > Key: KAFKA-2210 > URL: https://issues.apache.org/jira/browse/KAFKA-2210 > Project: Kafka > Issue Type: Sub-task > Components: security >Reporter: Parth Brahmbhatt >Assignee: Parth Brahmbhatt >Priority: Blocker > Fix For: 0.8.3 > > Attachments: KAFKA-2210.patch, KAFKA-2210_2015-06-03_16:36:11.patch, > KAFKA-2210_2015-06-04_16:07:39.patch, KAFKA-2210_2015-07-09_18:00:34.patch, > KAFKA-2210_2015-07-14_10:02:19.patch, KAFKA-2210_2015-07-14_14:13:19.patch, > KAFKA-2210_2015-07-20_16:42:18.patch, KAFKA-2210_2015-07-21_17:08:21.patch, > KAFKA-2210_2015-08-10_18:31:54.patch, KAFKA-2210_2015-08-20_11:27:18.patch, > KAFKA-2210_2015-08-25_17:59:22.patch, KAFKA-2210_2015-08-26_14:29:02.patch, > KAFKA-2210_2015-09-01_15:36:02.patch, KAFKA-2210_2015-09-02_14:50:29.patch, > KAFKA-2210_2015-09-02_17:32:06.patch, KAFKA-2210_2015-09-02_17:36:47.patch > > > This is the first subtask for Kafka-1688. As Part of this jira we intend to > agree on all the public entities, configs and changes to existing kafka > classes to allow pluggable authorizer implementation. > Please see KIP-11 > https://cwiki.apache.org/confluence/display/KAFKA/KIP-11+-+Authorization+Interface > for detailed design. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (KAFKA-2210) KafkaAuthorizer: Add all public entities, config changes and changes to KafkaAPI and kafkaServer to allow pluggable authorizer implementation.
[ https://issues.apache.org/jira/browse/KAFKA-2210?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Parth Brahmbhatt updated KAFKA-2210: Attachment: KAFKA-2210_2015-09-02_17:36:47.patch > KafkaAuthorizer: Add all public entities, config changes and changes to > KafkaAPI and kafkaServer to allow pluggable authorizer implementation. > -- > > Key: KAFKA-2210 > URL: https://issues.apache.org/jira/browse/KAFKA-2210 > Project: Kafka > Issue Type: Sub-task > Components: security >Reporter: Parth Brahmbhatt >Assignee: Parth Brahmbhatt >Priority: Blocker > Fix For: 0.8.3 > > Attachments: KAFKA-2210.patch, KAFKA-2210_2015-06-03_16:36:11.patch, > KAFKA-2210_2015-06-04_16:07:39.patch, KAFKA-2210_2015-07-09_18:00:34.patch, > KAFKA-2210_2015-07-14_10:02:19.patch, KAFKA-2210_2015-07-14_14:13:19.patch, > KAFKA-2210_2015-07-20_16:42:18.patch, KAFKA-2210_2015-07-21_17:08:21.patch, > KAFKA-2210_2015-08-10_18:31:54.patch, KAFKA-2210_2015-08-20_11:27:18.patch, > KAFKA-2210_2015-08-25_17:59:22.patch, KAFKA-2210_2015-08-26_14:29:02.patch, > KAFKA-2210_2015-09-01_15:36:02.patch, KAFKA-2210_2015-09-02_14:50:29.patch, > KAFKA-2210_2015-09-02_17:32:06.patch, KAFKA-2210_2015-09-02_17:36:47.patch > > > This is the first subtask for Kafka-1688. As Part of this jira we intend to > agree on all the public entities, configs and changes to existing kafka > classes to allow pluggable authorizer implementation. > Please see KIP-11 > https://cwiki.apache.org/confluence/display/KAFKA/KIP-11+-+Authorization+Interface > for detailed design. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (KAFKA-2210) KafkaAuthorizer: Add all public entities, config changes and changes to KafkaAPI and kafkaServer to allow pluggable authorizer implementation.
[ https://issues.apache.org/jira/browse/KAFKA-2210?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Parth Brahmbhatt updated KAFKA-2210: Attachment: KAFKA-2210_2015-09-02_14:50:29.patch > KafkaAuthorizer: Add all public entities, config changes and changes to > KafkaAPI and kafkaServer to allow pluggable authorizer implementation. > -- > > Key: KAFKA-2210 > URL: https://issues.apache.org/jira/browse/KAFKA-2210 > Project: Kafka > Issue Type: Sub-task > Components: security >Reporter: Parth Brahmbhatt >Assignee: Parth Brahmbhatt >Priority: Blocker > Fix For: 0.8.3 > > Attachments: KAFKA-2210.patch, KAFKA-2210_2015-06-03_16:36:11.patch, > KAFKA-2210_2015-06-04_16:07:39.patch, KAFKA-2210_2015-07-09_18:00:34.patch, > KAFKA-2210_2015-07-14_10:02:19.patch, KAFKA-2210_2015-07-14_14:13:19.patch, > KAFKA-2210_2015-07-20_16:42:18.patch, KAFKA-2210_2015-07-21_17:08:21.patch, > KAFKA-2210_2015-08-10_18:31:54.patch, KAFKA-2210_2015-08-20_11:27:18.patch, > KAFKA-2210_2015-08-25_17:59:22.patch, KAFKA-2210_2015-08-26_14:29:02.patch, > KAFKA-2210_2015-09-01_15:36:02.patch, KAFKA-2210_2015-09-02_14:50:29.patch > > > This is the first subtask for Kafka-1688. As Part of this jira we intend to > agree on all the public entities, configs and changes to existing kafka > classes to allow pluggable authorizer implementation. > Please see KIP-11 > https://cwiki.apache.org/confluence/display/KAFKA/KIP-11+-+Authorization+Interface > for detailed design. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (KAFKA-2210) KafkaAuthorizer: Add all public entities, config changes and changes to KafkaAPI and kafkaServer to allow pluggable authorizer implementation.
[ https://issues.apache.org/jira/browse/KAFKA-2210?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14728093#comment-14728093 ] Parth Brahmbhatt commented on KAFKA-2210: - Updated reviewboard https://reviews.apache.org/r/34492/diff/ against branch origin/trunk > KafkaAuthorizer: Add all public entities, config changes and changes to > KafkaAPI and kafkaServer to allow pluggable authorizer implementation. > -- > > Key: KAFKA-2210 > URL: https://issues.apache.org/jira/browse/KAFKA-2210 > Project: Kafka > Issue Type: Sub-task > Components: security >Reporter: Parth Brahmbhatt >Assignee: Parth Brahmbhatt >Priority: Blocker > Fix For: 0.8.3 > > Attachments: KAFKA-2210.patch, KAFKA-2210_2015-06-03_16:36:11.patch, > KAFKA-2210_2015-06-04_16:07:39.patch, KAFKA-2210_2015-07-09_18:00:34.patch, > KAFKA-2210_2015-07-14_10:02:19.patch, KAFKA-2210_2015-07-14_14:13:19.patch, > KAFKA-2210_2015-07-20_16:42:18.patch, KAFKA-2210_2015-07-21_17:08:21.patch, > KAFKA-2210_2015-08-10_18:31:54.patch, KAFKA-2210_2015-08-20_11:27:18.patch, > KAFKA-2210_2015-08-25_17:59:22.patch, KAFKA-2210_2015-08-26_14:29:02.patch, > KAFKA-2210_2015-09-01_15:36:02.patch, KAFKA-2210_2015-09-02_14:50:29.patch > > > This is the first subtask for Kafka-1688. As Part of this jira we intend to > agree on all the public entities, configs and changes to existing kafka > classes to allow pluggable authorizer implementation. > Please see KIP-11 > https://cwiki.apache.org/confluence/display/KAFKA/KIP-11+-+Authorization+Interface > for detailed design. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (KAFKA-2210) KafkaAuthorizer: Add all public entities, config changes and changes to KafkaAPI and kafkaServer to allow pluggable authorizer implementation.
[ https://issues.apache.org/jira/browse/KAFKA-2210?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Parth Brahmbhatt updated KAFKA-2210: Attachment: KAFKA-2210_2015-09-01_15:36:02.patch > KafkaAuthorizer: Add all public entities, config changes and changes to > KafkaAPI and kafkaServer to allow pluggable authorizer implementation. > -- > > Key: KAFKA-2210 > URL: https://issues.apache.org/jira/browse/KAFKA-2210 > Project: Kafka > Issue Type: Sub-task > Components: security >Reporter: Parth Brahmbhatt >Assignee: Parth Brahmbhatt > Fix For: 0.8.3 > > Attachments: KAFKA-2210.patch, KAFKA-2210_2015-06-03_16:36:11.patch, > KAFKA-2210_2015-06-04_16:07:39.patch, KAFKA-2210_2015-07-09_18:00:34.patch, > KAFKA-2210_2015-07-14_10:02:19.patch, KAFKA-2210_2015-07-14_14:13:19.patch, > KAFKA-2210_2015-07-20_16:42:18.patch, KAFKA-2210_2015-07-21_17:08:21.patch, > KAFKA-2210_2015-08-10_18:31:54.patch, KAFKA-2210_2015-08-20_11:27:18.patch, > KAFKA-2210_2015-08-25_17:59:22.patch, KAFKA-2210_2015-08-26_14:29:02.patch, > KAFKA-2210_2015-09-01_15:36:02.patch > > > This is the first subtask for Kafka-1688. As Part of this jira we intend to > agree on all the public entities, configs and changes to existing kafka > classes to allow pluggable authorizer implementation. > Please see KIP-11 > https://cwiki.apache.org/confluence/display/KAFKA/KIP-11+-+Authorization+Interface > for detailed design. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (KAFKA-2210) KafkaAuthorizer: Add all public entities, config changes and changes to KafkaAPI and kafkaServer to allow pluggable authorizer implementation.
[ https://issues.apache.org/jira/browse/KAFKA-2210?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14723866#comment-14723866 ] Parth Brahmbhatt commented on KAFKA-2210: - [~junrao] yes that is correct sequencing and I haven't updated 2211 and 2212 as we were changing 2210. Once 2210 is merged I will update 2211 and 2212 and send new pull requests. > KafkaAuthorizer: Add all public entities, config changes and changes to > KafkaAPI and kafkaServer to allow pluggable authorizer implementation. > -- > > Key: KAFKA-2210 > URL: https://issues.apache.org/jira/browse/KAFKA-2210 > Project: Kafka > Issue Type: Sub-task > Components: security >Reporter: Parth Brahmbhatt >Assignee: Parth Brahmbhatt > Fix For: 0.8.3 > > Attachments: KAFKA-2210.patch, KAFKA-2210_2015-06-03_16:36:11.patch, > KAFKA-2210_2015-06-04_16:07:39.patch, KAFKA-2210_2015-07-09_18:00:34.patch, > KAFKA-2210_2015-07-14_10:02:19.patch, KAFKA-2210_2015-07-14_14:13:19.patch, > KAFKA-2210_2015-07-20_16:42:18.patch, KAFKA-2210_2015-07-21_17:08:21.patch, > KAFKA-2210_2015-08-10_18:31:54.patch, KAFKA-2210_2015-08-20_11:27:18.patch, > KAFKA-2210_2015-08-25_17:59:22.patch, KAFKA-2210_2015-08-26_14:29:02.patch > > > This is the first subtask for Kafka-1688. As Part of this jira we intend to > agree on all the public entities, configs and changes to existing kafka > classes to allow pluggable authorizer implementation. > Please see KIP-11 > https://cwiki.apache.org/confluence/display/KAFKA/KIP-11+-+Authorization+Interface > for detailed design. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (KAFKA-2210) KafkaAuthorizer: Add all public entities, config changes and changes to KafkaAPI and kafkaServer to allow pluggable authorizer implementation.
[ https://issues.apache.org/jira/browse/KAFKA-2210?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Parth Brahmbhatt updated KAFKA-2210: Attachment: KAFKA-2210_2015-08-26_14:29:02.patch KafkaAuthorizer: Add all public entities, config changes and changes to KafkaAPI and kafkaServer to allow pluggable authorizer implementation. -- Key: KAFKA-2210 URL: https://issues.apache.org/jira/browse/KAFKA-2210 Project: Kafka Issue Type: Sub-task Components: security Reporter: Parth Brahmbhatt Assignee: Parth Brahmbhatt Fix For: 0.8.3 Attachments: KAFKA-2210.patch, KAFKA-2210_2015-06-03_16:36:11.patch, KAFKA-2210_2015-06-04_16:07:39.patch, KAFKA-2210_2015-07-09_18:00:34.patch, KAFKA-2210_2015-07-14_10:02:19.patch, KAFKA-2210_2015-07-14_14:13:19.patch, KAFKA-2210_2015-07-20_16:42:18.patch, KAFKA-2210_2015-07-21_17:08:21.patch, KAFKA-2210_2015-08-10_18:31:54.patch, KAFKA-2210_2015-08-20_11:27:18.patch, KAFKA-2210_2015-08-25_17:59:22.patch, KAFKA-2210_2015-08-26_14:29:02.patch This is the first subtask for Kafka-1688. As Part of this jira we intend to agree on all the public entities, configs and changes to existing kafka classes to allow pluggable authorizer implementation. Please see KIP-11 https://cwiki.apache.org/confluence/display/KAFKA/KIP-11+-+Authorization+Interface for detailed design. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (KAFKA-2210) KafkaAuthorizer: Add all public entities, config changes and changes to KafkaAPI and kafkaServer to allow pluggable authorizer implementation.
[ https://issues.apache.org/jira/browse/KAFKA-2210?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14712282#comment-14712282 ] Parth Brahmbhatt commented on KAFKA-2210: - Updated reviewboard https://reviews.apache.org/r/34492/diff/ against branch origin/trunk KafkaAuthorizer: Add all public entities, config changes and changes to KafkaAPI and kafkaServer to allow pluggable authorizer implementation. -- Key: KAFKA-2210 URL: https://issues.apache.org/jira/browse/KAFKA-2210 Project: Kafka Issue Type: Sub-task Components: security Reporter: Parth Brahmbhatt Assignee: Parth Brahmbhatt Fix For: 0.8.3 Attachments: KAFKA-2210.patch, KAFKA-2210_2015-06-03_16:36:11.patch, KAFKA-2210_2015-06-04_16:07:39.patch, KAFKA-2210_2015-07-09_18:00:34.patch, KAFKA-2210_2015-07-14_10:02:19.patch, KAFKA-2210_2015-07-14_14:13:19.patch, KAFKA-2210_2015-07-20_16:42:18.patch, KAFKA-2210_2015-07-21_17:08:21.patch, KAFKA-2210_2015-08-10_18:31:54.patch, KAFKA-2210_2015-08-20_11:27:18.patch, KAFKA-2210_2015-08-25_17:59:22.patch This is the first subtask for Kafka-1688. As Part of this jira we intend to agree on all the public entities, configs and changes to existing kafka classes to allow pluggable authorizer implementation. Please see KIP-11 https://cwiki.apache.org/confluence/display/KAFKA/KIP-11+-+Authorization+Interface for detailed design. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (KAFKA-2210) KafkaAuthorizer: Add all public entities, config changes and changes to KafkaAPI and kafkaServer to allow pluggable authorizer implementation.
[ https://issues.apache.org/jira/browse/KAFKA-2210?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Parth Brahmbhatt updated KAFKA-2210: Attachment: KAFKA-2210_2015-08-25_17:59:22.patch KafkaAuthorizer: Add all public entities, config changes and changes to KafkaAPI and kafkaServer to allow pluggable authorizer implementation. -- Key: KAFKA-2210 URL: https://issues.apache.org/jira/browse/KAFKA-2210 Project: Kafka Issue Type: Sub-task Components: security Reporter: Parth Brahmbhatt Assignee: Parth Brahmbhatt Fix For: 0.8.3 Attachments: KAFKA-2210.patch, KAFKA-2210_2015-06-03_16:36:11.patch, KAFKA-2210_2015-06-04_16:07:39.patch, KAFKA-2210_2015-07-09_18:00:34.patch, KAFKA-2210_2015-07-14_10:02:19.patch, KAFKA-2210_2015-07-14_14:13:19.patch, KAFKA-2210_2015-07-20_16:42:18.patch, KAFKA-2210_2015-07-21_17:08:21.patch, KAFKA-2210_2015-08-10_18:31:54.patch, KAFKA-2210_2015-08-20_11:27:18.patch, KAFKA-2210_2015-08-25_17:59:22.patch This is the first subtask for Kafka-1688. As Part of this jira we intend to agree on all the public entities, configs and changes to existing kafka classes to allow pluggable authorizer implementation. Please see KIP-11 https://cwiki.apache.org/confluence/display/KAFKA/KIP-11+-+Authorization+Interface for detailed design. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (KAFKA-2145) An option to add topic owners.
[ https://issues.apache.org/jira/browse/KAFKA-2145?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14712319#comment-14712319 ] Parth Brahmbhatt commented on KAFKA-2145: - [~junrao] Pinging again for review. An option to add topic owners. --- Key: KAFKA-2145 URL: https://issues.apache.org/jira/browse/KAFKA-2145 Project: Kafka Issue Type: Improvement Reporter: Parth Brahmbhatt Assignee: Parth Brahmbhatt We need to expose a way so users can identify users/groups that share ownership of topic. We discussed adding this as part of https://issues.apache.org/jira/browse/KAFKA-2035 and agreed that it will be simpler to add owner as a logconfig. The owner field can be used for auditing and also by authorization layer to grant access without having to explicitly configure acls. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (KAFKA-1683) Implement a session concept in the socket server
[ https://issues.apache.org/jira/browse/KAFKA-1683?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14706135#comment-14706135 ] Parth Brahmbhatt commented on KAFKA-1683: - [~gwenshap] I agree, the original KIP also assumes that when no authentication is done we will still set the Principal as ANONYMOUS. Only suggestion is , lets declare this constant in some Config in case we need to access the value from somewhere else in the code. Implement a session concept in the socket server -- Key: KAFKA-1683 URL: https://issues.apache.org/jira/browse/KAFKA-1683 Project: Kafka Issue Type: Sub-task Components: security Affects Versions: 0.9.0 Reporter: Jay Kreps Assignee: Gwen Shapira Fix For: 0.8.3 Attachments: KAFKA-1683.patch, KAFKA-1683.patch To implement authentication we need a way to keep track of some things between requests. The initial use for this would be remembering the authenticated user/principle info, but likely more uses would come up (for example we will also need to remember whether and which encryption or integrity measures are in place on the socket so we can wrap and unwrap writes and reads). I was thinking we could just add a Session object that might have a user field. The session object would need to get added to RequestChannel.Request so it is passed down to the API layer with each request. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (KAFKA-873) Consider replacing zkclient with curator (with zkclient-bridge)
[ https://issues.apache.org/jira/browse/KAFKA-873?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14705917#comment-14705917 ] Parth Brahmbhatt commented on KAFKA-873: Sorry, i misunderstood the original question. you are right guava is not bundled with kafka but with curator. and Flavio is correct that a client attempting to use guava with kafka could use shading to avoid conflicts or incompatibilities. Consider replacing zkclient with curator (with zkclient-bridge) --- Key: KAFKA-873 URL: https://issues.apache.org/jira/browse/KAFKA-873 Project: Kafka Issue Type: Improvement Affects Versions: 0.8.0 Reporter: Scott Clasen Assignee: Grant Henke If zkclient was replaced with curator and curator-x-zkclient-bridge it would be initially a drop-in replacement https://github.com/Netflix/curator/wiki/ZKClient-Bridge With the addition of a few more props to ZkConfig, and a bit of code this would open up the possibility of using ACLs in zookeeper (which arent supported directly by zkclient), as well as integrating with netflix exhibitor for those of us using that. Looks like KafkaZookeeperClient needs some love anyhow... -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (KAFKA-2210) KafkaAuthorizer: Add all public entities, config changes and changes to KafkaAPI and kafkaServer to allow pluggable authorizer implementation.
[ https://issues.apache.org/jira/browse/KAFKA-2210?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14705537#comment-14705537 ] Parth Brahmbhatt commented on KAFKA-2210: - [~ijuma] [~junrao] Thanks for the review and sorry for the bad quality in last patch. The code wont compile at this point as I have removed KAFKA-1683 changes from this patch and this patch depends on changes from KAFKA-1683. If you apply the old patch from KAFKA-1683 it should compile and locally all the tests passed for me. [~ijuma] I had every intention to just have authorizer call at top level in the switch case where we multiplex. Realized early on that it won't be possible given we don't have a common TopicRequest method and even though we have a top level response class some API's return their extensions even for error cases. In addition some of the methods need to verify topics , some topics and groups and some topics and cluster and sometimes there are side effects like topic creation. For the partition method, the best we might be able to do is create a method that would look awfully similar to authorize method but will take care of not duplicating boilerplate az.map().getOrElse() which IMO is not vastly different nor does it make the code more readable. Let me know if you come up with something better and I will be happy to incorporate your suggestion, I will also try the same on my end. KafkaAuthorizer: Add all public entities, config changes and changes to KafkaAPI and kafkaServer to allow pluggable authorizer implementation. -- Key: KAFKA-2210 URL: https://issues.apache.org/jira/browse/KAFKA-2210 Project: Kafka Issue Type: Sub-task Components: security Reporter: Parth Brahmbhatt Assignee: Parth Brahmbhatt Fix For: 0.8.3 Attachments: KAFKA-2210.patch, KAFKA-2210_2015-06-03_16:36:11.patch, KAFKA-2210_2015-06-04_16:07:39.patch, KAFKA-2210_2015-07-09_18:00:34.patch, KAFKA-2210_2015-07-14_10:02:19.patch, KAFKA-2210_2015-07-14_14:13:19.patch, KAFKA-2210_2015-07-20_16:42:18.patch, KAFKA-2210_2015-07-21_17:08:21.patch, KAFKA-2210_2015-08-10_18:31:54.patch, KAFKA-2210_2015-08-20_11:27:18.patch This is the first subtask for Kafka-1688. As Part of this jira we intend to agree on all the public entities, configs and changes to existing kafka classes to allow pluggable authorizer implementation. Please see KIP-11 https://cwiki.apache.org/confluence/display/KAFKA/KIP-11+-+Authorization+Interface for detailed design. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (KAFKA-1683) Implement a session concept in the socket server
[ https://issues.apache.org/jira/browse/KAFKA-1683?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14705538#comment-14705538 ] Parth Brahmbhatt commented on KAFKA-1683: - [~gwenshap] Given this blocks KAFKA-2210, if you are busy I can submit a PR for this one. Implement a session concept in the socket server -- Key: KAFKA-1683 URL: https://issues.apache.org/jira/browse/KAFKA-1683 Project: Kafka Issue Type: Sub-task Components: security Affects Versions: 0.9.0 Reporter: Jay Kreps Assignee: Gwen Shapira Fix For: 0.8.3 Attachments: KAFKA-1683.patch, KAFKA-1683.patch To implement authentication we need a way to keep track of some things between requests. The initial use for this would be remembering the authenticated user/principle info, but likely more uses would come up (for example we will also need to remember whether and which encryption or integrity measures are in place on the socket so we can wrap and unwrap writes and reads). I was thinking we could just add a Session object that might have a user field. The session object would need to get added to RequestChannel.Request so it is passed down to the API layer with each request. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (KAFKA-2210) KafkaAuthorizer: Add all public entities, config changes and changes to KafkaAPI and kafkaServer to allow pluggable authorizer implementation.
[ https://issues.apache.org/jira/browse/KAFKA-2210?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Parth Brahmbhatt updated KAFKA-2210: Attachment: KAFKA-2210_2015-08-20_11:27:18.patch KafkaAuthorizer: Add all public entities, config changes and changes to KafkaAPI and kafkaServer to allow pluggable authorizer implementation. -- Key: KAFKA-2210 URL: https://issues.apache.org/jira/browse/KAFKA-2210 Project: Kafka Issue Type: Sub-task Components: security Reporter: Parth Brahmbhatt Assignee: Parth Brahmbhatt Fix For: 0.8.3 Attachments: KAFKA-2210.patch, KAFKA-2210_2015-06-03_16:36:11.patch, KAFKA-2210_2015-06-04_16:07:39.patch, KAFKA-2210_2015-07-09_18:00:34.patch, KAFKA-2210_2015-07-14_10:02:19.patch, KAFKA-2210_2015-07-14_14:13:19.patch, KAFKA-2210_2015-07-20_16:42:18.patch, KAFKA-2210_2015-07-21_17:08:21.patch, KAFKA-2210_2015-08-10_18:31:54.patch, KAFKA-2210_2015-08-20_11:27:18.patch This is the first subtask for Kafka-1688. As Part of this jira we intend to agree on all the public entities, configs and changes to existing kafka classes to allow pluggable authorizer implementation. Please see KIP-11 https://cwiki.apache.org/confluence/display/KAFKA/KIP-11+-+Authorization+Interface for detailed design. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (KAFKA-2210) KafkaAuthorizer: Add all public entities, config changes and changes to KafkaAPI and kafkaServer to allow pluggable authorizer implementation.
[ https://issues.apache.org/jira/browse/KAFKA-2210?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14705505#comment-14705505 ] Parth Brahmbhatt commented on KAFKA-2210: - Updated reviewboard https://reviews.apache.org/r/34492/diff/ against branch origin/trunk KafkaAuthorizer: Add all public entities, config changes and changes to KafkaAPI and kafkaServer to allow pluggable authorizer implementation. -- Key: KAFKA-2210 URL: https://issues.apache.org/jira/browse/KAFKA-2210 Project: Kafka Issue Type: Sub-task Components: security Reporter: Parth Brahmbhatt Assignee: Parth Brahmbhatt Fix For: 0.8.3 Attachments: KAFKA-2210.patch, KAFKA-2210_2015-06-03_16:36:11.patch, KAFKA-2210_2015-06-04_16:07:39.patch, KAFKA-2210_2015-07-09_18:00:34.patch, KAFKA-2210_2015-07-14_10:02:19.patch, KAFKA-2210_2015-07-14_14:13:19.patch, KAFKA-2210_2015-07-20_16:42:18.patch, KAFKA-2210_2015-07-21_17:08:21.patch, KAFKA-2210_2015-08-10_18:31:54.patch, KAFKA-2210_2015-08-20_11:27:18.patch This is the first subtask for Kafka-1688. As Part of this jira we intend to agree on all the public entities, configs and changes to existing kafka classes to allow pluggable authorizer implementation. Please see KIP-11 https://cwiki.apache.org/confluence/display/KAFKA/KIP-11+-+Authorization+Interface for detailed design. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (KAFKA-2210) KafkaAuthorizer: Add all public entities, config changes and changes to KafkaAPI and kafkaServer to allow pluggable authorizer implementation.
[ https://issues.apache.org/jira/browse/KAFKA-2210?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Parth Brahmbhatt updated KAFKA-2210: Status: Patch Available (was: In Progress) KafkaAuthorizer: Add all public entities, config changes and changes to KafkaAPI and kafkaServer to allow pluggable authorizer implementation. -- Key: KAFKA-2210 URL: https://issues.apache.org/jira/browse/KAFKA-2210 Project: Kafka Issue Type: Sub-task Components: security Reporter: Parth Brahmbhatt Assignee: Parth Brahmbhatt Fix For: 0.8.3 Attachments: KAFKA-2210.patch, KAFKA-2210_2015-06-03_16:36:11.patch, KAFKA-2210_2015-06-04_16:07:39.patch, KAFKA-2210_2015-07-09_18:00:34.patch, KAFKA-2210_2015-07-14_10:02:19.patch, KAFKA-2210_2015-07-14_14:13:19.patch, KAFKA-2210_2015-07-20_16:42:18.patch, KAFKA-2210_2015-07-21_17:08:21.patch, KAFKA-2210_2015-08-10_18:31:54.patch, KAFKA-2210_2015-08-20_11:27:18.patch This is the first subtask for Kafka-1688. As Part of this jira we intend to agree on all the public entities, configs and changes to existing kafka classes to allow pluggable authorizer implementation. Please see KIP-11 https://cwiki.apache.org/confluence/display/KAFKA/KIP-11+-+Authorization+Interface for detailed design. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (KAFKA-1695) Authenticate connection to Zookeeper
[ https://issues.apache.org/jira/browse/KAFKA-1695?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14705211#comment-14705211 ] Parth Brahmbhatt commented on KAFKA-1695: - [~junrao] This is the patch https://github.com/apache/kafka/pull/93. I guess if I change the title after PR is submitted once apache jira is not updated. Authenticate connection to Zookeeper Key: KAFKA-1695 URL: https://issues.apache.org/jira/browse/KAFKA-1695 Project: Kafka Issue Type: Sub-task Components: security Reporter: Jay Kreps Assignee: Parth Brahmbhatt We need to make it possible to secure the Zookeeper cluster Kafka is using. This would make use of the normal authentication ZooKeeper provides. ZooKeeper supports a variety of authentication mechanisms so we will need to figure out what has to be passed in to the zookeeper client. The intention is that when the current round of client work is done it should be possible to run without clients needing access to Zookeeper so all we need here is to make it so that only the Kafka cluster is able to read and write to the Kafka znodes (we shouldn't need to set any kind of acl on a per-znode basis). -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (KAFKA-873) Consider replacing zkclient with curator (with zkclient-bridge)
[ https://issues.apache.org/jira/browse/KAFKA-873?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14705899#comment-14705899 ] Parth Brahmbhatt commented on KAFKA-873: you could, but then any client who relied on kafka to pull guava will be broken until they declare an explicit dependency on guava. Consider replacing zkclient with curator (with zkclient-bridge) --- Key: KAFKA-873 URL: https://issues.apache.org/jira/browse/KAFKA-873 Project: Kafka Issue Type: Improvement Affects Versions: 0.8.0 Reporter: Scott Clasen Assignee: Grant Henke If zkclient was replaced with curator and curator-x-zkclient-bridge it would be initially a drop-in replacement https://github.com/Netflix/curator/wiki/ZKClient-Bridge With the addition of a few more props to ZkConfig, and a bit of code this would open up the possibility of using ACLs in zookeeper (which arent supported directly by zkclient), as well as integrating with netflix exhibitor for those of us using that. Looks like KafkaZookeeperClient needs some love anyhow... -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (KAFKA-2145) An option to add topic owners.
[ https://issues.apache.org/jira/browse/KAFKA-2145?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14703539#comment-14703539 ] Parth Brahmbhatt commented on KAFKA-2145: - [~junrao] 3 week reminder ping to request review. An option to add topic owners. --- Key: KAFKA-2145 URL: https://issues.apache.org/jira/browse/KAFKA-2145 Project: Kafka Issue Type: Improvement Reporter: Parth Brahmbhatt Assignee: Parth Brahmbhatt We need to expose a way so users can identify users/groups that share ownership of topic. We discussed adding this as part of https://issues.apache.org/jira/browse/KAFKA-2035 and agreed that it will be simpler to add owner as a logconfig. The owner field can be used for auditing and also by authorization layer to grant access without having to explicitly configure acls. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (KAFKA-2210) KafkaAuthorizer: Add all public entities, config changes and changes to KafkaAPI and kafkaServer to allow pluggable authorizer implementation.
[ https://issues.apache.org/jira/browse/KAFKA-2210?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14703537#comment-14703537 ] Parth Brahmbhatt commented on KAFKA-2210: - [~junrao] [~ijuma] [~eribeiro] One week reminder ping for review request. KafkaAuthorizer: Add all public entities, config changes and changes to KafkaAPI and kafkaServer to allow pluggable authorizer implementation. -- Key: KAFKA-2210 URL: https://issues.apache.org/jira/browse/KAFKA-2210 Project: Kafka Issue Type: Sub-task Components: security Reporter: Parth Brahmbhatt Assignee: Parth Brahmbhatt Fix For: 0.8.3 Attachments: KAFKA-2210.patch, KAFKA-2210_2015-06-03_16:36:11.patch, KAFKA-2210_2015-06-04_16:07:39.patch, KAFKA-2210_2015-07-09_18:00:34.patch, KAFKA-2210_2015-07-14_10:02:19.patch, KAFKA-2210_2015-07-14_14:13:19.patch, KAFKA-2210_2015-07-20_16:42:18.patch, KAFKA-2210_2015-07-21_17:08:21.patch, KAFKA-2210_2015-08-10_18:31:54.patch This is the first subtask for Kafka-1688. As Part of this jira we intend to agree on all the public entities, configs and changes to existing kafka classes to allow pluggable authorizer implementation. Please see KIP-11 https://cwiki.apache.org/confluence/display/KAFKA/KIP-11+-+Authorization+Interface for detailed design. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (KAFKA-1695) Authenticate connection to Zookeeper
[ https://issues.apache.org/jira/browse/KAFKA-1695?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14697588#comment-14697588 ] Parth Brahmbhatt commented on KAFKA-1695: - [~fpj] When you have time can you please review this? Authenticate connection to Zookeeper Key: KAFKA-1695 URL: https://issues.apache.org/jira/browse/KAFKA-1695 Project: Kafka Issue Type: Sub-task Components: security Reporter: Jay Kreps Assignee: Parth Brahmbhatt We need to make it possible to secure the Zookeeper cluster Kafka is using. This would make use of the normal authentication ZooKeeper provides. ZooKeeper supports a variety of authentication mechanisms so we will need to figure out what has to be passed in to the zookeeper client. The intention is that when the current round of client work is done it should be possible to run without clients needing access to Zookeeper so all we need here is to make it so that only the Kafka cluster is able to read and write to the Kafka znodes (we shouldn't need to set any kind of acl on a per-znode basis). -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (KAFKA-1683) Implement a session concept in the socket server
[ https://issues.apache.org/jira/browse/KAFKA-1683?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14692317#comment-14692317 ] Parth Brahmbhatt commented on KAFKA-1683: - [~eugenstud] I believe this patch will set the foundation for the authorization. It will introduce the concept of a session where session will capture identity of the client so authorization layer can use that identity to authorize against some acl store. The Authorizer it self is being reviewed as part of KAFKA-2210. I am not sure what you mean by as different users. can you elaborate? Implement a session concept in the socket server -- Key: KAFKA-1683 URL: https://issues.apache.org/jira/browse/KAFKA-1683 Project: Kafka Issue Type: Sub-task Components: security Affects Versions: 0.9.0 Reporter: Jay Kreps Assignee: Gwen Shapira Fix For: 0.8.3 Attachments: KAFKA-1683.patch, KAFKA-1683.patch To implement authentication we need a way to keep track of some things between requests. The initial use for this would be remembering the authenticated user/principle info, but likely more uses would come up (for example we will also need to remember whether and which encryption or integrity measures are in place on the socket so we can wrap and unwrap writes and reads). I was thinking we could just add a Session object that might have a user field. The session object would need to get added to RequestChannel.Request so it is passed down to the API layer with each request. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (KAFKA-1695) Authenticate connection to Zookeeper
[ https://issues.apache.org/jira/browse/KAFKA-1695?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14680417#comment-14680417 ] Parth Brahmbhatt commented on KAFKA-1695: - [~ijuma] Don't have a date, have sent an E-mail to them. Right now I have not included the part that depends on the release so it's not a blocker. The APIs added in the new zkClient release are only required if we want to set the acls on already existing zookeeper nodes. This will be the case for anyone trying to move an existing kafka cluster to secure setup but for fresh installation or users willing to setting the zkAcls on existing kafka nodes manually the current patch should work as is. Authenticate connection to Zookeeper Key: KAFKA-1695 URL: https://issues.apache.org/jira/browse/KAFKA-1695 Project: Kafka Issue Type: Sub-task Components: security Reporter: Jay Kreps Assignee: Parth Brahmbhatt We need to make it possible to secure the Zookeeper cluster Kafka is using. This would make use of the normal authentication ZooKeeper provides. ZooKeeper supports a variety of authentication mechanisms so we will need to figure out what has to be passed in to the zookeeper client. The intention is that when the current round of client work is done it should be possible to run without clients needing access to Zookeeper so all we need here is to make it so that only the Kafka cluster is able to read and write to the Kafka znodes (we shouldn't need to set any kind of acl on a per-znode basis). -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (KAFKA-2210) KafkaAuthorizer: Add all public entities, config changes and changes to KafkaAPI and kafkaServer to allow pluggable authorizer implementation.
[ https://issues.apache.org/jira/browse/KAFKA-2210?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14680464#comment-14680464 ] Parth Brahmbhatt commented on KAFKA-2210: - [~ijuma] Sorry got busy with some other stuff.Will update the review before EOD. KafkaAuthorizer: Add all public entities, config changes and changes to KafkaAPI and kafkaServer to allow pluggable authorizer implementation. -- Key: KAFKA-2210 URL: https://issues.apache.org/jira/browse/KAFKA-2210 Project: Kafka Issue Type: Sub-task Components: security Reporter: Parth Brahmbhatt Assignee: Parth Brahmbhatt Fix For: 0.8.3 Attachments: KAFKA-2210.patch, KAFKA-2210_2015-06-03_16:36:11.patch, KAFKA-2210_2015-06-04_16:07:39.patch, KAFKA-2210_2015-07-09_18:00:34.patch, KAFKA-2210_2015-07-14_10:02:19.patch, KAFKA-2210_2015-07-14_14:13:19.patch, KAFKA-2210_2015-07-20_16:42:18.patch, KAFKA-2210_2015-07-21_17:08:21.patch This is the first subtask for Kafka-1688. As Part of this jira we intend to agree on all the public entities, configs and changes to existing kafka classes to allow pluggable authorizer implementation. Please see KIP-11 https://cwiki.apache.org/confluence/display/KAFKA/KIP-11+-+Authorization+Interface for detailed design. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (KAFKA-2210) KafkaAuthorizer: Add all public entities, config changes and changes to KafkaAPI and kafkaServer to allow pluggable authorizer implementation.
[ https://issues.apache.org/jira/browse/KAFKA-2210?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14681075#comment-14681075 ] Parth Brahmbhatt commented on KAFKA-2210: - Updated reviewboard https://reviews.apache.org/r/34492/diff/ against branch origin/trunk KafkaAuthorizer: Add all public entities, config changes and changes to KafkaAPI and kafkaServer to allow pluggable authorizer implementation. -- Key: KAFKA-2210 URL: https://issues.apache.org/jira/browse/KAFKA-2210 Project: Kafka Issue Type: Sub-task Components: security Reporter: Parth Brahmbhatt Assignee: Parth Brahmbhatt Fix For: 0.8.3 Attachments: KAFKA-2210.patch, KAFKA-2210_2015-06-03_16:36:11.patch, KAFKA-2210_2015-06-04_16:07:39.patch, KAFKA-2210_2015-07-09_18:00:34.patch, KAFKA-2210_2015-07-14_10:02:19.patch, KAFKA-2210_2015-07-14_14:13:19.patch, KAFKA-2210_2015-07-20_16:42:18.patch, KAFKA-2210_2015-07-21_17:08:21.patch, KAFKA-2210_2015-08-10_18:31:54.patch This is the first subtask for Kafka-1688. As Part of this jira we intend to agree on all the public entities, configs and changes to existing kafka classes to allow pluggable authorizer implementation. Please see KIP-11 https://cwiki.apache.org/confluence/display/KAFKA/KIP-11+-+Authorization+Interface for detailed design. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (KAFKA-2210) KafkaAuthorizer: Add all public entities, config changes and changes to KafkaAPI and kafkaServer to allow pluggable authorizer implementation.
[ https://issues.apache.org/jira/browse/KAFKA-2210?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Parth Brahmbhatt updated KAFKA-2210: Attachment: KAFKA-2210_2015-08-10_18:31:54.patch KafkaAuthorizer: Add all public entities, config changes and changes to KafkaAPI and kafkaServer to allow pluggable authorizer implementation. -- Key: KAFKA-2210 URL: https://issues.apache.org/jira/browse/KAFKA-2210 Project: Kafka Issue Type: Sub-task Components: security Reporter: Parth Brahmbhatt Assignee: Parth Brahmbhatt Fix For: 0.8.3 Attachments: KAFKA-2210.patch, KAFKA-2210_2015-06-03_16:36:11.patch, KAFKA-2210_2015-06-04_16:07:39.patch, KAFKA-2210_2015-07-09_18:00:34.patch, KAFKA-2210_2015-07-14_10:02:19.patch, KAFKA-2210_2015-07-14_14:13:19.patch, KAFKA-2210_2015-07-20_16:42:18.patch, KAFKA-2210_2015-07-21_17:08:21.patch, KAFKA-2210_2015-08-10_18:31:54.patch This is the first subtask for Kafka-1688. As Part of this jira we intend to agree on all the public entities, configs and changes to existing kafka classes to allow pluggable authorizer implementation. Please see KIP-11 https://cwiki.apache.org/confluence/display/KAFKA/KIP-11+-+Authorization+Interface for detailed design. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (KAFKA-1695) Authenticate connection to Zookeeper
[ https://issues.apache.org/jira/browse/KAFKA-1695?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Parth Brahmbhatt updated KAFKA-1695: Status: Patch Available (was: In Progress) Authenticate connection to Zookeeper Key: KAFKA-1695 URL: https://issues.apache.org/jira/browse/KAFKA-1695 Project: Kafka Issue Type: Sub-task Components: security Reporter: Jay Kreps Assignee: Parth Brahmbhatt We need to make it possible to secure the Zookeeper cluster Kafka is using. This would make use of the normal authentication ZooKeeper provides. ZooKeeper supports a variety of authentication mechanisms so we will need to figure out what has to be passed in to the zookeeper client. The intention is that when the current round of client work is done it should be possible to run without clients needing access to Zookeeper so all we need here is to make it so that only the Kafka cluster is able to read and write to the Kafka znodes (we shouldn't need to set any kind of acl on a per-znode basis). -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (KAFKA-2145) An option to add topic owners.
[ https://issues.apache.org/jira/browse/KAFKA-2145?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Parth Brahmbhatt updated KAFKA-2145: Status: Patch Available (was: In Progress) An option to add topic owners. --- Key: KAFKA-2145 URL: https://issues.apache.org/jira/browse/KAFKA-2145 Project: Kafka Issue Type: Improvement Reporter: Parth Brahmbhatt Assignee: Parth Brahmbhatt We need to expose a way so users can identify users/groups that share ownership of topic. We discussed adding this as part of https://issues.apache.org/jira/browse/KAFKA-2035 and agreed that it will be simpler to add owner as a logconfig. The owner field can be used for auditing and also by authorization layer to grant access without having to explicitly configure acls. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (KAFKA-1683) Implement a session concept in the socket server
[ https://issues.apache.org/jira/browse/KAFKA-1683?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14644805#comment-14644805 ] Parth Brahmbhatt commented on KAFKA-1683: - FYI, I relied on this Session object as part of Authorizer design and implementation. Implement a session concept in the socket server -- Key: KAFKA-1683 URL: https://issues.apache.org/jira/browse/KAFKA-1683 Project: Kafka Issue Type: Sub-task Components: security Affects Versions: 0.9.0 Reporter: Jay Kreps Assignee: Gwen Shapira Attachments: KAFKA-1683.patch, KAFKA-1683.patch To implement authentication we need a way to keep track of some things between requests. The initial use for this would be remembering the authenticated user/principle info, but likely more uses would come up (for example we will also need to remember whether and which encryption or integrity measures are in place on the socket so we can wrap and unwrap writes and reads). I was thinking we could just add a Session object that might have a user field. The session object would need to get added to RequestChannel.Request so it is passed down to the API layer with each request. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (KAFKA-1683) Implement a session concept in the socket server
[ https://issues.apache.org/jira/browse/KAFKA-1683?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14644830#comment-14644830 ] Parth Brahmbhatt commented on KAFKA-1683: - I did not include this as part of Authorizer work, I just applied this patch and developed against it. The pull request for authorizer does not have this patch, it assumes it is being reviewed in its own PR and will be merged separately. Implement a session concept in the socket server -- Key: KAFKA-1683 URL: https://issues.apache.org/jira/browse/KAFKA-1683 Project: Kafka Issue Type: Sub-task Components: security Affects Versions: 0.9.0 Reporter: Jay Kreps Assignee: Gwen Shapira Attachments: KAFKA-1683.patch, KAFKA-1683.patch To implement authentication we need a way to keep track of some things between requests. The initial use for this would be remembering the authenticated user/principle info, but likely more uses would come up (for example we will also need to remember whether and which encryption or integrity measures are in place on the socket so we can wrap and unwrap writes and reads). I was thinking we could just add a Session object that might have a user field. The session object would need to get added to RequestChannel.Request so it is passed down to the API layer with each request. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Work started] (KAFKA-1695) Authenticate connection to Zookeeper
[ https://issues.apache.org/jira/browse/KAFKA-1695?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on KAFKA-1695 started by Parth Brahmbhatt. --- Authenticate connection to Zookeeper Key: KAFKA-1695 URL: https://issues.apache.org/jira/browse/KAFKA-1695 Project: Kafka Issue Type: Sub-task Components: security Reporter: Jay Kreps Assignee: Parth Brahmbhatt We need to make it possible to secure the Zookeeper cluster Kafka is using. This would make use of the normal authentication ZooKeeper provides. ZooKeeper supports a variety of authentication mechanisms so we will need to figure out what has to be passed in to the zookeeper client. The intention is that when the current round of client work is done it should be possible to run without clients needing access to Zookeeper so all we need here is to make it so that only the Kafka cluster is able to read and write to the Kafka znodes (we shouldn't need to set any kind of acl on a per-znode basis). -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (KAFKA-2210) KafkaAuthorizer: Add all public entities, config changes and changes to KafkaAPI and kafkaServer to allow pluggable authorizer implementation.
[ https://issues.apache.org/jira/browse/KAFKA-2210?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Parth Brahmbhatt updated KAFKA-2210: Attachment: KAFKA-2210_2015-07-21_17:08:21.patch KafkaAuthorizer: Add all public entities, config changes and changes to KafkaAPI and kafkaServer to allow pluggable authorizer implementation. -- Key: KAFKA-2210 URL: https://issues.apache.org/jira/browse/KAFKA-2210 Project: Kafka Issue Type: Sub-task Components: security Reporter: Parth Brahmbhatt Assignee: Parth Brahmbhatt Attachments: KAFKA-2210.patch, KAFKA-2210_2015-06-03_16:36:11.patch, KAFKA-2210_2015-06-04_16:07:39.patch, KAFKA-2210_2015-07-09_18:00:34.patch, KAFKA-2210_2015-07-14_10:02:19.patch, KAFKA-2210_2015-07-14_14:13:19.patch, KAFKA-2210_2015-07-20_16:42:18.patch, KAFKA-2210_2015-07-21_17:08:21.patch This is the first subtask for Kafka-1688. As Part of this jira we intend to agree on all the public entities, configs and changes to existing kafka classes to allow pluggable authorizer implementation. Please see KIP-11 https://cwiki.apache.org/confluence/display/KAFKA/KIP-11+-+Authorization+Interface for detailed design. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (KAFKA-2210) KafkaAuthorizer: Add all public entities, config changes and changes to KafkaAPI and kafkaServer to allow pluggable authorizer implementation.
[ https://issues.apache.org/jira/browse/KAFKA-2210?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14636051#comment-14636051 ] Parth Brahmbhatt commented on KAFKA-2210: - Updated reviewboard https://reviews.apache.org/r/34492/diff/ against branch origin/trunk KafkaAuthorizer: Add all public entities, config changes and changes to KafkaAPI and kafkaServer to allow pluggable authorizer implementation. -- Key: KAFKA-2210 URL: https://issues.apache.org/jira/browse/KAFKA-2210 Project: Kafka Issue Type: Sub-task Components: security Reporter: Parth Brahmbhatt Assignee: Parth Brahmbhatt Attachments: KAFKA-2210.patch, KAFKA-2210_2015-06-03_16:36:11.patch, KAFKA-2210_2015-06-04_16:07:39.patch, KAFKA-2210_2015-07-09_18:00:34.patch, KAFKA-2210_2015-07-14_10:02:19.patch, KAFKA-2210_2015-07-14_14:13:19.patch, KAFKA-2210_2015-07-20_16:42:18.patch, KAFKA-2210_2015-07-21_17:08:21.patch This is the first subtask for Kafka-1688. As Part of this jira we intend to agree on all the public entities, configs and changes to existing kafka classes to allow pluggable authorizer implementation. Please see KIP-11 https://cwiki.apache.org/confluence/display/KAFKA/KIP-11+-+Authorization+Interface for detailed design. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Assigned] (KAFKA-824) java.lang.NullPointerException in commitOffsets
[ https://issues.apache.org/jira/browse/KAFKA-824?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Parth Brahmbhatt reassigned KAFKA-824: -- Assignee: Parth Brahmbhatt java.lang.NullPointerException in commitOffsets Key: KAFKA-824 URL: https://issues.apache.org/jira/browse/KAFKA-824 Project: Kafka Issue Type: Bug Components: consumer Affects Versions: 0.7.2, 0.8.2.0 Reporter: Yonghui Zhao Assignee: Parth Brahmbhatt Labels: newbie Attachments: ZkClient.0.3.txt, ZkClient.0.4.txt, screenshot-1.jpg Neha Narkhede Yes, I have. Unfortunately, I never quite around to fixing it. My guess is that it is caused due to a race condition between the rebalance thread and the offset commit thread when a rebalance is triggered or the client is being shutdown. Do you mind filing a bug ? 2013/03/25 12:08:32.020 WARN [ZookeeperConsumerConnector] [] 0_lu-ml-test10.bj-1364184411339-7c88f710 exception during commitOffsets java.lang.NullPointerException at org.I0Itec.zkclient.ZkConnection.writeData(ZkConnection.java:111) at org.I0Itec.zkclient.ZkClient$10.call(ZkClient.java:813) at org.I0Itec.zkclient.ZkClient.retryUntilConnected(ZkClient.java:675) at org.I0Itec.zkclient.ZkClient.writeData(ZkClient.java:809) at org.I0Itec.zkclient.ZkClient.writeData(ZkClient.java:777) at kafka.utils.ZkUtils$.updatePersistentPath(ZkUtils.scala:103) at kafka.consumer.ZookeeperConsumerConnector$$anonfun$commitOffsets$2$$anonfun$apply$4.apply(ZookeeperConsumerConnector.scala:251) at kafka.consumer.ZookeeperConsumerConnector$$anonfun$commitOffsets$2$$anonfun$apply$4.apply(ZookeeperConsumerConnector.scala:248) at scala.collection.Iterator$class.foreach(Iterator.scala:631) at scala.collection.JavaConversions$JIteratorWrapper.foreach(JavaConversions.scala:549) at scala.collection.IterableLike$class.foreach(IterableLike.scala:79) at scala.collection.JavaConversions$JCollectionWrapper.foreach(JavaConversions.scala:570) at kafka.consumer.ZookeeperConsumerConnector$$anonfun$commitOffsets$2.apply(ZookeeperConsumerConnector.scala:248) at kafka.consumer.ZookeeperConsumerConnector$$anonfun$commitOffsets$2.apply(ZookeeperConsumerConnector.scala:246) at scala.collection.Iterator$class.foreach(Iterator.scala:631) at kafka.utils.Pool$$anon$1.foreach(Pool.scala:53) at scala.collection.IterableLike$class.foreach(IterableLike.scala:79) at kafka.utils.Pool.foreach(Pool.scala:24) at kafka.consumer.ZookeeperConsumerConnector.commitOffsets(ZookeeperConsumerConnector.scala:246) at kafka.consumer.ZookeeperConsumerConnector.autoCommit(ZookeeperConsumerConnector.scala:232) at kafka.consumer.ZookeeperConsumerConnector$$anonfun$1.apply$mcV$sp(ZookeeperConsumerConnector.scala:126) at kafka.utils.Utils$$anon$2.run(Utils.scala:58) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) at java.util.concurrent.FutureTask$Sync.innerRunAndReset(FutureTask.java:351) at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:178) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:178) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603) at java.lang.Thread.run(Thread.java:722) -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (KAFKA-824) java.lang.NullPointerException in commitOffsets
[ https://issues.apache.org/jira/browse/KAFKA-824?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14633776#comment-14633776 ] Parth Brahmbhatt commented on KAFKA-824: [~techwhizbang] I upgraded to zkClient-0.5 so I will verify this is fixed and update the jira. java.lang.NullPointerException in commitOffsets Key: KAFKA-824 URL: https://issues.apache.org/jira/browse/KAFKA-824 Project: Kafka Issue Type: Bug Components: consumer Affects Versions: 0.7.2, 0.8.2.0 Reporter: Yonghui Zhao Assignee: Parth Brahmbhatt Labels: newbie Attachments: ZkClient.0.3.txt, ZkClient.0.4.txt, screenshot-1.jpg Neha Narkhede Yes, I have. Unfortunately, I never quite around to fixing it. My guess is that it is caused due to a race condition between the rebalance thread and the offset commit thread when a rebalance is triggered or the client is being shutdown. Do you mind filing a bug ? 2013/03/25 12:08:32.020 WARN [ZookeeperConsumerConnector] [] 0_lu-ml-test10.bj-1364184411339-7c88f710 exception during commitOffsets java.lang.NullPointerException at org.I0Itec.zkclient.ZkConnection.writeData(ZkConnection.java:111) at org.I0Itec.zkclient.ZkClient$10.call(ZkClient.java:813) at org.I0Itec.zkclient.ZkClient.retryUntilConnected(ZkClient.java:675) at org.I0Itec.zkclient.ZkClient.writeData(ZkClient.java:809) at org.I0Itec.zkclient.ZkClient.writeData(ZkClient.java:777) at kafka.utils.ZkUtils$.updatePersistentPath(ZkUtils.scala:103) at kafka.consumer.ZookeeperConsumerConnector$$anonfun$commitOffsets$2$$anonfun$apply$4.apply(ZookeeperConsumerConnector.scala:251) at kafka.consumer.ZookeeperConsumerConnector$$anonfun$commitOffsets$2$$anonfun$apply$4.apply(ZookeeperConsumerConnector.scala:248) at scala.collection.Iterator$class.foreach(Iterator.scala:631) at scala.collection.JavaConversions$JIteratorWrapper.foreach(JavaConversions.scala:549) at scala.collection.IterableLike$class.foreach(IterableLike.scala:79) at scala.collection.JavaConversions$JCollectionWrapper.foreach(JavaConversions.scala:570) at kafka.consumer.ZookeeperConsumerConnector$$anonfun$commitOffsets$2.apply(ZookeeperConsumerConnector.scala:248) at kafka.consumer.ZookeeperConsumerConnector$$anonfun$commitOffsets$2.apply(ZookeeperConsumerConnector.scala:246) at scala.collection.Iterator$class.foreach(Iterator.scala:631) at kafka.utils.Pool$$anon$1.foreach(Pool.scala:53) at scala.collection.IterableLike$class.foreach(IterableLike.scala:79) at kafka.utils.Pool.foreach(Pool.scala:24) at kafka.consumer.ZookeeperConsumerConnector.commitOffsets(ZookeeperConsumerConnector.scala:246) at kafka.consumer.ZookeeperConsumerConnector.autoCommit(ZookeeperConsumerConnector.scala:232) at kafka.consumer.ZookeeperConsumerConnector$$anonfun$1.apply$mcV$sp(ZookeeperConsumerConnector.scala:126) at kafka.utils.Utils$$anon$2.run(Utils.scala:58) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) at java.util.concurrent.FutureTask$Sync.innerRunAndReset(FutureTask.java:351) at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:178) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:178) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603) at java.lang.Thread.run(Thread.java:722) -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (KAFKA-2210) KafkaAuthorizer: Add all public entities, config changes and changes to KafkaAPI and kafkaServer to allow pluggable authorizer implementation.
[ https://issues.apache.org/jira/browse/KAFKA-2210?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14634280#comment-14634280 ] Parth Brahmbhatt commented on KAFKA-2210: - Updated reviewboard https://reviews.apache.org/r/34492/diff/ against branch origin/trunk KafkaAuthorizer: Add all public entities, config changes and changes to KafkaAPI and kafkaServer to allow pluggable authorizer implementation. -- Key: KAFKA-2210 URL: https://issues.apache.org/jira/browse/KAFKA-2210 Project: Kafka Issue Type: Sub-task Components: security Reporter: Parth Brahmbhatt Assignee: Parth Brahmbhatt Attachments: KAFKA-2210.patch, KAFKA-2210_2015-06-03_16:36:11.patch, KAFKA-2210_2015-06-04_16:07:39.patch, KAFKA-2210_2015-07-09_18:00:34.patch, KAFKA-2210_2015-07-14_10:02:19.patch, KAFKA-2210_2015-07-14_14:13:19.patch, KAFKA-2210_2015-07-20_16:42:18.patch This is the first subtask for Kafka-1688. As Part of this jira we intend to agree on all the public entities, configs and changes to existing kafka classes to allow pluggable authorizer implementation. Please see KIP-11 https://cwiki.apache.org/confluence/display/KAFKA/KIP-11+-+Authorization+Interface for detailed design. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (KAFKA-2210) KafkaAuthorizer: Add all public entities, config changes and changes to KafkaAPI and kafkaServer to allow pluggable authorizer implementation.
[ https://issues.apache.org/jira/browse/KAFKA-2210?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Parth Brahmbhatt updated KAFKA-2210: Attachment: KAFKA-2210_2015-07-20_16:42:18.patch KafkaAuthorizer: Add all public entities, config changes and changes to KafkaAPI and kafkaServer to allow pluggable authorizer implementation. -- Key: KAFKA-2210 URL: https://issues.apache.org/jira/browse/KAFKA-2210 Project: Kafka Issue Type: Sub-task Components: security Reporter: Parth Brahmbhatt Assignee: Parth Brahmbhatt Attachments: KAFKA-2210.patch, KAFKA-2210_2015-06-03_16:36:11.patch, KAFKA-2210_2015-06-04_16:07:39.patch, KAFKA-2210_2015-07-09_18:00:34.patch, KAFKA-2210_2015-07-14_10:02:19.patch, KAFKA-2210_2015-07-14_14:13:19.patch, KAFKA-2210_2015-07-20_16:42:18.patch This is the first subtask for Kafka-1688. As Part of this jira we intend to agree on all the public entities, configs and changes to existing kafka classes to allow pluggable authorizer implementation. Please see KIP-11 https://cwiki.apache.org/confluence/display/KAFKA/KIP-11+-+Authorization+Interface for detailed design. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (KAFKA-2210) KafkaAuthorizer: Add all public entities, config changes and changes to KafkaAPI and kafkaServer to allow pluggable authorizer implementation.
[ https://issues.apache.org/jira/browse/KAFKA-2210?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Parth Brahmbhatt updated KAFKA-2210: Status: Patch Available (was: In Progress) KafkaAuthorizer: Add all public entities, config changes and changes to KafkaAPI and kafkaServer to allow pluggable authorizer implementation. -- Key: KAFKA-2210 URL: https://issues.apache.org/jira/browse/KAFKA-2210 Project: Kafka Issue Type: Sub-task Components: security Reporter: Parth Brahmbhatt Assignee: Parth Brahmbhatt Attachments: KAFKA-2210.patch, KAFKA-2210_2015-06-03_16:36:11.patch, KAFKA-2210_2015-06-04_16:07:39.patch, KAFKA-2210_2015-07-09_18:00:34.patch, KAFKA-2210_2015-07-14_10:02:19.patch, KAFKA-2210_2015-07-14_14:13:19.patch, KAFKA-2210_2015-07-20_16:42:18.patch This is the first subtask for Kafka-1688. As Part of this jira we intend to agree on all the public entities, configs and changes to existing kafka classes to allow pluggable authorizer implementation. Please see KIP-11 https://cwiki.apache.org/confluence/display/KAFKA/KIP-11+-+Authorization+Interface for detailed design. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (KAFKA-824) java.lang.NullPointerException in commitOffsets
[ https://issues.apache.org/jira/browse/KAFKA-824?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14634327#comment-14634327 ] Parth Brahmbhatt commented on KAFKA-824: [~junrao] Looked at the fix code and ensure that it was part of 0.5 release. The fix also had a unit test so I believe this is ok to resolve. java.lang.NullPointerException in commitOffsets Key: KAFKA-824 URL: https://issues.apache.org/jira/browse/KAFKA-824 Project: Kafka Issue Type: Bug Components: consumer Affects Versions: 0.7.2, 0.8.2.0 Reporter: Yonghui Zhao Assignee: Parth Brahmbhatt Labels: newbie Attachments: ZkClient.0.3.txt, ZkClient.0.4.txt, screenshot-1.jpg Neha Narkhede Yes, I have. Unfortunately, I never quite around to fixing it. My guess is that it is caused due to a race condition between the rebalance thread and the offset commit thread when a rebalance is triggered or the client is being shutdown. Do you mind filing a bug ? 2013/03/25 12:08:32.020 WARN [ZookeeperConsumerConnector] [] 0_lu-ml-test10.bj-1364184411339-7c88f710 exception during commitOffsets java.lang.NullPointerException at org.I0Itec.zkclient.ZkConnection.writeData(ZkConnection.java:111) at org.I0Itec.zkclient.ZkClient$10.call(ZkClient.java:813) at org.I0Itec.zkclient.ZkClient.retryUntilConnected(ZkClient.java:675) at org.I0Itec.zkclient.ZkClient.writeData(ZkClient.java:809) at org.I0Itec.zkclient.ZkClient.writeData(ZkClient.java:777) at kafka.utils.ZkUtils$.updatePersistentPath(ZkUtils.scala:103) at kafka.consumer.ZookeeperConsumerConnector$$anonfun$commitOffsets$2$$anonfun$apply$4.apply(ZookeeperConsumerConnector.scala:251) at kafka.consumer.ZookeeperConsumerConnector$$anonfun$commitOffsets$2$$anonfun$apply$4.apply(ZookeeperConsumerConnector.scala:248) at scala.collection.Iterator$class.foreach(Iterator.scala:631) at scala.collection.JavaConversions$JIteratorWrapper.foreach(JavaConversions.scala:549) at scala.collection.IterableLike$class.foreach(IterableLike.scala:79) at scala.collection.JavaConversions$JCollectionWrapper.foreach(JavaConversions.scala:570) at kafka.consumer.ZookeeperConsumerConnector$$anonfun$commitOffsets$2.apply(ZookeeperConsumerConnector.scala:248) at kafka.consumer.ZookeeperConsumerConnector$$anonfun$commitOffsets$2.apply(ZookeeperConsumerConnector.scala:246) at scala.collection.Iterator$class.foreach(Iterator.scala:631) at kafka.utils.Pool$$anon$1.foreach(Pool.scala:53) at scala.collection.IterableLike$class.foreach(IterableLike.scala:79) at kafka.utils.Pool.foreach(Pool.scala:24) at kafka.consumer.ZookeeperConsumerConnector.commitOffsets(ZookeeperConsumerConnector.scala:246) at kafka.consumer.ZookeeperConsumerConnector.autoCommit(ZookeeperConsumerConnector.scala:232) at kafka.consumer.ZookeeperConsumerConnector$$anonfun$1.apply$mcV$sp(ZookeeperConsumerConnector.scala:126) at kafka.utils.Utils$$anon$2.run(Utils.scala:58) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) at java.util.concurrent.FutureTask$Sync.innerRunAndReset(FutureTask.java:351) at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:178) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:178) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603) at java.lang.Thread.run(Thread.java:722) -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (KAFKA-2032) ConsumerConfig doesn't validate partition.assignment.strategy values
[ https://issues.apache.org/jira/browse/KAFKA-2032?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14630102#comment-14630102 ] Parth Brahmbhatt commented on KAFKA-2032: - [~guozhang] The last review request was incorrectly attached to this jira Please review https://reviews.apache.org/r/32251/diff/5/ and apologies for the confusion. ConsumerConfig doesn't validate partition.assignment.strategy values Key: KAFKA-2032 URL: https://issues.apache.org/jira/browse/KAFKA-2032 Project: Kafka Issue Type: Bug Affects Versions: 0.8.1.2 Reporter: Jason Rosenberg Assignee: Parth Brahmbhatt Attachments: KAFKA-2032.patch, KAFKA-2032.patch, KAFKA-2032_2015-03-19_11:42:07.patch, KAFKA-2032_2015-03-19_11:44:48.patch, KAFKA-2032_2015-03-19_11:47:24.patch, KAFKA-2032_2015-03-19_12:19:45.patch In the ConsumerConfig class, there are validation checks to make sure that string based configuration properties conform to allowed values. However, this validation appears to be missing for the partition.assignment.strategy. E.g. there is validation for autooffset.reset and offsets.storage. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (KAFKA-2210) KafkaAuthorizer: Add all public entities, config changes and changes to KafkaAPI and kafkaServer to allow pluggable authorizer implementation.
[ https://issues.apache.org/jira/browse/KAFKA-2210?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14626663#comment-14626663 ] Parth Brahmbhatt commented on KAFKA-2210: - Updated reviewboard https://reviews.apache.org/r/34492/diff/ against branch origin/trunk KafkaAuthorizer: Add all public entities, config changes and changes to KafkaAPI and kafkaServer to allow pluggable authorizer implementation. -- Key: KAFKA-2210 URL: https://issues.apache.org/jira/browse/KAFKA-2210 Project: Kafka Issue Type: Sub-task Components: security Reporter: Parth Brahmbhatt Assignee: Parth Brahmbhatt Attachments: KAFKA-2210.patch, KAFKA-2210_2015-06-03_16:36:11.patch, KAFKA-2210_2015-06-04_16:07:39.patch, KAFKA-2210_2015-07-09_18:00:34.patch, KAFKA-2210_2015-07-14_10:02:19.patch This is the first subtask for Kafka-1688. As Part of this jira we intend to agree on all the public entities, configs and changes to existing kafka classes to allow pluggable authorizer implementation. Please see KIP-11 https://cwiki.apache.org/confluence/display/KAFKA/KIP-11+-+Authorization+Interface for detailed design. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (KAFKA-2210) KafkaAuthorizer: Add all public entities, config changes and changes to KafkaAPI and kafkaServer to allow pluggable authorizer implementation.
[ https://issues.apache.org/jira/browse/KAFKA-2210?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Parth Brahmbhatt updated KAFKA-2210: Attachment: KAFKA-2210_2015-07-14_10:02:19.patch KafkaAuthorizer: Add all public entities, config changes and changes to KafkaAPI and kafkaServer to allow pluggable authorizer implementation. -- Key: KAFKA-2210 URL: https://issues.apache.org/jira/browse/KAFKA-2210 Project: Kafka Issue Type: Sub-task Components: security Reporter: Parth Brahmbhatt Assignee: Parth Brahmbhatt Attachments: KAFKA-2210.patch, KAFKA-2210_2015-06-03_16:36:11.patch, KAFKA-2210_2015-06-04_16:07:39.patch, KAFKA-2210_2015-07-09_18:00:34.patch, KAFKA-2210_2015-07-14_10:02:19.patch This is the first subtask for Kafka-1688. As Part of this jira we intend to agree on all the public entities, configs and changes to existing kafka classes to allow pluggable authorizer implementation. Please see KIP-11 https://cwiki.apache.org/confluence/display/KAFKA/KIP-11+-+Authorization+Interface for detailed design. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (KAFKA-2210) KafkaAuthorizer: Add all public entities, config changes and changes to KafkaAPI and kafkaServer to allow pluggable authorizer implementation.
[ https://issues.apache.org/jira/browse/KAFKA-2210?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14627049#comment-14627049 ] Parth Brahmbhatt commented on KAFKA-2210: - Updated reviewboard https://reviews.apache.org/r/34492/diff/ against branch origin/trunk KafkaAuthorizer: Add all public entities, config changes and changes to KafkaAPI and kafkaServer to allow pluggable authorizer implementation. -- Key: KAFKA-2210 URL: https://issues.apache.org/jira/browse/KAFKA-2210 Project: Kafka Issue Type: Sub-task Components: security Reporter: Parth Brahmbhatt Assignee: Parth Brahmbhatt Attachments: KAFKA-2210.patch, KAFKA-2210_2015-06-03_16:36:11.patch, KAFKA-2210_2015-06-04_16:07:39.patch, KAFKA-2210_2015-07-09_18:00:34.patch, KAFKA-2210_2015-07-14_10:02:19.patch, KAFKA-2210_2015-07-14_14:13:19.patch This is the first subtask for Kafka-1688. As Part of this jira we intend to agree on all the public entities, configs and changes to existing kafka classes to allow pluggable authorizer implementation. Please see KIP-11 https://cwiki.apache.org/confluence/display/KAFKA/KIP-11+-+Authorization+Interface for detailed design. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (KAFKA-2210) KafkaAuthorizer: Add all public entities, config changes and changes to KafkaAPI and kafkaServer to allow pluggable authorizer implementation.
[ https://issues.apache.org/jira/browse/KAFKA-2210?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Parth Brahmbhatt updated KAFKA-2210: Attachment: KAFKA-2210_2015-07-14_14:13:19.patch KafkaAuthorizer: Add all public entities, config changes and changes to KafkaAPI and kafkaServer to allow pluggable authorizer implementation. -- Key: KAFKA-2210 URL: https://issues.apache.org/jira/browse/KAFKA-2210 Project: Kafka Issue Type: Sub-task Components: security Reporter: Parth Brahmbhatt Assignee: Parth Brahmbhatt Attachments: KAFKA-2210.patch, KAFKA-2210_2015-06-03_16:36:11.patch, KAFKA-2210_2015-06-04_16:07:39.patch, KAFKA-2210_2015-07-09_18:00:34.patch, KAFKA-2210_2015-07-14_10:02:19.patch, KAFKA-2210_2015-07-14_14:13:19.patch This is the first subtask for Kafka-1688. As Part of this jira we intend to agree on all the public entities, configs and changes to existing kafka classes to allow pluggable authorizer implementation. Please see KIP-11 https://cwiki.apache.org/confluence/display/KAFKA/KIP-11+-+Authorization+Interface for detailed design. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Work started] (KAFKA-2145) An option to add topic owners.
[ https://issues.apache.org/jira/browse/KAFKA-2145?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on KAFKA-2145 started by Parth Brahmbhatt. --- An option to add topic owners. --- Key: KAFKA-2145 URL: https://issues.apache.org/jira/browse/KAFKA-2145 Project: Kafka Issue Type: Improvement Reporter: Parth Brahmbhatt Assignee: Parth Brahmbhatt We need to expose a way so users can identify users/groups that share ownership of topic. We discussed adding this as part of https://issues.apache.org/jira/browse/KAFKA-2035 and agreed that it will be simpler to add owner as a logconfig. The owner field can be used for auditing and also by authorization layer to grant access without having to explicitly configure acls. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (KAFKA-2162) Kafka Auditing functionality
[ https://issues.apache.org/jira/browse/KAFKA-2162?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14625176#comment-14625176 ] Parth Brahmbhatt commented on KAFKA-2162: - [~gwenshap] [~harsha_ch] I don't completely agree with need to audit session/connection establishments and termination. In a secure system with authorization, connecting/establishing a session with a server does not buy a client anything unless they have authorization on operations so auditing those events don't seem to be useful to me. Ddosing based on authentication seems a different story and I don't think auditing can really help much in that situation , we should rather rely on quotas to prevent something like that from happening to begin with. Ticket renewals: Given the server is going to use keytabs (or should use keytabs) I think this is also not very useful but I know very little about kerberos and it never seizes to surprise me so may be we do need this. If we want to audit anything more than the authorizer operations we will have to provide a pluggable auditor just like authorizer which means another config and another interface. Kafka Auditing functionality Key: KAFKA-2162 URL: https://issues.apache.org/jira/browse/KAFKA-2162 Project: Kafka Issue Type: Bug Reporter: Sriharsha Chintalapani Assignee: Parth Brahmbhatt During Kafka authorization discussion thread . There was concerns raised about not having Auditing. Auditing is important functionality but its not part of authorizer. This jira will track adding audit functionality to kafka. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (KAFKA-2145) An option to add topic owners.
[ https://issues.apache.org/jira/browse/KAFKA-2145?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14624975#comment-14624975 ] Parth Brahmbhatt commented on KAFKA-2145: - @singhashish Given I am the original reporter I have some context on this. Do you mind if I take this one over? I have a few more jiras assigned to me most of which are blocked for one reason or another so I have some time that I could allocate to this one. An option to add topic owners. --- Key: KAFKA-2145 URL: https://issues.apache.org/jira/browse/KAFKA-2145 Project: Kafka Issue Type: Improvement Reporter: Parth Brahmbhatt Assignee: Ashish K Singh We need to expose a way so users can identify users/groups that share ownership of topic. We discussed adding this as part of https://issues.apache.org/jira/browse/KAFKA-2035 and agreed that it will be simpler to add owner as a logconfig. The owner field can be used for auditing and also by authorization layer to grant access without having to explicitly configure acls. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (KAFKA-1788) producer record can stay in RecordAccumulator forever if leader is no available
[ https://issues.apache.org/jira/browse/KAFKA-1788?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14624873#comment-14624873 ] Parth Brahmbhatt commented on KAFKA-1788: - [~becket_qin] So is this jira irrelavant at this point? If yes can I resolve it? If no, can you describe what needs to be done? I know you had a KIP and multiple discussions but I am not sure if you are taking of it as part of KAFKA-2142. I will be happy to continue working on this jira if you can describe what needs to be done. producer record can stay in RecordAccumulator forever if leader is no available --- Key: KAFKA-1788 URL: https://issues.apache.org/jira/browse/KAFKA-1788 Project: Kafka Issue Type: Bug Components: core, producer Affects Versions: 0.8.2.0 Reporter: Jun Rao Assignee: Parth Brahmbhatt Labels: newbie++ Fix For: 0.8.3 Attachments: KAFKA-1788.patch, KAFKA-1788_2015-01-06_13:42:37.patch, KAFKA-1788_2015-01-06_13:44:41.patch In the new producer, when a partition has no leader for a long time (e.g., all replicas are down), the records for that partition will stay in the RecordAccumulator until the leader is available. This may cause the bufferpool to be full and the callback for the produced message to block for a long time. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Comment Edited] (KAFKA-2145) An option to add topic owners.
[ https://issues.apache.org/jira/browse/KAFKA-2145?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14624975#comment-14624975 ] Parth Brahmbhatt edited comment on KAFKA-2145 at 7/13/15 5:18 PM: -- [~singhashish] Given I am the original reporter I have some context on this. Do you mind if I take this one over? I have a few more jiras assigned to me most of which are blocked for one reason or another so I have some time that I could allocate to this one. was (Author: parth.brahmbhatt): @singhashish Given I am the original reporter I have some context on this. Do you mind if I take this one over? I have a few more jiras assigned to me most of which are blocked for one reason or another so I have some time that I could allocate to this one. An option to add topic owners. --- Key: KAFKA-2145 URL: https://issues.apache.org/jira/browse/KAFKA-2145 Project: Kafka Issue Type: Improvement Reporter: Parth Brahmbhatt Assignee: Ashish K Singh We need to expose a way so users can identify users/groups that share ownership of topic. We discussed adding this as part of https://issues.apache.org/jira/browse/KAFKA-2035 and agreed that it will be simpler to add owner as a logconfig. The owner field can be used for auditing and also by authorization layer to grant access without having to explicitly configure acls. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Assigned] (KAFKA-2145) An option to add topic owners.
[ https://issues.apache.org/jira/browse/KAFKA-2145?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Parth Brahmbhatt reassigned KAFKA-2145: --- Assignee: Parth Brahmbhatt An option to add topic owners. --- Key: KAFKA-2145 URL: https://issues.apache.org/jira/browse/KAFKA-2145 Project: Kafka Issue Type: Improvement Reporter: Parth Brahmbhatt Assignee: Parth Brahmbhatt We need to expose a way so users can identify users/groups that share ownership of topic. We discussed adding this as part of https://issues.apache.org/jira/browse/KAFKA-2035 and agreed that it will be simpler to add owner as a logconfig. The owner field can be used for auditing and also by authorization layer to grant access without having to explicitly configure acls. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (KAFKA-2182) zkClient dies if there is any exception while reconnecting
[
https://issues.apache.org/jira/browse/KAFKA-2182?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14623077#comment-14623077
]
Parth Brahmbhatt commented on KAFKA-2182:
-
[~junrao] I think I took care of this as part of KAFKA-2169. We now just
system.exit when this exception is caught at least on the borker side. Can we
close this jira as Fixed? Or am I missing the intent of this jira?
zkClient dies if there is any exception while reconnecting
--
Key: KAFKA-2182
URL: https://issues.apache.org/jira/browse/KAFKA-2182
Project: Kafka
Issue Type: Bug
Components: core
Affects Versions: 0.8.1
Reporter: Igor Maravić
Assignee: Parth Brahmbhatt
Priority: Critical
We, Spotify, have just been hit by a BUG that's related to ZkClient. It made
a whole Kafka cluster go down.
Long story short, we've restarted TOR switch and all of our brokers from the
cluster lost all the connectivity with the zookeeper cluster, which was
living in another rack. Because of that, all the connections to Zookeeper got
expired.
Everything would be fine if we haven't lost hostname to IP Address mapping
for some reason. Since we did lost that mapping, we got a
UnknownHostException when the broker tried to reconnect. This exception got
swallowed up, and we never got reconnected to Zookeeper, which in turn made
our cluster of brokers useless.
If we had handleSessionEstablishmentError function, the whole exception
could be caught, we could just completely kill KafkaServer process and start
it cleanly, since this kind of exception is fatal for the KafkaCluster.
{code}
2015-05-05T12:49:01.709+00:00 127.0.0.1 apache-kafka[main-EventThread] INFO
zookeeper.ZooKeeper - Initiating client connection,
connectString=zookeeper1.spotify.net:2181,zookeeper2.spotify.net:2181,zookeeper3.spotify.net:2181/gabobroker-local
sessionTimeout=6000 watcher=org.I0Itec.zkclient.ZkClient@7303d690
2015-05-05T12:49:01.711+00:00 127.0.0.1 apache-kafka[main-EventThread] ERROR
zookeeper.ClientCnxn - Error while calling watcher
2015-05-05T12:49:01.711+00:00 127.0.0.1 java.lang.RuntimeException: Exception
while restarting zk client
2015-05-05T12:49:01.711+00:00 127.0.0.1 at
org.I0Itec.zkclient.ZkClient.processStateChanged(ZkClient.java:462)
2015-05-05T12:49:01.711+00:00 127.0.0.1 at
org.I0Itec.zkclient.ZkClient.process(ZkClient.java:368)
2015-05-05T12:49:01.711+00:00 127.0.0.1 at
org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:522)
2015-05-05T12:49:01.711+00:00 127.0.0.1 at
org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:498)
2015-05-05T12:49:01.711+00:00 127.0.0.1 Caused by:
org.I0Itec.zkclient.exception.ZkException: Unable to connect to
zookeeper1.spotify.net:2181,zookeeper2.spotify.net:2181,zookeeper3.spotify.net:2181/gabobroker-local
2015-05-05T12:49:01.711+00:00 127.0.0.1 at
org.I0Itec.zkclient.ZkConnection.connect(ZkConnection.java:66)
2015-05-05T12:49:01.711+00:00 127.0.0.1 at
org.I0Itec.zkclient.ZkClient.reconnect(ZkClient.java:939)
2015-05-05T12:49:01.711+00:00 127.0.0.1 at
org.I0Itec.zkclient.ZkClient.processStateChanged(ZkClient.java:458)
2015-05-05T12:49:01.711+00:00 127.0.0.1 ... 3 more
2015-05-05T12:49:01.712+00:00 127.0.0.1 Caused by:
java.net.UnknownHostException: zookeeper1.spotify.net: Name or service not
known
2015-05-05T12:49:01.712+00:00 127.0.0.1 at
java.net.Inet6AddressImpl.lookupAllHostAddr(Native Method)
2015-05-05T12:49:01.712+00:00 127.0.0.1 at
java.net.InetAddress$1.lookupAllHostAddr(InetAddress.java:901)
2015-05-05T12:49:01.712+00:00 127.0.0.1 at
java.net.InetAddress.getAddressesFromNameService(InetAddress.java:1293)
2015-05-05T12:49:01.712+00:00 127.0.0.1 at
java.net.InetAddress.getAllByName0(InetAddress.java:1246)
2015-05-05T12:49:01.712+00:00 127.0.0.1 at
java.net.InetAddress.getAllByName(InetAddress.java:1162)
2015-05-05T12:49:01.712+00:00 127.0.0.1 at
java.net.InetAddress.getAllByName(InetAddress.java:1098)
2015-05-05T12:49:01.712+00:00 127.0.0.1 at
org.apache.zookeeper.client.StaticHostProvider.init(StaticHostProvider.java:61)
2015-05-05T12:49:01.712+00:00 127.0.0.1 at
org.apache.zookeeper.ZooKeeper.init(ZooKeeper.java:445)
2015-05-05T12:49:01.712+00:00 127.0.0.1 at
org.apache.zookeeper.ZooKeeper.init(ZooKeeper.java:380)
2015-05-05T12:49:01.713+00:00 127.0.0.1 at
org.I0Itec.zkclient.ZkConnection.connect(ZkConnection.java:64)
2015-05-05T12:49:01.713+00:00 127.0.0.1 ... 5 more
2015-05-05T12:49:01.713+00:00 127.0.0.1
apache-kafka[ZkClient-EventThread-18-zookeeper1.spotify.net:2181,zookeeper2.spotify.net:2181,zookeeper3.spotify.net:2181/gabobroker-local]
ERROR zkclient.ZkEventThread - Error handling event ZkEvent[Children of
/config/changes
[jira] [Commented] (KAFKA-2145) An option to add topic owners.
[ https://issues.apache.org/jira/browse/KAFKA-2145?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14623106#comment-14623106 ] Parth Brahmbhatt commented on KAFKA-2145: - [~neelesh77] sorry for the delay on this 1) The objective is to have a Topic Owner field associated with the topic. Shall this be associated with the topic at all times? Not sure what do you mean by Shall this be associated with the topic at all times. The intent is to add a topic.config called owner which can either be a a string or a list of string (comma separated string) which is filled with the value of user executing the create topic. This user could then add other users as co-owners in the same list. 2) TopicCommand should be updated to reflect the owner field. An option to add topic owners. --- Key: KAFKA-2145 URL: https://issues.apache.org/jira/browse/KAFKA-2145 Project: Kafka Issue Type: Improvement Reporter: Parth Brahmbhatt Assignee: Neelesh Srinivas Salian We need to expose a way so users can identify users/groups that share ownership of topic. We discussed adding this as part of https://issues.apache.org/jira/browse/KAFKA-2035 and agreed that it will be simpler to add owner as a logconfig. The owner field can be used for auditing and also by authorization layer to grant access without having to explicitly configure acls. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
