[jira] [Commented] (KAFKA-2920) Storage module
[ https://issues.apache.org/jira/browse/KAFKA-2920?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15056278#comment-15056278 ] Andrii Biletskyi commented on KAFKA-2920: - The idea is to implement plug-gable interface of the Storage module defined by https://cwiki.apache.org/confluence/display/KAFKA/KIP-30+-+Allow+for+brokers+to+have+plug-able+consensus+and+meta+data+storage+sub+systems (got to Storage tab). Current zkUtils should be transformed so it becomes interface implementation. There are some unclear things though. Like what exception contract we need to follow. E.g. when we are trying to update value that doesn't exist or insert value that already exists. > Storage module > -- > > Key: KAFKA-2920 > URL: https://issues.apache.org/jira/browse/KAFKA-2920 > Project: Kafka > Issue Type: Sub-task > Components: core >Reporter: Andrii Biletskyi > > Add Storage module and implement it on top of Zookeeper as a default version. > Replace ZkUtils calls with calls to Storage interface. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (KAFKA-2921) Plug-able implementations support
[ https://issues.apache.org/jira/browse/KAFKA-2921?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrii Biletskyi updated KAFKA-2921: Attachment: KIP-30-LE-WIP.patch > Plug-able implementations support > - > > Key: KAFKA-2921 > URL: https://issues.apache.org/jira/browse/KAFKA-2921 > Project: Kafka > Issue Type: Sub-task > Components: core >Reporter: Andrii Biletskyi > Attachments: KIP-30-LE-WIP.patch > > > Add infrastructure to support plug-able implementations in runtime. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (KAFKA-2916) KIP-30 Plug-able interface for consensus and storage sub-systems
Andrii Biletskyi created KAFKA-2916: --- Summary: KIP-30 Plug-able interface for consensus and storage sub-systems Key: KAFKA-2916 URL: https://issues.apache.org/jira/browse/KAFKA-2916 Project: Kafka Issue Type: New Feature Components: core Reporter: Andrii Biletskyi Checklist for KIP-30 (https://cwiki.apache.org/confluence/display/KAFKA/KIP-30+-+Allow+for+brokers+to+have+plug-able+consensus+and+meta+data+storage+sub+systems) -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (KAFKA-2918) GroupMembership module
Andrii Biletskyi created KAFKA-2918: --- Summary: GroupMembership module Key: KAFKA-2918 URL: https://issues.apache.org/jira/browse/KAFKA-2918 Project: Kafka Issue Type: Sub-task Reporter: Andrii Biletskyi Add GroupMembership interface and implement it on top of Zookeeper as a default version. Integrate implementation for: a) cluster (brokers) membership b) consumer group (High Level Consumer) membership -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (KAFKA-2917) LeaderElection module
Andrii Biletskyi created KAFKA-2917: --- Summary: LeaderElection module Key: KAFKA-2917 URL: https://issues.apache.org/jira/browse/KAFKA-2917 Project: Kafka Issue Type: Sub-task Reporter: Andrii Biletskyi Add LeaderElection interface and turn current ZookeeperLeaderElector into a default implementation -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (KAFKA-2919) ListenerRegistry
Andrii Biletskyi created KAFKA-2919: --- Summary: ListenerRegistry Key: KAFKA-2919 URL: https://issues.apache.org/jira/browse/KAFKA-2919 Project: Kafka Issue Type: Sub-task Reporter: Andrii Biletskyi Add ListenerRegistry interface and implement it on top of Zookeeper as a default version. Integrate it into Kafka instead of zookeeper data watchers. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (KAFKA-2920) Storage module
Andrii Biletskyi created KAFKA-2920: --- Summary: Storage module Key: KAFKA-2920 URL: https://issues.apache.org/jira/browse/KAFKA-2920 Project: Kafka Issue Type: Sub-task Reporter: Andrii Biletskyi Add Storage module and implement it on top of Zookeeper as a default version. Replace ZkUtils calls with calls to Storage interface. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (KAFKA-2921) Plug-able implementations support
Andrii Biletskyi created KAFKA-2921: --- Summary: Plug-able implementations support Key: KAFKA-2921 URL: https://issues.apache.org/jira/browse/KAFKA-2921 Project: Kafka Issue Type: Sub-task Reporter: Andrii Biletskyi Add infrastructure to support plug-able implementations in runtime. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (KAFKA-2922) System and Replication tools verification
Andrii Biletskyi created KAFKA-2922: --- Summary: System and Replication tools verification Key: KAFKA-2922 URL: https://issues.apache.org/jira/browse/KAFKA-2922 Project: Kafka Issue Type: Sub-task Reporter: Andrii Biletskyi Go through System (https://cwiki.apache.org/confluence/display/KAFKA/System+Tools) and Replication tools (https://cwiki.apache.org/confluence/display/KAFKA/Replication+tools) to verify (update/deprecate) which tools need to be abstracted from Zookeeper as the exclusive storage sub-system for Kafka. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (KAFKA-1694) kafka command line and centralized operations
[ https://issues.apache.org/jira/browse/KAFKA-1694?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14984933#comment-14984933 ] Andrii Biletskyi commented on KAFKA-1694: - [~granthenke], it was decided to split the work into 3 phases: api, admin client, cli. The Phase 1 was implemented under KAFKA-2229, the patch was moved to github (https://github.com/apache/kafka/pull/223). There were some minor comments under this pull request, they were fixed, though not rebased. IMO it lacks some deeper review and maybe testing. In short, you can pickup this ticket. I'm happy to help to close this issue asap too. If anything is needed (i.e. rebase) let me know. > kafka command line and centralized operations > - > > Key: KAFKA-1694 > URL: https://issues.apache.org/jira/browse/KAFKA-1694 > Project: Kafka > Issue Type: Bug >Reporter: Joe Stein >Assignee: Andrii Biletskyi >Priority: Critical > Attachments: KAFKA-1694.patch, KAFKA-1694_2014-12-24_21:21:51.patch, > KAFKA-1694_2015-01-12_15:28:41.patch, KAFKA-1694_2015-01-12_18:54:48.patch, > KAFKA-1694_2015-01-13_19:30:11.patch, KAFKA-1694_2015-01-14_15:42:12.patch, > KAFKA-1694_2015-01-14_18:07:39.patch, KAFKA-1694_2015-03-12_13:04:37.patch, > KAFKA-1772_1802_1775_1774_v2.patch > > > https://cwiki.apache.org/confluence/display/KAFKA/Kafka+Command+Line+and+Related+Improvements -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (KAFKA-2702) ConfigDef toHtmlTable() sorts in a way that is a bit confusing
[ https://issues.apache.org/jira/browse/KAFKA-2702?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14982679#comment-14982679 ] Andrii Biletskyi commented on KAFKA-2702: - It's been a while, but, yes, as far as I remember I added required field because not all configs had default values and we couldn't instantiate Config unless all settings have their value - the default one, or the one came from the user (config file). > ConfigDef toHtmlTable() sorts in a way that is a bit confusing > -- > > Key: KAFKA-2702 > URL: https://issues.apache.org/jira/browse/KAFKA-2702 > Project: Kafka > Issue Type: Bug >Reporter: Gwen Shapira >Assignee: Grant Henke > Attachments: ConsumerConfig-After.html, ConsumerConfig-Before.html > > > Because we put everything without default first (without prioritizing), > critical parameters get placed below low priority ones when they both have > no defaults. Some parameters are without default and optional (SASL server in > ConsumerConfig for instance). > Try printing ConsumerConfig parameters and see the mandatory group.id show up > as #15. > I suggest sorting the no-default parameters by priority as well, or perhaps > adding a "REQUIRED" category that gets printed first no matter what. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (KAFKA-2229) Phase 1: Requests and KafkaApis
[ https://issues.apache.org/jira/browse/KAFKA-2229?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14742109#comment-14742109 ] Andrii Biletskyi commented on KAFKA-2229: - Sorry I was on vacation so didn't have time to work on this issue. I think I resolved all your comments now I need to rebase changes to the trunk and commit corresponding fixes per your comments. It's been a while since last rebase, but still I think it will be quicker if I do it. Your help will be very appreciated in reviewing again after rebase. As I can see there were some changes in replica assignment code which my changes use a lot, this part is pretty critical. Also you could provide more details (or even recommendations) to your comment "This logic is fairly complex and difficult to track. I am not sure the best way to resolve it. Breaking it into smaller functions may help." -> I've updated on it in RB. Thanks, Andrii > Phase 1: Requests and KafkaApis > --- > > Key: KAFKA-2229 > URL: https://issues.apache.org/jira/browse/KAFKA-2229 > Project: Kafka > Issue Type: Sub-task >Reporter: Andrii Biletskyi >Assignee: Andrii Biletskyi > Attachments: KAFKA-2229.patch, KAFKA-2229_2015-06-30_16:59:17.patch > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (KAFKA-2229) Phase 1: Requests and KafkaApis
[ https://issues.apache.org/jira/browse/KAFKA-2229?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14742108#comment-14742108 ] Andrii Biletskyi commented on KAFKA-2229: - Sorry I was on vacation so didn't have time to work on this issue. I think I resolved all your comments now I need to rebase changes to the trunk and commit corresponding fixes per your comments. It's been a while since last rebase, but still I think it will be quicker if I do it. Your help will be very appreciated in reviewing again after rebase. As I can see there were some changes in replica assignment code which my changes use a lot, this part is pretty critical. Also you could provide more details (or even recommendations) to your comment "This logic is fairly complex and difficult to track. I am not sure the best way to resolve it. Breaking it into smaller functions may help." -> I've updated on it in RB. Thanks, Andrii > Phase 1: Requests and KafkaApis > --- > > Key: KAFKA-2229 > URL: https://issues.apache.org/jira/browse/KAFKA-2229 > Project: Kafka > Issue Type: Sub-task >Reporter: Andrii Biletskyi >Assignee: Andrii Biletskyi > Attachments: KAFKA-2229.patch, KAFKA-2229_2015-06-30_16:59:17.patch > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (KAFKA-2310) Add config to prevent broker becoming controller
[ https://issues.apache.org/jira/browse/KAFKA-2310?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14616401#comment-14616401 ] Andrii Biletskyi commented on KAFKA-2310: - [~jjkoshy] yes, it might be a duplicate. Although we didn't agree there what we want to do - force re-elect command or do it via config (as Joe mentioned in his comment). This patch is the implementation of the latter one. Not sure what should we do though - duplicate, link or maybe convert as sub-task of the KAFKA-1778. Add config to prevent broker becoming controller Key: KAFKA-2310 URL: https://issues.apache.org/jira/browse/KAFKA-2310 Project: Kafka Issue Type: Bug Reporter: Andrii Biletskyi Assignee: Andrii Biletskyi Attachments: KAFKA-2310.patch, KAFKA-2310_0.8.1.patch, KAFKA-2310_0.8.2.patch The goal is to be able to specify which cluster brokers can serve as a controller and which cannot. This way it will be possible to reserve particular, not overloaded with partitions and other operations, broker as controller. Proposed to add config _controller.eligibility_ defaulted to true (for backward compatibility, since now any broker can become a controller) Patch will be available for trunk, 0.8.2 and 0.8.1 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (KAFKA-2310) Add config to prevent broker becoming controller
Andrii Biletskyi created KAFKA-2310: --- Summary: Add config to prevent broker becoming controller Key: KAFKA-2310 URL: https://issues.apache.org/jira/browse/KAFKA-2310 Project: Kafka Issue Type: Bug Reporter: Andrii Biletskyi Assignee: Andrii Biletskyi The goal is to be able to specify which cluster brokers can serve as a controller and which cannot. This way it will be possible to reserve particular, not overloaded with partitions and other operations, broker as controller. Proposed to add config _controller.eligibility_ defaulted to true (for backward compatibility, since now any broker can become a controller) Patch will be available for trunk, 0.8.2 and 0.8.1 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (KAFKA-2310) Add config to prevent broker becoming controller
[ https://issues.apache.org/jira/browse/KAFKA-2310?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrii Biletskyi updated KAFKA-2310: Attachment: (was: KAFKA-2310_0.8.2.patch) Add config to prevent broker becoming controller Key: KAFKA-2310 URL: https://issues.apache.org/jira/browse/KAFKA-2310 Project: Kafka Issue Type: Bug Reporter: Andrii Biletskyi Assignee: Andrii Biletskyi Attachments: KAFKA-2310.patch, KAFKA-2310_0.8.1.patch, KAFKA-2310_0.8.2.patch The goal is to be able to specify which cluster brokers can serve as a controller and which cannot. This way it will be possible to reserve particular, not overloaded with partitions and other operations, broker as controller. Proposed to add config _controller.eligibility_ defaulted to true (for backward compatibility, since now any broker can become a controller) Patch will be available for trunk, 0.8.2 and 0.8.1 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (KAFKA-2310) Add config to prevent broker becoming controller
[ https://issues.apache.org/jira/browse/KAFKA-2310?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrii Biletskyi updated KAFKA-2310: Attachment: KAFKA-2310_0.8.2.patch Add config to prevent broker becoming controller Key: KAFKA-2310 URL: https://issues.apache.org/jira/browse/KAFKA-2310 Project: Kafka Issue Type: Bug Reporter: Andrii Biletskyi Assignee: Andrii Biletskyi Attachments: KAFKA-2310.patch, KAFKA-2310_0.8.1.patch, KAFKA-2310_0.8.2.patch The goal is to be able to specify which cluster brokers can serve as a controller and which cannot. This way it will be possible to reserve particular, not overloaded with partitions and other operations, broker as controller. Proposed to add config _controller.eligibility_ defaulted to true (for backward compatibility, since now any broker can become a controller) Patch will be available for trunk, 0.8.2 and 0.8.1 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (KAFKA-2310) Add config to prevent broker becoming controller
[ https://issues.apache.org/jira/browse/KAFKA-2310?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrii Biletskyi updated KAFKA-2310: Attachment: KAFKA-2310_0.8.1.patch Add config to prevent broker becoming controller Key: KAFKA-2310 URL: https://issues.apache.org/jira/browse/KAFKA-2310 Project: Kafka Issue Type: Bug Reporter: Andrii Biletskyi Assignee: Andrii Biletskyi Attachments: KAFKA-2310.patch, KAFKA-2310_0.8.1.patch, KAFKA-2310_0.8.2.patch The goal is to be able to specify which cluster brokers can serve as a controller and which cannot. This way it will be possible to reserve particular, not overloaded with partitions and other operations, broker as controller. Proposed to add config _controller.eligibility_ defaulted to true (for backward compatibility, since now any broker can become a controller) Patch will be available for trunk, 0.8.2 and 0.8.1 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (KAFKA-2310) Add config to prevent broker becoming controller
[ https://issues.apache.org/jira/browse/KAFKA-2310?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrii Biletskyi updated KAFKA-2310: Attachment: KAFKA-2310_0.8.2.patch Add config to prevent broker becoming controller Key: KAFKA-2310 URL: https://issues.apache.org/jira/browse/KAFKA-2310 Project: Kafka Issue Type: Bug Reporter: Andrii Biletskyi Assignee: Andrii Biletskyi Attachments: KAFKA-2310.patch, KAFKA-2310_0.8.2.patch The goal is to be able to specify which cluster brokers can serve as a controller and which cannot. This way it will be possible to reserve particular, not overloaded with partitions and other operations, broker as controller. Proposed to add config _controller.eligibility_ defaulted to true (for backward compatibility, since now any broker can become a controller) Patch will be available for trunk, 0.8.2 and 0.8.1 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (KAFKA-2310) Add config to prevent broker becoming controller
[ https://issues.apache.org/jira/browse/KAFKA-2310?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrii Biletskyi updated KAFKA-2310: Attachment: KAFKA-2310.patch Add config to prevent broker becoming controller Key: KAFKA-2310 URL: https://issues.apache.org/jira/browse/KAFKA-2310 Project: Kafka Issue Type: Bug Reporter: Andrii Biletskyi Assignee: Andrii Biletskyi Attachments: KAFKA-2310.patch The goal is to be able to specify which cluster brokers can serve as a controller and which cannot. This way it will be possible to reserve particular, not overloaded with partitions and other operations, broker as controller. Proposed to add config _controller.eligibility_ defaulted to true (for backward compatibility, since now any broker can become a controller) Patch will be available for trunk, 0.8.2 and 0.8.1 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (KAFKA-2310) Add config to prevent broker becoming controller
[ https://issues.apache.org/jira/browse/KAFKA-2310?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrii Biletskyi updated KAFKA-2310: Status: Patch Available (was: Open) Add config to prevent broker becoming controller Key: KAFKA-2310 URL: https://issues.apache.org/jira/browse/KAFKA-2310 Project: Kafka Issue Type: Bug Reporter: Andrii Biletskyi Assignee: Andrii Biletskyi Attachments: KAFKA-2310.patch The goal is to be able to specify which cluster brokers can serve as a controller and which cannot. This way it will be possible to reserve particular, not overloaded with partitions and other operations, broker as controller. Proposed to add config _controller.eligibility_ defaulted to true (for backward compatibility, since now any broker can become a controller) Patch will be available for trunk, 0.8.2 and 0.8.1 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (KAFKA-2310) Add config to prevent broker becoming controller
[ https://issues.apache.org/jira/browse/KAFKA-2310?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14615111#comment-14615111 ] Andrii Biletskyi commented on KAFKA-2310: - Created reviewboard https://reviews.apache.org/r/36203/diff/ against branch origin/trunk Add config to prevent broker becoming controller Key: KAFKA-2310 URL: https://issues.apache.org/jira/browse/KAFKA-2310 Project: Kafka Issue Type: Bug Reporter: Andrii Biletskyi Assignee: Andrii Biletskyi Attachments: KAFKA-2310.patch The goal is to be able to specify which cluster brokers can serve as a controller and which cannot. This way it will be possible to reserve particular, not overloaded with partitions and other operations, broker as controller. Proposed to add config _controller.eligibility_ defaulted to true (for backward compatibility, since now any broker can become a controller) Patch will be available for trunk, 0.8.2 and 0.8.1 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (KAFKA-2229) Phase 1: Requests and KafkaApis
[ https://issues.apache.org/jira/browse/KAFKA-2229?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrii Biletskyi updated KAFKA-2229: Attachment: KAFKA-2229_2015-06-30_16:59:17.patch Phase 1: Requests and KafkaApis --- Key: KAFKA-2229 URL: https://issues.apache.org/jira/browse/KAFKA-2229 Project: Kafka Issue Type: Sub-task Reporter: Andrii Biletskyi Assignee: Andrii Biletskyi Fix For: 0.8.3 Attachments: KAFKA-2229.patch, KAFKA-2229_2015-06-30_16:59:17.patch -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (KAFKA-2229) Phase 1: Requests and KafkaApis
[ https://issues.apache.org/jira/browse/KAFKA-2229?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14608307#comment-14608307 ] Andrii Biletskyi commented on KAFKA-2229: - Updated reviewboard https://reviews.apache.org/r/34766/diff/ against branch origin/trunk Phase 1: Requests and KafkaApis --- Key: KAFKA-2229 URL: https://issues.apache.org/jira/browse/KAFKA-2229 Project: Kafka Issue Type: Sub-task Reporter: Andrii Biletskyi Assignee: Andrii Biletskyi Fix For: 0.8.3 Attachments: KAFKA-2229.patch, KAFKA-2229_2015-06-30_16:59:17.patch -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (KAFKA-2229) Phase 1: Requests and KafkaApis
[ https://issues.apache.org/jira/browse/KAFKA-2229?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14608309#comment-14608309 ] Andrii Biletskyi commented on KAFKA-2229: - Rebased to include KAFKA-2195. Phase 1: Requests and KafkaApis --- Key: KAFKA-2229 URL: https://issues.apache.org/jira/browse/KAFKA-2229 Project: Kafka Issue Type: Sub-task Reporter: Andrii Biletskyi Assignee: Andrii Biletskyi Fix For: 0.8.3 Attachments: KAFKA-2229.patch, KAFKA-2229_2015-06-30_16:59:17.patch -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (KAFKA-2195) Add versionId to AbstractRequest.getErrorResponse and AbstractRequest.getRequest
[ https://issues.apache.org/jira/browse/KAFKA-2195?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14588607#comment-14588607 ] Andrii Biletskyi commented on KAFKA-2195: - Updated reviewboard https://reviews.apache.org/r/34415/diff/ against branch origin/trunk Add versionId to AbstractRequest.getErrorResponse and AbstractRequest.getRequest Key: KAFKA-2195 URL: https://issues.apache.org/jira/browse/KAFKA-2195 Project: Kafka Issue Type: Sub-task Reporter: Andrii Biletskyi Assignee: Jun Rao Fix For: 0.8.3 Attachments: KAFKA-2195.patch, KAFKA-2195_2015-05-24_22:48:55.patch, KAFKA-2195_2015-06-15_09:54:53.patch, KAFKA-2195_2015-06-16_22:28:19.patch This is needed to support versioning. 1) getRequest: to parse bytes into correct schema you need to know it's version; by default it's the latest version (current_schema) 2) getErrorResponse: after filling map with error codes you need to create respective Response message which dependes on versionId -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (KAFKA-2195) Add versionId to AbstractRequest.getErrorResponse and AbstractRequest.getRequest
[ https://issues.apache.org/jira/browse/KAFKA-2195?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrii Biletskyi updated KAFKA-2195: Attachment: KAFKA-2195_2015-06-16_22:28:19.patch Add versionId to AbstractRequest.getErrorResponse and AbstractRequest.getRequest Key: KAFKA-2195 URL: https://issues.apache.org/jira/browse/KAFKA-2195 Project: Kafka Issue Type: Sub-task Reporter: Andrii Biletskyi Assignee: Jun Rao Fix For: 0.8.3 Attachments: KAFKA-2195.patch, KAFKA-2195_2015-05-24_22:48:55.patch, KAFKA-2195_2015-06-15_09:54:53.patch, KAFKA-2195_2015-06-16_22:28:19.patch This is needed to support versioning. 1) getRequest: to parse bytes into correct schema you need to know it's version; by default it's the latest version (current_schema) 2) getErrorResponse: after filling map with error codes you need to create respective Response message which dependes on versionId -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (KAFKA-2195) Add versionId to AbstractRequest.getErrorResponse and AbstractRequest.getRequest
[ https://issues.apache.org/jira/browse/KAFKA-2195?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrii Biletskyi updated KAFKA-2195: Attachment: KAFKA-2195_2015-06-15_09:54:53.patch Add versionId to AbstractRequest.getErrorResponse and AbstractRequest.getRequest Key: KAFKA-2195 URL: https://issues.apache.org/jira/browse/KAFKA-2195 Project: Kafka Issue Type: Sub-task Reporter: Andrii Biletskyi Assignee: Jun Rao Fix For: 0.8.3 Attachments: KAFKA-2195.patch, KAFKA-2195_2015-05-24_22:48:55.patch, KAFKA-2195_2015-06-15_09:54:53.patch This is needed to support versioning. 1) getRequest: to parse bytes into correct schema you need to know it's version; by default it's the latest version (current_schema) 2) getErrorResponse: after filling map with error codes you need to create respective Response message which dependes on versionId -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (KAFKA-2195) Add versionId to AbstractRequest.getErrorResponse and AbstractRequest.getRequest
[ https://issues.apache.org/jira/browse/KAFKA-2195?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14585515#comment-14585515 ] Andrii Biletskyi commented on KAFKA-2195: - Updated reviewboard https://reviews.apache.org/r/34415/diff/ against branch origin/trunk Add versionId to AbstractRequest.getErrorResponse and AbstractRequest.getRequest Key: KAFKA-2195 URL: https://issues.apache.org/jira/browse/KAFKA-2195 Project: Kafka Issue Type: Sub-task Reporter: Andrii Biletskyi Assignee: Jun Rao Fix For: 0.8.3 Attachments: KAFKA-2195.patch, KAFKA-2195_2015-05-24_22:48:55.patch, KAFKA-2195_2015-06-15_09:54:53.patch This is needed to support versioning. 1) getRequest: to parse bytes into correct schema you need to know it's version; by default it's the latest version (current_schema) 2) getErrorResponse: after filling map with error codes you need to create respective Response message which dependes on versionId -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (KAFKA-2195) Add versionId to AbstractRequest.getErrorResponse and AbstractRequest.getRequest
[ https://issues.apache.org/jira/browse/KAFKA-2195?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14584234#comment-14584234 ] Andrii Biletskyi commented on KAFKA-2195: - Yes, thanks, I'll address it by Monday. Add versionId to AbstractRequest.getErrorResponse and AbstractRequest.getRequest Key: KAFKA-2195 URL: https://issues.apache.org/jira/browse/KAFKA-2195 Project: Kafka Issue Type: Sub-task Reporter: Andrii Biletskyi Assignee: Jun Rao Fix For: 0.8.3 Attachments: KAFKA-2195.patch, KAFKA-2195_2015-05-24_22:48:55.patch This is needed to support versioning. 1) getRequest: to parse bytes into correct schema you need to know it's version; by default it's the latest version (current_schema) 2) getErrorResponse: after filling map with error codes you need to create respective Response message which dependes on versionId -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (KAFKA-2228) Delete me
[ https://issues.apache.org/jira/browse/KAFKA-2228?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrii Biletskyi resolved KAFKA-2228. - Resolution: Duplicate Delete me - Key: KAFKA-2228 URL: https://issues.apache.org/jira/browse/KAFKA-2228 Project: Kafka Issue Type: Sub-task Reporter: Andrii Biletskyi Fix For: 0.8.3 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (KAFKA-2227) Delete me
[ https://issues.apache.org/jira/browse/KAFKA-2227?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrii Biletskyi resolved KAFKA-2227. - Resolution: Duplicate Delete me - Key: KAFKA-2227 URL: https://issues.apache.org/jira/browse/KAFKA-2227 Project: Kafka Issue Type: Sub-task Reporter: Andrii Biletskyi Fix For: 0.8.3 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (KAFKA-2227) Delete me
[ https://issues.apache.org/jira/browse/KAFKA-2227?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrii Biletskyi updated KAFKA-2227: Summary: Delete me (was: Phase 1: Requests and KafkaApis) Delete me - Key: KAFKA-2227 URL: https://issues.apache.org/jira/browse/KAFKA-2227 Project: Kafka Issue Type: Sub-task Reporter: Andrii Biletskyi Fix For: 0.8.3 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Assigned] (KAFKA-2228) Delete me
[ https://issues.apache.org/jira/browse/KAFKA-2228?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrii Biletskyi reassigned KAFKA-2228: --- Assignee: (was: Andrii Biletskyi) Delete me - Key: KAFKA-2228 URL: https://issues.apache.org/jira/browse/KAFKA-2228 Project: Kafka Issue Type: Sub-task Reporter: Andrii Biletskyi Fix For: 0.8.3 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (KAFKA-2228) Delete me
[ https://issues.apache.org/jira/browse/KAFKA-2228?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrii Biletskyi updated KAFKA-2228: Summary: Delete me (was: Phase 1: Requests and KafkaApis) Delete me - Key: KAFKA-2228 URL: https://issues.apache.org/jira/browse/KAFKA-2228 Project: Kafka Issue Type: Sub-task Reporter: Andrii Biletskyi Assignee: Andrii Biletskyi Fix For: 0.8.3 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (KAFKA-2229) Phase 1: Requests and KafkaApis
[ https://issues.apache.org/jira/browse/KAFKA-2229?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrii Biletskyi updated KAFKA-2229: Status: Patch Available (was: Open) Phase 1: Requests and KafkaApis --- Key: KAFKA-2229 URL: https://issues.apache.org/jira/browse/KAFKA-2229 Project: Kafka Issue Type: Sub-task Reporter: Andrii Biletskyi Assignee: Andrii Biletskyi Fix For: 0.8.3 Attachments: KAFKA-2229.patch -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (KAFKA-2228) Phase 1: Requests and KafkaApis
Andrii Biletskyi created KAFKA-2228: --- Summary: Phase 1: Requests and KafkaApis Key: KAFKA-2228 URL: https://issues.apache.org/jira/browse/KAFKA-2228 Project: Kafka Issue Type: Sub-task Reporter: Andrii Biletskyi Assignee: Andrii Biletskyi -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (KAFKA-2229) Phase 1: Requests and KafkaApis
Andrii Biletskyi created KAFKA-2229: --- Summary: Phase 1: Requests and KafkaApis Key: KAFKA-2229 URL: https://issues.apache.org/jira/browse/KAFKA-2229 Project: Kafka Issue Type: Sub-task Reporter: Andrii Biletskyi Assignee: Andrii Biletskyi -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (KAFKA-2229) Phase 1: Requests and KafkaApis
[ https://issues.apache.org/jira/browse/KAFKA-2229?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrii Biletskyi updated KAFKA-2229: Attachment: KAFKA-2229.patch Phase 1: Requests and KafkaApis --- Key: KAFKA-2229 URL: https://issues.apache.org/jira/browse/KAFKA-2229 Project: Kafka Issue Type: Sub-task Reporter: Andrii Biletskyi Assignee: Andrii Biletskyi Fix For: 0.8.3 Attachments: KAFKA-2229.patch -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (KAFKA-2229) Phase 1: Requests and KafkaApis
[ https://issues.apache.org/jira/browse/KAFKA-2229?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14562877#comment-14562877 ] Andrii Biletskyi commented on KAFKA-2229: - Created reviewboard https://reviews.apache.org/r/34766/diff/ against branch origin/trunk Phase 1: Requests and KafkaApis --- Key: KAFKA-2229 URL: https://issues.apache.org/jira/browse/KAFKA-2229 Project: Kafka Issue Type: Sub-task Reporter: Andrii Biletskyi Assignee: Andrii Biletskyi Fix For: 0.8.3 Attachments: KAFKA-2229.patch -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (KAFKA-2195) Add versionId to AbstractRequest.getErrorResponse and AbstractRequest.getRequest
[ https://issues.apache.org/jira/browse/KAFKA-2195?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrii Biletskyi updated KAFKA-2195: Status: Patch Available (was: In Progress) Add versionId to AbstractRequest.getErrorResponse and AbstractRequest.getRequest Key: KAFKA-2195 URL: https://issues.apache.org/jira/browse/KAFKA-2195 Project: Kafka Issue Type: Sub-task Reporter: Andrii Biletskyi Assignee: Andrii Biletskyi Fix For: 0.8.3 Attachments: KAFKA-2195.patch, KAFKA-2195_2015-05-24_22:48:55.patch This is needed to support versioning. 1) getRequest: to parse bytes into correct schema you need to know it's version; by default it's the latest version (current_schema) 2) getErrorResponse: after filling map with error codes you need to create respective Response message which dependes on versionId -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (KAFKA-2195) Add versionId to AbstractRequest.getErrorResponse and AbstractRequest.getRequest
[ https://issues.apache.org/jira/browse/KAFKA-2195?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14557829#comment-14557829 ] Andrii Biletskyi commented on KAFKA-2195: - Updated reviewboard https://reviews.apache.org/r/34415/diff/ against branch origin/trunk Add versionId to AbstractRequest.getErrorResponse and AbstractRequest.getRequest Key: KAFKA-2195 URL: https://issues.apache.org/jira/browse/KAFKA-2195 Project: Kafka Issue Type: Sub-task Reporter: Andrii Biletskyi Assignee: Andrii Biletskyi Fix For: 0.8.3 Attachments: KAFKA-2195.patch, KAFKA-2195_2015-05-24_22:48:55.patch This is needed to support versioning. 1) getRequest: to parse bytes into correct schema you need to know it's version; by default it's the latest version (current_schema) 2) getErrorResponse: after filling map with error codes you need to create respective Response message which dependes on versionId -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (KAFKA-2195) Add versionId to AbstractRequest.getErrorResponse and AbstractRequest.getRequest
[ https://issues.apache.org/jira/browse/KAFKA-2195?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrii Biletskyi updated KAFKA-2195: Attachment: KAFKA-2195_2015-05-24_22:48:55.patch Add versionId to AbstractRequest.getErrorResponse and AbstractRequest.getRequest Key: KAFKA-2195 URL: https://issues.apache.org/jira/browse/KAFKA-2195 Project: Kafka Issue Type: Sub-task Reporter: Andrii Biletskyi Assignee: Andrii Biletskyi Fix For: 0.8.3 Attachments: KAFKA-2195.patch, KAFKA-2195_2015-05-24_22:48:55.patch This is needed to support versioning. 1) getRequest: to parse bytes into correct schema you need to know it's version; by default it's the latest version (current_schema) 2) getErrorResponse: after filling map with error codes you need to create respective Response message which dependes on versionId -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Assigned] (KAFKA-2195) Add versionId to AbstractRequest.getErrorResponse and AbstractRequest.getRequest
[ https://issues.apache.org/jira/browse/KAFKA-2195?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrii Biletskyi reassigned KAFKA-2195: --- Assignee: Jun Rao (was: Andrii Biletskyi) Add versionId to AbstractRequest.getErrorResponse and AbstractRequest.getRequest Key: KAFKA-2195 URL: https://issues.apache.org/jira/browse/KAFKA-2195 Project: Kafka Issue Type: Sub-task Reporter: Andrii Biletskyi Assignee: Jun Rao Fix For: 0.8.3 Attachments: KAFKA-2195.patch, KAFKA-2195_2015-05-24_22:48:55.patch This is needed to support versioning. 1) getRequest: to parse bytes into correct schema you need to know it's version; by default it's the latest version (current_schema) 2) getErrorResponse: after filling map with error codes you need to create respective Response message which dependes on versionId -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (KAFKA-2195) Add versionId to AbstractRequest.getErrorResponse and AbstractRequest.getRequest
[ https://issues.apache.org/jira/browse/KAFKA-2195?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14550682#comment-14550682 ] Andrii Biletskyi commented on KAFKA-2195: - Created reviewboard https://reviews.apache.org/r/34415/diff/ against branch origin/trunk Add versionId to AbstractRequest.getErrorResponse and AbstractRequest.getRequest Key: KAFKA-2195 URL: https://issues.apache.org/jira/browse/KAFKA-2195 Project: Kafka Issue Type: Sub-task Reporter: Andrii Biletskyi Assignee: Andrii Biletskyi Fix For: 0.8.3 Attachments: KAFKA-2195.patch This is needed to support versioning. 1) getRequest: to parse bytes into correct schema you need to know it's version; by default it's the latest version (current_schema) 2) getErrorResponse: after filling map with error codes you need to create respective Response message which dependes on versionId -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (KAFKA-2195) Add versionId to AbstractRequest.getErrorResponse and AbstractRequest.getRequest
[ https://issues.apache.org/jira/browse/KAFKA-2195?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrii Biletskyi updated KAFKA-2195: Status: Patch Available (was: Open) Add versionId to AbstractRequest.getErrorResponse and AbstractRequest.getRequest Key: KAFKA-2195 URL: https://issues.apache.org/jira/browse/KAFKA-2195 Project: Kafka Issue Type: Sub-task Reporter: Andrii Biletskyi Assignee: Andrii Biletskyi Fix For: 0.8.3 Attachments: KAFKA-2195.patch This is needed to support versioning. 1) getRequest: to parse bytes into correct schema you need to know it's version; by default it's the latest version (current_schema) 2) getErrorResponse: after filling map with error codes you need to create respective Response message which dependes on versionId -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (KAFKA-2195) Add versionId to AbstractRequest.getErrorResponse and AbstractRequest.getRequest
Andrii Biletskyi created KAFKA-2195: --- Summary: Add versionId to AbstractRequest.getErrorResponse and AbstractRequest.getRequest Key: KAFKA-2195 URL: https://issues.apache.org/jira/browse/KAFKA-2195 Project: Kafka Issue Type: Sub-task Reporter: Andrii Biletskyi Assignee: Andrii Biletskyi This is needed to support versioning. 1) getRequest: to parse bytes into correct schema you need to know it's version; by default it's the latest version (current_schema) 2) getErrorResponse: after filling map with error codes you need to create respective Response message which dependes on versionId -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Assigned] (KAFKA-2073) Replace TopicMetadata request/response with o.a.k.requests.metadata
[ https://issues.apache.org/jira/browse/KAFKA-2073?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrii Biletskyi reassigned KAFKA-2073: --- Assignee: Andrii Biletskyi (was: Sriharsha Chintalapani) Replace TopicMetadata request/response with o.a.k.requests.metadata --- Key: KAFKA-2073 URL: https://issues.apache.org/jira/browse/KAFKA-2073 Project: Kafka Issue Type: Sub-task Reporter: Gwen Shapira Assignee: Andrii Biletskyi Fix For: 0.8.3 Replace TopicMetadata request/response with o.a.k.requests.metadata. Note, this is more challenging that it appears because while the wire protocol is identical, the objects are completely different. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (KAFKA-2073) Replace TopicMetadata request/response with o.a.k.requests.metadata
[ https://issues.apache.org/jira/browse/KAFKA-2073?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14545841#comment-14545841 ] Andrii Biletskyi commented on KAFKA-2073: - [~sriharsha] is there any update on this one? This blocks me in KIP-4 where we will evolve TopicMetadataRequest. If you are not working on in - let me know, I can continue with this ticket. Thanks. Replace TopicMetadata request/response with o.a.k.requests.metadata --- Key: KAFKA-2073 URL: https://issues.apache.org/jira/browse/KAFKA-2073 Project: Kafka Issue Type: Sub-task Reporter: Gwen Shapira Assignee: Sriharsha Chintalapani Fix For: 0.8.3 Replace TopicMetadata request/response with o.a.k.requests.metadata. Note, this is more challenging that it appears because while the wire protocol is identical, the objects are completely different. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (KAFKA-2188) JBOD Support
Andrii Biletskyi created KAFKA-2188: --- Summary: JBOD Support Key: KAFKA-2188 URL: https://issues.apache.org/jira/browse/KAFKA-2188 Project: Kafka Issue Type: Bug Reporter: Andrii Biletskyi Assignee: Andrii Biletskyi https://cwiki.apache.org/confluence/display/KAFKA/KIP-18+-+JBOD+Support -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (KAFKA-2188) JBOD Support
[ https://issues.apache.org/jira/browse/KAFKA-2188?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14539777#comment-14539777 ] Andrii Biletskyi commented on KAFKA-2188: - Created reviewboard https://reviews.apache.org/r/34103/diff/ against branch origin/trunk JBOD Support Key: KAFKA-2188 URL: https://issues.apache.org/jira/browse/KAFKA-2188 Project: Kafka Issue Type: Bug Reporter: Andrii Biletskyi Assignee: Andrii Biletskyi Attachments: KAFKA-2188.patch https://cwiki.apache.org/confluence/display/KAFKA/KIP-18+-+JBOD+Support -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (KAFKA-2188) JBOD Support
[ https://issues.apache.org/jira/browse/KAFKA-2188?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrii Biletskyi updated KAFKA-2188: Status: Patch Available (was: Open) JBOD Support Key: KAFKA-2188 URL: https://issues.apache.org/jira/browse/KAFKA-2188 Project: Kafka Issue Type: Bug Reporter: Andrii Biletskyi Assignee: Andrii Biletskyi Attachments: KAFKA-2188.patch https://cwiki.apache.org/confluence/display/KAFKA/KIP-18+-+JBOD+Support -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (KAFKA-2188) JBOD Support
[ https://issues.apache.org/jira/browse/KAFKA-2188?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrii Biletskyi updated KAFKA-2188: Attachment: KAFKA-2188.patch JBOD Support Key: KAFKA-2188 URL: https://issues.apache.org/jira/browse/KAFKA-2188 Project: Kafka Issue Type: Bug Reporter: Andrii Biletskyi Assignee: Andrii Biletskyi Attachments: KAFKA-2188.patch https://cwiki.apache.org/confluence/display/KAFKA/KIP-18+-+JBOD+Support -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (KAFKA-1367) Broker topic metadata not kept in sync with ZooKeeper
[ https://issues.apache.org/jira/browse/KAFKA-1367?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14514763#comment-14514763 ] Andrii Biletskyi commented on KAFKA-1367: - [~jjkoshy] Yes, it appears that topic commands don't require ISR information so it was proposed to remove it at all from the TopicMetadataRequest. There was an idea to create some sort of BrokerMetadataRequest which will include correct topic metadata but since it's not related to KIP-4 directly it won't be a part of it. So everyone is welcome to create a separate KIP for it. Atleast to this conclusion we came last time. Broker topic metadata not kept in sync with ZooKeeper - Key: KAFKA-1367 URL: https://issues.apache.org/jira/browse/KAFKA-1367 Project: Kafka Issue Type: Bug Affects Versions: 0.8.0, 0.8.1 Reporter: Ryan Berdeen Assignee: Ashish K Singh Labels: newbie++ Fix For: 0.8.3 Attachments: KAFKA-1367.txt When a broker is restarted, the topic metadata responses from the brokers will be incorrect (different from ZooKeeper) until a preferred replica leader election. In the metadata, it looks like leaders are correctly removed from the ISR when a broker disappears, but followers are not. Then, when a broker reappears, the ISR is never updated. I used a variation of the Vagrant setup created by Joe Stein to reproduce this with latest from the 0.8.1 branch: https://github.com/also/kafka/commit/dba36a503a5e22ea039df0f9852560b4fb1e067c -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (KAFKA-1367) Broker topic metadata not kept in sync with ZooKeeper
[ https://issues.apache.org/jira/browse/KAFKA-1367?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14514760#comment-14514760 ] Andrii Biletskyi commented on KAFKA-1367: - [~jjkoshy] Yes, it appears that topic commands don't require ISR information so it was proposed to remove it at all from the TopicMetadataRequest. There was an idea to create some sort of BrokerMetadataRequest which will include correct topic metadata but since it's not related to KIP-4 directly it won't be a part of it. So everyone is welcome to create a separate KIP for it. Atleast to this conclusion we came last time. Broker topic metadata not kept in sync with ZooKeeper - Key: KAFKA-1367 URL: https://issues.apache.org/jira/browse/KAFKA-1367 Project: Kafka Issue Type: Bug Affects Versions: 0.8.0, 0.8.1 Reporter: Ryan Berdeen Assignee: Ashish K Singh Labels: newbie++ Fix For: 0.8.3 Attachments: KAFKA-1367.txt When a broker is restarted, the topic metadata responses from the brokers will be incorrect (different from ZooKeeper) until a preferred replica leader election. In the metadata, it looks like leaders are correctly removed from the ISR when a broker disappears, but followers are not. Then, when a broker reappears, the ISR is never updated. I used a variation of the Vagrant setup created by Joe Stein to reproduce this with latest from the 0.8.1 branch: https://github.com/also/kafka/commit/dba36a503a5e22ea039df0f9852560b4fb1e067c -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (KAFKA-1367) Broker topic metadata not kept in sync with ZooKeeper
[ https://issues.apache.org/jira/browse/KAFKA-1367?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14514762#comment-14514762 ] Andrii Biletskyi commented on KAFKA-1367: - [~jjkoshy] Yes, it appears that topic commands don't require ISR information so it was proposed to remove it at all from the TopicMetadataRequest. There was an idea to create some sort of BrokerMetadataRequest which will include correct topic metadata but since it's not related to KIP-4 directly it won't be a part of it. So everyone is welcome to create a separate KIP for it. Atleast to this conclusion we came last time. Broker topic metadata not kept in sync with ZooKeeper - Key: KAFKA-1367 URL: https://issues.apache.org/jira/browse/KAFKA-1367 Project: Kafka Issue Type: Bug Affects Versions: 0.8.0, 0.8.1 Reporter: Ryan Berdeen Assignee: Ashish K Singh Labels: newbie++ Fix For: 0.8.3 Attachments: KAFKA-1367.txt When a broker is restarted, the topic metadata responses from the brokers will be incorrect (different from ZooKeeper) until a preferred replica leader election. In the metadata, it looks like leaders are correctly removed from the ISR when a broker disappears, but followers are not. Then, when a broker reappears, the ISR is never updated. I used a variation of the Vagrant setup created by Joe Stein to reproduce this with latest from the 0.8.1 branch: https://github.com/also/kafka/commit/dba36a503a5e22ea039df0f9852560b4fb1e067c -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (KAFKA-1367) Broker topic metadata not kept in sync with ZooKeeper
[ https://issues.apache.org/jira/browse/KAFKA-1367?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14514764#comment-14514764 ] Andrii Biletskyi commented on KAFKA-1367: - [~jjkoshy] Yes, it appears that topic commands don't require ISR information so it was proposed to remove it at all from the TopicMetadataRequest. There was an idea to create some sort of BrokerMetadataRequest which will include correct topic metadata but since it's not related to KIP-4 directly it won't be a part of it. So everyone is welcome to create a separate KIP for it. Atleast to this conclusion we came last time. Broker topic metadata not kept in sync with ZooKeeper - Key: KAFKA-1367 URL: https://issues.apache.org/jira/browse/KAFKA-1367 Project: Kafka Issue Type: Bug Affects Versions: 0.8.0, 0.8.1 Reporter: Ryan Berdeen Assignee: Ashish K Singh Labels: newbie++ Fix For: 0.8.3 Attachments: KAFKA-1367.txt When a broker is restarted, the topic metadata responses from the brokers will be incorrect (different from ZooKeeper) until a preferred replica leader election. In the metadata, it looks like leaders are correctly removed from the ISR when a broker disappears, but followers are not. Then, when a broker reappears, the ISR is never updated. I used a variation of the Vagrant setup created by Joe Stein to reproduce this with latest from the 0.8.1 branch: https://github.com/also/kafka/commit/dba36a503a5e22ea039df0f9852560b4fb1e067c -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Issue Comment Deleted] (KAFKA-1367) Broker topic metadata not kept in sync with ZooKeeper
[ https://issues.apache.org/jira/browse/KAFKA-1367?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrii Biletskyi updated KAFKA-1367: Comment: was deleted (was: [~jjkoshy] Yes, it appears that topic commands don't require ISR information so it was proposed to remove it at all from the TopicMetadataRequest. There was an idea to create some sort of BrokerMetadataRequest which will include correct topic metadata but since it's not related to KIP-4 directly it won't be a part of it. So everyone is welcome to create a separate KIP for it. Atleast to this conclusion we came last time.) Broker topic metadata not kept in sync with ZooKeeper - Key: KAFKA-1367 URL: https://issues.apache.org/jira/browse/KAFKA-1367 Project: Kafka Issue Type: Bug Affects Versions: 0.8.0, 0.8.1 Reporter: Ryan Berdeen Assignee: Ashish K Singh Labels: newbie++ Fix For: 0.8.3 Attachments: KAFKA-1367.txt When a broker is restarted, the topic metadata responses from the brokers will be incorrect (different from ZooKeeper) until a preferred replica leader election. In the metadata, it looks like leaders are correctly removed from the ISR when a broker disappears, but followers are not. Then, when a broker reappears, the ISR is never updated. I used a variation of the Vagrant setup created by Joe Stein to reproduce this with latest from the 0.8.1 branch: https://github.com/also/kafka/commit/dba36a503a5e22ea039df0f9852560b4fb1e067c -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (KAFKA-1367) Broker topic metadata not kept in sync with ZooKeeper
[ https://issues.apache.org/jira/browse/KAFKA-1367?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14514765#comment-14514765 ] Andrii Biletskyi commented on KAFKA-1367: - [~jjkoshy] Yes, it appears that topic commands don't require ISR information so it was proposed to remove it at all from the TopicMetadataRequest. There was an idea to create some sort of BrokerMetadataRequest which will include correct topic metadata but since it's not related to KIP-4 directly it won't be a part of it. So everyone is welcome to create a separate KIP for it. Atleast to this conclusion we came last time. Broker topic metadata not kept in sync with ZooKeeper - Key: KAFKA-1367 URL: https://issues.apache.org/jira/browse/KAFKA-1367 Project: Kafka Issue Type: Bug Affects Versions: 0.8.0, 0.8.1 Reporter: Ryan Berdeen Assignee: Ashish K Singh Labels: newbie++ Fix For: 0.8.3 Attachments: KAFKA-1367.txt When a broker is restarted, the topic metadata responses from the brokers will be incorrect (different from ZooKeeper) until a preferred replica leader election. In the metadata, it looks like leaders are correctly removed from the ISR when a broker disappears, but followers are not. Then, when a broker reappears, the ISR is never updated. I used a variation of the Vagrant setup created by Joe Stein to reproduce this with latest from the 0.8.1 branch: https://github.com/also/kafka/commit/dba36a503a5e22ea039df0f9852560b4fb1e067c -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (KAFKA-1367) Broker topic metadata not kept in sync with ZooKeeper
[ https://issues.apache.org/jira/browse/KAFKA-1367?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14514761#comment-14514761 ] Andrii Biletskyi commented on KAFKA-1367: - [~jjkoshy] Yes, it appears that topic commands don't require ISR information so it was proposed to remove it at all from the TopicMetadataRequest. There was an idea to create some sort of BrokerMetadataRequest which will include correct topic metadata but since it's not related to KIP-4 directly it won't be a part of it. So everyone is welcome to create a separate KIP for it. Atleast to this conclusion we came last time. Broker topic metadata not kept in sync with ZooKeeper - Key: KAFKA-1367 URL: https://issues.apache.org/jira/browse/KAFKA-1367 Project: Kafka Issue Type: Bug Affects Versions: 0.8.0, 0.8.1 Reporter: Ryan Berdeen Assignee: Ashish K Singh Labels: newbie++ Fix For: 0.8.3 Attachments: KAFKA-1367.txt When a broker is restarted, the topic metadata responses from the brokers will be incorrect (different from ZooKeeper) until a preferred replica leader election. In the metadata, it looks like leaders are correctly removed from the ISR when a broker disappears, but followers are not. Then, when a broker reappears, the ISR is never updated. I used a variation of the Vagrant setup created by Joe Stein to reproduce this with latest from the 0.8.1 branch: https://github.com/also/kafka/commit/dba36a503a5e22ea039df0f9852560b4fb1e067c -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Issue Comment Deleted] (KAFKA-1367) Broker topic metadata not kept in sync with ZooKeeper
[ https://issues.apache.org/jira/browse/KAFKA-1367?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrii Biletskyi updated KAFKA-1367: Comment: was deleted (was: [~jjkoshy] Yes, it appears that topic commands don't require ISR information so it was proposed to remove it at all from the TopicMetadataRequest. There was an idea to create some sort of BrokerMetadataRequest which will include correct topic metadata but since it's not related to KIP-4 directly it won't be a part of it. So everyone is welcome to create a separate KIP for it. Atleast to this conclusion we came last time.) Broker topic metadata not kept in sync with ZooKeeper - Key: KAFKA-1367 URL: https://issues.apache.org/jira/browse/KAFKA-1367 Project: Kafka Issue Type: Bug Affects Versions: 0.8.0, 0.8.1 Reporter: Ryan Berdeen Assignee: Ashish K Singh Labels: newbie++ Fix For: 0.8.3 Attachments: KAFKA-1367.txt When a broker is restarted, the topic metadata responses from the brokers will be incorrect (different from ZooKeeper) until a preferred replica leader election. In the metadata, it looks like leaders are correctly removed from the ISR when a broker disappears, but followers are not. Then, when a broker reappears, the ISR is never updated. I used a variation of the Vagrant setup created by Joe Stein to reproduce this with latest from the 0.8.1 branch: https://github.com/also/kafka/commit/dba36a503a5e22ea039df0f9852560b4fb1e067c -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Issue Comment Deleted] (KAFKA-1367) Broker topic metadata not kept in sync with ZooKeeper
[ https://issues.apache.org/jira/browse/KAFKA-1367?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrii Biletskyi updated KAFKA-1367: Comment: was deleted (was: [~jjkoshy] Yes, it appears that topic commands don't require ISR information so it was proposed to remove it at all from the TopicMetadataRequest. There was an idea to create some sort of BrokerMetadataRequest which will include correct topic metadata but since it's not related to KIP-4 directly it won't be a part of it. So everyone is welcome to create a separate KIP for it. Atleast to this conclusion we came last time.) Broker topic metadata not kept in sync with ZooKeeper - Key: KAFKA-1367 URL: https://issues.apache.org/jira/browse/KAFKA-1367 Project: Kafka Issue Type: Bug Affects Versions: 0.8.0, 0.8.1 Reporter: Ryan Berdeen Assignee: Ashish K Singh Labels: newbie++ Fix For: 0.8.3 Attachments: KAFKA-1367.txt When a broker is restarted, the topic metadata responses from the brokers will be incorrect (different from ZooKeeper) until a preferred replica leader election. In the metadata, it looks like leaders are correctly removed from the ISR when a broker disappears, but followers are not. Then, when a broker reappears, the ISR is never updated. I used a variation of the Vagrant setup created by Joe Stein to reproduce this with latest from the 0.8.1 branch: https://github.com/also/kafka/commit/dba36a503a5e22ea039df0f9852560b4fb1e067c -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Issue Comment Deleted] (KAFKA-1367) Broker topic metadata not kept in sync with ZooKeeper
[ https://issues.apache.org/jira/browse/KAFKA-1367?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrii Biletskyi updated KAFKA-1367: Comment: was deleted (was: [~jjkoshy] Yes, it appears that topic commands don't require ISR information so it was proposed to remove it at all from the TopicMetadataRequest. There was an idea to create some sort of BrokerMetadataRequest which will include correct topic metadata but since it's not related to KIP-4 directly it won't be a part of it. So everyone is welcome to create a separate KIP for it. Atleast to this conclusion we came last time.) Broker topic metadata not kept in sync with ZooKeeper - Key: KAFKA-1367 URL: https://issues.apache.org/jira/browse/KAFKA-1367 Project: Kafka Issue Type: Bug Affects Versions: 0.8.0, 0.8.1 Reporter: Ryan Berdeen Assignee: Ashish K Singh Labels: newbie++ Fix For: 0.8.3 Attachments: KAFKA-1367.txt When a broker is restarted, the topic metadata responses from the brokers will be incorrect (different from ZooKeeper) until a preferred replica leader election. In the metadata, it looks like leaders are correctly removed from the ISR when a broker disappears, but followers are not. Then, when a broker reappears, the ISR is never updated. I used a variation of the Vagrant setup created by Joe Stein to reproduce this with latest from the 0.8.1 branch: https://github.com/also/kafka/commit/dba36a503a5e22ea039df0f9852560b4fb1e067c -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Issue Comment Deleted] (KAFKA-1367) Broker topic metadata not kept in sync with ZooKeeper
[ https://issues.apache.org/jira/browse/KAFKA-1367?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrii Biletskyi updated KAFKA-1367: Comment: was deleted (was: [~jjkoshy] Yes, it appears that topic commands don't require ISR information so it was proposed to remove it at all from the TopicMetadataRequest. There was an idea to create some sort of BrokerMetadataRequest which will include correct topic metadata but since it's not related to KIP-4 directly it won't be a part of it. So everyone is welcome to create a separate KIP for it. Atleast to this conclusion we came last time.) Broker topic metadata not kept in sync with ZooKeeper - Key: KAFKA-1367 URL: https://issues.apache.org/jira/browse/KAFKA-1367 Project: Kafka Issue Type: Bug Affects Versions: 0.8.0, 0.8.1 Reporter: Ryan Berdeen Assignee: Ashish K Singh Labels: newbie++ Fix For: 0.8.3 Attachments: KAFKA-1367.txt When a broker is restarted, the topic metadata responses from the brokers will be incorrect (different from ZooKeeper) until a preferred replica leader election. In the metadata, it looks like leaders are correctly removed from the ISR when a broker disappears, but followers are not. Then, when a broker reappears, the ISR is never updated. I used a variation of the Vagrant setup created by Joe Stein to reproduce this with latest from the 0.8.1 branch: https://github.com/also/kafka/commit/dba36a503a5e22ea039df0f9852560b4fb1e067c -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (KAFKA-2021) Consolidate test classes for KafkaConfig
[ https://issues.apache.org/jira/browse/KAFKA-2021?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrii Biletskyi updated KAFKA-2021: Status: Patch Available (was: Open) Consolidate test classes for KafkaConfig Key: KAFKA-2021 URL: https://issues.apache.org/jira/browse/KAFKA-2021 Project: Kafka Issue Type: Task Reporter: Gwen Shapira Assignee: Andrii Biletskyi Priority: Minor Attachments: KAFKA-2021.patch We have kafka.server.KafkaConfigTest, KafkaConfigConfigDefTest and kafka.unit.KafkaTest (in a file called KafkaConfigTest.scala) I think consolidating them into one test class (or at list renaming so it will be clear how they are different) will make a lot of sense. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (KAFKA-2021) Consolidate test classes for KafkaConfig
[ https://issues.apache.org/jira/browse/KAFKA-2021?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrii Biletskyi updated KAFKA-2021: Attachment: KAFKA-2021.patch Consolidate test classes for KafkaConfig Key: KAFKA-2021 URL: https://issues.apache.org/jira/browse/KAFKA-2021 Project: Kafka Issue Type: Task Reporter: Gwen Shapira Assignee: Andrii Biletskyi Priority: Minor Attachments: KAFKA-2021.patch We have kafka.server.KafkaConfigTest, KafkaConfigConfigDefTest and kafka.unit.KafkaTest (in a file called KafkaConfigTest.scala) I think consolidating them into one test class (or at list renaming so it will be clear how they are different) will make a lot of sense. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (KAFKA-2021) Consolidate test classes for KafkaConfig
[ https://issues.apache.org/jira/browse/KAFKA-2021?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14375085#comment-14375085 ] Andrii Biletskyi commented on KAFKA-2021: - Created reviewboard https://reviews.apache.org/r/32376/diff/ against branch origin/trunk Consolidate test classes for KafkaConfig Key: KAFKA-2021 URL: https://issues.apache.org/jira/browse/KAFKA-2021 Project: Kafka Issue Type: Task Reporter: Gwen Shapira Assignee: Andrii Biletskyi Priority: Minor Attachments: KAFKA-2021.patch We have kafka.server.KafkaConfigTest, KafkaConfigConfigDefTest and kafka.unit.KafkaTest (in a file called KafkaConfigTest.scala) I think consolidating them into one test class (or at list renaming so it will be clear how they are different) will make a lot of sense. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (KAFKA-1912) Create a simple request re-routing facility
[ https://issues.apache.org/jira/browse/KAFKA-1912?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14367331#comment-14367331 ] Andrii Biletskyi commented on KAFKA-1912: - [~jkreps] thanks for adding the code. A few questions / comments: 1. Regarding RequestRerouter shim: If we store and re-route unsent messages in the separate thread, outside the main io thread pool, wouldn't it blur num.io.threads setting? With this facility in place there will be a RequestHandlerPool bounded by num.io.threads and a message queue with a worker managing not-routed messages and unsent responses to clients. 2. On a message queue: if we bombard some broker with requests which require re-routing to controller wouldn't it be strange when client receives TimeoutException because _broker_ wasn't able to process that but not the controller who is the real request executor. 3. Is it possible for NetworkClient (on the client side) to know whether target broker's message queue is full and thus broker may delay re-routing? Consider there is a high priority admin task (like alter topic's config) but the target broker is busy with produce / fetch requests. 4. This was mentioned by [~guozhang] in the mailing list. Let me add this question too: the broker instance which carries the current controller is agnostic of its existence, and use KafkaApis to handle general Kafka requests. Having all admin requests redirected to the controller instance will force the broker to be aware of its carried controller, and access its internal data for handling these requests. Plus with the number of clients out of Kafka's control, this may easily cause the controller to be a hot spot in terms of request load. Create a simple request re-routing facility --- Key: KAFKA-1912 URL: https://issues.apache.org/jira/browse/KAFKA-1912 Project: Kafka Issue Type: Improvement Reporter: Jay Kreps Fix For: 0.8.3 We are accumulating a lot of requests that have to be directed to the correct server. This makes sense for high volume produce or fetch requests. But it is silly to put the extra burden on the client for the many miscellaneous requests such as fetching or committing offsets and so on. This adds a ton of practical complexity to the clients with little or no payoff in performance. We should add a generic request-type agnostic re-routing facility on the server. This would allow any server to accept a request and forward it to the correct destination, proxying the response back to the user. Naturally it needs to do this without blocking the thread. The result is that a client implementation can choose to be optimally efficient and manage a local cache of cluster state and attempt to always direct its requests to the proper server OR it can choose simplicity and just send things all to a single host and let that host figure out where to forward it. I actually think we should implement this more or less across the board, but some requests such as produce and fetch require more logic to proxy since they have to be scattered out to multiple servers and gathered back to create the response. So these could be done in a second phase. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Assigned] (KAFKA-2021) Consolidate test classes for KafkaConfig
[ https://issues.apache.org/jira/browse/KAFKA-2021?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrii Biletskyi reassigned KAFKA-2021: --- Assignee: Andrii Biletskyi Consolidate test classes for KafkaConfig Key: KAFKA-2021 URL: https://issues.apache.org/jira/browse/KAFKA-2021 Project: Kafka Issue Type: Task Reporter: Gwen Shapira Assignee: Andrii Biletskyi Priority: Minor We have kafka.server.KafkaConfigTest, KafkaConfigConfigDefTest and kafka.unit.KafkaTest (in a file called KafkaConfigTest.scala) I think consolidating them into one test class (or at list renaming so it will be clear how they are different) will make a lot of sense. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (KAFKA-1694) kafka command line and centralized operations
[ https://issues.apache.org/jira/browse/KAFKA-1694?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14359409#comment-14359409 ] Andrii Biletskyi commented on KAFKA-1694: - Hi, The list of changes from the latest patch: - removed MaybeOf type and changed RQ/RP protocol - switched over to java protocol definitions (removed scala classes) - removed workaround for KAFKA-1867 (since it was fixed and committed) - code review fixes What's left (pending discussion on mailing list): - Batching Admin Operations - Remove Cluster Metadata, evolve TopicMetadata to V1 instead - Introduce fine-grained error codes instead of single AdminRequestFailedError I also update KIP-4 accordingly and added proposed changes to cover all pending items. kafka command line and centralized operations - Key: KAFKA-1694 URL: https://issues.apache.org/jira/browse/KAFKA-1694 Project: Kafka Issue Type: Bug Reporter: Joe Stein Assignee: Andrii Biletskyi Priority: Critical Fix For: 0.8.3 Attachments: KAFKA-1694.patch, KAFKA-1694_2014-12-24_21:21:51.patch, KAFKA-1694_2015-01-12_15:28:41.patch, KAFKA-1694_2015-01-12_18:54:48.patch, KAFKA-1694_2015-01-13_19:30:11.patch, KAFKA-1694_2015-01-14_15:42:12.patch, KAFKA-1694_2015-01-14_18:07:39.patch, KAFKA-1694_2015-03-12_13:04:37.patch, KAFKA-1772_1802_1775_1774_v2.patch https://cwiki.apache.org/confluence/display/KAFKA/Kafka+Command+Line+and+Related+Improvements -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (KAFKA-1694) kafka command line and centralized operations
[ https://issues.apache.org/jira/browse/KAFKA-1694?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14358489#comment-14358489 ] Andrii Biletskyi commented on KAFKA-1694: - Updated reviewboard https://reviews.apache.org/r/29301/diff/ against branch origin/trunk kafka command line and centralized operations - Key: KAFKA-1694 URL: https://issues.apache.org/jira/browse/KAFKA-1694 Project: Kafka Issue Type: Bug Reporter: Joe Stein Assignee: Andrii Biletskyi Priority: Critical Fix For: 0.8.3 Attachments: KAFKA-1694.patch, KAFKA-1694_2014-12-24_21:21:51.patch, KAFKA-1694_2015-01-12_15:28:41.patch, KAFKA-1694_2015-01-12_18:54:48.patch, KAFKA-1694_2015-01-13_19:30:11.patch, KAFKA-1694_2015-01-14_15:42:12.patch, KAFKA-1694_2015-01-14_18:07:39.patch, KAFKA-1694_2015-03-12_13:04:37.patch, KAFKA-1772_1802_1775_1774_v2.patch https://cwiki.apache.org/confluence/display/KAFKA/Kafka+Command+Line+and+Related+Improvements -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (KAFKA-1694) kafka command line and centralized operations
[ https://issues.apache.org/jira/browse/KAFKA-1694?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrii Biletskyi updated KAFKA-1694: Attachment: KAFKA-1694_2015-03-12_13:04:37.patch kafka command line and centralized operations - Key: KAFKA-1694 URL: https://issues.apache.org/jira/browse/KAFKA-1694 Project: Kafka Issue Type: Bug Reporter: Joe Stein Assignee: Andrii Biletskyi Priority: Critical Fix For: 0.8.3 Attachments: KAFKA-1694.patch, KAFKA-1694_2014-12-24_21:21:51.patch, KAFKA-1694_2015-01-12_15:28:41.patch, KAFKA-1694_2015-01-12_18:54:48.patch, KAFKA-1694_2015-01-13_19:30:11.patch, KAFKA-1694_2015-01-14_15:42:12.patch, KAFKA-1694_2015-01-14_18:07:39.patch, KAFKA-1694_2015-03-12_13:04:37.patch, KAFKA-1772_1802_1775_1774_v2.patch https://cwiki.apache.org/confluence/display/KAFKA/Kafka+Command+Line+and+Related+Improvements -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (KAFKA-1845) KafkaConfig should use ConfigDef
[ https://issues.apache.org/jira/browse/KAFKA-1845?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrii Biletskyi updated KAFKA-1845: Attachment: KAFKA-1845_2015-03-05_01:12:22.patch KafkaConfig should use ConfigDef - Key: KAFKA-1845 URL: https://issues.apache.org/jira/browse/KAFKA-1845 Project: Kafka Issue Type: Sub-task Reporter: Gwen Shapira Assignee: Andrii Biletskyi Labels: newbie Fix For: 0.8.3 Attachments: KAFKA-1845.patch, KAFKA-1845_2015-02-08_17:05:22.patch, KAFKA-1845_2015-03-05_01:12:22.patch ConfigDef is already used for the new producer and for TopicConfig. Will be nice to standardize and use one configuration and validation library across the board. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (KAFKA-1845) KafkaConfig should use ConfigDef
[ https://issues.apache.org/jira/browse/KAFKA-1845?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14347751#comment-14347751 ] Andrii Biletskyi commented on KAFKA-1845: - Updated reviewboard https://reviews.apache.org/r/30126/diff/ against branch origin/trunk KafkaConfig should use ConfigDef - Key: KAFKA-1845 URL: https://issues.apache.org/jira/browse/KAFKA-1845 Project: Kafka Issue Type: Sub-task Reporter: Gwen Shapira Assignee: Andrii Biletskyi Labels: newbie Fix For: 0.8.3 Attachments: KAFKA-1845.patch, KAFKA-1845_2015-02-08_17:05:22.patch, KAFKA-1845_2015-03-05_01:12:22.patch ConfigDef is already used for the new producer and for TopicConfig. Will be nice to standardize and use one configuration and validation library across the board. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (KAFKA-1694) kafka command line and centralized operations
[ https://issues.apache.org/jira/browse/KAFKA-1694?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14326171#comment-14326171 ] Andrii Biletskyi commented on KAFKA-1694: - [~guozhang], yes, initially I did split it to several patches. The problem is that almost all tickets depend on each other. Also I believe that single patch lets you better understand the whole picture (and the use case - CLI) so people can comment, argue in mail list. But I totally agree with you this way we can miss some hidden issues. I would prefer to collect some feedback to be sure we more or less agree on approach and after that I can split this big feature to sub-patches. kafka command line and centralized operations - Key: KAFKA-1694 URL: https://issues.apache.org/jira/browse/KAFKA-1694 Project: Kafka Issue Type: Bug Reporter: Joe Stein Assignee: Andrii Biletskyi Priority: Critical Fix For: 0.8.3 Attachments: KAFKA-1694.patch, KAFKA-1694_2014-12-24_21:21:51.patch, KAFKA-1694_2015-01-12_15:28:41.patch, KAFKA-1694_2015-01-12_18:54:48.patch, KAFKA-1694_2015-01-13_19:30:11.patch, KAFKA-1694_2015-01-14_15:42:12.patch, KAFKA-1694_2015-01-14_18:07:39.patch, KAFKA-1772_1802_1775_1774_v2.patch https://cwiki.apache.org/confluence/display/KAFKA/Kafka+Command+Line+and+Related+Improvements -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Issue Comment Deleted] (KAFKA-1694) kafka command line and centralized operations
[ https://issues.apache.org/jira/browse/KAFKA-1694?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrii Biletskyi updated KAFKA-1694: Comment: was deleted (was: [~guozhang], yes, initially I did split it to several patches. The problem is that almost all tickets depend on each other. Also I believe that single patch lets you better understand the whole picture (and the use case - CLI) so people can comment, argue in mail list. But I totally agree with you this way we can miss some hidden issues. I would prefer to collect some feedback to be sure we more or less agree on approach and after that I can split this big feature to sub-patches. ) kafka command line and centralized operations - Key: KAFKA-1694 URL: https://issues.apache.org/jira/browse/KAFKA-1694 Project: Kafka Issue Type: Bug Reporter: Joe Stein Assignee: Andrii Biletskyi Priority: Critical Fix For: 0.8.3 Attachments: KAFKA-1694.patch, KAFKA-1694_2014-12-24_21:21:51.patch, KAFKA-1694_2015-01-12_15:28:41.patch, KAFKA-1694_2015-01-12_18:54:48.patch, KAFKA-1694_2015-01-13_19:30:11.patch, KAFKA-1694_2015-01-14_15:42:12.patch, KAFKA-1694_2015-01-14_18:07:39.patch, KAFKA-1772_1802_1775_1774_v2.patch https://cwiki.apache.org/confluence/display/KAFKA/Kafka+Command+Line+and+Related+Improvements -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (KAFKA-1694) kafka command line and centralized operations
[ https://issues.apache.org/jira/browse/KAFKA-1694?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14326172#comment-14326172 ] Andrii Biletskyi commented on KAFKA-1694: - [~guozhang], yes, initially I did split it to several patches. The problem is that almost all tickets depend on each other. Also I believe that single patch lets you better understand the whole picture (and the use case - CLI) so people can comment, argue in mail list. But I totally agree with you this way we can miss some hidden issues. I would prefer to collect some feedback to be sure we more or less agree on approach and after that I can split this big feature to sub-patches. kafka command line and centralized operations - Key: KAFKA-1694 URL: https://issues.apache.org/jira/browse/KAFKA-1694 Project: Kafka Issue Type: Bug Reporter: Joe Stein Assignee: Andrii Biletskyi Priority: Critical Fix For: 0.8.3 Attachments: KAFKA-1694.patch, KAFKA-1694_2014-12-24_21:21:51.patch, KAFKA-1694_2015-01-12_15:28:41.patch, KAFKA-1694_2015-01-12_18:54:48.patch, KAFKA-1694_2015-01-13_19:30:11.patch, KAFKA-1694_2015-01-14_15:42:12.patch, KAFKA-1694_2015-01-14_18:07:39.patch, KAFKA-1772_1802_1775_1774_v2.patch https://cwiki.apache.org/confluence/display/KAFKA/Kafka+Command+Line+and+Related+Improvements -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (KAFKA-1845) KafkaConfig should use ConfigDef
[ https://issues.apache.org/jira/browse/KAFKA-1845?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14311353#comment-14311353 ] Andrii Biletskyi commented on KAFKA-1845: - Updated reviewboard https://reviews.apache.org/r/30126/diff/ against branch origin/trunk KafkaConfig should use ConfigDef - Key: KAFKA-1845 URL: https://issues.apache.org/jira/browse/KAFKA-1845 Project: Kafka Issue Type: Sub-task Reporter: Gwen Shapira Assignee: Andrii Biletskyi Labels: newbie Fix For: 0.8.3 Attachments: KAFKA-1845.patch, KAFKA-1845_2015-02-08_17:05:22.patch ConfigDef is already used for the new producer and for TopicConfig. Will be nice to standardize and use one configuration and validation library across the board. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (KAFKA-1845) KafkaConfig should use ConfigDef
[ https://issues.apache.org/jira/browse/KAFKA-1845?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrii Biletskyi updated KAFKA-1845: Attachment: KAFKA-1845_2015-02-08_17:05:22.patch KafkaConfig should use ConfigDef - Key: KAFKA-1845 URL: https://issues.apache.org/jira/browse/KAFKA-1845 Project: Kafka Issue Type: Sub-task Reporter: Gwen Shapira Assignee: Andrii Biletskyi Labels: newbie Fix For: 0.8.3 Attachments: KAFKA-1845.patch, KAFKA-1845_2015-02-08_17:05:22.patch ConfigDef is already used for the new producer and for TopicConfig. Will be nice to standardize and use one configuration and validation library across the board. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (KAFKA-1904) run sanity failed test
[ https://issues.apache.org/jira/browse/KAFKA-1904?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14308964#comment-14308964 ] Andrii Biletskyi commented on KAFKA-1904: - [~joestein] - I was able to run tests both on trunk and 0.8.2... after fixing some operational stuff locally. A Gwen noted, it is required to specify your values in cluster_config.json for kafka_home and java_home (this instructions are present in readme for system_tests). Then I had to setup my ssh so I can ssh localhost without password. After that I had some permission denied while tests write or create log files inside system_test dirs - I chmod'ed that and all went fine. So, I guess, we can consider that tests are working (at least running). P.S.: having said that, I'd vote for patch to print errors that may happen during executing commands via ssh. I had to to execute all commands from debug log myself to find and fix the issues. run sanity failed test -- Key: KAFKA-1904 URL: https://issues.apache.org/jira/browse/KAFKA-1904 Project: Kafka Issue Type: Bug Reporter: Joe Stein Priority: Blocker Fix For: 0.8.3 Attachments: run_sanity.log.gz _test_case_name : testcase_1 _test_class_name : ReplicaBasicTest arg : bounce_broker : true arg : broker_type : leader arg : message_producing_free_time_sec : 15 arg : num_iteration : 2 arg : num_messages_to_produce_per_producer_call : 50 arg : num_partition : 2 arg : replica_factor : 3 arg : sleep_seconds_between_producer_calls : 1 validation_status : Test completed : FAILED -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (KAFKA-1786) implement a global configuration feature for brokers
[ https://issues.apache.org/jira/browse/KAFKA-1786?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrii Biletskyi updated KAFKA-1786: Attachment: KAFKA_1786.patch Patch KAFKA_1786.patch is based on KAFKA-1845 patch (KafkaConfig should use ConfigDef) implement a global configuration feature for brokers Key: KAFKA-1786 URL: https://issues.apache.org/jira/browse/KAFKA-1786 Project: Kafka Issue Type: Sub-task Reporter: Joe Stein Assignee: Andrii Biletskyi Fix For: 0.8.3 Attachments: KAFKA_1786.patch Global level configurations (much like topic level) for brokers are managed by humans and automation systems through server.properties. Some configuration make sense to use default (like it is now) or override from central location (zookeeper for now). We can modify this through the new CLI tool so that every broker can have exact same setting. Some configurations we should allow to be overriden from server.properties (like port) but others we should use the global store as source of truth (e.g. auto topic enable, fetch replica message size, etc). Since most configuration I believe are going to fall into this category we should have the list of server.properties that can override the global config in the code in a list which we can manage... everything else the global takes precedence. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Comment Edited] (KAFKA-1786) implement a global configuration feature for brokers
[ https://issues.apache.org/jira/browse/KAFKA-1786?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14262248#comment-14262248 ] Andrii Biletskyi edited comment on KAFKA-1786 at 1/23/15 12:05 PM: --- Created reviewboard https://reviews.apache.org/r/29513/diff/ against branch origin/trunk UPD: discraded for now. Uploaded a patch KAFAK_1786.patch based on KAFKA-1845 (since it's a separate big patch) was (Author: abiletskyi): Created reviewboard https://reviews.apache.org/r/29513/diff/ against branch origin/trunk implement a global configuration feature for brokers Key: KAFKA-1786 URL: https://issues.apache.org/jira/browse/KAFKA-1786 Project: Kafka Issue Type: Sub-task Reporter: Joe Stein Assignee: Andrii Biletskyi Fix For: 0.8.3 Attachments: KAFKA_1786.patch Global level configurations (much like topic level) for brokers are managed by humans and automation systems through server.properties. Some configuration make sense to use default (like it is now) or override from central location (zookeeper for now). We can modify this through the new CLI tool so that every broker can have exact same setting. Some configurations we should allow to be overriden from server.properties (like port) but others we should use the global store as source of truth (e.g. auto topic enable, fetch replica message size, etc). Since most configuration I believe are going to fall into this category we should have the list of server.properties that can override the global config in the code in a list which we can manage... everything else the global takes precedence. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (KAFKA-1786) implement a global configuration feature for brokers
[ https://issues.apache.org/jira/browse/KAFKA-1786?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrii Biletskyi updated KAFKA-1786: Attachment: (was: KAFKA-1786.patch) implement a global configuration feature for brokers Key: KAFKA-1786 URL: https://issues.apache.org/jira/browse/KAFKA-1786 Project: Kafka Issue Type: Sub-task Reporter: Joe Stein Assignee: Andrii Biletskyi Fix For: 0.8.3 Global level configurations (much like topic level) for brokers are managed by humans and automation systems through server.properties. Some configuration make sense to use default (like it is now) or override from central location (zookeeper for now). We can modify this through the new CLI tool so that every broker can have exact same setting. Some configurations we should allow to be overriden from server.properties (like port) but others we should use the global store as source of truth (e.g. auto topic enable, fetch replica message size, etc). Since most configuration I believe are going to fall into this category we should have the list of server.properties that can override the global config in the code in a list which we can manage... everything else the global takes precedence. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (KAFKA-1845) KafkaConfig should use ConfigDef
[ https://issues.apache.org/jira/browse/KAFKA-1845?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14285979#comment-14285979 ] Andrii Biletskyi commented on KAFKA-1845: - In uploaded patch (KAFKA-1845.patch) all config settings were moved to ConfigDef. ConfigDef.define method requires Importance field. This information is not provided in current trunk version of KafkaConfig, so I used Importance.HIGH everywhere. Please add our comments in review or provide setting to importance map. KafkaConfig should use ConfigDef - Key: KAFKA-1845 URL: https://issues.apache.org/jira/browse/KAFKA-1845 Project: Kafka Issue Type: Sub-task Reporter: Gwen Shapira Assignee: Andrii Biletskyi Labels: newbie Attachments: KAFKA-1845.patch ConfigDef is already used for the new producer and for TopicConfig. Will be nice to standardize and use one configuration and validation library across the board. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (KAFKA-1845) KafkaConfig should use ConfigDef
[ https://issues.apache.org/jira/browse/KAFKA-1845?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrii Biletskyi updated KAFKA-1845: Status: Patch Available (was: Open) KafkaConfig should use ConfigDef - Key: KAFKA-1845 URL: https://issues.apache.org/jira/browse/KAFKA-1845 Project: Kafka Issue Type: Sub-task Reporter: Gwen Shapira Assignee: Andrii Biletskyi Labels: newbie Attachments: KAFKA-1845.patch ConfigDef is already used for the new producer and for TopicConfig. Will be nice to standardize and use one configuration and validation library across the board. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (KAFKA-1333) Add consumer co-ordinator module to the server
[ https://issues.apache.org/jira/browse/KAFKA-1333?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14279930#comment-14279930 ] Andrii Biletskyi commented on KAFKA-1333: - [~guozhang] many thanks for such informative update! Add consumer co-ordinator module to the server -- Key: KAFKA-1333 URL: https://issues.apache.org/jira/browse/KAFKA-1333 Project: Kafka Issue Type: Sub-task Components: consumer Affects Versions: 0.9.0 Reporter: Neha Narkhede Assignee: Guozhang Wang Scope of this JIRA is to just add a consumer co-ordinator module that do the following: 1) coordinator start-up, metadata initialization 2) simple join group handling (just updating metadata, no failure detection / rebalancing): this should be sufficient for consumers doing self offset / partition management. Offset manager will still run side-by-side with the coordinator in this JIRA, and we will merge it in KAFKA-1740. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (KAFKA-1333) Add consumer co-ordinator module to the server
[ https://issues.apache.org/jira/browse/KAFKA-1333?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14278739#comment-14278739 ] Andrii Biletskyi commented on KAFKA-1333: - Hey [~guozhang], We are considering using 0.9 Consumer parts for full-fledged high level Consumer which supports all basic stuff like rebalance etc plus some additional specific to our needs features.Since you are the assignee of this ticket can you please tell where are you on this piece, when there will be some first alpha version patch to review / contribute? Thank you in advance! Add consumer co-ordinator module to the server -- Key: KAFKA-1333 URL: https://issues.apache.org/jira/browse/KAFKA-1333 Project: Kafka Issue Type: Sub-task Components: consumer Affects Versions: 0.9.0 Reporter: Neha Narkhede Assignee: Guozhang Wang Scope of this JIRA is to just add a consumer co-ordinator module that do the following: 1) coordinator start-up, metadata initialization 2) simple join group handling (just updating metadata, no failure detection / rebalancing): this should be sufficient for consumers doing self offset / partition management. Offset manager will still run side-by-side with the coordinator in this JIRA, and we will merge it in KAFKA-1740. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (KAFKA-1694) kafka command line and centralized operations
[ https://issues.apache.org/jira/browse/KAFKA-1694?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14277126#comment-14277126 ] Andrii Biletskyi commented on KAFKA-1694: - Updated reviewboard https://reviews.apache.org/r/29301/diff/ against branch origin/trunk kafka command line and centralized operations - Key: KAFKA-1694 URL: https://issues.apache.org/jira/browse/KAFKA-1694 Project: Kafka Issue Type: Bug Reporter: Joe Stein Assignee: Andrii Biletskyi Priority: Critical Fix For: 0.8.3 Attachments: KAFKA-1694.patch, KAFKA-1694_2014-12-24_21:21:51.patch, KAFKA-1694_2015-01-12_15:28:41.patch, KAFKA-1694_2015-01-12_18:54:48.patch, KAFKA-1694_2015-01-13_19:30:11.patch, KAFKA-1694_2015-01-14_15:42:12.patch, KAFKA-1694_2015-01-14_18:07:39.patch, KAFKA-1772_1802_1775_1774_v2.patch https://cwiki.apache.org/confluence/display/KAFKA/Kafka+Command+Line+and+Related+Improvements -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (KAFKA-1694) kafka command line and centralized operations
[ https://issues.apache.org/jira/browse/KAFKA-1694?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrii Biletskyi updated KAFKA-1694: Attachment: KAFKA-1694_2015-01-14_18:07:39.patch kafka command line and centralized operations - Key: KAFKA-1694 URL: https://issues.apache.org/jira/browse/KAFKA-1694 Project: Kafka Issue Type: Bug Reporter: Joe Stein Assignee: Andrii Biletskyi Priority: Critical Fix For: 0.8.3 Attachments: KAFKA-1694.patch, KAFKA-1694_2014-12-24_21:21:51.patch, KAFKA-1694_2015-01-12_15:28:41.patch, KAFKA-1694_2015-01-12_18:54:48.patch, KAFKA-1694_2015-01-13_19:30:11.patch, KAFKA-1694_2015-01-14_15:42:12.patch, KAFKA-1694_2015-01-14_18:07:39.patch, KAFKA-1772_1802_1775_1774_v2.patch https://cwiki.apache.org/confluence/display/KAFKA/Kafka+Command+Line+and+Related+Improvements -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (KAFKA-1694) kafka command line and centralized operations
[ https://issues.apache.org/jira/browse/KAFKA-1694?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrii Biletskyi updated KAFKA-1694: Attachment: KAFKA-1694_2015-01-14_15:42:12.patch kafka command line and centralized operations - Key: KAFKA-1694 URL: https://issues.apache.org/jira/browse/KAFKA-1694 Project: Kafka Issue Type: Bug Reporter: Joe Stein Assignee: Andrii Biletskyi Priority: Critical Fix For: 0.8.3 Attachments: KAFKA-1694.patch, KAFKA-1694_2014-12-24_21:21:51.patch, KAFKA-1694_2015-01-12_15:28:41.patch, KAFKA-1694_2015-01-12_18:54:48.patch, KAFKA-1694_2015-01-13_19:30:11.patch, KAFKA-1694_2015-01-14_15:42:12.patch, KAFKA-1772_1802_1775_1774_v2.patch https://cwiki.apache.org/confluence/display/KAFKA/Kafka+Command+Line+and+Related+Improvements -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (KAFKA-1694) kafka command line and centralized operations
[ https://issues.apache.org/jira/browse/KAFKA-1694?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14276910#comment-14276910 ] Andrii Biletskyi commented on KAFKA-1694: - Updated reviewboard https://reviews.apache.org/r/29301/diff/ against branch origin/trunk kafka command line and centralized operations - Key: KAFKA-1694 URL: https://issues.apache.org/jira/browse/KAFKA-1694 Project: Kafka Issue Type: Bug Reporter: Joe Stein Assignee: Andrii Biletskyi Priority: Critical Fix For: 0.8.3 Attachments: KAFKA-1694.patch, KAFKA-1694_2014-12-24_21:21:51.patch, KAFKA-1694_2015-01-12_15:28:41.patch, KAFKA-1694_2015-01-12_18:54:48.patch, KAFKA-1694_2015-01-13_19:30:11.patch, KAFKA-1694_2015-01-14_15:42:12.patch, KAFKA-1772_1802_1775_1774_v2.patch https://cwiki.apache.org/confluence/display/KAFKA/Kafka+Command+Line+and+Related+Improvements -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (KAFKA-1694) kafka command line and centralized operations
[ https://issues.apache.org/jira/browse/KAFKA-1694?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14275577#comment-14275577 ] Andrii Biletskyi commented on KAFKA-1694: - Updated reviewboard https://reviews.apache.org/r/29301/diff/ against branch origin/trunk kafka command line and centralized operations - Key: KAFKA-1694 URL: https://issues.apache.org/jira/browse/KAFKA-1694 Project: Kafka Issue Type: Bug Reporter: Joe Stein Assignee: Andrii Biletskyi Priority: Critical Fix For: 0.8.3 Attachments: KAFKA-1694.patch, KAFKA-1694_2014-12-24_21:21:51.patch, KAFKA-1694_2015-01-12_15:28:41.patch, KAFKA-1694_2015-01-12_18:54:48.patch, KAFKA-1694_2015-01-13_19:30:11.patch, KAFKA-1772_1802_1775_1774_v2.patch https://cwiki.apache.org/confluence/display/KAFKA/Kafka+Command+Line+and+Related+Improvements -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (KAFKA-1694) kafka command line and centralized operations
[ https://issues.apache.org/jira/browse/KAFKA-1694?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrii Biletskyi updated KAFKA-1694: Attachment: KAFKA-1694_2015-01-13_19:30:11.patch kafka command line and centralized operations - Key: KAFKA-1694 URL: https://issues.apache.org/jira/browse/KAFKA-1694 Project: Kafka Issue Type: Bug Reporter: Joe Stein Assignee: Andrii Biletskyi Priority: Critical Fix For: 0.8.3 Attachments: KAFKA-1694.patch, KAFKA-1694_2014-12-24_21:21:51.patch, KAFKA-1694_2015-01-12_15:28:41.patch, KAFKA-1694_2015-01-12_18:54:48.patch, KAFKA-1694_2015-01-13_19:30:11.patch, KAFKA-1772_1802_1775_1774_v2.patch https://cwiki.apache.org/confluence/display/KAFKA/Kafka+Command+Line+and+Related+Improvements -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (KAFKA-1694) kafka command line and centralized operations
[ https://issues.apache.org/jira/browse/KAFKA-1694?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrii Biletskyi updated KAFKA-1694: Attachment: KAFKA-1694_2015-01-12_15:28:41.patch kafka command line and centralized operations - Key: KAFKA-1694 URL: https://issues.apache.org/jira/browse/KAFKA-1694 Project: Kafka Issue Type: Bug Reporter: Joe Stein Assignee: Andrii Biletskyi Priority: Critical Fix For: 0.8.3 Attachments: KAFKA-1694.patch, KAFKA-1694_2014-12-24_21:21:51.patch, KAFKA-1694_2015-01-12_15:28:41.patch, KAFKA-1772_1802_1775_1774_v2.patch https://cwiki.apache.org/confluence/display/KAFKA/Kafka+Command+Line+and+Related+Improvements -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (KAFKA-1694) kafka command line and centralized operations
[ https://issues.apache.org/jira/browse/KAFKA-1694?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14273588#comment-14273588 ] Andrii Biletskyi commented on KAFKA-1694: - Updated reviewboard https://reviews.apache.org/r/29301/diff/ against branch origin/trunk kafka command line and centralized operations - Key: KAFKA-1694 URL: https://issues.apache.org/jira/browse/KAFKA-1694 Project: Kafka Issue Type: Bug Reporter: Joe Stein Assignee: Andrii Biletskyi Priority: Critical Fix For: 0.8.3 Attachments: KAFKA-1694.patch, KAFKA-1694_2014-12-24_21:21:51.patch, KAFKA-1694_2015-01-12_15:28:41.patch, KAFKA-1772_1802_1775_1774_v2.patch https://cwiki.apache.org/confluence/display/KAFKA/Kafka+Command+Line+and+Related+Improvements -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (KAFKA-1694) kafka command line and centralized operations
[ https://issues.apache.org/jira/browse/KAFKA-1694?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14273773#comment-14273773 ] Andrii Biletskyi commented on KAFKA-1694: - Updated reviewboard https://reviews.apache.org/r/29301/diff/ against branch origin/trunk kafka command line and centralized operations - Key: KAFKA-1694 URL: https://issues.apache.org/jira/browse/KAFKA-1694 Project: Kafka Issue Type: Bug Reporter: Joe Stein Assignee: Andrii Biletskyi Priority: Critical Fix For: 0.8.3 Attachments: KAFKA-1694.patch, KAFKA-1694_2014-12-24_21:21:51.patch, KAFKA-1694_2015-01-12_15:28:41.patch, KAFKA-1694_2015-01-12_18:54:48.patch, KAFKA-1772_1802_1775_1774_v2.patch https://cwiki.apache.org/confluence/display/KAFKA/Kafka+Command+Line+and+Related+Improvements -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (KAFKA-1694) kafka command line and centralized operations
[ https://issues.apache.org/jira/browse/KAFKA-1694?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrii Biletskyi updated KAFKA-1694: Attachment: KAFKA-1694_2015-01-12_18:54:48.patch kafka command line and centralized operations - Key: KAFKA-1694 URL: https://issues.apache.org/jira/browse/KAFKA-1694 Project: Kafka Issue Type: Bug Reporter: Joe Stein Assignee: Andrii Biletskyi Priority: Critical Fix For: 0.8.3 Attachments: KAFKA-1694.patch, KAFKA-1694_2014-12-24_21:21:51.patch, KAFKA-1694_2015-01-12_15:28:41.patch, KAFKA-1694_2015-01-12_18:54:48.patch, KAFKA-1772_1802_1775_1774_v2.patch https://cwiki.apache.org/confluence/display/KAFKA/Kafka+Command+Line+and+Related+Improvements -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (KAFKA-1845) KafkaConfig should use ConfigDef
[ https://issues.apache.org/jira/browse/KAFKA-1845?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14269348#comment-14269348 ] Andrii Biletskyi commented on KAFKA-1845: - I converted it to sub-task and put under CLI tool since we will require this feature in GlobalConfiguration task (namely validating ConfigChangeRequest). KafkaConfig should use ConfigDef - Key: KAFKA-1845 URL: https://issues.apache.org/jira/browse/KAFKA-1845 Project: Kafka Issue Type: Sub-task Reporter: Gwen Shapira Labels: newbie ConfigDef is already used for the new producer and for TopicConfig. Will be nice to standardize and use one configuration and validation library across the board. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (KAFKA-1845) KafkaConfig should use ConfigDef
[ https://issues.apache.org/jira/browse/KAFKA-1845?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrii Biletskyi updated KAFKA-1845: Issue Type: Sub-task (was: Improvement) Parent: KAFKA-1694 KafkaConfig should use ConfigDef - Key: KAFKA-1845 URL: https://issues.apache.org/jira/browse/KAFKA-1845 Project: Kafka Issue Type: Sub-task Reporter: Gwen Shapira Labels: newbie ConfigDef is already used for the new producer and for TopicConfig. Will be nice to standardize and use one configuration and validation library across the board. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (KAFKA-1786) implement a global configuration feature for brokers
[ https://issues.apache.org/jira/browse/KAFKA-1786?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrii Biletskyi updated KAFKA-1786: Attachment: KAFKA-1786.patch implement a global configuration feature for brokers Key: KAFKA-1786 URL: https://issues.apache.org/jira/browse/KAFKA-1786 Project: Kafka Issue Type: Sub-task Reporter: Joe Stein Fix For: 0.8.3 Attachments: KAFKA-1786.patch Global level configurations (much like topic level) for brokers are managed by humans and automation systems through server.properties. Some configuration make sense to use default (like it is now) or override from central location (zookeeper for now). We can modify this through the new CLI tool so that every broker can have exact same setting. Some configurations we should allow to be overriden from server.properties (like port) but others we should use the global store as source of truth (e.g. auto topic enable, fetch replica message size, etc). Since most configuration I believe are going to fall into this category we should have the list of server.properties that can override the global config in the code in a list which we can manage... everything else the global takes precedence. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
