[jira] [Commented] (KAFKA-1862) Pass in the Time object into OffsetManager
[ https://issues.apache.org/jira/browse/KAFKA-1862?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14298961#comment-14298961 ] Aditya A Auradkar commented on KAFKA-1862: -- [~guozhang] I cannot find the testOffsetExpiration case in OffsetCommitTest. Does this already exist? Pass in the Time object into OffsetManager -- Key: KAFKA-1862 URL: https://issues.apache.org/jira/browse/KAFKA-1862 Project: Kafka Issue Type: Bug Reporter: Guozhang Wang Assignee: Aditya Auradkar Labels: newbie++ Fix For: 0.9.0 We should improve OffsetManager to take in a Time instance as we do for LogManager and ReplicaManager. That way we can advance time with MockTime in test cases. Then we can move the testOffsetExpiration case from OffsetCommitTest to OffsetManagerTest. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (KAFKA-1760) Implement new consumer client
[ https://issues.apache.org/jira/browse/KAFKA-1760?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guozhang Wang updated KAFKA-1760: - Resolution: Fixed Status: Resolved (was: Patch Available) Implement new consumer client - Key: KAFKA-1760 URL: https://issues.apache.org/jira/browse/KAFKA-1760 Project: Kafka Issue Type: Sub-task Components: consumer Reporter: Jay Kreps Assignee: Jay Kreps Fix For: 0.8.3 Attachments: KAFKA-1760.patch, KAFKA-1760_2015-01-11_16:57:15.patch, KAFKA-1760_2015-01-18_19:10:13.patch, KAFKA-1760_2015-01-21_08:42:20.patch, KAFKA-1760_2015-01-22_10:03:26.patch, KAFKA-1760_2015-01-22_20:21:56.patch, KAFKA-1760_2015-01-23_13:13:00.patch, KAFKA-1760_2015-01-29_03:20:20.patch Implement a consumer client. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (KAFKA-1760) Implement new consumer client
[ https://issues.apache.org/jira/browse/KAFKA-1760?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14299027#comment-14299027 ] Guozhang Wang commented on KAFKA-1760: -- Has committed to trunk, closing this ticket. Implement new consumer client - Key: KAFKA-1760 URL: https://issues.apache.org/jira/browse/KAFKA-1760 Project: Kafka Issue Type: Sub-task Components: consumer Reporter: Jay Kreps Assignee: Jay Kreps Fix For: 0.8.3 Attachments: KAFKA-1760.patch, KAFKA-1760_2015-01-11_16:57:15.patch, KAFKA-1760_2015-01-18_19:10:13.patch, KAFKA-1760_2015-01-21_08:42:20.patch, KAFKA-1760_2015-01-22_10:03:26.patch, KAFKA-1760_2015-01-22_20:21:56.patch, KAFKA-1760_2015-01-23_13:13:00.patch, KAFKA-1760_2015-01-29_03:20:20.patch Implement a consumer client. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (KAFKA-1909) VerifiableProperties does not see default properties of the wrapped Properties instance
Tommy Becker created KAFKA-1909: --- Summary: VerifiableProperties does not see default properties of the wrapped Properties instance Key: KAFKA-1909 URL: https://issues.apache.org/jira/browse/KAFKA-1909 Project: Kafka Issue Type: Bug Components: config Affects Versions: 0.8.1.1 Reporter: Tommy Becker The VerifiableProperties class wraps a java.util.Properties instance. The various getXXX methods in VerifiableProperties that do not take a default value issue a containsKey() call to the underlying Properties instance to determine if the property exists. Unfortunately, the containsKey method is merely (mis)-inherited from Hashtable; it doesn't query the Properties instance own defaults. The net effect of this is that only key value pairs defined directly in the Properties instance are usable by Kafka. We have a base config that is used throughout our application but one particular consumer needs different settings. Trying to achieve this by using new Properties(baseProperties) and setting the consumer specific values in that doesn't work :( VerifiableProperties already provides its own containsKey method that should simply be changed to return getProperty(propname) != null to avoid this issue. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (KAFKA-1862) Pass in the Time object into OffsetManager
[ https://issues.apache.org/jira/browse/KAFKA-1862?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14299119#comment-14299119 ] Aditya A Auradkar commented on KAFKA-1862: -- Waiting for this to get committed. https://issues.apache.org/jira/browse/KAFKA-1634 Pass in the Time object into OffsetManager -- Key: KAFKA-1862 URL: https://issues.apache.org/jira/browse/KAFKA-1862 Project: Kafka Issue Type: Bug Reporter: Guozhang Wang Assignee: Aditya Auradkar Labels: newbie++ Fix For: 0.9.0 We should improve OffsetManager to take in a Time instance as we do for LogManager and ReplicaManager. That way we can advance time with MockTime in test cases. Then we can move the testOffsetExpiration case from OffsetCommitTest to OffsetManagerTest. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (KAFKA-1476) Get a list of consumer groups
[ https://issues.apache.org/jira/browse/KAFKA-1476?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14299071#comment-14299071 ] Onur Karaman commented on KAFKA-1476: - Updated reviewboard https://reviews.apache.org/r/29831/diff/ against branch origin/trunk Get a list of consumer groups - Key: KAFKA-1476 URL: https://issues.apache.org/jira/browse/KAFKA-1476 Project: Kafka Issue Type: Wish Components: tools Affects Versions: 0.8.1.1 Reporter: Ryan Williams Assignee: Balaji Seshadri Labels: newbie Fix For: 0.9.0 Attachments: ConsumerCommand.scala, KAFKA-1476-LIST-GROUPS.patch, KAFKA-1476-RENAME.patch, KAFKA-1476-REVIEW-COMMENTS.patch, KAFKA-1476.patch, KAFKA-1476.patch, KAFKA-1476.patch, KAFKA-1476.patch, KAFKA-1476_2014-11-10_11:58:26.patch, KAFKA-1476_2014-11-10_12:04:01.patch, KAFKA-1476_2014-11-10_12:06:35.patch, KAFKA-1476_2014-12-05_12:00:12.patch, KAFKA-1476_2015-01-12_16:22:26.patch, KAFKA-1476_2015-01-12_16:31:20.patch, KAFKA-1476_2015-01-13_10:36:18.patch, KAFKA-1476_2015-01-15_14:30:04.patch, KAFKA-1476_2015-01-22_02:32:52.patch, KAFKA-1476_2015-01-30_11:09:59.patch, sample-kafka-consumer-groups-sh-output-1-23-2015.txt, sample-kafka-consumer-groups-sh-output.txt It would be useful to have a way to get a list of consumer groups currently active via some tool/script that ships with kafka. This would be helpful so that the system tools can be explored more easily. For example, when running the ConsumerOffsetChecker, it requires a group option bin/kafka-run-class.sh kafka.tools.ConsumerOffsetChecker --topic test --group ? But, when just getting started with kafka, using the console producer and consumer, it is not clear what value to use for the group option. If a list of consumer groups could be listed, then it would be clear what value to use. Background: http://mail-archives.apache.org/mod_mbox/kafka-users/201405.mbox/%3cCAOq_b1w=slze5jrnakxvak0gu9ctdkpazak1g4dygvqzbsg...@mail.gmail.com%3e -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (KAFKA-1476) Get a list of consumer groups
[ https://issues.apache.org/jira/browse/KAFKA-1476?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Onur Karaman updated KAFKA-1476: Attachment: KAFKA-1476_2015-01-30_11:09:59.patch Get a list of consumer groups - Key: KAFKA-1476 URL: https://issues.apache.org/jira/browse/KAFKA-1476 Project: Kafka Issue Type: Wish Components: tools Affects Versions: 0.8.1.1 Reporter: Ryan Williams Assignee: Balaji Seshadri Labels: newbie Fix For: 0.9.0 Attachments: ConsumerCommand.scala, KAFKA-1476-LIST-GROUPS.patch, KAFKA-1476-RENAME.patch, KAFKA-1476-REVIEW-COMMENTS.patch, KAFKA-1476.patch, KAFKA-1476.patch, KAFKA-1476.patch, KAFKA-1476.patch, KAFKA-1476_2014-11-10_11:58:26.patch, KAFKA-1476_2014-11-10_12:04:01.patch, KAFKA-1476_2014-11-10_12:06:35.patch, KAFKA-1476_2014-12-05_12:00:12.patch, KAFKA-1476_2015-01-12_16:22:26.patch, KAFKA-1476_2015-01-12_16:31:20.patch, KAFKA-1476_2015-01-13_10:36:18.patch, KAFKA-1476_2015-01-15_14:30:04.patch, KAFKA-1476_2015-01-22_02:32:52.patch, KAFKA-1476_2015-01-30_11:09:59.patch, sample-kafka-consumer-groups-sh-output-1-23-2015.txt, sample-kafka-consumer-groups-sh-output.txt It would be useful to have a way to get a list of consumer groups currently active via some tool/script that ships with kafka. This would be helpful so that the system tools can be explored more easily. For example, when running the ConsumerOffsetChecker, it requires a group option bin/kafka-run-class.sh kafka.tools.ConsumerOffsetChecker --topic test --group ? But, when just getting started with kafka, using the console producer and consumer, it is not clear what value to use for the group option. If a list of consumer groups could be listed, then it would be clear what value to use. Background: http://mail-archives.apache.org/mod_mbox/kafka-users/201405.mbox/%3cCAOq_b1w=slze5jrnakxvak0gu9ctdkpazak1g4dygvqzbsg...@mail.gmail.com%3e -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (KAFKA-1810) Add IP Filtering / Whitelists-Blacklists
[ https://issues.apache.org/jira/browse/KAFKA-1810?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14299202#comment-14299202 ] Tong Li commented on KAFKA-1810: rather than add specific security measures, can we add some kind of plugin point so that any plugins can be configured to do that type of work. Either it is a IP filter or certificate filter or basic authentication filter we can simply enable these plugins according to our own needs. This way, kafka only provide the plugin point, nothing else, how the plugin gets developed , performs, are not really the concern of the kafka community, we can have a clear separation of concerns. This has been done in many other successful projects, new to kafka, just saying we can do some thing like middle ware (in python term) or servlet filter in java world. Add IP Filtering / Whitelists-Blacklists - Key: KAFKA-1810 URL: https://issues.apache.org/jira/browse/KAFKA-1810 Project: Kafka Issue Type: New Feature Components: core, network, security Reporter: Jeff Holoman Assignee: Jeff Holoman Priority: Minor Fix For: 0.8.3 Attachments: KAFKA-1810.patch, KAFKA-1810_2015-01-15_19:47:14.patch While longer-term goals of security in Kafka are on the roadmap there exists some value for the ability to restrict connection to Kafka brokers based on IP address. This is not intended as a replacement for security but more of a precaution against misconfiguration and to provide some level of control to Kafka administrators about who is reading/writing to their cluster. 1) In some organizations software administration vs o/s systems administration and network administration is disjointed and not well choreographed. Providing software administrators the ability to configure their platform relatively independently (after initial configuration) from Systems administrators is desirable. 2) Configuration and deployment is sometimes error prone and there are situations when test environments could erroneously read/write to production environments 3) An additional precaution against reading sensitive data is typically welcomed in most large enterprise deployments. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
