Re: [DISCUSS] KIP-47 - Add timestamp-based log deletion policy

Sorry that I didn't see this comment before the meeting Joel. I'll try to clarify what I said at the meeting: - The KIP currently states that timestamp-based log deletion will only work with LogAppendTime. I need to update the KIP to reflect that, after the work is done for KIP-33, it will work

[jira] [Commented] (KAFKA-3042) updateIsr should stop after failed several times due to zkVersion issue

[ https://issues.apache.org/jira/browse/KAFKA-3042?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15159676#comment-15159676 ] James Cheng commented on KAFKA-3042: [~fpj], I have controller.log, state-change.log, and server.log

Re: [DISCUSS] KIP-43: Kafka SASL enhancements

I have updated the KIP based on the discussion in the KIP meeting today. Comments and feedback are welcome. On Wed, Feb 3, 2016 at 7:20 PM, Rajini Sivaram wrote: > Hi Harsha, > > Thank you for the review. Can you clarify - I think you are saying that > the client

Re: Kafka KIP meeting Feb 23 at 11:00am PST

The following are the notes from today's KIP discussion. - KIP-33 and KIP-47: No issues. Will start the voting thread. - KIP-43: We discussed whether there is a need to support multiple SASL mechanisms at the same time and what's the best way to implement this. Will discuss this in

[GitHub] kafka pull request: Kafka 3270 Added some Happy Path Tests for the...

GitHub user benstopford opened a pull request: https://github.com/apache/kafka/pull/956 Kafka 3270 Added some Happy Path Tests for the Reassign Partitions Command You can merge this pull request into a Git repository by running: $ git pull https://github.com/benstopford/kafka

Build failed in Jenkins: kafka-trunk-jdk7 #1058

See Changes: [me] KAFKA-2802: kafka streams system tests -- [...truncated 1826 lines...] kafka.log.BrokerCompressionTest > testBrokerSideCompression[1] PASSED kafka.log.BrokerCompressionTest

[jira] [Commented] (KAFKA-3242) "Add Partition" log message doesn't actually indicate adding a partition

[ https://issues.apache.org/jira/browse/KAFKA-3242?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15159629#comment-15159629 ] ASF GitHub Bot commented on KAFKA-3242: --- Github user asfgit closed the pull request at:

[jira] [Resolved] (KAFKA-3242) "Add Partition" log message doesn't actually indicate adding a partition

[ https://issues.apache.org/jira/browse/KAFKA-3242?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gwen Shapira resolved KAFKA-3242. - Resolution: Fixed Fix Version/s: 0.9.1.0 Issue resolved by pull request 924

[GitHub] kafka pull request: KAFKA-3242: minor rename / logging change to C...

Github user asfgit closed the pull request at: https://github.com/apache/kafka/pull/924 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

Re: [VOTE] Make next Kafka release 0.10.0.0 instead of 0.9.1.0

+1. On Sat, Feb 20, 2016 at 7:28 AM, Becket Qin wrote: > Hi Grant, > > Thanks for bringing up the compatibility and feature discussion. > > Ismael had similar concern about the short release interval between 0.9.0.0 > and 0.10.0.0. > > From wire protocol perspective, the

[jira] [Commented] (KAFKA-2802) Add integration tests for Kafka Streams

[ https://issues.apache.org/jira/browse/KAFKA-2802?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15159573#comment-15159573 ] ASF GitHub Bot commented on KAFKA-2802: --- Github user asfgit closed the pull request at:

[GitHub] kafka pull request: KAFKA-2802: kafka streams system tests

Github user asfgit closed the pull request at: https://github.com/apache/kafka/pull/930 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[jira] [Resolved] (KAFKA-2802) Add integration tests for Kafka Streams

[ https://issues.apache.org/jira/browse/KAFKA-2802?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ewen Cheslack-Postava resolved KAFKA-2802. -- Resolution: Fixed Fix Version/s: 0.9.1.0 Issue resolved by pull request

[jira] [Updated] (KAFKA-2802) Add integration tests for Kafka Streams

[ https://issues.apache.org/jira/browse/KAFKA-2802?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ewen Cheslack-Postava updated KAFKA-2802: - Assignee: Yasuhiro Matsuda > Add integration tests for Kafka Streams >

[jira] [Comment Edited] (KAFKA-3262) Make KafkaStreams debugging friendly

[ https://issues.apache.org/jira/browse/KAFKA-3262?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15159545#comment-15159545 ] Jay Kreps edited comment on KAFKA-3262 at 2/23/16 7:58 PM: --- This is a good

[jira] [Commented] (KAFKA-3262) Make KafkaStreams debugging friendly

[ https://issues.apache.org/jira/browse/KAFKA-3262?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15159545#comment-15159545 ] Jay Kreps commented on KAFKA-3262: -- This is a good catch, I ran into this issue too. Another issue is

[GitHub] kafka pull request: Fix for https://issues.apache.org/jira/browse/...

GitHub user christian-posta opened a pull request: https://github.com/apache/kafka/pull/955 Fix for https://issues.apache.org/jira/browse/KAFKA-3272 to easily en… …able remote debugging to Kafka tools scripts You can merge this pull request into a Git repository by running:

[jira] [Commented] (KAFKA-3272) Add debugging options to kafka-run-class.sh so we can easily run remote debugging

[ https://issues.apache.org/jira/browse/KAFKA-3272?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15159515#comment-15159515 ] ASF GitHub Bot commented on KAFKA-3272: --- GitHub user christian-posta opened a pull request:

[jira] [Created] (KAFKA-3272) Add debugging options to kafka-run-class.sh so we can easily run remote debugging

Christian Posta created KAFKA-3272: -- Summary: Add debugging options to kafka-run-class.sh so we can easily run remote debugging Key: KAFKA-3272 URL: https://issues.apache.org/jira/browse/KAFKA-3272

[jira] [Created] (KAFKA-3271) Notification upon unclean leader election

Yasuhiro Matsuda created KAFKA-3271: --- Summary: Notification upon unclean leader election Key: KAFKA-3271 URL: https://issues.apache.org/jira/browse/KAFKA-3271 Project: Kafka Issue Type:

[jira] [Commented] (KAFKA-1215) Rack-Aware replica assignment option

[ https://issues.apache.org/jira/browse/KAFKA-1215?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15159491#comment-15159491 ] Aditya Auradkar commented on KAFKA-1215: [~allenxwang] - Is this patch ready for review? I noticed

RE: [DISCUSS] Deprecating the old Scala producers for the next release

>From my point of view, it seems very odd to deprecate the Scala producer but >not the consumer. So, I would vote to deprecate them both in 0.10. It doesn't sound like there's an established mechanism for deprecation. So, for the sake of discussion, how about: * Start with deprecation

[jira] [Work started] (KAFKA-3214) Add consumer system tests for compressed topics

[ https://issues.apache.org/jira/browse/KAFKA-3214?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on KAFKA-3214 started by Anna Povzner. --- > Add consumer system tests for compressed topics >

[jira] [Updated] (KAFKA-3214) Add consumer system tests for compressed topics

[ https://issues.apache.org/jira/browse/KAFKA-3214?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jason Gustafson updated KAFKA-3214: --- Assignee: Anna Povzner (was: Jason Gustafson) > Add consumer system tests for compressed

[jira] [Created] (KAFKA-3270) Add Basic Integration Tests Around ReassignPartitionsCommand

Ben Stopford created KAFKA-3270: --- Summary: Add Basic Integration Tests Around ReassignPartitionsCommand Key: KAFKA-3270 URL: https://issues.apache.org/jira/browse/KAFKA-3270 Project: Kafka

[jira] [Assigned] (KAFKA-3270) Add Basic Integration Tests Around ReassignPartitionsCommand

[ https://issues.apache.org/jira/browse/KAFKA-3270?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ben Stopford reassigned KAFKA-3270: --- Assignee: Ben Stopford > Add Basic Integration Tests Around ReassignPartitionsCommand >

[jira] [Commented] (KAFKA-3201) Add system test for KIP-31 and KIP-32 - Upgrade Test

[ https://issues.apache.org/jira/browse/KAFKA-3201?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15159416#comment-15159416 ] Anna Povzner commented on KAFKA-3201: - Here is a set of tests: 1. Setup: Producer (0.8) → Kafka

Re: [DISCUSS] Deprecating the old Scala producers for the next release

It does make sense, thanks for the clarification. If we deprecate the producer first, does it mean that the following release won't have a scala producer but will have a scala consumer? Actually I should have asked this question first: what's the deprecation path precisely? -Flavio > On 23

Re: [DISCUSS] Deprecating the old Scala producers for the next release

Would it make more sense to be able to announce both new clients stable and deprecate both the scala clients at the same time (irrespective of whether that is next release or one after)? Otherwise is an intermediate state where we recommend you get off the scala producer but not the scala consumer

Re: [DISCUSS] KIP-47 - Add timestamp-based log deletion policy

I'm having some trouble reconciling the current proposal with your original requirement which was essentially being able to purge log data up to a precise point (an offset). The KIP currently suggests that timestamp-based deletion would only work with LogAppendTime, so it does not seem

[jira] [Work started] (KAFKA-3267) Describe/Alter Configs protocol and server side implementation

[ https://issues.apache.org/jira/browse/KAFKA-3267?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on KAFKA-3267 started by Grant Henke. -- > Describe/Alter Configs protocol and server side implementation >

[jira] [Work started] (KAFKA-3265) Create Java Admin Client

[ https://issues.apache.org/jira/browse/KAFKA-3265?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on KAFKA-3265 started by Grant Henke. -- > Create Java Admin Client > > > Key:

[jira] [Created] (KAFKA-3269) Update topic metadata request to included needed information for polling

Grant Henke created KAFKA-3269: -- Summary: Update topic metadata request to included needed information for polling Key: KAFKA-3269 URL: https://issues.apache.org/jira/browse/KAFKA-3269 Project: Kafka

[jira] [Work started] (KAFKA-2947) AlterTopic - protocol and server side implementation

[ https://issues.apache.org/jira/browse/KAFKA-2947?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on KAFKA-2947 started by Grant Henke. -- > AlterTopic - protocol and server side implementation >

[jira] [Work started] (KAFKA-3266) List/Alter Acls - protocol and server side implementation

[ https://issues.apache.org/jira/browse/KAFKA-3266?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on KAFKA-3266 started by Grant Henke. -- > List/Alter Acls - protocol and server side implementation >

[jira] [Created] (KAFKA-3265) Create Java Admin Client

Grant Henke created KAFKA-3265: -- Summary: Create Java Admin Client Key: KAFKA-3265 URL: https://issues.apache.org/jira/browse/KAFKA-3265 Project: Kafka Issue Type: Sub-task

[jira] [Updated] (KAFKA-1777) Re-factor reasign-partitions into CLI

[ https://issues.apache.org/jira/browse/KAFKA-1777?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Grant Henke updated KAFKA-1777: --- Issue Type: Improvement (was: Sub-task) Parent: (was: KAFKA-1694) > Re-factor

[jira] [Resolved] (KAFKA-1777) Re-factor reasign-partitions into CLI

[ https://issues.apache.org/jira/browse/KAFKA-1777?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Grant Henke resolved KAFKA-1777. Resolution: Duplicate > Re-factor reasign-partitions into CLI >

[jira] [Commented] (KAFKA-3241) JmxReporter produces invalid JSON when a value is Infinity

[ https://issues.apache.org/jira/browse/KAFKA-3241?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15159272#comment-15159272 ] Babak Behzad commented on KAFKA-3241: - sure [~vahid]. We use Kafka's Metrics sink in Hadoop and as

[jira] [Resolved] (KAFKA-1802) Add a new type of request for the discovery of the controller

[ https://issues.apache.org/jira/browse/KAFKA-1802?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Grant Henke resolved KAFKA-1802. Resolution: Won't Fix Assignee: Grant Henke (was: Andrii Biletskyi) > Add a new type of

[jira] [Updated] (KAFKA-1125) Add options to let admin tools block until finish

[ https://issues.apache.org/jira/browse/KAFKA-1125?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Grant Henke updated KAFKA-1125: --- Issue Type: Improvement (was: Sub-task) Parent: (was: KAFKA-1694) > Add options to let

[jira] [Resolved] (KAFKA-1125) Add options to let admin tools block until finish

[ https://issues.apache.org/jira/browse/KAFKA-1125?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Grant Henke resolved KAFKA-1125. Resolution: Duplicate Assignee: Grant Henke (was: Guozhang Wang) > Add options to let admin

[jira] [Updated] (KAFKA-1775) Re-factor TopicCommand into thew handerAdminMessage call

[ https://issues.apache.org/jira/browse/KAFKA-1775?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Grant Henke updated KAFKA-1775: --- Issue Type: Improvement (was: Sub-task) Parent: (was: KAFKA-1694) > Re-factor

[jira] [Resolved] (KAFKA-1775) Re-factor TopicCommand into thew handerAdminMessage call

[ https://issues.apache.org/jira/browse/KAFKA-1775?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Grant Henke resolved KAFKA-1775. Resolution: Won't Fix > Re-factor TopicCommand into thew handerAdminMessage call >

[jira] [Updated] (KAFKA-1802) Add a new type of request for the discovery of the controller

[ https://issues.apache.org/jira/browse/KAFKA-1802?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Grant Henke updated KAFKA-1802: --- Issue Type: Improvement (was: Sub-task) Parent: (was: KAFKA-1694) > Add a new type of

[jira] [Commented] (KAFKA-1786) implement a global configuration feature for brokers

[ https://issues.apache.org/jira/browse/KAFKA-1786?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15159256#comment-15159256 ] Grant Henke commented on KAFKA-1786: Moved out of KIP-4 scope > implement a global configuration

[jira] [Updated] (KAFKA-1786) implement a global configuration feature for brokers

[ https://issues.apache.org/jira/browse/KAFKA-1786?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Grant Henke updated KAFKA-1786: --- Issue Type: New Feature (was: Sub-task) Parent: (was: KAFKA-1694) > implement a global

Re: [DISCUSS] Deprecating the old Scala producers for the next release

Hi Flavio, On Tue, Feb 23, 2016 at 9:23 AM, Flavio Junqueira wrote: > Ismael, I'm curious about why you want to deprecate one and not the other. > You say that it is premature to deprecate the old scala consumers and I'm > wondering about the reasoning behind it. > The new

Re: [DISCUSS] Deprecating the old Scala producers for the next release

Ismael, I'm curious about why you want to deprecate one and not the other. You say that it is premature to deprecate the old scala consumers and I'm wondering about the reasoning behind it. -Flavio > On 22 Feb 2016, at 17:36, Ismael Juma wrote: > > Hi all, > > The new

Re: [DISCUSS] Deprecating the old Scala producers for the next release

Thanks Grant. On Tue, Feb 23, 2016 at 8:44 AM, Grant Henke wrote: > +1 > > I had a jira tracking marking both as deprecated. I agree we should wait to > deprecate the old consumer. I changed the existing jira to be for the > producer and created a new jira to track

Re: [DISCUSS] Deprecating the old Scala producers for the next release

+1 I had a jira tracking marking both as deprecated. I agree we should wait to deprecate the old consumer. I changed the existing jira to be for the producer and created a new jira to track deprecating the the consumer sometime in the future (likely in the 0.11 release). - KAFKA-2982: Mark

[jira] [Updated] (KAFKA-3264) Mark the old Scala consumer and related classes as deprecated

[ https://issues.apache.org/jira/browse/KAFKA-3264?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Grant Henke updated KAFKA-3264: --- Affects Version/s: (was: 0.9.0.0) 0.10.0.0 > Mark the old Scala consumer

[jira] [Updated] (KAFKA-3264) Mark the old Scala consumer and related classes as deprecated

[ https://issues.apache.org/jira/browse/KAFKA-3264?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ismael Juma updated KAFKA-3264: --- Description: Once the new consumer is out of beta, we should consider deprecating the old Scala

[jira] [Updated] (KAFKA-3264) Mark the old Scala consumer and related classes as deprecated

[ https://issues.apache.org/jira/browse/KAFKA-3264?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Grant Henke updated KAFKA-3264: --- Affects Version/s: (was: 0.10.0.0) > Mark the old Scala consumer and related classes as

[jira] [Updated] (KAFKA-2982) Mark the old Scala producer related classes as deprecated

[ https://issues.apache.org/jira/browse/KAFKA-2982?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Grant Henke updated KAFKA-2982: --- Fix Version/s: 0.10.0.0 > Mark the old Scala producer related classes as deprecated >

[jira] [Created] (KAFKA-3264) Mark the old Scala consumer and related classes as deprecated

Grant Henke created KAFKA-3264: -- Summary: Mark the old Scala consumer and related classes as deprecated Key: KAFKA-3264 URL: https://issues.apache.org/jira/browse/KAFKA-3264 Project: Kafka

[jira] [Updated] (KAFKA-3264) Mark the old Scala consumer and related classes as deprecated

[ https://issues.apache.org/jira/browse/KAFKA-3264?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Grant Henke updated KAFKA-3264: --- Fix Version/s: (was: 0.10.0.0) > Mark the old Scala consumer and related classes as deprecated >

[jira] [Updated] (KAFKA-2982) Mark the old Scala producer and related classes as deprecated

[ https://issues.apache.org/jira/browse/KAFKA-2982?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Grant Henke updated KAFKA-2982: --- Summary: Mark the old Scala producer and related classes as deprecated (was: Mark the old Scala

[jira] [Updated] (KAFKA-2982) Mark the old Scala producer related classes as deprecated

[ https://issues.apache.org/jira/browse/KAFKA-2982?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Grant Henke updated KAFKA-2982: --- Summary: Mark the old Scala producer related classes as deprecated (was: Mark the old Scala producer

[jira] [Updated] (KAFKA-2982) Mark the old Scala producer related classes as deprecated

[ https://issues.apache.org/jira/browse/KAFKA-2982?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Grant Henke updated KAFKA-2982: --- Affects Version/s: 0.9.0.0 > Mark the old Scala producer related classes as deprecated >

[jira] [Updated] (KAFKA-2982) Mark the old Scala producer related classes as deprecated

[ https://issues.apache.org/jira/browse/KAFKA-2982?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Grant Henke updated KAFKA-2982: --- Assignee: (was: Grant Henke) > Mark the old Scala producer related classes as deprecated >

[jira] [Commented] (KAFKA-2066) Replace FetchRequest / FetchResponse with their org.apache.kafka.common.requests equivalents

[ https://issues.apache.org/jira/browse/KAFKA-2066?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15158985#comment-15158985 ] Ismael Juma commented on KAFKA-2066: [~dajac], the PR for KIP-31 and KIP-32 has been merged (FYI). >

[GitHub] kafka pull request: [MINOR] Fixed documenation of parameter "block...

GitHub user mjsax opened a pull request: https://github.com/apache/kafka/pull/954 [MINOR] Fixed documenation of parameter "block.on.buffer.full" default value is "false" and not "true" See:

Tests taking too long (> 120mins) cause PR's to fail

I've noticed a few PRs recently (including one of mine :) ) are not passing initial checks because the Jenkins job times out (looks like all tests are passing, just takes too long) Is this a known issue? Can we increase the timeout (or make the tests faster :) )?

[jira] [Commented] (KAFKA-3042) updateIsr should stop after failed several times due to zkVersion issue

[ https://issues.apache.org/jira/browse/KAFKA-3042?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15158807#comment-15158807 ] Flavio Junqueira commented on KAFKA-3042: - bq. I think this is because the broker consider itself

[jira] [Commented] (KAFKA-1382) Update zkVersion on partition state update failures

[ https://issues.apache.org/jira/browse/KAFKA-1382?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15158799#comment-15158799 ] Flavio Junqueira commented on KAFKA-1382: - hi there, it isn't clear to me if you've hit this bug

[jira] [Commented] (KAFKA-2729) Cached zkVersion not equal to that in zookeeper, broker not recovering.

[ https://issues.apache.org/jira/browse/KAFKA-2729?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15158771#comment-15158771 ] Petri Lehtinen commented on KAFKA-2729: --- This happened to me (again) a few days ago on 0.9.0.0 on a

[GitHub] kafka pull request: In daemon mode,redirect stdout to /dev/null

GitHub user tomjiang1987 opened a pull request: https://github.com/apache/kafka/pull/953 In daemon mode,redirect stdout to /dev/null when start in deamon mode,don't output console to file,because the console output file will keep increased. so redirect stdout to /dev/null.

<    1   2