[jira] [Updated] (KAFKA-4570) How to transfer extended fields in producing or consuming requests.

2017-01-09 Thread Ewen Cheslack-Postava (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-4570?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ewen Cheslack-Postava updated KAFKA-4570: - Component/s: security core > How to transfer extended fields in

Re: [DISCUSS] KIP-110: Add Codec for ZStandard Compression

2017-01-09 Thread Dongjin Lee
Ismael, 1. Is the benchmark in the KIP page not enough? You mean we need a whole performance test using kafka-producer-perf-test.sh? 2. It seems like no major project is relying on it currently. However, after reviewing the code, I concluded that at least this project has a good test coverage.

[jira] [Commented] (KAFKA-4570) How to transfer extended fields in producing or consuming requests.

2017-01-09 Thread zander (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-4570?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15814218#comment-15814218 ] zander commented on KAFKA-4570: --- [~ewencp] In my use case, the user has Access Key and Secret Key and he

Jenkins build is back to normal : kafka-trunk-jdk8 #1160

2017-01-09 Thread Apache Jenkins Server
See

[GitHub] kafka pull request #2336: MINOR: Fix small error in javadoc for persistent S...

2017-01-09 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/kafka/pull/2336 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[jira] [Resolved] (KAFKA-4455) CommitFailedException during rebalance doesn't release resources in tasks/processors

2017-01-09 Thread Ewen Cheslack-Postava (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-4455?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ewen Cheslack-Postava resolved KAFKA-4455. -- Resolution: Fixed Assignee: Ewen Cheslack-Postava Fix Version/s:

[jira] [Commented] (KAFKA-4610) getting error:Batch containing 3 record(s) expired due to timeout while requesting metadata from brokers for test2R2P2-1

2017-01-09 Thread Ewen Cheslack-Postava (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-4610?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15814103#comment-15814103 ] Ewen Cheslack-Postava commented on KAFKA-4610: -- Are any of your messages making it to Kafka

[jira] [Updated] (KAFKA-4610) getting error:Batch containing 3 record(s) expired due to timeout while requesting metadata from brokers for test2R2P2-1

2017-01-09 Thread Ewen Cheslack-Postava (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-4610?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ewen Cheslack-Postava updated KAFKA-4610: - Fix Version/s: (was: 0.10.2.0) > getting error:Batch containing 3 record(s)

[jira] [Commented] (KAFKA-4556) unordered messages when multiple topics are combined in single topic through stream

2017-01-09 Thread Ewen Cheslack-Postava (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-4556?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15814086#comment-15814086 ] Ewen Cheslack-Postava commented on KAFKA-4556: -- The reason you are seeing inconsistent

[GitHub] kafka pull request #2338: MINOR: remove unnecessary store info from Topology...

2017-01-09 Thread guozhangwang
GitHub user guozhangwang opened a pull request: https://github.com/apache/kafka/pull/2338 MINOR: remove unnecessary store info from TopologyBuilder This PR is extracted from https://github.com/apache/kafka/pull/2333 as an incremental fix to ease the reviewing: 1. Removed

Re: 0.10.2.0 release planning

2017-01-09 Thread Ewen Cheslack-Postava
As a quick follow up, for community members looking for areas they are familiar with to review I have updated the KIP table at the bottom of https://cwiki.apache.org/confluence/display/KAFKA/Release+Plan+0.10.2.0 with a column annotating the area of code the KIP/PR is related to (e.g. consumer,

[GitHub] kafka pull request #2335: TRIVIAL: Fix spelling of log message

2017-01-09 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/kafka/pull/2335 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

Re: [DISCUSS] KIP-109: Old Consumer Deprecation

2017-01-09 Thread Ewen Cheslack-Postava
I don't mean to derail this at all, but given Ismael's question I'm wondering what exactly we consider a message format change? Are we assuming that additions like a new compression format (i.e. semantic changes where previously undefined bits, but bits which existed) don't require a format

[jira] [Commented] (KAFKA-4570) How to transfer extended fields in producing or consuming requests.

2017-01-09 Thread Ewen Cheslack-Postava (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-4570?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15813908#comment-15813908 ] Ewen Cheslack-Postava commented on KAFKA-4570: -- [~zander] I am not super familiar with the

[jira] [Commented] (KAFKA-4569) Transient failure in org.apache.kafka.clients.consumer.KafkaConsumerTest.testWakeupWithFetchDataAvailable

2017-01-09 Thread Umesh Chaudhary (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-4569?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15813892#comment-15813892 ] Umesh Chaudhary commented on KAFKA-4569: Hi [~ijuma], One observation regarding the case. I saw

Re: [VOTE] KIP-104: Granular Sensors for Streams

2017-01-09 Thread Neha Narkhede
+1. Thanks Eno! On Mon, Jan 9, 2017 at 1:05 PM Guozhang Wang wrote: > +1, thanks. > > On Jan 9, 2017 12:27 PM, "Ismael Juma" wrote: > > Thanks for the KIP, +1 on the latest update. > > Ismael > > On Fri, Jan 6, 2017 at 9:37 AM, Eno Thereska

[GitHub] kafka pull request #2320: MINOR: Maybe decorate inner topics for SourceNode

2017-01-09 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/kafka/pull/2320 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[jira] [Updated] (KAFKA-4603) the argument of shell in doc wrong and command parsed error

2017-01-09 Thread Guozhang Wang (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-4603?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guozhang Wang updated KAFKA-4603: - Assignee: Xin > the argument of shell in doc wrong and command parsed error >

[jira] [Commented] (KAFKA-4570) How to transfer extended fields in producing or consuming requests.

2017-01-09 Thread zander (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-4570?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15813696#comment-15813696 ] zander commented on KAFKA-4570: --- [~ewencp] Can I interpret it this way that there is no way to verify the

Re: [DISCUSS] KIP-109: Old Consumer Deprecation

2017-01-09 Thread Ismael Juma
Hi Joel, Great to hear that LinkedIn is likely to implement KAFKA-4513. :) Yes, the KIP as it stands is a compromise given the situation. We could push the deprecation to the subsequent release: likely to be 0.11.0.0 since there are a number of KIPs that require message format changes. This

[jira] [Updated] (KAFKA-4603) the argument of shell in doc wrong and command parsed error

2017-01-09 Thread Xin (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-4603?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xin updated KAFKA-4603: --- Summary: the argument of shell in doc wrong and command parsed error (was: shell in doc argument error and command

[jira] [Updated] (KAFKA-3751) Add support for SASL/SCRAM mechanisms

2017-01-09 Thread Ismael Juma (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-3751?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ismael Juma updated KAFKA-3751: --- Reviewer: Jun Rao (was: Ismael Juma) > Add support for SASL/SCRAM mechanisms >

Build failed in Jenkins: kafka-trunk-jdk8 #1159

2017-01-09 Thread Apache Jenkins Server
See Changes: [wangguoz] MINOR: some public JavaDoc cleanup -- [...truncated 18066 lines...] org.apache.kafka.streams.KafkaStreamsTest > testCleanup STARTED

[jira] [Commented] (KAFKA-4612) Simple Kafka Streams app causes "ClassCastException: java.lang.String cannot be cast to [B"

2017-01-09 Thread Matthias J. Sax (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-4612?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15813501#comment-15813501 ] Matthias J. Sax commented on KAFKA-4612: Did you configure global default key and value Serdes via

[jira] [Updated] (KAFKA-4603) shell in doc argument error and command parsed error

2017-01-09 Thread Xin (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-4603?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xin updated KAFKA-4603: --- Summary: shell in doc argument error and command parsed error (was: argument error,and command parsed error) >

Re: 0.10.2.0 release planning

2017-01-09 Thread Ewen Cheslack-Postava
Hey folks, Here's a quick update and reminder that feature freeze is coming up on Friday (don't worry, you'll have through the end of the day to merge stuff). Most importantly, *everyone* can contribute to landing the many KIPs and features we're trying to get in by Friday. Committers need to

[jira] [Updated] (KAFKA-4218) Enable access to key in {{ValueTransformer}}

2017-01-09 Thread Matthias J. Sax (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-4218?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthias J. Sax updated KAFKA-4218: --- Labels: api needs-kip newbie (was: api newbie) > Enable access to key in

[jira] [Updated] (KAFKA-4218) Enable access to key in ValueTransformer

2017-01-09 Thread Matthias J. Sax (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-4218?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthias J. Sax updated KAFKA-4218: --- Summary: Enable access to key in ValueTransformer (was: Enable access to key in

[jira] [Updated] (KAFKA-4217) KStream.transform equivalent of flatMap

2017-01-09 Thread Matthias J. Sax (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-4217?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthias J. Sax updated KAFKA-4217: --- Labels: api needs-kip newbie (was: api newbie) > KStream.transform equivalent of flatMap >

[jira] [Updated] (KAFKA-4346) Add foreachValue method to KStream

2017-01-09 Thread Matthias J. Sax (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-4346?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthias J. Sax updated KAFKA-4346: --- Labels: needs-kip newbie (was: newbie) > Add foreachValue method to KStream >

Re: [VOTE] KIP-108: Create Topic Policy

2017-01-09 Thread Sriram Subramanian
+1 On Mon, Jan 9, 2017 at 3:29 PM, Apurva Mehta wrote: > (hit send too soon) > > +1 (non-binding).. that is a very well written KIP! > > On Mon, Jan 9, 2017 at 3:28 PM, Apurva Mehta wrote: > > > +1, that1 > > > > On Mon, Jan 9, 2017 at 2:47 PM, Neha

[jira] [Created] (KAFKA-4612) Simple Kafka Streams app causes "ClassCastException: java.lang.String cannot be cast to [B"

2017-01-09 Thread Kurt Ostfeld (JIRA)
Kurt Ostfeld created KAFKA-4612: --- Summary: Simple Kafka Streams app causes "ClassCastException: java.lang.String cannot be cast to [B" Key: KAFKA-4612 URL: https://issues.apache.org/jira/browse/KAFKA-4612

Re: [DISCUSS] KIP-109: Old Consumer Deprecation

2017-01-09 Thread Joel Koshy
> > > The ideal scenario would be for us to provide a tool for no downtime > migration as discussed in the original thread (I filed > https://issues.apache.org/jira/browse/KAFKA-4513 in response to that > discussion). There are a few issues, however: > >- There doesn't seem to be much demand

[jira] [Work started] (KAFKA-4558) throttling_test fails if the producer starts too fast.

2017-01-09 Thread Apurva Mehta (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-4558?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on KAFKA-4558 started by Apurva Mehta. --- > throttling_test fails if the producer starts too fast. >

[jira] [Updated] (KAFKA-4426) Add consumer.close(timeout, unit) for graceful close with timeout

2017-01-09 Thread Ewen Cheslack-Postava (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-4426?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ewen Cheslack-Postava updated KAFKA-4426: - Status: Patch Available (was: Open) > Add consumer.close(timeout, unit) for

[jira] [Updated] (KAFKA-4381) Add per partition lag metric to KafkaConsumer.

2017-01-09 Thread Ewen Cheslack-Postava (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-4381?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ewen Cheslack-Postava updated KAFKA-4381: - Reviewer: Ewen Cheslack-Postava > Add per partition lag metric to KafkaConsumer.

[jira] [Updated] (KAFKA-4565) Separation of Internal and External traffic (KIP-103)

2017-01-09 Thread Ewen Cheslack-Postava (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-4565?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ewen Cheslack-Postava updated KAFKA-4565: - Reviewer: Jason Gustafson > Separation of Internal and External traffic (KIP-103)

[jira] [Updated] (KAFKA-4591) Create Topic Policy

2017-01-09 Thread Ewen Cheslack-Postava (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-4591?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ewen Cheslack-Postava updated KAFKA-4591: - Reviewer: Jason Gustafson > Create Topic Policy > --- > >

[jira] [Updated] (KAFKA-4426) Add consumer.close(timeout, unit) for graceful close with timeout

2017-01-09 Thread Ewen Cheslack-Postava (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-4426?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ewen Cheslack-Postava updated KAFKA-4426: - Reviewer: Jason Gustafson > Add consumer.close(timeout, unit) for graceful close

[jira] [Updated] (KAFKA-4462) Improved Kafka Client Compatibility Policy

2017-01-09 Thread Ewen Cheslack-Postava (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-4462?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ewen Cheslack-Postava updated KAFKA-4462: - Reviewer: Jason Gustafson > Improved Kafka Client Compatibility Policy >

[jira] [Updated] (KAFKA-3751) Add support for SASL/SCRAM mechanisms

2017-01-09 Thread Ewen Cheslack-Postava (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-3751?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ewen Cheslack-Postava updated KAFKA-3751: - Reviewer: Ismael Juma > Add support for SASL/SCRAM mechanisms >

[jira] [Updated] (KAFKA-4602) KIP-72 Allow putting a bound on memory consumed by Incoming requests

2017-01-09 Thread Ewen Cheslack-Postava (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-4602?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ewen Cheslack-Postava updated KAFKA-4602: - Reviewer: Ewen Cheslack-Postava > KIP-72 Allow putting a bound on memory consumed

[jira] [Updated] (KAFKA-4481) Relax Kafka Streams API type constraints

2017-01-09 Thread Ewen Cheslack-Postava (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-4481?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ewen Cheslack-Postava updated KAFKA-4481: - Reviewer: Guozhang Wang > Relax Kafka Streams API type constraints >

[jira] [Updated] (KAFKA-3853) Report offsets for empty groups in ConsumerGroupCommand

2017-01-09 Thread Ewen Cheslack-Postava (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-3853?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ewen Cheslack-Postava updated KAFKA-3853: - Reviewer: Ewen Cheslack-Postava > Report offsets for empty groups in

[jira] [Updated] (KAFKA-3209) Support single message transforms in Kafka Connect

2017-01-09 Thread Ewen Cheslack-Postava (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-3209?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ewen Cheslack-Postava updated KAFKA-3209: - Reviewer: Ewen Cheslack-Postava > Support single message transforms in Kafka

Re: [VOTE] KIP-108: Create Topic Policy

2017-01-09 Thread Apurva Mehta
(hit send too soon) +1 (non-binding).. that is a very well written KIP! On Mon, Jan 9, 2017 at 3:28 PM, Apurva Mehta wrote: > +1, that1 > > On Mon, Jan 9, 2017 at 2:47 PM, Neha Narkhede wrote: > >> +1 - thanks Ismael! >> >> On Mon, Jan 9, 2017 at 2:30

Re: [VOTE] KIP-108: Create Topic Policy

2017-01-09 Thread Apurva Mehta
+1, that1 On Mon, Jan 9, 2017 at 2:47 PM, Neha Narkhede wrote: > +1 - thanks Ismael! > > On Mon, Jan 9, 2017 at 2:30 PM Gwen Shapira wrote: > > > +1 - thanks for the proposal, I think it will be super useful for admins. > > > > On Sun, Jan 8, 2017 at 6:50

[jira] [Commented] (KAFKA-4125) Provide low-level Processor API meta data in DSL layer

2017-01-09 Thread Guozhang Wang (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-4125?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15813181#comment-15813181 ] Guozhang Wang commented on KAFKA-4125: -- [~bbejeck] Could you lead a KIP proposal / discussion on this

[GitHub] kafka pull request #2332: MINOR: some public JavaDoc cleanup

2017-01-09 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/kafka/pull/2332 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

Re: [DISCUSS] KIP-104: Granular Sensors for Streams

2017-01-09 Thread Guozhang Wang
Thanks!! On Mon, Jan 9, 2017 at 12:18 PM, Eno Thereska wrote: > Sure, updated the KIP for both. > > Thanks > Eno > > On 9 Jan 2017, at 19:23, Guozhang Wang wrote: > > > > Thanks Eno, a couple of follow-up comments: > > > > 1. About `sensor()`

Re: [VOTE] KIP-108: Create Topic Policy

2017-01-09 Thread Neha Narkhede
+1 - thanks Ismael! On Mon, Jan 9, 2017 at 2:30 PM Gwen Shapira wrote: > +1 - thanks for the proposal, I think it will be super useful for admins. > > On Sun, Jan 8, 2017 at 6:50 AM, Ismael Juma wrote: > > Hi all, > > > > As the discussion seems to have

Re: [DISCUSS] KIP-105: Addition of Record Level for Sensors

2017-01-09 Thread Eno Thereska
Thanks Joel, I'll address these tomorrow, no problem. Eno > On 9 Jan 2017, at 22:04, Joel Koshy wrote: > > Didn't get a chance to review this earlier, but this LGTM. Minor comments: > - The current name is fine, but I would have preferred calling it > RecordingLevel to

Re: [VOTE] KIP-108: Create Topic Policy

2017-01-09 Thread Gwen Shapira
+1 - thanks for the proposal, I think it will be super useful for admins. On Sun, Jan 8, 2017 at 6:50 AM, Ismael Juma wrote: > Hi all, > > As the discussion seems to have settled down, I would like to initiate the > voting process for KIP-108: Create Topic Policy: > >

[GitHub] kafka pull request #2337: MINOR: rework JavaDoc for windowing related public...

2017-01-09 Thread mjsax
GitHub user mjsax opened a pull request: https://github.com/apache/kafka/pull/2337 MINOR: rework JavaDoc for windowing related public API - also some code refactoring and bug fixes You can merge this pull request into a Git repository by running: $ git pull

Re: [VOTE] KIP-105: Addition of Record Level for Sensors

2017-01-09 Thread Joel Koshy
+1 (although I added a minor comment on the discussion thread) On Mon, Jan 9, 2017 at 3:43 AM, Michael Noll wrote: > +1 (non-binding) > > On Fri, Jan 6, 2017 at 6:12 PM, Matthias J. Sax > wrote: > > > +1 > > > > On 1/6/17 9:09 AM, Neha Narkhede

Re: [DISCUSS] KIP-105: Addition of Record Level for Sensors

2017-01-09 Thread Joel Koshy
Didn't get a chance to review this earlier, but this LGTM. Minor comments: - The current name is fine, but I would have preferred calling it RecordingLevel to RecordLevel - first thing that comes to my mind with RecordLevel is Kafka records. - Under future work: it may be useful to allow

Re: [VOTE] KIP-103: Separation of Internal and External traffic

2017-01-09 Thread Joel Koshy
+1 On Fri, Jan 6, 2017 at 4:53 PM, Ismael Juma wrote: > Hi all, > > Since a few people (including myself) felt that listener name was clearer > than protocol label, I updated the KIP to use that (as mentioned in the > discuss thread). Given that this is a minor change, I

[jira] [Assigned] (KAFKA-4064) Add support for infinite endpoints for range queries in Kafka Streams KV stores

2017-01-09 Thread JIRA
[ https://issues.apache.org/jira/browse/KAFKA-4064?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xavier Léauté reassigned KAFKA-4064: Assignee: Xavier Léauté (was: Roger Hoover) > Add support for infinite endpoints for

Re: [VOTE] KIP-108: Create Topic Policy

2017-01-09 Thread Jon Chiu
+1 (non-binding) On Mon, Jan 9, 2017 at 1:49 PM, Joel Koshy wrote: > +1 > > Thanks for the KIP - this is a very useful feature. > > Thanks, > > Joel > > On Mon, Jan 9, 2017 at 2:15 AM, Edoardo Comar wrote: > > > Ismael, > > thanks for the KIP I see it as

Re: [VOTE] KIP-108: Create Topic Policy

2017-01-09 Thread Roger Hoover
+1 (non-binding) On Mon, Jan 9, 2017 at 2:15 AM, Edoardo Comar wrote: > Ismael, > thanks for the KIP I see it as quite useful in a managed cloud > environment. > > +1 (non-binding) > -- > Edoardo Comar > IBM MessageHub >

Re: [DISCUSS] KIP-108: Create Topic Policy

2017-01-09 Thread Roger Hoover
Got it. Thanks, Ismael. On Mon, Jan 9, 2017 at 10:42 AM, Ismael Juma wrote: > Hi Roger, > > That's a good question. The server defaults are passed via the `configure` > method of the `Configurable` interface that is implemented by > `CreateTopicPolicy`. I'll mention this

[jira] [Commented] (KAFKA-4125) Provide low-level Processor API meta data in DSL layer

2017-01-09 Thread Matthias J. Sax (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-4125?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15812918#comment-15812918 ] Matthias J. Sax commented on KAFKA-4125: I think we need a KIP for this -- we did a lot of API

Re: [VOTE] KIP-104: Granular Sensors for Streams

2017-01-09 Thread Guozhang Wang
+1, thanks. On Jan 9, 2017 12:27 PM, "Ismael Juma" wrote: Thanks for the KIP, +1 on the latest update. Ismael On Fri, Jan 6, 2017 at 9:37 AM, Eno Thereska wrote: > The discussion points for KIP-104 are addressed. At this point we'd like > to start

Re: out of the box security?

2017-01-09 Thread Ismael Juma
Hi Radai, This is a good point. At Confluent, we discussed this briefly a while back, but we never got around to working out a concrete proposal. It would be great if someone were interested in working out the details in a KIP. Ismael On Mon, Jan 9, 2017 at 6:13 PM, radai

Build failed in Jenkins: kafka-trunk-jdk8 #1158

2017-01-09 Thread Apache Jenkins Server
See Changes: [wangguoz] KAFKA-4551: StreamsSmokeTest.test_streams intermittent failure -- [...truncated 7998 lines...] kafka.utils.timer.TimerTaskListTest > testAll STARTED

Re: [VOTE] KIP-104: Granular Sensors for Streams

2017-01-09 Thread Ismael Juma
Thanks for the KIP, +1 on the latest update. Ismael On Fri, Jan 6, 2017 at 9:37 AM, Eno Thereska wrote: > The discussion points for KIP-104 are addressed. At this point we'd like > to start the vote for it: > > https://cwiki.apache.org/confluence/display/KAFKA/KIP- >

Re: [DISCUSS] KIP-104: Granular Sensors for Streams

2017-01-09 Thread Eno Thereska
Sure, updated the KIP for both. Thanks Eno > On 9 Jan 2017, at 19:23, Guozhang Wang wrote: > > Thanks Eno, a couple of follow-up comments: > > 1. About `sensor()` functions, could we rename them to `addSensor()` to be > consistent with other APIs? Also do we want to still

[GitHub] kafka pull request #2336: MINOR: Fix small error in javadoc for persistent S...

2017-01-09 Thread nixsticks
GitHub user nixsticks opened a pull request: https://github.com/apache/kafka/pull/2336 MINOR: Fix small error in javadoc for persistent Stores Closed the last PR I made for this because I accidentally borked it with my other PR. Small error, I figure this is from copy-pasting the

[jira] [Commented] (KAFKA-4547) Consumer.position returns incorrect results for Kafka 0.10.1.0 client

2017-01-09 Thread Vahid Hashemian (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-4547?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15812615#comment-15812615 ] Vahid Hashemian commented on KAFKA-4547: Thanks for clarifying. I'll submit a PR and will also

[jira] [Comment Edited] (KAFKA-4547) Consumer.position returns incorrect results for Kafka 0.10.1.0 client

2017-01-09 Thread Jason Gustafson (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-4547?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15812595#comment-15812595 ] Jason Gustafson edited comment on KAFKA-4547 at 1/9/17 7:23 PM: I was

Re: [DISCUSS] KIP-104: Granular Sensors for Streams

2017-01-09 Thread Guozhang Wang
Thanks Eno, a couple of follow-up comments: 1. About `sensor()` functions, could we rename them to `addSensor()` to be consistent with other APIs? Also do we want to still add the "String scopeName, String entityName, String operationName" parameters to enforce the naming hierarchy or just let

[jira] [Commented] (KAFKA-4547) Consumer.position returns incorrect results for Kafka 0.10.1.0 client

2017-01-09 Thread Jason Gustafson (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-4547?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15812595#comment-15812595 ] Jason Gustafson commented on KAFKA-4547: I was thinking in addition to your suggestion, which

[GitHub] kafka pull request #2327: MINOR: Fix small error in javadoc for persistent S...

2017-01-09 Thread nixsticks
Github user nixsticks closed the pull request at: https://github.com/apache/kafka/pull/2327 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[jira] [Commented] (KAFKA-4125) Provide low-level Processor API meta data in DSL layer

2017-01-09 Thread Bill Bejeck (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-4125?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15812586#comment-15812586 ] Bill Bejeck commented on KAFKA-4125: Is this task still viable/desired? If so I'll pick it up. >

[jira] [Commented] (KAFKA-4551) StreamsSmokeTest.test_streams intermittent failure

2017-01-09 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-4551?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15812576#comment-15812576 ] ASF GitHub Bot commented on KAFKA-4551: --- Github user asfgit closed the pull request at:

[GitHub] kafka pull request #2319: KAFKA-4551: StreamsSmokeTest.test_streams intermit...

2017-01-09 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/kafka/pull/2319 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[jira] [Resolved] (KAFKA-4551) StreamsSmokeTest.test_streams intermittent failure

2017-01-09 Thread Guozhang Wang (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-4551?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guozhang Wang resolved KAFKA-4551. -- Resolution: Fixed Fix Version/s: (was: 0.10.1.2) 0.10.2.0 Issue

[jira] [Commented] (KAFKA-4547) Consumer.position returns incorrect results for Kafka 0.10.1.0 client

2017-01-09 Thread Vahid Hashemian (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-4547?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15812544#comment-15812544 ] Vahid Hashemian commented on KAFKA-4547: [~hachikuji] Thanks for the feedback. What you're

[jira] [Updated] (KAFKA-4611) Support custom authentication mechanism

2017-01-09 Thread mahendiran chandrasekar (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-4611?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] mahendiran chandrasekar updated KAFKA-4611: --- Description: Currently there are two login mechanisms supported by kafka

Re: [DISCUSS] KIP-108: Create Topic Policy

2017-01-09 Thread Ismael Juma
Hi Roger, That's a good question. The server defaults are passed via the `configure` method of the `Configurable` interface that is implemented by `CreateTopicPolicy`. I'll mention this explicitly in the KIP. Ismael On Mon, Jan 9, 2017 at 6:04 PM, Roger Hoover wrote: >

out of the box security?

2017-01-09 Thread radai
in light of things like this - https://www.bleepingcomputer.com/news/security/mongodb-apocalypse-is-here-as-ransom-attacks-hit-10-000-servers/ and given that plenty of people/orgs have public facing kafka installations that are wide open - https://www.shodan.io/search?query=kafka (yes, i realize

Re: [DISCUSS] KIP-104: Granular Sensors for Streams

2017-01-09 Thread Eno Thereska
One hopefully final update to the KIP to re-add two methods for generic sensor creation. Also the interface should have been marked unstable from the beginning, so did that now. Thanks, and please vote if you haven't done so. The above changes shouldn't trigger a re-vote since they are more

[jira] [Commented] (KAFKA-4547) Consumer.position returns incorrect results for Kafka 0.10.1.0 client

2017-01-09 Thread Jason Gustafson (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-4547?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15812429#comment-15812429 ] Jason Gustafson commented on KAFKA-4547: [~vahid] Thanks for looking into this. The suggested fix

Re: [VOTE] KIP-66: Single Message Transforms for Kafka Connect

2017-01-09 Thread Shikhar Bhushan
Thanks all. The vote passed with +5 (binding). On Fri, Jan 6, 2017 at 11:37 AM Shikhar Bhushan wrote: That makes sense to me, I'll fold that into the PR and update the KIP if it gets committed in that form. On Fri, Jan 6, 2017 at 9:44 AM Jason Gustafson

Re: [DISCUSS] KIP-108: Create Topic Policy

2017-01-09 Thread Roger Hoover
This is great. Thanks, Ismael. One question. When TopicDetails are passed to the policy implementation, would the server defaults already have been merged? If not, I think the policy also needs access to the server defaults. Cheers, Roger On Fri, Jan 6, 2017 at 9:26 AM, Ismael Juma

[jira] [Commented] (KAFKA-4441) Kafka Monitoring is incorrect during rapid topic creation and deletion

2017-01-09 Thread Edoardo Comar (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-4441?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15812411#comment-15812411 ] Edoardo Comar commented on KAFKA-4441: -- Please ignore the previous comment - the

[jira] [Commented] (KAFKA-4455) CommitFailedException during rebalance doesn't release resources in tasks/processors

2017-01-09 Thread Davor Poldrugo (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-4455?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15812326#comment-15812326 ] Davor Poldrugo commented on KAFKA-4455: --- This should be resolved by KAFKA-4561, so this issue can

[jira] [Comment Edited] (KAFKA-3745) Consider adding join key to ValueJoiner interface

2017-01-09 Thread Matthias J. Sax (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-3745?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15812244#comment-15812244 ] Matthias J. Sax edited comment on KAFKA-3745 at 1/9/17 5:01 PM: [~sree2k]

[jira] [Commented] (KAFKA-3745) Consider adding join key to ValueJoiner interface

2017-01-09 Thread Matthias J. Sax (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-3745?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15812244#comment-15812244 ] Matthias J. Sax commented on KAFKA-3745: [~sree2k] thanks for picking this up. Just wanted to

[VOTE] KIP-109: Old Consumer Deprecation

2017-01-09 Thread Vahid S Hashemian
Happy Monday, I'd like to thank everyone who participated in the discussion around this KIP and shared their opinion. The only concern that was raised was not having a defined migration plan yet for existing users of the old consumer. I hope that responses to this concern (on the discussion

[jira] [Updated] (KAFKA-3745) Consider adding join key to ValueJoiner interface

2017-01-09 Thread Matthias J. Sax (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-3745?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthias J. Sax updated KAFKA-3745: --- Labels: api needs-kip newbie (was: api newbie) > Consider adding join key to ValueJoiner

[jira] [Created] (KAFKA-4610) getting error:Batch containing 3 record(s) expired due to timeout while requesting metadata from brokers for test2R2P2-1

2017-01-09 Thread sandeep kumar singh (JIRA)
sandeep kumar singh created KAFKA-4610: -- Summary: getting error:Batch containing 3 record(s) expired due to timeout while requesting metadata from brokers for test2R2P2-1 Key: KAFKA-4610 URL:

[jira] [Created] (KAFKA-4609) KTable/KTable join followed by groupBy and aggregate/count can result in incorrect results

2017-01-09 Thread Damian Guy (JIRA)
Damian Guy created KAFKA-4609: - Summary: KTable/KTable join followed by groupBy and aggregate/count can result in incorrect results Key: KAFKA-4609 URL: https://issues.apache.org/jira/browse/KAFKA-4609

[jira] [Updated] (KAFKA-4556) unordered messages when multiple topics are combined in single topic through stream

2017-01-09 Thread Savdeep Singh (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-4556?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Savdeep Singh updated KAFKA-4556: - Attachment: stream topology.png Stream topology example > unordered messages when multiple

[jira] [Commented] (KAFKA-4556) unordered messages when multiple topics are combined in single topic through stream

2017-01-09 Thread Savdeep Singh (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-4556?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15811854#comment-15811854 ] Savdeep Singh commented on KAFKA-4556: -- [~ewencp] Thanks for the response, please find attached

[GitHub] kafka pull request #2335: TRIVIAL: Fix spelling of log message

2017-01-09 Thread reftel
GitHub user reftel opened a pull request: https://github.com/apache/kafka/pull/2335 TRIVIAL: Fix spelling of log message You can merge this pull request into a Git repository by running: $ git pull https://github.com/reftel/kafka feature/triggering Alternatively you can

[jira] [Commented] (KAFKA-4353) Add semantic types to Kafka Connect

2017-01-09 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-4353?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15811608#comment-15811608 ] ASF GitHub Bot commented on KAFKA-4353: --- GitHub user rnpridgeon opened a pull request:

[GitHub] kafka pull request #2334: subset of KAFKA-4353: Add uuid

2017-01-09 Thread rnpridgeon
GitHub user rnpridgeon opened a pull request: https://github.com/apache/kafka/pull/2334 subset of KAFKA-4353: Add uuid Alternatively I am open to using Avro's 'Fixed' type with a 16 byte size. However most requests I have seen received wish to see UUID represented as type String

[jira] [Assigned] (KAFKA-4588) QueryableStateIntegrationTest.shouldNotMakeStoreAvailableUntilAllStoresAvailable is occasionally failing on jenkins

2017-01-09 Thread Damian Guy (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-4588?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Damian Guy reassigned KAFKA-4588: - Assignee: Damian Guy >

Re: [VOTE] KIP-105: Addition of Record Level for Sensors

2017-01-09 Thread Michael Noll
+1 (non-binding) On Fri, Jan 6, 2017 at 6:12 PM, Matthias J. Sax wrote: > +1 > > On 1/6/17 9:09 AM, Neha Narkhede wrote: > > +1 > > > > On Fri, Jan 6, 2017 at 9:04 AM Sriram Subramanian > wrote: > > > >> +1 > >> > >> On Fri, Jan 6, 2017 at 8:40 AM,

  1   2   >