[GitHub] kafka pull request #4342: KAFKA-4263: fix flaky test QueryableStateIntegrati...

GitHub user mjsax opened a pull request: https://github.com/apache/kafka/pull/4342 KAFKA-4263: fix flaky test QueryableStateIntegrationTest.concurrentAccess You can merge this pull request into a Git repository by running: $ git pull https://github.com/mjsax/kafka

Re: [VOTE] KIP-218: Make KafkaFuture.Function java 8 lambda compatible

Xavier, I agree that we can close the vote with 4 binding and 3 non-binding votes. I will update, confluence, jira and the PR accordingly. Thank you all Op ma 18 dec. 2017 om 20:23 schreef Xavier Léauté : > Steven, I think you can mark the vote as closed now. > > On Tue,

Re: [DISCUSS] KIP-228 Negative record timestamp support

I think the trivial change of just recognizing using -1 was a mistake for a sentinel value and special casing it while allowing other negative values through is the most practical, reasonable change. Realistically, the scope of impact for that -1 is pretty tiny, as has been pointed out. A single

[GitHub] kafka pull request #4334: [KAFKA-6265] GlobalKTable missing #queryableStoreN...

Github user ConcurrencyPractitioner closed the pull request at: https://github.com/apache/kafka/pull/4334 ---

Build failed in Jenkins: kafka-trunk-jdk8 #2284

See Changes: [wangguoz] MINOR: Fix race condition in KafkaStreamsTest.shouldReturnThreadMetadata -- [...truncated 408.93 KB...] kafka.utils.CoreUtilsTest > testReadInt

Re: [VOTE] KIP-238: Expose Kafka cluster ID in Connect REST API

Seems we've passed 72h and have the necessary 3 +1 binding votes, 2 non-binding, and 0 otherwise, so this KIP passes. I'll follow up w/ another committer about landing the patch. Thanks everyone! -Ewen On Mon, Dec 18, 2017 at 11:27 AM, Jason Gustafson wrote: > +1

[jira] [Created] (KAFKA-6384) TransactionsTest#testFencingOnSendOffsets sometimes fails with ProducerFencedException

Ted Yu created KAFKA-6384: - Summary: TransactionsTest#testFencingOnSendOffsets sometimes fails with ProducerFencedException Key: KAFKA-6384 URL: https://issues.apache.org/jira/browse/KAFKA-6384 Project:

Jenkins build is back to normal : kafka-1.0-jdk7 #109

See

Build failed in Jenkins: kafka-trunk-jdk8 #2283

See Changes: [wangguoz] MINOR: Stabilize flaky smoke system tests before KIP-91 -- [...truncated 411.31 KB...] kafka.utils.CoreUtilsTest > testReadInt STARTED

Re: [DISCUSS] KIP-232: Detect outdated metadata by adding ControllerMetadataEpoch field

Hey Jason, Thanks much for reviewing the KIP. I think it is a good idea to let coordinator do the additional sanity check to ensure the leader epoch from OffsetCommitRequest never decreases. This can help us detect bug. The next question will be what should we do if OffsetCommitRequest provides

[GitHub] kafka pull request #4337: MINOR: fix race condition in KafkaStreamsTest.shou...

Github user asfgit closed the pull request at: https://github.com/apache/kafka/pull/4337 ---

[GitHub] kafka pull request #4329: MINOR: stabilize flaky system tests

Github user asfgit closed the pull request at: https://github.com/apache/kafka/pull/4329 ---

Re: [DISCUSS] KIP 226 - Dynamic Broker Configuration

Hey Rajini, Thanks, makes sense. A couple replies: 2. I haven't changed the way Configurable is used. It is still used for > initial configuration (if the class implements it). Reconfigurable is used > at the moment only for reconfiguration. The reason I did it that way is > because for some of

Re: [DISCUSS]: KIP-159: Introducing Rich functions to Streams

I just want to point out that the basic idea is great and that we should apply optimizations like "filter first" and other. But we should NOT convolute this KIP with orthogonal improvements. In fact, we have an umbrella JIRA for DSL optimization already:

Re: [DISCUSS] KIP 226 - Dynamic Broker Configuration

Hi Jason, Thank you for reviewing the KIP. 1. ConfigDef is used for validating the type of the value and the constraints. But I am doing a lot more validation of security configs. For example, for keystore configuration update, validate() loads the keystore and if it is an inter-broker listener,

[jira] [Created] (KAFKA-6383) StreamThread.shutdown doesn't clean up completely when called beforeStreamThread.start

Rohan Desai created KAFKA-6383: -- Summary: StreamThread.shutdown doesn't clean up completely when called beforeStreamThread.start Key: KAFKA-6383 URL: https://issues.apache.org/jira/browse/KAFKA-6383

[jira] [Reopened] (KAFKA-4263) QueryableStateIntegrationTest.concurrentAccess is failing occasionally in jenkins builds

[ https://issues.apache.org/jira/browse/KAFKA-4263?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthias J. Sax reopened KAFKA-4263: Assignee: Matthias J. Sax (was: Damian Guy) Reopening this. Happened again:

Re: [VOTE] KIP-243: Make ProducerConfig and ConsumerConfig constructors public

+1 Thanks for the KIP. --Vahid From: Ted Yu To: dev@kafka.apache.org Date: 12/18/2017 02:45 PM Subject:Re: [VOTE] KIP-243: Make ProducerConfig and ConsumerConfig constructors public +1 nit: via "copy and past" an 'e' is missing at the end. On Mon,

Re: [DISCUSS] KIP-232: Detect outdated metadata by adding ControllerMetadataEpoch field

Hi Dong, Thanks for the KIP. Good job identifying the problem. One minor question I had is whether the coordinator should enforce that the leader epoch associated with an offset commit can only go forward for each partition? Currently it looks like we just depend on the client for this, but since

Re: [VOTE] KIP-243: Make ProducerConfig and ConsumerConfig constructors public

+1 nit: via "copy and past" an 'e' is missing at the end. On Mon, Dec 18, 2017 at 2:38 PM, Matthias J. Sax wrote: > Hi, > > I want to propose the following KIP: > https://cwiki.apache.org/confluence/display/KAFKA/KIP- >

[GitHub] kafka pull request #4341: KAFKA-6382: Make ProducerConfig and ConsumerConfig...

GitHub user mjsax opened a pull request: https://github.com/apache/kafka/pull/4341 KAFKA-6382: Make ProducerConfig and ConsumerConfig constructors public You can merge this pull request into a Git repository by running: $ git pull https://github.com/mjsax/kafka

[VOTE] KIP-243: Make ProducerConfig and ConsumerConfig constructors public

Hi, I want to propose the following KIP: https://cwiki.apache.org/confluence/display/KAFKA/KIP-243%3A+Make+ProducerConfig+and+ConsumerConfig+constructors+public This is a rather straight forward change, thus I skip the DISCUSS thread and call for a vote immediately. -Matthias signature.asc

[jira] [Created] (KAFKA-6382) Make ProducerConfig and ConsumerConfig constructors public

Matthias J. Sax created KAFKA-6382: -- Summary: Make ProducerConfig and ConsumerConfig constructors public Key: KAFKA-6382 URL: https://issues.apache.org/jira/browse/KAFKA-6382 Project: Kafka

[jira] [Created] (KAFKA-6381) Improve KafkaStreamsTest for localThreadMetadata

Matthias J. Sax created KAFKA-6381: -- Summary: Improve KafkaStreamsTest for localThreadMetadata Key: KAFKA-6381 URL: https://issues.apache.org/jira/browse/KAFKA-6381 Project: Kafka Issue

[GitHub] kafka pull request #4340: [KAFKA-6265] GlobalKTable missing #queryableStoreN...

GitHub user ConcurrencyPractitioner opened a pull request: https://github.com/apache/kafka/pull/4340 [KAFKA-6265] GlobalKTable missing #queryableStoreName() You can merge this pull request into a Git repository by running: $ git pull

[GitHub] kafka pull request #4339: [KAFKA-6328] Exclude node groups belonging to glob...

GitHub user ConcurrencyPractitioner opened a pull request: https://github.com/apache/kafka/pull/4339 [KAFKA-6328] Exclude node groups belonging to global stores in InternalTopologyBuilder#makeNodeGroups You can merge this pull request into a Git repository by running: $ git

[VOTE] KIP-237: More Controller Health Metrics

Hi all, Since there are no more outstanding comments, I would like to start voting thread for KIP-237: https://cwiki.apache.org/confluence/display/KAFKA/KIP-237%3A+More+Controller+Health+Metrics The KIP proposes to add a few more metrics to help monitor Kafka Controller health. Thanks, Dong

Re: [DISCUSS] KIP-232: Detect outdated metadata by adding ControllerMetadataEpoch field

Hey Jun, Thanks much for your comments. These are very thoughtful ideas. Please see my comments below. On Thu, Dec 14, 2017 at 6:38 PM, Jun Rao wrote: > Hi, Dong, > > Thanks for the update. A few more comments below. > > 10. It seems that we need to return the leader epoch

Re: [DISCUSS] KIP 226 - Dynamic Broker Configuration

Hi Rajini, Looking good. Just a few questions. 1. (Related to Jay's comment) Is the validate() method on Reconfigurable necessary? I would have thought we'd validate using the ConfigDef. Do you have a use case in mind in which the reconfigurable component only permits certain reconfigurations?

Build failed in Jenkins: kafka-1.0-jdk7 #108

See Changes: [jason] KAFKA-6258; SSLTransportLayer should keep reading from socket until -- Started by an SCM change Started by an SCM change [EnvInject] - Loading node

Build failed in Jenkins: kafka-1.0-jdk7 #107

See Changes: [wangguoz] MINOR: improve JavaDocs for KafkaStreams, KafkaProducer, KafkaConsumer -- [...truncated 1.83 MB...]

Jenkins build is back to normal : kafka-trunk-jdk7 #3045

See

[GitHub] kafka pull request #4338: [WIP] MINOR: web docs for KIP-220

GitHub user guozhangwang opened a pull request: https://github.com/apache/kafka/pull/4338 [WIP] MINOR: web docs for KIP-220 1. added functions for KafkaStreams and KafkaClientSupplier. 2. added prefix for admin client in StreamsConfig. ### Committer Checklist (excluded

Re: Request to become contributor for Kafka project

Hello Chen, I have added you to the contributor list of Kafka now. Cheers, Guozhang On Mon, Dec 18, 2017 at 10:24 AM, Chen He wrote: > Dear Kafka project committees > > It is my pleasure to contribute to Kafka community. I am interested in join > Kafka community as a

Re: Request to become contributor for Kafka project

Hi, Chen, Thanks for your interest. Just added you to the contributor list. Jun On Mon, Dec 18, 2017 at 10:24 AM, Chen He wrote: > Dear Kafka project committees > > It is my pleasure to contribute to Kafka community. I am interested in join > Kafka community as a

Request to become contributor for Kafka project

Dear Kafka project committees It is my pleasure to contribute to Kafka community. I am interested in join Kafka community as a contributor. I reported following issues for Kafka community: KAFKA-5518 KAFKA-5705

Re: [VOTE] KIP-238: Expose Kafka cluster ID in Connect REST API

+1 (binding) On Sun, Dec 17, 2017 at 8:44 PM, Konstantine Karantasis < konstant...@confluent.io> wrote: > +1 > > Nice KIP. Thanks! > > - Konstantine > > On Sat, Dec 16, 2017 at 6:19 AM, Gwen Shapira wrote: > > > +1 (binding). Thanks! > > On Fri, Dec 15, 2017 at 10:55 AM Ted

[jira] [Resolved] (KAFKA-6258) SSLTransportLayer should keep reading from socket until either the buffer is full or the socket has no more data

[ https://issues.apache.org/jira/browse/KAFKA-6258?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jason Gustafson resolved KAFKA-6258. Resolution: Fixed Fix Version/s: 1.0.1 1.1.0 Issue resolved by

Re: [VOTE] KIP-218: Make KafkaFuture.Function java 8 lambda compatible

Steven, I think you can mark the vote as closed now. On Tue, Dec 12, 2017 at 5:24 PM Guozhang Wang wrote: > +1 > > On Mon, Dec 11, 2017 at 7:07 PM, Colin McCabe wrote: > > > +1 (non-binding) > > > > P.S. Suggest to use whenComplete instead of making

[GitHub] kafka pull request #4248: KAFKA-6258; SSLTransportLayer should keep reading ...

Github user asfgit closed the pull request at: https://github.com/apache/kafka/pull/4248 ---

[DISCUSS] KIP-242: Mask password fields in Kafka Connect REST response

Hi all, I've created KIP-242, a proposal to secure credentials in kafka connect rest endpoint. https://cwiki.apache.org/confluence/display/KAFKA/KIP-242%3A+Mask+password+in+Kafka+Connect+Rest+API+response Here are something I'd like to discuss: - The "masked" value is set to "*" (9

[GitHub] kafka pull request #4335: MINOR: improve JavaDocs for KafkaStreams, KafkaPro...

Github user asfgit closed the pull request at: https://github.com/apache/kafka/pull/4335 ---

Build failed in Jenkins: kafka-trunk-jdk7 #3044

See -- Started by an SCM change [EnvInject] - Loading node environment variables. Building remotely on ubuntu-eu3 (ubuntu xenial) in workspace

Build failed in Jenkins: kafka-trunk-jdk7 #3043

See Changes: [wangguoz] MINOR: improve JavaDocs for KafkaStreams, KafkaProducer, KafkaConsumer -- Started by an SCM change [EnvInject] - Loading node environment

[GitHub] kafka pull request #4337: MINOR: fix race condition in KafkaStreamsTest.shou...

GitHub user mjsax opened a pull request: https://github.com/apache/kafka/pull/4337 MINOR: fix race condition in KafkaStreamsTest.shouldReturnThreadMetadata You can merge this pull request into a Git repository by running: $ git pull https://github.com/mjsax/kafka

[jira] [Resolved] (KAFKA-6373) Log end offset of input table changing during restore

[ https://issues.apache.org/jira/browse/KAFKA-6373?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bill Bejeck resolved KAFKA-6373. Resolution: Duplicate Duplicate issue of KAFKA-6269 > Log end offset of input table changing

Re: [DISCUSS] KIP 226 - Dynamic Broker Configuration

Hi Jay, Thank you for reviewing the KIP. 1) Yes, makes sense. I will update the PR. There are some config updates that may be allowed depending on the context (e.g. some security configs can be updated for new listeners, but not existing listeners). Perhaps it is ok to mark them dynamic in the

Re: [DISCUSS] KIP-236 Interruptible Partition Reassignment

Hi, Tom, Thanks for the reply. A few more followup comments below. 10. Another concern of mine is on consistency with the current pattern. The current pattern for change notification based on ZK is (1) we first write the actual value in the entity path and then write the change notification

Re: [DISCUSS] KIP 226 - Dynamic Broker Configuration

Two thoughts on implementation (shouldn't effect the KIP): 1. It might be nice to add a parameter to ConfigDef which says whether a configuration is dynamically updatable or not so that we can give error messages if it isn't and also have it reflected in the auto-generated docs. 2.

Re: [DISCUSS]: KIP-159: Introducing Rich functions to Streams

Jan, I apologize for the delayed response. my suggestion would be that instead of > > SOURCE -> KTABLESOURCE -> KTABLEFILTER -> JOIN -> SINK > > we build > > SOURCE -> KTABLEFILTER -> KTABLESOURCE -> JOIN -> SINK I agree that filtering before the KTable source makes sense and would be a

Re: [DISCUSS] KIP-213 Support non-key joining in KTable

Hi Guozhang, thanks for the update. On 15.12.2017 22:54, Guozhang Wang wrote: Jan, Thanks for the updated KIP, and the raised questions. Here are my thoughts around the "back and forth mapper" approach on your wiki: 1) regarding the key-value types of KTableValueGetter, we do not

Re: [DISCUSS] KIP-240: AdminClient.listReassignments AdminClient.describeReassignments

I've removed the option to pass a null reassignments argument to AdminClient.describeReassignments(), because to support this would require passing the topic and partition of each reassignment in the response, for what might be many partitions. It just seems like unnecessary bloat.

Re: [DISCUSS] KIP-240: AdminClient.listReassignments AdminClient.describeReassignments

Hi Ted, For class Reassignment, it seems you forgot to include set of brokers. I omitted a set of brokers intentionally, because the Reassignment is a an immutable reference to a (mutable) reassignment (in ZooKeeper). See KIP-236 to understand the background. I've fixed the other two issues,

Re: [DISCUSS] KIP-240: AdminClient.listReassignments AdminClient.describeReassignments

Hi Steven, I think it would be useful to be able to subscribe yourself on updates of > reassignment changes. I agree this would be really useful, but, to the extent I understand the networking underpinnings of the admin client, it might be difficult to do well in practice. Part of the problem

Re: [DISCUSS] KIP-236 Interruptible Partition Reassignment

Hi Jun, Thanks for replying, some answers below: > 10. The proposal now stores the reassignment for all partitions in > /admin/reassignment_requests/request_xxx. If the number of reassigned > partitions is larger, the ZK write may hit the default 1MB limit and fail. > An alternative approach is

[jira] [Created] (KAFKA-6380) transient failure in MetricsTest.testMetrics

Manikumar created KAFKA-6380: Summary: transient failure in MetricsTest.testMetrics Key: KAFKA-6380 URL: https://issues.apache.org/jira/browse/KAFKA-6380 Project: Kafka Issue Type: Bug

[jira] [Created] (KAFKA-6379) Work for KIP-240

Tom Bentley created KAFKA-6379: -- Summary: Work for KIP-240 Key: KAFKA-6379 URL: https://issues.apache.org/jira/browse/KAFKA-6379 Project: Kafka Issue Type: Bug Reporter: Tom Bentley