Re: [DISCUSS] KIP-458: Connector Client Config Override Policy

Randall, Thanks a lot for the suggestions. I have incorporated the comments in the KIP. Thanks, Magesh On Mon, May 6, 2019 at 6:52 PM Randall Hauch wrote: > Thanks, Magesh. I do have a few pretty minor suggestions. > > 1) Define a bit more clearly in the "Proposed Changes" whether the configs

[jira] [Created] (KAFKA-8327) Connection to Node2 was disconnected

suseendramani created KAFKA-8327: Summary: Connection to Node2 was disconnected Key: KAFKA-8327 URL: https://issues.apache.org/jira/browse/KAFKA-8327 Project: Kafka Issue Type: New Feature

Re: [DISCUSS] KIP-466: Add support for List serialization and deserialization

Hi John, I’m really sorry for the confusion. I cloned that JIRA ticket from an old one about introducing UUID Serde, and I guess was too hasty while editing the copy to notice the mistake. Just edited the ticket. Sorry for any inconvenience . As per comparator, I agree. Let’s make user be

Re: [DISCUSS] KIP-458: Connector Client Config Override Policy

Thanks, Magesh. I do have a few pretty minor suggestions. 1) Define a bit more clearly in the "Proposed Changes" whether the configs passed to the validate method via the ConnectorClientConfigRequest object have or do not have the prefixes. 2) Specify more clearly in (or around) the table which

Re: [VOTE] KIP-453: Add close() method to RocksDBConfigSetter

+1 (non-binding). Thanks for the KIP. On Tue, May 7, 2019 at 3:50 AM Bill Bejeck wrote: > Thanks for the KIP Sophie. > > +1(binding) > > On Mon, May 6, 2019 at 4:51 PM John Roesler wrote: > > > Thanks, Sophie, I reviewed the KIP, and I agree this is the best / > > only-practical approach. > >

Jenkins build is back to normal : kafka-trunk-jdk11 #490

See

Re: [DISCUSS] KIP-458: Connector Client Config Override Policy

Hi Randall, I have incorporated your comments and updated the KIP with the following 1. update the config key to be `connector.client.config.override.policy` 2. update the interface name to be `ConnectorClientConfigOverridePolicy`. Updated the implementation names also to match the

Re: [DISCUSS] KIP-458: Connector Client Config Override Policy

Konstantine, Thanks a lot for your feedback on the KIP. I have incorporated the feedback using generics for Class. I have also updated the KIP to handle the default value per Randall's suggestion. Let me know if you have any questions. Thanks, Magesh On Mon, May 6, 2019 at 1:58 PM Konstantine

Re: [DISCUSS] KIP-458: Connector Client Config Override Policy

That's fine. I don't think there's much difference between the two option anyway. I'm looking forward to the updated KIP. On Mon, May 6, 2019 at 7:38 PM Magesh Nandakumar wrote: > Randall, > > Thanks a lot for your feedback. > > If I understand it correctly, we could do one of the following,

Re: [DISCUSS] KIP-458: Connector Client Config Override Policy

Randall, Thanks a lot for your feedback. If I understand it correctly, we could do one of the following, right? 1. introduce a new config `allow.client.config.overrides` with a default value of false. The default value for the policy would be `None`. So by default, we will still preserve the

Re: [DISCUSS] KIP-458: Connector Client Config Override Policy

I actually like a separate config for whether to pass or filter client override properties to the connector. I generally dislike adding more properties, but in this case it keeps the two orthogonal behaviors independent and reduces the need to implement policies that cover all permutations.

Re: [VOTE] KIP-460: Admin Leader Election RPC

All committer votes are binding, so you have binding +1 from Colin, Jason and myself - which is just the 3 you need for the KIP to be accepted. Mickael added non-binding community support, which is great signal as well. On Mon, May 6, 2019 at 4:39 PM Jose Armando Garcia Sancio <

Re: [VOTE] KIP-460: Admin Leader Election RPC

I am closing the voting. KIP is accepted with: +1 (binding): Colin McCabe +1 (non-binding): Jason Gustafson, Gwen Shapira, Mickael Maison Thanks! On Fri, May 3, 2019 at 9:59 AM Mickael Maison wrote: > +1 (non binding) > Thanks for the KIP > > On Thu, May 2, 2019 at 11:02 PM Colin McCabe

Re: [VOTE] KIP-258: Allow to Store Record Timestamps in RocksDB

Another small change to the KIP: We want to add two more public static helper methods `make()` and `getValueOrNull` to simplify `null` handling. Also, the constructor of `ValueAndTimestamp` should be private. > public class ValueAndTimestamp { > private ValueAndTimestamp(final V value,

Re: [VOTE] KIP-453: Add close() method to RocksDBConfigSetter

Thanks for the KIP Sophie. +1(binding) On Mon, May 6, 2019 at 4:51 PM John Roesler wrote: > Thanks, Sophie, I reviewed the KIP, and I agree this is the best / > only-practical approach. > > +1 (non-binding) > > On Mon, May 6, 2019 at 2:23 PM Matthias J. Sax > wrote: > > > +1 (binding) > > > >

[jira] [Resolved] (KAFKA-8255) Replica fetcher thread exits with OffsetOutOfRangeException

[ https://issues.apache.org/jira/browse/KAFKA-8255?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jason Gustafson resolved KAFKA-8255. Resolution: Resolved I'm going to resolve this. We believe the root cause was KAFKA-8306,

Error connecting to kafka machine

Hello Team, I have installed Kafka on a centos machine and trying to connect to that machine to send data to my broker and getting an error message as follows "Message: pyembed: ConnectionError(: Failed to establish a new connection: [Errno 111] Connection refused) caused by:

Re: Cleaning up command line tools argument parsing a little

On Mon, May 6, 2019, at 10:21, Sönke Liebau wrote: > Hi Colin, > > it was my intention to keep the structure of the commands mostly intact > while doing the refactoring - if that is possible, have not really checked > yet to be honest. > > But what I wanted to try and do is recreate the current

Not Able to connect to kafka

Hello Team, I have installed kafka on a centos machine and trying to connect to that machine to send data to my broker and getting a error message as

Re: [DISCUSS] KIP-466: Add support for List serialization and deserialization

Hi Daniyar, Thanks for the proposal! If I understand the point about the comparator, is it just to capture the generic type parameter? If so, then anything that implements a known interface would work just as well, right? I've been considering adding something like the Jackson TypeReference (or

Re: [VOTE] KIP-454: Expansion of the ConnectClusterState interface

Nice and useful KIP. Thanks Chris! +1 (non-binding) Konstantine On Fri, May 3, 2019 at 5:14 PM Randall Hauch wrote: > Nice job, Chris! > > +1 (binding) > > On Thu, May 2, 2019 at 8:16 PM Magesh Nandakumar > wrote: > > > Thanks a lot for the work on this KIP Chris. > > > > +1(non-binding) >

Re: [VOTE] KIP-437: Custom replacement for MaskField SMT

I think is is a useful improvement proposal. Thanks Valeria! I'm +1 (non-binding) on this KIP Konstantine On Mon, Apr 15, 2019 at 2:04 AM Valeria Vasylieva < valeria.vasyli...@gmail.com> wrote: > Hi all, > > Since there are no more objections/proposals I would like to start the > vote on

Kafka 2.3 KIP freeze on May 10th (Friday)

Hi all, As per the release plan for 2.3, we're going to have a KIP freeze this week. Let's have it on Friday (May 10th). For more details about the 2.3 release, check out https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=112820648 cheers, Colin

[jira] [Resolved] (KAFKA-8056) Replace FindCoordinator request/response with automated protocol

[ https://issues.apache.org/jira/browse/KAFKA-8056?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jason Gustafson resolved KAFKA-8056. Resolution: Fixed > Replace FindCoordinator request/response with automated protocol >

Re: [DISCUSS] KIP-458: Connector Client Config Override Policy

Thanks for the KIP Magesh, it's quite useful towards the goals for more general multi-tenancy in Connect. Couple of comments from me too: I think the fact that the default policy is 'null' (no implementation) should be mentioned on the table next to the available implementations. Currently the

Re: [VOT] KIP-449: Add connector contexts to Connect worker logs (vote thread)

Great improvement for multi-tenancy. Thanks Randall! +1 (non-binding) Konstantine On Tue, Apr 30, 2019 at 9:18 PM Chris Egerton wrote: > +1 (non-binding) > > Really looking forward to this. Thanks, Randall! > > On Tue, Apr 30, 2019, 20:47 Magesh Nandakumar > wrote: > > > This will make

Re: [VOTE] KIP-453: Add close() method to RocksDBConfigSetter

Thanks, Sophie, I reviewed the KIP, and I agree this is the best / only-practical approach. +1 (non-binding) On Mon, May 6, 2019 at 2:23 PM Matthias J. Sax wrote: > +1 (binding) > > > > On 5/6/19 6:28 PM, Sophie Blee-Goldman wrote: > > Hi all, > > > > I'd like to call for a vote on a minor KIP

[DISCUSS] KIP-461 - Improving replica fetcher behavior in case of partition failures

Hey All! I have created a KIP to improve the behavior of replica fetcher when partition failure occurs. Please do have a look at it and let me know what you think. KIP 461 - https://cwiki.apache.org/confluence/display/KAFKA/KIP-461+-+Improve+Replica+Fetcher+behavior+at+handling+partition+failure

Re: [VOTE] KIP-453: Add close() method to RocksDBConfigSetter

+1 (binding) On 5/6/19 6:28 PM, Sophie Blee-Goldman wrote: > Hi all, > > I'd like to call for a vote on a minor KIP that adds a close() method to > the RocksDBConfigSetter interface of Streams. > > Link: >

Re: [VOTE] KIP-465: Add Consolidated Connector Endpoint to Connect REST API

Useful and concise KIP +1 (non-binding) Konstantine On Mon, May 6, 2019 at 10:43 AM Randall Hauch wrote: > Thanks, Dan. As mentioned on the discussion, this is really a nice little > addition that was alway missing from the API. > > +1 (binding) > > Randall > > On Mon, May 6, 2019 at 9:23 AM

Re: [DISCUSS] KIP-465: Add Consolidated Connector Endpoint to Connect REST API

This is a nice improvement, both from an operational standpoint as well as for testing purposes, as we scale the number of connectors that a connect cluster can run. LGTM, thanks for the KIP Dan! On Fri, May 3, 2019 at 2:50 PM Alex Liu wrote: > Good question, > > `info` is probably the best

[VOTE] KIP-453: Add close() method to RocksDBConfigSetter

Hi all, I'd like to call for a vote on a minor KIP that adds a close() method to the RocksDBConfigSetter interface of Streams. Link: https://cwiki.apache.org/confluence/display/KAFKA/KIP-453%3A+Add+close%28%29+method+to+RocksDBConfigSetter This is important for users who have created

Re: [DISCUSS] KIP-458: Connector Client Config Override Policy

Randall, I was wondering if you had any thoughts on the above alternatives to deal with a default policy. If it's possible, I would like to finalize the discussions and start a vote. Let me know your thoughts. Thanks, Magesh On Tue, Apr 30, 2019 at 8:46 PM Magesh Nandakumar wrote: > Randall,

Re: [DISCUSS] KIP-440: Extend Connect Converter to support headers

Have we started a vote on this? I don't see the separate "[VOTE]" thread. On Mon, Apr 29, 2019 at 6:19 PM Konstantine Karantasis < konstant...@confluent.io> wrote: > Thanks Yaroslav, this KIP LGTM now too! > > To give some context regarding my previous comment: headers in Connect > would

Re: [VOTE] KIP-465: Add Consolidated Connector Endpoint to Connect REST API

Thanks, Dan. As mentioned on the discussion, this is really a nice little addition that was alway missing from the API. +1 (binding) Randall On Mon, May 6, 2019 at 9:23 AM dan wrote: > I would like to start voting for > >

Re: [VOTE] KIP-422: Use the default value of max.poll.interval in Streams

You're absolutely right. I realized that at some point after the initial message and just tried to play it cool. Sorry about that. -John On Mon, Apr 15, 2019 at 11:59 AM Jun Rao wrote: > The voting thread seems to be referring to the wrong KIP number. It should > be KIP-442 instead of KIP-422.

Re: [VOTE] KIP-411: Make default Kafka Connect worker task client IDs distinct

Thanks Arjun. I've updated the KIP using your suggestion - just a few slight changes. On Fri, May 3, 2019 at 4:48 PM Arjun Satish wrote: > Maybe we can say something like: > > This change can have an indirect impact on resource usage by a Connector. > For example, systems that were enforcing

Re: Cleaning up command line tools argument parsing a little

Hi Colin, it was my intention to keep the structure of the commands mostly intact while doing the refactoring - if that is possible, have not really checked yet to be honest. But what I wanted to try and do is recreate the current parsing with argparse as much as possible. And in the process

Re: Cleaning up command line tools argument parsing a little

Hi Sönke, #2 is a bit tough because people have come to rely on the way the commands are structured right now. If we want to make big changes, it might be easier just to create a separate tool and deprecate the old one(s). One thing we've talked about doing in the past is creating a single

[DISCUSS] KIP-466: Add support for List serialization and deserialization

Hello, Starting a discussion for KIP-466 adding support for List Serde. PR is created under https://github.com/apache/kafka/pull/6592 There are two topics I would like to discuss: 1. Since type for List serve needs to be declared before hand, I could

Re: [DISCUSS] KIP-455: Create an Administrative API for Replica Reassignment

On Mon, May 6, 2019, at 07:39, Ismael Juma wrote: > Hi Colin, > > A quick comment. > > On Sat, May 4, 2019 at 11:18 PM Colin McCabe wrote: > > > The big advantage of doing batching on the controller is that the > > controller has more information about what is going on in the cluster. So > >

Re: [DISCUSS] KIP-456: Helper classes to make it simpler to write test logic with TopologyTestDriver

Thanks for the KIP, Jukka! Sorry it took so long for me to review it. Just a few questions, in no particular order: InputTopic: 1. Have you considered adding a ProducerRecord input method to the input topic? This might be unnecessary, given the overloads you provide. I'm wondering if it

Re: [DISCUSS] KIP-460: Admin Leader Election RPC

On Sun, May 5, 2019 at 12:35 PM Stanislav Kozlovski wrote: > Hey there Jose, thanks for the KIP! > > I have one small nit regarding the `kafka-leader-election.sh` tool. I agree > with Jason that it is probably better to force users be explicit in their > desired election type. I was wondering

Re: [DISCUSS] KIP-439: Deprecate Interface WindowStoreIterator

Thanks all (or range? ;) ) for the discussion. Good points all around. Although I see the allure of naming the metrics the same as the things they're measuring, it seems not to be perfect. Seconding Matthias's latter thought, I think it's likely you'd want to measure the method calls

[jira] [Resolved] (KAFKA-8319) Flaky Test KafkaStreamsTest.statefulTopologyShouldCreateStateDirectory

[ https://issues.apache.org/jira/browse/KAFKA-8319?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bill Bejeck resolved KAFKA-8319. Resolution: Cannot Reproduce > Flaky Test

[jira] [Created] (KAFKA-8326) Add List Serde

Daniyar Yeralin created KAFKA-8326: -- Summary: Add List Serde Key: KAFKA-8326 URL: https://issues.apache.org/jira/browse/KAFKA-8326 Project: Kafka Issue Type: Improvement

Re: [DISCUSSION] KIP-418: A method-chaining way to branch KStream

Ivan, That's a very good point about the "start" operator in the dynamic case. I had no problem with "split()"; I was just questioning the necessity. Since you've provided a proof of necessity, I'm in favor of the "split()" start operator. Thanks! Separately, I'm interested to see where the

Re: [DISCUSS] KIP-315: Stream Join Sticky Assignor

Hi Matthias, Once KIP-429 is released, all non-sticky assignors will be as useful. Any user that wants to take advantage of KIP-429 needs to use a sticky assignor. Currently there is only 1 sticky assignor in the kafka project, which is similar to RoundRobinAssignor, but a sticky verion. I

Re: [DISCUSS] KIP-455: Create an Administrative API for Replica Reassignment

Hi Colin, A quick comment. On Sat, May 4, 2019 at 11:18 PM Colin McCabe wrote: > The big advantage of doing batching on the controller is that the > controller has more information about what is going on in the cluster. So > it can schedule reassignments in a more optimal way. For instance,

Build failed in Jenkins: kafka-trunk-jdk11 #489

See Changes: [github] MINOR: Remove workarounds for lz4-java bug affecting byte buffers -- Started by an SCM change [EnvInject] - Loading node environment variables.

[VOTE] KIP-465: Add Consolidated Connector Endpoint to Connect REST API

I would like to start voting for https://cwiki.apache.org/confluence/display/KAFKA/KIP-465%3A+Add+Consolidated+Connector+Endpoint+to+Connect+REST+API thanks dan

[jira] [Created] (KAFKA-8325) Remove from the incomplete set failed. This should be impossible

Mattia Barbon created KAFKA-8325: Summary: Remove from the incomplete set failed. This should be impossible Key: KAFKA-8325 URL: https://issues.apache.org/jira/browse/KAFKA-8325 Project: Kafka

Build failed in Jenkins: kafka-trunk-jdk11 #488

See Changes: [manikumar.reddy] KAFKA-7455: Support JmxTool to connect to a secured RMI port. (#5968) -- Started by an SCM change [EnvInject] - Loading node environment

[jira] [Resolved] (KAFKA-7455) JmxTool cannot connect to an SSL-enabled JMX RMI port

[ https://issues.apache.org/jira/browse/KAFKA-7455?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Manikumar resolved KAFKA-7455. -- Resolution: Fixed Fix Version/s: 2.3.0 > JmxTool cannot connect to an SSL-enabled JMX RMI port

Re: [DISCUSS] KIP-455: Create an Administrative API for Replica Reassignment

Hi Colin, Thanks for explaining all this, it makes sense. Viktor On Sun, May 5, 2019 at 8:18 AM Colin McCabe wrote: > On Thu, May 2, 2019, at 09:35, Viktor Somogyi-Vass wrote: > > Hey Colin & George, > > > > Thinking on George's points I was wondering if it's feasible to submit a > > big

Re: [DISCUSS] KIP-456: Helper classes to make it simpler to write test logic with TopologyTestDriver

Hi, Now everything what I planned to add including tests, samples and document changes are in my branch: https://github.com/jukkakarvanen/kafka/compare/trunk...jukkakarvanen:KAFKA-8233_InputOutputTopics So I can create PR as soon as this KIP is getting green light to proceed. Jukka la 4.

Build failed in Jenkins: kafka-trunk-jdk11 #487

See Changes: [manikumar.reddy] MINOR: Fix ThrottledReplicaListValidator doc error. (#6537) [manikumar.reddy] MINOR: Document improvement (#6682) -- Started by an SCM

Re: [VOTE] KIP-334 Include partitions in exceptions raised during consumer record deserialization/validation

Hey there Kamal, I'm sincerely sorry for missing your earlier message. As I open this thread up, I see I have an unsent draft message about resuming discussion from some time ago. In retrospect, I think I may have been too pedantic with the exception naming and hierarchy. I now believe a single