Re: [VOTE] - KIP-213 (new vote) - Simplified and revised.

Hello Adam, Thank YOU for all the patience going through this process! Could you update the KIP pages (including the parent page) with the newest status as well? Thanks. Guozhang On Mon, Mar 11, 2019 at 6:12 PM Adam Bellemare wrote: > Thanks to everyone, including John, Guozhang, Matthias

[DISCUSS] KIP-440: Extend Connect Converter to support headers

Hello, I'd like to propose a KIP that extends Kafka Connect Converter interface: https://cwiki.apache.org/confluence/display/KAFKA/KIP-440%3A+Extend+Connect+Converter+to+support+headers Thanks for considering! -- Yaroslav Tkachenko sap1ens.com

Re: [VOTE] - KIP-213 (new vote) - Simplified and revised.

Thanks to everyone, including John, Guozhang, Matthias and Jan for all the help! I do believe that makes 3 binding and 3 non-binding, vote passes. Thanks folks > On Mar 11, 2019, at 5:02 PM, Matthias J. Sax wrote: > > Thanks a lot for putting this together Adam and Jan! > > +1 (binding) >

Re: [VOTE] KIP-415: Incremental Cooperative Rebalancing in Kafka Connect

Thanks Jason! That makes perfect sense. The change is reflected in the KIP now. "compatible" will be the default mode for "connect.protocol" Cheers, Konstantine On Mon, Mar 11, 2019 at 4:31 PM Jason Gustafson wrote: > +1 Thanks for all the work on this. My only minor comment is that >

Re: [VOTE] KIP-415: Incremental Cooperative Rebalancing in Kafka Connect

+1 Thanks for all the work on this. My only minor comment is that `connect.protocol` probably should be `compatible` by default. The cost is low and it will save upgrade confusion. Best, Jason On Fri, Mar 8, 2019 at 10:37 AM Robert Yokota wrote: > Thanks for the great KIP Konstantine! > > +1

Re: [DISCUSS] KIP-392: Allow consumers to fetch from the closest replica

Hey Everyone, Apologies for the long delay. I am picking this work back up. After giving this some further thought, I decided it makes the most sense to move replica selection logic into the broker. It is much more difficult to coordinate selection logic in a multi-tenant environment if

Build failed in Jenkins: kafka-trunk-jdk11 #363

See Changes: [github] KAFKA-8091; Wait for processor shutdown before testing removed listeners -- [...truncated 2.34 MB...] WARNING: All illegal access operations will

[jira] [Created] (KAFKA-8094) Iterating over cache with get(key) is inefficient

Sophie Blee-Goldman created KAFKA-8094: -- Summary: Iterating over cache with get(key) is inefficient Key: KAFKA-8094 URL: https://issues.apache.org/jira/browse/KAFKA-8094 Project: Kafka

Re: [DISCUSS] KIP-424: Allow suppression of intermediate events based on wall clock time

Thanks for the response, Matthias, I agree on both of these points. I didn't mean to question whether we should discuss it; we should, since as you point out both points affect the behavior of the API. Regarding checking system time, After reviewing the motivation of the KIP, it seems like a

[jira] [Created] (KAFKA-8093) Fix JavaDoc markup

Matthias J. Sax created KAFKA-8093: -- Summary: Fix JavaDoc markup Key: KAFKA-8093 URL: https://issues.apache.org/jira/browse/KAFKA-8093 Project: Kafka Issue Type: Bug Components:

Re: KIP-213 - Scalable/Usable Foreign-Key KTable joins - Rebooted.

I'd agree with Adam as well: we could consider optimizations leveraging headers as a separate topic that applies more broadly in the future. On Mon, Mar 11, 2019 at 11:29 AM Adam Bellemare wrote: > Hi John > > Thanks for the explanation. I wasn't sure how KTable repartition topics > were

Re: [VOTE] - KIP-213 (new vote) - Simplified and revised.

Thanks a lot for putting this together Adam and Jan! +1 (binding) -Matthias On 2/15/19 11:27 AM, Adam Bellemare wrote: > Hi Bill > > Now that you are a committer, does your vote add a +1 to binding? Can you > recast it if you believe this is a sound decision? I am eager to finally >

Re: KIP-213 - Scalable/Usable Foreign-Key KTable joins - Rebooted.

> I am more inclined to go with keeping it consistent and >> separated into a normal repartition topic and a normal changelog topic >> otherwise. SGTM. -Matthias On 3/11/19 11:18 AM, Adam Bellemare wrote: > Hi John > > Thanks for the explanation. I wasn't sure how KTable repartition topics >

[jira] [Created] (KAFKA-8092) Flaky Test GroupAuthorizerIntegrationTest#testSendOffsetsWithNoConsumerGroupDescribeAccess

Matthias J. Sax created KAFKA-8092: -- Summary: Flaky Test GroupAuthorizerIntegrationTest#testSendOffsetsWithNoConsumerGroupDescribeAccess Key: KAFKA-8092 URL: https://issues.apache.org/jira/browse/KAFKA-8092

Re: [DISCUSS] KIP-424: Allow suppression of intermediate events based on wall clock time

I agree that there are multiple ways how to avoid calling `System.currentTimeMillis()`. However, the KIP needs to define the public contract to explain users what behavior they can expect (the simplest thing might be to say, it's based on `punctuation()` schedule -- not sure if this is desired or

Jenkins build is back to normal : kafka-trunk-jdk8 #3456

See

Jenkins build is back to normal : kafka-2.2-jdk8 #55

See

Help us understand the effects of developers' personality on collaboration in OSS development

(** Apologies for multiple emails **) Dear Apache developer, We are a team of researchers from the Collab research group (http://collab.di.uniba.it), in the Department of Computer Science at the University of Bari, Italy. We would be grateful if you could help us understand the effects of

[jira] [Resolved] (KAFKA-8091) Flaky test DynamicBrokerReconfigurationTest#testAddRemoveSaslListener

[ https://issues.apache.org/jira/browse/KAFKA-8091?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajini Sivaram resolved KAFKA-8091. --- Resolution: Fixed Reviewer: Manikumar > Flaky test

Re: KIP-213 - Scalable/Usable Foreign-Key KTable joins - Rebooted.

Hi John Thanks for the explanation. I wasn't sure how KTable repartition topics were handled with regards to cleanup but I just wanted to double check to see if it could cause an issue. @Matthias My inclination is to keep the DSL topologies consistent with one another. I am a bit concerned about

Re: Using Kafka Internals from Within Plugin

Yeah, I agree. I looked at CruiseControl and it seems to be using the same kind of model (as a producer). The auth model we’re planning has a set of apps which each self-sign a token, so we need a list of keys and identities we trust, which changes over time. Kinda sounds like a log ;), so our

Re: [DISCUSSION] KIP-421: Support resolving externalized secrets in AbstractConfig

Hi Folks, I have accommodated most of the review comments for https://cwiki.apache.org/confluence/display/KAFKA/KIP-421%3A+Support+resolving+externalized+secrets+in+AbstractConfig . Reopening the thread for further discussion. Please let me know your thoughts on it. Thanks, Tejal On

Re: [DISCUSS] KIP-439: Deprecate Interface WindowStoreIterator

I am open to change the return type to KeyValueIterator, V> However, this requires to rename #fetch(K key, long startTimestamp, long endTimestamp) #fetch(K key, Instant startTimestamp, Instant endTimestamp) to avoid naming conflicts. What new name would you suggest? The existing methods are

Re: [DISCUSS] KIP-439: Deprecate Interface WindowStoreIterator

I was thinking about changing the return type even, to `KeyValueIterator, V>` since it is confusing to users about the key typed `Long` (Streams javadoc today did not explain it clearly either), note it is not backward compatible at all. Personally I'd prefer to just deprecate the API and new new

Re: KIP-213 - Scalable/Usable Foreign-Key KTable joins - Rebooted.

I guess Adam suggests, to use compaction for the repartition topic and don't purge data. Doing this, would allow us to avoid a store changelog topic for the "subscription store" on the RHS. This would be a nice optimization. But the concern about breaking compaction is correct. However, I see it

Re: [DISCUSS] KIP-439: Deprecate Interface WindowStoreIterator

I remember thinking this while working on window stores, am definitely for it. On Mon, Mar 11, 2019 at 9:20 AM John Roesler wrote: > Sounds great to me. Thanks, Matthias! > -John > > On Sun, Mar 10, 2019 at 11:58 PM Matthias J. Sax > wrote: > > > Hi, > > > > I would like to propose KIP-439 to

Re: KIP-213 - Scalable/Usable Foreign-Key KTable joins - Rebooted.

Hey Adam, That's a good observation, but it wouldn't be a problem for repartition topics because Streams aggressively deletes messages from the reparation topics once it knows they are handled. Thus, we don't need to try and cater to the log compactor. Thanks, -John On Mon, Mar 11, 2019 at 9:10

Re: [DISCUSS] KIP-439: Deprecate Interface WindowStoreIterator

Sounds great to me. Thanks, Matthias! -John On Sun, Mar 10, 2019 at 11:58 PM Matthias J. Sax wrote: > Hi, > > I would like to propose KIP-439 to deprecate interface > `WindowStoreIterator`. > > >

Re: [DISCUSS] KIP-424: Allow suppression of intermediate events based on wall clock time

Hey, all, just chiming in to keep the discussion moving... Regarding whether to flush or not on shutdown, I'm curious why we *would* flush... The record cache does this, but that's because it's not durable. The suppression buffer is already backed by a changelog specifically so that it can

[jira] [Resolved] (KAFKA-7939) Flaky Test KafkaAdminClientTest#testCreateTopicsRetryBackoff

[ https://issues.apache.org/jira/browse/KAFKA-7939?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Manikumar resolved KAFKA-7939. -- Resolution: Fixed Issue resolved by pull request 6418 [https://github.com/apache/kafka/pull/6418] >

Re: KIP-213 - Scalable/Usable Foreign-Key KTable joins - Rebooted.

For the sake of expediency, I updated the KIP with what I believe we have discussed. https://cwiki.apache.org/confluence/display/KAFKA/KIP-213+Support+non-key+joining+in+KTable#KIP-213Supportnon-keyjoininginKTable-Tombstones On Mon, Mar 11, 2019 at 8:20 AM Adam Bellemare wrote: > My only

Re: KIP-213 - Scalable/Usable Foreign-Key KTable joins - Rebooted.

My only concern was around compaction of records in the repartition topic. This would simply mean that these records would stick around as their value isn't truly null. Though I know about the usage of compaction on changelog topics, I am a bit fuzzy on where we use compaction in other internal

Re: Speeding up integration tests

I agree with Ron. I think improving the framework with a configurable number of retries on some tests will yield the highest ROI in terms of passing builds. On Fri, Mar 8, 2019 at 10:48 PM Ron Dagostino wrote: > It's a classic problem: you can't string N things together serially and > expect

[jira] [Created] (KAFKA-8091) Flaky test DynamicBrokerReconfigurationTest#testAddRemoveSaslListener

Rajini Sivaram created KAFKA-8091: - Summary: Flaky test DynamicBrokerReconfigurationTest#testAddRemoveSaslListener Key: KAFKA-8091 URL: https://issues.apache.org/jira/browse/KAFKA-8091 Project:

[jira] [Created] (KAFKA-8090) Replace ControlledShutdown request/response with automated protocol

Mickael Maison created KAFKA-8090: - Summary: Replace ControlledShutdown request/response with automated protocol Key: KAFKA-8090 URL: https://issues.apache.org/jira/browse/KAFKA-8090 Project: Kafka

[DISCUSS] KIP-437: Custom replacement for MaskField SMT

Hi All, I would like to start a discussion about adding new functionality to MaskField SMT. The existing implementation allows to mask out any field value with the null equivalent of the field type. I suggest to add a possibility to provide a literal replacement for the field. This way you can

[jira] [Resolved] (KAFKA-8072) Transient failure in SslSelectorTest.testCloseOldestConnectionWithMultipleStagedReceives

[ https://issues.apache.org/jira/browse/KAFKA-8072?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajini Sivaram resolved KAFKA-8072. --- Resolution: Fixed Fix Version/s: (was: 2.2.1) (was: 2.3.0)

[jira] [Created] (KAFKA-8089) High level consumer from MirrorMaker is slow to deal with SSL certification expiration

Henry Cai created KAFKA-8089: Summary: High level consumer from MirrorMaker is slow to deal with SSL certification expiration Key: KAFKA-8089 URL: https://issues.apache.org/jira/browse/KAFKA-8089