[jira] [Created] (KAFKA-9108) Flaky Test LogCleanerIntegrationTest#testMarksPartitionsAsOfflineAndPopulatesUncleanableMetrics

Matthias J. Sax created KAFKA-9108: -- Summary: Flaky Test LogCleanerIntegrationTest#testMarksPartitionsAsOfflineAndPopulatesUncleanableMetrics Key: KAFKA-9108 URL: https://issues.apache.org/jira/browse/KAFKA-9108

[DISCUSS] KIP-501 Avoid out-of-sync or offline partitions when follower fetch requests not processed in time

Hi All, I wrote a short KIP about avoiding out-of-sync or offline partitions when follower fetch requests are not processed in time by the leader replica. KIP-501 is located at https://s.apache.org/jhbpn Please take a look, I would like to hear your feedback and suggestions. JIRA:

Re: [DISCUSS] KIP-405: Kafka Tiered Storage

Hi Viktor, >1. Can we allow RLM Followers to serve read requests? After all segments on the cold storage are closed ones, no modification is allowed. Besides KIP-392 ( https://cwiki.apache.org/confluence/display/KAFKA/KIP-392%3A+Allow+consumers+to+fetch+from+closest+replica) would introduce

Re: [DISCUSS] KIP-150 - Kafka-Streams Cogroup

Hi Walker, This is a good point about compatibility breakage while overloading the existing classes; while reading John and your exchanges, I think I still need to clarify the motivations a bit more: 1) Multiple streams need to be aggregated together, inputs are always *KStreams* and end result

Jenkins build is back to normal : kafka-trunk-jdk8 #4003

See

Re: [DISCUSS] KIP-150 - Kafka-Streams Cogroup

Hi John, Thank you for the background information. I think I understand your point. I believe that this could be fixed by making the motivation a little clearer in the KIP. I think that the motivation is when you have multiple streams that need to aggregate together to form a single object the

Re: [DISCUSS] KIP-150 - Kafka-Streams Cogroup

Hi Walker, Sorry for the delay in responding. Thanks for your response earlier. I think there might be a subtlety getting overlooked in considering whether we're talking about streams versus tables in cogroup. As I'm sure you know, Kafka Streams treats "stream" records as independent, immutable,

[jira] [Resolved] (KAFKA-8700) Flaky Test QueryableStateIntegrationTest#queryOnRebalance

[ https://issues.apache.org/jira/browse/KAFKA-8700?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chris Pettitt resolved KAFKA-8700. -- Resolution: Fixed Checked in a fix for this which may reduce flakiness. Also improved

Re: [VOTE] KIP-441: Smooth Scaling Out for Kafka Streams

Hey all, Now that the 2.4 release storm is over, I'd like to bump this vote thread. Currently, we have two binding +1s (Guozhang and Bill), and four non-binding ones (Bruno, Vinoth, Sophie, and myself), and no vetoes. Thanks, -John On Thu, Sep 12, 2019 at 12:54 PM Bill Bejeck wrote: > > +1

Re: [DISCUSS] KIP-405: Kafka Tiered Storage

Got it, thanks Ying. Ryanne On Mon, Oct 28, 2019, 2:42 PM Ying Zheng wrote: > > > > > > > > > - Why wait until local segments expire before offloading them to cold > > storage? Why not stream to HDFS/S3 on an ongoing basis? I'd think this > > would reduce bursty behavior from periodic

Re: [VOTE] KIP-523 Add KStream#toTable to the Streams DSL

Thanks, Aishwarya! I'm +1 (non-binding) -John On Mon, Oct 28, 2019 at 11:58 AM aishwarya kumar wrote: > > Thank you, > > Two binding votes so far. > > I'll keep this thread open for a couple of days. > > Best, > Aishwarya > > On Thu, Oct 24, 2019, 3:05 PM Bill Bejeck wrote: > > > Thanks for

Re: [DISCUSS] KIP-150 - Kafka-Streams Cogroup

Hi Gouzhang, Matthias and I did talk about overloading different a type of aggregate methods in the cogroup that would take in the windows and returns a windowed KTable. We decided that it would break too much with the current pattern that was established in the normal KStream. We can revisit

Re: [DISCUSS] KIP-405: Kafka Tiered Storage

On Thu, Oct 24, 2019 at 7:13 AM Eno Thereska wrote: > Going back to initial thread with general questions on KIP. I think > aspects of the user experience still need clarification: > > - if a user has a mix of compacted and non-compacted topics it will be > hard to reason about storage needs

Re: [DISCUSS] KIP-405: Kafka Tiered Storage

> > > > - Why wait until local segments expire before offloading them to cold > storage? Why not stream to HDFS/S3 on an ongoing basis? I'd think this > would reduce bursty behavior from periodic uploads. > > I think you misunderstood the KIP. We do plan to ship the segment files to remote storage

Re: [DISCUSS] KIP-150 - Kafka-Streams Cogroup

Thank you Bill, I can get behind keeping the correct pattern and adding the Named object into the aggregate. And adding something like Cogrouped Object is a bit beyond the scope of this KIP. I updated the KIP to include the changes to include the Named parameter and some minor text fixes. If

Re: [DISCUSS] KIP-150 - Kafka-Streams Cogroup

Hi Walker, On Fri, Oct 25, 2019 at 1:34 PM Walker Carlson wrote: > Hi Guozhang, > > 1. I am familiar with the cogroup of spark, it is very similar to > their join operator but instead it makes the values iterable. I think that > the use cases are different enough that it makes sense to specify

Build failed in Jenkins: kafka-trunk-jdk11 #918

See Changes: [github] KAFKA-9048 Pt1: Remove Unnecessary lookup in Fetch Building (#7576) -- [...truncated 2.72 MB...] org.apache.kafka.streams.MockTimeTest >

[Jira contributor list] Adding request

Hi all, I'm new to apache kafka project and would love to contribute to it by fixing bugs/documentation. As per https://kafka.apache.org/contributing.html, could you please add me to the contributor list so that I can start working on starter bugs. JIRA id: Hmed06 Thank you,

[jira] [Resolved] (KAFKA-8800) Flaky Test SaslScramSslEndToEndAuthorizationTest#testNoDescribeProduceOrConsumeWithoutTopicDescribeAcl

[ https://issues.apache.org/jira/browse/KAFKA-8800?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guozhang Wang resolved KAFKA-8800. -- Fix Version/s: (was: 2.5.0) Resolution: Duplicate > Flaky Test >

[jira] [Resolved] (KAFKA-8937) Flaky Test SaslPlainSslEndToEndAuthorizationTest#testNoDescribeProduceOrConsumeWithoutTopicDescribeAcl

[ https://issues.apache.org/jira/browse/KAFKA-8937?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guozhang Wang resolved KAFKA-8937. -- Resolution: Duplicate > Flaky Test >

[jira] [Resolved] (KAFKA-9045) Flaky Test DelegationTokenEndToEndAuthorizationTest#testNoDescribeProduceOrConsumeWithoutTopicDescribeAcl

[ https://issues.apache.org/jira/browse/KAFKA-9045?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guozhang Wang resolved KAFKA-9045. -- Resolution: Duplicate > Flaky Test >

Re: [VOTE] KIP-523 Add KStream#toTable to the Streams DSL

Thank you, Two binding votes so far. I'll keep this thread open for a couple of days. Best, Aishwarya On Thu, Oct 24, 2019, 3:05 PM Bill Bejeck wrote: > Thanks for the KIP, this is something that will be appreciated by the > community. > > +1(binding) > > -Bill > > On Thu, Oct 24, 2019 at

Re: [DISCUSS] KIP-543: Expand ConfigCommand's non-ZK functionality

Hi Stanislav, That's fair, I wouldn't imagine it being necessary to add --describe for all broker-loggers, but was intended mainly for consistency with the other entity types. One might find it useful to grep for certain entries, or it's possible that the output is improved to only show

Re: [DISCUSS] KIP-150 - Kafka-Streams Cogroup

Hi Walker, Thanks for taking on KIP-150, the co-group will be very useful. Regarding the naming, IMHO, we should stick to the current pattern, and that is, we provide overloads with a "Named" operator for the "aggregate" methods (I believe those are the only ones that create a processor).

Re: [DISCUSS] KIP-544: Make metrics exposed via JMX configurable

Hi Xavier, How would the practical application look like if this was implemented? Would monitoring agents switch between the whitelist and blacklist periodically if they wanted to monitor every metrics? I think we should make some usage recommendations. Thanks, Viktor On Sun, Oct 27, 2019 at

Re: [DISCUSSION] KIP-422: Add support for user/client configuration in the Kafka Admin Client

Hi Yaodong, Is this KIP still active? In case you have no time for it, I'd like to continue this. Thanks, Viktor On Thu, May 23, 2019 at 6:37 PM Yaodong Yang wrote: > friendly ping... > > On Wed, May 15, 2019 at 10:57 AM Yaodong Yang > wrote: > > > Hello Colin, Rajini and Jun, > > > > I have