[jira] [Created] (KAFKA-10540) Convert KStream aggregations to new PAPI

2020-09-29 Thread John Roesler (Jira)
John Roesler created KAFKA-10540: Summary: Convert KStream aggregations to new PAPI Key: KAFKA-10540 URL: https://issues.apache.org/jira/browse/KAFKA-10540 Project: Kafka Issue Type: Sub

[jira] [Created] (KAFKA-10539) Convert KStreamImpl joins to new PAPI

2020-09-29 Thread John Roesler (Jira)
John Roesler created KAFKA-10539: Summary: Convert KStreamImpl joins to new PAPI Key: KAFKA-10539 URL: https://issues.apache.org/jira/browse/KAFKA-10539 Project: Kafka Issue Type: Sub-task

[jira] [Created] (KAFKA-10538) Convert KStreamImpl maps to new PAPI

2020-09-29 Thread John Roesler (Jira)
John Roesler created KAFKA-10538: Summary: Convert KStreamImpl maps to new PAPI Key: KAFKA-10538 URL: https://issues.apache.org/jira/browse/KAFKA-10538 Project: Kafka Issue Type: Sub-task

[jira] [Created] (KAFKA-10537) Convert KStreamImpl filters to new PAPI

2020-09-29 Thread John Roesler (Jira)
John Roesler created KAFKA-10537: Summary: Convert KStreamImpl filters to new PAPI Key: KAFKA-10537 URL: https://issues.apache.org/jira/browse/KAFKA-10537 Project: Kafka Issue Type: Sub-task

[jira] [Created] (KAFKA-10536) KIP-478: Implement KStream changes

2020-09-29 Thread John Roesler (Jira)
John Roesler created KAFKA-10536: Summary: KIP-478: Implement KStream changes Key: KAFKA-10536 URL: https://issues.apache.org/jira/browse/KAFKA-10536 Project: Kafka Issue Type: Sub-task

[jira] [Created] (KAFKA-10535) KIP-478: Implement StateStoreContext and Record

2020-09-29 Thread John Roesler (Jira)
John Roesler created KAFKA-10535: Summary: KIP-478: Implement StateStoreContext and Record Key: KAFKA-10535 URL: https://issues.apache.org/jira/browse/KAFKA-10535 Project: Kafka Issue Type

Re: [VOTE] KIP-663: API to Start and Shut Down Stream Threads and to Request Closing of Kafka Streams Clients

2020-09-29 Thread John Roesler
Streams client will > transit to ERROR if the last alive stream thread dies exceptionally. > This corresponds to the current behavior. > > I will not restart voting and keep the votes so far. > > Best, > Bruno > > On 22.09.20 01:19, John Roesler wrote: > >

Re: [DISCUSS] KIP-508: Make Suppression State Queriable - rebooted.

2020-09-28 Thread John Roesler
t; > > I updated the PR applying the API changes we discussed above. I am now > > updating the KIP document. > > > > Thanks, > > Dongjin > > > > On Sat, Sep 19, 2020 at 10:42 AM John Roesler wrote: > > > >> Hi Dongjin, > >> > >

Re: [VOTE] KIP-671: Add method to Shutdown entire Streams Application

2020-09-24 Thread John Roesler
Thanks for the KIP, Walker! I’m +1 (binding) -John On Mon, Sep 21, 2020, at 17:04, Guozhang Wang wrote: > Thanks for finalizing the KIP. +1 (binding) > > > Guozhang > > On Mon, Sep 21, 2020 at 1:38 PM Walker Carlson > wrote: > > > Hello all, > > > > I would like to start a thread to vote fo

Re: [DISCUSS] KIP-671: Shutdown Streams Application when appropriate exception is thrown

2020-09-24 Thread John Roesler
Hello all, Thanks for bringing this up, Bruno. It’s a really good point that a disconnected node would miss the signal and then resurrect a single-node “zombie cluster” when it reconnects. Offhand, I can’t think of a simple and reliable way to distinguish this case from one in which an operato

Re: [VOTE] KIP-666: Add Instant-based methods to ReadOnlySessionStore

2020-09-21 Thread John Roesler
Thanks, Jorge! I’m +1 (binding) -John On Mon, Sep 21, 2020, at 12:35, Sophie Blee-Goldman wrote: > Thanks for pointing out the vote in the discussion thread, this email > somehow skipped my inbox ¯\_(ツ)_/¯ > > I'm +1 (non-binding) > > -Sophie > > On Mon, Sep 7, 2020 at 4:18 AM Jorge Esteban

Re: [VOTE] KIP-663: API to Start and Shut Down Stream Threads and to Request Closing of Kafka Streams Clients

2020-09-21 Thread John Roesler
till +1, please vote again. > > > > Best, > > Bruno > > > > On 04.09.20 23:12, John Roesler wrote: > > > Hi Sophie, > > > > > > Uh, oh, it's never a good sign when the discussion moves > > > into the vote thread :) > >

Re: [DISCUSS] KIP-666: Add Instant-based methods to ReadOnlySessionStore

2020-09-18 Thread John Roesler
Thanks for the KIP, Jorge! Sorry it took me so long. I just reviewed the KIP, and it looks good to me. Thanks, John On Tue, Sep 1, 2020, at 12:35, Jorge Esteban Quilcate Otoya wrote: > Thanks Sophie! > > > one nit: you missed updating the startTime long to Instant in both > appearances of the

Re: [DISCUSS] KIP-508: Make Suppression State Queriable - rebooted.

2020-09-18 Thread John Roesler
l update the KIP by adding the discussion > above. > > Best, > Dongjin > > On Thu, Sep 17, 2020 at 11:06 AM John Roesler wrote: > > > Hi Dongjin, > > > > Thanks for the reply. Yes, that’s correct, we added that method to name > > the operation. But

Re: [DISCUSS] KIP-508: Make Suppression State Queriable - rebooted.

2020-09-16 Thread John Roesler
ason that you prefer the second > approach? > > Thanks, > Dongjin > > [^1]: > https://cwiki.apache.org/confluence/display/KAFKA/KIP-307%3A+Allow+to+define+custom+processor+names+with+KStreams+DSL > > > > On Wed, Sep 16, 2020 at 11:48 PM John Roesler wrote: >

Re: [DISCUSS] KIP-508: Make Suppression State Queriable - rebooted.

2020-09-16 Thread John Roesler
fine API with the > other related KIPs later. > > Do I understand correctly? > > Thanks, > Dongjin > > On Wed, Sep 16, 2020 at 2:17 AM John Roesler wrote: > > > Hi Dongjin, > > > > Thanks for presenting these options. The concern that > > Matthias broug

Re: [DISCUSS] KIP-508: Make Suppression State Queriable - rebooted.

2020-09-15 Thread John Roesler
> > > > > > -Matthias > > > > > > On 9/11/20 12:26 AM, Dongjin Lee wrote: > > > Hi All, > > > > > > Here is the voting thread: > > > > > > > https://lists.apache.org/thread.html/r5653bf2dafbb27b247

Re: [VOTE] KIP-478 Strongly Typed Streams Processor API

2020-09-14 Thread John Roesler
is case. > > Thus, for both cases, accessing the record context does not seems to be > a requirement. And providing access to the processor context to, eg., > `forward()` or similar seems safe. > > > -Matthias > > On 9/10/20 7:25 PM, John Roesler wrote: > > Than

Re: [DISCUSS] KIP-406: GlobalStreamThread should honor custom reset policy

2020-09-14 Thread John Roesler
on’t want. So, > > restore for global store and get() might have to be exclusive. Is there a way > for a > > store to know if it global store or not because now internally global and > normal > > stores will behave differently. Although if everyone is fine with the above

Re: Java Kafka Streams

2020-09-14 Thread John Roesler
Hello Sebastián, Thanks for the question. I think the reason you don't find documentation for this is that it's quite unusual to have a data stream that just stops at some point and never sends any more data. Can you elaborate a little whether you meant that the whole input topic just "ends" at s

Re: How do I place a JIRA in status "Patch Available"? (For KAFKA-10473)

2020-09-11 Thread John Roesler
t; Thanks for the help, > -James > > > On Sep 11, 2020, at 12:53 PM, John Roesler wrote: > > > > Hi James, > > > > Sorry, I overlooked your reply until now. I've granted you > > access. > > > > Thanks, > > -John > >

Re: How do I place a JIRA in status "Patch Available"? (For KAFKA-10473)

2020-09-11 Thread John Roesler
Hi James, Sorry, I overlooked your reply until now. I've granted you access. Thanks, -John On Wed, 2020-09-09 at 21:44 -0700, James Cheng wrote: > Thanks John. My wiki user ID is wushujames > > -James > > Sent from my iPhone > > > On Sep 9, 2020, at

Re: [DISCUSS] KIP-516: Topic Identifiers

2020-09-11 Thread John Roesler
Have you thought about whether it makes sense to support authorizing a > > principal for a topic ID rather than a topic name to achieve tighter > > security? > > > > Or is the topic ID fundamentally an internal detail similar to epochs used > > in a bunch of other places in Kafk

Re: [DISCUSS] KIP-516: Topic Identifiers

2020-09-11 Thread John Roesler
Hello Justine, Thanks for the KIP! I happen to have been confronted recently with the need to keep track of a large number of topics as compactly as possible. I was going to come up with some way to dictionary encode the topic names as integers, but this seems much better! Apologies if this h

Re: [VOTE] KIP-478 Strongly Typed Streams Processor API

2020-09-10 Thread John Roesler
r previous response regarding the > > range query after all. I thought you just meant a time-range query inside a > > punctuator. It genuinely did not occur to me that users might be looking up > > and/or updating records of other keys from within a Processor. Sorry for > &

Re: [VOTE] KIP-478 Strongly Typed Streams Processor API

2020-09-10 Thread John Roesler
hat might be > a valid use case or just a lurking bug in itself :) > > Thanks for humoring me. The current proposal for KIP-478 sounds good to me > > On Thu, Sep 10, 2020 at 3:43 PM John Roesler wrote: > > > Ah, thanks Sophie, > > > > I'm sorry for misint

Re: [VOTE] KIP-478 Strongly Typed Streams Processor API

2020-09-10 Thread John Roesler
lear use case I'd > find > that convincing. In the absence of any examples, the conservative approach > sounds good to me. > > If it turns out that someone did need the record context in their custom > state > store, I'm sure they'll submit a politely worded

Re: [VOTE] KIP-478 Strongly Typed Streams Processor API

2020-09-10 Thread John Roesler
ore > from their punctuator. > > Again, I think I do agree that we should leave the current record context > off of > the StateStoreContext, but I don't think the Punctuator argument against it > is > very convincing. It sounds to me like we need to disallow access to the >

Re: [DISCUSS] KIP-508: Make Suppression State Queriable - rebooted.

2020-09-10 Thread John Roesler
have a > look after your work is done. > > Thanks, > Dongjin > > On Sun, Mar 8, 2020 at 12:20 PM John Roesler wrote: > > > Thanks Matthias, > > > > Good idea. I've changed the ticket name and added a note > > clarifying that this ticket is not th

Re: [VOTE] KIP-478 Strongly Typed Streams Processor API

2020-09-10 Thread John Roesler
store. But I can see some other > things being useful, for example the current record's headers. Maybe if/when > we add better (or any) support for headers in state stores this will be > less true. > > Of course as John has made clear, it's pretty hard to judge withou

Re: [VOTE] KIP-478 Strongly Typed Streams Processor API

2020-09-09 Thread John Roesler
/org/apache/kafka/streams/state/internals/ChangeLoggingKeyValueBytesStore.java#L121 > > You could have the writer client of the state store pass this through, but > it would be nice to be able to write state stores where the client did not > have this responsibility. I'm not s

Re: How do I place a JIRA in status "Patch Available"? (For KAFKA-10473)

2020-09-09 Thread John Roesler
atus of the JIRA to "Patch Available" if it's ready > for review. Do this by clicking the "Submit Patch" button in JIRA, and > then in the resulting dialog, click "Submit Patch". > > -James > > > On Sep 9, 2020, at 6:24 PM, John Roesl

Re: How do I place a JIRA in status "Patch Available"? (For KAFKA-10473)

2020-09-09 Thread John Roesler
Hi James, I think the button on Jira says “Add Patch” or something confusing like that. Thanks, John On Wed, Sep 9, 2020, at 17:34, James Cheng wrote: > I have a JIRA that I am working on, and a pull request available for it. > > [KAFKA-10473] Website is missing docs on JMX metrics for partit

Re: [VOTE] KIP-478 Strongly Typed Streams Processor API

2020-09-09 Thread John Roesler
Hello all, I've been slowly pushing KIP-478 forward over the last year, and I'm happy to say that we're making good progress now. However, several issues with the original design have come to light. The major changes: We discovered that the original plan of just adding generic parameters to Proc

[jira] [Created] (KAFKA-10472) Consider migrating Topology methods to the Builder pattern

2020-09-09 Thread John Roesler (Jira)
John Roesler created KAFKA-10472: Summary: Consider migrating Topology methods to the Builder pattern Key: KAFKA-10472 URL: https://issues.apache.org/jira/browse/KAFKA-10472 Project: Kafka

Re: [VOTE] KIP-667: Remove deprecated methods from ReadOnlyWindowStore

2020-09-08 Thread John Roesler
Hello Jorge, Thanks for the KIP! FYI, it looks like there's still a boilerplate paragraph in the KIP document about it being a template. The KIP looks good to me; I'm +1 (binding) for doing this in 3.0. Thanks, -John On Mon, 2020-09-07 at 12:18 +0100, Jorge Esteban Quilcate Otoya wrote: > Hi e

Re: [VOTE] KIP-663: API to Start and Shut Down Stream Threads and to Request Closing of Kafka Streams Clients

2020-09-04 Thread John Roesler
Hi Sophie, Uh, oh, it's never a good sign when the discussion moves into the vote thread :) I agree with you, it seems like a good touch for removeStreamThread() to return the name of the thread that got removed, rather than a boolean flag. Maybe the return value would be `null` if there is no th

Re: [DISCUSS] KIP-406: GlobalStreamThread should honor custom reset policy

2020-09-04 Thread John Roesler
een > > the > > > > two at all? > > > > Would it make sense to avoid a special GLOBAL_RESTORING state and just > > > > introduce > > > > a single unified RESTORING state to cover both the regular and global > > case? > > > >

Re: [VOTE] KIP-614: Add Prefix Scan support for State Stores

2020-09-03 Thread John Roesler
> > > Just thought of bumping this voting thread again to see if we can > > > form > > > > > any > > > > > > consensus around this. > > > > > > > > > > > > Thanks! > > > > > > Sagar. >

Re: [DISCUSS] KIP-659: Improve TimeWindowedDeserializer and TimeWindowedSerde to handle window size

2020-09-02 Thread John Roesler
ood to me as well, I added a few lines in the > > KIP about logging an error when the *timeWindowedSerde *implicit is called. > > > > Let me know if there are any other concerns, else I'll resume voting. > > > > Cheers, > > Leah > > > >

Re: [VOTE] KIP-616: Rename implicit Serdes instances in kafka-streams-scala

2020-09-01 Thread John Roesler
. Sax wrote: > > > > > Thanks for the KIP Yuriy! > > > > > > +1 (binding) > > > > > > > > > -Matthias > > > > > > On 8/13/20 9:49 PM, John Roesler wrote: > > > > Hi Yuriy, > > > > > > >

Re: [DISCUSS] KIP-659: Improve TimeWindowedDeserializer and TimeWindowedSerde to handle window size

2020-09-01 Thread John Roesler
; > > > > > I do feel like we would want to add an implicit to `Serdes.scala` that > > > takes a serde and a window size so that users can access the constructor > > > that initializes with the correct window size. I agree with your comment > > on > > > the K

Re: Request to Add in Contributor list

2020-09-01 Thread John Roesler
Hi Sharath, Please forgive me if this is a mistake, but I think I've just granted this permission for your username "DOJI". Do you need access as "DOJI45"? Thanks, -John On Tue, 2020-09-01 at 16:11 +0530, Sharath Bhat wrote: > Hi TeamPlease guide me on how I can request contributor access for ji

Re: Need contributor access to Jira

2020-09-01 Thread John Roesler
Hi Sandeep, I've added you to the contributors group in Jira. Thanks for your interest in Apache Kafka! Thanks, -John On Tue, 2020-09-01 at 16:13 +0530, Sandeep Kumar wrote: > Hi Team , > > Please guide me how I can request for the contributor access for jira so > that I can assign some jira ti

Re: Request to Add in Contributor list

2020-09-01 Thread John Roesler
Hi Sharath, I've added you to the contributor group in Jira. Thanks for your interest in Apache Kafka! Thanks, -John On Tue, 2020-09-01 at 17:22 +0530, Sharath Bhat wrote: > Hi TeamPlease guide me on how I can request contributor access for jira so > that i can assign some jira tickets to myself

Re: [DISCUSS] KIP-667: Remove deprecated methods from ReadOnlyWindowStore

2020-09-01 Thread John Roesler
Hi all, Thanks for bringing this up, Jorge! I'm not sure if we have a documented rule; I seem to recall having had the same discussion for 2.0. My personal suggestion would be to have a deprecation period of at least one year, and beyond that to weigh how good we feel about removing each deprecat

Re: [DISCUSS] KIP-663: API to Start and Shut Down Stream Threads and to Request Closing of Kafka Streams Clients

2020-09-01 Thread John Roesler
Hello Bruno, Thanks for the update! The KIP looks good to me; I only have a grammatical complaint about the proposed metric name. "Died" is a verb, the past tense of "to die", but in the expression,"x stream threads", x should be an adjective. To be fair, "died" is also the past participle of "to

Re: [DISCUSS] KIP-406: GlobalStreamThread should honor custom reset policy

2020-08-29 Thread John Roesler
er to make progress on some > tasks. > > Of course, if we collect real world experience and it becomes an issue, > we could still try to change it? > > > -Matthias > > > On 8/18/20 3:31 PM, John Roesler wrote: > > Thanks Matthias, > > > > Sounds go

[jira] [Resolved] (KAFKA-10307) Topology cycles in KTableKTableForeignKeyInnerJoinMultiIntegrationTest#shouldInnerJoinMultiPartitionQueryable

2020-08-28 Thread John Roesler (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-10307?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] John Roesler resolved KAFKA-10307. -- Resolution: Not A Bug This bug report was based on an invalid assumption that topologies

Re: [VOTE] KIP-662: Throw Exception when Source Topics of a Streams App are Deleted

2020-08-27 Thread John Roesler
Thanks, Bruno! I'm +1 (binding) -John On Thu, 2020-08-27 at 15:35 -0400, Bill Bejeck wrote: > Thanks for the KIP Bruno. > > +1 (binding) > > -Bill > > On Thu, Aug 27, 2020 at 3:15 PM Walker Carlson > wrote: > > > +1 (Non Binding). Good Kip Bruno > > > > Walker > > > > On Tue, Aug 25, 2020

Re: [DISCUSS] KIP-663: API to Start and Shut Down Stream Threads and to Request Closing of Kafka Streams Clients

2020-08-26 Thread John Roesler
Hi Bruno, Thanks for the well motivated and throrough KIP! It's a good point that the record cache should be re- distributed over the threads. Reading your KIP leads me to a few questions: 1. Start vs. Add Maybe this is paranoid, but I'm mildly concerned that users who don't read the docs too

Re: [VOTE] KIP-659: Improve TimeWindowedDeserializer and TimeWindowedSerde to handle window size

2020-08-26 Thread John Roesler
Hi all, I've just sent a new message to the DISCUSS thread. We forgot to include the Scala API in the proposal. Thanks, -John On Mon, 2020-08-24 at 18:00 -0700, Sophie Blee-Goldman wrote: > Thanks for the KIP! +1 (non-binding) > > Sophie > > On Mon, Aug 24, 2020 at 5:06 PM

Re: [DISCUSS] KIP-659: Improve TimeWindowedDeserializer and TimeWindowedSerde to handle window size

2020-08-26 Thread John Roesler
s, > Leah > > On Fri, Aug 21, 2020 at 2:50 PM John Roesler wrote: > > > Thanks, all, > > > > Based on my reading of the conversation, it sounds like I > > have some legwork to do in KIP-645, but our collective > > instinct is that Leah's proposal d

[jira] [Created] (KAFKA-10437) Convert test-utils (and StateStore) for KIP-478

2020-08-25 Thread John Roesler (Jira)
John Roesler created KAFKA-10437: Summary: Convert test-utils (and StateStore) for KIP-478 Key: KAFKA-10437 URL: https://issues.apache.org/jira/browse/KAFKA-10437 Project: Kafka Issue Type

[jira] [Created] (KAFKA-10436) Implement KIP-478 Topology changes

2020-08-25 Thread John Roesler (Jira)
John Roesler created KAFKA-10436: Summary: Implement KIP-478 Topology changes Key: KAFKA-10436 URL: https://issues.apache.org/jira/browse/KAFKA-10436 Project: Kafka Issue Type: Sub-task

[jira] [Resolved] (KAFKA-10379) Implement the KIP-478 StreamBuilder#addGlobalStore()

2020-08-25 Thread John Roesler (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-10379?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] John Roesler resolved KAFKA-10379. -- Resolution: Fixed > Implement the KIP-478 StreamBuilder#addGlobalSt

Re: [VOTE] KIP-659: Improve TimeWindowedDeserializer and TimeWindowedSerde to handle window size

2020-08-24 Thread John Roesler
Thanks Leah, I’m +1 (binding) -John On Mon, Aug 24, 2020, at 16:54, Leah Thomas wrote: > Hi everyone, > > I'd like to kick-off the vote for KIP-659: Improve > TimeWindowedDeserializer > and TimeWindowedSerde to handle window size. > https://cwiki.apache.org/confluence/display/KAFKA/KIP-659%3A+I

Re: [DISCUSS] KIP-659: Improve TimeWindowedDeserializer and TimeWindowedSerde to handle window size

2020-08-21 Thread John Roesler
on Windows are handled separately from time > > windows and also have variable window sizes. I assume that the TimeWindowed > > option is preferable for variably sized windows because you still want to > > access the window end times? But I think one alternative could be >

Re: [DISCUSS] KIP-662: Throw Exception when Source Topics of a Streams App are Deleted

2020-08-21 Thread John Roesler
Thanks for the KIP, Bruno! Your proposal sounds good to me. -John On Fri, 2020-08-21 at 11:18 -0700, Sophie Blee-Goldman wrote: > Thanks for the KIP! I'm totally in favor of this approach and to be honest, > have > always wondered why we just silently shut down instead of throwing an > exception

Re: [DISCUSS] KIP-659: Improve TimeWindowedDeserializer and TimeWindowedSerde to handle window size

2020-08-21 Thread John Roesler
Hi Leah, Thanks for the KIP! This has been a real pain for some use cases, so it's really good to see a proposal to fix it. We do need a default constructor so that it can be dynamically instantiated by the consumer (or any other component). But I'm +1 on deprecating the constructor you're propos

Re: [DISCUSS] KIP-406: GlobalStreamThread should honor custom reset policy

2020-08-18 Thread John Roesler
dd a new state to KafkaStreams client for this case? > > Having a new state might also be helpful for > https://issues.apache.org/jira/browse/KAFKA-7380 ? > > > > -Matthias > > > > > On 8/17/20 7:34 AM, John Roesler wrote: > > Hi Navinder, > > > >

Re: [DISCUSS] KIP-406: GlobalStreamThread should honor custom reset policy

2020-08-17 Thread John Roesler
ster, and we have already lost k1, k2 from the global > topic so it will start consuming from the earliest point in the global topic. > So, wouldn’t this global store on the new instance has 2 keys less than all > the other global stores already available in the cluster? Please let

Re: [DISCUSS] Apache Kafka 2.7.0 release

2020-08-14 Thread John Roesler
Thanks, Bill! -John On Thu, 2020-08-13 at 15:19 -0700, Ismael Juma wrote: > Thanks for volunteering Bill. :) > > Ismael > > On Thu, Aug 13, 2020 at 3:13 PM Bill Bejeck wrote: > > > Hi All, > > > > I'd like to volunteer to be the release manager for our next feature > > release, 2.7. If there

Re: [DISCUSS] KIP-639 Move nodeLevelSensor and storeLevelSensor methods from StreamsMetricsImpl to StreamsMetrics

2020-08-14 Thread John Roesler
Thanks Mohamed, I think Bruno raised a good point in the ticket that the node name is not well known from within the Processor at the time of init(), so users would basically have to make up a name to pass into the sensor. Maybe this is ok, but it doesn't seem too nice. Offhand, it seems like the

Re: [DISCUSSION] Upgrade system tests to python 3

2020-08-14 Thread John Roesler
Thanks Nikolay, No objection. This would be very nice to have. Thanks, John On Fri, Aug 14, 2020, at 09:18, Nikolay Izhikov wrote: > Hello. > > > If anyone's interested in porting it to Python 3 it would be a good change. > > I’ve created a ticket [1] to upgrade system tests to python3. > Do

Re: [DISCUSS] KIP-622 Add currentSystemTimeMs and currentStreamTimeMs to ProcessorContext

2020-08-13 Thread John Roesler
If it's not good enough, we can > still change the TimestampExtractor interface as a follow up? > > > -Matthias > > On 7/28/20 10:03 AM, John Roesler wrote: > > Thanks Matthias, > > > > This is a really good point. It might be a bummer > > to have

Re: [VOTE] KIP-616: Rename implicit Serdes instances in kafka-streams-scala

2020-08-13 Thread John Roesler
> > > > -Yuriy > > > > > > On Fri, Aug 7, 2020 at 8:10 AM Yuriy Badalyantc > > wrote: > > > > > > > Hi everybody. > > > > > > > > There was some minor change since the voting process started (nullSerde > > > > ad

Re: [DISCUSS] KIP-406: GlobalStreamThread should honor custom reset policy

2020-08-13 Thread John Roesler
for this or > > just handling the reset in GlobalStreamThread should be sufficient? > > > > Regards, > > Navinder > > > >     On Tuesday, 7 July, 2020, 12:53:36 am IST, Matthias J. Sax > > wrote:  > >  > >  Atm, the config should be ign

Re: [ANNOUNCE] New Kafka PMC Member: John Roesler

2020-08-11 Thread John Roesler
atulations! > > > > > > > > > On Mon, Aug 10, 2020 at 1:11 PM Jun Rao wrote: > > > > > > > > > > Hi, Everyone, > > > > > > > > > > John Roesler has been a Kafka committer since Nov. 5, 2019. He has > > > &

[ANNOUNCE] Apache Kafka 2.5.1

2020-08-11 Thread John Roesler
“Andy” Fang, Dima Reznik, Ego, Evelyn Bayes, Ewen Cheslack-Postava, Greg Harris, Guozhang Wang, Ismael Juma, Jason Gustafson, Jeff Widman, Jeremy Custenborder, jiameixie, John Roesler, Jorge Esteban Quilcate Otoya, Konstantine Karantasis, Lucent-Wong, Mario Molina, Matthias J. Sax, Navinder Pal

[jira] [Created] (KAFKA-10379) Implement the KIP-478 StreamBuilder#addGlobalStore()

2020-08-09 Thread John Roesler (Jira)
John Roesler created KAFKA-10379: Summary: Implement the KIP-478 StreamBuilder#addGlobalStore() Key: KAFKA-10379 URL: https://issues.apache.org/jira/browse/KAFKA-10379 Project: Kafka Issue

[jira] [Resolved] (KAFKA-10261) Introduce the KIP-478 processors with shims

2020-08-09 Thread John Roesler (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-10261?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] John Roesler resolved KAFKA-10261. -- Resolution: Fixed > Introduce the KIP-478 processors with sh

Re: [DISCUSS] KIP-616: Rename implicit Serdes instances in kafka-streams-scala

2020-08-06 Thread John Roesler
of values and deserialization. > > About Null serde - I'm ok with this. But, it should be a low priority > implicit to not intersect will other implicits. I will add it. > > -Yuriy > > On Sat, Jul 11, 2020 at 9:18 PM John Roesler wrote: > > > Thanks, Yuriy,

[RESULTS] [VOTE] Release Kafka version 2.5.1

2020-08-04 Thread John Roesler
Hello all, This vote passes with four +1 votes (3 binding) and no 0 or -1 votes. +1 votes PMC Members (in voting order): * Ismael Juma * Manikumar Reddy * Mickael Maison Committers (in voting order): * John Roesler Community: * No votes 0 votes * No votes -1 votes * No votes Vote thread

Re: [VOTE] 2.5.1 RC0

2020-08-04 Thread John Roesler
ified signatures, ran the tests on the source archive > > > with > > > Scala 2.13 and Java 11 and verified the quickstart with the source archive > > > and Scala 2.13 binary archive. > > > > > > Ismael > > > > > > On Thu, Jul 23, 2020

Re: [DISCUSS] KIP-645: Replace abstract class Windows with a proper interface

2020-08-01 Thread John Roesler
g good and reasonable, > > and the benefits are pretty clear. > > > > Haven't had a careful look through the POC yet but the proposal and > > public changes you described sound great. Thanks for the KIP! > > > > On Tue, Jul 28, 2020 at 10:27 AM John Roesler

Re: [ANNOUNCE] New Kafka PMC Member: Mickael Maison

2020-07-31 Thread John Roesler
Congrats, Mickael! -John On Fri, Jul 31, 2020, at 23:29, Gwen Shapira wrote: > Congratulations Mickael. Thrilled to have you as part of the PMC :) > > On Fri, Jul 31, 2020 at 8:14 AM Jun Rao wrote: > > > Hi, Everyone, > > > > Mickael Maison has been a Kafka committer since Nov. 5, 2019. He has

[jira] [Created] (KAFKA-10336) Rolling upgrade with Suppression AND Standbys may throw exceptions

2020-07-31 Thread John Roesler (Jira)
John Roesler created KAFKA-10336: Summary: Rolling upgrade with Suppression AND Standbys may throw exceptions Key: KAFKA-10336 URL: https://issues.apache.org/jira/browse/KAFKA-10336 Project: Kafka

Re: [VOTE] 2.5.1 RC0

2020-07-30 Thread John Roesler
Hello again all, Just a reminder that the 2.5.1 RC0 is available for verification. Thanks, John On Thu, Jul 23, 2020, at 21:39, John Roesler wrote: > Hello Kafka users, developers and client-developers, > > This is the first candidate for release of Apache Kafka 2.5.1. > > Apa

[jira] [Resolved] (KAFKA-9210) kafka stream loss data

2020-07-29 Thread John Roesler (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-9210?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] John Roesler resolved KAFKA-9210. - Resolution: Fixed > kafka stream loss data > -- > >

Re: [VOTE] KIP-648: Renaming getter method for Interactive Queries

2020-07-29 Thread John Roesler
Hi John, Thanks for picking this up! I’m +1 (binding). -John On Wed, Jul 29, 2020, at 09:14, Jorge Esteban Quilcate Otoya wrote: > +1 (non-binding). > > Thanks John! > > On Wed, Jul 29, 2020 at 3:00 PM Navinder Brar > wrote: > > > +1 (non-binding). Thanks John, looks good to me. > > > > ~Na

Re: [VOTE] KIP-450: Sliding Window Aggregations in the DSL

2020-07-29 Thread John Roesler
Thanks for the awesome KIP, Leah, I’m +1 (binding) Thanks, John On Tue, Jul 28, 2020, at 19:10, Guozhang Wang wrote: > +1 (binding) > > On Tue, Jul 28, 2020 at 4:44 PM Matthias J. Sax wrote: > > > +1 (binding) > > > > On 7/28/20 4:35 PM, Sophie Blee-Goldman wrote: > > > Thanks for the KIP! It

Re: [DISCUSS] KIP-645: Replace abstract class Windows with a proper interface

2020-07-28 Thread John Roesler
that sounds reasonable to you, I can update the KIP accordingly. Thanks, -John On Mon, Jul 27, 2020, at 22:11, John Roesler wrote: > Thanks for the reply, Sophie. > > Yes, I'd neglected to specify that Windows will implement maxSize() > by delegating to size(). It's updated

Re: [DISCUSS] KIP-622 Add currentSystemTimeMs and currentStreamTimeMs to ProcessorContext

2020-07-28 Thread John Roesler
he > >>> new API, could we try to address that? It should help with the motivation > >>> and follow-up meta comments as John proposed. > >>> > >>> Boyang > >>> > >>> On Mon, Jul 6, 2020 at 12:04 PM Matthias J. Sax wrote: &

[jira] [Created] (KAFKA-10319) Fix unknown offset sum on trunk

2020-07-28 Thread John Roesler (Jira)
John Roesler created KAFKA-10319: Summary: Fix unknown offset sum on trunk Key: KAFKA-10319 URL: https://issues.apache.org/jira/browse/KAFKA-10319 Project: Kafka Issue Type: Sub-task

Re: [DISCUSS] KIP-645: Replace abstract class Windows with a proper interface

2020-07-27 Thread John Roesler
throughout the KIP but > just to be clear, > you will be replacing any DSL methods that accept Windows with identical > DSL methods > that take the new EnumerableWindowDefinition as argument. So there is > nothing deprecated > and nothing added, but there are public methods changed. Is t

Re: [DISCUSS] KIP-645: Replace abstract class Windows with a proper interface

2020-07-26 Thread John Roesler
olved by your proposal? > > > > By the way, I assume we would also need to deprecate all APIs that accept a > > Windows > > parameter in favor of new ones that accept a FixedSizeWindowDefinition? Off > > the > > top of my head that would be the windowedBy methods in KGroup

Re: [VOTE] KIP-622 Add currentSystemTimeMs and currentStreamTimeMs to ProcessorContext

2020-07-26 Thread John Roesler
Thanks William, I’m +1 (binding) Thanks, John On Fri, Jul 24, 2020, at 20:22, Sophie Blee-Goldman wrote: > Thanks all, +1 (non-binding) > > Cheers, > Sophie > > On Wed, Jul 8, 2020 at 4:02 AM Bruno Cadonna wrote: > > > Thanks Will and Piotr, > > > > +1 (non-binding) > > > > Best, > > Bruno >

Re: [VOTE] 2.6.0 RC1

2020-07-24 Thread John Roesler
Hi Randall, I'm sorry to say we have also identified that this flaky test failure turned out to be a real blocker bug: https://issues.apache.org/jira/browse/KAFKA-10287 There is a PR in progress. Thanks, -John On Fri, Jul 24, 2020, at 12:26, Matthias J. Sax wrote: > We found a regression bug th

Re: [VOTE] KIP-607: Add Metrics to Record the Memory Used by RocksDB to Kafka Streams

2020-07-24 Thread John Roesler
9.05.20 10:00, Bruno Cadonna wrote: > > Thank you for voting! > > > > This KIP passes with: > > 4 binding +1 > > 1 non-binding +1 > > 0 -1 > > > > Best, > > Bruno > > > > On Fri, May 15, 2020 at 11:34 PM Matthias J. Sax w

Re: [DISCUSS] 2.5.1 Bug Fix Release

2020-07-23 Thread John Roesler
Hello all, Sorry again for the delay. While completing the RC, I found and had to fix a broken system test. However, I've just created the vote thread for 2.5.1 RC0! Please take a look at it and let me know what you think. Thanks, -John On Tue, Jul 14, 2020, at 17:22, John Roesler

[VOTE] 2.5.1 RC0

2020-07-23 Thread John Roesler
Hello Kafka users, developers and client-developers, This is the first candidate for release of Apache Kafka 2.5.1. Apache Kafka 2.5.1 is a bugfix release and fixes 72 issues since the 2.5.0 release. Please see the release notes for more information. Release notes for the 2.5.1 release: https:/

Re: [DISCUSS] KIP-450: Sliding Windows

2020-07-22 Thread John Roesler
ake me nervous to go against the "future-proof" direction, though. > Are there any other examples of things we might want to add to one window > type but not to another? > > On another note, this actually brings up a new question: should > SlidingWindows > also be final?

Re: [DISCUSS] KIP-450: Sliding Windows

2020-07-22 Thread John Roesler
ppens to be. > > > > Also, the fact that they can all rely on the same aggregation algorithm > > seems like an > > implementation detail and it would be weird to force a separate/new DSL API > > just because > > under the covers we swap in a different process

Re: [DISCUSS] KIP-450: Sliding Windows

2020-07-22 Thread John Roesler
e put in a new window can be > > found > > >> on the implementation page > > >> < > > >> > > https://cwiki.apache.org/confluence/display/KAFKA/Aggregation+details+for+KIP-450 > > >>> > > >> . > > >> > > &g

[DISCUSS] KIP-645: Replace abstract class Windows with a proper interface

2020-07-21 Thread John Roesler
Hello all, I'd like to propose KIP-645, to correct a small API mistake in Streams. Fixing this now allows us to avoid perpetuating the mistake in new work. For example, it will allow us to implement KIP-450 cleanly. The change itself should be seamless for users. Please see https://cwiki.apache.

[jira] [Created] (KAFKA-10298) Replace Windows with a proper interface

2020-07-21 Thread John Roesler (Jira)
John Roesler created KAFKA-10298: Summary: Replace Windows with a proper interface Key: KAFKA-10298 URL: https://issues.apache.org/jira/browse/KAFKA-10298 Project: Kafka Issue Type

Re: [DISCUSS] KIP-607: Add Metrics to Record the Memory Used by RocksDB to Kafka Streams

2020-07-21 Thread John Roesler
gt; > on > > >> the scheduled rocksdb metrics recorder whereas previously, we can > > decide to > > >> not turn on the recorder at all if all are set as DEBUG and we > > configure at > > >> INFO level in production. But this is an implementation det

[jira] [Resolved] (KAFKA-10254) 100% cpu usage by kafkaConsumer poll , when broker can‘t be connect

2020-07-17 Thread John Roesler (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-10254?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] John Roesler resolved KAFKA-10254. -- Resolution: Fixed Changing resolution to "fixed", since the linked duplicate is fix

[jira] [Reopened] (KAFKA-10254) 100% cpu usage by kafkaConsumer poll , when broker can‘t be connect

2020-07-17 Thread John Roesler (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-10254?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] John Roesler reopened KAFKA-10254: -- > 100% cpu usage by kafkaConsumer poll , when broker can‘t be conn

<    1   2   3   4   5   6   7   8   9   10   >