; Bill
>
> On Thu, Oct 17, 2019 at 7:42 PM Richard Yu
> wrote:
>
> > Hi all,
> >
> > I wish to discuss this KIP which would help us in resolving some issues
> we
> > have with suppression buffers.
> > Below is the link:
> >
> >
> >
>
Hi all,
Want to try to get this KIP wrapped up. So it would be great if we can get
some votes.
Cheers,
Richard
On Tue, Oct 15, 2019 at 12:58 PM Jun Rao wrote:
> Hi, Richard,
>
> Thanks for the updated KIP. +1 from me.
>
> Jun
>
> On Tue, Oct 15, 2019 at 12:46 PM Richard
Hi all,
I wish to discuss this KIP which would help us in resolving some issues we
have with suppression buffers.
Below is the link:
https://cwiki.apache.org/confluence/display/KAFKA/KIP-539%3A+Implement+mechanism+to+flush+out+records+in+low+volume+suppression+buffers
@John Roesler if you have
gt; With a
> > record batch of 512 in practice, and suppose after compaction each record
> > would take 2 more byte for encoding deltas, that would be 1K more per
> > batch. Usually it would not be too big of an issue with reasonable sized
> > message, but I just want
to
> get events flushed out of the suppression buffer after some wall clock
> time passes, rather than solely based on stream time.
>
> WDYT?
> Thanks,
> -John
>
> On Sun, Oct 20, 2019 at 11:50 AM Richard Yu
> wrote:
> >
> > Hi Bill, John,
> >
at. Perhaps we can just document this
> limitation.
>
> Hi, Richard,
>
> Could you update the KIP with Jason's approach? Also, it seems that KIP-515
> is already taken by another KIP. Could you use a new KIP number for this?
>
> Thanks,
>
> Jun
>
> On Fri, Sep 2
Hi all,
Bumping this. If you feel that this KIP is not too urgent. Then let me
know. :)
Cheers,
Richard
On Thu, Feb 6, 2020 at 4:55 PM Richard Yu
wrote:
> Hi all,
>
> I've had just a few thoughts regarding the forwarding of change>. As Matthias already mentioned, there are tw
d"
> and
> "new" result.
>
> It's certainly a good observation, but I think we can just make a note of
> it
> in "rejected alternatives" for now, and plan to refine it later, if it does
> pose a big performance problem.
>
> Thanks!
> -John
>
Hi all,
Just some updates. Below is the vote thread:
https://sematext.com/opensee/m/Kafka/uyzND1h1NPW1tLVQR?subj=+VOTE+KIP+557+Add+emit+on+change+support+for+Kafka+Streams
It would be great if we can include this change to Kafka. :)
Cheers,
Richard
On Thu, Feb 27, 2020 at 6:45 PM Richard Yu
> >
> >
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-557%3A+Add+emit+on+change+support+for+Kafka+Streams
> > > >
> > > > late-records-drop: INFO at processor node level, replaced by INFO
> > > > task-level "dropped-records".
>
lly not confident that there's anyone
> really using the
> untilTimeLimit suppression, and even if they are, if they would really see
> consecutive
> idempotent updates for long enough to really have an observable impact on
> what gets emitted from the suppression buffer. In fact,
> > 1. Could you change "idempotent update operations will only be dropped
> > > from KTables, not from other classes." -> idempotent update operations
> > > will only be dropped from materialized KTables? For non-materialized
> > > KTables -- as they c
Hi all,
I might've made a minor mistake. The processor node level is level 3, not
level 1.
I will correct the KIP accordingly.
After looking over things, I decided to start the voting thread this
afternoon.
Cheers,
Richard
On Thu, Feb 27, 2020 at 12:29 PM Richard Yu
wrote:
> Hi Bruno,
Hi all,
I am proposing a new optimization to Kafka Streams which would greatly
reduce the number of idempotent updates (or no-ops) in the Kafka Streams
DAG.
A number of users have been interested in this feature, so it would be nice
to pass this one in.
For information, the KIP is described
ed-idempotent-updates" to be consistent with the
> > "dropped-records".
> >
> > Best,
> > Bruno
> >
> > On Tue, Mar 3, 2020 at 11:57 PM Richard Yu
> > wrote:
> > >
> > > Hi all,
> > >
> > > Thanks for the disc
Thanks for the KIP!
> >
> > I'm +1 (binding)
> >
> > -john
> >
> > On Thu, Feb 27, 2020, at 14:40, Richard Yu wrote:
> > > Hi all,
> > >
> > > I am proposing a new optimization to Kafka Streams which would
25, 2020 at 11:12 AM Richard Yu
wrote:
> Hi all,
>
> Thanks for all the discussion!
>
> @John and @Bruno I will survey other possible systems and see what I can
> do.
> Just a question, by systems, I suppose you would mean the pros and cons of
> different repor
; to add an opt-out config. Can you update the KIP to mention the exact
> > >>> config key and value(s) you'd propose?
> > >>>
> > >>> Just to move the discussion forward, maybe something like:
> > >>> emit.on := change|update
> &g
ses like
> "there has
> >>> been some discussion" or "possibly we could do X". For the final
> version of the
> >>> KIP, it should just say, "Streams will do X, Streams will do Y". Feel
> free to
> >>> add an elaboration section t
results of stateless
> operations, could you please add that those stateless operations are
> on KTables? IMO adding this information makes the KIP easier
> approachable by people that are not that familiar with the matter.
> Best,
> Bruno
>
> On Sat, Feb 1, 2020 at 11:33 PM R
__
> > > >>
> > > >> Hi Thomas and yuzhihong, That’s an interesting idea. Can you help
> > > >> think of a use case that isn’t also served by filtering or
> > > >> mapping beforehand? Thanks for helping to design this feature!
> > > >>
and it would be
> nice
> for them to know that this feature is saving them X-thousand updates per
> second,
> etc.
>
> What does everyone think about this? Note, as I read it, what I've said
> above is
> already reflected in the text of the KIP.
>
> Thanks,
> -John
f
> > > > idempotent updates if the aggregate is updated with the same key,
> > > > value, AND timestamp. I am also fine if we do not include this into
> > > > this KIP (remember: baby steps).
> > > >
> > > > You write that "emit-on-change
of the number of records
dropped.
I hope that this is more on point.
Best,
Richard
On Fri, Feb 21, 2020 at 2:20 PM Richard Yu
wrote:
> Hi all,
>
> Thanks for the clarification. I was just confused a little on what was
> going on.
>
> So I guess then that for the actua
), but the underlying values of the result might be the same.
(therefore equals() might return true)
Do you think this would be plausible?
Cheers,
Richard
On Fri, Feb 21, 2020 at 2:37 PM Richard Yu
wrote:
> Hello,
>
> Just to make some updates. I changed the name of the metric so that it w
add an option to send `oldValues` instead of materialization (this
> > would at least save the store overhead). As we consider the KIP an
> > optimization, a "config" seems to make sense.
> >
> >
> > -Matthias
> >
> >
> > On 2/17/20 5:21 PM, Ri
Hi everybody!
I'd like to propose a change that we probably should've added for a long
time now.
The key benefit of this KIP would be reduced traffic in Kafka Streams since
a lot of no-op results would no longer be sent downstream.
Here is the KIP for reference.
Hi Jun,
Can the SSL username really include the comma?
>From what I could tell, when I searched it up, I couldn't find anything
that indicated comma can be a delimiter.
A related doc below:
https://knowledge.digicert.com/solution/SO12401.html
Cheers,
Richard
On Thu, Jan 16, 2020 at 1:37 PM
Hi all,
I have decided to pass this KIP with 2 binding votes and 3 non-binding
votes (including mine).
I will update KIP status shortly after this.
Best,
Richard
On Thu, Mar 5, 2020 at 3:45 PM Richard Yu
wrote:
> Hi all,
>
> Just polling for some last changes on the name.
> I thin
is case, because it's computed as maximum over
> all input record for the window).
>
>
> (3) The discussion about stream time is very interesting. I agree that
> it's an orthogonal concern to this KIP.
>
>
>
> - -Matthias
>
>
> On 3/6/20 1:52 PM, Richard Yu wrote:
Hi all,
This reminds me of a previous issue I think that we were discussing.
@John Roesler I think you should remember this one.
A while back, we were talking about having suppress operator emit records
by wall-clock time instead of stream time.
If we are adding this, wouldn't that make it more
Richard Yu created KAFKA-7127:
-
Summary: Add asynchronous support for methods in KafkaConsumer
Key: KAFKA-7127
URL: https://issues.apache.org/jira/browse/KAFKA-7127
Project: Kafka
Issue Type
Richard Yu created KAFKA-6583:
-
Summary: Metadata should include number of state stores for task
Key: KAFKA-6583
URL: https://issues.apache.org/jira/browse/KAFKA-6583
Project: Kafka
Issue Type
Richard Yu created KAFKA-7132:
-
Summary: Consider adding multithreaded form of recovery
Key: KAFKA-7132
URL: https://issues.apache.org/jira/browse/KAFKA-7132
Project: Kafka
Issue Type
Richard Yu created KAFKA-6608:
-
Summary: Add timeout parameter to methods which fetches and resets
offsets
Key: KAFKA-6608
URL: https://issues.apache.org/jira/browse/KAFKA-6608
Project: Kafka
Richard Yu created KAFKA-7118:
-
Summary: Currently, it was discovered that KafkaConsumer's close()
method might not be multi-thread safe when multiple cores are calling the same
consumer.
Key: KAFKA-7118
URL: https
Richard Yu created KAFKA-7850:
-
Summary: Remove deprecated KStreamTestDriver
Key: KAFKA-7850
URL: https://issues.apache.org/jira/browse/KAFKA-7850
Project: Kafka
Issue Type: Improvement
Richard Yu created KAFKA-8020:
-
Summary: Consider making ThreadCache a time-aware LRU Cache
Key: KAFKA-8020
URL: https://issues.apache.org/jira/browse/KAFKA-8020
Project: Kafka
Issue Type
Richard Yu created KAFKA-8438:
-
Summary: Add API to allow user to define end behavior of consumer
failure
Key: KAFKA-8438
URL: https://issues.apache.org/jira/browse/KAFKA-8438
Project: Kafka
Richard Yu created KAFKA-8431:
-
Summary: Add a onTimeoutExpired callback to Kafka Consumer
Key: KAFKA-8431
URL: https://issues.apache.org/jira/browse/KAFKA-8431
Project: Kafka
Issue Type
Richard Yu created KAFKA-8434:
-
Summary: Make global stream time consistent over all stream tasks
Key: KAFKA-8434
URL: https://issues.apache.org/jira/browse/KAFKA-8434
Project: Kafka
Issue Type
[
https://issues.apache.org/jira/browse/KAFKA-8434?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Richard Yu resolved KAFKA-8434.
---
Resolution: Fixed
> Make global stream time consistent over all stream ta
Richard Yu created KAFKA-8516:
-
Summary: Consider allowing all replicas to have read/write
permissions
Key: KAFKA-8516
URL: https://issues.apache.org/jira/browse/KAFKA-8516
Project: Kafka
Issue
Richard Yu created KAFKA-8388:
-
Summary: Add methods to query for entries in KTable using timestamp
Key: KAFKA-8388
URL: https://issues.apache.org/jira/browse/KAFKA-8388
Project: Kafka
Issue
Richard Yu created KAFKA-9285:
-
Summary: Implement failed message topic to account for processing
lag during failure
Key: KAFKA-9285
URL: https://issues.apache.org/jira/browse/KAFKA-9285
Project: Kafka
[
https://issues.apache.org/jira/browse/KAFKA-9285?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Richard Yu resolved KAFKA-9285.
---
Resolution: Fixed
Already resolved by Kafka Connect.
> Implement failed message topic to acco
Richard Yu created KAFKA-9808:
-
Summary: Refactor State Store Hierarchy
Key: KAFKA-9808
URL: https://issues.apache.org/jira/browse/KAFKA-9808
Project: Kafka
Issue Type: Improvement
Richard Yu created KAFKA-9733:
-
Summary: Consider addition to Kafka's replication model
Key: KAFKA-9733
URL: https://issues.apache.org/jira/browse/KAFKA-9733
Project: Kafka
Issue Type: New
101 - 148 of 148 matches
Mail list logo
