This looks useful, I think the only nit I would pick would be to name the
MetricsReporter method contextChanged (past tense), which seems more
conventional for methods like this.
On Tue, 2020-05-05 at 16:58 -0700, Xavier Léauté wrote:
[EXTERNAL EMAIL] Attention: This email was sent from
Bumping to get and get some attention on this KIP before initiating a vote.
Using ConsumerInterceptor for its intended purpose quite difficult without this.
On Mon, 2020-02-10 at 15:50 +, Thomas Becker wrote:
[EXTERNAL EMAIL] Attention: This email was sent from outside TiVo. DO NOT CLICK
Bumping this again for visibility. If no one has any comments, maybe I'll just
start the VOTE thread?
On Wed, 2020-01-29 at 22:24 +, Thomas Becker wrote:
[EXTERNAL EMAIL] Attention: This email was sent from outside TiVo. DO NOT CLICK
any links or attachments unless you expected them
our point, support seems to be lacking). I feel like there
would be other cases where this feature could be valuable, but I admit I can't
come up with anything right this second. Perhaps yuzhihong had an example in
mind?
I.e., simple things should be easy, and complex things should be possible.
What
of a use case that isn’t also
served by filtering or mapping beforehand?
Thanks for helping to design this feature!
-John
On Fri, Jan 31, 2020, at 18:56, yuzhih...@gmail.com<mailto:yuzhih...@gmail.com>
wrote:
I think this is good idea.
On Jan 31, 2020, at 4:49 PM, Thomas
How do folks feel about allowing the mechanism by which no-ops are detected to
be pluggable? Meaning use something like a hash by default, but you could
optionally provide an implementation of something to use instead, like a
ChangeDetector. This could be useful for example to ignore changes to
from outside TiVo. DO NOT CLICK
any links or attachments unless you expected them.
Hey Thomas,
On Thu, 23 Jan 2020 at 21:17, Thomas Becker wrote:
> Hi folks,
> I'd like to open the discussion for KIP-566: Add rebalance callbacks to
> ConsumerIntercept
Hi folks,
I'd like to open the discussion for KIP-566: Add rebalance callbacks to
ConsumerInterceptor. We've been looking to implement some custom metrics via
ConsumerInterceptor, and not knowing when partition ownership changes is a
significant impediment. I'd appreciate your thoughts.
I'd like permission to create a KIP please. My confluence account is twbecker.
This email and any attachments may contain confidential and privileged material
for the sole use of the intended recipient. Any review, copying, or
distribution of this email (or
ficient and that can be more synchronous, but still presents
the same API as a broker. This would be a ton of work to design an build,
though, which I assume is why no one has done it.
Thanks,
-John
On Fri, Dec 6, 2019, at 12:21, Thomas Becker wrote:
>
> Personally, I would love to see
Personally, I would love to see EmbeddedKafkaCluster moved to a public test
artifact, similarly to kafka-streams-test-utils. Having to copy/paste internal
classes into your project is...unfortunate.
On Fri, 2019-12-06 at 10:42 -0600, John Roesler wrote:
[EXTERNAL EMAIL] Attention: This email
+1 non-binding
We've run into issues trying to decorate the AdminClient due it being an
abstract class. Will be nice to have consistency with Producer/Consumer as well.
On Tue, 2019-06-04 at 17:17 +0100, Andy Coates wrote:
Hi folks
As there's been no chatter on this KIP I'm assuming it's
Yes, I think this type of strategy interface would be valuable.
On Wed, 2019-01-16 at 15:41 +, Jan Filipiak wrote:
On 16.01.2019 14:05, Thomas Becker wrote:
I'm going to bow out of this discussion since it's been made clear that
the feature is not targeted at streams. But for the record
I'm going to bow out of this discussion since it's been made clear that the
feature is not targeted at streams. But for the record, my desire is to have an
alternative to the timestamp based message choosing strategy streams currently
imposes, and I thought topic prioritization in the consumer
ity
topic partitions are at the HW before I can decide if I want to poll the lower
priority ones. Right?
On Fri, 2018-10-05 at 11:34 -0700, Colin McCabe wrote:
On Fri, Oct 5, 2018, at 10:58, Thomas Becker wrote:
Colin,
Would you mind sharing your vision for how this looks with multiple
consum
Colin,
Would you mind sharing your vision for how this looks with multiple consumers?
I'm still getting my bearings with the new consumer but it's not immediately
obvious to me how this would work. In particular, it doesn't seem particularly
easy to know when you are at the high watermark of a
, period. Trying
to force a temporal relationship between that and an event where the item was
viewed is non-sensical.
On Mon, 2018-09-17 at 18:18 +, Thomas Becker wrote:
Hi Matthias,
I'm familiar with how the timestamp synchronization currently works. I also
submit that it does not work
w---is
semantically incorrect.
Shameless plug: you might want to read
https://www.confluent.io/blog/streams-tables-two-sides-same-coin
-Matthias
On 9/17/18 8:23 AM, Thomas Becker wrote:
For my part, a major use-case for this feature is stream-table joins.
Currently, KafkaStreams does the wro
For my part, a major use-case for this feature is stream-table joins.
Currently, KafkaStreams does the wrong thing in some cases because the only
message choosing strategy available is timestamp-based. So table records that
have been updated recently will not be read until the stream records
I agree with Jan. A strategy interface for choosing processing order is nice,
and would hopefully be a step towards getting this in streams.
-Tommy
On Mon, 2018-08-20 at 12:52 +0200, Jan Filipiak wrote:
On 20.08.2018 00:19, Matthias J. Sax wrote:
@Nick: A KIP is only accepted if it got 3
Guozhang
On Tue, Aug 7, 2018 at 9:59 AM, Thomas Becker
mailto:thomas.bec...@tivo.com>>
wrote:
Thanks Guozhang. So in the scenario you describe, where one topic has
vastly lower throughput, you're saying that when the lower throughput topic
is fully caught up (no messages in the
elf should be discussed as a separate KIP, maybe for
both Streams and Consumer clients, and hence I intentionally avoid
overlapping with it and stays with a static messaging choosing mechanism in
my KIP.
Guozhang
On Tue, Aug 7, 2018 at 4:55 AM, Thomas Becker
mailto:thomas.bec...@tivo.com>>
wro
+1 (non-binding)
We've hit issues with the log cleaner in the past, and this would be a great
improvement.
On Tue, 2018-08-07 at 12:19 +0100, Stanislav Kozlovski wrote:
Hey everybody,
I'm starting a vote on KIP-346
This looks like a big step in the right direction IMO. So am I correct in
assuming this idle period would only come into play after startup when waiting
for initial records to be fetched? In other words, once we have seen records
from all topics and have established the stream time processing
Personally, I like the idea of builders for the producer/consumer themselves,
but I'm less enthusiastic about one for ProducerRecord. Mostly because I think
the following is overly verbose/reads poorly:
producer.send(ProducerRecord.builder()
.topic("mytopic")
.key("Key")
`-repartition` or
`-changelog`.
Thus, from my point of view, it would make sense to keep the current
distinction.
-Matthias
On 11/6/17 4:45 PM, Thomas Becker wrote:
I think this sounds good as well. It's worth clarifying whether topics that are
named by the user but created by streams
I think this sounds good as well. It's worth clarifying whether topics that are
named by the user but created by streams are considered "internal" topics also.
On Sun, 2017-11-05 at 23:02 +0100, Matthias J. Sax wrote:
My idea was, to relax the requirement for through() that a topic must be
I think it would be helpful to clarify what happens if consumers rejoin an
empty group. I would presume that the expiration timer is stopped and reset
back to offsets.retention.minutes when it is empty again but the KIP doesn't
say.
On Wed, 2017-10-18 at 16:45 -0700, Vahid S Hashemian wrote:
ther topic and then constructing the GlobalKTable from the latter?
The GlobalKTable has the limitations you mention since it was primarily
designed for joins only. We should consider allowing a less restrictive
interface if it makes sense.
Eno
> On 25 May 2017, at 14:48, Thomas Becker <tobec...@tivo.com>
We need to do a series of joins against a KTable that we can't co-
partition with the stream, so we're looking at GlobalKTable. But the
topic backing the table is not ideally keyed for the sort of lookups
this particular processor needs to do. Unfortunately, GlobalKTable is
very limited in that
+1
On Wed, 2017-05-10 at 10:52 +0100, Michal Borowiecki wrote:
> Hi all,
>
> This vote thread has gone quiet.
>
> In view of the looming cut-off for 0.11.0.0 I'd like to encourage
> anyone
> who cares about this to have a look and vote and/or comment on this
> proposal.
>
> Thanks,
>
> Michał
>
>
I'm experimenting with a streams application that does a KStream-
GlobalKTable join, and I'm seeing some unexpected behavior when re-
running the application. First, it does not appear that the offsets in
the topic backing the GlobalKTable are being checkpointed to a file as
I expected. This
We have had a number of situations where we need to migrate data in a
Kafka topic to a new topic that is keyed differently. Stream processing
is a good fit for this use-case with one exception: there is no easy
way to know when your "migration job" is finished. Has any thought been
given to adding
+1 (non-binding)
On Tue, 2017-02-28 at 08:59 +, Jeyhun Karimov wrote:
> Dear community,
>
> I'd like to start the vote for KIP-123:
> https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=6871
> 4788
>
>
> Cheers,
> Jeyhun
--
Tommy Becker
Senior Software Engineer
O
uggest
>
> enum PunctuationType {
> EVENT_TIME,
> SYSTEM_TIME,
> }
>
> or similar. Just to keep the door open -- it's easier to add new
> stuff
> if the name is more generic.
>
>
> -Matthias
>
>
> On 4/4/17 5:30 AM, Thomas Becker wrote:
> >
> > I
ctual use case where you might need
> > anything
> > else then those two). Hence I also proposed the option to allow
> > users
> > to, effectively, decide what "stream time" is for them given the
> > presence or absence of messages, much like they can decide what msg
Although I fully agree we need a way to trigger periodic processing
that is independent from whether and when messages arrive, I'm not sure
I like the idea of changing the existing semantics across the board.
What if we added an additional callback to Processor that can be
scheduled similarly to
We ran into an incident a while back where one of our broker machines
abruptly went down (AWS is fun). While the leadership transitions and
so forth seemed to work correctly with the remaining brokers, our
producers hung shortly thereafter. I should point out that we are using
the old Scala
38 matches
Mail list logo
