Hi Guozhang,
KIP looks great, I have one suggestion: in addition to
"restore-records-total", it would also be useful to track the number of
records *remaining*, that is, the records that have not yet been restored.
This is actually the metric I was attempting to implement in the
StateRestoreListen
her before,
after or inside the "recursively" method. I can't see a scenario where the
recursion would cause problems with it.
Nick
On Tue, 6 Sept 2022 at 18:08, Nick Telford wrote:
> Hi Guozhang,
>
> I mentioned this in the "Rejected Alternatives" section. Reparti
vides no further optimizations. Is my understanding correct? If yes,
> I'm wondering if it's worthy the complexity to allow repartitioning inside
> the unary operator, or should we just restrict the recursion inside a
> single sub-topology.
>
>
> Guozhang
>
> On Tue, S
itioning" in my head, as I
don't think it's quite right. It may turn out that the additional overload
(with the Produced argument) is not necessary.
Thanks for all your feedback so far. Let me know what you think!
Regards,
Nick
On Thu, 25 Aug 2022 at 17:46, Nick Telford wrote:
> In our current examples, the linked output stream and input stream are on
> > the same sub-topology, in which case this API allows us to avoid creating
> > unnecessary intermediate topics; when the linked output/input streams are
> > not on the same sub-topology, then using this AP
hen you say that it's
> > an alternative to something already existing. My suggestion would be to
> > still modify the KIP around the new API, highlight how it's an
> alternative
> > to something already existing and why we should add the new API. You have
> >
emphasizing the new functionality and algorithms that
they enable.
What additional context would you like to see in the KIP? Some more
examples of recursive algorithms that would benefit? A more concrete
example than generic graph traversal? Something else?
Regards,
Nick Telford
On Fri, 5 Aug 2022
Hi Alex,
Excellent proposal, I'm very keen to see this land!
Would it be useful to permit configuring the type of store used for
uncommitted offsets on a store-by-store basis? This way, users could choose
whether to use, e.g. an in-memory store or RocksDB, potentially reducing
the overheads assoc
of recursively counting nodes in a graph.
I haven't included changes for the PAPI, mostly because I don't use it, so
I'm not as familiar with the idioms there. If you can think of a good
analogue for a new PAPI method, I'm happy to include it in the KIP.
Regards,
Nick Telford
m/a/48850722
>
> Cheers,
>
> Chris
>
> On Mon, Mar 28, 2022 at 11:46 PM Matthias J. Sax wrote:
>
> > I think Java does not allow to have both overloads, because it would
> > result in ambiguity?
> >
> > If you call `s1.merge(s2)` it's unclear whi
per method themselves if they have such a pattern?
>
>
> -Matthias
>
>
>
> On 1/31/22 7:35 AM, Nick Telford wrote:
> > Hi everyone,
> >
> > I'd like to discuss KIP 819:
> >
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-819%3A+Merge+mu
/__{topo_name}__/1_0/rocksdb/mystore
> > {app_id}/__{topo_name}__/1_3/rocksdb/mystore
> >
> > Which is pretty convenient because it means there are no
> > implications for your proposal. If you implement the above
> > code, then we'll just wind up with:
> >
at 8:59 AM John Roesler wrote:
> >
> >> Thanks, Nick,
> >>
> >> It sounds like we're on the same page. I didn't think (A)
> >> would be fundamentally "hard", just that it might be a pain
> >> in practice. Reading your response
easy to go around as long as users reuse
> > the same appId, 3) can be tackled with the help of the named topology but
> > each named topology can still be composed of multiple sub-topologies so
> > extra work is still needed to align the sub-topology ids, but we still
> need
>
angelog topics within the sub-topology
> > * used repartition topics
> > * etc
> >
> > So let's assume in the long run we can indeed automatically determine if
> a
> > topology or part of it (a sub-topology) is structurally the same, what we
> > can do
Hi everyone,
I'd like to discuss KIP 819:
https://cwiki.apache.org/confluence/display/KAFKA/KIP-819%3A+Merge+multiple+KStreams+in+one+operation
This is a simple KIP that adds/modifies the KStream#merge API to enable
many streams to be merged in a single graph node.
Regards,
Nick Telford
ikely have
unintended consequences. C is also the least complete solution, and will
need significant additional work to make it work.
Please let me know if the Motivation and Background sections need more
clarity.
Regards,
Nick Telford
n Fri, Jan 14, 2022, at 07:40, Bruno Cadonna wrote:
> > > >> Hi Nick,
> > > >>
> > > >> Since the title of the KIP slightly changed after the vote was
> opened
> > > >> also the link to the KIP changed as a result. This is should be a
>
, I leave it up to you if you want to change it in this case or
> not. I am fine either way.
>
> Best,
> Bruno
>
> On 13.01.22 10:00, Nick Telford wrote:
> > Hi Matthias,
> >
> > You raise a good point: commit.interval.ms only specifies a minimum
> > i
st to strip the `min.` prefix.
>
>
> For the config description, I would remove
>
> > The default value is the same as the default for commit.interval.ms
> (3).
>
> It seems unnecessary to me.
>
>
> -Matthias
>
>
>
> On 1/12/22 7:21 AM, Nick Telfo
Hi everyone,
I'd like to call a vote to adopt KIP-811: Add config
min.repartition.purge.interval.ms to Kafka Streams
<https://cwiki.apache.org/confluence/display/KAFKA/KIP-811%3A+Add+config+min.repartition.purge.interval.ms+to+Kafka+Streams>
.
Regards
Nick Telford
Thanks Bruno, I've made those changes.
I'll call a vote on the KIP later today.
Regards,
Nick Telford
On Wed, 12 Jan 2022 at 12:13, Bruno Cadonna wrote:
> Hi Nick,
>
> Great!
>
> I think the KIP is ready for voting. I have just a couple of minor
> comments.
>
&
ly_once_v2)."
This should make it clear that this is just a minimum interval, without
leaking too much detail in to the specification.
If there are no other issues, I'll put this to a vote.
Regards,
Nick Telford
On Tue, 11 Jan 2022 at 15:34, Bruno Cadonna wrote:
> Hi Nick,
automatic commits.
Good point. I'm holding off on amending this for now until we agree on an
outcome for point 7 above, because I suspect it may at least influence the
wording of this section.
Thanks for the feedback, and sorry for the delay. Now that the holidays are
over I'm able to foc
; >> config type (presumably a Long), the doc
> > >> string, and the importance (probably "low", similar to
> > >> commit.interval.ms
> > >> )
> > >> 2. Might be worth considering a slightly more descriptive name for
> &
This is a KIP for a proposed solution to KAFKA-13549
<https://issues.apache.org/jira/browse/KAFKA-13549>. The solution is very
simple, so the KIP is pretty short.
The suggested changes are implemented by this PR
<https://github.com/apache/kafka/pull/11610>.
--
Nick Telford
Hi,
I would like permission to contribute to Apache Kafka.
Wiki user: nick.telford
JIRA: nicktelford
Regards,
Nick Telford
101 - 127 of 127 matches
Mail list logo
