Thanks Mike, very helpful to have all that context. I'm in agreement with
everything you've said. Accepting duplicates may be a tradeoff we accept
to keep performance high.
Your comments are centered around Kafka but how does this apply to other
writers? Since we're now handling multiple messag
Totally on board with everybody's comments above this point.
Jon
On Fri, Jan 18, 2019, 6:07 PM Michael Miklavcic
wrote:
> Thanks for the write up, Ryan. I had to touch on some of this when
> refactoring the kafka writer away from the async model so we could
> guarantee delivery. We had potentia
Thanks for the write up, Ryan. I had to touch on some of this when
refactoring the kafka writer away from the async model so we could
guarantee delivery. We had potential to drop messages before that change
because of the async producer calls, which would ack the Storm tuple as
soon as the writer r
Agreed
On January 18, 2019 at 14:52:32, Ryan Merriman (merrim...@gmail.com) wrote:
I am on board with that. In that case, I think it's even more important
that we get the Writer interfaces right.
On Fri, Jan 18, 2019 at 1:34 PM Otto Fowler
wrote:
> I think that the writers should be loaded as
I am on board with that. In that case, I think it's even more important
that we get the Writer interfaces right.
On Fri, Jan 18, 2019 at 1:34 PM Otto Fowler wrote:
> I think that the writers should be loaded as, and act as extension points,
> such that it is possible to have 3rd party writers,
I think that the writers should be loaded as, and act as extension points,
such that it is possible to have 3rd party writers, and would structure
them as such.
On January 18, 2019 at 13:55:00, Ryan Merriman (merrim...@gmail.com) wrote:
Recently there was a bug reported by a user where a parser