Yeah I think the angle makes sense to pursue. I don't feel strongly about
whether now or later is the right time to pursue it, but it does seem like
it's not the immediate highest priority.
On Tue, Aug 21, 2018 at 1:57 PM, Tianyi Wang wrote:
> GetCatalogDelta used to block catalogd from
GetCatalogDelta used to block catalogd from executing DDLs and the pending
struct was yet another cache to smooth things a little.
On Tue, Aug 21, 2018 at 11:28 AM Todd Lipcon wrote:
> One more parting thought: why don't we just call 'GetCatalogDelta()'
> directly from the catalog callback in
One more parting thought: why don't we just call 'GetCatalogDelta()'
directly from the catalog callback in order to do a direct handoff, instead
of storing them in this 'pending' struct? Given the statestore uses a
dedicated thread per subscriber (right?) it seems like it would be fine for
the
Thanks, Tim. I'm guessing once we switch over these RPCs to KRPC instead of
Thrift we'll alleviate some of the scalability issues and maybe we can look
into increasing frequency or doing a "push" to the statestore, etc. I
probably won't work on this in the near term to avoid complicating the
This is somewhat relevant for admission control too - I had thought about
some of these issues in that context, because reducing the latency of
admission controls state propagation helps avoid overadmission but having a
very low statestore frequency is very inefficient and doesn't scale well to
