I believe Ildar is just looking for a way to ensure, in doing
experiments, that things are all in disk components. His
stats-gathering extensions camp on the LSM lifecycle - flushes in
particular - and he wants to finish that process in his testing and
experiments. Wail's schema inference stuff has a similar flavor. So
the goal is to flush any lingering memory components to disk for a given
dataset at the end of the "experiment lifecycle".
We have DDL to compact a dataset - which flushes AND compacts - it might
also be useful to have DDL to flush a dataset without also forcing
compaction - as a way for an administrator to release that dataset's
in-memory component related resources. (Not that it's "necessary" for
any correctness reason - just might be nice to be able to do that. That
could also be useful in scripting more user-level-oriented recovery tests.)
Thus, I'd likely vote for adding a harmless new DDL statement - another
arm of the one that supports compaction - for this.
Cheers,
Mike
On 1/21/17 6:21 AM, Till Westmann wrote:
Hi Ildar,
On 19 Jan 2017, at 4:02, Ildar Absalyamov wrote:
Since I was out for quite a while and a lot of things happened in a
meantime in a codebase I wanted to clarify couple of things.
I was wondering if there is any legitimate way to force the data of
in-memory components to be flushed, other then stop the whole instance?
It used to be that choosing a different default dataverse with “use”
statement did that trick, but that is not the case anymore.
Just wondering, why do you want to flush the in-memory components to
disk?
Another question is regarding CC<->NC & NC<->NC messaging. Does the
sender get some kind of ACK that the message was received by the
addressee? Say if I send a message just before the instance shutdown
will the shutdown hook wait until the message is delivered and
processed?
I agree with Murtadha, that I can certainly be done. However, we also
need to assume that some shutdowns won’t be clean and so the messages
might not be received. So it might be easier to just be able to
recover from missing messages than to be able to recover *and* to
synchronize on shutdown. Just a thought - maybe that’s not even an
issue for your use-case.
Cheers,
Till
