It's only TEMPORARY... :-)
On 11/20/17 12:41 PM, Ian Maxon wrote:
Are we going to preserve the keyword?
On Mon, Nov 20, 2017 at 12:06 PM, Michael Blow <[email protected]> wrote:
removing temporary datasets?
+1
On Mon, Nov 20, 2017 at 1:52 PM Murtadha Hubail <[email protected]> wrote:
Hi all,
As you might be aware, we have a feature in AsterixDB to create temporary
datasets that differ from regular datasets in some ways such as:
Their existence is not persisted in metadata, but only in the CC metadata
cache.
They don’t’ generate any transaction logs
Their files are deleted on NC restart.
If they are not accessed for some period of time, their metadata records
are removed from the CC metadata cache.
Temporary datasets were originally introduced to serve as a staging area
between AsterixDB and external systems such as Perglix. However, as the
system evolved over the years, the assumptions they were built on don’t
hold anymore and they could lead to undesired consequences such as leaking
files after a CC restart or inability to access the dataset files on a
restarted NC. Therefore, I’m proposing to remove the support for the
current temporary datasets and we may add the feature with a careful design
at a later stage.
Any thoughts or concerns on removing temporary datasets?
Cheers,
Murtadha
