+1
Till
On 20 Nov 2017, at 12:05, Mike Carey wrote:
+1 to remove them and then re-create them later - based on the state
of the AsterixDB storage world and cluster dynamics at that time. (I
think we'll have a better chance of getting them perfect if we re-do
them then - I don't remember that it took Yingyi very long to do them
the first time, so I think the re-do path will beat the fix-up path if
we want them again.) As far as I know, since we don't document them,
nobody is using them - and I think the engineering cost of maintaining
orphaned code is too high (not worth it).
Any thoughts to the contrary?
Cheers,
Mike
On 11/20/17 10:52 AM, Murtadha Hubail wrote:
Hi all,
As you might be aware, we have a feature in AsterixDB to create
temporary datasets that differ from regular datasets in some ways
such as:
Their existence is not persisted in metadata, but only in the CC
metadata cache.
They don’t’ generate any transaction logs
Their files are deleted on NC restart.
If they are not accessed for some period of time, their metadata
records are removed from the CC metadata cache.
Temporary datasets were originally introduced to serve as a staging
area between AsterixDB and external systems such as Perglix. However,
as the system evolved over the years, the assumptions they were built
on don’t hold anymore and they could lead to undesired consequences
such as leaking files after a CC restart or inability to access the
dataset files on a restarted NC. Therefore, I’m proposing to remove
the support for the current temporary datasets and we may add the
feature with a careful design at a later stage.
Any thoughts or concerns on removing temporary datasets?
Cheers,
Murtadha
