Let me prefix this with that statement that I haven’t looked into our
metadata storage implementation.
But looking at the relatively frequent changes to the metadata format it
seems that we should either
a) make the types for metadata records open or
b) be very careful in designing the next (and future) revision(s) of it.
I don’t know why the metadata storage was done the way it is now and
what the considerations were when we did that.
Does anybody remember those (or is there even a document that contains
the design and rationale)?
Thanks,
Till
On 14 Dec 2015, at 11:35, Steven Jacobs wrote:
It could easily be done as reverse-compatible, but my thinking was
that
this is the "wrong" choice. I can easily make the datatype dataverse
an
open field for the metadata. The question is, why do we have open vs
closed
fields for metadata at all? If it is okay for them to be open, should
we
get rid of the schema entirely? if it's not okay, then shouldn't this
field
be closed? From a design standpoint it seems that if reverse
compatibility
were not an issue than the field should be closed.
Steven
On Mon, Dec 14, 2015 at 11:30 AM, Ildar Absalyamov <
[email protected]> wrote:
If fix for 2) will break the backwards-compatibility 1) might do that
as
well and be submitted together.
Now 2) was a long overdue problem, I don’t think there is any
reason even
to try make changes backwards-compatible, because it was broken in
the
first place.
On Dec 14, 2015, at 11:16, Steven Jacobs <[email protected]> wrote:
It's a new attribute, but it's a closed field, which means it isn't
backwards compatible.
Steven
On Mon, Dec 14, 2015 at 11:13 AM, Ian Maxon <[email protected]> wrote:
For 1), I guess the question is whether it would be a backwards
compatible change. Since it's just a new attribute (right?...), and
it
is also sort of a new feature rather than a fix for something that
was
critically broken, I would tend toward putting it on master. If
it's
not backwards compatible though maybe it needs more careful
consideration.
-Ian
On Mon, Dec 14, 2015 at 10:12 AM, Steven Jacobs <[email protected]>
wrote:
Hi all,
I'm implementing a change so that datasets can use datatypes from
alternate
data verses (previously the type and set had to be from the same
dataverse). Unfortunately this means another change for Dataset
Metadata
(which will now store the dataverse for its type).
As such, I had a couple of questions:
1) Should this change be thrown into the release branch, as it is
another
Metadata change?
2) In implementing this change, I've been looking at the Metadata
secondary
indexes. I had a discussion with Ildar, and it seems the thread on
Metadata
secondary indexes being "hacked" has been lost. Is this also
something
that
should get into the release? Is there anyone currently looking at
it?
Steven
Best regards,
Ildar
