I should specify that the types are all open already, but there is still a question of whether there should be closed fields or not and what they should be. Steven
On Monday, December 14, 2015, Till Westmann <[email protected]> wrote: > Let me prefix this with that statement that I haven’t looked into our > metadata storage implementation. > But looking at the relatively frequent changes to the metadata format it > seems that we should either > a) make the types for metadata records open or > b) be very careful in designing the next (and future) revision(s) of it. > I don’t know why the metadata storage was done the way it is now and what > the considerations were when we did that. > > Does anybody remember those (or is there even a document that contains the > design and rationale)? > > Thanks, > Till > > On 14 Dec 2015, at 11:35, Steven Jacobs wrote: > > It could easily be done as reverse-compatible, but my thinking was that >> this is the "wrong" choice. I can easily make the datatype dataverse an >> open field for the metadata. The question is, why do we have open vs >> closed >> fields for metadata at all? If it is okay for them to be open, should we >> get rid of the schema entirely? if it's not okay, then shouldn't this >> field >> be closed? From a design standpoint it seems that if reverse compatibility >> were not an issue than the field should be closed. >> Steven >> >> On Mon, Dec 14, 2015 at 11:30 AM, Ildar Absalyamov < >> [email protected]> wrote: >> >> If fix for 2) will break the backwards-compatibility 1) might do that as >>> well and be submitted together. >>> >>> Now 2) was a long overdue problem, I don’t think there is any reason even >>> to try make changes backwards-compatible, because it was broken in the >>> first place. >>> >>> On Dec 14, 2015, at 11:16, Steven Jacobs <[email protected]> wrote: >>>> >>>> It's a new attribute, but it's a closed field, which means it isn't >>>> backwards compatible. >>>> Steven >>>> >>>> On Mon, Dec 14, 2015 at 11:13 AM, Ian Maxon <[email protected]> wrote: >>>> >>>> For 1), I guess the question is whether it would be a backwards >>>>> compatible change. Since it's just a new attribute (right?...), and it >>>>> is also sort of a new feature rather than a fix for something that was >>>>> critically broken, I would tend toward putting it on master. If it's >>>>> not backwards compatible though maybe it needs more careful >>>>> consideration. >>>>> >>>>> -Ian >>>>> >>>>> On Mon, Dec 14, 2015 at 10:12 AM, Steven Jacobs <[email protected]> >>>>> >>>> wrote: >>> >>>> Hi all, >>>>>> I'm implementing a change so that datasets can use datatypes from >>>>>> >>>>> alternate >>>>> >>>>>> data verses (previously the type and set had to be from the same >>>>>> dataverse). Unfortunately this means another change for Dataset >>>>>> >>>>> Metadata >>> >>>> (which will now store the dataverse for its type). >>>>>> >>>>>> As such, I had a couple of questions: >>>>>> >>>>>> 1) Should this change be thrown into the release branch, as it is >>>>>> >>>>> another >>> >>>> Metadata change? >>>>>> >>>>>> 2) In implementing this change, I've been looking at the Metadata >>>>>> >>>>> secondary >>>>> >>>>>> indexes. I had a discussion with Ildar, and it seems the thread on >>>>>> >>>>> Metadata >>>>> >>>>>> secondary indexes being "hacked" has been lost. Is this also something >>>>>> >>>>> that >>>>> >>>>>> should get into the release? Is there anyone currently looking at it? >>>>>> >>>>>> Steven >>>>>> >>>>> >>>>> >>> Best regards, >>> Ildar >>> >>> >>>
