Re: Metadata Dependencies

2016-10-13 Thread Steven Jacobs
For example, the "Channel" Metadata Dataset includes the following fields:
<dataverse, resultsDataset, subscriptionsDataset> (plus others)

When deleting a dataset, we need to know that "Channel" has a BLOCKING
dependency on dataset through both <dataverse, resultsDataset> and
<dataverse, subscriptionsDataset>
When deleting a dataverse, we need to know that "Channel" has a CHAINING
dependency on dataverse through 

Steven

On Thu, Oct 13, 2016 at 12:14 PM, Mike Carey <dtab...@gmail.com> wrote:

> Do we absolutely need that fine level of granularity?  (Maybe a few
> examples would help.)
>
>
>
> On 10/13/16 11:40 AM, Steven Jacobs wrote:
>
>> I lean towards B as well. The interesting question is how to succinctly
>> store the dependency field information in this catalog, i.e. I have these
>> fields which correspond to the primary key fields of this other metadata
>> dataset.
>> Steven
>>
>> On Wednesday, October 12, 2016, Mike Carey <dtab...@gmail.com> wrote:
>>
>> Got it.  IMO option B) is probably "better" in the sense that it seems
>>> "safer" and "more transparent".  With A) not much might be known about
>>> why
>>> those catalog entries are there, assuming a generic notion of "entity",
>>> and
>>> there's always the potential for a buggy extension to forget to delete a
>>> dependency and therefore force the non-deletion of a given dataverse.
>>> With
>>> B) you know what the issue is - the supposed existence of a metadata
>>> dataset - and you have the possible recourse of verifying the existence
>>> of
>>> such a dataset.  (E.g., if a buggy extension drops its dataset but
>>> forgets
>>> to update this registry, you can at least discover manually that the
>>> dataset is already gone.)  Maybe not a big deal, but
>>>
>>>
>>> On 10/12/16 3:11 PM, Steven Jacobs wrote:
>>>
>>> Yes, "catalog" would be the better word here. Thanks for the
>>>> clarification.
>>>>
>>>> Steven
>>>>
>>>> On Wed, Oct 12, 2016 at 12:38 PM, Mike Carey <dtab...@gmail.com> wrote:
>>>>
>>>> I assume that by "index" you really mean "catalog" (as opposed to index
>>>> in
>>>>
>>>>> the physical sense)?  I.e., your two proposals are about a possible
>>>>> additional metadata dataset in support of extension dependency
>>>>> management?
>>>>>
>>>>>
>>>>> On 10/12/16 9:47 AM, Steven Jacobs wrote:
>>>>>
>>>>> Hi,
>>>>>
>>>>>> I am trying to get feedback from the group in general about how best
>>>>>> to
>>>>>> add
>>>>>> Metadata Dependencies, particularly in the context of extensions. The
>>>>>> functionality that is needed is as follows:
>>>>>>
>>>>>> *When dropping a metadata entity A (e.g. a dataverse), first check
>>>>>> whether
>>>>>> A has any metadata entities that depend on it. Then allow these
>>>>>> entities
>>>>>> two options:*
>>>>>>
>>>>>> *1) block: don't drop A while this dependent entity exists*
>>>>>> *2) chain: when dropping A, drop this dependent entity as well.*
>>>>>>
>>>>>> One issue to keep in mind here is that with extensions, Asterix will
>>>>>> not
>>>>>> currently know about metadata datasets added by extensions.
>>>>>>
>>>>>> So far we have two proposals for this issue:
>>>>>>
>>>>>> A) Metadata dependency index. When you create an entity that depends
>>>>>> on
>>>>>> another, add an entry to this index. Check this index when dropping an
>>>>>> entity.
>>>>>>
>>>>>> B) Metadata data set index. All Metadata datasets would be registered
>>>>>> in
>>>>>> this index, and then we can query them as needed when dropping an
>>>>>> entity.
>>>>>> In this case we would need some way to specify which fields in these
>>>>>> datasets represent dependencies in order to query them properly.
>>>>>>
>>>>>> We would like any feedback on these two solutions or proposals for
>>>>>> alternate solutions.
>>>>>>
>>>>>> Steven
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>


Re: Metadata Dependencies

2016-10-12 Thread Steven Jacobs
Yes, "catalog" would be the better word here. Thanks for the clarification.

Steven

On Wed, Oct 12, 2016 at 12:38 PM, Mike Carey <dtab...@gmail.com> wrote:

> I assume that by "index" you really mean "catalog" (as opposed to index in
> the physical sense)?  I.e., your two proposals are about a possible
> additional metadata dataset in support of extension dependency management?
>
>
> On 10/12/16 9:47 AM, Steven Jacobs wrote:
>
>> Hi,
>> I am trying to get feedback from the group in general about how best to
>> add
>> Metadata Dependencies, particularly in the context of extensions. The
>> functionality that is needed is as follows:
>>
>> *When dropping a metadata entity A (e.g. a dataverse), first check whether
>> A has any metadata entities that depend on it. Then allow these entities
>> two options:*
>>
>> *1) block: don't drop A while this dependent entity exists*
>> *2) chain: when dropping A, drop this dependent entity as well.*
>>
>> One issue to keep in mind here is that with extensions, Asterix will not
>> currently know about metadata datasets added by extensions.
>>
>> So far we have two proposals for this issue:
>>
>> A) Metadata dependency index. When you create an entity that depends on
>> another, add an entry to this index. Check this index when dropping an
>> entity.
>>
>> B) Metadata data set index. All Metadata datasets would be registered in
>> this index, and then we can query them as needed when dropping an entity.
>> In this case we would need some way to specify which fields in these
>> datasets represent dependencies in order to query them properly.
>>
>> We would like any feedback on these two solutions or proposals for
>> alternate solutions.
>>
>> Steven
>>
>>
>