[jira] [Commented] (ATLAS-3546) isOptional for composition relationship category?
[ https://issues.apache.org/jira/browse/ATLAS-3546?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16998540#comment-16998540 ] Bolke de Bruin commented on ATLAS-3546: --- +1 on that last comment see also ATLAS-3254 > isOptional for composition relationship category? > - > > Key: ATLAS-3546 > URL: https://issues.apache.org/jira/browse/ATLAS-3546 > Project: Atlas > Issue Type: Improvement > Components: atlas-core >Affects Versions: 2.0.0 >Reporter: charles shen >Priority: Major > Attachments: json.jpg, xml.jpg, xml.jpg > > > I noticed since > [ATLAS-3051|https://issues.apache.org/jira/browse/ATLAS-3051], the > relationship attribute must be specified in the end def which is not > container and relationship category is composition. > I understand it's to prohibit orphan children but is it too strong? Reason > below: > # I have to provide all the entities along the relationship path in the > payload when creating a child, eg, for RDBMS, I have to provide > rdbms_instance, rdbms_db, rdbms_table, rdbms_column where I just want to > create a single rdbms_column, it brings performance overhead to check > existence of rdbms_instance, rdbms_db, etc..., > # I have defined a composition relationship type where each end is the same > entity type, it couldn't be created successfully anymore since it always > requires the mandatory attribute where it's the same type and then falls into > infinite loop. > Three possible fixes: > # Remove the isOptional constraint, since ownedRef/inverseRef doesn't have > such constraint. > # Add isOptional to relationship type end def. > # Add option in Rest to ignore the isOptional constraint for relationship > type. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (ATLAS-3546) isOptional for composition relationship category?
[ https://issues.apache.org/jira/browse/ATLAS-3546?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16998011#comment-16998011 ] charles shen commented on ATLAS-3546: - Also, it brings a side effect that load a parent will load all contained children(through ownedref), it's somehow conflict with [ATLAS-3056|https://issues.apache.org/jira/browse/ATLAS-3056], consider aws S3 as an example, there might be thousands of pseudo dirs under one bucket and thousands of S3 object under one pseudo dirs, it's a huge performance issue. > isOptional for composition relationship category? > - > > Key: ATLAS-3546 > URL: https://issues.apache.org/jira/browse/ATLAS-3546 > Project: Atlas > Issue Type: Improvement > Components: atlas-core >Affects Versions: 2.0.0 >Reporter: charles shen >Priority: Major > Attachments: json.jpg, xml.jpg, xml.jpg > > > I noticed since > [ATLAS-3051|https://issues.apache.org/jira/browse/ATLAS-3051], the > relationship attribute must be specified in the end def which is not > container and relationship category is composition. > I understand it's to prohibit orphan children but is it too strong? Reason > below: > # I have to provide all the entities along the relationship path in the > payload when creating a child, eg, for RDBMS, I have to provide > rdbms_instance, rdbms_db, rdbms_table, rdbms_column where I just want to > create a single rdbms_column, it brings performance overhead to check > existence of rdbms_instance, rdbms_db, etc..., > # I have defined a composition relationship type where each end is the same > entity type, it couldn't be created successfully anymore since it always > requires the mandatory attribute where it's the same type and then falls into > infinite loop. > Three possible fixes: > # Remove the isOptional constraint, since ownedRef/inverseRef doesn't have > such constraint. > # Add isOptional to relationship type end def. > # Add option in Rest to ignore the isOptional constraint for relationship > type. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (ATLAS-3546) isOptional for composition relationship category?
[ https://issues.apache.org/jira/browse/ATLAS-3546?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16989445#comment-16989445 ] charles shen commented on ATLAS-3546: - Thanks [~madhan]! For the first scenario, the case is that we couldn't guarantee the creation order so we would like to "create if not exist" for rdbms_db when creating rdbms_table, so we always include the full properties of rdbms_db as referred entities when crating each rdbms_table. I will take a look whether [ATLAS-3405|https://issues.apache.org/jira/browse/ATLAS-3405] is a good candidate to resolve this case. Thanks for the information. For the second scenario, I use two concrete examples below to explain in detail: # XML schema One xs:schema can contain several xs:element and one xs:element can contain a xs:sequence where a xs:sequence can contain several xs:element. Below is a simple graph to express this relationship. !xml.jpg! It's natural that when XML schema is deleted, all the XML elements under it should be deleted, when the XML element is deleted, all the XML elements under it should be deleted as well. To have this cascade deleting capability, I have to define composition relationship between xs:schema and xs:element and between xs:sequence and xs:element. But one specific xs:element should have one and only parent xs:schema or xs:sequence, but from the ATLAS composition relationship constraint, I have to provide both xs:schema and xs:sequence which is not valid. 2. Json Schema One Json document can contain several Json properties while one Json property can contain several Json properties. See below graph: !json.jpg! It's natural that when one Json property is deleted, all the Json properties under it should be deleted. To have this capability, I have to define composition relationship between jsonProperty and jsonProperty. Then it always requires the mandatory attribute where it's the same type and then falls into infinite loop. > isOptional for composition relationship category? > - > > Key: ATLAS-3546 > URL: https://issues.apache.org/jira/browse/ATLAS-3546 > Project: Atlas > Issue Type: Improvement > Components: atlas-core >Affects Versions: 2.0.0 >Reporter: charles shen >Priority: Major > Attachments: json.jpg, xml.jpg, xml.jpg > > > I noticed since > [ATLAS-3051|https://issues.apache.org/jira/browse/ATLAS-3051], the > relationship attribute must be specified in the end def which is not > container and relationship category is composition. > I understand it's to prohibit orphan children but is it too strong? Reason > below: > # I have to provide all the entities along the relationship path in the > payload when creating a child, eg, for RDBMS, I have to provide > rdbms_instance, rdbms_db, rdbms_table, rdbms_column where I just want to > create a single rdbms_column, it brings performance overhead to check > existence of rdbms_instance, rdbms_db, etc..., > # I have defined a composition relationship type where each end is the same > entity type, it couldn't be created successfully anymore since it always > requires the mandatory attribute where it's the same type and then falls into > infinite loop. > Three possible fixes: > # Remove the isOptional constraint, since ownedRef/inverseRef doesn't have > such constraint. > # Add isOptional to relationship type end def. > # Add option in Rest to ignore the isOptional constraint for relationship > type. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (ATLAS-3546) isOptional for composition relationship category?
[
https://issues.apache.org/jira/browse/ATLAS-3546?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16988422#comment-16988422
]
Madhan Neethiraj commented on ATLAS-3546:
-
[~api123] - about the first scenario: to create a single rdbms_column,
providing object-id of an existing rdbms_table entity should be enough; there
should be no need to provide rdbms_instance/rdbms_db/... For example:
{noformat}
{
"guid": "-1",
"typeName": "rdbms_column",
"attributes": {
"name": "id",
"qualifiedName": "sales.order_items.id@prod",
"data_type": "long",
"isPrimaryKey": false
},
"relationshipAttributes": {
"table": {
"relationshipType": "rdbms_table_columns",
"typeName": "rdbms_table",
"uniqueAttributes": {
"qualifiedName": "sales.order_items@prod"
}
}
}
}
{noformat}
For the second scenario, perhaps you can consider modeling this as an
aggregation. Composition, by definition needs a container entity to be present
for a contained entity to exist. If this constraint can't be met, then perhaps
the relationship shouldn't be modeled as composition.
You might also want to consider the enhancement added in ATLAS-3405, which will
create a shell/empty entity for non-existing object-references. In your
example, if rdbms_table with {{qualifiedName=sales.order_items@prod doesn't
exist}}, Atlas will create an shell entity for the rdbms_table (with only
qualifiedName attribute populated), and create rdbms_column entity with link to
the created table entity. The table entity can be populated later in a
subsequent call. Note that configuration
{{atlas.rest.create.shell.entity.for.non-existing.ref=true}} should be set for
the above handling; else Atlas will fail creation of the column entity.
> isOptional for composition relationship category?
> -
>
> Key: ATLAS-3546
> URL: https://issues.apache.org/jira/browse/ATLAS-3546
> Project: Atlas
> Issue Type: Improvement
> Components: atlas-core
>Affects Versions: 2.0.0
>Reporter: charles shen
>Priority: Major
>
> I noticed since
> [ATLAS-3051|https://issues.apache.org/jira/browse/ATLAS-3051], the
> relationship attribute must be specified in the end def which is not
> container and relationship category is composition.
> I understand it's to prohibit orphan children but is it too strong? Reason
> below:
> # I have to provide all the entities along the relationship path in the
> payload when creating a child, eg, for RDBMS, I have to provide
> rdbms_instance, rdbms_db, rdbms_table, rdbms_column where I just want to
> create a single rdbms_column, it brings performance overhead to check
> existence of rdbms_instance, rdbms_db, etc...,
> # I have defined a composition relationship type where each end is the same
> entity type, it couldn't be created successfully anymore since it always
> requires the mandatory attribute where it's the same type and then falls into
> infinite loop.
> Three possible fixes:
> # Remove the isOptional constraint, since ownedRef/inverseRef doesn't have
> such constraint.
> # Add isOptional to relationship type end def.
> # Add option in Rest to ignore the isOptional constraint for relationship
> type.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
