[jira] [Commented] (ATLAS-2456) Implement tag propagation using relationships
[
https://issues.apache.org/jira/browse/ATLAS-2456?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16377205#comment-16377205
]
Madhan Neethiraj commented on ATLAS-2456:
-
Committed [^ATLAS-2456-7.patch] to master:
[http://git-wip-us.apache.org/repos/asf/atlas/commit/be9b39bf]
> Implement tag propagation using relationships
> -
>
> Key: ATLAS-2456
> URL: https://issues.apache.org/jira/browse/ATLAS-2456
> Project: Atlas
> Issue Type: New Feature
> Components: atlas-core
>Affects Versions: 1.0.0
>Reporter: Sarath Subramanian
>Assignee: Sarath Subramanian
>Priority: Major
> Fix For: 1.0.0
>
> Attachments: ATLAS-2456-6.patch, ATLAS-2456-7.patch, TAG PROPAGATION
> IN ATLAS v.1.pdf, tag_propagation_rest_api
>
>
> {color:#172b4d}Scalable way to quickly and efficiently propagate tags for
> efficient searches and tag based security. Likewise tags for derivative
> dataset should be inherited from the parent. For example, if an entity is
> tagged "PII" then resulting entity created from a CTAS operation should also
> be tagged "secret" to maintain the classification of the parent. In the case
> where 2 or more datasets are aggregated the derivative dataset should be a
> union of all parent tags.{color}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (ATLAS-2456) Implement tag propagation using relationships
[
https://issues.apache.org/jira/browse/ATLAS-2456?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16375844#comment-16375844
]
Sarath Subramanian commented on ATLAS-2456:
---
+1 for the patch [~madhan.neethiraj]. Looks good.
> Implement tag propagation using relationships
> -
>
> Key: ATLAS-2456
> URL: https://issues.apache.org/jira/browse/ATLAS-2456
> Project: Atlas
> Issue Type: New Feature
> Components: atlas-core
>Affects Versions: 1.0.0
>Reporter: Sarath Subramanian
>Assignee: Sarath Subramanian
>Priority: Major
> Fix For: 1.0.0
>
> Attachments: ATLAS-2456-6.patch, ATLAS-2456-7.patch, TAG PROPAGATION
> IN ATLAS v.1.pdf, tag_propagation_rest_api
>
>
> {color:#172b4d}Scalable way to quickly and efficiently propagate tags for
> efficient searches and tag based security. Likewise tags for derivative
> dataset should be inherited from the parent. For example, if an entity is
> tagged "PII" then resulting entity created from a CTAS operation should also
> be tagged "secret" to maintain the classification of the parent. In the case
> where 2 or more datasets are aggregated the derivative dataset should be a
> union of all parent tags.{color}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (ATLAS-2456) Implement tag propagation using relationships
[
https://issues.apache.org/jira/browse/ATLAS-2456?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16375114#comment-16375114
]
Madhan Neethiraj commented on ATLAS-2456:
-
[~sarath.ku...@gmail.com] - added another minor fix in handling of string
conversion to EntityAuditAction V1/V2. Please review [^ATLAS-2456-7.patch] .
> Implement tag propagation using relationships
> -
>
> Key: ATLAS-2456
> URL: https://issues.apache.org/jira/browse/ATLAS-2456
> Project: Atlas
> Issue Type: New Feature
> Components: atlas-core
>Affects Versions: 1.0.0
>Reporter: Sarath Subramanian
>Assignee: Sarath Subramanian
>Priority: Major
> Fix For: 1.0.0
>
> Attachments: ATLAS-2456-6.patch, ATLAS-2456-7.patch, TAG PROPAGATION
> IN ATLAS v.1.pdf, tag_propagation_rest_api
>
>
> {color:#172b4d}Scalable way to quickly and efficiently propagate tags for
> efficient searches and tag based security. Likewise tags for derivative
> dataset should be inherited from the parent. For example, if an entity is
> tagged "PII" then resulting entity created from a CTAS operation should also
> be tagged "secret" to maintain the classification of the parent. In the case
> where 2 or more datasets are aggregated the derivative dataset should be a
> union of all parent tags.{color}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (ATLAS-2456) Implement tag propagation using relationships
[
https://issues.apache.org/jira/browse/ATLAS-2456?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16374660#comment-16374660
]
Madhan Neethiraj commented on ATLAS-2456:
-
Committed following additional patches to master:
- http://git-wip-us.apache.org/repos/asf/atlas/commit/5e496e5e
- http://git-wip-us.apache.org/repos/asf/atlas/commit/bf4bf7fe
> Implement tag propagation using relationships
> -
>
> Key: ATLAS-2456
> URL: https://issues.apache.org/jira/browse/ATLAS-2456
> Project: Atlas
> Issue Type: New Feature
> Components: atlas-core
>Affects Versions: 1.0.0
>Reporter: Sarath Subramanian
>Assignee: Sarath Subramanian
>Priority: Major
> Fix For: 1.0.0
>
> Attachments: ATLAS-2456-6.patch, TAG PROPAGATION IN ATLAS v.1.pdf,
> tag_propagation_rest_api
>
>
> {color:#172b4d}Scalable way to quickly and efficiently propagate tags for
> efficient searches and tag based security. Likewise tags for derivative
> dataset should be inherited from the parent. For example, if an entity is
> tagged "PII" then resulting entity created from a CTAS operation should also
> be tagged "secret" to maintain the classification of the parent. In the case
> where 2 or more datasets are aggregated the derivative dataset should be a
> union of all parent tags.{color}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (ATLAS-2456) Implement tag propagation using relationships
[
https://issues.apache.org/jira/browse/ATLAS-2456?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16374074#comment-16374074
]
Nixon Rodrigues commented on ATLAS-2456:
+1 for ATLAS-2456-6.patch, the changes LGTM.
Thanks [~madhan.neethiraj] for the patch.
> Implement tag propagation using relationships
> -
>
> Key: ATLAS-2456
> URL: https://issues.apache.org/jira/browse/ATLAS-2456
> Project: Atlas
> Issue Type: New Feature
> Components: atlas-core
>Affects Versions: 1.0.0
>Reporter: Sarath Subramanian
>Assignee: Sarath Subramanian
>Priority: Major
> Fix For: 1.0.0
>
> Attachments: ATLAS-2456-6.patch, TAG PROPAGATION IN ATLAS v.1.pdf,
> tag_propagation_rest_api
>
>
> {color:#172b4d}Scalable way to quickly and efficiently propagate tags for
> efficient searches and tag based security. Likewise tags for derivative
> dataset should be inherited from the parent. For example, if an entity is
> tagged "PII" then resulting entity created from a CTAS operation should also
> be tagged "secret" to maintain the classification of the parent. In the case
> where 2 or more datasets are aggregated the derivative dataset should be a
> union of all parent tags.{color}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (ATLAS-2456) Implement tag propagation using relationships
[
https://issues.apache.org/jira/browse/ATLAS-2456?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16374070#comment-16374070
]
Madhan Neethiraj commented on ATLAS-2456:
-
[~sarath.ku...@gmail.com] - added a fix about entity-notification
deserialization to support V2 notification message. Please review.
> Implement tag propagation using relationships
> -
>
> Key: ATLAS-2456
> URL: https://issues.apache.org/jira/browse/ATLAS-2456
> Project: Atlas
> Issue Type: New Feature
> Components: atlas-core
>Affects Versions: 1.0.0
>Reporter: Sarath Subramanian
>Assignee: Sarath Subramanian
>Priority: Major
> Fix For: 1.0.0
>
> Attachments: ATLAS-2456-6.patch, TAG PROPAGATION IN ATLAS v.1.pdf,
> tag_propagation_rest_api
>
>
> {color:#172b4d}Scalable way to quickly and efficiently propagate tags for
> efficient searches and tag based security. Likewise tags for derivative
> dataset should be inherited from the parent. For example, if an entity is
> tagged "PII" then resulting entity created from a CTAS operation should also
> be tagged "secret" to maintain the classification of the parent. In the case
> where 2 or more datasets are aggregated the derivative dataset should be a
> union of all parent tags.{color}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (ATLAS-2456) Implement tag propagation using relationships
[
https://issues.apache.org/jira/browse/ATLAS-2456?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16369846#comment-16369846
]
David Radley commented on ATLAS-2456:
-
[~sarath.ku...@gmail.com] Thanks for your reply.
>From your responses, is it true that:
* many propagated classifications of the same type can exist for an entity
* a propagated classification can exist with the same type as an explicitly
defined classification
* on point 7 - does this mean that only one classification type or any of its
sub types can be applied to an entity
I suggest:
* we do not propagate a classification of a given type to an entity if there
is already a classification of that type defined (we should account for sub
types here)
* we should not allow more than one classification of a given type against an
entity across explicitly defined classification and propagated classifications.
This means that a policy only has to consider one classification and its
ramifications. I am not sure how we resolve the potential conflicts; ideally
this would be resolved using some rules and a data steward. If we could search
for these conflicts (maybe with a new conflicted status on the entity) , then a
classification could be then explicitly defined on the entity and would force
which classification values were effective.
> Implement tag propagation using relationships
> -
>
> Key: ATLAS-2456
> URL: https://issues.apache.org/jira/browse/ATLAS-2456
> Project: Atlas
> Issue Type: New Feature
> Components: atlas-core
>Affects Versions: 1.0.0
>Reporter: Sarath Subramanian
>Assignee: Sarath Subramanian
>Priority: Major
> Fix For: 1.0.0
>
> Attachments: TAG PROPAGATION IN ATLAS v.1.pdf,
> tag_propagation_rest_api
>
>
> {color:#172b4d}Scalable way to quickly and efficiently propagate tags for
> efficient searches and tag based security. Likewise tags for derivative
> dataset should be inherited from the parent. For example, if an entity is
> tagged "PII" then resulting entity created from a CTAS operation should also
> be tagged "secret" to maintain the classification of the parent. In the case
> where 2 or more datasets are aggregated the derivative dataset should be a
> union of all parent tags.{color}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (ATLAS-2456) Implement tag propagation using relationships
[
https://issues.apache.org/jira/browse/ATLAS-2456?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16369779#comment-16369779
]
Sarath Subramanian commented on ATLAS-2456:
---
[~davidrad],
1) Yes, the entity will have both assigned classification and propagated
classifications each having different attributes. In the code this is handled
by creating an edge (propagated) for propagated classifications. So an entity
can be associated with only one classification type but can have any number of
propagated classification types (from upstream entities).
4) Entity will provide both - associated and propagated classification
information. The client will need to decide on which classification value to
work on.
5) Yes, only one classification of a type is allowed for an entity, exception
will be thrown if same classification is added again.
6) Yes, propagated classifications will also follow the
classification-entitytype restriction rules.
7) Both 'confidentiality' and its subclass will share the same confidentiality
'level' attribute, hence will only have a single value for 'level' attribute.
There won't be a conflict for the shared attribute value.
> Implement tag propagation using relationships
> -
>
> Key: ATLAS-2456
> URL: https://issues.apache.org/jira/browse/ATLAS-2456
> Project: Atlas
> Issue Type: New Feature
> Components: atlas-core
>Affects Versions: 1.0.0
>Reporter: Sarath Subramanian
>Assignee: Sarath Subramanian
>Priority: Major
> Fix For: 1.0.0
>
> Attachments: TAG PROPAGATION IN ATLAS v.1.pdf,
> tag_propagation_rest_api
>
>
> {color:#172b4d}Scalable way to quickly and efficiently propagate tags for
> efficient searches and tag based security. Likewise tags for derivative
> dataset should be inherited from the parent. For example, if an entity is
> tagged "PII" then resulting entity created from a CTAS operation should also
> be tagged "secret" to maintain the classification of the parent. In the case
> where 2 or more datasets are aggregated the derivative dataset should be a
> union of all parent tags.{color}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (ATLAS-2456) Implement tag propagation using relationships
[
https://issues.apache.org/jira/browse/ATLAS-2456?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16368955#comment-16368955
]
David Radley commented on ATLAS-2456:
-
[~sarath.ku...@gmail.com] I have not had time to go through the code in a lot
of detail. I think it looks pretty good. I wanted to check how you handle the
following situations:
1) If there is a classification that is applied to an entity with certain
attributes defined, and you decide to propagate the same classification to that
entity, in this situation there must be 2 classification instances - each with
different property values.
So how does the code handle the case when there are 2 instances of the same
classification with different property values that propagate to the same target
entity.
2) I think the classifications defined in the open type
GovernanceActionclassifications
[https://cwiki.apache.org/confluence/display/ATLAS/Area+4+-+Governance] are
going to be the key classifications around governance. There classifications
are defined by their enums- which have an implied low-high order. So for
example confifidentialityLevel of sensitive is higher than internal. For point
1 scenario- we would want the higher value to win.
4) I think that any classification that is explicitly defined on an entity wins
over propagated values.
5) Can you confirm that only one classification of a type is allowed for a
given entity.
6) I assume that the entitytypes defined on a classificationdef will always
constrain - so classification and propagated classification (or their subtypes)
can only ever classify entities of those types (or their subtypes).
7) I assume that a classification and subtypes of a classification could both
be applied to an entity. This could compromise the idea of one classification
per entity. How do you see this working or is this policed as invalid? For
example a Confidentiality classification and its subclass could be applied to
an entity. - each specifying different levels of confidentiality - which one is
effective?
> Implement tag propagation using relationships
> -
>
> Key: ATLAS-2456
> URL: https://issues.apache.org/jira/browse/ATLAS-2456
> Project: Atlas
> Issue Type: New Feature
> Components: atlas-core
>Affects Versions: 1.0.0
>Reporter: Sarath Subramanian
>Assignee: Sarath Subramanian
>Priority: Major
> Fix For: 1.0.0
>
> Attachments: TAG PROPAGATION IN ATLAS v.1.pdf,
> tag_propagation_rest_api
>
>
> {color:#172b4d}Scalable way to quickly and efficiently propagate tags for
> efficient searches and tag based security. Likewise tags for derivative
> dataset should be inherited from the parent. For example, if an entity is
> tagged "PII" then resulting entity created from a CTAS operation should also
> be tagged "secret" to maintain the classification of the parent. In the case
> where 2 or more datasets are aggregated the derivative dataset should be a
> union of all parent tags.{color}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
