Repository: atlas Updated Branches: refs/heads/master f65aeceeb -> 024484407
ATLAS-2687: Create twiki documentation about classification propagation Project: http://git-wip-us.apache.org/repos/asf/atlas/repo Commit: http://git-wip-us.apache.org/repos/asf/atlas/commit/02448440 Tree: http://git-wip-us.apache.org/repos/asf/atlas/tree/02448440 Diff: http://git-wip-us.apache.org/repos/asf/atlas/diff/02448440 Branch: refs/heads/master Commit: 024484407290ad664141c1a8c52012c3e7708c3c Parents: f65aece Author: Sarath Subramanian <[email protected]> Authored: Tue May 15 16:29:04 2018 -0700 Committer: Sarath Subramanian <[email protected]> Committed: Tue May 15 22:34:03 2018 -0700 ---------------------------------------------------------------------- .../twiki/classification-propagation-1.png | Bin 0 -> 98689 bytes .../twiki/classification-propagation-2.png | Bin 0 -> 128774 bytes .../twiki/classification-propagation-3.png | Bin 0 -> 128278 bytes .../twiki/classification-propagation-4.png | Bin 0 -> 106518 bytes .../twiki/classification-propagation-5.png | Bin 0 -> 78040 bytes .../twiki/classification-propagation-6.png | Bin 0 -> 69127 bytes .../twiki/classification-propagation-7.png | Bin 0 -> 72418 bytes .../twiki/classification-propagation-8.png | Bin 0 -> 140839 bytes .../site/twiki/ClassificationPropagation.twiki | 86 +++++++++++++++++++ docs/src/site/twiki/index.twiki | 1 + 10 files changed, 87 insertions(+) ---------------------------------------------------------------------- http://git-wip-us.apache.org/repos/asf/atlas/blob/02448440/docs/src/site/resources/images/twiki/classification-propagation-1.png ---------------------------------------------------------------------- diff --git a/docs/src/site/resources/images/twiki/classification-propagation-1.png b/docs/src/site/resources/images/twiki/classification-propagation-1.png new file mode 100644 index 0000000..2e27424 Binary files /dev/null and b/docs/src/site/resources/images/twiki/classification-propagation-1.png differ http://git-wip-us.apache.org/repos/asf/atlas/blob/02448440/docs/src/site/resources/images/twiki/classification-propagation-2.png ---------------------------------------------------------------------- diff --git a/docs/src/site/resources/images/twiki/classification-propagation-2.png b/docs/src/site/resources/images/twiki/classification-propagation-2.png new file mode 100644 index 0000000..352db3b Binary files /dev/null and b/docs/src/site/resources/images/twiki/classification-propagation-2.png differ http://git-wip-us.apache.org/repos/asf/atlas/blob/02448440/docs/src/site/resources/images/twiki/classification-propagation-3.png ---------------------------------------------------------------------- diff --git a/docs/src/site/resources/images/twiki/classification-propagation-3.png b/docs/src/site/resources/images/twiki/classification-propagation-3.png new file mode 100644 index 0000000..355bdf9 Binary files /dev/null and b/docs/src/site/resources/images/twiki/classification-propagation-3.png differ http://git-wip-us.apache.org/repos/asf/atlas/blob/02448440/docs/src/site/resources/images/twiki/classification-propagation-4.png ---------------------------------------------------------------------- diff --git a/docs/src/site/resources/images/twiki/classification-propagation-4.png b/docs/src/site/resources/images/twiki/classification-propagation-4.png new file mode 100644 index 0000000..74d8ea1 Binary files /dev/null and b/docs/src/site/resources/images/twiki/classification-propagation-4.png differ http://git-wip-us.apache.org/repos/asf/atlas/blob/02448440/docs/src/site/resources/images/twiki/classification-propagation-5.png ---------------------------------------------------------------------- diff --git a/docs/src/site/resources/images/twiki/classification-propagation-5.png b/docs/src/site/resources/images/twiki/classification-propagation-5.png new file mode 100644 index 0000000..66350e4 Binary files /dev/null and b/docs/src/site/resources/images/twiki/classification-propagation-5.png differ http://git-wip-us.apache.org/repos/asf/atlas/blob/02448440/docs/src/site/resources/images/twiki/classification-propagation-6.png ---------------------------------------------------------------------- diff --git a/docs/src/site/resources/images/twiki/classification-propagation-6.png b/docs/src/site/resources/images/twiki/classification-propagation-6.png new file mode 100644 index 0000000..1669bed Binary files /dev/null and b/docs/src/site/resources/images/twiki/classification-propagation-6.png differ http://git-wip-us.apache.org/repos/asf/atlas/blob/02448440/docs/src/site/resources/images/twiki/classification-propagation-7.png ---------------------------------------------------------------------- diff --git a/docs/src/site/resources/images/twiki/classification-propagation-7.png b/docs/src/site/resources/images/twiki/classification-propagation-7.png new file mode 100644 index 0000000..ff0cfb0 Binary files /dev/null and b/docs/src/site/resources/images/twiki/classification-propagation-7.png differ http://git-wip-us.apache.org/repos/asf/atlas/blob/02448440/docs/src/site/resources/images/twiki/classification-propagation-8.png ---------------------------------------------------------------------- diff --git a/docs/src/site/resources/images/twiki/classification-propagation-8.png b/docs/src/site/resources/images/twiki/classification-propagation-8.png new file mode 100644 index 0000000..70985f9 Binary files /dev/null and b/docs/src/site/resources/images/twiki/classification-propagation-8.png differ http://git-wip-us.apache.org/repos/asf/atlas/blob/02448440/docs/src/site/twiki/ClassificationPropagation.twiki ---------------------------------------------------------------------- diff --git a/docs/src/site/twiki/ClassificationPropagation.twiki b/docs/src/site/twiki/ClassificationPropagation.twiki new file mode 100755 index 0000000..75d6165 --- /dev/null +++ b/docs/src/site/twiki/ClassificationPropagation.twiki @@ -0,0 +1,86 @@ +---+ Classification Propagation + +Classification propagation enables classifications associated to an entity to be automatically associated with other +related entities of the entity. This is very useful in dealing with scenarios where a dataset derives it data from other +datasets - like a table loaded with data in a file, a report generated from a table/view, etc. For example, when a table +is classified as *"PII"*, tables or views that derive data from this table (via CTAS or âcreate viewâ operation) will be +automatically classified as *"PII"*. + + +---++ Use Cases +Consider the following lineage where data from a 'hdfs_path' entity is loaded into a table, which is further made +available through views. We will go through various scenarios to understand the classification propagation feature. + +<img src="images/twiki/classification-propagation-1.png"/> + + +---++ Add classification to an entity +When classification âPIIâ is added to 'hdfs_path' entity, the classification is propagated to all impacted entities in the +lineage path, including 'employees' table, views 'us_employees' and 'uk_employees' - as shown below. + +<img src="images/twiki/classification-propagation-2.png"/> + +---++ Update classification associated with an entity +Any updates to classifications associated with an entity will be seen in all entities the classification is +propagated to as well. + +<img src="images/twiki/classification-propagation-3.png"/> + +---++ Remove classification associated with an entity +When a classification is deleted from an entity, the classification will be removed from all entities the classification +is propagated to as well. + +<img src="images/twiki/classification-propagation-4.png"/> + +---++ Add lineage between entities +When lineage is added between entities, for example to capture loading of data in a file to a table, the classifications +associated with the source entity are propagated to all impacted entities as well. + +For example, when a view is created from a table, classifications associated with the table are propagated to the newly +created view as well. + +<img src="images/twiki/classification-propagation-5.png"/> + +---++ Delete an entity +When an entity is deleted, classifications associated with this entity will be removed from all entities the +classifications are propagated to. + +For example. when _employees_ table is deleted, classifications associated with this table are removed from +'employees_view' view. + +<img src="images/twiki/classification-propagation-6.png"/> + +---++ Control Propagation +Apache Atlas provides few options to control whether/where a classification is propagated. +This section details available options. + +---++ Propagate flag in classification +Each association of classification to an entity has a boolean flag that controls whether the classification is +propagated or not. When a classification is associated with an entity, this flag is set to âtrueâ i.e. the classification +will be propagated to all impacted entities. This flag can be updated to desired value during initial association or later. + +<img src="images/twiki/classification-propagation-7.png"/> + +---++ Propagate flag in lineage edge +Apache Atlas supports a flag at lineage edge to enable/disable propagation of classifications through the edge. By default, +the propagation is enabled for lineage edges. When the flag is turned off, no classification will be propagated through +this edge; and propagation of currently propagated classifications through the edge will be reevaluated, so that they can +be removed from impacted entities. When the flag is turned on, propagation of classifications of the source entity will +be reevaluated, so that they can be propagated to all impacted entities. + +---++ Block propagation of specific classifications in lineage edge +Apache Atlas supports blocking propagation of specific classifications in at lineage edges. This can be useful, for example, +to handle scenarios like: a column classified as PII is masked when creating a view; in such scenario, if corresponding +column in the created view might not have PII, hence the propagation of PII classification should be blocked. This can be +done by updating the lineage edge to add the PII classification in _âblocked propagated classificationsâ_ list. +Classifications in blocked propagated classifications will not be propagated in the derivative/downstream entities. + +<img src="images/twiki/classification-propagation-8.png"/> + +---++ Notifications and Audit +When propagated classifications are added/update/deleted, Apache Atlas sends notifications to 'ATLAS_ENTITIES' topic for +each entity affected by the propagation. + +---++ Glossary +When a classification is associated with a glossary term, the classification is automatically propagated to all entities +associated with the term. \ No newline at end of file http://git-wip-us.apache.org/repos/asf/atlas/blob/02448440/docs/src/site/twiki/index.twiki ---------------------------------------------------------------------- diff --git a/docs/src/site/twiki/index.twiki b/docs/src/site/twiki/index.twiki index 4f74702..40621af 100755 --- a/docs/src/site/twiki/index.twiki +++ b/docs/src/site/twiki/index.twiki @@ -58,6 +58,7 @@ capabilities around these data assets for data scientists, analysts and the data * [[Atlas-Authentication][Authentication]] * [[Atlas-Authorization-Model][Atlas Authorization Model]] * [[Configure-simple-authorizer][Steps to configure Atlas Simple Authorizer]] + * [[ClassificationPropagation][Classification Propagation]] * [[Configuration][Configuration]] * [[Notifications][Notifications]] * Hooks & Bridges
