[
https://issues.apache.org/jira/browse/ATLAS-2774?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16687596#comment-16687596
]
ASF subversion and git services commented on ATLAS-2774:
--------------------------------------------------------
Commit daabed1312c73916136eacf09218300d75bb8f9b in atlas's branch
refs/heads/branch-0.8 from [~mad...@apache.org]
[ https://git-wip-us.apache.org/repos/asf?p=atlas.git;h=daabed1 ]
ATLAS-2774: enhancement to support parameterized delete-type (hard or soft) per
API call
(cherry picked from commit d003ddb33b16a0a67e6b7f7eae29c1638a4d0971)
> Options for hard and soft delete of instances
> ---------------------------------------------
>
> Key: ATLAS-2774
> URL: https://issues.apache.org/jira/browse/ATLAS-2774
> Project: Atlas
> Issue Type: New Feature
> Reporter: Graham Wallis
> Assignee: Ashutosh Mestry
> Priority: Major
> Attachments: ATLAS-2774-Soft-and-hard-delete-impl.patch,
> ATLAS-2774-branch-0.8.patch, ATLAS-2774-master.patch
>
>
> *Overview*
> For Entities and Relationships, the following delete behaviors are desired.
> Atlas can be configured to offer hard delete (only). Alternatively it can be
> configured to offer soft-delete. Configuration is achieved by setting the
> delete handler configuration property (see atlas-application.properties) to
> either the SoftDeleteHandlerV1 or the HardDeleteHandlerV1. The default (for
> when the property is not set) is to use the SoftDeleteHandlerV1.
> The AtlasEntityStoreV2 deleteById() and deleteByIds() and methods and the
> AtlasRelationshipStoreV2 deleteById() method are sensitive to the above
> configuration. When the configuration is soft these methods will perform a
> soft delete; when it is hard they will perform a hard delete.
> In addition to the above methods, the AtlasEntityStoreV2 and
> AtlasRelationshipStoreV2 offer a (new) purgeById() method that ALWAYS
> performs a hard delete. This is true regardless of which delete handler has
> been configured. When the configuration is hard, the purgeById() methods and
> deleteById() methods are essentially equivalent. When the configuration is
> soft, the purgeById() methods continue to provide a hard delete capability.
> The AtlasEntityStoreV2 deleteById(), deleteByIds() and purgeById() methods
> will delete the specified entities and any relationships to which they are
> connected. The AtlasRelationshipStoreV2 deleteById() and purgeById() methods
> will delete the specified relationship. Deletion of a relationship may cause
> an upate to an entity to which the relationhsip is connected, if it changes
> the propagation of classifications, for example.
> In a hard delete or purge operation, an affected entity and relationship will
> be removed from the graph and will not be returned in response to any future
> queries. This is true regardless of whether the affected entity or
> relationship has already been soft deleted or not. i.e. it's status could
> initially be ACTIVE or DELETED. Following the operation it will not exist.
> In a soft delete, the affected entities and relationships (that initially
> have status ACTIVE) are updated to set the status to DELETED. These instances
> can still be returned from queries and searches if the appropriate control is
> selected (to include deleted instances). Without that control selected they
> will not be returned.
> *Use Case for soft delete*
> Soft delete provides a 'safe' means of deleting instances from the repository
> whilst preserving an audit trail and (if supported by the repository)
> enabling a deleted instance to be restored (to ACTIVE status).
> *Use Cases for hard delete*
> One use case for hard delete is to permanently remove soft-deleted items some
> period of time after they were soft deleted.
> Another use case for hard delete is to is to correct (clean up after) a
> mistake in which a potentially large batch of incomplete/corrupted/wrong
> metadata is loaded and needs to be fully removed.
> *Use Scenario for an OMAS user*
> An OMAS provides a delete function that does NOT offer the user a choice of
> hard/soft. The OMAS's delete function will invoke the relevant soft delete
> method provided by the OMRS - deleteEntity or deleteRelationship. These are
> both soft delete methods.
> The Atlas OMRS Connector's deleteEntity/deleteRelationship methods will check
> whether Atlas is configured for hard or soft delete. If Atlas is configured
> for hard delete then soft-delete is not possible, and the method will throw a
> FunctionNotSupported exception which is caught by the OMAS.
> On catching this exception the OMAS author should automatically escalate the
> soft delete to a hard delete by calling either of the mandatory OMRS
> purgeEntity() or purgeRelationship() methods. In the Atlas OMRS Connector
> these methods will invoke AtlasEntityStoreV2 purgeById() or
> AtlasRelationshipStoreV2 purgeById(). The effect of this is that if soft
> delete is not possible, a hard delete is used instead.
> An OMAS does NOT provide a purge capability to a non-privileged user, but a
> suitably expert OMRS or Atlas repository administrator can explicitly issue a
> purge call at the OMRS or Atlas interface, by calling purgeEntity() or
> purgeRelationship() (or the relevant Atlas store's purgeById() method).
> *Use Scenario for an Administrator*
> As highlighted above, a suitably expert repository Administrator can invoke
> the purgeEntity(), purgeRelationship() methods directly. They may need to do
> this to permanently remove instances following an earlier soft-delete and the
> timeout of a period of grace, or following a failed import or batch update.
> In addition to the above direct use by a repository administrator of the
> 'purge' methods, an Administrator could alternatively use the deleteEntity(),
> deleteRelationship() methods to perform soft deletes. The invoked method will
> behave as described above for an OMAS - performing a soft-delete if possible
> and throwing a FunctionNotSupported exception otherwise. In the latter case
> (no support for soft-delete), the Administrator COULD then choose to issue a
> purgeEntity(), purgeRelationship() call.
> *Restore of an entity or relationship*
> If an instance has been deleted using a soft-delete, the object still exists
> but has a status of 'DELETED'. Whilst in the DELETED state, the object can be:
> * optionally included/excluded from search results
> * restored using the OMRS method for restoreEntity or restoreRelationship.
> * purged from the repository using the purgeEntity() or purgeRelationship()
> method (or the underlying Atlas store purgeById() method).
> The use case for restoring a soft-deleted entity or relationship is to
> correct an accidental deletion, or possibly a deliberately deletion that was
> performed for impact analysis.
> To restore an entity or relationship that has been soft-deleted, an OMAS user
> or repository administrator can use the restoreEntity() or
> restoreRealtionship() method of the OMRS connector.
> In the case of the Atlas OMRS Connector, the restoreEntity() and
> restoreRealtionship() methods will invoke (new) Atlas store methods to update
> the affected entities and relationships, leaving them in ACTIVE state and
> with classifications that reflect the propagation options between all the
> resulting active entities and relationships.
> *Within the Atlas Repository*
> The AtlasEntityStoreV2 and AtlasRelationshipStoreV2 provide additional
> methods for purgeById() that perform a hard delete, regardless of the choice
> of configured delete handler.
> The stores also provide additional methods to enable restore of an entity or
> relationship that has been soft-deleted.
> *Within the Atlas OMRS Connector*
> The Atlas OMRS Connector will interrogate the Atlas repository configuration
> to determine whether it is configuraed for soft or hard deletes. The Atlas
> OMRS Connector implementations of deleteEntity() and deleteRelationship()
> methods will therefore have prior knowledge as to whether it is feasible to
> attempt to perform a soft-delete. If feasible, the soft-delete will be issued
> to the relevant Atlas store. If not feasible, the methods will throw a
> FunctionNotSupported exception.
> Also within the Atlas OMRS Connector, the mandatory OMRS purgeEntity(),
> purgeRelationship() methods will invoke the Atlas stores' purgeById() methods
> to perform permanent (hard) removal of an entity or relationship from the
> repository. As outline above, these methods may be called either by an OMAS
> that has attempted a soft-delete and been 'bounced' or called directly by an
> Adminstrator.
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
