date:20180425

[jira] [Assigned] (ATLAS-2608) UI - Cache is not cleared if user click on cancel button while editing propagation flow

2018-04-25 Thread pratik pandey (JIRA)


 [ 
https://issues.apache.org/jira/browse/ATLAS-2608?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

pratik pandey reassigned ATLAS-2608:


Assignee: pratik pandey

> UI - Cache is not cleared if user click on cancel button while editing 
> propagation flow
> ---
>
> Key: ATLAS-2608
> URL: https://issues.apache.org/jira/browse/ATLAS-2608
> Project: Atlas
>  Issue Type: Bug
>Affects Versions: 1.0.0
>Reporter: Tejas Rawool
>Assignee: pratik pandey
>Priority: Major
> Fix For: 1.0.0
>
>
> Steps to reproduce:-
>  
> 1.Do a Basic Search for Tables
> 2.Now go to Lineage Tab of  sales_fact
> 3.Click on arrow between sales_fact and loadSalesDaily
> 4.Now switch to Block Propagation
> 5.Checked any of the entity and click on cancel button
> 6.Now again click on that arrow and go to block propagation without 
> refreshing the page
> Error:-
> An Duplicate entity is created which is incorrect



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Resolved] (ATLAS-2127) Import of an entity associated with a tag into backup cluster with updateTypeDefinition options set to false

2018-04-25 Thread Ashutosh Mestry (JIRA)


 [ 
https://issues.apache.org/jira/browse/ATLAS-2127?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ashutosh Mestry resolved ATLAS-2127.

Resolution: Fixed

> Import of an entity associated with a tag into backup cluster with 
> updateTypeDefinition options set to false
> 
>
> Key: ATLAS-2127
> URL: https://issues.apache.org/jira/browse/ATLAS-2127
> Project: Atlas
>  Issue Type: Bug
>  Components:  atlas-core
>Affects Versions: 1.0.0
>Reporter: Sharmadha Sainath
>Assignee: Ashutosh Mestry
>Priority: Major
> Attachments: ImportExceptionWithTag.txt, hdfs_path1.zip
>
>
> 1. On cluster1 , created an entity and associated it to a tag.
> 2. Exported the entity into a zipfile.
> 3. cluster2 is in clean state.(i.e no entity / tag is created)
> 4. Tried to import the entity zip into cluster2 with import option 
> "updateTypeDefinition" set to false.
> 5. Import failed with NPE. 
> Since updateTypeDefinition is set to false , NPE is thrown when the entity is 
> attempted to be associated to the tag in the backup cluster.
> This is the expected behavior , but the cause of the issue is not very 
> explicit to the end user when Atlas throws NPE.
> Attached the exception stack trace.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Resolved] (ATLAS-1716) Export API is successfully exporting metadata when deleted entity is given as starting point.

2018-04-25 Thread Ashutosh Mestry (JIRA)


 [ 
https://issues.apache.org/jira/browse/ATLAS-1716?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ashutosh Mestry resolved ATLAS-1716.

Resolution: Fixed

> Export API is successfully exporting metadata when deleted entity is given as 
> starting point.
> -
>
> Key: ATLAS-1716
> URL: https://issues.apache.org/jira/browse/ATLAS-1716
> Project: Atlas
>  Issue Type: Bug
>  Components:  atlas-core
>Affects Versions: 0.8-incubating
>Reporter: Ayub Pathan
>Assignee: Ashutosh Mestry
>Priority: Critical
> Fix For: 1.0.0
>
>
> export API should not honor the metadata export operation when deleted entity 
> is given as starting point. But this is not the case, the API is exporting 
> the deleted entity metadata as well. Is this a new requiremen?



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Resolved] (ATLAS-2610) Import Process: Import from Earlier Versions Should Set Relationship Defaults

2018-04-25 Thread Ashutosh Mestry (JIRA)


 [ 
https://issues.apache.org/jira/browse/ATLAS-2610?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ashutosh Mestry resolved ATLAS-2610.

Resolution: Fixed

> Import Process: Import from Earlier Versions Should Set Relationship Defaults
> -
>
> Key: ATLAS-2610
> URL: https://issues.apache.org/jira/browse/ATLAS-2610
> Project: Atlas
>  Issue Type: Bug
>  Components:  atlas-core
>Affects Versions: trunk, 1.0.0
>Reporter: Ashutosh Mestry
>Assignee: Ashutosh Mestry
>Priority: Major
> Fix For: trunk
>
> Attachments: 
> ATLAS-2610-Relationship-id-during-import.-Tag-propga.patch
>
>
> *Scenario*
> User exports data from v0.8 or before. Importing to v1.0 should set correct 
> defaults for features specific to v1.0.
> *Cases*
>  * Relationship GUID should be present on edges. API calls like _findByGuid_ 
> should return _AtlasEntity_ with _relationshipGuid_ set.
>  * For classification, _propagateTag_  should be set to _false_ by default.
>  * Export-Import-Export verification should set inverse attributes such that 
> they referenced in parent and referring to parent from child.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Re: Review Request 66797: ATLAS-2609: Update audit listener to handle propagated classification add/delete/update

2018-04-25 Thread Sarath Subramanian


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/66797/
---

(Updated April 25, 2018, 9:39 p.m.)


Review request for atlas, Apoorv Naik, Ashutosh Mestry, and Madhan Neethiraj.


Changes
---

addressed review comments


Bugs: ATLAS-2609
https://issues.apache.org/jira/browse/ATLAS-2609


Repository: atlas


Description
---

Currently audit are not generated for propagated classification add/update or 
delete.

For Audit:
* Adding Propagated Classification   => Send 'Propagated Classifications Added' 
message to audit repository. 
* Removing Propagated Classification => Send 'Propagated Classifications 
Deleted' message to audit repository.
* Updating Propagated Classification => Send 'Propagated Classifications 
Updated' message to audit repository.


Diffs (updated)
-

  client/client-v1/src/main/java/org/apache/atlas/EntityAuditEvent.java 
3cf3e212 
  dashboardv2/public/js/utils/Enums.js 8c0518bf 
  intg/src/main/java/org/apache/atlas/listener/EntityChangeListenerV2.java 
70877d22 
  intg/src/main/java/org/apache/atlas/model/audit/EntityAuditEventV2.java 
1045c774 
  
repository/src/main/java/org/apache/atlas/repository/audit/AbstractStorageBasedAuditRepository.java
 af6c4e62 
  
repository/src/main/java/org/apache/atlas/repository/audit/EntityAuditListener.java
 1c04eea4 
  
repository/src/main/java/org/apache/atlas/repository/audit/EntityAuditListenerV2.java
 bb510149 
  
repository/src/main/java/org/apache/atlas/repository/audit/HBaseBasedAuditRepository.java
 c5967b27 
  
repository/src/main/java/org/apache/atlas/repository/converters/AtlasInstanceConverter.java
 20147a0c 
  
repository/src/main/java/org/apache/atlas/repository/store/graph/v1/AtlasEntityChangeNotifier.java
 9aebde2b 
  
repository/src/main/java/org/apache/atlas/repository/store/graph/v1/AtlasRelationshipStoreV1.java
 51dcc3a5 
  
repository/src/main/java/org/apache/atlas/repository/store/graph/v1/DeleteHandlerV1.java
 c0f2fc3d 
  
repository/src/main/java/org/apache/atlas/repository/store/graph/v1/EntityGraphMapper.java
 43e67df7 
  
repository/src/main/java/org/apache/atlas/repository/store/graph/v1/EntityGraphRetriever.java
 7d5b7fb6 
  
repository/src/test/java/org/apache/atlas/repository/store/graph/v1/AtlasRelationshipStoreV1Test.java
 8fc0327a 
  server-api/src/main/java/org/apache/atlas/RequestContextV1.java b7c56865 
  server-api/src/main/java/org/apache/atlas/listener/EntityChangeListener.java 
19fae4fd 
  
tools/atlas-migration-exporter/src/main/java/org/apache/atlas/migration/NoOpNotificationChangeListener.java
 6c2bf220 
  
webapp/src/main/java/org/apache/atlas/notification/EntityNotificationListenerV2.java
 ac097b60 
  
webapp/src/main/java/org/apache/atlas/notification/NotificationEntityChangeListener.java
 a3e5949c 


Diff: https://reviews.apache.org/r/66797/diff/3/

Changes: https://reviews.apache.org/r/66797/diff/2-3/


Testing
---

Precommit validated: 
https://builds.apache.org/view/A/view/Atlas/job/PreCommit-ATLAS-Build-Test/307/console

Validated audit/notifications manually for propagated classification 
add/update/delete


Thanks,

Sarath Subramanian

Re: Review Request 66797: ATLAS-2609: Update audit listener to handle propagated classification add/delete/update

2018-04-25 Thread Madhan Neethiraj


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/66797/#review201979
---


Fix it, then Ship it!





repository/src/main/java/org/apache/atlas/repository/audit/EntityAuditListenerV2.java
Lines 148 (patched)


Use of isPropagatedClassificationUpdated can be avoided by having #148 - 
#150 as 'else' block at line #146.



repository/src/main/java/org/apache/atlas/repository/audit/EntityAuditListenerV2.java
Line 138 (original), 164 (patched)


Consider replacing 
"entity.getGuid().equals(classification.getEntityGuid())" with the following:

  StringUtils.equals(entity.getGuid(), classification.getEntityGuid())



repository/src/main/java/org/apache/atlas/repository/audit/EntityAuditListenerV2.java
Lines 234 (patched)


Consider replacing "equals()" with StringUtil.equals().



server-api/src/main/java/org/apache/atlas/RequestContextV1.java
Lines 110 (patched)


recordAddPropagation() ==> recordAddedPropagation()



server-api/src/main/java/org/apache/atlas/RequestContextV1.java
Lines 124 (patched)


recordRemovePropagation() ==> recordRemovedPropagation()


- Madhan Neethiraj


On April 25, 2018, 8:29 p.m., Sarath Subramanian wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/66797/
> ---
> 
> (Updated April 25, 2018, 8:29 p.m.)
> 
> 
> Review request for atlas, Apoorv Naik, Ashutosh Mestry, and Madhan Neethiraj.
> 
> 
> Bugs: ATLAS-2609
> https://issues.apache.org/jira/browse/ATLAS-2609
> 
> 
> Repository: atlas
> 
> 
> Description
> ---
> 
> Currently audit are not generated for propagated classification add/update or 
> delete.
> 
> For Audit:
> * Adding Propagated Classification   => Send 'Propagated Classifications 
> Added' message to audit repository. 
> * Removing Propagated Classification => Send 'Propagated Classifications 
> Deleted' message to audit repository.
> * Updating Propagated Classification => Send 'Propagated Classifications 
> Updated' message to audit repository.
> 
> 
> Diffs
> -
> 
>   client/client-v1/src/main/java/org/apache/atlas/EntityAuditEvent.java 
> 3cf3e212 
>   dashboardv2/public/js/utils/Enums.js 8c0518bf 
>   intg/src/main/java/org/apache/atlas/listener/EntityChangeListenerV2.java 
> 70877d22 
>   intg/src/main/java/org/apache/atlas/model/audit/EntityAuditEventV2.java 
> 1045c774 
>   
> repository/src/main/java/org/apache/atlas/repository/audit/AbstractStorageBasedAuditRepository.java
>  af6c4e62 
>   
> repository/src/main/java/org/apache/atlas/repository/audit/EntityAuditListener.java
>  1c04eea4 
>   
> repository/src/main/java/org/apache/atlas/repository/audit/EntityAuditListenerV2.java
>  bb510149 
>   
> repository/src/main/java/org/apache/atlas/repository/audit/HBaseBasedAuditRepository.java
>  c5967b27 
>   
> repository/src/main/java/org/apache/atlas/repository/converters/AtlasInstanceConverter.java
>  20147a0c 
>   
> repository/src/main/java/org/apache/atlas/repository/store/graph/v1/AtlasEntityChangeNotifier.java
>  9aebde2b 
>   
> repository/src/main/java/org/apache/atlas/repository/store/graph/v1/AtlasRelationshipStoreV1.java
>  51dcc3a5 
>   
> repository/src/main/java/org/apache/atlas/repository/store/graph/v1/DeleteHandlerV1.java
>  c0f2fc3d 
>   
> repository/src/main/java/org/apache/atlas/repository/store/graph/v1/EntityGraphMapper.java
>  4225a80d 
>   
> repository/src/main/java/org/apache/atlas/repository/store/graph/v1/EntityGraphRetriever.java
>  7d5b7fb6 
>   
> repository/src/test/java/org/apache/atlas/repository/store/graph/v1/AtlasRelationshipStoreV1Test.java
>  8fc0327a 
>   server-api/src/main/java/org/apache/atlas/RequestContextV1.java b7c56865 
>   
> server-api/src/main/java/org/apache/atlas/listener/EntityChangeListener.java 
> 19fae4fd 
>   
> tools/atlas-migration-exporter/src/main/java/org/apache/atlas/migration/NoOpNotificationChangeListener.java
>  6c2bf220 
>   
> webapp/src/main/java/org/apache/atlas/notification/EntityNotificationListenerV2.java
>  ac097b60 
>   
> webapp/src/main/java/org/apache/atlas/notification/NotificationEntityChangeListener.java
>  a3e5949c 
> 
> 
> Diff: https://reviews.apache.org/r/66797/diff/2/
> 
> 
> Testing
> ---
> 
> Precommit validated: 
> https://builds.apache.org/view/A/view/Atlas/job/PreCommit-ATLAS-Build-Test/307/console
> 
> Validated audit/notifications manually for propagated classification 
> add/update/delete
> 
> 
> Thanks,
> 
> Sarath Subramanian
> 
>

[jira] [Commented] (ATLAS-2139) Map IGC meta model and Atlas open metadata types

2018-04-25 Thread V Seth (JIRA)


[ 
https://issues.apache.org/jira/browse/ATLAS-2139?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16453276#comment-16453276
 ] 

V Seth commented on ATLAS-2139:
---

1 [~mandy_chessell] when OMRSRepositoryHelper.getSkeletonEntity() is called for 
an entity.  The supertype attributes are not being returned.  For e.g. for a 
Host type, attributes of Asset and Referenceable are not being returned.

2 Currently we are working on IGC to IGC through OMRS.  In OMRS entity and 
relationships are passed in separate methods.  In IGC for example when we 
create a table, it needs the parent rid so schema rid in this case.  What is 
the best way to pass the parent rid in OMRS when sending the entity from IGC to 
OMRS?   Passing table as entity and then a relationship between table and 
schema separately would not work because when table entity is getting saved in 
IGC (saveEntityReferenceCopy) it needs the schema rid at the same time.  Same 
thing would happen when column needs to be saved, parent table rid would be 
needed.  Are there any standards for this in OMRS?

> Map IGC meta model and Atlas open metadata types
> 
>
> Key: ATLAS-2139
> URL: https://issues.apache.org/jira/browse/ATLAS-2139
> Project: Atlas
>  Issue Type: Sub-task
>Reporter: Jerry He
>Assignee: Jerry He
>Priority: Major
> Attachments: Relational Tables and Columns.png
>
>
> Create the mapping between IGC meta model and Atlas open metadata types.
> It is used by the IGC Event Mapper.
> The outcome of this subtask can be a design document.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Re: Review Request 66804: Import: Set Relationship Defaults when Importing from pre-v1.0

2018-04-25 Thread Sarath Subramanian


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/66804/#review201974
---


Ship it!




Ship It!

- Sarath Subramanian


On April 25, 2018, 3:09 p.m., Ashutosh Mestry wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/66804/
> ---
> 
> (Updated April 25, 2018, 3:09 p.m.)
> 
> 
> Review request for atlas, Madhan Neethiraj and Sarath Subramanian.
> 
> 
> Bugs: ATLAS-2610
> https://issues.apache.org/jira/browse/ATLAS-2610
> 
> 
> Repository: atlas
> 
> 
> Description
> ---
> 
> **Cases**
> _Relationship GUID_: Import from pre-v1.0 will not have relationship guid. 
> This condition was checked before setting guid.
> _Tag Propagation_: Default for pre-v1.0 is _false_. Whereas default from v1.0 
> to v1.0 is true. This case was handled.
> _Inverse Attribute_: 
>   Case of edge creation from table to SD, table to column:
>- There are 2 edges. 
>- Edge creation from child to parent happens during regular entity 
> creation flow.
>- During child entity creation, additional check is performed by looking 
> at value of _inverseAttribute_.
>- Additional methods were added to _EntityMutationContext_ to get parent 
> entity.
> 
> 
> Diffs
> -
> 
>   
> repository/src/main/java/org/apache/atlas/repository/store/graph/v1/EntityGraphMapper.java
>  4225a80d5 
>   
> repository/src/main/java/org/apache/atlas/repository/store/graph/v1/EntityMutationContext.java
>  0a55d55e6 
>   
> repository/src/test/java/org/apache/atlas/repository/impexp/ImportServiceTest.java
>  5089070b2 
> 
> 
> Diff: https://reviews.apache.org/r/66804/diff/3/
> 
> 
> Testing
> ---
> 
> **Unit tests**
> Additional unit tests added to address new cases.
> 
> 
> Thanks,
> 
> Ashutosh Mestry
> 
>

Re: Review Request 66804: Import: Set Relationship Defaults when Importing from pre-v1.0

2018-04-25 Thread Madhan Neethiraj


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/66804/#review201973
---


Fix it, then Ship it!





repository/src/main/java/org/apache/atlas/repository/store/graph/v1/EntityGraphMapper.java
Lines 575 (patched)


ret ==> edge


- Madhan Neethiraj


On April 25, 2018, 10:09 p.m., Ashutosh Mestry wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/66804/
> ---
> 
> (Updated April 25, 2018, 10:09 p.m.)
> 
> 
> Review request for atlas, Madhan Neethiraj and Sarath Subramanian.
> 
> 
> Bugs: ATLAS-2610
> https://issues.apache.org/jira/browse/ATLAS-2610
> 
> 
> Repository: atlas
> 
> 
> Description
> ---
> 
> **Cases**
> _Relationship GUID_: Import from pre-v1.0 will not have relationship guid. 
> This condition was checked before setting guid.
> _Tag Propagation_: Default for pre-v1.0 is _false_. Whereas default from v1.0 
> to v1.0 is true. This case was handled.
> _Inverse Attribute_: 
>   Case of edge creation from table to SD, table to column:
>- There are 2 edges. 
>- Edge creation from child to parent happens during regular entity 
> creation flow.
>- During child entity creation, additional check is performed by looking 
> at value of _inverseAttribute_.
>- Additional methods were added to _EntityMutationContext_ to get parent 
> entity.
> 
> 
> Diffs
> -
> 
>   
> repository/src/main/java/org/apache/atlas/repository/store/graph/v1/EntityGraphMapper.java
>  4225a80d5 
>   
> repository/src/main/java/org/apache/atlas/repository/store/graph/v1/EntityMutationContext.java
>  0a55d55e6 
>   
> repository/src/test/java/org/apache/atlas/repository/impexp/ImportServiceTest.java
>  5089070b2 
> 
> 
> Diff: https://reviews.apache.org/r/66804/diff/2/
> 
> 
> Testing
> ---
> 
> **Unit tests**
> Additional unit tests added to address new cases.
> 
> 
> Thanks,
> 
> Ashutosh Mestry
> 
>

[jira] [Updated] (ATLAS-2610) Import Process: Import from Earlier Versions Should Set Relationship Defaults

2018-04-25 Thread Ashutosh Mestry (JIRA)


 [ 
https://issues.apache.org/jira/browse/ATLAS-2610?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ashutosh Mestry updated ATLAS-2610:
---
Attachment: ATLAS-2610-Relationship-id-during-import.-Tag-propga.patch

> Import Process: Import from Earlier Versions Should Set Relationship Defaults
> -
>
> Key: ATLAS-2610
> URL: https://issues.apache.org/jira/browse/ATLAS-2610
> Project: Atlas
>  Issue Type: Bug
>  Components:  atlas-core
>Affects Versions: trunk, 1.0.0
>Reporter: Ashutosh Mestry
>Assignee: Ashutosh Mestry
>Priority: Major
> Fix For: trunk
>
> Attachments: 
> ATLAS-2610-Relationship-id-during-import.-Tag-propga.patch
>
>
> *Scenario*
> User exports data from v0.8 or before. Importing to v1.0 should set correct 
> defaults for features specific to v1.0.
> *Cases*
>  * Relationship GUID should be present on edges. API calls like _findByGuid_ 
> should return _AtlasEntity_ with _relationshipGuid_ set.
>  * For classification, _propagateTag_  should be set to _false_ by default.
>  * Export-Import-Export verification should set inverse attributes such that 
> they referenced in parent and referring to parent from child.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Updated] (ATLAS-2610) Import Process: Import from Earlier Versions Should Set Relationship Defaults

2018-04-25 Thread Ashutosh Mestry (JIRA)


 [ 
https://issues.apache.org/jira/browse/ATLAS-2610?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ashutosh Mestry updated ATLAS-2610:
---
Attachment: (was: 
ATLAS-2610-Relationship-id-during-import.-Tag-propga.patch)

> Import Process: Import from Earlier Versions Should Set Relationship Defaults
> -
>
> Key: ATLAS-2610
> URL: https://issues.apache.org/jira/browse/ATLAS-2610
> Project: Atlas
>  Issue Type: Bug
>  Components:  atlas-core
>Affects Versions: trunk, 1.0.0
>Reporter: Ashutosh Mestry
>Assignee: Ashutosh Mestry
>Priority: Major
> Fix For: trunk
>
> Attachments: 
> ATLAS-2610-Relationship-id-during-import.-Tag-propga.patch
>
>
> *Scenario*
> User exports data from v0.8 or before. Importing to v1.0 should set correct 
> defaults for features specific to v1.0.
> *Cases*
>  * Relationship GUID should be present on edges. API calls like _findByGuid_ 
> should return _AtlasEntity_ with _relationshipGuid_ set.
>  * For classification, _propagateTag_  should be set to _false_ by default.
>  * Export-Import-Export verification should set inverse attributes such that 
> they referenced in parent and referring to parent from child.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Updated] (ATLAS-2610) Import Process: Import from Earlier Versions Should Set Relationship Defaults

2018-04-25 Thread Ashutosh Mestry (JIRA)


 [ 
https://issues.apache.org/jira/browse/ATLAS-2610?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ashutosh Mestry updated ATLAS-2610:
---
Attachment: ATLAS-2610-Relationship-id-during-import.-Tag-propga.patch

> Import Process: Import from Earlier Versions Should Set Relationship Defaults
> -
>
> Key: ATLAS-2610
> URL: https://issues.apache.org/jira/browse/ATLAS-2610
> Project: Atlas
>  Issue Type: Bug
>  Components:  atlas-core
>Affects Versions: trunk, 1.0.0
>Reporter: Ashutosh Mestry
>Assignee: Ashutosh Mestry
>Priority: Major
> Fix For: trunk
>
> Attachments: 
> ATLAS-2610-Relationship-id-during-import.-Tag-propga.patch
>
>
> *Scenario*
> User exports data from v0.8 or before. Importing to v1.0 should set correct 
> defaults for features specific to v1.0.
> *Cases*
>  * Relationship GUID should be present on edges. API calls like _findByGuid_ 
> should return _AtlasEntity_ with _relationshipGuid_ set.
>  * For classification, _propagateTag_  should be set to _false_ by default.
>  * Export-Import-Export verification should set inverse attributes such that 
> they referenced in parent and referring to parent from child.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Review Request 66804: Import: Set Relationship Defaults when Importing from pre-v1.0

2018-04-25 Thread Ashutosh Mestry


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/66804/
---

Review request for atlas, Madhan Neethiraj and Sarath Subramanian.


Bugs: ATLAS-2610
https://issues.apache.org/jira/browse/ATLAS-2610


Repository: atlas


Description
---

**Cases**
_Relationship GUID_: Import from pre-v1.0 will not have relationship guid. This 
condition was checked before setting guid.
_Tag Propagation_: Default for pre-v1.0 is _false_. Whereas default from v1.0 
to v1.0 is true. This case was handled.
_Inverse Attribute_: 
  Case of edge creation from table to SD, table to column:
   - There are 2 edges. 
   - Edge creation from child to parent happens during regular entity creation 
flow.
   - During child entity creation, additional check is performed by looking at 
value of _inverseAttribute_.
   - Additional methods were added to _EntityMutationContext_ to get parent 
entity.


Diffs
-

  
repository/src/main/java/org/apache/atlas/repository/store/graph/v1/EntityGraphMapper.java
 4225a80d5 
  
repository/src/main/java/org/apache/atlas/repository/store/graph/v1/EntityMutationContext.java
 0a55d55e6 
  
repository/src/test/java/org/apache/atlas/repository/impexp/ImportServiceTest.java
 5089070b2 


Diff: https://reviews.apache.org/r/66804/diff/1/


Testing
---

**Unit tests**
Additional unit tests added to address new cases.


Thanks,

Ashutosh Mestry

[jira] [Updated] (ATLAS-2610) Import Process: Import from Earlier Versions Should Set Relationship Defaults

2018-04-25 Thread Ashutosh Mestry (JIRA)


 [ 
https://issues.apache.org/jira/browse/ATLAS-2610?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ashutosh Mestry updated ATLAS-2610:
---
Description: 
*Scenario*

User exports data from v0.8 or before. Importing to v1.0 should set correct 
defaults for features specific to v1.0.

*Cases*
 * Relationship GUID should be present on edges. API calls like _findByGuid_ 
should return _AtlasEntity_ with _relationshipGuid_ set.
 * For classification, _propagateTag_  should be set to _false_ by default.
 * Export-Import-Export verification should set inverse attributes such that 
they referenced in parent and referring to parent from child.

  was:
*Scenario*

User exports data from v0.8 or before. Importing to v1.0 should set correct 
defaults for features specific to v1.0.

*Cases*
 * Relationship GUID should be present on edges. API calls like _findByGuid_ 
should return _AtlasEntity_ with _relationshipGuid_ set.
 * For classification, _propogateTag_ should be set to _false_ by default.


> Import Process: Import from Earlier Versions Should Set Relationship Defaults
> -
>
> Key: ATLAS-2610
> URL: https://issues.apache.org/jira/browse/ATLAS-2610
> Project: Atlas
>  Issue Type: Bug
>  Components:  atlas-core
>Affects Versions: trunk, 1.0.0
>Reporter: Ashutosh Mestry
>Assignee: Ashutosh Mestry
>Priority: Major
> Fix For: trunk
>
>
> *Scenario*
> User exports data from v0.8 or before. Importing to v1.0 should set correct 
> defaults for features specific to v1.0.
> *Cases*
>  * Relationship GUID should be present on edges. API calls like _findByGuid_ 
> should return _AtlasEntity_ with _relationshipGuid_ set.
>  * For classification, _propagateTag_  should be set to _false_ by default.
>  * Export-Import-Export verification should set inverse attributes such that 
> they referenced in parent and referring to parent from child.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Created] (ATLAS-2610) Import Process: Import from Earlier Versions Should Set Relationship Defaults

2018-04-25 Thread Ashutosh Mestry (JIRA)

Ashutosh Mestry created ATLAS-2610:
--

 Summary: Import Process: Import from Earlier Versions Should Set 
Relationship Defaults
 Key: ATLAS-2610
 URL: https://issues.apache.org/jira/browse/ATLAS-2610
 Project: Atlas
  Issue Type: Bug
  Components:  atlas-core
Affects Versions: trunk, 1.0.0
Reporter: Ashutosh Mestry
Assignee: Ashutosh Mestry
 Fix For: trunk


*Scenario*

User exports data from v0.8 or before. Importing to v1.0 should set correct 
defaults for features specific to v1.0.

*Cases*
 * Relationship GUID should be present on edges. API calls like _findByGuid_ 
should return _AtlasEntity_ with _relationshipGuid_ set.
 * For classification, _propogateTag_ should be set to _false_ by default.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Re: Review Request 66797: ATLAS-2609: Update audit and notification listener to handle propagated classification add/delete/update

2018-04-25 Thread Sarath Subramanian


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/66797/
---

(Updated April 25, 2018, 1:29 p.m.)


Review request for atlas, Apoorv Naik, Ashutosh Mestry, and Madhan Neethiraj.


Bugs: ATLAS-2609
https://issues.apache.org/jira/browse/ATLAS-2609


Repository: atlas


Description (updated)
---

Currently audit are not generated for propagated classification add/update or 
delete.

For Audit:
* Adding Propagated Classification   => Send 'Propagated Classifications Added' 
message to audit repository. 
* Removing Propagated Classification => Send 'Propagated Classifications 
Deleted' message to audit repository.
* Updating Propagated Classification => Send 'Propagated Classifications 
Updated' message to audit repository.


Diffs
-

  client/client-v1/src/main/java/org/apache/atlas/EntityAuditEvent.java 
3cf3e212 
  dashboardv2/public/js/utils/Enums.js 8c0518bf 
  intg/src/main/java/org/apache/atlas/listener/EntityChangeListenerV2.java 
70877d22 
  intg/src/main/java/org/apache/atlas/model/audit/EntityAuditEventV2.java 
1045c774 
  
repository/src/main/java/org/apache/atlas/repository/audit/AbstractStorageBasedAuditRepository.java
 af6c4e62 
  
repository/src/main/java/org/apache/atlas/repository/audit/EntityAuditListener.java
 1c04eea4 
  
repository/src/main/java/org/apache/atlas/repository/audit/EntityAuditListenerV2.java
 bb510149 
  
repository/src/main/java/org/apache/atlas/repository/audit/HBaseBasedAuditRepository.java
 c5967b27 
  
repository/src/main/java/org/apache/atlas/repository/converters/AtlasInstanceConverter.java
 20147a0c 
  
repository/src/main/java/org/apache/atlas/repository/store/graph/v1/AtlasEntityChangeNotifier.java
 9aebde2b 
  
repository/src/main/java/org/apache/atlas/repository/store/graph/v1/AtlasRelationshipStoreV1.java
 51dcc3a5 
  
repository/src/main/java/org/apache/atlas/repository/store/graph/v1/DeleteHandlerV1.java
 c0f2fc3d 
  
repository/src/main/java/org/apache/atlas/repository/store/graph/v1/EntityGraphMapper.java
 4225a80d 
  
repository/src/main/java/org/apache/atlas/repository/store/graph/v1/EntityGraphRetriever.java
 7d5b7fb6 
  
repository/src/test/java/org/apache/atlas/repository/store/graph/v1/AtlasRelationshipStoreV1Test.java
 8fc0327a 
  server-api/src/main/java/org/apache/atlas/RequestContextV1.java b7c56865 
  server-api/src/main/java/org/apache/atlas/listener/EntityChangeListener.java 
19fae4fd 
  
tools/atlas-migration-exporter/src/main/java/org/apache/atlas/migration/NoOpNotificationChangeListener.java
 6c2bf220 
  
webapp/src/main/java/org/apache/atlas/notification/EntityNotificationListenerV2.java
 ac097b60 
  
webapp/src/main/java/org/apache/atlas/notification/NotificationEntityChangeListener.java
 a3e5949c 


Diff: https://reviews.apache.org/r/66797/diff/2/


Testing
---

Precommit validated: 
https://builds.apache.org/view/A/view/Atlas/job/PreCommit-ATLAS-Build-Test/307/console

Validated audit/notifications manually for propagated classification 
add/update/delete


Thanks,

Sarath Subramanian

[jira] [Updated] (ATLAS-2609) Update audit listener to handle propagated classification add/delete/update

2018-04-25 Thread Sarath Subramanian (JIRA)


 [ 
https://issues.apache.org/jira/browse/ATLAS-2609?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sarath Subramanian updated ATLAS-2609:
--
Summary: Update audit listener to handle propagated classification 
add/delete/update  (was: Update audit and notification listener to handle 
propagated classification add/delete/update)

> Update audit listener to handle propagated classification add/delete/update
> ---
>
> Key: ATLAS-2609
> URL: https://issues.apache.org/jira/browse/ATLAS-2609
> Project: Atlas
>  Issue Type: Bug
>  Components:  atlas-core
>Affects Versions: 1.0.0
>Reporter: Sarath Subramanian
>Assignee: Sarath Subramanian
>Priority: Major
> Fix For: 1.0.0
>
>
> Currently audit and notifications are not generated for propagated 
> classification add/update or delete.
>  For Audit:
>  * Adding Propagated Classification      => Send '_Propagated Classifications 
> Added_' message to audit repository. 
>  * Removing Propagated Classification =>  Send '_Propagated Classifications 
> Deleted_' message to audit repository.
>  * Updating Propagated Classification   => Send '_Propagated Classifications 
> Updated_' message to audit repository.
>  For Notification:
>  * Adding Propagated Classification      =>  Send 
> *PROPAGATED_CLASSIFICATION_ADD* message type.
>  * Removing Propagated Classification =>  Send 
> *PROPAGATED_CLASSIFICATION_DELETE* message type.
>  * Updating Propagated Classification   => Send 
> *PROPAGATED_CLASSIFICATION_UPDATE* message type.
>  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Updated] (ATLAS-2609) Update audit listener to handle propagated classification add/delete/update

2018-04-25 Thread Sarath Subramanian (JIRA)


 [ 
https://issues.apache.org/jira/browse/ATLAS-2609?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sarath Subramanian updated ATLAS-2609:
--
Description: 
Currently audit events are not generated for propagated classification 
add/update or delete.

 For Audit:
 * Adding Propagated Classification      => Send '_Propagated Classifications 
Added_' message to audit repository. 
 * Removing Propagated Classification =>  Send '_Propagated Classifications 
Deleted_' message to audit repository.
 * Updating Propagated Classification   => Send '_Propagated Classifications 
Updated_' message to audit repository

 

  was:
Currently audit and notifications are not generated for propagated 
classification add/update or delete.

 For Audit:
 * Adding Propagated Classification      => Send '_Propagated Classifications 
Added_' message to audit repository. 
 * Removing Propagated Classification =>  Send '_Propagated Classifications 
Deleted_' message to audit repository.
 * Updating Propagated Classification   => Send '_Propagated Classifications 
Updated_' message to audit repository.

 For Notification:
 * Adding Propagated Classification      =>  Send 
*PROPAGATED_CLASSIFICATION_ADD* message type.
 * Removing Propagated Classification =>  Send 
*PROPAGATED_CLASSIFICATION_DELETE* message type.
 * Updating Propagated Classification   => Send 
*PROPAGATED_CLASSIFICATION_UPDATE* message type.

 


> Update audit listener to handle propagated classification add/delete/update
> ---
>
> Key: ATLAS-2609
> URL: https://issues.apache.org/jira/browse/ATLAS-2609
> Project: Atlas
>  Issue Type: Bug
>  Components:  atlas-core
>Affects Versions: 1.0.0
>Reporter: Sarath Subramanian
>Assignee: Sarath Subramanian
>Priority: Major
> Fix For: 1.0.0
>
>
> Currently audit events are not generated for propagated classification 
> add/update or delete.
>  For Audit:
>  * Adding Propagated Classification      => Send '_Propagated Classifications 
> Added_' message to audit repository. 
>  * Removing Propagated Classification =>  Send '_Propagated Classifications 
> Deleted_' message to audit repository.
>  * Updating Propagated Classification   => Send '_Propagated Classifications 
> Updated_' message to audit repository
>  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Re: Review Request 66797: ATLAS-2609: Update audit and notification listener to handle propagated classification add/delete/update

2018-04-25 Thread Sarath Subramanian


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/66797/
---

(Updated April 25, 2018, 1:29 p.m.)


Review request for atlas, Apoorv Naik, Ashutosh Mestry, and Madhan Neethiraj.


Changes
---

removed notification listener changes for propagated classification 
add/update/delete


Bugs: ATLAS-2609
https://issues.apache.org/jira/browse/ATLAS-2609


Repository: atlas


Description
---

Currently audit and notifications are not generated for propagated 
classification add/update or delete.

For Audit:
* Adding Propagated Classification   => Send 'Propagated Classifications Added' 
message to audit repository. 
* Removing Propagated Classification => Send 'Propagated Classifications 
Deleted' message to audit repository.
* Updating Propagated Classification => Send 'Propagated Classifications 
Updated' message to audit repository.

For Notification:
* Adding Propagated Classification   => Send PROPAGATED_CLASSIFICATION_ADD 
message type.
* Removing Propagated Classification => Send PROPAGATED_CLASSIFICATION_DELETE 
message type.
* Updating Propagated Classification => Send PROPAGATED_CLASSIFICATION_UPDATE 
message type.


Diffs (updated)
-

  client/client-v1/src/main/java/org/apache/atlas/EntityAuditEvent.java 
3cf3e212 
  dashboardv2/public/js/utils/Enums.js 8c0518bf 
  intg/src/main/java/org/apache/atlas/listener/EntityChangeListenerV2.java 
70877d22 
  intg/src/main/java/org/apache/atlas/model/audit/EntityAuditEventV2.java 
1045c774 
  
repository/src/main/java/org/apache/atlas/repository/audit/AbstractStorageBasedAuditRepository.java
 af6c4e62 
  
repository/src/main/java/org/apache/atlas/repository/audit/EntityAuditListener.java
 1c04eea4 
  
repository/src/main/java/org/apache/atlas/repository/audit/EntityAuditListenerV2.java
 bb510149 
  
repository/src/main/java/org/apache/atlas/repository/audit/HBaseBasedAuditRepository.java
 c5967b27 
  
repository/src/main/java/org/apache/atlas/repository/converters/AtlasInstanceConverter.java
 20147a0c 
  
repository/src/main/java/org/apache/atlas/repository/store/graph/v1/AtlasEntityChangeNotifier.java
 9aebde2b 
  
repository/src/main/java/org/apache/atlas/repository/store/graph/v1/AtlasRelationshipStoreV1.java
 51dcc3a5 
  
repository/src/main/java/org/apache/atlas/repository/store/graph/v1/DeleteHandlerV1.java
 c0f2fc3d 
  
repository/src/main/java/org/apache/atlas/repository/store/graph/v1/EntityGraphMapper.java
 4225a80d 
  
repository/src/main/java/org/apache/atlas/repository/store/graph/v1/EntityGraphRetriever.java
 7d5b7fb6 
  
repository/src/test/java/org/apache/atlas/repository/store/graph/v1/AtlasRelationshipStoreV1Test.java
 8fc0327a 
  server-api/src/main/java/org/apache/atlas/RequestContextV1.java b7c56865 
  server-api/src/main/java/org/apache/atlas/listener/EntityChangeListener.java 
19fae4fd 
  
tools/atlas-migration-exporter/src/main/java/org/apache/atlas/migration/NoOpNotificationChangeListener.java
 6c2bf220 
  
webapp/src/main/java/org/apache/atlas/notification/EntityNotificationListenerV2.java
 ac097b60 
  
webapp/src/main/java/org/apache/atlas/notification/NotificationEntityChangeListener.java
 a3e5949c 


Diff: https://reviews.apache.org/r/66797/diff/2/

Changes: https://reviews.apache.org/r/66797/diff/1-2/


Testing
---

Precommit validated: 
https://builds.apache.org/view/A/view/Atlas/job/PreCommit-ATLAS-Build-Test/307/console

Validated audit/notifications manually for propagated classification 
add/update/delete


Thanks,

Sarath Subramanian

[jira] [Comment Edited] (ATLAS-2607) Classification lifecycle through metadata properties and relationships

2018-04-25 Thread Madhan Neethiraj (JIRA)


[ 
https://issues.apache.org/jira/browse/ATLAS-2607?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16452955#comment-16452955
 ] 

Madhan Neethiraj edited comment on ATLAS-2607 at 4/25/18 7:45 PM:
--

bq. Asset can only be attached to classifications with the status of ACTIVE
This implies that the status is on the classification-type - and not on the 
entity-classification association. I think having the status for 
entity-classification association would be more useful - like: PII => 
employee.ssn, PII => employee.phone_num, PII => customer.name

bq. All new classifications after this change will start out with DRAFT status
Clients that use current APIs will now know of this new attribute 'status'; 
they expect the classifications given in the API to be active, I think the 
default should be ACTIVE. Newer clients can choose to specify different value 
for status.

bq.  a steward or an admin with appropriate permissions can move them into an 
ACTIVE state
Existing entity-classification-update permission will be used to enforce change 
to status; new new permission will be introduced for this. If this becomes 
necessary, I would suggest we take this up in a separate JIRA, post 1.0 release.



was (Author: madhan.neethiraj):
bq. Asset can only be attached to classifications with the status of ACTIVE
This implies that the status is on the classification-type - and not on the 
entity-classification association. I think having the status for 
entity-classification association would be more useful - like: PII => 
employee.ssn, PII => employee.phone_num, PII => customer.name

bq. All new classifications after this change will start out with DRAFT status 
and a steward or an admin with appropriate permissions can move them into an 
ACTIVE state
Clients that use current APIs will now know of this new attribute 'status'; 
they expect the classifications given in the API to be active, I think the 
default should be ACTIVE. Newer clients can choose to specify different value 
for status. 


> Classification lifecycle through metadata properties and relationships
> --
>
> Key: ATLAS-2607
> URL: https://issues.apache.org/jira/browse/ATLAS-2607
> Project: Atlas
>  Issue Type: Improvement
>  Components:  atlas-core
>Reporter: Srikanth Venkat
>Assignee: Madhan Neethiraj
>Priority: Critical
>
> Currently tags or classifications in Atlas are considered active once they 
> are defined. For governance and stewardship purposes, it would be important 
> to attach the notion of what state in its lifecycle a particular 
> classification is. This would help with workflows to manage the lifecycle 
> aspects and provide any filtering needed to take appropriate actions. For 
> example only active classifications should be considered for classification 
> based policy enforcement. Additionally lifecycle status would help with 
> filtering and search as well as reporting and compliance/audit scenarios.
> Implementation Proposal:
>  * All tags or classifications have a "Lifecycle Status" property
>  * They can go through the following list of states during their lifecycle: 
> DRAFT -> ACTIVE  -> RETIRED
>  * Lifecycle Status can be set as an enum property that is mandatory or 
> required for all classifications.
>  * All existing classifications already present in Atlas before this change 
> will default to an ACTIVE status so that all pre-existing classifications 
> will continue to work as before.
>  * All new classifications after this change will start out with DRAFT status 
> and a steward or an admin with appropriate permissions can move them into an 
> ACTIVE state (controlled via Metadata security policies)
>  * Policy enforcement for authorization on classifications can ignore any 
> that are not in ACTIVE state. 
>  * Asset can only be attached to classifications with the status of ACTIVE
>  * For a classification in RETIRED state, we might have an optional 
> relationship with another classification called "Replaced By" which is the 
> new classification that the current one was remapped or replaced with. The 
> inverse relationship could be labeled "Replaces" which is on the new 
> classification and points to the removed classification that it replaces. 
>  * The state RETIRED implies this classification is no longer used and policy 
> enforcement will ignore any classifications in such states. 
>  * Additionally UI can filter and show by default only classifications that 
> are not RETIRED and a checkbox to "Show Retired"
>  * Deletion of classifications should work as it currently does with no 
> behavioral changes.
>  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (ATLAS-2607) Classification lifecycle through metadata properties and relationships

2018-04-25 Thread Madhan Neethiraj (JIRA)


[ 
https://issues.apache.org/jira/browse/ATLAS-2607?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16452955#comment-16452955
 ] 

Madhan Neethiraj commented on ATLAS-2607:
-

bq. Asset can only be attached to classifications with the status of ACTIVE
This implies that the status is on the classification-type - and not on the 
entity-classification association. I think having the status for 
entity-classification association would be more useful - like: PII => 
employee.ssn, PII => employee.phone_num, PII => customer.name

bq. All new classifications after this change will start out with DRAFT status 
and a steward or an admin with appropriate permissions can move them into an 
ACTIVE state
Clients that use current APIs will now know of this new attribute 'status'; 
they expect the classifications given in the API to be active, I think the 
default should be ACTIVE. Newer clients can choose to specify different value 
for status. 


> Classification lifecycle through metadata properties and relationships
> --
>
> Key: ATLAS-2607
> URL: https://issues.apache.org/jira/browse/ATLAS-2607
> Project: Atlas
>  Issue Type: Improvement
>  Components:  atlas-core
>Reporter: Srikanth Venkat
>Assignee: Madhan Neethiraj
>Priority: Critical
>
> Currently tags or classifications in Atlas are considered active once they 
> are defined. For governance and stewardship purposes, it would be important 
> to attach the notion of what state in its lifecycle a particular 
> classification is. This would help with workflows to manage the lifecycle 
> aspects and provide any filtering needed to take appropriate actions. For 
> example only active classifications should be considered for classification 
> based policy enforcement. Additionally lifecycle status would help with 
> filtering and search as well as reporting and compliance/audit scenarios.
> Implementation Proposal:
>  * All tags or classifications have a "Lifecycle Status" property
>  * They can go through the following list of states during their lifecycle: 
> DRAFT -> ACTIVE  -> RETIRED
>  * Lifecycle Status can be set as an enum property that is mandatory or 
> required for all classifications.
>  * All existing classifications already present in Atlas before this change 
> will default to an ACTIVE status so that all pre-existing classifications 
> will continue to work as before.
>  * All new classifications after this change will start out with DRAFT status 
> and a steward or an admin with appropriate permissions can move them into an 
> ACTIVE state (controlled via Metadata security policies)
>  * Policy enforcement for authorization on classifications can ignore any 
> that are not in ACTIVE state. 
>  * Asset can only be attached to classifications with the status of ACTIVE
>  * For a classification in RETIRED state, we might have an optional 
> relationship with another classification called "Replaced By" which is the 
> new classification that the current one was remapped or replaced with. The 
> inverse relationship could be labeled "Replaces" which is on the new 
> classification and points to the removed classification that it replaces. 
>  * The state RETIRED implies this classification is no longer used and policy 
> enforcement will ignore any classifications in such states. 
>  * Additionally UI can filter and show by default only classifications that 
> are not RETIRED and a checkbox to "Show Retired"
>  * Deletion of classifications should work as it currently does with no 
> behavioral changes.
>  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Resolved] (ATLAS-2602) Import: Glossary Import Does not Display Imported Entities

2018-04-25 Thread Ashutosh Mestry (JIRA)


 [ 
https://issues.apache.org/jira/browse/ATLAS-2602?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ashutosh Mestry resolved ATLAS-2602.

Resolution: Fixed

> Import: Glossary Import Does not Display Imported Entities
> --
>
> Key: ATLAS-2602
> URL: https://issues.apache.org/jira/browse/ATLAS-2602
> Project: Atlas
>  Issue Type: Bug
>  Components:  atlas-core
>Affects Versions: trunk, 1.0.0
>Reporter: Ashutosh Mestry
>Assignee: Ashutosh Mestry
>Priority: Major
> Fix For: trunk
>
> Attachments: 
> ATLAS-2602-Walkthrough-includes-relationship-attribu.patch
>
>
> *Pre-requisites*
>  * Create Glossary hierarchy with few terms.
>  * Export to ZIP using Atlas Export.
> *Steps to Duplicate*
>  * Import the file create in previous step.
>  * Open Atlas Web UI and navigate to Glossary tab. 
> _Expected:_ All terms created are visible.
> _Actual_: Few of the terms will be visible from in the tab.
> _Additional information:_  Using JanusGraph Gremlin shell, it appears that 
> the vertices are created but the edges (containing relationships) are missing.
>  
>  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Re: Review Request 66791: Import: Fix for Glossary Terms

2018-04-25 Thread Madhan Neethiraj


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/66791/#review201951
---


Ship it!




Ship It!

- Madhan Neethiraj


On April 25, 2018, 5:49 p.m., Ashutosh Mestry wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/66791/
> ---
> 
> (Updated April 25, 2018, 5:49 p.m.)
> 
> 
> Review request for atlas, Madhan Neethiraj and Sarath Subramanian.
> 
> 
> Bugs: ATLAS-2602
> https://issues.apache.org/jira/browse/ATLAS-2602
> 
> 
> Repository: atlas
> 
> 
> Description
> ---
> 
> **Background**
> During entity creation, has a pre-create phase that discovers entities that 
> need to be created within _AtlasEntity_. For import, this discovery was 
> ignoring the entities present within relationship attribute.
> 
> **Fix**
> The fix removes the condition.
> 
> 
> Diffs
> -
> 
>   
> repository/src/main/java/org/apache/atlas/repository/store/graph/v1/AtlasEntityGraphDiscoveryV1.java
>  f092d1847 
>   
> repository/src/main/java/org/apache/atlas/repository/store/graph/v1/BulkImporterImpl.java
>  af310954f 
>   
> repository/src/test/java/org/apache/atlas/repository/impexp/ImportServiceTest.java
>  91d56ab80 
>   repository/src/test/resources/stocks-glossary.zip PRE-CREATION 
> 
> 
> Diff: https://reviews.apache.org/r/66791/diff/3/
> 
> 
> Testing
> ---
> 
> **Unit tests**
> - New unit tests cover this case.
> - Additional checks added using EntityDiscovery to query the Graph with known 
> guids after import succeeds.
> 
> 
> Thanks,
> 
> Ashutosh Mestry
> 
>

Re: Review Request 66791: Import: Fix for Glossary Terms

2018-04-25 Thread Ashutosh Mestry


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/66791/
---

(Updated April 25, 2018, 5:49 p.m.)


Review request for atlas, Madhan Neethiraj and Sarath Subramanian.


Changes
---

Updates include: Addressed review comments.


Bugs: ATLAS-2602
https://issues.apache.org/jira/browse/ATLAS-2602


Repository: atlas


Description
---

**Background**
During entity creation, has a pre-create phase that discovers entities that 
need to be created within _AtlasEntity_. For import, this discovery was 
ignoring the entities present within relationship attribute.

**Fix**
The fix removes the condition.


Diffs (updated)
-

  
repository/src/main/java/org/apache/atlas/repository/store/graph/v1/AtlasEntityGraphDiscoveryV1.java
 f092d1847 
  
repository/src/main/java/org/apache/atlas/repository/store/graph/v1/BulkImporterImpl.java
 af310954f 
  
repository/src/test/java/org/apache/atlas/repository/impexp/ImportServiceTest.java
 91d56ab80 
  repository/src/test/resources/stocks-glossary.zip PRE-CREATION 


Diff: https://reviews.apache.org/r/66791/diff/3/

Changes: https://reviews.apache.org/r/66791/diff/2-3/


Testing
---

**Unit tests**
- New unit tests cover this case.
- Additional checks added using EntityDiscovery to query the Graph with known 
guids after import succeeds.


Thanks,

Ashutosh Mestry

[jira] [Resolved] (ATLAS-2317) [Docs] Add HBase Bridge Documents

2018-04-25 Thread Madhan Neethiraj (JIRA)


 [ 
https://issues.apache.org/jira/browse/ATLAS-2317?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Madhan Neethiraj resolved ATLAS-2317.
-
Resolution: Fixed

Committed to master: 
http://git-wip-us.apache.org/repos/asf/atlas/commit/540129f5

> [Docs] Add HBase Bridge Documents
> -
>
> Key: ATLAS-2317
> URL: https://issues.apache.org/jira/browse/ATLAS-2317
> Project: Atlas
>  Issue Type: Bug
>Affects Versions: 0.8.1
>Reporter: ShuMing Li
>Assignee: Ramesh Mani
>Priority: Major
> Fix For: 1.0.0
>
> Attachments: 0001-ATLAS-2317-Docs-Add-HBase-Bridge-Documents.patch
>
>
> Since HBase Bridge has been already 
> supported(https://issues.apache.org/jira/browse/ATLAS-1805), We should update 
> index 
> docs(https://github.com/apache/atlas/blob/master/docs/src/site/twiki/index.twiki)
>  to add HBase Config knowledge.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Updated] (ATLAS-2603) [Docs] Add Atlas Kafka Bridge Documents

2018-04-25 Thread Ramesh Mani (JIRA)


 [ 
https://issues.apache.org/jira/browse/ATLAS-2603?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ramesh Mani updated ATLAS-2603:
---
Attachment: 0001-ATLAS-2603-Docs-Add-Atlas-Kafka-Bridge-Documents_003.patch

> [Docs] Add Atlas Kafka Bridge Documents
> ---
>
> Key: ATLAS-2603
> URL: https://issues.apache.org/jira/browse/ATLAS-2603
> Project: Atlas
>  Issue Type: Bug
>Reporter: Ramesh Mani
>Assignee: Ramesh Mani
>Priority: Major
> Fix For: 1.0.0
>
> Attachments: 
> 0001-ATLAS-2603-Docs-Add-Atlas-Kafka-Bridge-Documents.patch, 
> 0001-ATLAS-2603-Docs-Add-Atlas-Kafka-Bridge-Documents_002.patch, 
> 0001-ATLAS-2603-Docs-Add-Atlas-Kafka-Bridge-Documents_003.patch
>
>
> [Docs] Add Atlas Kafka Bridge Documents



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Re: Review Request 66785: ATLAS-2603: [Docs] Add Atlas Kafka Bridge Documents

2018-04-25 Thread Madhan Neethiraj


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/66785/#review201950
---


Ship it!




Ship It!

- Madhan Neethiraj


On April 25, 2018, 5:30 p.m., Ramesh Mani wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/66785/
> ---
> 
> (Updated April 25, 2018, 5:30 p.m.)
> 
> 
> Review request for atlas, Apoorv Naik, Madhan Neethiraj, and Sarath 
> Subramanian.
> 
> 
> Bugs: ATLAS-2603
> https://issues.apache.org/jira/browse/ATLAS-2603
> 
> 
> Repository: atlas
> 
> 
> Description
> ---
> 
> ATLAS-2603: [Docs] Add Atlas Kafka Bridge Documents
> 
> 
> Diffs
> -
> 
>   docs/src/site/twiki/Bridge-Kafka.twiki PRE-CREATION 
> 
> 
> Diff: https://reviews.apache.org/r/66785/diff/3/
> 
> 
> Testing
> ---
> 
> 
> Thanks,
> 
> Ramesh Mani
> 
>

Re: Review Request 66785: ATLAS-2603: [Docs] Add Atlas Kafka Bridge Documents

2018-04-25 Thread Ramesh Mani


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/66785/
---

(Updated April 25, 2018, 5:30 p.m.)


Review request for atlas, Apoorv Naik, Madhan Neethiraj, and Sarath Subramanian.


Changes
---

Fixed review comments


Bugs: ATLAS-2603
https://issues.apache.org/jira/browse/ATLAS-2603


Repository: atlas


Description
---

ATLAS-2603: [Docs] Add Atlas Kafka Bridge Documents


Diffs (updated)
-

  docs/src/site/twiki/Bridge-Kafka.twiki PRE-CREATION 


Diff: https://reviews.apache.org/r/66785/diff/3/

Changes: https://reviews.apache.org/r/66785/diff/2-3/


Testing
---


Thanks,

Ramesh Mani

Re: Review Request 66797: ATLAS-2609: Update audit and notification listener to handle propagated classification add/delete/update

2018-04-25 Thread Madhan Neethiraj


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/66797/#review201947
---




webapp/src/main/java/org/apache/atlas/notification/EntityNotificationListenerV2.java
Lines 125 (patched)


Can 'classifications' list contain some directly associated classifications 
and some propagated? If yes, shouldn't 2 notifications & 2 audits be created?


- Madhan Neethiraj


On April 25, 2018, 8:40 a.m., Sarath Subramanian wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/66797/
> ---
> 
> (Updated April 25, 2018, 8:40 a.m.)
> 
> 
> Review request for atlas, Apoorv Naik, Ashutosh Mestry, and Madhan Neethiraj.
> 
> 
> Bugs: ATLAS-2609
> https://issues.apache.org/jira/browse/ATLAS-2609
> 
> 
> Repository: atlas
> 
> 
> Description
> ---
> 
> Currently audit and notifications are not generated for propagated 
> classification add/update or delete.
> 
> For Audit:
> * Adding Propagated Classification   => Send 'Propagated Classifications 
> Added' message to audit repository. 
> * Removing Propagated Classification => Send 'Propagated Classifications 
> Deleted' message to audit repository.
> * Updating Propagated Classification => Send 'Propagated Classifications 
> Updated' message to audit repository.
> 
> For Notification:
> * Adding Propagated Classification   => Send PROPAGATED_CLASSIFICATION_ADD 
> message type.
> * Removing Propagated Classification => Send PROPAGATED_CLASSIFICATION_DELETE 
> message type.
> * Updating Propagated Classification => Send PROPAGATED_CLASSIFICATION_UPDATE 
> message type.
> 
> 
> Diffs
> -
> 
>   client/client-v1/src/main/java/org/apache/atlas/EntityAuditEvent.java 
> 3cf3e212 
>   dashboardv2/public/js/utils/Enums.js 8c0518bf 
>   intg/src/main/java/org/apache/atlas/listener/EntityChangeListenerV2.java 
> 70877d22 
>   intg/src/main/java/org/apache/atlas/model/audit/EntityAuditEventV2.java 
> 1045c774 
>   
> intg/src/main/java/org/apache/atlas/v1/model/notification/EntityNotificationV2.java
>  a8dfd235 
>   
> repository/src/main/java/org/apache/atlas/repository/audit/EntityAuditListener.java
>  1c04eea4 
>   
> repository/src/main/java/org/apache/atlas/repository/audit/EntityAuditListenerV2.java
>  bb510149 
>   
> repository/src/main/java/org/apache/atlas/repository/converters/AtlasInstanceConverter.java
>  20147a0c 
>   
> repository/src/main/java/org/apache/atlas/repository/store/graph/v1/AtlasEntityChangeNotifier.java
>  9aebde2b 
>   
> repository/src/main/java/org/apache/atlas/repository/store/graph/v1/AtlasRelationshipStoreV1.java
>  51dcc3a5 
>   
> repository/src/main/java/org/apache/atlas/repository/store/graph/v1/DeleteHandlerV1.java
>  c0f2fc3d 
>   
> repository/src/main/java/org/apache/atlas/repository/store/graph/v1/EntityGraphMapper.java
>  4225a80d 
>   
> repository/src/main/java/org/apache/atlas/repository/store/graph/v1/EntityGraphRetriever.java
>  7d5b7fb6 
>   
> repository/src/test/java/org/apache/atlas/repository/store/graph/v1/AtlasRelationshipStoreV1Test.java
>  8fc0327a 
>   server-api/src/main/java/org/apache/atlas/RequestContextV1.java b7c56865 
>   
> server-api/src/main/java/org/apache/atlas/listener/EntityChangeListener.java 
> 19fae4fd 
>   
> tools/atlas-migration-exporter/src/main/java/org/apache/atlas/migration/NoOpNotificationChangeListener.java
>  6c2bf220 
>   
> webapp/src/main/java/org/apache/atlas/notification/EntityNotificationListenerV2.java
>  ac097b60 
>   
> webapp/src/main/java/org/apache/atlas/notification/NotificationEntityChangeListener.java
>  a3e5949c 
> 
> 
> Diff: https://reviews.apache.org/r/66797/diff/1/
> 
> 
> Testing
> ---
> 
> Precommit in progress: 
> https://builds.apache.org/view/A/view/Atlas/job/PreCommit-ATLAS-Build-Test/307/console
> 
> Validated audit/notifications manually for propagated classification 
> add/update/delete
> 
> 
> Thanks,
> 
> Sarath Subramanian
> 
>

Re: Review Request 66791: Import: Fix for Glossary Terms

2018-04-25 Thread Madhan Neethiraj


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/66791/#review201946
---


Fix it, then Ship it!





repository/src/main/java/org/apache/atlas/repository/store/graph/v1/BulkImporterImpl.java
Lines 78 (patched)


Consider moving line #78 before line #76.


- Madhan Neethiraj


On April 25, 2018, 4:35 p.m., Ashutosh Mestry wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/66791/
> ---
> 
> (Updated April 25, 2018, 4:35 p.m.)
> 
> 
> Review request for atlas, Madhan Neethiraj and Sarath Subramanian.
> 
> 
> Bugs: ATLAS-2602
> https://issues.apache.org/jira/browse/ATLAS-2602
> 
> 
> Repository: atlas
> 
> 
> Description
> ---
> 
> **Background**
> During entity creation, has a pre-create phase that discovers entities that 
> need to be created within _AtlasEntity_. For import, this discovery was 
> ignoring the entities present within relationship attribute.
> 
> **Fix**
> The fix removes the condition.
> 
> 
> Diffs
> -
> 
>   
> repository/src/main/java/org/apache/atlas/repository/store/graph/v1/AtlasEntityGraphDiscoveryV1.java
>  f092d1847 
>   
> repository/src/main/java/org/apache/atlas/repository/store/graph/v1/BulkImporterImpl.java
>  af310954f 
>   
> repository/src/test/java/org/apache/atlas/repository/impexp/ImportServiceTest.java
>  91d56ab80 
>   repository/src/test/resources/stocks-glossary.zip PRE-CREATION 
> 
> 
> Diff: https://reviews.apache.org/r/66791/diff/2/
> 
> 
> Testing
> ---
> 
> **Unit tests**
> - New unit tests cover this case.
> - Additional checks added using EntityDiscovery to query the Graph with known 
> guids after import succeeds.
> 
> 
> Thanks,
> 
> Ashutosh Mestry
> 
>

Re: Review Request 66791: Import: Fix for Glossary Terms

2018-04-25 Thread Ashutosh Mestry


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/66791/
---

(Updated April 25, 2018, 4:35 p.m.)


Review request for atlas, Madhan Neethiraj and Sarath Subramanian.


Changes
---

Updates include: Addressed review comment.


Bugs: ATLAS-2602
https://issues.apache.org/jira/browse/ATLAS-2602


Repository: atlas


Description
---

**Background**
During entity creation, has a pre-create phase that discovers entities that 
need to be created within _AtlasEntity_. For import, this discovery was 
ignoring the entities present within relationship attribute.

**Fix**
The fix removes the condition.


Diffs (updated)
-

  
repository/src/main/java/org/apache/atlas/repository/store/graph/v1/AtlasEntityGraphDiscoveryV1.java
 f092d1847 
  
repository/src/main/java/org/apache/atlas/repository/store/graph/v1/BulkImporterImpl.java
 af310954f 
  
repository/src/test/java/org/apache/atlas/repository/impexp/ImportServiceTest.java
 91d56ab80 
  repository/src/test/resources/stocks-glossary.zip PRE-CREATION 


Diff: https://reviews.apache.org/r/66791/diff/2/

Changes: https://reviews.apache.org/r/66791/diff/1-2/


Testing
---

**Unit tests**
- New unit tests cover this case.
- Additional checks added using EntityDiscovery to query the Graph with known 
guids after import succeeds.


Thanks,

Ashutosh Mestry

Re: Dependent component versions for 1.0 release

2018-04-25 Thread Madhan Neethiraj

Mandy,

What is the Spring version to target?

Thanks,
Madhan




On 4/25/18, 1:36 AM, "Mandy Chessell"  wrote:

Dear all,
I think we should move to the latest level of the spring framework/boot

All the best
Mandy
___
Mandy Chessell CBE FREng CEng FBCS
IBM Distinguished Engineer

Master Inventor
Member of the IBM Academy of Technology
Visiting Professor, Department of Computer Science, University of 
Sheffield

Email: mandy_chess...@uk.ibm.com
LinkedIn: http://www.linkedin.com/pub/mandy-chessell/22/897/a49

Assistant: Janet Brooks - jsbrook...@uk.ibm.com



From:   Nigel Jones 
To: 
Date:   24/04/2018 17:01
Subject:Re: Dependent component versions for 1.0 release





On 2018/04/24 13:04:59, Graham Wallis  wrote: 

> Do we need to include anything about development versions including 
Maven, 
> Java, etc?

I think it would be worth mentioning in readme/ wiki build, though now we 
have the maven-enforcer in place we know the min prereq java is java 8 
update 151 & maven 3.5.0 which aren't far off (latest 172 - jdk 11/18.10 
is next lts and could be 'interesting', and maven 3.5.3)

Post-release I think there's a few other questions about specific 
component versions (and pom standardization where it makes sense), but I 
wouldn't change anything close to release

[jira] [Commented] (ATLAS-2601) OMAG server Swagger

2018-04-25 Thread Nigel Jones (JIRA)


[ 
https://issues.apache.org/jira/browse/ATLAS-2601?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16452025#comment-16452025
 ] 

Nigel Jones commented on ATLAS-2601:


It does seem as if enunciate and swagger are broadly comparable. In terms of 
build there are some advantages I think to the current enunciate approach if 
[https://stackoverflow.com/questions/43249895/generating-swagger-json-for-spring-mvc-springfox-or-enunciate]
 is correct, especially as it seems less annotations are needed?

Overall  I agree it's better to have one mechanism unless there are very clear 
advantages in having two - or moving to springfox for the base atlas code.

> OMAG server Swagger 
> 
>
> Key: ATLAS-2601
> URL: https://issues.apache.org/jira/browse/ATLAS-2601
> Project: Atlas
>  Issue Type: Improvement
>Reporter: Maryna Strelchuk
>Assignee: Maryna Strelchuk
>Priority: Major
> Attachments: 0001-Atlas-2601-OMAG-server-Swagger.patch
>
>
> This Jira addresses the addition of the Swagger for the OMAG server. 
> Currently Swagger was added as a dependency for
>  * omag-server
>  * omag-api
>  * omas-connectedasset



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Re: Review Request 66784: ATLAS2604 Minor fixups to open metadata archive types

2018-04-25 Thread Mandy Chessell


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/66784/#review201902
---




omrs/src/test/java/org/apache/atlas/omrs/archivemanager/opentypes/TestOpenMetadataTypesArchive.java
Lines 35 (patched)


This code is not in the most effective place.  It only ensures the open 
metadata types do not have duplicate attributes for an entity and only after 
the archive is built.  It should be in the archive builder ensuring that no 
archives can be built with duplicate names.  It should also be in the 
repository content manager validating that types from different archives do not 
cause an entity to have duplicate properties.  This test code should be removed 
and reformatted to run in the main OMRS code.


- Mandy Chessell


On April 24, 2018, 8:48 p.m., David Radley wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/66784/
> ---
> 
> (Updated April 24, 2018, 8:48 p.m.)
> 
> 
> Review request for atlas, Madhan Neethiraj and Mandy Chessell.
> 
> 
> Bugs: ATLAS2604
> https://issues.apache.org/jira/browse/ATLAS2604
> 
> 
> Repository: atlas
> 
> 
> Description
> ---
> 
> ATLAS2604 Minor fixups to open metadata archive types to avoid duplicate 
> attributes due to relationship end names.
> 
> 
> Diffs
> -
> 
>   
> omrs/src/main/java/org/apache/atlas/omrs/archivemanager/opentypes/OpenMetadataTypesArchive.java
>  faab96cac 
>   
> omrs/src/test/java/org/apache/atlas/omrs/archivemanager/opentypes/TestOpenMetadataTypesArchive.java
>  PRE-CREATION 
> 
> 
> Diff: https://reviews.apache.org/r/66784/diff/1/
> 
> 
> Testing
> ---
> 
> Ran junit that failed without the patch applied. I ran this in Intellij using 
> the  Maven view to run the test phase on the OMRS module. This successfully 
> ran the test.
> 
> 
> Thanks,
> 
> David Radley
> 
>

Review Request 66797: ATLAS-2609: Update audit and notification listener to handle propagated classification add/delete/update

2018-04-25 Thread Sarath Subramanian


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/66797/
---

Review request for atlas, Apoorv Naik, Ashutosh Mestry, and Madhan Neethiraj.


Bugs: ATLAS-2609
https://issues.apache.org/jira/browse/ATLAS-2609


Repository: atlas


Description
---

Currently audit and notifications are not generated for propagated 
classification add/update or delete.

 For Audit:

Adding Propagated Classification  => Send 'Propagated Classifications 
Added' message to audit repository. 
Removing Propagated Classification =>  Send 'Propagated Classifications 
Deleted' message to audit repository.
Updating Propagated Classification   => Send 'Propagated Classifications 
Updated' message to audit repository.
 For Notification:

Adding Propagated Classification  =>  Send PROPAGATED_CLASSIFICATION_ADD 
message type.
Removing Propagated Classification =>  Send PROPAGATED_CLASSIFICATION_DELETE 
message type.
Updating Propagated Classification   => Send PROPAGATED_CLASSIFICATION_UPDATE 
message type.


Diffs
-

  client/client-v1/src/main/java/org/apache/atlas/EntityAuditEvent.java 
3cf3e212 
  dashboardv2/public/js/utils/Enums.js 8c0518bf 
  intg/src/main/java/org/apache/atlas/listener/EntityChangeListenerV2.java 
70877d22 
  intg/src/main/java/org/apache/atlas/model/audit/EntityAuditEventV2.java 
1045c774 
  
intg/src/main/java/org/apache/atlas/v1/model/notification/EntityNotificationV2.java
 a8dfd235 
  
repository/src/main/java/org/apache/atlas/repository/audit/EntityAuditListener.java
 1c04eea4 
  
repository/src/main/java/org/apache/atlas/repository/audit/EntityAuditListenerV2.java
 bb510149 
  
repository/src/main/java/org/apache/atlas/repository/converters/AtlasInstanceConverter.java
 20147a0c 
  
repository/src/main/java/org/apache/atlas/repository/store/graph/v1/AtlasEntityChangeNotifier.java
 9aebde2b 
  
repository/src/main/java/org/apache/atlas/repository/store/graph/v1/AtlasRelationshipStoreV1.java
 51dcc3a5 
  
repository/src/main/java/org/apache/atlas/repository/store/graph/v1/DeleteHandlerV1.java
 c0f2fc3d 
  
repository/src/main/java/org/apache/atlas/repository/store/graph/v1/EntityGraphMapper.java
 4225a80d 
  
repository/src/main/java/org/apache/atlas/repository/store/graph/v1/EntityGraphRetriever.java
 7d5b7fb6 
  
repository/src/test/java/org/apache/atlas/repository/store/graph/v1/AtlasRelationshipStoreV1Test.java
 8fc0327a 
  server-api/src/main/java/org/apache/atlas/RequestContextV1.java b7c56865 
  server-api/src/main/java/org/apache/atlas/listener/EntityChangeListener.java 
19fae4fd 
  
tools/atlas-migration-exporter/src/main/java/org/apache/atlas/migration/NoOpNotificationChangeListener.java
 6c2bf220 
  
webapp/src/main/java/org/apache/atlas/notification/EntityNotificationListenerV2.java
 ac097b60 
  
webapp/src/main/java/org/apache/atlas/notification/NotificationEntityChangeListener.java
 a3e5949c 


Diff: https://reviews.apache.org/r/66797/diff/1/


Testing
---

Precommit in progress: 
https://builds.apache.org/view/A/view/Atlas/job/PreCommit-ATLAS-Build-Test/307/console

Validated audit/notifications manually for propagated classification 
add/update/delete


Thanks,

Sarath Subramanian

Re: Dependent component versions for 1.0 release

2018-04-25 Thread Mandy Chessell

Dear all,
I think we should move to the latest level of the spring framework/boot

All the best
Mandy
___
Mandy Chessell CBE FREng CEng FBCS
IBM Distinguished Engineer

Master Inventor
Member of the IBM Academy of Technology
Visiting Professor, Department of Computer Science, University of 
Sheffield

Email: mandy_chess...@uk.ibm.com
LinkedIn: http://www.linkedin.com/pub/mandy-chessell/22/897/a49

Assistant: Janet Brooks - jsbrook...@uk.ibm.com



From:   Nigel Jones 
To: 
Date:   24/04/2018 17:01
Subject:Re: Dependent component versions for 1.0 release





On 2018/04/24 13:04:59, Graham Wallis  wrote: 

> Do we need to include anything about development versions including 
Maven, 
> Java, etc?

I think it would be worth mentioning in readme/ wiki build, though now we 
have the maven-enforcer in place we know the min prereq java is java 8 
update 151 & maven 3.5.0 which aren't far off (latest 172 - jdk 11/18.10 
is next lts and could be 'interesting', and maven 3.5.3)

Post-release I think there's a few other questions about specific 
component versions (and pom standardization where it makes sense), but I 
wouldn't change anything close to release

[jira] [Commented] (ATLAS-2607) Classification lifecycle through metadata properties and relationships

2018-04-25 Thread Graham Wallis (JIRA)


[ 
https://issues.apache.org/jira/browse/ATLAS-2607?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16451874#comment-16451874
 ] 

Graham Wallis commented on ATLAS-2607:
--

Can a steward/admin move a Classification from Retired state back into Active 
state?

> Classification lifecycle through metadata properties and relationships
> --
>
> Key: ATLAS-2607
> URL: https://issues.apache.org/jira/browse/ATLAS-2607
> Project: Atlas
>  Issue Type: Improvement
>  Components:  atlas-core
>Reporter: Srikanth Venkat
>Assignee: Madhan Neethiraj
>Priority: Critical
>
> Currently tags or classifications in Atlas are considered active once they 
> are defined. For governance and stewardship purposes, it would be important 
> to attach the notion of what state in its lifecycle a particular 
> classification is. This would help with workflows to manage the lifecycle 
> aspects and provide any filtering needed to take appropriate actions. For 
> example only active classifications should be considered for classification 
> based policy enforcement. Additionally lifecycle status would help with 
> filtering and search as well as reporting and compliance/audit scenarios.
> Implementation Proposal:
>  * All tags or classifications have a "Lifecycle Status" property
>  * They can go through the following list of states during their lifecycle: 
> DRAFT -> ACTIVE  -> RETIRED
>  * Lifecycle Status can be set as an enum property that is mandatory or 
> required for all classifications.
>  * All existing classifications already present in Atlas before this change 
> will default to an ACTIVE status so that all pre-existing classifications 
> will continue to work as before.
>  * All new classifications after this change will start out with DRAFT status 
> and a steward or an admin with appropriate permissions can move them into an 
> ACTIVE state (controlled via Metadata security policies)
>  * Policy enforcement for authorization on classifications can ignore any 
> that are not in ACTIVE state. 
>  * Asset can only be attached to classifications with the status of ACTIVE
>  * For a classification in RETIRED state, we might have an optional 
> relationship with another classification called "Replaced By" which is the 
> new classification that the current one was remapped or replaced with. The 
> inverse relationship could be labeled "Replaces" which is on the new 
> classification and points to the removed classification that it replaces. 
>  * The state RETIRED implies this classification is no longer used and policy 
> enforcement will ignore any classifications in such states. 
>  * Additionally UI can filter and show by default only classifications that 
> are not RETIRED and a checkbox to "Show Retired"
>  * Deletion of classifications should work as it currently does with no 
> behavioral changes.
>  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (ATLAS-2605) Drop Titan 0 support

2018-04-25 Thread Graham Wallis (JIRA)


[ 
https://issues.apache.org/jira/browse/ATLAS-2605?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16451828#comment-16451828
 ] 

Graham Wallis commented on ATLAS-2605:
--

I think this might be a dup of 2521?

> Drop Titan 0 support
> 
>
> Key: ATLAS-2605
> URL: https://issues.apache.org/jira/browse/ATLAS-2605
> Project: Atlas
>  Issue Type: Improvement
>Affects Versions: 1.0.0-alpha
>Reporter: Apoorv Naik
>Assignee: Apoorv Naik
>Priority: Major
> Fix For: 1.0.0
>
>
> Since the titan 0 project is no longer maintained and Atlas 1.0 is moving 
> towards JanusGraph which is much active and an improvement over Titan 1



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Updated] (ATLAS-2609) Update audit and notification listener to handle propagated classification add/delete/update

2018-04-25 Thread Sarath Subramanian (JIRA)


 [ 
https://issues.apache.org/jira/browse/ATLAS-2609?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sarath Subramanian updated ATLAS-2609:
--
Description: 
Currently audit and notifications are not generated for propagated 
classification add/update or delete.

 For Audit:
 * Adding Propagated Classification      => Send '_Propagated Classifications 
Added_' message to audit repository. 
 * Removing Propagated Classification =>  Send '_Propagated Classifications 
Deleted_' message to audit repository.
 * Updating Propagated Classification   => Send '_Propagated Classifications 
Updated_' message to audit repository.

 For Notification:
 * Adding Propagated Classification      =>  Send 
*PROPAGATED_CLASSIFICATION_ADD* message type.
 * Removing Propagated Classification =>  Send 
*PROPAGATED_CLASSIFICATION_DELETE* message type.
 * Updating Propagated Classification   => Send 
*PROPAGATED_CLASSIFICATION_UPDATE* message type.

 

  was:
Currently audit and notifications are not generated for propagated 
classification add/update or delete.

 

 


> Update audit and notification listener to handle propagated classification 
> add/delete/update
> 
>
> Key: ATLAS-2609
> URL: https://issues.apache.org/jira/browse/ATLAS-2609
> Project: Atlas
>  Issue Type: Bug
>  Components:  atlas-core
>Affects Versions: 1.0.0
>Reporter: Sarath Subramanian
>Assignee: Sarath Subramanian
>Priority: Major
> Fix For: 1.0.0
>
>
> Currently audit and notifications are not generated for propagated 
> classification add/update or delete.
>  For Audit:
>  * Adding Propagated Classification      => Send '_Propagated Classifications 
> Added_' message to audit repository. 
>  * Removing Propagated Classification =>  Send '_Propagated Classifications 
> Deleted_' message to audit repository.
>  * Updating Propagated Classification   => Send '_Propagated Classifications 
> Updated_' message to audit repository.
>  For Notification:
>  * Adding Propagated Classification      =>  Send 
> *PROPAGATED_CLASSIFICATION_ADD* message type.
>  * Removing Propagated Classification =>  Send 
> *PROPAGATED_CLASSIFICATION_DELETE* message type.
>  * Updating Propagated Classification   => Send 
> *PROPAGATED_CLASSIFICATION_UPDATE* message type.
>  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Created] (ATLAS-2609) Update audit and notification listener to handle propagated classification add/delete/update

2018-04-25 Thread Sarath Subramanian (JIRA)

Sarath Subramanian created ATLAS-2609:
-

 Summary: Update audit and notification listener to handle 
propagated classification add/delete/update
 Key: ATLAS-2609
 URL: https://issues.apache.org/jira/browse/ATLAS-2609
 Project: Atlas
  Issue Type: Bug
  Components:  atlas-core
Affects Versions: 1.0.0
Reporter: Sarath Subramanian
Assignee: Sarath Subramanian
 Fix For: 1.0.0


Currently audit and notifications are not generated for propagated 
classification add/update or delete.

 

 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Updated] (ATLAS-2608) UI - Cache is not cleared if user click on cancel button while editing propagation flow

2018-04-25 Thread Tejas Rawool (JIRA)


 [ 
https://issues.apache.org/jira/browse/ATLAS-2608?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Tejas Rawool updated ATLAS-2608:

Summary: UI - Cache is not cleared if user click on cancel button while 
editing propagation flow  (was: UI - Cached is not cleared if user click on 
cancel button while editing propagation flow)

> UI - Cache is not cleared if user click on cancel button while editing 
> propagation flow
> ---
>
> Key: ATLAS-2608
> URL: https://issues.apache.org/jira/browse/ATLAS-2608
> Project: Atlas
>  Issue Type: Bug
>Affects Versions: 1.0.0
>Reporter: Tejas Rawool
>Priority: Major
> Fix For: 1.0.0
>
>
> Steps to reproduce:-
>  
> 1.Do a Basic Search for Tables
> 2.Now go to Lineage Tab of  sales_fact
> 3.Click on arrow between sales_fact and loadSalesDaily
> 4.Now switch to Block Propagation
> 5.Checked any of the entity and click on cancel button
> 6.Now again click on that arrow and go to block propagation without 
> refreshing the page
> Error:-
> An Duplicate entity is created which is incorrect



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Updated] (ATLAS-2608) UI - Cached is not cleared if user click on cancel button while editing propagation flow

2018-04-25 Thread Tejas Rawool (JIRA)


 [ 
https://issues.apache.org/jira/browse/ATLAS-2608?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Tejas Rawool updated ATLAS-2608:

Summary: UI - Cached is not cleared if user click on cancel button while 
editing propagation flow  (was: UI - Cashed is not cleared if user click on 
cancel button while editing propagation flow)

> UI - Cached is not cleared if user click on cancel button while editing 
> propagation flow
> 
>
> Key: ATLAS-2608
> URL: https://issues.apache.org/jira/browse/ATLAS-2608
> Project: Atlas
>  Issue Type: Bug
>Affects Versions: 1.0.0
>Reporter: Tejas Rawool
>Priority: Major
> Fix For: 1.0.0
>
>
> Steps to reproduce:-
>  
> 1.Do a Basic Search for Tables
> 2.Now go to Lineage Tab of  sales_fact
> 3.Click on arrow between sales_fact and loadSalesDaily
> 4.Now switch to Block Propagation
> 5.Checked any of the entity and click on cancel button
> 6.Now again click on that arrow and go to block propagation without 
> refreshing the page
> Error:-
> An Duplicate entity is created which is incorrect



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Created] (ATLAS-2608) UI - Cashed is not cleared if user click on cancel button while editing propagation flow

2018-04-25 Thread Tejas Rawool (JIRA)

Tejas Rawool created ATLAS-2608:
---

 Summary: UI - Cashed is not cleared if user click on cancel button 
while editing propagation flow
 Key: ATLAS-2608
 URL: https://issues.apache.org/jira/browse/ATLAS-2608
 Project: Atlas
  Issue Type: Bug
Affects Versions: 1.0.0
Reporter: Tejas Rawool
 Fix For: 1.0.0


Steps to reproduce:-

 

1.Do a Basic Search for Tables

2.Now go to Lineage Tab of  sales_fact

3.Click on arrow between sales_fact and loadSalesDaily

4.Now switch to Block Propagation

5.Checked any of the entity and click on cancel button

6.Now again click on that arrow and go to block propagation without refreshing 
the page

Error:-

An Duplicate entity is created which is incorrect



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Assigned] (ATLAS-2607) Classification lifecycle through metadata properties and relationships

2018-04-25 Thread Srikanth Venkat (JIRA)


 [ 
https://issues.apache.org/jira/browse/ATLAS-2607?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Srikanth Venkat reassigned ATLAS-2607:
--

Assignee: Madhan Neethiraj

> Classification lifecycle through metadata properties and relationships
> --
>
> Key: ATLAS-2607
> URL: https://issues.apache.org/jira/browse/ATLAS-2607
> Project: Atlas
>  Issue Type: Improvement
>  Components:  atlas-core
>Reporter: Srikanth Venkat
>Assignee: Madhan Neethiraj
>Priority: Critical
>
> Currently tags or classifications in Atlas are considered active once they 
> are defined. For governance and stewardship purposes, it would be important 
> to attach the notion of what state in its lifecycle a particular 
> classification is. This would help with workflows to manage the lifecycle 
> aspects and provide any filtering needed to take appropriate actions. For 
> example only active classifications should be considered for classification 
> based policy enforcement. Additionally lifecycle status would help with 
> filtering and search as well as reporting and compliance/audit scenarios.
> Implementation Proposal:
>  * All tags or classifications have a "Lifecycle Status" property
>  * They can go through the following list of states during their lifecycle: 
> DRAFT -> ACTIVE  -> RETIRED
>  * Lifecycle Status can be set as an enum property that is mandatory or 
> required for all classifications.
>  * All existing classifications already present in Atlas before this change 
> will default to an ACTIVE status so that all pre-existing classifications 
> will continue to work as before.
>  * All new classifications after this change will start out with DRAFT status 
> and a steward or an admin with appropriate permissions can move them into an 
> ACTIVE state (controlled via Metadata security policies)
>  * Policy enforcement for authorization on classifications can ignore any 
> that are not in ACTIVE state. 
>  * Asset can only be attached to classifications with the status of ACTIVE
>  * For a classification in RETIRED state, we might have an optional 
> relationship with another classification called "Replaced By" which is the 
> new classification that the current one was remapped or replaced with. The 
> inverse relationship could be labeled "Replaces" which is on the new 
> classification and points to the removed classification that it replaces. 
>  * The state RETIRED implies this classification is no longer used and policy 
> enforcement will ignore any classifications in such states. 
>  * Additionally UI can filter and show by default only classifications that 
> are not RETIRED and a checkbox to "Show Retired"
>  * Deletion of classifications should work as it currently does with no 
> behavioral changes.
>  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Created] (ATLAS-2607) Classification lifecycle through metadata properties and relationships

2018-04-25 Thread Srikanth Venkat (JIRA)

Srikanth Venkat created ATLAS-2607:
--

 Summary: Classification lifecycle through metadata properties and 
relationships
 Key: ATLAS-2607
 URL: https://issues.apache.org/jira/browse/ATLAS-2607
 Project: Atlas
  Issue Type: Improvement
  Components:  atlas-core
Reporter: Srikanth Venkat


Currently tags or classifications in Atlas are considered active once they are 
defined. For governance and stewardship purposes, it would be important to 
attach the notion of what state in its lifecycle a particular classification 
is. This would help with workflows to manage the lifecycle aspects and provide 
any filtering needed to take appropriate actions. For example only active 
classifications should be considered for classification based policy 
enforcement. Additionally lifecycle status would help with filtering and search 
as well as reporting and compliance/audit scenarios.

Implementation Proposal:
 * All tags or classifications have a "Lifecycle Status" property
 * They can go through the following list of states during their lifecycle: 
DRAFT -> ACTIVE  -> RETIRED
 * Lifecycle Status can be set as an enum property that is mandatory or 
required for all classifications.
 * All existing classifications already present in Atlas before this change 
will default to an ACTIVE status so that all pre-existing classifications will 
continue to work as before.
 * All new classifications after this change will start out with DRAFT status 
and a steward or an admin with appropriate permissions can move them into an 
ACTIVE state (controlled via Metadata security policies)
 * Policy enforcement for authorization on classifications can ignore any that 
are not in ACTIVE state. 
 * Asset can only be attached to classifications with the status of ACTIVE
 * For a classification in RETIRED state, we might have an optional 
relationship with another classification called "Replaced By" which is the new 
classification that the current one was remapped or replaced with. The inverse 
relationship could be labeled "Replaces" which is on the new classification and 
points to the removed classification that it replaces. 
 * The state RETIRED implies this classification is no longer used and policy 
enforcement will ignore any classifications in such states. 
 * Additionally UI can filter and show by default only classifications that are 
not RETIRED and a checkbox to "Show Retired"
 * Deletion of classifications should work as it currently does with no 
behavioral changes.

 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

45 matches

Mail list logo