Re: Review Request 71115: ATLAS-3335 Update Sqoop/Storm hook to use relationship attributes

2019-07-18 Thread Nixon Rodrigues 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/71115/#review216757
---


Ship it!




Precommit passed
https://builds.apache.org/job/PreCommit-ATLAS-Build-Test/1285/console

- Nixon Rodrigues


On July 18, 2019, 1:50 p.m., Mandar Ambawane wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/71115/
> ---
> 
> (Updated July 18, 2019, 1:50 p.m.)
> 
> 
> Review request for atlas, Ashutosh Mestry, Madhan Neethiraj, Nixon Rodrigues, 
> and Sarath Subramanian.
> 
> 
> Bugs: ATLAS-3335
> https://issues.apache.org/jira/browse/ATLAS-3335
> 
> 
> Repository: atlas
> 
> 
> Description
> ---
> 
> ATLAS-3335 Update Sqoop/Storm hook to use relationship attributes
> 
> 
> Diffs
> -
> 
>   
> addons/sqoop-bridge/src/main/java/org/apache/atlas/sqoop/hook/SqoopHook.java 
> 3ccd426 
>   
> addons/storm-bridge/src/main/java/org/apache/atlas/storm/hook/StormAtlasHook.java
>  517a3c3 
> 
> 
> Diff: https://reviews.apache.org/r/71115/diff/1/
> 
> 
> Testing
> ---
> 
> Executed sqoop import with mysql to hive.
> 
> pre-commit 
> https://builds.apache.org/job/PreCommit-ATLAS-Build-Test/1285/console
> 
> 
> Thanks,
> 
> Mandar Ambawane
> 
>

Re: Review Request 71115: ATLAS-3335 Update Sqoop/Storm hook to use relationship attributes

2019-07-18 Thread Nixon Rodrigues 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/71115/#review216756
---


Ship it!




Ship It!

- Nixon Rodrigues


On July 18, 2019, 1:50 p.m., Mandar Ambawane wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/71115/
> ---
> 
> (Updated July 18, 2019, 1:50 p.m.)
> 
> 
> Review request for atlas, Ashutosh Mestry, Madhan Neethiraj, Nixon Rodrigues, 
> and Sarath Subramanian.
> 
> 
> Bugs: ATLAS-3335
> https://issues.apache.org/jira/browse/ATLAS-3335
> 
> 
> Repository: atlas
> 
> 
> Description
> ---
> 
> ATLAS-3335 Update Sqoop/Storm hook to use relationship attributes
> 
> 
> Diffs
> -
> 
>   
> addons/sqoop-bridge/src/main/java/org/apache/atlas/sqoop/hook/SqoopHook.java 
> 3ccd426 
>   
> addons/storm-bridge/src/main/java/org/apache/atlas/storm/hook/StormAtlasHook.java
>  517a3c3 
> 
> 
> Diff: https://reviews.apache.org/r/71115/diff/1/
> 
> 
> Testing
> ---
> 
> Executed sqoop import with mysql to hive.
> 
> pre-commit 
> https://builds.apache.org/job/PreCommit-ATLAS-Build-Test/1285/console
> 
> 
> Thanks,
> 
> Mandar Ambawane
> 
>

[jira] [Commented] (ATLAS-3067) update hive types to remove use of ownedRef/inverseRef constraints for relationships

2019-07-18 Thread ASF subversion and git services (JIRA) 


[ 
https://issues.apache.org/jira/browse/ATLAS-3067?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16888424#comment-16888424
 ] 

ASF subversion and git services commented on ATLAS-3067:


Commit a69166e3ba87eaa6339fd44c0198e436b3c7e9d8 in atlas's branch 
refs/heads/branch-2.0 from Madhan Neethiraj
[ https://gitbox.apache.org/repos/asf?p=atlas.git;h=a69166e ]

ATLAS-3067: fixed failure in EntityJerseyResourceIT.testUTF8()

(cherry picked from commit 709db096badfad3c17cf3adf9d65585bf1c0bc04)


> update hive types to remove use of ownedRef/inverseRef constraints for 
> relationships
> 
>
> Key: ATLAS-3067
> URL: https://issues.apache.org/jira/browse/ATLAS-3067
> Project: Atlas
>  Issue Type: Improvement
>  Components:  atlas-core
>Reporter: Madhan Neethiraj
>Assignee: Madhan Neethiraj
>Priority: Major
> Fix For: 1.2.0, 2.0.0
>
> Attachments: ATLAS-3067.patch
>
>
> Similar to recent updates in RDBMS entity-types (ATLAS-3056), Hive 
> entity-types should be updated to remove attributes that refer to other 
> entities, as equivalent relationship-types already cover the relationship 
> between these entity-types. Here is the list of attributes to be removed:
>  - hive_table.db (covered by hive_table_db)
>  - hive_table.sd, hive_storagedesc.table (covered by hive_table_storagedesc)
>  - hive_table.partitionKeys, hive_column.table (covered by 
> hive_table_partitionkeys)
>  - hive_table.columns, hive_column.table (covered by hive_table_columns)
>  - hive_column_lineage.query (covered by hive_process_column_lineage)



--
This message was sent by Atlassian JIRA
(v7.6.14#76016)

[jira] [Commented] (ATLAS-3067) update hive types to remove use of ownedRef/inverseRef constraints for relationships

2019-07-18 Thread ASF subversion and git services (JIRA) 


[ 
https://issues.apache.org/jira/browse/ATLAS-3067?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16888423#comment-16888423
 ] 

ASF subversion and git services commented on ATLAS-3067:


Commit 709db096badfad3c17cf3adf9d65585bf1c0bc04 in atlas's branch 
refs/heads/master from Madhan Neethiraj
[ https://gitbox.apache.org/repos/asf?p=atlas.git;h=709db09 ]

ATLAS-3067: fixed failure in EntityJerseyResourceIT.testUTF8()


> update hive types to remove use of ownedRef/inverseRef constraints for 
> relationships
> 
>
> Key: ATLAS-3067
> URL: https://issues.apache.org/jira/browse/ATLAS-3067
> Project: Atlas
>  Issue Type: Improvement
>  Components:  atlas-core
>Reporter: Madhan Neethiraj
>Assignee: Madhan Neethiraj
>Priority: Major
> Fix For: 1.2.0, 2.0.0
>
> Attachments: ATLAS-3067.patch
>
>
> Similar to recent updates in RDBMS entity-types (ATLAS-3056), Hive 
> entity-types should be updated to remove attributes that refer to other 
> entities, as equivalent relationship-types already cover the relationship 
> between these entity-types. Here is the list of attributes to be removed:
>  - hive_table.db (covered by hive_table_db)
>  - hive_table.sd, hive_storagedesc.table (covered by hive_table_storagedesc)
>  - hive_table.partitionKeys, hive_column.table (covered by 
> hive_table_partitionkeys)
>  - hive_table.columns, hive_column.table (covered by hive_table_columns)
>  - hive_column_lineage.query (covered by hive_process_column_lineage)



--
This message was sent by Atlassian JIRA
(v7.6.14#76016)

[jira] [Commented] (ATLAS-3067) update hive types to remove use of ownedRef/inverseRef constraints for relationships

2019-07-18 Thread ASF subversion and git services (JIRA) 


[ 
https://issues.apache.org/jira/browse/ATLAS-3067?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16888014#comment-16888014
 ] 

ASF subversion and git services commented on ATLAS-3067:


Commit 86057ff3faee44d9459ae3b627e2ca938184fd28 in atlas's branch 
refs/heads/branch-2.0 from Madhan Neethiraj
[ https://gitbox.apache.org/repos/asf?p=atlas.git;h=86057ff ]

ATLAS-3067: updated EntityJerseyResourceIT.testUTF8() to aid troubleshooting 
failures - #3

(cherry picked from commit bd31c50539c7b4df5fb43065844a007ce68a0106)


> update hive types to remove use of ownedRef/inverseRef constraints for 
> relationships
> 
>
> Key: ATLAS-3067
> URL: https://issues.apache.org/jira/browse/ATLAS-3067
> Project: Atlas
>  Issue Type: Improvement
>  Components:  atlas-core
>Reporter: Madhan Neethiraj
>Assignee: Madhan Neethiraj
>Priority: Major
> Fix For: 1.2.0, 2.0.0
>
> Attachments: ATLAS-3067.patch
>
>
> Similar to recent updates in RDBMS entity-types (ATLAS-3056), Hive 
> entity-types should be updated to remove attributes that refer to other 
> entities, as equivalent relationship-types already cover the relationship 
> between these entity-types. Here is the list of attributes to be removed:
>  - hive_table.db (covered by hive_table_db)
>  - hive_table.sd, hive_storagedesc.table (covered by hive_table_storagedesc)
>  - hive_table.partitionKeys, hive_column.table (covered by 
> hive_table_partitionkeys)
>  - hive_table.columns, hive_column.table (covered by hive_table_columns)
>  - hive_column_lineage.query (covered by hive_process_column_lineage)



--
This message was sent by Atlassian JIRA
(v7.6.14#76016)

Review Request 71115: ATLAS-3335 Update Sqoop/Storm hook to use relationship attributes

2019-07-18 Thread Mandar Ambawane 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/71115/
---

Review request for atlas, Ashutosh Mestry, Madhan Neethiraj, Nixon Rodrigues, 
and Sarath Subramanian.


Bugs: ATLAS-3335
https://issues.apache.org/jira/browse/ATLAS-3335


Repository: atlas


Description
---

ATLAS-3335 Update Sqoop/Storm hook to use relationship attributes


Diffs
-

  addons/sqoop-bridge/src/main/java/org/apache/atlas/sqoop/hook/SqoopHook.java 
3ccd426 
  
addons/storm-bridge/src/main/java/org/apache/atlas/storm/hook/StormAtlasHook.java
 517a3c3 


Diff: https://reviews.apache.org/r/71115/diff/1/


Testing
---

Executed sqoop import with mysql to hive.

pre-commit https://builds.apache.org/job/PreCommit-ATLAS-Build-Test/1285/console


Thanks,

Mandar Ambawane

[jira] [Commented] (ATLAS-3067) update hive types to remove use of ownedRef/inverseRef constraints for relationships

2019-07-18 Thread ASF subversion and git services (JIRA) 


[ 
https://issues.apache.org/jira/browse/ATLAS-3067?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16887996#comment-16887996
 ] 

ASF subversion and git services commented on ATLAS-3067:


Commit bd31c50539c7b4df5fb43065844a007ce68a0106 in atlas's branch 
refs/heads/master from Madhan Neethiraj
[ https://gitbox.apache.org/repos/asf?p=atlas.git;h=bd31c50 ]

ATLAS-3067: updated EntityJerseyResourceIT.testUTF8() to aid troubleshooting 
failures - #3


> update hive types to remove use of ownedRef/inverseRef constraints for 
> relationships
> 
>
> Key: ATLAS-3067
> URL: https://issues.apache.org/jira/browse/ATLAS-3067
> Project: Atlas
>  Issue Type: Improvement
>  Components:  atlas-core
>Reporter: Madhan Neethiraj
>Assignee: Madhan Neethiraj
>Priority: Major
> Fix For: 1.2.0, 2.0.0
>
> Attachments: ATLAS-3067.patch
>
>
> Similar to recent updates in RDBMS entity-types (ATLAS-3056), Hive 
> entity-types should be updated to remove attributes that refer to other 
> entities, as equivalent relationship-types already cover the relationship 
> between these entity-types. Here is the list of attributes to be removed:
>  - hive_table.db (covered by hive_table_db)
>  - hive_table.sd, hive_storagedesc.table (covered by hive_table_storagedesc)
>  - hive_table.partitionKeys, hive_column.table (covered by 
> hive_table_partitionkeys)
>  - hive_table.columns, hive_column.table (covered by hive_table_columns)
>  - hive_column_lineage.query (covered by hive_process_column_lineage)



--
This message was sent by Atlassian JIRA
(v7.6.14#76016)

[jira] [Resolved] (ATLAS-3326) Update Hbase hook/bridge to use relationship attribute.

2019-07-18 Thread Nixon Rodrigues (JIRA) 


 [ 
https://issues.apache.org/jira/browse/ATLAS-3326?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Nixon Rodrigues resolved ATLAS-3326.

   Resolution: Fixed
Fix Version/s: 3.0.0

Thanks [~mandar_va] for the patch and [~sarath.ku...@gmail.com] review. 
committed patch to master.

> Update Hbase hook/bridge to use relationship attribute.
> ---
>
> Key: ATLAS-3326
> URL: https://issues.apache.org/jira/browse/ATLAS-3326
> Project: Atlas
>  Issue Type: Bug
>Reporter: Nixon Rodrigues
>Assignee: Mandar Ambawane
>Priority: Major
> Fix For: 3.0.0
>
>
> Update Hbase hook/bridge to use relationship attribute.



--
This message was sent by Atlassian JIRA
(v7.6.14#76016)

[jira] [Commented] (ATLAS-3326) Update Hbase hook/bridge to use relationship attribute.

2019-07-18 Thread ASF subversion and git services (JIRA) 


[ 
https://issues.apache.org/jira/browse/ATLAS-3326?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16887970#comment-16887970
 ] 

ASF subversion and git services commented on ATLAS-3326:


Commit ca62d8659db632954ed28e35f3a5e7d27d3fe8b4 in atlas's branch 
refs/heads/master from Mandar Ambawane
[ https://gitbox.apache.org/repos/asf?p=atlas.git;h=ca62d86 ]

ATLAS-3326 Update Hbase hook/bridge to use relationship attribute and move 
Conversion methods to AtlasTypeUtil

Signed-off-by: nixonrodrigues 


> Update Hbase hook/bridge to use relationship attribute.
> ---
>
> Key: ATLAS-3326
> URL: https://issues.apache.org/jira/browse/ATLAS-3326
> Project: Atlas
>  Issue Type: Bug
>Reporter: Nixon Rodrigues
>Assignee: Mandar Ambawane
>Priority: Major
>
> Update Hbase hook/bridge to use relationship attribute.



--
This message was sent by Atlassian JIRA
(v7.6.14#76016)

Re: Review Request 71064: ATLAS-3326 Update Hbase hook/bridge to use relationship attribute

2019-07-18 Thread Nixon Rodrigues 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/71064/#review216722
---


Ship it!




Ship It!

- Nixon Rodrigues


On July 15, 2019, 11:17 a.m., Mandar Ambawane wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/71064/
> ---
> 
> (Updated July 15, 2019, 11:17 a.m.)
> 
> 
> Review request for atlas, Ashutosh Mestry, Madhan Neethiraj, Nixon Rodrigues, 
> and Sarath Subramanian.
> 
> 
> Repository: atlas
> 
> 
> Description
> ---
> 
> ATLAS-3326 Update Hbase hook/bridge to use relationship attribute
> 
> 
> Diffs
> -
> 
>   
> addons/hbase-bridge/src/main/java/org/apache/atlas/hbase/bridge/HBaseAtlasHook.java
>  6d062e2 
>   
> addons/hbase-bridge/src/main/java/org/apache/atlas/hbase/bridge/HBaseBridge.java
>  4a4b4d9 
>   
> addons/hive-bridge/src/main/java/org/apache/atlas/hive/bridge/HiveMetaStoreBridge.java
>  a61a3e6 
>   
> addons/hive-bridge/src/main/java/org/apache/atlas/hive/hook/events/BaseHiveEvent.java
>  a74273a 
>   
> addons/hive-bridge/src/main/java/org/apache/atlas/hive/hook/events/CreateHiveProcess.java
>  6b050d4 
>   
> addons/hive-bridge/src/test/java/org/apache/atlas/hive/bridge/HiveMetaStoreBridgeTest.java
>  4403aaf 
>   intg/src/main/java/org/apache/atlas/type/AtlasTypeUtil.java 6ac176d 
> 
> 
> Diff: https://reviews.apache.org/r/71064/diff/2/
> 
> 
> Testing
> ---
> 
> Steps done for updating 
> 1. Disable hbase-hook from Ambari.
> 2. Create table in hbase.
> 3. Check whether table entity is created in Atlas side or not.(It should not 
> be created) 
> 4. Run script import-hbsae.sh
> 5. Check again in Atlas side, hbase table and namespace entities are created 
> with relationship.
> 
> 
> Thanks,
> 
> Mandar Ambawane
> 
>

Re: Review Request 71113: [ATLAS-3319] Add api to get lineage by uniqueAttributes

2019-07-18 Thread jiaqi shan 


> On 七月 18, 2019, 9:33 a.m., Nixon Rodrigues wrote:
> > webapp/src/main/java/org/apache/atlas/web/rest/LineageREST.java
> > Lines 110 (patched)
> > 
> >
> > Can you add java doc here with usage of this API here
> > 
> > eg : 
> > v2/lineage/uniqueAttribute/type/hive_table?attr:qualifiedName=sample.test2@bd
> > 
> > So that it is be availabe in API documentation.
> > 
> > 
> > http://atlas.apache.org/api/v2/resource_EntityREST.html#resource_EntityREST_getEntitiesByUniqueAttributes_GET

ok, I've update the java doc


- jiaqi


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/71113/#review216715
---


On 七月 18, 2019, 1:01 p.m., jiaqi shan wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/71113/
> ---
> 
> (Updated 七月 18, 2019, 1:01 p.m.)
> 
> 
> Review request for atlas.
> 
> 
> Bugs: ATLAS-3319
> https://issues.apache.org/jira/browse/ATLAS-3319
> 
> 
> Repository: atlas
> 
> 
> Description
> ---
> 
> This patch add the follow api to support getting lineage by uniqueAttributes, 
> also add the api to AtlasClientV2 accordingly:
> 
> /v2/lineage/uniqueAttribute/type/{typeName}
> 
> 
> Diffs
> -
> 
>   client/client-v2/src/main/java/org/apache/atlas/AtlasClientV2.java 7c8caee 
>   webapp/src/main/java/org/apache/atlas/web/rest/LineageREST.java 8b0316b 
> 
> 
> Diff: https://reviews.apache.org/r/71113/diff/2/
> 
> 
> Testing
> ---
> 
> link used: 
> localhost:21000/api/atlas/v2/lineage/uniqueAttribute/type/hive_table?attr:qualifiedName=sample.test2@bd
> Request Type : GET
> JSON :
> {
> "baseEntityGuid": "0c583c2b-d378-4a7b-98aa-be6c621f11b6",
> "lineageDirection": "BOTH",
> "lineageDepth": 3,
> "guidEntityMap": {
> "4ed7fb2e-e7bc-4384-86cd-fba9303879ae": {
> "typeName": "hive_process_v1",
> "attributes": {
> "qualifiedName": 
> "QUERY:sample.test3@bd->:INSERT:sample.test2@bd",
> "name": "insert overwrite table test2 select * from test3"
> },
> "guid": "4ed7fb2e-e7bc-4384-86cd-fba9303879ae",
> "status": "ACTIVE",
> "displayText": "insert overwrite table test2 select * from test3",
> "classificationNames": [],
> "meaningNames": [],
> "meanings": []
> },
> "ac6f5856-5881-4055-99de-8ecf874b74ee": {
> "typeName": "hive_table",
> "attributes": {
> "owner": "cloud_service",
> "createTime": 1562205245000,
> "qualifiedName": "sample.test3@bd",
> "name": "test3"
> },
> "guid": "ac6f5856-5881-4055-99de-8ecf874b74ee",
> "status": "ACTIVE",
> "displayText": "test3",
> "classificationNames": [],
> "meaningNames": [],
> "meanings": []
> },
> "0c583c2b-d378-4a7b-98aa-be6c621f11b6": {
> "typeName": "hive_table",
> "attributes": {
> "owner": "cloud_service",
> "createTime": 1562145763000,
> "qualifiedName": "sample.test2@bd",
> "name": "test2"
> },
> "guid": "0c583c2b-d378-4a7b-98aa-be6c621f11b6",
> "status": "ACTIVE",
> "displayText": "test2",
> "classificationNames": [],
> "meaningNames": [],
> "meanings": []
> }
> },
> "relations": [
> {
> "fromEntityId": "ac6f5856-5881-4055-99de-8ecf874b74ee",
> "toEntityId": "4ed7fb2e-e7bc-4384-86cd-fba9303879ae",
> "relationshipId": "1a6f0644-cf51-485b-a1ea-0918f7652605"
> },
> {
> "fromEntityId": "4ed7fb2e-e7bc-4384-86cd-fba9303879ae",
> "toEntityId": "0c583c2b-d378-4a7b-98aa-be6c621f11b6",
> "relationshipId": "7031e072-928e-4b8c-b7b9-6f368dc1be90"
> }
> ]
> }
> 
> 
> Thanks,
> 
> jiaqi shan
> 
>

Re: Review Request 71113: [ATLAS-3319] Add api to get lineage by uniqueAttributes

2019-07-18 Thread jiaqi shan 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/71113/
---

(Updated 七月 18, 2019, 1:01 p.m.)


Review request for atlas.


Bugs: ATLAS-3319
https://issues.apache.org/jira/browse/ATLAS-3319


Repository: atlas


Description
---

This patch add the follow api to support getting lineage by uniqueAttributes, 
also add the api to AtlasClientV2 accordingly:

/v2/lineage/uniqueAttribute/type/{typeName}


Diffs (updated)
-

  client/client-v2/src/main/java/org/apache/atlas/AtlasClientV2.java 7c8caee 
  webapp/src/main/java/org/apache/atlas/web/rest/LineageREST.java 8b0316b 


Diff: https://reviews.apache.org/r/71113/diff/2/

Changes: https://reviews.apache.org/r/71113/diff/1-2/


Testing
---

link used: 
localhost:21000/api/atlas/v2/lineage/uniqueAttribute/type/hive_table?attr:qualifiedName=sample.test2@bd
Request Type : GET
JSON :
{
"baseEntityGuid": "0c583c2b-d378-4a7b-98aa-be6c621f11b6",
"lineageDirection": "BOTH",
"lineageDepth": 3,
"guidEntityMap": {
"4ed7fb2e-e7bc-4384-86cd-fba9303879ae": {
"typeName": "hive_process_v1",
"attributes": {
"qualifiedName": 
"QUERY:sample.test3@bd->:INSERT:sample.test2@bd",
"name": "insert overwrite table test2 select * from test3"
},
"guid": "4ed7fb2e-e7bc-4384-86cd-fba9303879ae",
"status": "ACTIVE",
"displayText": "insert overwrite table test2 select * from test3",
"classificationNames": [],
"meaningNames": [],
"meanings": []
},
"ac6f5856-5881-4055-99de-8ecf874b74ee": {
"typeName": "hive_table",
"attributes": {
"owner": "cloud_service",
"createTime": 1562205245000,
"qualifiedName": "sample.test3@bd",
"name": "test3"
},
"guid": "ac6f5856-5881-4055-99de-8ecf874b74ee",
"status": "ACTIVE",
"displayText": "test3",
"classificationNames": [],
"meaningNames": [],
"meanings": []
},
"0c583c2b-d378-4a7b-98aa-be6c621f11b6": {
"typeName": "hive_table",
"attributes": {
"owner": "cloud_service",
"createTime": 1562145763000,
"qualifiedName": "sample.test2@bd",
"name": "test2"
},
"guid": "0c583c2b-d378-4a7b-98aa-be6c621f11b6",
"status": "ACTIVE",
"displayText": "test2",
"classificationNames": [],
"meaningNames": [],
"meanings": []
}
},
"relations": [
{
"fromEntityId": "ac6f5856-5881-4055-99de-8ecf874b74ee",
"toEntityId": "4ed7fb2e-e7bc-4384-86cd-fba9303879ae",
"relationshipId": "1a6f0644-cf51-485b-a1ea-0918f7652605"
},
{
"fromEntityId": "4ed7fb2e-e7bc-4384-86cd-fba9303879ae",
"toEntityId": "0c583c2b-d378-4a7b-98aa-be6c621f11b6",
"relationshipId": "7031e072-928e-4b8c-b7b9-6f368dc1be90"
}
]
}


Thanks,

jiaqi shan

[ATLAS-3319] Add api to get lineage by uniqueAttributes

2019-07-18 Thread SJQ 
[ATLAS-3319] was created Jiaqi Shan

Re: Review Request 71113: [ATLAS-3319] Add api to get lineage by uniqueAttributes

2019-07-18 Thread Nixon Rodrigues 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/71113/#review216715
---




webapp/src/main/java/org/apache/atlas/web/rest/LineageREST.java
Lines 110 (patched)


Can you add java doc here with usage of this API here

eg : 
v2/lineage/uniqueAttribute/type/hive_table?attr:qualifiedName=sample.test2@bd

So that it is be availabe in API documentation.


http://atlas.apache.org/api/v2/resource_EntityREST.html#resource_EntityREST_getEntitiesByUniqueAttributes_GET


- Nixon Rodrigues


On July 18, 2019, 5:20 a.m., jiaqi shan wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/71113/
> ---
> 
> (Updated July 18, 2019, 5:20 a.m.)
> 
> 
> Review request for atlas.
> 
> 
> Bugs: ATLAS-3319
> https://issues.apache.org/jira/browse/ATLAS-3319
> 
> 
> Repository: atlas
> 
> 
> Description
> ---
> 
> This patch add the follow api to support getting lineage by uniqueAttributes, 
> also add the api to AtlasClientV2 accordingly:
> 
> /v2/lineage/uniqueAttribute/type/{typeName}
> 
> 
> Diffs
> -
> 
>   client/client-v2/src/main/java/org/apache/atlas/AtlasClientV2.java f582fbd 
>   webapp/src/main/java/org/apache/atlas/web/rest/LineageREST.java 4210077 
> 
> 
> Diff: https://reviews.apache.org/r/71113/diff/1/
> 
> 
> Testing
> ---
> 
> link used: 
> localhost:21000/api/atlas/v2/lineage/uniqueAttribute/type/hive_table?attr:qualifiedName=sample.test2@bd
> Request Type : GET
> JSON :
> {
> "baseEntityGuid": "0c583c2b-d378-4a7b-98aa-be6c621f11b6",
> "lineageDirection": "BOTH",
> "lineageDepth": 3,
> "guidEntityMap": {
> "4ed7fb2e-e7bc-4384-86cd-fba9303879ae": {
> "typeName": "hive_process_v1",
> "attributes": {
> "qualifiedName": 
> "QUERY:sample.test3@bd->:INSERT:sample.test2@bd",
> "name": "insert overwrite table test2 select * from test3"
> },
> "guid": "4ed7fb2e-e7bc-4384-86cd-fba9303879ae",
> "status": "ACTIVE",
> "displayText": "insert overwrite table test2 select * from test3",
> "classificationNames": [],
> "meaningNames": [],
> "meanings": []
> },
> "ac6f5856-5881-4055-99de-8ecf874b74ee": {
> "typeName": "hive_table",
> "attributes": {
> "owner": "cloud_service",
> "createTime": 1562205245000,
> "qualifiedName": "sample.test3@bd",
> "name": "test3"
> },
> "guid": "ac6f5856-5881-4055-99de-8ecf874b74ee",
> "status": "ACTIVE",
> "displayText": "test3",
> "classificationNames": [],
> "meaningNames": [],
> "meanings": []
> },
> "0c583c2b-d378-4a7b-98aa-be6c621f11b6": {
> "typeName": "hive_table",
> "attributes": {
> "owner": "cloud_service",
> "createTime": 1562145763000,
> "qualifiedName": "sample.test2@bd",
> "name": "test2"
> },
> "guid": "0c583c2b-d378-4a7b-98aa-be6c621f11b6",
> "status": "ACTIVE",
> "displayText": "test2",
> "classificationNames": [],
> "meaningNames": [],
> "meanings": []
> }
> },
> "relations": [
> {
> "fromEntityId": "ac6f5856-5881-4055-99de-8ecf874b74ee",
> "toEntityId": "4ed7fb2e-e7bc-4384-86cd-fba9303879ae",
> "relationshipId": "1a6f0644-cf51-485b-a1ea-0918f7652605"
> },
> {
> "fromEntityId": "4ed7fb2e-e7bc-4384-86cd-fba9303879ae",
> "toEntityId": "0c583c2b-d378-4a7b-98aa-be6c621f11b6",
> "relationshipId": "7031e072-928e-4b8c-b7b9-6f368dc1be90"
> }
> ]
> }
> 
> 
> Thanks,
> 
> jiaqi shan
> 
>

Re: Review Request 71089: ATLAS-3338: avoid repeated calls to get index-field name during startup

2019-07-18 Thread Sridhar K 


> On July 18, 2019, 8:10 a.m., Sridhar K wrote:
> > Ship It!

Thanks for fixing it.


- Sridhar


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/71089/#review216713
---


On July 17, 2019, 6:45 a.m., Madhan Neethiraj wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/71089/
> ---
> 
> (Updated July 17, 2019, 6:45 a.m.)
> 
> 
> Review request for atlas, Ashutosh Mestry, Aadarsh Jajodia, keval bhatt, 
> Sridhar K, Le Ma, Mandar Ambawane, mayank jain, Nikhil Bonte, Nixon 
> Rodrigues, Saqeeb Shaikh, and Sarath Subramanian.
> 
> 
> Bugs: ATLAS-3338
> https://issues.apache.org/jira/browse/ATLAS-3338
> 
> 
> Repository: atlas
> 
> 
> Description
> ---
> 
> updated resolveIndexFieldName() to check the attribute instance in super-type 
> i.e. the type where the attribute is defined-in; only when this instance 
> doesn't have index-field name set, graph API is called to retrieve the value
> 
> 
> Diffs
> -
> 
>   
> repository/src/main/java/org/apache/atlas/repository/graph/GraphBackedSearchIndexer.java
>  95ae5049e 
> 
> 
> Diff: https://reviews.apache.org/r/71089/diff/2/
> 
> 
> Testing
> ---
> 
> - manaully verified that only one call is made to graph API during 
> type-system initialization/update
> - pre-commit tests run: 
> https://builds.apache.org/view/A/view/Atlas/job/PreCommit-ATLAS-Build-Test/1271
> 
> 
> Thanks,
> 
> Madhan Neethiraj
> 
>

Re: Review Request 71089: ATLAS-3338: avoid repeated calls to get index-field name during startup

2019-07-18 Thread Sridhar K 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/71089/#review216713
---


Ship it!




Ship It!

- Sridhar K


On July 17, 2019, 6:45 a.m., Madhan Neethiraj wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/71089/
> ---
> 
> (Updated July 17, 2019, 6:45 a.m.)
> 
> 
> Review request for atlas, Ashutosh Mestry, Aadarsh Jajodia, keval bhatt, 
> Sridhar K, Le Ma, Mandar Ambawane, mayank jain, Nikhil Bonte, Nixon 
> Rodrigues, Saqeeb Shaikh, and Sarath Subramanian.
> 
> 
> Bugs: ATLAS-3338
> https://issues.apache.org/jira/browse/ATLAS-3338
> 
> 
> Repository: atlas
> 
> 
> Description
> ---
> 
> updated resolveIndexFieldName() to check the attribute instance in super-type 
> i.e. the type where the attribute is defined-in; only when this instance 
> doesn't have index-field name set, graph API is called to retrieve the value
> 
> 
> Diffs
> -
> 
>   
> repository/src/main/java/org/apache/atlas/repository/graph/GraphBackedSearchIndexer.java
>  95ae5049e 
> 
> 
> Diff: https://reviews.apache.org/r/71089/diff/2/
> 
> 
> Testing
> ---
> 
> - manaully verified that only one call is made to graph API during 
> type-system initialization/update
> - pre-commit tests run: 
> https://builds.apache.org/view/A/view/Atlas/job/PreCommit-ATLAS-Build-Test/1271
> 
> 
> Thanks,
> 
> Madhan Neethiraj
> 
>