Re: Review Request 66027: ATLAS-2491:Atlas Hive hook should use atlas v2 api for Entity propagation

2018-03-19 Thread Madhan Neethiraj 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/66027/#review199470
---


Ship it!




Ship It!

- Madhan Neethiraj


On March 19, 2018, 5:57 p.m., Ramesh Mani wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/66027/
> ---
> 
> (Updated March 19, 2018, 5:57 p.m.)
> 
> 
> Review request for atlas and Madhan Neethiraj.
> 
> 
> Bugs: ATLAS-2491
> https://issues.apache.org/jira/browse/ATLAS-2491
> 
> 
> Repository: atlas
> 
> 
> Description
> ---
> 
> ATLAS-2491:Atlas Hive hook should use atlas v2 api for Entity propagation
> 
> 
> Diffs
> -
> 
>   
> addons/falcon-bridge/src/main/java/org/apache/atlas/falcon/bridge/FalconBridge.java
>  11d13b38 
>   addons/hive-bridge/pom.xml 14e42959 
>   
> addons/hive-bridge/src/main/java/org/apache/atlas/hive/bridge/ColumnLineageUtils.java
>  ba100085 
>   
> addons/hive-bridge/src/main/java/org/apache/atlas/hive/bridge/HiveMetaStoreBridge.java
>  4047c164 
>   
> addons/hive-bridge/src/main/java/org/apache/atlas/hive/hook/AtlasHiveHookContext.java
>  PRE-CREATION 
>   addons/hive-bridge/src/main/java/org/apache/atlas/hive/hook/HiveHook.java 
> 69d294cc 
>   
> addons/hive-bridge/src/main/java/org/apache/atlas/hive/hook/events/AlterDatabase.java
>  PRE-CREATION 
>   
> addons/hive-bridge/src/main/java/org/apache/atlas/hive/hook/events/AlterTable.java
>  PRE-CREATION 
>   
> addons/hive-bridge/src/main/java/org/apache/atlas/hive/hook/events/AlterTableRename.java
>  PRE-CREATION 
>   
> addons/hive-bridge/src/main/java/org/apache/atlas/hive/hook/events/AlterTableRenameCol.java
>  PRE-CREATION 
>   
> addons/hive-bridge/src/main/java/org/apache/atlas/hive/hook/events/AtlasEvent.java
>  PRE-CREATION 
>   
> addons/hive-bridge/src/main/java/org/apache/atlas/hive/hook/events/CreateDatabase.java
>  PRE-CREATION 
>   
> addons/hive-bridge/src/main/java/org/apache/atlas/hive/hook/events/CreateHiveProcess.java
>  PRE-CREATION 
>   
> addons/hive-bridge/src/main/java/org/apache/atlas/hive/hook/events/CreateTable.java
>  PRE-CREATION 
>   
> addons/hive-bridge/src/main/java/org/apache/atlas/hive/hook/events/DropDatabase.java
>  PRE-CREATION 
>   
> addons/hive-bridge/src/main/java/org/apache/atlas/hive/hook/events/DropTable.java
>  PRE-CREATION 
>   
> addons/hive-bridge/src/main/java/org/apache/atlas/hive/rewrite/ASTRewriter.java
>  3a2506bd 
>   
> addons/hive-bridge/src/main/java/org/apache/atlas/hive/rewrite/HiveASTRewriter.java
>  4cd219f3 
>   
> addons/hive-bridge/src/main/java/org/apache/atlas/hive/rewrite/LiteralRewriter.java
>  789b9812 
>   
> addons/hive-bridge/src/main/java/org/apache/atlas/hive/rewrite/RewriteContext.java
>  505616e9 
>   
> addons/hive-bridge/src/main/java/org/apache/atlas/hive/rewrite/RewriteException.java
>  c87bf6ba 
>   addons/hive-bridge/src/test/java/org/apache/atlas/hive/HiveITBase.java 
> 217c6aac 
>   
> addons/hive-bridge/src/test/java/org/apache/atlas/hive/bridge/HiveLiteralRewriterTest.java
>  f4abfb6a 
>   
> addons/hive-bridge/src/test/java/org/apache/atlas/hive/bridge/HiveMetaStoreBridgeTest.java
>  ea669f34 
>   
> addons/hive-bridge/src/test/java/org/apache/atlas/hive/bridge/HiveMetastoreBridgeIT.java
>  22d3c599 
>   addons/hive-bridge/src/test/java/org/apache/atlas/hive/hook/HiveHookIT.java 
> f96d47d4 
>   
> addons/sqoop-bridge/src/main/java/org/apache/atlas/sqoop/hook/SqoopHook.java 
> 06a0e9c4 
>   
> addons/storm-bridge/src/main/java/org/apache/atlas/storm/hook/StormAtlasHook.java
>  b0a4dab8 
> 
> 
> Diff: https://reviews.apache.org/r/66027/diff/3/
> 
> 
> Testing
> ---
> 
> Tested in local vm
> 
> 
> Thanks,
> 
> Ramesh Mani
> 
>

Re: Review Request 66027: ATLAS-2491:Atlas Hive hook should use atlas v2 api for Entity propagation

2018-03-19 Thread Ramesh Mani 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/66027/
---

(Updated March 19, 2018, 5:57 p.m.)


Review request for atlas and Madhan Neethiraj.


Changes
---

IT and Unit test fixes


Bugs: ATLAS-2491
https://issues.apache.org/jira/browse/ATLAS-2491


Repository: atlas


Description
---

ATLAS-2491:Atlas Hive hook should use atlas v2 api for Entity propagation


Diffs (updated)
-

  
addons/falcon-bridge/src/main/java/org/apache/atlas/falcon/bridge/FalconBridge.java
 11d13b38 
  addons/hive-bridge/pom.xml 14e42959 
  
addons/hive-bridge/src/main/java/org/apache/atlas/hive/bridge/ColumnLineageUtils.java
 ba100085 
  
addons/hive-bridge/src/main/java/org/apache/atlas/hive/bridge/HiveMetaStoreBridge.java
 4047c164 
  
addons/hive-bridge/src/main/java/org/apache/atlas/hive/hook/AtlasHiveHookContext.java
 PRE-CREATION 
  addons/hive-bridge/src/main/java/org/apache/atlas/hive/hook/HiveHook.java 
69d294cc 
  
addons/hive-bridge/src/main/java/org/apache/atlas/hive/hook/events/AlterDatabase.java
 PRE-CREATION 
  
addons/hive-bridge/src/main/java/org/apache/atlas/hive/hook/events/AlterTable.java
 PRE-CREATION 
  
addons/hive-bridge/src/main/java/org/apache/atlas/hive/hook/events/AlterTableRename.java
 PRE-CREATION 
  
addons/hive-bridge/src/main/java/org/apache/atlas/hive/hook/events/AlterTableRenameCol.java
 PRE-CREATION 
  
addons/hive-bridge/src/main/java/org/apache/atlas/hive/hook/events/AtlasEvent.java
 PRE-CREATION 
  
addons/hive-bridge/src/main/java/org/apache/atlas/hive/hook/events/CreateDatabase.java
 PRE-CREATION 
  
addons/hive-bridge/src/main/java/org/apache/atlas/hive/hook/events/CreateHiveProcess.java
 PRE-CREATION 
  
addons/hive-bridge/src/main/java/org/apache/atlas/hive/hook/events/CreateTable.java
 PRE-CREATION 
  
addons/hive-bridge/src/main/java/org/apache/atlas/hive/hook/events/DropDatabase.java
 PRE-CREATION 
  
addons/hive-bridge/src/main/java/org/apache/atlas/hive/hook/events/DropTable.java
 PRE-CREATION 
  
addons/hive-bridge/src/main/java/org/apache/atlas/hive/rewrite/ASTRewriter.java 
3a2506bd 
  
addons/hive-bridge/src/main/java/org/apache/atlas/hive/rewrite/HiveASTRewriter.java
 4cd219f3 
  
addons/hive-bridge/src/main/java/org/apache/atlas/hive/rewrite/LiteralRewriter.java
 789b9812 
  
addons/hive-bridge/src/main/java/org/apache/atlas/hive/rewrite/RewriteContext.java
 505616e9 
  
addons/hive-bridge/src/main/java/org/apache/atlas/hive/rewrite/RewriteException.java
 c87bf6ba 
  addons/hive-bridge/src/test/java/org/apache/atlas/hive/HiveITBase.java 
217c6aac 
  
addons/hive-bridge/src/test/java/org/apache/atlas/hive/bridge/HiveLiteralRewriterTest.java
 f4abfb6a 
  
addons/hive-bridge/src/test/java/org/apache/atlas/hive/bridge/HiveMetaStoreBridgeTest.java
 ea669f34 
  
addons/hive-bridge/src/test/java/org/apache/atlas/hive/bridge/HiveMetastoreBridgeIT.java
 22d3c599 
  addons/hive-bridge/src/test/java/org/apache/atlas/hive/hook/HiveHookIT.java 
f96d47d4 
  addons/sqoop-bridge/src/main/java/org/apache/atlas/sqoop/hook/SqoopHook.java 
06a0e9c4 
  
addons/storm-bridge/src/main/java/org/apache/atlas/storm/hook/StormAtlasHook.java
 b0a4dab8 


Diff: https://reviews.apache.org/r/66027/diff/3/

Changes: https://reviews.apache.org/r/66027/diff/2-3/


Testing
---

Tested in local vm


Thanks,

Ramesh Mani

Re: Review Request 66027: ATLAS-2491:Atlas Hive hook should use atlas v2 api for Entity propagation

2018-03-15 Thread Ramesh Mani 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/66027/
---

(Updated March 15, 2018, 8:31 a.m.)


Review request for atlas and Madhan Neethiraj.


Changes
---

Fixed review comments


Bugs: ATLAS-2491
https://issues.apache.org/jira/browse/ATLAS-2491


Repository: atlas


Description
---

ATLAS-2491:Atlas Hive hook should use atlas v2 api for Entity propagation


Diffs (updated)
-

  addons/hive-bridge/pom.xml 14e4295 
  
addons/hive-bridge/src/main/java/org/apache/atlas/hive/bridge/HiveMetaStoreBridge.java
 4047c16 
  
addons/hive-bridge/src/main/java/org/apache/atlas/hive/hook/AtlasHiveHookContext.java
 PRE-CREATION 
  addons/hive-bridge/src/main/java/org/apache/atlas/hive/hook/HiveHook.java 
69d294c 
  
addons/hive-bridge/src/main/java/org/apache/atlas/hive/hook/events/AlterDatabase.java
 PRE-CREATION 
  
addons/hive-bridge/src/main/java/org/apache/atlas/hive/hook/events/AlterTable.java
 PRE-CREATION 
  
addons/hive-bridge/src/main/java/org/apache/atlas/hive/hook/events/AlterTableRename.java
 PRE-CREATION 
  
addons/hive-bridge/src/main/java/org/apache/atlas/hive/hook/events/AlterTableRenameCol.java
 PRE-CREATION 
  
addons/hive-bridge/src/main/java/org/apache/atlas/hive/hook/events/AtlasEvent.java
 PRE-CREATION 
  
addons/hive-bridge/src/main/java/org/apache/atlas/hive/hook/events/CreateDatabase.java
 PRE-CREATION 
  
addons/hive-bridge/src/main/java/org/apache/atlas/hive/hook/events/CreateHiveProcess.java
 PRE-CREATION 
  
addons/hive-bridge/src/main/java/org/apache/atlas/hive/hook/events/CreateTable.java
 PRE-CREATION 
  
addons/hive-bridge/src/main/java/org/apache/atlas/hive/hook/events/DropDatabase.java
 PRE-CREATION 
  
addons/hive-bridge/src/main/java/org/apache/atlas/hive/hook/events/DropTable.java
 PRE-CREATION 
  addons/hive-bridge/src/test/java/org/apache/atlas/hive/HiveITBase.java 
217c6aa 
  
addons/hive-bridge/src/test/java/org/apache/atlas/hive/bridge/HiveLiteralRewriterTest.java
 f4abfb6 
  
addons/hive-bridge/src/test/java/org/apache/atlas/hive/bridge/HiveMetaStoreBridgeTest.java
 ea669f3 
  
addons/hive-bridge/src/test/java/org/apache/atlas/hive/bridge/HiveMetastoreBridgeIT.java
 22d3c59 
  addons/hive-bridge/src/test/java/org/apache/atlas/hive/hook/HiveHookIT.java 
f96d47d 
  intg/src/main/java/org/apache/atlas/model/instance/AtlasEntity.java fce46da 


Diff: https://reviews.apache.org/r/66027/diff/2/

Changes: https://reviews.apache.org/r/66027/diff/1-2/


Testing
---

Tested in local vm


Thanks,

Ramesh Mani

Re: Review Request 66027: ATLAS-2491:Atlas Hive hook should use atlas v2 api for Entity propagation

2018-03-12 Thread Madhan Neethiraj 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/66027/#review199003
---




addons/hive-bridge/src/main/java/org/apache/atlas/hive/bridge/HiveMetaStoreBridge.java
Line 321 (original), 348 (patched)


V2 APIs don't allow 'AtlasEntity' in attributes.
- use AtlasEntitiesWithExtInfo as type for 'lineageProcess'
- replace AtlasEntity/List in attributes with 
AtlasObjectId/List
- add AtlasEntity instances in lineageProcess.referredEntities

registerInstance() should take AtlasEntitiesWithExtInfo as argument type.


- Madhan Neethiraj


On March 12, 2018, 4:24 a.m., Ramesh Mani wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/66027/
> ---
> 
> (Updated March 12, 2018, 4:24 a.m.)
> 
> 
> Review request for atlas and Madhan Neethiraj.
> 
> 
> Bugs: ATLAS-2491
> https://issues.apache.org/jira/browse/ATLAS-2491
> 
> 
> Repository: atlas
> 
> 
> Description
> ---
> 
> ATLAS-2491:Atlas Hive hook should use atlas v2 api for Entity propagation
> 
> 
> Diffs
> -
> 
>   addons/hive-bridge/pom.xml 14e4295 
>   
> addons/hive-bridge/src/main/java/org/apache/atlas/hive/bridge/HiveMetaStoreBridge.java
>  4047c16 
>   
> addons/hive-bridge/src/main/java/org/apache/atlas/hive/hook/AtlasHiveHookContext.java
>  PRE-CREATION 
>   addons/hive-bridge/src/main/java/org/apache/atlas/hive/hook/HiveHook.java 
> 69d294c 
>   
> addons/hive-bridge/src/main/java/org/apache/atlas/hive/hook/events/AlterDatabase.java
>  PRE-CREATION 
>   
> addons/hive-bridge/src/main/java/org/apache/atlas/hive/hook/events/AlterTable.java
>  PRE-CREATION 
>   
> addons/hive-bridge/src/main/java/org/apache/atlas/hive/hook/events/AlterTableRename.java
>  PRE-CREATION 
>   
> addons/hive-bridge/src/main/java/org/apache/atlas/hive/hook/events/AlterTableRenameCol.java
>  PRE-CREATION 
>   
> addons/hive-bridge/src/main/java/org/apache/atlas/hive/hook/events/AtlasEvent.java
>  PRE-CREATION 
>   
> addons/hive-bridge/src/main/java/org/apache/atlas/hive/hook/events/CreateDatabase.java
>  PRE-CREATION 
>   
> addons/hive-bridge/src/main/java/org/apache/atlas/hive/hook/events/CreateHiveProcess.java
>  PRE-CREATION 
>   
> addons/hive-bridge/src/main/java/org/apache/atlas/hive/hook/events/CreateTable.java
>  PRE-CREATION 
>   
> addons/hive-bridge/src/main/java/org/apache/atlas/hive/hook/events/DropDatabase.java
>  PRE-CREATION 
>   
> addons/hive-bridge/src/main/java/org/apache/atlas/hive/hook/events/DropTable.java
>  PRE-CREATION 
>   addons/hive-bridge/src/test/java/org/apache/atlas/hive/HiveITBase.java 
> 217c6aa 
>   
> addons/hive-bridge/src/test/java/org/apache/atlas/hive/bridge/HiveLiteralRewriterTest.java
>  f4abfb6 
>   
> addons/hive-bridge/src/test/java/org/apache/atlas/hive/bridge/HiveMetaStoreBridgeTest.java
>  ea669f3 
>   
> addons/hive-bridge/src/test/java/org/apache/atlas/hive/bridge/HiveMetastoreBridgeIT.java
>  22d3c59 
>   addons/hive-bridge/src/test/java/org/apache/atlas/hive/hook/HiveHookIT.java 
> f96d47d 
>   intg/src/main/java/org/apache/atlas/model/instance/AtlasEntity.java fce46da 
> 
> 
> Diff: https://reviews.apache.org/r/66027/diff/1/
> 
> 
> Testing
> ---
> 
> Tested in local vm
> 
> 
> Thanks,
> 
> Ramesh Mani
> 
>

Re: Review Request 66027: ATLAS-2491:Atlas Hive hook should use atlas v2 api for Entity propagation

2018-03-11 Thread Ramesh Mani 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/66027/
---

(Updated March 12, 2018, 4:24 a.m.)


Review request for atlas and Madhan Neethiraj.


Bugs: ATLAS-2491
https://issues.apache.org/jira/browse/ATLAS-2491


Repository: atlas


Description
---

ATLAS-2491:Atlas Hive hook should use atlas v2 api for Entity propagation


Diffs
-

  addons/hive-bridge/pom.xml 14e4295 
  
addons/hive-bridge/src/main/java/org/apache/atlas/hive/bridge/HiveMetaStoreBridge.java
 4047c16 
  
addons/hive-bridge/src/main/java/org/apache/atlas/hive/hook/AtlasHiveHookContext.java
 PRE-CREATION 
  addons/hive-bridge/src/main/java/org/apache/atlas/hive/hook/HiveHook.java 
69d294c 
  
addons/hive-bridge/src/main/java/org/apache/atlas/hive/hook/events/AlterDatabase.java
 PRE-CREATION 
  
addons/hive-bridge/src/main/java/org/apache/atlas/hive/hook/events/AlterTable.java
 PRE-CREATION 
  
addons/hive-bridge/src/main/java/org/apache/atlas/hive/hook/events/AlterTableRename.java
 PRE-CREATION 
  
addons/hive-bridge/src/main/java/org/apache/atlas/hive/hook/events/AlterTableRenameCol.java
 PRE-CREATION 
  
addons/hive-bridge/src/main/java/org/apache/atlas/hive/hook/events/AtlasEvent.java
 PRE-CREATION 
  
addons/hive-bridge/src/main/java/org/apache/atlas/hive/hook/events/CreateDatabase.java
 PRE-CREATION 
  
addons/hive-bridge/src/main/java/org/apache/atlas/hive/hook/events/CreateHiveProcess.java
 PRE-CREATION 
  
addons/hive-bridge/src/main/java/org/apache/atlas/hive/hook/events/CreateTable.java
 PRE-CREATION 
  
addons/hive-bridge/src/main/java/org/apache/atlas/hive/hook/events/DropDatabase.java
 PRE-CREATION 
  
addons/hive-bridge/src/main/java/org/apache/atlas/hive/hook/events/DropTable.java
 PRE-CREATION 
  addons/hive-bridge/src/test/java/org/apache/atlas/hive/HiveITBase.java 
217c6aa 
  
addons/hive-bridge/src/test/java/org/apache/atlas/hive/bridge/HiveLiteralRewriterTest.java
 f4abfb6 
  
addons/hive-bridge/src/test/java/org/apache/atlas/hive/bridge/HiveMetaStoreBridgeTest.java
 ea669f3 
  
addons/hive-bridge/src/test/java/org/apache/atlas/hive/bridge/HiveMetastoreBridgeIT.java
 22d3c59 
  addons/hive-bridge/src/test/java/org/apache/atlas/hive/hook/HiveHookIT.java 
f96d47d 
  intg/src/main/java/org/apache/atlas/model/instance/AtlasEntity.java fce46da 


Diff: https://reviews.apache.org/r/66027/diff/1/


Testing
---

Tested in local vm


Thanks,

Ramesh Mani

Review Request 66027: ATLAS-2491:Atlas Hive hook should use atlas v2 api for Entity propagation

2018-03-11 Thread Ramesh Mani 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/66027/
---

Review request for atlas and Madhan Neethiraj.


Repository: atlas


Description
---

ATLAS-2491:Atlas Hive hook should use atlas v2 api for Entity propagation


Diffs
-

  addons/hive-bridge/pom.xml 14e4295 
  
addons/hive-bridge/src/main/java/org/apache/atlas/hive/bridge/HiveMetaStoreBridge.java
 4047c16 
  
addons/hive-bridge/src/main/java/org/apache/atlas/hive/hook/AtlasHiveHookContext.java
 PRE-CREATION 
  addons/hive-bridge/src/main/java/org/apache/atlas/hive/hook/HiveHook.java 
69d294c 
  
addons/hive-bridge/src/main/java/org/apache/atlas/hive/hook/events/AlterDatabase.java
 PRE-CREATION 
  
addons/hive-bridge/src/main/java/org/apache/atlas/hive/hook/events/AlterTable.java
 PRE-CREATION 
  
addons/hive-bridge/src/main/java/org/apache/atlas/hive/hook/events/AlterTableRename.java
 PRE-CREATION 
  
addons/hive-bridge/src/main/java/org/apache/atlas/hive/hook/events/AlterTableRenameCol.java
 PRE-CREATION 
  
addons/hive-bridge/src/main/java/org/apache/atlas/hive/hook/events/AtlasEvent.java
 PRE-CREATION 
  
addons/hive-bridge/src/main/java/org/apache/atlas/hive/hook/events/CreateDatabase.java
 PRE-CREATION 
  
addons/hive-bridge/src/main/java/org/apache/atlas/hive/hook/events/CreateHiveProcess.java
 PRE-CREATION 
  
addons/hive-bridge/src/main/java/org/apache/atlas/hive/hook/events/CreateTable.java
 PRE-CREATION 
  
addons/hive-bridge/src/main/java/org/apache/atlas/hive/hook/events/DropDatabase.java
 PRE-CREATION 
  
addons/hive-bridge/src/main/java/org/apache/atlas/hive/hook/events/DropTable.java
 PRE-CREATION 
  addons/hive-bridge/src/test/java/org/apache/atlas/hive/HiveITBase.java 
217c6aa 
  
addons/hive-bridge/src/test/java/org/apache/atlas/hive/bridge/HiveLiteralRewriterTest.java
 f4abfb6 
  
addons/hive-bridge/src/test/java/org/apache/atlas/hive/bridge/HiveMetaStoreBridgeTest.java
 ea669f3 
  
addons/hive-bridge/src/test/java/org/apache/atlas/hive/bridge/HiveMetastoreBridgeIT.java
 22d3c59 
  addons/hive-bridge/src/test/java/org/apache/atlas/hive/hook/HiveHookIT.java 
f96d47d 
  intg/src/main/java/org/apache/atlas/model/instance/AtlasEntity.java fce46da 


Diff: https://reviews.apache.org/r/66027/diff/1/


Testing
---

Tested in local vm


Thanks,

Ramesh Mani