Re: Review Request 71620: ATLAS-3461: Impala process and lineage is not getting created appropriately if query contains variation like comment or extra spaces

[Sidharth Mishra]

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/71620/
-----------------------------------------------------------

(Updated Oct. 21, 2019, 9:01 p.m.)


Review request for atlas, Ashutosh Mestry, Madhan Neethiraj, and Sarath 
Subramanian.


Bugs: ATLAS-3461
    https://issues.apache.org/jira/browse/ATLAS-3461


Repository: atlas


Description
-------

Now even if the Impala queries contains comment/ extra spaces, Atlas will 
create the impala process approproately.
Even "create external table" will create Impala process at Atlas and user can 
see lineage.


Diffs (updated)
-----

  
addons/impala-bridge/src/main/java/org/apache/atlas/impala/hook/ImpalaOperationParser.java
 b9dd894f8 
  
addons/impala-bridge/src/test/java/org/apache/atlas/impala/ImpalaLineageToolIT.java
 dc4d1333e 
  
addons/impala-bridge/src/test/resources/impalaAlterViewAsSelectWithCommentSpaces.json
 PRE-CREATION 
  
addons/impala-bridge/src/test/resources/impalaCreateTableAsSelectWithCommentSpaces.json
 PRE-CREATION 
  
addons/impala-bridge/src/test/resources/impalaCreateViewWithCommentSpaces.json 
PRE-CREATION 


Diff: https://reviews.apache.org/r/71620/diff/3/

Changes: https://reviews.apache.org/r/71620/diff/2-3/


Testing
-------

1. Unit Tests added to ImpalaLineageToolIT and without fix it was failing. Now 
after the fix it works.
2. Validated manually by running below mentioned query from Impala:JDBC, Hue 
and Impala Shell

a) create     table testMon4 as select * from t1;

b) creAte external table /*test*/ testMo3 aS -- temp  fff
      /* temp dddd*/ -- sdfrnfkjef select
      select * from t1;
      

c) Create /*dd*/ table testMon4 aS select * from t1;

d) alter view  testMonView2 as select * from t1;

e) create     view testMonView4 as select * from t1;

f) alter view  testMonView4 as select * from t1;

g) create EXTERNAL TABLE IF NOT EXISTS thusday_ext1
   (STUD_ID int, 
   DEPT_ID int, 
   NAME string 
   ) 
   ROW FORMAT DELIMITED 
   FIELDS TERMINATED BY ',' 
   STORED AS TEXTFILE 
   LOCATION '/warehouse/tablespace/external/hive/db1.db/thusday_ext1';


Thanks,

Sidharth Mishra