> On May 21, 2019, 7:09 p.m., Sridhar K wrote: > > addons/impala-bridge/src/main/java/org.apache.atlas.impala/hook/ImpalaHook.java > > Lines 19 (patched) > > <https://reviews.apache.org/r/70619/diff/6/?file=2146017#file2146017line19> > > > > Please package name w.r.t. directories. There are lot of classes in > > wrong folder strucutre. Please fix it. > > Na Li wrote: > will be fixed in https://issues.apache.org/jira/browse/ATLAS-3221
I have fixed the package issue in review version 9. However, do you really want to have integration change and fixing packaging in the same patch? As you can see, so many changes, the review gets really big, and you don't what has changed for integration with Impala. That is preciously why I want to focus this issue ATLAS-3080 on integration, and ATLAS-3221 on fixing packaging issue. - Na ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/70619/#review215417 ----------------------------------------------------------- On May 22, 2019, 2:15 a.m., Na Li wrote: > > ----------------------------------------------------------- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/70619/ > ----------------------------------------------------------- > > (Updated May 22, 2019, 2:15 a.m.) > > > Review request for atlas, Fredy Wijaya, Madhan Neethiraj, and Sarath > Subramanian. > > > Repository: atlas > > > Description > ------- > > Integrate with Impala hook using Impala private jar with post exec hook > > changes: > > 1) handle changed lineage record format change > The lineage record format has changed from having seperate table vertex > > { > "id":4, > "vertexType":"TABLE", > "vertexId":"db_1.table_1", > "createTime":1554750000 > } > > To embed the table create time inside column vertex > > { > "id": 3, > "vertexType": "COLUMN", > "vertexId": "db_1.table_1.tinyint_col", > "metadata": { > "createTime": 1557860978, > "tableName": "db_1.table_1" > } > } > > 2) add Impala hook API code to allow Atlas integration code to compile > without Impala updates > > > Diffs > ----- > > addons/impala-bridge-shim/pom.xml PRE-CREATION > > addons/impala-bridge-shim/src/main/java/org/apache/atlas/impala/hook/ImpalaLineageHook.java > PRE-CREATION > addons/impala-bridge/pom.xml 8e7e952 > > addons/impala-bridge/src/main/java/org.apache.atlas.impala/ImpalaLineageTool.java > 7c9abc8 > > addons/impala-bridge/src/main/java/org.apache.atlas.impala/hook/AtlasImpalaHookContext.java > 88faace > > addons/impala-bridge/src/main/java/org.apache.atlas.impala/hook/ImpalaIdentifierParser.java > b9d6cbb > > addons/impala-bridge/src/main/java/org.apache.atlas.impala/hook/ImpalaLineageHook.java > fdb6748 > > addons/impala-bridge/src/main/java/org.apache.atlas.impala/hook/ImpalaOperationParser.java > b9dd894 > > addons/impala-bridge/src/main/java/org.apache.atlas.impala/hook/events/BaseImpalaEvent.java > afe296c > > addons/impala-bridge/src/main/java/org.apache.atlas.impala/hook/events/CreateImpalaProcess.java > e4a38a3 > > addons/impala-bridge/src/main/java/org.apache.atlas.impala/model/IImpalaLineageHook.java > 7c1103a > > addons/impala-bridge/src/main/java/org.apache.atlas.impala/model/ImpalaDataType.java > 10ce448 > > addons/impala-bridge/src/main/java/org.apache.atlas.impala/model/ImpalaDependencyType.java > 892ee9b > > addons/impala-bridge/src/main/java/org.apache.atlas.impala/model/ImpalaNode.java > a3ddf53 > > addons/impala-bridge/src/main/java/org.apache.atlas.impala/model/ImpalaOperationType.java > a893b88 > > addons/impala-bridge/src/main/java/org.apache.atlas.impala/model/ImpalaQuery.java > 27bdc72 > > addons/impala-bridge/src/main/java/org.apache.atlas.impala/model/ImpalaVertexType.java > 8ec3f85 > > addons/impala-bridge/src/main/java/org.apache.atlas.impala/model/LineageEdge.java > 251507e > > addons/impala-bridge/src/main/java/org.apache.atlas.impala/model/LineageVertex.java > 82672c9 > > addons/impala-bridge/src/main/java/org/apache/atlas/impala/ImpalaLineageTool.java > PRE-CREATION > > addons/impala-bridge/src/main/java/org/apache/atlas/impala/hook/AtlasImpalaHookContext.java > PRE-CREATION > > addons/impala-bridge/src/main/java/org/apache/atlas/impala/hook/ImpalaHook.java > PRE-CREATION > > addons/impala-bridge/src/main/java/org/apache/atlas/impala/hook/ImpalaIdentifierParser.java > PRE-CREATION > > addons/impala-bridge/src/main/java/org/apache/atlas/impala/hook/ImpalaLineageHook.java > PRE-CREATION > > addons/impala-bridge/src/main/java/org/apache/atlas/impala/hook/ImpalaOperationParser.java > PRE-CREATION > > addons/impala-bridge/src/main/java/org/apache/atlas/impala/hook/events/BaseImpalaEvent.java > PRE-CREATION > > addons/impala-bridge/src/main/java/org/apache/atlas/impala/hook/events/CreateImpalaProcess.java > PRE-CREATION > > addons/impala-bridge/src/main/java/org/apache/atlas/impala/model/ImpalaDataType.java > PRE-CREATION > > addons/impala-bridge/src/main/java/org/apache/atlas/impala/model/ImpalaDependencyType.java > PRE-CREATION > > addons/impala-bridge/src/main/java/org/apache/atlas/impala/model/ImpalaNode.java > PRE-CREATION > > addons/impala-bridge/src/main/java/org/apache/atlas/impala/model/ImpalaOperationType.java > PRE-CREATION > > addons/impala-bridge/src/main/java/org/apache/atlas/impala/model/ImpalaQuery.java > PRE-CREATION > > addons/impala-bridge/src/main/java/org/apache/atlas/impala/model/ImpalaVertexType.java > PRE-CREATION > > addons/impala-bridge/src/main/java/org/apache/atlas/impala/model/LineageEdge.java > PRE-CREATION > > addons/impala-bridge/src/main/java/org/apache/atlas/impala/model/LineageVertex.java > PRE-CREATION > > addons/impala-bridge/src/main/java/org/apache/atlas/impala/model/LineageVertexMetadata.java > PRE-CREATION > > addons/impala-bridge/src/test/java/org/apache/atlas/impala/ImpalaLineageITBase.java > cc62955 > > addons/impala-bridge/src/test/java/org/apache/atlas/impala/ImpalaLineageToolIT.java > 6e4d321 > addons/impala-bridge/src/test/resources/impala1.json 8f747f6 > addons/impala-bridge/src/test/resources/impala2.json 239797b > addons/impala-bridge/src/test/resources/impala3.json 6a7d171 > addons/impala-bridge/src/test/resources/impala5.json 854969b > addons/impala-bridge/src/test/resources/impala6.json f136180 > addons/impala-bridge/src/test/resources/impala7.json f9ee670 > addons/impala-bridge/src/test/resources/impalaAlterViewAsSelect.json > PRE-CREATION > addons/impala-bridge/src/test/resources/impalaCreateTableAsSelect.json > PRE-CREATION > addons/impala-bridge/src/test/resources/impalaCreateView.json PRE-CREATION > addons/impala-bridge/src/test/resources/impalaCreateViewNoCreateTime.json > PRE-CREATION > addons/impala-bridge/src/test/resources/impalaInsertIntoAsSelect.json > PRE-CREATION > addons/impala-hook-api/pom.xml PRE-CREATION > > addons/impala-hook-api/src/main/java/org/apache/impala/hooks/PostQueryHookContext.java > PRE-CREATION > > addons/impala-hook-api/src/main/java/org/apache/impala/hooks/QueryExecHook.java > PRE-CREATION > pom.xml 98c2d69 > > > Diff: https://reviews.apache.org/r/70619/diff/9/ > > > Testing > ------- > > add integration tests > ImpalaLineageToolIT.testCreateViewFromFileWithoutCreateTime to handle the > situation that there is no createTime in lineage record > change the file content for other integration tests in ImpalaLineageToolIT to > handle new lineage record format > > > Thanks, > > Na Li > >
