[jira] [Commented] (ATLAS-284) UI not displaying the lineage for complex hive table (CTAS)

2015-11-05 Thread Shwetha G S (JIRA) 

[ 
https://issues.apache.org/jira/browse/ATLAS-284?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14993145#comment-14993145
 ] 

Shwetha G S commented on ATLAS-284:
---

Yes, hive hook is fired even through hive CLI

> UI not displaying the lineage for complex hive table (CTAS)
> ---
>
> Key: ATLAS-284
> URL: https://issues.apache.org/jira/browse/ATLAS-284
> Project: Atlas
>  Issue Type: Bug
>Affects Versions: 0.6-incubating
>Reporter: Ayub Khan
>Assignee: Rohit
> Fix For: 0.6-incubating
>
> Attachments: graph_not_shown_application.log
>
>
> UI not displaying the lineage for complex hive table (CTAS). 
> 1. On hive cli, create a hive table using CTAS just like below..
> Use this query:
> create table table_1251 as  select employee.salary as salary, t2.temp as 
> integer, t1.temp as number, nested.propertyid as
> string, nested.propertyName as propname, table_1243.number as num, 
> table_1250.number as prism,  temp_batting.col_value as
> temp1  from employee, t2, t1, nested, table_1243, table_1250, temp_batting;
> 2. Now try to view the graph on ATLAS UI for the above table. - No Graph is 
> displayed where as the backend API request is successful.
> {noformat}
> curl 
> 'http://localhost:21000/api/atlas/lineage/hive/table/primary.default.table_1261/inputs/graph'
>  -H 'Accept: application/json, text/plain, */*' -H 'Connection: keep-alive' 
> -H 'Accept-Encoding: gzip, deflate, sdch' -H 'Referer: 
> http://localhost:21000/' -H 'Accept-Language: en-US,en;q=0.8' -H 'User-Agent: 
> Mozilla/5.0 (Macintosh; Intel Mac OS X 10_10_5) AppleWebKit/537.36 (KHTML, 
> like Gecko) Chrome/46.0.2490.80 Safari/537.36' --compressed | python -m 
> json.tool
>   % Total% Received % Xferd  Average Speed   TimeTime Time  
> Current
>  Dload  Upload   Total   SpentLeft  Speed
> 100  38250  38250 0   3813  0 --:--:--  0:00:01 --:--:--  3813
> {
> "requestId": "qtp1610708593-415 - fb672eb4-2ce2-4485-8350-ba1f919ece02",
> "results": {
> "jsonClass": 
> "org.apache.atlas.typesystem.json.InstanceSerialization$_Struct",
> "typeName": "__tempQueryResultStruct65",
> "values": {
> "edges": {
> "5a3ecb0e-adc9-40c9-8296-9132d7ad1838": [
> "ebde1571-2ced-46d6-b65c-558835224764"
> ],
> "ebde1571-2ced-46d6-b65c-558835224764": [
> "364d6dc1-fdcc-4d03-8df7-cf1d835730af",
> "6b52aa8f-134d-4ab7-9389-4568fac87533",
> "7c1af593-b7b1-4838-815b-872b8a163a40",
> "ce01cbb6-ffd5-42c8-8c15-3f72bcef45c1",
> "b040d1e4-dd96-4729-8f55-4da4c958ff8f",
> "f82337ea-1e0c-4c28-a491-4916be594b49",
> "0f8b8f57-5aa4-4c30-be0a-c89eeedc2012"
> ]
> },
> "vertices": {
> "0f8b8f57-5aa4-4c30-be0a-c89eeedc2012": {
> "jsonClass": 
> "org.apache.atlas.typesystem.json.InstanceSerialization$_Struct",
> "typeName": "__tempQueryResultStruct64",
> "values": {
> "name": "primary.default.employee",
> "vertexId": {
> "jsonClass": 
> "org.apache.atlas.typesystem.json.InstanceSerialization$_Struct",
> "typeName": "__IdType",
> "values": {
> "guid": 
> "0f8b8f57-5aa4-4c30-be0a-c89eeedc2012",
> "typeName": "hive_table"
> }
> }
> }
> },
> "364d6dc1-fdcc-4d03-8df7-cf1d835730af": {
> "jsonClass": 
> "org.apache.atlas.typesystem.json.InstanceSerialization$_Struct",
> "typeName": "__tempQueryResultStruct64",
> "values": {
> "name": "primary.default.t1",
> "vertexId": {
> "jsonClass": 
> "org.apache.atlas.typesystem.json.InstanceSerialization$_Struct",
> "typeName": "__IdType",
> "values": {
> "guid": 
> "364d6dc1-fdcc-4d03-8df7-cf1d835730af",
> "typeName": "hive_table"
> }
> }
> }
> },
> "5a3ecb0e-adc9-40c9-8296-9132d7ad1838": {
> "jsonClass": 
> "org.apache.atlas.typesystem.json.InstanceSerialization$_Struct",
> "typeName": "__tempQueryResultStruct64",
>

[jira] [Commented] (ATLAS-284) UI not displaying the lineage for complex hive table (CTAS)

2015-11-05 Thread Ayub Khan (JIRA) 

[ 
https://issues.apache.org/jira/browse/ATLAS-284?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14992340#comment-14992340
 ] 

Ayub Khan commented on ATLAS-284:
-

[~bergenholtz] Yes, it can be seen with beeline as well.

UI snapshot link:https://monosnap.com/file/fIOtvrfD0WPEgrrvI4bGwcqZCQ0Bxj

> UI not displaying the lineage for complex hive table (CTAS)
> ---
>
> Key: ATLAS-284
> URL: https://issues.apache.org/jira/browse/ATLAS-284
> Project: Atlas
>  Issue Type: Bug
>Affects Versions: 0.6-incubating
>Reporter: Ayub Khan
>Assignee: Rohit
> Fix For: 0.6-incubating
>
> Attachments: graph_not_shown_application.log
>
>
> UI not displaying the lineage for complex hive table (CTAS). 
> 1. On hive cli, create a hive table using CTAS just like below..
> Use this query:
> create table table_1251 as  select employee.salary as salary, t2.temp as 
> integer, t1.temp as number, nested.propertyid as
> string, nested.propertyName as propname, table_1243.number as num, 
> table_1250.number as prism,  temp_batting.col_value as
> temp1  from employee, t2, t1, nested, table_1243, table_1250, temp_batting;
> 2. Now try to view the graph on ATLAS UI for the above table. - No Graph is 
> displayed where as the backend API request is successful.
> {noformat}
> curl 
> 'http://localhost:21000/api/atlas/lineage/hive/table/primary.default.table_1261/inputs/graph'
>  -H 'Accept: application/json, text/plain, */*' -H 'Connection: keep-alive' 
> -H 'Accept-Encoding: gzip, deflate, sdch' -H 'Referer: 
> http://localhost:21000/' -H 'Accept-Language: en-US,en;q=0.8' -H 'User-Agent: 
> Mozilla/5.0 (Macintosh; Intel Mac OS X 10_10_5) AppleWebKit/537.36 (KHTML, 
> like Gecko) Chrome/46.0.2490.80 Safari/537.36' --compressed | python -m 
> json.tool
>   % Total% Received % Xferd  Average Speed   TimeTime Time  
> Current
>  Dload  Upload   Total   SpentLeft  Speed
> 100  38250  38250 0   3813  0 --:--:--  0:00:01 --:--:--  3813
> {
> "requestId": "qtp1610708593-415 - fb672eb4-2ce2-4485-8350-ba1f919ece02",
> "results": {
> "jsonClass": 
> "org.apache.atlas.typesystem.json.InstanceSerialization$_Struct",
> "typeName": "__tempQueryResultStruct65",
> "values": {
> "edges": {
> "5a3ecb0e-adc9-40c9-8296-9132d7ad1838": [
> "ebde1571-2ced-46d6-b65c-558835224764"
> ],
> "ebde1571-2ced-46d6-b65c-558835224764": [
> "364d6dc1-fdcc-4d03-8df7-cf1d835730af",
> "6b52aa8f-134d-4ab7-9389-4568fac87533",
> "7c1af593-b7b1-4838-815b-872b8a163a40",
> "ce01cbb6-ffd5-42c8-8c15-3f72bcef45c1",
> "b040d1e4-dd96-4729-8f55-4da4c958ff8f",
> "f82337ea-1e0c-4c28-a491-4916be594b49",
> "0f8b8f57-5aa4-4c30-be0a-c89eeedc2012"
> ]
> },
> "vertices": {
> "0f8b8f57-5aa4-4c30-be0a-c89eeedc2012": {
> "jsonClass": 
> "org.apache.atlas.typesystem.json.InstanceSerialization$_Struct",
> "typeName": "__tempQueryResultStruct64",
> "values": {
> "name": "primary.default.employee",
> "vertexId": {
> "jsonClass": 
> "org.apache.atlas.typesystem.json.InstanceSerialization$_Struct",
> "typeName": "__IdType",
> "values": {
> "guid": 
> "0f8b8f57-5aa4-4c30-be0a-c89eeedc2012",
> "typeName": "hive_table"
> }
> }
> }
> },
> "364d6dc1-fdcc-4d03-8df7-cf1d835730af": {
> "jsonClass": 
> "org.apache.atlas.typesystem.json.InstanceSerialization$_Struct",
> "typeName": "__tempQueryResultStruct64",
> "values": {
> "name": "primary.default.t1",
> "vertexId": {
> "jsonClass": 
> "org.apache.atlas.typesystem.json.InstanceSerialization$_Struct",
> "typeName": "__IdType",
> "values": {
> "guid": 
> "364d6dc1-fdcc-4d03-8df7-cf1d835730af",
> "typeName": "hive_table"
> }
> }
> }
> },
> "5a3ecb0e-adc9-40c9-8296-9132d7ad1838": {
> "jsonClass": 
> "org.apache.atlas.typesystem.json.InstanceSerial

[jira] [Commented] (ATLAS-284) UI not displaying the lineage for complex hive table (CTAS)

2015-11-05 Thread Erik Bergenholtz (JIRA) 

[ 
https://issues.apache.org/jira/browse/ATLAS-284?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14992316#comment-14992316
 ] 

Erik Bergenholtz commented on ATLAS-284:


[~ayubkhan] - do we see the same behavior with Beeline?

[~shwethags] - does the atlas post execution hive hook fire through the hive 
cli? For some reason I recall some specific problem with this.

> UI not displaying the lineage for complex hive table (CTAS)
> ---
>
> Key: ATLAS-284
> URL: https://issues.apache.org/jira/browse/ATLAS-284
> Project: Atlas
>  Issue Type: Bug
>Affects Versions: 0.6-incubating
>Reporter: Ayub Khan
>Assignee: Rohit
> Fix For: 0.6-incubating
>
> Attachments: graph_not_shown_application.log
>
>
> UI not displaying the lineage for complex hive table (CTAS). 
> 1. On hive cli, create a hive table using CTAS just like below..
> Use this query:
> create table table_1251 as  select employee.salary as salary, t2.temp as 
> integer, t1.temp as number, nested.propertyid as
> string, nested.propertyName as propname, table_1243.number as num, 
> table_1250.number as prism,  temp_batting.col_value as
> temp1  from employee, t2, t1, nested, table_1243, table_1250, temp_batting;
> 2. Now try to view the graph on ATLAS UI for the above table. - No Graph is 
> displayed where as the backend API request is successful.
> {noformat}
> curl 
> 'http://localhost:21000/api/atlas/lineage/hive/table/primary.default.table_1261/inputs/graph'
>  -H 'Accept: application/json, text/plain, */*' -H 'Connection: keep-alive' 
> -H 'Accept-Encoding: gzip, deflate, sdch' -H 'Referer: 
> http://localhost:21000/' -H 'Accept-Language: en-US,en;q=0.8' -H 'User-Agent: 
> Mozilla/5.0 (Macintosh; Intel Mac OS X 10_10_5) AppleWebKit/537.36 (KHTML, 
> like Gecko) Chrome/46.0.2490.80 Safari/537.36' --compressed | python -m 
> json.tool
>   % Total% Received % Xferd  Average Speed   TimeTime Time  
> Current
>  Dload  Upload   Total   SpentLeft  Speed
> 100  38250  38250 0   3813  0 --:--:--  0:00:01 --:--:--  3813
> {
> "requestId": "qtp1610708593-415 - fb672eb4-2ce2-4485-8350-ba1f919ece02",
> "results": {
> "jsonClass": 
> "org.apache.atlas.typesystem.json.InstanceSerialization$_Struct",
> "typeName": "__tempQueryResultStruct65",
> "values": {
> "edges": {
> "5a3ecb0e-adc9-40c9-8296-9132d7ad1838": [
> "ebde1571-2ced-46d6-b65c-558835224764"
> ],
> "ebde1571-2ced-46d6-b65c-558835224764": [
> "364d6dc1-fdcc-4d03-8df7-cf1d835730af",
> "6b52aa8f-134d-4ab7-9389-4568fac87533",
> "7c1af593-b7b1-4838-815b-872b8a163a40",
> "ce01cbb6-ffd5-42c8-8c15-3f72bcef45c1",
> "b040d1e4-dd96-4729-8f55-4da4c958ff8f",
> "f82337ea-1e0c-4c28-a491-4916be594b49",
> "0f8b8f57-5aa4-4c30-be0a-c89eeedc2012"
> ]
> },
> "vertices": {
> "0f8b8f57-5aa4-4c30-be0a-c89eeedc2012": {
> "jsonClass": 
> "org.apache.atlas.typesystem.json.InstanceSerialization$_Struct",
> "typeName": "__tempQueryResultStruct64",
> "values": {
> "name": "primary.default.employee",
> "vertexId": {
> "jsonClass": 
> "org.apache.atlas.typesystem.json.InstanceSerialization$_Struct",
> "typeName": "__IdType",
> "values": {
> "guid": 
> "0f8b8f57-5aa4-4c30-be0a-c89eeedc2012",
> "typeName": "hive_table"
> }
> }
> }
> },
> "364d6dc1-fdcc-4d03-8df7-cf1d835730af": {
> "jsonClass": 
> "org.apache.atlas.typesystem.json.InstanceSerialization$_Struct",
> "typeName": "__tempQueryResultStruct64",
> "values": {
> "name": "primary.default.t1",
> "vertexId": {
> "jsonClass": 
> "org.apache.atlas.typesystem.json.InstanceSerialization$_Struct",
> "typeName": "__IdType",
> "values": {
> "guid": 
> "364d6dc1-fdcc-4d03-8df7-cf1d835730af",
> "typeName": "hive_table"
> }
> }
> }
> },
> "5a3ecb0e-adc9-40c9-8296-9132d7ad1838": {
>