[
https://issues.apache.org/jira/browse/HIVE-9779?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14341607#comment-14341607
]
Abdelrahman Shettia commented on HIVE-9779:
-------------------------------------------
Hi [~thejas],
I have uploaded the patch file called 9979.002.patch. And here are the testing
results:
{code}
Beeline
curl
http://127.0.0.1:8188/ws/v1/timeline/HIVE_QUERY_ID?primaryFilter=requestuser:hue
{"entities":[{"events":[{"timestamp":1425088884579,"eventtype":"QUERY_COMPLETED","eventinfo":{}},{"timestamp":0,"eventtype":"QUERY_SUBMITTED","eventinfo":{}}],"entitytype":"HIVE_QUERY_ID","entity":"hive_20150228020000_e5581a7d-b0cf-4348-87af-a95c5405602c","starttime":0,"domain":"DEFAULT","otherinfo":{"QUERY":"{\"queryText\":\"select
count(1) from poke1 \",\"queryPlan\":{\"STAGE PLANS\":{\"Stage-1\":{\"Map
Reduce\":{\"Reduce Operator Tree:\":{\"Group By
Operator\":{\"mode:\":\"mergepartial\",\"aggregations:\":[\"count(VALUE._col0)\"],\"outputColumnNames:\":[\"_col0\"],\"children\":{\"File
Output
Operator\":{\"compressed:\":\"false\",\"table:\":{\"serde:\":\"org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe\",\"input
format:\":\"org.apache.hadoop.mapred.TextInputFormat\",\"output
format:\":\"org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat\"}}}}},\"Map
Operator Tree:\":[{\"TableScan\":{\"alias:\":\"poke1\",\"children\":{\"Select
Operator\":{\"children\":{\"Group By
Operator\":{\"mode:\":\"hash\",\"aggregations:\":[\"count(1)\"],\"outputColumnNames:\":[\"_col0\"],\"children\":{\"Reduce
Output Operator\":{\"sort order:\":\"\",\"value expressions:\":\"_col0 (type:
bigint)\"}}}}}}}}]}},\"Stage-0\":{\"Fetch
Operator\":{\"limit:\":\"-1\",\"Processor Tree:\":{\"ListSink\":{}}}}},\"STAGE
DEPENDENCIES\":{\"Stage-1\":{\"ROOT STAGE\":\"TRUE\"},\"Stage-0\":{\"DEPENDENT
STAGES\":\"Stage-1\"}}}}","TEZ":false,"STATUS":true,"MAPRED":true},"relatedentities":{},"primaryfilters":{"operationid":["l3T-7AI4Su-1m44E2RL05A"],"requestuser":["hue"],"user":["hive"]}}]}
curl
http://127.0.0.1:8188/ws/v1/timeline/HIVE_QUERY_ID?primaryFilter=requestuser:hue
{"entities":[{"events":[{"timestamp":1425088884579,"eventtype":"QUERY_COMPLETED","eventinfo":{}},{"timestamp":0,"eventtype":"QUERY_SUBMITTED","eventinfo":{}}],"entitytype":"HIVE_QUERY_ID","entity":"hive_20150228020000_e5581a7d-b0cf-4348-87af-a95c5405602c","starttime":0,"domain":"DEFAULT","otherinfo":{"QUERY":"{\"queryText\":\"select
count(1) from poke1 \",\"queryPlan\":{\"STAGE PLANS\":{\"Stage-1\":{\"Map
Reduce\":{\"Reduce Operator Tree:\":{\"Group By
Operator\":{\"mode:\":\"mergepartial\",\"aggregations:\":[\"count(VALUE._col0)\"],\"outputColumnNames:\":[\"_col0\"],\"children\":{\"File
Output
Operator\":{\"compressed:\":\"false\",\"table:\":{\"serde:\":\"org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe\",\"input
format:\":\"org.apache.hadoop.mapred.TextInputFormat\",\"output
format:\":\"org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat\"}}}}},\"Map
Operator Tree:\":[{\"TableScan\":{\"alias:\":\"poke1\",\"children\":{\"Select
Operator\":{\"children\":{\"Group By
Operator\":{\"mode:\":\"hash\",\"aggregations:\":[\"count(1)\"],\"outputColumnNames:\":[\"_col0\"],\"children\":{\"Reduce
Output Operator\":{\"sort order:\":\"\",\"value expressions:\":\"_col0 (type:
bigint)\"}}}}}}}}]}},\"Stage-0\":{\"Fetch
Operator\":{\"limit:\":\"-1\",\"Processor Tree:\":{\"ListSink\":{}}}}},\"STAGE
DEPENDENCIES\":{\"Stage-1\":{\"ROOT STAGE\":\"TRUE\"},\"Stage-0\":{\"DEPENDENT
STAGES\":\"Stage-1\"}}}}","TEZ":false,"STATUS":true,"MAPRED":true},"relatedentities":{},"primaryfilters":{"operationid":["l3T-7AI4Su-1m44E2RL05A"],"requestuser":["hue"],"user":["hive"]}}]}
Hive CLI :
curl http://127.0.0.1:8188/ws/v1/timeline/HIVE_QUERY_ID?primaryFilter=user:hue
{"entities":[{"events":[{"timestamp":1425091326014,"eventtype":"QUERY_COMPLETED","eventinfo":{}},{"timestamp":1425091289943,"eventtype":"QUERY_SUBMITTED","eventinfo":{}}],"entitytype":"HIVE_QUERY_ID","entity":"hue_20150228024141_3e6c0287-84b2-453b-ab19-67dddf87d42e","starttime":1425091289943,"domain":"DEFAULT","otherinfo":{"QUERY":"{\"queryText\":\"select
count(1) from poke1 \",\"queryPlan\":{\"STAGE PLANS\":{\"Stage-1\":{\"Map
Reduce\":{\"Reduce Operator Tree:\":{\"Group By
Operator\":{\"mode:\":\"mergepartial\",\"aggregations:\":[\"count(VALUE._col0)\"],\"outputColumnNames:\":[\"_col0\"],\"children\":{\"File
Output
Operator\":{\"compressed:\":\"false\",\"table:\":{\"serde:\":\"org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe\",\"input
format:\":\"org.apache.hadoop.mapred.TextInputFormat\",\"output
format:\":\"org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat\"}}}}},\"Map
Operator Tree:\":[{\"TableScan\":{\"alias:\":\"poke1\",\"children\":{\"Select
Operator\":{\"children\":{\"Group By
Operator\":{\"mode:\":\"hash\",\"aggregations:\":[\"count(1)\"],\"outputColumnNames:\":[\"_col0\"],\"children\":{\"Reduce
Output Operator\":{\"sort order:\":\"\",\"value expressions:\":\"_col0 (type:
bigint)\"}}}}}}}}]}},\"Stage-0\":{\"Fetch
Operator\":{\"limit:\":\"-1\",\"Processor Tree:\":{\"ListSink\":{}}}}},\"STAGE
DEPENDENCIES\":{\"Stage-1\":{\"ROOT STAGE\":\"TRUE\"},\"Stage-0\":{\"DEPENDENT
STAGES\":\"Stage-1\"}}}}","TEZ":false,"STATUS":true,"MAPRED":true},"relatedentities":{},"primaryfilters":{"requestuser":["hue"],"user":["hue"]}},{"events":[{"timestamp":1425089222614,"eventtype":"QUERY_COMPLETED","eventinfo":{}},{"timestamp":1425089185488,"eventtype":"QUERY_SUBMITTED","eventinfo":{}}],"entitytype":"HIVE_QUERY_ID","entity":"hue_20150228020606_9280ae8c-a2f1-4547-897f-b34100fb0d37","starttime":1425089185488,"domain":"DEFAULT","otherinfo":{"QUERY":"{\"queryText\":\"select
count(1) from poke1 \",\"queryPlan\":{\"STAGE PLANS\":{\"Stage-1\":{\"Map
Reduce\":{\"Reduce Operator Tree:\":{\"Group By
Operator\":{\"mode:\":\"mergepartial\",\"aggregations:\":[\"count(VALUE._col0)\"],\"outputColumnNames:\":[\"_col0\"],\"children\":{\"File
Output
Operator\":{\"compressed:\":\"false\",\"table:\":{\"serde:\":\"org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe\",\"input
format:\":\"org.apache.hadoop.mapred.TextInputFormat\",\"output
format:\":\"org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat\"}}}}},\"Map
Operator Tree:\":[{\"TableScan\":{\"alias:\":\"poke1\",\"children\":{\"Select
Operator\":{\"children\":{\"Group By
Operator\":{\"mode:\":\"hash\",\"aggregations:\":[\"count(1)\"],\"outputColumnNames:\":[\"_col0\"],\"children\":{\"Reduce
Output Operator\":{\"sort order:\":\"\",\"value expressions:\":\"_col0 (type:
bigint)\"}}}}}}}}]}},\"Stage-0\":{\"Fetch
Operator\":{\"limit:\":\"-1\",\"Processor Tree:\":{\"ListSink\":{}}}}},\"STAGE
DEPENDENCIES\":{\"Stage-1\":{\"ROOT STAGE\":\"TRUE\"},\"Stage-0\":{\"DEPENDENT
STAGES\":\"Stage-1\"}}}}","TEZ":false,"STATUS":true,"MAPRED":true},"relatedentities":{},"primaryfilters":{"requestuser":["hue"],"user":["hue"]}},{"events":[{"timestamp":1425085219623,"eventtype":"QUERY_COMPLETED","eventinfo":{}},{"timestamp":1425085173742,"eventtype":"QUERY_SUBMITTED","eventinfo":{}}],"entitytype":"HIVE_QUERY_ID","entity":"hive_20150228005959_8b59a353-5fd5-4886-af76-a076aa8c3470","starttime":1425085173742,"domain":"DEFAULT","otherinfo":{"QUERY":"{\"queryText\":\"select
count(1) from poke1 \",\"queryPlan\":{\"STAGE
PLANS\":{\"Stage-1\":{\"Tez\":{\"DagName:\":\"hive_20150228005959_8b59a353-5fd5-4886-af76-a076aa8c3470:1\",\"Vertices:\":{\"Reducer
2\":{\"Reduce Operator Tree:\":{\"Group By
Operator\":{\"mode:\":\"mergepartial\",\"aggregations:\":[\"count(VALUE._col0)\"],\"outputColumnNames:\":[\"_col0\"],\"children\":{\"File
Output Operator\":{\"Statistics:\":\"Num rows: 1 Data size: 8 Basic stats:
COMPLETE Column stats:
COMPLETE\",\"compressed:\":\"false\",\"table:\":{\"serde:\":\"org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe\",\"input
format:\":\"org.apache.hadoop.mapred.TextInputFormat\",\"output
format:\":\"org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat\"}}},\"Statistics:\":\"Num
rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: COMPLETE\"}}},\"Map
1\":{\"Map Operator
Tree:\":[{\"TableScan\":{\"alias:\":\"poke1\",\"children\":{\"Select
Operator\":{\"children\":{\"Group By
Operator\":{\"mode:\":\"hash\",\"aggregations:\":[\"count(1)\"],\"outputColumnNames:\":[\"_col0\"],\"children\":{\"Reduce
Output Operator\":{\"sort order:\":\"\",\"value expressions:\":\"_col0 (type:
bigint)\",\"Statistics:\":\"Num rows: 1 Data size: 8 Basic stats: COMPLETE
Column stats: COMPLETE\"}},\"Statistics:\":\"Num rows: 1 Data size: 8 Basic
stats: COMPLETE Column stats: COMPLETE\"}},\"Statistics:\":\"Num rows: 0 Data
size: 0 Basic stats: NONE Column stats: COMPLETE\"}},\"Statistics:\":\"Num
rows: 0 Data size: 0 Basic stats: NONE Column stats:
COMPLETE\"}}]}},\"Edges:\":{\"Reducer 2\":{\"parent\":\"Map
1\",\"type\":\"SIMPLE_EDGE\"}}}},\"Stage-0\":{\"Fetch
Operator\":{\"limit:\":\"-1\",\"Processor Tree:\":{\"ListSink\":{}}}}},\"STAGE
DEPENDENCIES\":{\"Stage-1\":{\"ROOT STAGE\":\"TRUE\"},\"Stage-0\":{\"DEPENDENT
STAGES\":\"Stage-1\"}}}}","TEZ":true,"STATUS":true,"MAPRED":false},"relatedentities":{},"primaryfilters":{"operationid":["8K0lnZIwTnOV3_4Bp_0cqA"],"user":["hive","hue"]}}]}
curl
http://127.0.0.1:8188/ws/v1/timeline/HIVE_QUERY_ID?primaryFilter=requestuser:hive
{"entities":[{"events":[{"timestamp":1425091144316,"eventtype":"QUERY_COMPLETED","eventinfo":{}},{"timestamp":1425091105837,"eventtype":"QUERY_SUBMITTED","eventinfo":{}}],"entitytype":"HIVE_QUERY_ID","entity":"hive_20150228023838_e2030858-4927-4ae6-97e4-afbad68c43f8","starttime":1425091105837,"domain":"DEFAULT","otherinfo":{"QUERY":"{\"queryText\":\"select
count(1) from poke1 \",\"queryPlan\":{\"STAGE PLANS\":{\"Stage-1\":{\"Map
Reduce\":{\"Reduce Operator Tree:\":{\"Group By
Operator\":{\"mode:\":\"mergepartial\",\"aggregations:\":[\"count(VALUE._col0)\"],\"outputColumnNames:\":[\"_col0\"],\"children\":{\"File
Output
Operator\":{\"compressed:\":\"false\",\"table:\":{\"serde:\":\"org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe\",\"input
format:\":\"org.apache.hadoop.mapred.TextInputFormat\",\"output
format:\":\"org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat\"}}}}},\"Map
Operator Tree:\":[{\"TableScan\":{\"alias:\":\"poke1\",\"children\":{\"Select
Operator\":{\"children\":{\"Group By
Operator\":{\"mode:\":\"hash\",\"aggregations:\":[\"count(1)\"],\"outputColumnNames:\":[\"_col0\"],\"children\":{\"Reduce
Output Operator\":{\"sort order:\":\"\",\"value expressions:\":\"_col0 (type:
bigint)\"}}}}}}}}]}},\"Stage-0\":{\"Fetch
Operator\":{\"limit:\":\"-1\",\"Processor Tree:\":{\"ListSink\":{}}}}},\"STAGE
DEPENDENCIES\":{\"Stage-1\":{\"ROOT STAGE\":\"TRUE\"},\"Stage-0\":{\"DEPENDENT
STAGES\":\"Stage-1\"}}}}","TEZ":false,"STATUS":true,"MAPRED":true},"relatedentities":{},"primaryfilters":{"requestuser":["hive"],"user":["hive"]}}]}
{code}
Please let me know if you have questions.
Thanks
-Rahman
> ATSHook does not log the end user if doAs=false (it logs the hs2 server user)
> -----------------------------------------------------------------------------
>
> Key: HIVE-9779
> URL: https://issues.apache.org/jira/browse/HIVE-9779
> Project: Hive
> Issue Type: Bug
> Components: HiveServer2
> Affects Versions: 0.13.0, 0.14.0
> Reporter: Abdelrahman Shettia
> Assignee: Abdelrahman Shettia
> Attachments: 9979.001.patch, 9979.002.patch
>
>
> When doAs=false, ATSHook should log the end username in ATS instead of
> logging the hiveserver2 user's name.
> The way things are, it is not possible for an admin to identify which query
> is being run by which user. The end user information is already available in
> the HookContext.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
