[
https://issues.apache.org/jira/browse/IMPALA-10401?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17303138#comment-17303138
]
Quanlong Huang edited comment on IMPALA-10401 at 3/17/21, 6:26 AM:
-------------------------------------------------------------------
By adding the following configs in fe/src/test/resources/ranger-hive-audit.xml,
we can enabled audit logs to log4j:
{code:xml}
<property>
<name>xasecure.audit.is.enabled</name>
<value>true</value>
</property>
<property>
<name>xasecure.audit.destination.log4j</name>
<value>true</value>
</property>
{code}
However, it's just audits in the client side. E.g. we can find these in
impalad.INFO after executing a query:
{code:java}
I0317 14:19:16.222668 18674 Log4JAuditDestination.java:107]
{"repoType":3,"repo":"test_impala","reqUser":"quanlong","evtTime":"2021-03-17
14:19:16.089","access":"select","resource":"functional/alltypestiny","resType":"@table","action":"select","result":1,"agent":"impala","policy":2,"enforcer":"ranger-acl","cliIP":"::ffff:127.0.0.1","reqData":"select
id from
functional.alltypestiny","agentHost":"quanlong-OptiPlex-BJ","logType":"RangerAudit","id":"662c97a5-0cb5-42e0-bd09-13e8473ef0f7-0","seq_num":0,"event_count":1,"event_dur_ms":0,"tags":[],"cluster_name":"test-cluster","policy_version":2207}
I0317 14:19:16.222872 18674 Log4JAuditDestination.java:107]
{"repoType":3,"repo":"test_impala","reqUser":"quanlong","evtTime":"2021-03-17
14:19:16.089","access":"select","resource":"functional/alltypestiny/id","resType":"@column","action":"select","result":1,"agent":"impala","policy":2,"enforcer":"ranger-acl","cliIP":"::ffff:127.0.0.1","reqData":"select
id from
functional.alltypestiny","agentHost":"quanlong-OptiPlex-BJ","logType":"RangerAudit","id":"f788e4f9-8689-4fb9-a4d0-afd4f5e50a2e-0","seq_num":1,"event_count":1,"event_dur_ms":0,"tags":[],"cluster_name":"test-cluster","policy_version":2207}
{code}
The "policy" field shows which policy takes effect. The "result" field shows
whether the access is allowed.
For debugging e2e test like IMPALA-10587, we still need the access audits about
policies. Something that's shown in Ranger Admin Web UI: Audits->Admin
!Ranger Admin Access Audit.png|width=780,height=367!
I think we should either dumping the results of it or enable DEBUG logging for
ranger.
Refs:
[https://cwiki.apache.org/confluence/display/RANGER/Ranger+0.5+Audit+Configuration#Ranger0.5AuditConfiguration-AudittoLog4j]
[https://tomcat.apache.org/tomcat-8.0-doc/config/valve.html]
was (Author: stiga-huang):
By adding the following configs in fe/src/test/resources/ranger-hive-audit.xml,
we can enabled audit logs to log4j:
{code:xml}
<property>
<name>xasecure.audit.is.enabled</name>
<value>true</value>
</property>
<property>
<name>xasecure.audit.destination.log4j</name>
<value>true</value>
</property>
{code}
However, it's just audits in the client side. E.g. we can find these in
impalad.INFO after executing a query:
{code:java}
I0317 14:19:16.222668 18674 Log4JAuditDestination.java:107]
{"repoType":3,"repo":"test_impala","reqUser":"quanlong","evtTime":"2021-03-17
14:19:16.089","access":"select","resource":"functional/alltypestiny","resType":"@table","action":"select","result":1,"agent":"impala","policy":2,"enforcer":"ranger-acl","cliIP":"::ffff:127.0.0.1","reqData":"select
id from
functional.alltypestiny","agentHost":"quanlong-OptiPlex-BJ","logType":"RangerAudit","id":"662c97a5-0cb5-42e0-bd09-13e8473ef0f7-0","seq_num":0,"event_count":1,"event_dur_ms":0,"tags":[],"cluster_name":"test-cluster","policy_version":2207}
I0317 14:19:16.222872 18674 Log4JAuditDestination.java:107]
{"repoType":3,"repo":"test_impala","reqUser":"quanlong","evtTime":"2021-03-17
14:19:16.089","access":"select","resource":"functional/alltypestiny/id","resType":"@column","action":"select","result":1,"agent":"impala","policy":2,"enforcer":"ranger-acl","cliIP":"::ffff:127.0.0.1","reqData":"select
id from
functional.alltypestiny","agentHost":"quanlong-OptiPlex-BJ","logType":"RangerAudit","id":"f788e4f9-8689-4fb9-a4d0-afd4f5e50a2e-0","seq_num":1,"event_count":1,"event_dur_ms":0,"tags":[],"cluster_name":"test-cluster","policy_version":2207}
{code}
The "policy" field shows which policy takes effect. The "result" field shows
whether the access is allowed.
For debugging e2e test like IMPALA-10587, we still need the access audits about
policies. Something that's shown in Ranger Admin Web UI: Audits->Admin
!Ranger Admin Access Audit.png|width=780,height=367!
I think we should either dumping the results of it or enable DEBUG logging for
ranger.
> Enable Ranger Audit logs in minicluster
> ---------------------------------------
>
> Key: IMPALA-10401
> URL: https://issues.apache.org/jira/browse/IMPALA-10401
> Project: IMPALA
> Issue Type: New Feature
> Components: Infrastructure
> Reporter: Quanlong Huang
> Assignee: Quanlong Huang
> Priority: Major
> Attachments: Ranger Admin Access Audit.png
>
>
> It'd be helpful for debugging e2e ranger authz tests if we can get the audit
> logs. Ranger supports ingesting audit logs to Solr or HDFS files. We
> currently don't have Solr in our minicluster. We can try enabling ranger
> audit logs in HDFS and save the file for each job.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
