1. All audit messages go into a single table. Based on the type of the
database you could create partition the table by date so that it is easy to
drop stale partitions. It should be then possible to setup a job that can
periodically drop stale partitions. Different databases have different ways to
do so.
* If you want to preserve the logs for longer than the database might
allow then also consider sending audit logging to HDFS in parallel. Note that
queries are run against data in database not HDFS. Tee’ing logging to HFDS
would be for archival purposes at timelines longer than the db can support.
2. It is not possible to piecemeal limit what is logged. Policy permissions
are not at the level of user action, e.g. PUT v/s SCAN, etc. Instead several
of user actions are clustered into a few access types like Read, Write, Create,
Admin.
Hopefully automating the periodic truncation of audit logs per the above means
would help to mitigate the need for this. The ideal solution is provided
however by https://issues.apache.org/jira/browse/RANGER-276. This is available
from 0.5 onwards. This can aggregate logs at source and put less burden on the
database or Solr in 0.5 onwards.
Thanks
On 6/8/15, 5:45 AM, "Bradman, Dale"
<[email protected]<mailto:[email protected]>> wrote:
Reason being was because the ranger MySql database had taken up all disk space
on the machine it was installed on.
As I am running HBase REST server with lots of messages coming through every
minute, the audit logs are getting absolutely hammered by “PUT” commands.
This is great and all, but is there a way to:
a) prune the logs without having to delete straight from the MySQL database?
b) Limit what is sent to the MySQL DB? For example, if it is a “PUT” command
coming in from HBase then do not write to the audit log for this. I know I can
turn the HBase auditing off completely but I would rather filter through what I
would and would not like filtered.
Thank you in advance.
Dale
On 21 May 2015, at 18:07, Alok Lal
<[email protected]<mailto:[email protected]>> wrote:
Perhaps the policy manager logs could point you the problem? They should
be under <install-dir>/ews/logs. If not then the command line of the
policy manager process contains the location of log files: run "ps -eaf |
grep Embed² and then look for logdir argument to the process.
Thanks
On 5/21/15, 5:22 AM, "Bradman, Dale"
<[email protected]<mailto:[email protected]>> wrote:
Hi,
When I log in to the Ranger UI and click on the audit tab as an Admin, it
says ³No Access Audit found!² Which is strange because it has been
working fine up until now. I have checked the MySql ranger_audit database
and it is still heavily populated with information. One thing that has
changed is the that there is no more disk space available on the node
that MySql is hosted on. This will be expanded in the next day or so but
in the mean time, surely I should still be able to retrieve the audit
logs?
Thanks,
Dale
________________________________
Capgemini is a trading name used by the Capgemini Group of companies
which includes Capgemini UK plc, a company registered in England and
Wales (number 943935) whose registered office is at No. 1, Forge End,
Woking, Surrey, GU21 6DB.
________________________________
Capgemini is a trading name used by the Capgemini Group of companies which
includes Capgemini UK plc, a company registered in England and Wales (number
943935) whose registered office is at No. 1, Forge End, Woking, Surrey, GU21
6DB.
This message contains information that may be privileged or confidential and is
the property of the Capgemini Group. It is intended only for the person to whom
it is addressed. If you are not the intended recipient, you are not authorized
to read, print, retain, copy, disseminate, distribute, or use this message or
any part thereof. If you receive this message in error, please notify the
sender immediately and delete all copies of this message.
