Gautam, >> Do we need to add timezone info also with event time, if yes then what >> should be the new format ? current format is "yyyy-MM-dd HH:mm:ss.SSS". For JSON, how about including the timezone? Please find how/if the new format (that includes timezone) can co-exist with the earlier format? If not, then we would have to support writing in older format – depending upon a configuration; by default the audit log should be written with the timezone.
Thanks, Madhan From: Gautam Borad <[email protected]<mailto:[email protected]>> on behalf of Gautam Borad <[email protected]<mailto:[email protected]>> Reply-To: Gautam Borad <[email protected]<mailto:[email protected]>> Date: Thursday, January 28, 2016 at 5:13 AM To: Abhay Kulkarni <[email protected]<mailto:[email protected]>>, Ramesh Mani <[email protected]<mailto:[email protected]>>, Velmurugan Periasamy <[email protected]<mailto:[email protected]>>, Alok Lal <[email protected]<mailto:[email protected]>>, Selvamohan Neethiraj <[email protected]<mailto:[email protected]>>, Madhan Neethiraj <[email protected]<mailto:[email protected]>>, Don Bosco Durai <[email protected]<mailto:[email protected]>> Cc: ranger <[email protected]<mailto:[email protected]>>, Gautam Borad <[email protected]<mailto:[email protected]>> Subject: Re: Review Request 42601: RANGER-798 - Approach 1 : Handle different timezone issue while saving audit logs to Solr This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/42601/ On January 28th, 2016, 8:32 a.m. UTC, Madhan Neethiraj wrote: agents-audit/src/main/java/org/apache/ranger/audit/destination/DBAuditDestination.java<https://reviews.apache.org/r/42601/diff/1/?file=1205404#file1205404line97>(Diff revision 1) public boolean log(Collection<AuditEventBase> events) { 97 eventCopy=cloneAuthzAuditEvent((AuthzAuditEvent)event); Instead of cloning and updating the eventDate in every destination (db/hdfs/solr/log4j), consider sending the "local" time (as set in AuthzAuditEvent.eventTime) to all audit destinations - exception DB, which require the time in UTC. For DB, consider updating the time in AuthzAuditEventDbObj(AuthzAuditEvent event) constructor. Do we need to add timezone info also with event time, if yes then what should be the new format ? current format is "yyyy-MM-dd HH:mm:ss.SSS". - Gautam On January 28th, 2016, 1:12 p.m. UTC, Gautam Borad wrote: Review request for ranger, Alok Lal, Don Bosco Durai, Abhay Kulkarni, Madhan Neethiraj, Ramesh Mani, Selvamohan Neethiraj, and Velmurugan Periasamy. By Gautam Borad. Updated Jan. 28, 2016, 1:12 p.m. Bugs: RANGER-798<https://issues.apache.org/jira/browse/RANGER-798> Repository: ranger Description Problem Statement: The current implementation of “auditEvent.getEventTime()” contains time in UTC and since it's a date object it will contain component machine's local timezone. When Solr receives this date object and timezone, it tries to convert it from given timezone to UTC timestamp, which leads to double conversion of actual time before it get stored in Solr. Proposed Solution: If we can provide server local time and timezone to Solr then Solr will convert the received time from given timezone to UTC. As an alternate solution, replaced getUTCDate() with new Date() object at various places for audit event time, all audit destination will receive local Date object, for Solr there will be no conversion on received Date object but for all other audit destination we need to convert the received Date value to UTC timestamp as audit logs are being stored in UTC timestamp for all service/component. If all destination thread are enabled then changing the received event object may create issue in other audit destination as same event object is refferred everywhere. Hence received event object attributes value are being copied in another local event object and the updated event time can be stored there, after this local event object will be used to convert that in JSON to write in HDFS, or can be persisted in DB. Testing Steps performed(after patch) : 1) Changed plugin system time zone to IST and restarted all components. 2) Initiated an HDFS audit event. 3) Checked event time of newly created audit log in Solr, Audit log event time was matching with UTC. 4) Checked event time in Ranger UI, newly generated Audit event is matching with current time. 5) Checked event time of newly created audit log in xa_access_audit table, Audit log event time was matching with UTC. 6) Checked event time of newly created audit log in HDFS logs, Audit log event time was matching with UTC. Note: Will test other services audit logs after this approach is reviewed. Diffs * agents-audit/src/main/java/org/apache/ranger/audit/entity/AuthzAuditEventDbObj.java (d52a60a) * agents-audit/src/main/java/org/apache/ranger/audit/provider/MiscUtil.java (9586f73) * agents-common/src/main/java/org/apache/ranger/plugin/policyengine/RangerAccessRequestImpl.java (fe50ca6) * hdfs-agent/src/main/java/org/apache/ranger/authorization/hadoop/RangerHdfsAuthorizer.java (5125af7) * hive-agent/src/main/java/org/apache/ranger/authorization/hive/authorizer/RangerHiveAccessRequest.java (2ae4149) * hive-agent/src/main/java/org/apache/ranger/authorization/hive/authorizer/RangerHiveAuditHandler.java (0f13577) * plugin-kafka/src/main/java/org/apache/ranger/authorization/kafka/authorizer/RangerKafkaAuthorizer.java (bb6a337) * plugin-kms/src/main/java/org/apache/ranger/authorization/kms/authorizer/RangerKmsAuthorizer.java (04b8b91) * plugin-solr/src/main/java/org/apache/ranger/authorization/solr/authorizer/RangerSolrAuthorizer.java (a8ecf15) * plugin-yarn/src/main/java/org/apache/ranger/authorization/yarn/authorizer/RangerYarnAuthorizer.java (ab9b7a9) View Diff<https://reviews.apache.org/r/42601/diff/>
