----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/44377/#review122169 -----------------------------------------------------------
security-admin/src/main/java/org/apache/ranger/db/XXAccessAuditDao.java (line 43) <https://reviews.apache.org/r/44377/#comment184066> Long.valueOf(0L) - see RANGER-871 for more details. security-admin/src/main/java/org/apache/ranger/patch/cliutil/DbToSolrMigrationUtil.java (line 98) <https://reviews.apache.org/r/44377/#comment184075> log.error("Failed to read migration status from file " + CHECK_FILE_NAME, el); security-admin/src/main/java/org/apache/ranger/patch/cliutil/DbToSolrMigrationUtil.java (line 100) <https://reviews.apache.org/r/44377/#comment184069> logger.info("maxXXAccessAuditID:" + maxXXAccessAuditID) ==> logger.info("ID of the last available audit log: + " maxXXAccessAuditID) security-admin/src/main/java/org/apache/ranger/patch/cliutil/DbToSolrMigrationUtil.java (line 101) <https://reviews.apache.org/r/44377/#comment184068> logger.info("maxMigratedID: " + maxMigratedID); ==> if(maxMigratedID > 0) { logger.info("ID of the last migrated audit log: + " maxMigratedID); } security-admin/src/main/java/org/apache/ranger/patch/cliutil/DbToSolrMigrationUtil.java (line 103) <https://reviews.apache.org/r/44377/#comment184067> "No DB Audit logs available for migration" ==> "No more DB Audit logs to migrate. Last migrated audit log ID: " + maxMigratedID security-admin/src/main/java/org/apache/ranger/patch/cliutil/DbToSolrMigrationUtil.java (line 109) <https://reviews.apache.org/r/44377/#comment184070> minRange ==> rangeStart maxRange does not seem necessary. Use "rangeStart + maxRowsPerBatch" instead. security-admin/src/main/java/org/apache/ranger/patch/cliutil/DbToSolrMigrationUtil.java (line 124) <https://reviews.apache.org/r/44377/#comment184072> "to json" ==> "to Solr". Shouldn't processing stop here?? So that the next migration attempt will resume from this audit log? Also, make sure to update the status file with ID of the last migrated audit log. security-admin/src/main/java/org/apache/ranger/patch/cliutil/DbToSolrMigrationUtil.java (line 129) <https://reviews.apache.org/r/44377/#comment184073> "TimeTaken:" ==> "Batch #" + index + ": time taken:" security-admin/src/main/java/org/apache/ranger/patch/cliutil/DbToSolrMigrationUtil.java (line 137) <https://reviews.apache.org/r/44377/#comment184076> log.error("Failed to update migration status to file " + CHECK_FILE_NAME, el); security-admin/src/main/java/org/apache/ranger/patch/cliutil/DbToSolrMigrationUtil.java (line 184) <https://reviews.apache.org/r/44377/#comment184083> "Aborting." ?? Consider the following: "Failed to send audit event ID=" + xXAccessAudit.getId() security-admin/src/main/java/org/apache/ranger/solr/SolrAccessAuditsService.java (line 286) <https://reviews.apache.org/r/44377/#comment184086> if this added method is not needed, please remove and revert changes to this file. security-admin/src/main/resources/META-INF/jpa_named_queries.xml (line 910) <https://reviews.apache.org/r/44377/#comment184085> ">" ==> "≥" - else one record might be dropped for each iteration. Please review. - Madhan Neethiraj On March 4, 2016, 2:26 p.m., Pradeep Agrawal wrote: > > ----------------------------------------------------------- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/44377/ > ----------------------------------------------------------- > > (Updated March 4, 2016, 2:26 p.m.) > > > Review request for ranger, Alok Lal, Don Bosco Durai, Abhay Kulkarni, Madhan > Neethiraj, Ramesh Mani, Selvamohan Neethiraj, and Velmurugan Periasamy. > > > Bugs: RANGER-271 > https://issues.apache.org/jira/browse/RANGER-271 > > > Repository: ranger > > > Description > ------- > > **Problem Statement:** A migration script is required to move access audit > log from RDBMS to Solr. > > **Proposed Solution :** > > Make sure below given property are having required values in > conf/ranger-admin-site.xml , if you have upgraded Ranger with DB to Solr > support then there is possibility that you might not be having required > values in some of the below given properties. you can refer previous (before > upgrade) conf/ranger-admin-site.xml and manually copy-paste the value of > required properties which are not populated after upgrade. > > ranger.audit.solr.urls > ranger.audit.source.type > ranger.jpa.audit.jdbc.url > ranger.jpa.audit.jdbc.user > ranger.jpa.audit.jdbc.password > ranger.jpa.audit.jdbc.dialect > ranger.jpa.audit.jdbc.driver > > Sample Command : > > cd ranger-install-dir > > java -Dlogdir=ews/logs -Dlog4j.configuration=db_patch.log4j.xml -cp > ews/webapp/WEB-INF/classes/conf:ews/webapp/WEB-INF/classes/lib/*:ews/webapp/WEB-INF/:ews/webapp/META-INF/:ews/webapp/WEB-INF/lib/*:ews/webapp/WEB-INF/classes/:ews/webapp/WEB-INF/classes/META-INF:/usr/share/java/mysql-connector-java.jar > org.apache.ranger.patch.cliutil.DbToSolrMigrationUtil > > > Diffs > ----- > > security-admin/src/main/java/org/apache/ranger/common/DateUtil.java b710020 > security-admin/src/main/java/org/apache/ranger/db/XXAccessAuditDao.java > cd1add6 > > security-admin/src/main/java/org/apache/ranger/patch/cliutil/DbToSolrMigrationUtil.java > PRE-CREATION > > security-admin/src/main/java/org/apache/ranger/solr/SolrAccessAuditsService.java > 0d6eefa > security-admin/src/main/resources/META-INF/jpa_named_queries.xml 89764f5 > security-admin/src/main/resources/META-INF/persistence.xml 344e2e9 > > Diff: https://reviews.apache.org/r/44377/diff/ > > > Testing > ------- > > **Steps performed (with patch) :** > 1. Installed and Started Ranger Admin with Audit source from DB, Enabled HDFS > plugin with Audit to DB. > 2. Created HDFS service and policies and executed few hdfs command to make > sure that audit to db is working and audit logs are going to db and appears > under Ranger Admin Audit tab. > 3. Stopped Ranger Admin and updated value of below given properties in > ranger-admin-site.xml : > ranger.audit.solr.urls => http://localhost:6083/solr/ranger_audits > ranger.audit.source.type => solr > 4. Installed and configured Solr. > 5. Restarted Ranger Admin so that Ranger can read the audit data from Solr. > 6. On Ranger Admin UI Audit Tab don't have any records as there is no data in > Solr. > 7. Executed below given command from my current Ranger admin directory > > java -Dlogdir=ews/logs -Dlog4j.configuration=db_patch.log4j.xml -cp > ews/webapp/WEB-INF/classes/conf:ews/webapp/WEB-INF/classes/lib/*:ews/webapp/WEB-INF/:ews/webapp/META-INF/:ews/webapp/WEB-INF/lib/*:ews/webapp/WEB-INF/classes/:ews/webapp/WEB-INF/classes/META-INF:/usr/share/java/mysql-connector-java.jar > org.apache.ranger.patch.cliutil.DbToSolrMigrationUtil > > Results : > After above command DB records were added in Solr, and Audit logs can be seen > in Solr UI and Audit tab of Ranger Admin UI. > > > Thanks, > > Pradeep Agrawal > >
