> On April 14, 2016, 4:39 p.m., Velmurugan Periasamy wrote: > > security-admin/scripts/db_setup.py, line 2109 > > <https://reviews.apache.org/r/45091/diff/2/?file=1341082#file1341082line2109> > > > > Is it required to use db_name as audit_db_name here? Same for other > > properties?
Have commented these assignments; please review. - Pradeep ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/45091/#review128903 ----------------------------------------------------------- On April 14, 2016, 6:39 p.m., Pradeep Agrawal wrote: > > ----------------------------------------------------------- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/45091/ > ----------------------------------------------------------- > > (Updated April 14, 2016, 6:39 p.m.) > > > Review request for ranger, Alok Lal, Don Bosco Durai, Gautam Borad, Abhay > Kulkarni, Madhan Neethiraj, Ramesh Mani, Selvamohan Neethiraj, and Velmurugan > Periasamy. > > > Bugs: RANGER-900 > https://issues.apache.org/jira/browse/RANGER-900 > > > Repository: ranger > > > Description > ------- > > **Problem Statement :** > Remove option to store audit to DB as storing audit logs in db requires lots > of data management activity and frequent backup-restore process might hamper > Ranger application performance. Production team might face frequent down-time > issues due to db disk space reclaim activities. > > > **Proposed Solution :** > Proposed solution is having below mentioned approch : > 1. Remove audit to DB related properties from install.properties of all > components. > 2. Disable shell script code to read audit to DB related properties from > install.properties of all components. > 3. Disable code from dba_script.py which is invoked to create audit DB > schema, audit User and executes grants privileges. > 4. Disable code from db_setup.py to skip 'xa_access_audit' table creation and > stop executing audit to Db related sql patches. > 5. Make solr as mandatory audit data store/source. > > > Diffs > ----- > > agents-audit/src/main/java/org/apache/ranger/audit/test/TestEvents.java > 3e89cc4 > agents-common/scripts/enable-agent.sh b9511d2 > > agents-common/src/test/java/org/apache/ranger/plugin/policyengine/TestPolicyEngine.java > 8ee6bea > hbase-agent/conf/ranger-hbase-audit-changes.cfg e29ccd5 > hbase-agent/scripts/install.properties 795ea3e > hdfs-agent/conf/ranger-hdfs-audit-changes.cfg 9c88450 > hdfs-agent/scripts/install.properties b4dda13 > hive-agent/conf/ranger-hive-audit-changes.cfg 4e61c7d > hive-agent/scripts/install.properties 6b71a85 > kms/scripts/install.properties 7762948 > knox-agent/conf/ranger-knox-audit-changes.cfg f722e53 > knox-agent/scripts/install.properties 1febd49 > plugin-kafka/conf/ranger-kafka-audit-changes.cfg 46ee29a > plugin-kafka/scripts/install.properties 79ea6db > plugin-kms/conf/ranger-kms-audit-changes.cfg 5a51455 > plugin-kms/scripts/enable-kms-plugin.sh 7bf6c62 > plugin-solr/conf/ranger-solr-audit-changes.cfg 2742bc1 > plugin-solr/scripts/install.properties a3d9887 > plugin-yarn/conf/ranger-yarn-audit-changes.cfg b650be1 > plugin-yarn/scripts/install.properties 3780068 > security-admin/scripts/db_setup.py 3d20fcd > security-admin/scripts/dba_script.py 0ebd90b > security-admin/scripts/install.properties 1d9d207 > security-admin/scripts/setup.sh bf29ed6 > security-admin/src/main/resources/conf.dist/ranger-admin-default-site.xml > 3333827 > security-admin/src/main/resources/conf.dist/ranger-admin-site.xml 6ee48f4 > storm-agent/conf/ranger-storm-audit-changes.cfg b650be1 > storm-agent/scripts/install.properties f2aa5c4 > > Diff: https://reviews.apache.org/r/45091/diff/ > > > Testing > ------- > > **Steps Performaed (With patch) :** > > **Use-case 1:** > Fresh Ranger Admin Installation : > > Steps: > 1. After Ranger installation did not find any audit to DB related properties > in install.properties file so provided 'solr' as audit data store and > configured solr URL in solr_url property. > 2. Executed setup.sh to install Ranger > > Expected Behaviour : > 1. Installation script should complete successfully and after starting > Ranger, Ranger UI should work; user should able to create services, policies, > users and groups. > > Actual Behaviour : > 1. In installation log it was observed that installation process skipped > creation of audit DB, audit user and execution of audit db related sql > patches. > 2. Ranger installation was finished successfully. > 3. After starting Ranger; was able to login to Ranger and Ranger UI was > working fine. Was able to create services, policies, users and groups. > > **Use-case 2:** > Enabling Ranger plugin and writing audit logs To solr : > > Steps: > 1. Enabled HDFS plugin with solr and provided solr url so that hdfs component > should write audit logs in solr. > 2. Excecuted HDFS command to READ/CREATE resources of hdfs for which Ranger > policies were created. > > Expected Behaviour : > HDFS plugin should write audit logs to provided solr url and same logs should > appear in Ranger admin UI -> Audit menu -> Access tab. > > Actual Behaviour : > Expected logs were available in Ranger Admin UI -> Audit menu -> Access tab. > > > **Use-case 3:** > Ranger admin and Ranger plugins upgrade: > > Steps: > 1. Installed Ranger admin without patch and started Ranger admin with audit > source as DB; enabled HDFS plugin with Audit logs to all three audit > destination DB, HDFS and solr. > 2. Created HDFS service and policies; assigned policies to users with > different combination of access permissions. > 3. From console window excecuted HDFS command to READ/CREATE HDFS resources > on which Ranger policies was created. > 4. It was observed that HDFS plugin was writing audit logs to all three audit > stores. > 5. Expected logs were available in Ranger admin UI -> Audit menu -> Access > tab. > 6. Stopped Ranger Admin, disabled plugins to stop communicating to Ranger > Admin, Stopped HDFS component. > 7. Copied Ranger admin install.properties and installed Ranger with patch and > used same properties of previous installation, since new install.properties > did not have audit to DB related properties so skipped that and provided solr > url which was used in solr related config of HDFS plugin. > 7. Executed Ranger setup script and restarted Ranger admin. > 8. Now Ranger UI was reading audit logs from solr source and expected logs > were available in Ranger Admin UI -> Audit menu -> Access tab. > 9. Enabled HDFS plugin(patched version) with audit destination as HDFS and > solr. > 10. Excecuted HDFS command to READ/CREATE resources on which Ranger policies > exist. > > Expected Behaviour : > 1. HDFS plugin should write new logs to provided solr url and same logs > should appear in Ranger admin UI -> Audit menu -> Access tab. > 2. Expected logs should appear in Solr UI also. > 3. HDFS plugin must not write any new audit logs to DB. > > > Actual Behaviour : > 1. Expected logs were available in Ranger admin UI -> Audit menu -> Access > tab. > 2. Expected logs were available in solr UI. > 3. There was no new logs in 'xa_access_audit' table of Ranger audit DB. > > > Thanks, > > Pradeep Agrawal > >
