> On April 25, 2024, 11:19 a.m., Madhan Neethiraj wrote: > > Consider adding few samples and test cases, to make it easier to understand > > audit filter functioanlity.
this patch includes a few tests which will help demonstrate the following usages of rules: control audits based on single or multiple criteria/conditions control audits with same criteria for multiple types : CSV of type-names is supported control audits for all types under a hook type : using Regex support for wildcard character '*' control audits using same rule for a type and all its subtypes > On April 25, 2024, 11:19 a.m., Madhan Neethiraj wrote: > > repository/src/main/java/org/apache/atlas/repository/audit/EntityAuditListenerV2.java > > Lines 462 (patched) > > <https://reviews.apache.org/r/74608/diff/13/?file=2286898#file2286898line462> > > > > prepareDataObject() seems expensive - especially given it will be > > executed for each entity operation in Atlas. Are all fields of AtlasEntity > > used in rules? currently only basic metadata fields with primitive data types are being supported. > On April 25, 2024, 11:19 a.m., Madhan Neethiraj wrote: > > repository/src/main/java/org/apache/atlas/rulesengine/AtlasRulesEngine.java > > Lines 83 (patched) > > <https://reviews.apache.org/r/74608/diff/13/?file=2286905#file2286905line83> > > > > This conversion seems expensive. Is this conversion necessary for every > > evalation of a rule? yes, currently we do not have any caching mechanism for rules to store expressions of all rules in memory. Hence, we are currently fetching all the rules from the database everytime just before applying them on the incoming event and using this conversion to execute the pertaining rule evaluation in the flow > On April 25, 2024, 11:19 a.m., Madhan Neethiraj wrote: > > repository/src/main/java/org/apache/atlas/rulesengine/AtlasRulesEngine.java > > Lines 97 (patched) > > <https://reviews.apache.org/r/74608/diff/13/?file=2286905#file2286905line97> > > > > Rule evaluation continues even after ACCEPT result from a rule; is this > > necessary? yes. The evaluation continues for all the rules one after the other or until a matching rule with action DISCARD is encountered. Once a rule matches to DISCARD, no rules will be executed further and the audit event will be discarded. In case if no rules match, the default action is taken. > On April 25, 2024, 11:19 a.m., Madhan Neethiraj wrote: > > webapp/src/main/java/org/apache/atlas/web/resources/AdminResource.java > > Lines 1327 (patched) > > <https://reviews.apache.org/r/74608/diff/13/?file=2286910#file2286910line1327> > > > > Doesn't "/audits/rules/all" conflict with "/audits/rules/{ruleGuid}" > > (line #1269 above)? No, the API paths "/audits/rules/all" and "/audits/rules/{ruleGuid}" do not conflict because they are distinct routes that the framework should be able to differentiate. If a request is sent to "/audits/rules/all", the static route is matched, and the dynamic route is not evaluated. For a request like "/audits/rules/1234", the dynamic route matches because no static route corresponds. I have still made changes to the path definition to avoid any understanding conflicts - Sheetal ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/74608/#review226400 ----------------------------------------------------------- On Dec. 12, 2024, 5:36 p.m., Sheetal Shah wrote: > > ----------------------------------------------------------- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/74608/ > ----------------------------------------------------------- > > (Updated Dec. 12, 2024, 5:36 p.m.) > > > Review request for atlas, Jayendra Parab, Mandar Ambawane, Pinal Shah, Prasad > Pawar, Radhika Kundam, and Sidharth Mishra. > > > Bugs: ATLAS-4797 > https://issues.apache.org/jira/browse/ATLAS-4797 > > > Repository: atlas > > > Description > ------- > > Currently, Atlas has to persist all audit events without any constraints > which can make audit data grow exponentially after some time. > > This feature is specifically to reduce Atlas audit storage where generation > of audit events can be controlled based on the user's use-case/criteria > > Note: Audit data mentioned here is exclusive of Admin Audit Data which > persists in atlas_janus table. Filtering will be applicable to the entity > audit data which persists in ATLAS_ENTITY_AUDIT_EVENTS table. > > Precommit details : > Latest PC build (PASSED) 12-12-2024 : > https://ci-builds.apache.org/job/Atlas/job/PreCommit-ATLAS-Build-Test/1757/ > > > Diffs > ----- > > addons/hbase-bridge/pom.xml 968d8039c > addons/hbase-testing-util/pom.xml 05ed88401 > addons/hive-bridge-shim/pom.xml 08a9ee7c4 > addons/hive-bridge/pom.xml aa32e033f > addons/kafka-bridge/pom.xml a1dd3a666 > addons/models/0000-Area0/0010-base_model.json a4a9248ec > addons/sqoop-bridge/pom.xml 9ddd4efcd > authorization/pom.xml 0e7d75a46 > client/client-v2/src/main/java/org/apache/atlas/AtlasClientV2.java > 6477b4091 > distro/src/conf/atlas-application.properties b5734d7a8 > graphdb/api/pom.xml a01668b64 > intg/pom.xml 0fa19b605 > intg/src/main/java/org/apache/atlas/AtlasConfiguration.java 6c39ee406 > intg/src/main/java/org/apache/atlas/AtlasErrorCode.java 61b8155b9 > intg/src/main/java/org/apache/atlas/model/audit/EntityAuditEventV2.java > 3afd27e7d > intg/src/main/java/org/apache/atlas/model/instance/AtlasRule.java > PRE-CREATION > repository/pom.xml 24dee7e30 > > repository/src/main/java/org/apache/atlas/repository/audit/EntityAuditListenerV2.java > 9b011ab06 > > repository/src/main/java/org/apache/atlas/repository/audit/HBaseBasedAuditRepository.java > e0ecacc6d > repository/src/main/java/org/apache/atlas/repository/ogm/AtlasRuleDTO.java > PRE-CREATION > repository/src/main/java/org/apache/atlas/repository/ogm/DataAccess.java > e63152475 > > repository/src/main/java/org/apache/atlas/rulesengine/AtlasEntityAuditFilterService.java > PRE-CREATION > repository/src/main/java/org/apache/atlas/rulesengine/AtlasRuleUtils.java > PRE-CREATION > repository/src/main/java/org/apache/atlas/rulesengine/AtlasRulesEngine.java > PRE-CREATION > repository/src/main/java/org/apache/atlas/rulesengine/RuleAction.java > PRE-CREATION > server-api/pom.xml 3ecef5df7 > test-tools/src/main/resources/solr/core-template/solrconfig.xml 7a0e8dd16 > webapp/src/main/java/org/apache/atlas/web/filters/ActiveServerFilter.java > e264fa30f > webapp/src/main/java/org/apache/atlas/web/resources/AdminResource.java > 475756981 > webapp/src/test/java/org/apache/atlas/web/filters/CAFTests.java > PRE-CREATION > webapp/src/test/java/org/apache/atlas/web/resources/AdminResourceTest.java > 268fca717 > webapp/src/test/resources/atlas-application.properties b5c707ed1 > > > Diff: https://reviews.apache.org/r/74608/diff/14/ > > > Testing > ------- > > Manual testing done > > > Thanks, > > Sheetal Shah > >
