Re: Review Request 52917: Improve Audit Log processing by Logfeeder
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/52917/ --- (Updated Nov. 7, 2016, 9:23 a.m.) Review request for Ambari, Oliver Szabo, Robert Nettleton, and Sumit Mohanty. Changes --- fix branches Bugs: AMBARI-18606 https://issues.apache.org/jira/browse/AMBARI-18606 Repository: ambari Description --- Logfeeder should be able to handle dynamic audit fields. Ambari Audit log has some information which should be indexed, searchable, and present in the log table on the UI. Also fixed: - If filter key value is based on parentheses then inner parentheses should be handled appropriately - A(B(C)) --> A = B(C) - New Mapper introduced to copy a value, if it needs to be loaded into more than one field ( both the original value and a derived ) - dynamic field definition was added to the audit_logs collection - added some ambari-audit log and configuration to the docker test environment - fixed a typo in the falcon configuration - improved code quality by some minor changes Diffs - ambari-logsearch/ambari-logsearch-logfeeder/src/main/java/org/apache/ambari/logfeeder/filter/FilterKeyValue.java 5bb15ff ambari-logsearch/ambari-logsearch-logfeeder/src/main/java/org/apache/ambari/logfeeder/mapper/MapperFieldCopy.java PRE-CREATION ambari-logsearch/ambari-logsearch-logfeeder/src/main/resources/alias_config.json b15db63 ambari-logsearch/ambari-logsearch-logfeeder/src/test/java/org/apache/ambari/logfeeder/mapper/MapperFieldCopyTest.java PRE-CREATION ambari-logsearch/ambari-logsearch-portal/src/main/configsets/audit_logs/conf/managed-schema f77bec0 ambari-logsearch/ambari-logsearch-portal/src/main/java/org/apache/ambari/logsearch/common/LogSearchConstants.java b13768f ambari-logsearch/ambari-logsearch-portal/src/main/java/org/apache/ambari/logsearch/dao/AuditSolrDao.java 959d6f4 ambari-logsearch/ambari-logsearch-portal/src/main/java/org/apache/ambari/logsearch/dao/SolrCollectionDao.java 1cbac31 ambari-logsearch/ambari-logsearch-portal/src/main/java/org/apache/ambari/logsearch/dao/SolrSchemaFieldDao.java f16dc41 ambari-logsearch/ambari-logsearch-portal/src/main/java/org/apache/ambari/logsearch/dao/UserDao.java 912442d ambari-logsearch/ambari-logsearch-portal/src/main/java/org/apache/ambari/logsearch/graph/GraphDataGenerator.java 85b5fda ambari-logsearch/ambari-logsearch-portal/src/main/java/org/apache/ambari/logsearch/manager/ManagerBase.java 45d66cf ambari-logsearch/ambari-logsearch-portal/src/main/java/org/apache/ambari/logsearch/manager/ServiceLogsManager.java d276769 ambari-logsearch/ambari-logsearch-portal/src/main/java/org/apache/ambari/logsearch/manager/UserConfigManager.java 00ae332 ambari-logsearch/ambari-logsearch-portal/src/main/java/org/apache/ambari/logsearch/model/response/CommonLogData.java e2bba3f ambari-logsearch/ambari-logsearch-portal/src/main/java/org/apache/ambari/logsearch/solr/model/SolrAuditLogData.java 0fd65a3 ambari-logsearch/ambari-logsearch-portal/src/main/java/org/apache/ambari/logsearch/solr/model/SolrCommonLogData.java bac5080 ambari-logsearch/ambari-logsearch-portal/src/main/java/org/apache/ambari/logsearch/solr/model/SolrServiceLogData.java 1d4cecf ambari-logsearch/ambari-logsearch-portal/src/main/java/org/apache/ambari/logsearch/util/SolrUtil.java 6369107 ambari-logsearch/ambari-logsearch-portal/src/main/java/org/apache/ambari/logsearch/web/filters/LogsearchKRBAuthenticationFilter.java 29fd5b2 ambari-logsearch/ambari-logsearch-portal/src/main/java/org/apache/ambari/logsearch/web/security/LdapProperties.java 2a1b4ee ambari-logsearch/ambari-logsearch-portal/src/main/webapp/scripts/views/audit/AuditTabLayoutView.js 4e09e88 ambari-logsearch/ambari-logsearch-portal/src/main/webapp/static/schema_fields.json 7b1a9a9 ambari-logsearch/docker/test-config/logfeeder/logfeeder.properties f930ec9 ambari-logsearch/docker/test-config/logfeeder/shipper-conf/input.config-ambari.json PRE-CREATION ambari-logsearch/docker/test-logs/ambari-server/ambari-audit.log PRE-CREATION ambari-server/src/main/resources/common-services/LOGSEARCH/0.5.0/package/templates/input.config-ambari.json.j2 e2646ff ambari-server/src/main/resources/common-services/LOGSEARCH/0.5.0/package/templates/input.config-falcon.json.j2 3fea3d3 Diff: https://reviews.apache.org/r/52917/diff/ Testing --- Done. Thanks, Miklos Gergely
Re: Review Request 52917: Improve Audit Log processing by Logfeeder
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/52917/#review152885 --- Ship it! Ship It! - Robert Nettleton On Oct. 15, 2016, 11:48 p.m., Miklos Gergely wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/52917/ > --- > > (Updated Oct. 15, 2016, 11:48 p.m.) > > > Review request for Ambari, Oliver Szabo, Robert Nettleton, and Sumit Mohanty. > > > Bugs: AMBARI-18606 > https://issues.apache.org/jira/browse/AMBARI-18606 > > > Repository: ambari > > > Description > --- > > Logfeeder should be able to handle dynamic audit fields. > Ambari Audit log has some information which should be indexed, searchable, > and present in the log table on the UI. > > Also fixed: > - If filter key value is based on parentheses then inner parentheses should > be handled appropriately - A(B(C)) --> A = B(C) > - New Mapper introduced to copy a value, if it needs to be loaded into more > than one field ( both the original value and a derived ) > - dynamic field definition was added to the audit_logs collection > - added some ambari-audit log and configuration to the docker test environment > - fixed a typo in the falcon configuration > - improved code quality by some minor changes > > > Diffs > - > > > ambari-logsearch/ambari-logsearch-logfeeder/src/main/java/org/apache/ambari/logfeeder/filter/FilterKeyValue.java > 5bb15ff > > ambari-logsearch/ambari-logsearch-logfeeder/src/main/java/org/apache/ambari/logfeeder/mapper/MapperFieldCopy.java > PRE-CREATION > > ambari-logsearch/ambari-logsearch-logfeeder/src/main/resources/alias_config.json > b15db63 > > ambari-logsearch/ambari-logsearch-logfeeder/src/test/java/org/apache/ambari/logfeeder/mapper/MapperFieldCopyTest.java > PRE-CREATION > > ambari-logsearch/ambari-logsearch-portal/src/main/configsets/audit_logs/conf/managed-schema > f77bec0 > > ambari-logsearch/ambari-logsearch-portal/src/main/java/org/apache/ambari/logsearch/common/LogSearchConstants.java > b13768f > > ambari-logsearch/ambari-logsearch-portal/src/main/java/org/apache/ambari/logsearch/dao/AuditSolrDao.java > 959d6f4 > > ambari-logsearch/ambari-logsearch-portal/src/main/java/org/apache/ambari/logsearch/dao/SolrCollectionDao.java > 1cbac31 > > ambari-logsearch/ambari-logsearch-portal/src/main/java/org/apache/ambari/logsearch/dao/SolrSchemaFieldDao.java > f16dc41 > > ambari-logsearch/ambari-logsearch-portal/src/main/java/org/apache/ambari/logsearch/dao/UserDao.java > 912442d > > ambari-logsearch/ambari-logsearch-portal/src/main/java/org/apache/ambari/logsearch/graph/GraphDataGenerator.java > 85b5fda > > ambari-logsearch/ambari-logsearch-portal/src/main/java/org/apache/ambari/logsearch/manager/ManagerBase.java > 45d66cf > > ambari-logsearch/ambari-logsearch-portal/src/main/java/org/apache/ambari/logsearch/manager/ServiceLogsManager.java > d276769 > > ambari-logsearch/ambari-logsearch-portal/src/main/java/org/apache/ambari/logsearch/manager/UserConfigManager.java > 00ae332 > > ambari-logsearch/ambari-logsearch-portal/src/main/java/org/apache/ambari/logsearch/model/response/CommonLogData.java > e2bba3f > > ambari-logsearch/ambari-logsearch-portal/src/main/java/org/apache/ambari/logsearch/solr/model/SolrAuditLogData.java > 0fd65a3 > > ambari-logsearch/ambari-logsearch-portal/src/main/java/org/apache/ambari/logsearch/solr/model/SolrCommonLogData.java > bac5080 > > ambari-logsearch/ambari-logsearch-portal/src/main/java/org/apache/ambari/logsearch/solr/model/SolrServiceLogData.java > 1d4cecf > > ambari-logsearch/ambari-logsearch-portal/src/main/java/org/apache/ambari/logsearch/util/SolrUtil.java > 6369107 > > ambari-logsearch/ambari-logsearch-portal/src/main/java/org/apache/ambari/logsearch/web/filters/LogsearchKRBAuthenticationFilter.java > 29fd5b2 > > ambari-logsearch/ambari-logsearch-portal/src/main/java/org/apache/ambari/logsearch/web/security/LdapProperties.java > 2a1b4ee > > ambari-logsearch/ambari-logsearch-portal/src/main/webapp/scripts/views/audit/AuditTabLayoutView.js > 4e09e88 > > ambari-logsearch/ambari-logsearch-portal/src/main/webapp/static/schema_fields.json > 7b1a9a9 > ambari-logsearch/docker/test-config/logfeeder/logfeeder.properties f930ec9 > > ambari-logsearch/docker/test-config/logfeeder/shipper-conf/input.config-ambari.json > PRE-CREATION > ambari-logsearch/docker/test-logs/ambari-server/ambari-audit.log > PRE-CREATION > > ambari-server/src/main/resources/common-services/LOGSEARCH/0.5.0/package/templates/input.config-ambari.json.j2 > e2646ff > > ambari-server/src/main/resources/common-services/LOGSEARCH/0.5.0/package/tem
Re: Review Request 52917: Improve Audit Log processing by Logfeeder
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/52917/#review152807 --- Ship it! Ship It! - Oliver Szabo On Oct. 15, 2016, 11:48 p.m., Miklos Gergely wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/52917/ > --- > > (Updated Oct. 15, 2016, 11:48 p.m.) > > > Review request for Ambari, Oliver Szabo, Robert Nettleton, and Sumit Mohanty. > > > Bugs: AMBARI-18606 > https://issues.apache.org/jira/browse/AMBARI-18606 > > > Repository: ambari > > > Description > --- > > Logfeeder should be able to handle dynamic audit fields. > Ambari Audit log has some information which should be indexed, searchable, > and present in the log table on the UI. > > Also fixed: > - If filter key value is based on parentheses then inner parentheses should > be handled appropriately - A(B(C)) --> A = B(C) > - New Mapper introduced to copy a value, if it needs to be loaded into more > than one field ( both the original value and a derived ) > - dynamic field definition was added to the audit_logs collection > - added some ambari-audit log and configuration to the docker test environment > - fixed a typo in the falcon configuration > - improved code quality by some minor changes > > > Diffs > - > > > ambari-logsearch/ambari-logsearch-logfeeder/src/main/java/org/apache/ambari/logfeeder/filter/FilterKeyValue.java > 5bb15ff > > ambari-logsearch/ambari-logsearch-logfeeder/src/main/java/org/apache/ambari/logfeeder/mapper/MapperFieldCopy.java > PRE-CREATION > > ambari-logsearch/ambari-logsearch-logfeeder/src/main/resources/alias_config.json > b15db63 > > ambari-logsearch/ambari-logsearch-logfeeder/src/test/java/org/apache/ambari/logfeeder/mapper/MapperFieldCopyTest.java > PRE-CREATION > > ambari-logsearch/ambari-logsearch-portal/src/main/configsets/audit_logs/conf/managed-schema > f77bec0 > > ambari-logsearch/ambari-logsearch-portal/src/main/java/org/apache/ambari/logsearch/common/LogSearchConstants.java > b13768f > > ambari-logsearch/ambari-logsearch-portal/src/main/java/org/apache/ambari/logsearch/dao/AuditSolrDao.java > 959d6f4 > > ambari-logsearch/ambari-logsearch-portal/src/main/java/org/apache/ambari/logsearch/dao/SolrCollectionDao.java > 1cbac31 > > ambari-logsearch/ambari-logsearch-portal/src/main/java/org/apache/ambari/logsearch/dao/SolrSchemaFieldDao.java > f16dc41 > > ambari-logsearch/ambari-logsearch-portal/src/main/java/org/apache/ambari/logsearch/dao/UserDao.java > 912442d > > ambari-logsearch/ambari-logsearch-portal/src/main/java/org/apache/ambari/logsearch/graph/GraphDataGenerator.java > 85b5fda > > ambari-logsearch/ambari-logsearch-portal/src/main/java/org/apache/ambari/logsearch/manager/ManagerBase.java > 45d66cf > > ambari-logsearch/ambari-logsearch-portal/src/main/java/org/apache/ambari/logsearch/manager/ServiceLogsManager.java > d276769 > > ambari-logsearch/ambari-logsearch-portal/src/main/java/org/apache/ambari/logsearch/manager/UserConfigManager.java > 00ae332 > > ambari-logsearch/ambari-logsearch-portal/src/main/java/org/apache/ambari/logsearch/model/response/CommonLogData.java > e2bba3f > > ambari-logsearch/ambari-logsearch-portal/src/main/java/org/apache/ambari/logsearch/solr/model/SolrAuditLogData.java > 0fd65a3 > > ambari-logsearch/ambari-logsearch-portal/src/main/java/org/apache/ambari/logsearch/solr/model/SolrCommonLogData.java > bac5080 > > ambari-logsearch/ambari-logsearch-portal/src/main/java/org/apache/ambari/logsearch/solr/model/SolrServiceLogData.java > 1d4cecf > > ambari-logsearch/ambari-logsearch-portal/src/main/java/org/apache/ambari/logsearch/util/SolrUtil.java > 6369107 > > ambari-logsearch/ambari-logsearch-portal/src/main/java/org/apache/ambari/logsearch/web/filters/LogsearchKRBAuthenticationFilter.java > 29fd5b2 > > ambari-logsearch/ambari-logsearch-portal/src/main/java/org/apache/ambari/logsearch/web/security/LdapProperties.java > 2a1b4ee > > ambari-logsearch/ambari-logsearch-portal/src/main/webapp/scripts/views/audit/AuditTabLayoutView.js > 4e09e88 > > ambari-logsearch/ambari-logsearch-portal/src/main/webapp/static/schema_fields.json > 7b1a9a9 > ambari-logsearch/docker/test-config/logfeeder/logfeeder.properties f930ec9 > > ambari-logsearch/docker/test-config/logfeeder/shipper-conf/input.config-ambari.json > PRE-CREATION > ambari-logsearch/docker/test-logs/ambari-server/ambari-audit.log > PRE-CREATION > > ambari-server/src/main/resources/common-services/LOGSEARCH/0.5.0/package/templates/input.config-ambari.json.j2 > e2646ff > > ambari-server/src/main/resources/common-services/LOGSEARCH/0.5.0/package/templat
