> On March 29, 2016, 12:33 p.m., Jonathan Hurley wrote: > > ambari-server/src/main/java/org/apache/ambari/server/actionmanager/ActionDBAccessorImpl.java, > > lines 134-139 > > <https://reviews.apache.org/r/44265/diff/4/?file=1313493#file1313493line134> > > > > I don't see these being cleared anywhere. > > Daniel Gergely wrote: > These are not cleared, because there is no expression to decide when to > clear. The problem is that agents sends the completed status multiple times. > So if I deleted a completed task from the cache, the agent would send it > again. If the task is not in the cache, it is logged, even it is in completed > state. (consider the case when ambari server is stopped during a task > execution, task finishes and server is started. That point the completed > status is needed to be logged, since this is a new information) > > Sebastian Toader wrote: > HostRoleStatus should be cached by host/component in > temporaryTaskStatusCache. When a host is deleted then all host role commands > associated with the deleted host are deleted as well. Upon the deletion of > the host role command the appropriate entries from these caches should be > removed.
You have a good point; the maps are long->enum, which should not really cause too much memory overhead. It would be nice though to clean them up using some mechanism. We could hook into the events fired for things like host removal. We could also use an expiring cache as well. I'll drop this issue for now and file a Jira for this. - Jonathan ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/44265/#review125892 ----------------------------------------------------------- On March 30, 2016, 11:20 a.m., Daniel Gergely wrote: > > ----------------------------------------------------------- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/44265/ > ----------------------------------------------------------- > > (Updated March 30, 2016, 11:20 a.m.) > > > Review request for Ambari, Laszlo Puskas, Oliver Szabo, Robert Levas, Sandor > Magyari, and Sebastian Toader. > > > Bugs: AMBARI-15241 > https://issues.apache.org/jira/browse/AMBARI-15241 > > > Repository: ambari > > > Description > ------- > > Entry points for logging events: > - Login and logout: AmbariAuthenticationFilter, AmbariAuthorizationFilter, > LogoutService > - Operation and tasks: ActionDBAAccessorImpl > - Kerberos related events: CreateKeytabFilesServerAction, > CreatePrincipalServerAction, DestroyPrincipalsServerAction, > FinalizeKerberosServerAction > - REST api: BaseService > > Architecture: > AuditLogger injectable is created and wired in by Guice. The default > implementation (AuditLoggerDefaultImpl) for this interface logs to a file. > Considering performance impact, there is a buffered audit logger > implementation (BufferedAuditLogger) that is a wrapper for an audit logger > implementation and does the logging asynchronously. > > Audit loggers have a single log method that accepts an AuditEvent object. At > the entry points an AuditEvent is assembled and log method is called (except > for the REST api, see below...) > > REST Api: > In BaseService, there is a RequestAuditLogger, that is responsible for > handling and assembling AuditEvents. It also has a log method, but the > parameters for that is the Request and Result objects. > RequestAuditLogger is a small framework for that plugins can be implemented > to handle different type of requests. > Plugins implements RequestAuditEventCreator interface and can be set to > handle one or multiple request types (PUT, POST, DELETE, etc...), resource > types (HOST_COMPONENT, BLUEPRINT, etc..) and results statuses (200 OK, 202 > ACCEPTED, 404 NOT_FOUND, etc). If null is returned by these getters, it means > "all". If there are more than one RequestAuditEventCreators can handle a > request, then the most specific is called. (Priority order: resourceType > > resultStatus > requestType) > For example it is possible to define a plugin for all the POST requests, but > if there is an other plugin for POST request and for resource type > HOST_COMPONENT, then this latter is used (because it is more specific). The > method createAuditEvent is responsible for creating the AuditEvent object for > the RequestAuditEventLogger. If null is returned, then request is not logged > (can be used for ignoring events). > Plugins are registered in ControllerModule and wired by guice to the > RequestAuditLogger. If a new plugin is needed, then implementing the > RequestAuditEventCreator interface and registering it in Guice is the only > things to do. (open-closed principle) > > > Diffs > ----- > > ambari-project/pom.xml b3d9ca2 > ambari-server/conf/unix/log4j.properties ab3ea0b > ambari-server/conf/windows/log4j.properties cc40f74 > ambari-server/pom.xml 07a315d > ambari-server/src/main/conf/log4j.properties 8e6652d > > ambari-server/src/main/java/org/apache/ambari/server/actionmanager/ActionDBAccessorImpl.java > a75a000 > > ambari-server/src/main/java/org/apache/ambari/server/api/services/BaseRequest.java > a33e8a7 > > ambari-server/src/main/java/org/apache/ambari/server/api/services/BaseService.java > 7945599 > > ambari-server/src/main/java/org/apache/ambari/server/api/services/LogoutService.java > df8faaa > > ambari-server/src/main/java/org/apache/ambari/server/api/services/Request.java > ff9b6c7 > > ambari-server/src/main/java/org/apache/ambari/server/audit/AsyncAuditLogger.java > PRE-CREATION > ambari-server/src/main/java/org/apache/ambari/server/audit/AuditLogger.java > PRE-CREATION > > ambari-server/src/main/java/org/apache/ambari/server/audit/AuditLoggerDefaultImpl.java > PRE-CREATION > > ambari-server/src/main/java/org/apache/ambari/server/audit/AuditLoggerModule.java > PRE-CREATION > > ambari-server/src/main/java/org/apache/ambari/server/audit/event/AbstractAuditEvent.java > PRE-CREATION > > ambari-server/src/main/java/org/apache/ambari/server/audit/event/AbstractUserAuditEvent.java > PRE-CREATION > > ambari-server/src/main/java/org/apache/ambari/server/audit/event/AccessUnauthorizedAuditEvent.java > PRE-CREATION > > ambari-server/src/main/java/org/apache/ambari/server/audit/event/AuditEvent.java > PRE-CREATION > > ambari-server/src/main/java/org/apache/ambari/server/audit/event/LoginAuditEvent.java > PRE-CREATION > > ambari-server/src/main/java/org/apache/ambari/server/audit/event/LogoutAuditEvent.java > PRE-CREATION > > ambari-server/src/main/java/org/apache/ambari/server/audit/event/OperationStatusAuditEvent.java > PRE-CREATION > > ambari-server/src/main/java/org/apache/ambari/server/audit/event/TaskStatusAuditEvent.java > PRE-CREATION > > ambari-server/src/main/java/org/apache/ambari/server/audit/event/kerberos/AbstractKerberosAuditEvent.java > PRE-CREATION > > ambari-server/src/main/java/org/apache/ambari/server/audit/event/kerberos/ChangeSecurityStateKerberosAuditEvent.java > PRE-CREATION > > ambari-server/src/main/java/org/apache/ambari/server/audit/event/kerberos/CreateKeyTabKerberosAuditEvent.java > PRE-CREATION > > ambari-server/src/main/java/org/apache/ambari/server/audit/event/kerberos/CreatePrincipalKerberosAuditEvent.java > PRE-CREATION > > ambari-server/src/main/java/org/apache/ambari/server/audit/event/kerberos/DestroyPrincipalKerberosAuditEvent.java > PRE-CREATION > > ambari-server/src/main/java/org/apache/ambari/server/audit/event/request/ActivateUserRequestAuditEvent.java > PRE-CREATION > > ambari-server/src/main/java/org/apache/ambari/server/audit/event/request/AddAlertGroupRequestAuditEvent.java > PRE-CREATION > > ambari-server/src/main/java/org/apache/ambari/server/audit/event/request/AddAlertTargetRequestAuditEvent.java > PRE-CREATION > > ambari-server/src/main/java/org/apache/ambari/server/audit/event/request/AddBlueprintRequestAuditEvent.java > PRE-CREATION > > ambari-server/src/main/java/org/apache/ambari/server/audit/event/request/AddComponentToHostRequestAuditEvent.java > PRE-CREATION > > ambari-server/src/main/java/org/apache/ambari/server/audit/event/request/AddCredentialRequestAuditEvent.java > PRE-CREATION > > ambari-server/src/main/java/org/apache/ambari/server/audit/event/request/AddHostRequestAuditEvent.java > PRE-CREATION > > ambari-server/src/main/java/org/apache/ambari/server/audit/event/request/AddRepositoryRequestAuditEvent.java > PRE-CREATION > > ambari-server/src/main/java/org/apache/ambari/server/audit/event/request/AddRepositoryVersionRequestAuditEvent.java > PRE-CREATION > > ambari-server/src/main/java/org/apache/ambari/server/audit/event/request/AddRequestRequestAuditEvent.java > PRE-CREATION > > ambari-server/src/main/java/org/apache/ambari/server/audit/event/request/AddUpgradeRequestAuditEvent.java > PRE-CREATION > > ambari-server/src/main/java/org/apache/ambari/server/audit/event/request/AddUserToGroupRequestAuditEvent.java > PRE-CREATION > > ambari-server/src/main/java/org/apache/ambari/server/audit/event/request/AddViewInstanceRequestAuditEvent.java > PRE-CREATION > > ambari-server/src/main/java/org/apache/ambari/server/audit/event/request/AdminUserRequestAuditEvent.java > PRE-CREATION > > ambari-server/src/main/java/org/apache/ambari/server/audit/event/request/BlueprintExportRequestAuditEvent.java > PRE-CREATION > > ambari-server/src/main/java/org/apache/ambari/server/audit/event/request/ChangeAlertGroupRequestAuditEvent.java > PRE-CREATION > > ambari-server/src/main/java/org/apache/ambari/server/audit/event/request/ChangeAlertTargetRequestAuditEvent.java > PRE-CREATION > > ambari-server/src/main/java/org/apache/ambari/server/audit/event/request/ChangeRepositoryVersionRequestAuditEvent.java > PRE-CREATION > > ambari-server/src/main/java/org/apache/ambari/server/audit/event/request/ChangeViewInstanceRequestAuditEvent.java > PRE-CREATION > > ambari-server/src/main/java/org/apache/ambari/server/audit/event/request/ClientConfigDownloadRequestAuditEvent.java > PRE-CREATION > > ambari-server/src/main/java/org/apache/ambari/server/audit/event/request/ClusterNameChangeRequestAuditEvent.java > PRE-CREATION > > ambari-server/src/main/java/org/apache/ambari/server/audit/event/request/ClusterPrivilegeChangeRequestAuditEvent.java > PRE-CREATION > > ambari-server/src/main/java/org/apache/ambari/server/audit/event/request/ConfigurationChangeRequestAuditEvent.java > PRE-CREATION > > ambari-server/src/main/java/org/apache/ambari/server/audit/event/request/CreateGroupRequestAuditEvent.java > PRE-CREATION > > ambari-server/src/main/java/org/apache/ambari/server/audit/event/request/CreateUserRequestAuditEvent.java > PRE-CREATION > > ambari-server/src/main/java/org/apache/ambari/server/audit/event/request/DeleteAlertGroupRequestAuditEvent.java > PRE-CREATION > > ambari-server/src/main/java/org/apache/ambari/server/audit/event/request/DeleteAlertTargetRequestAuditEvent.java > PRE-CREATION > > ambari-server/src/main/java/org/apache/ambari/server/audit/event/request/DeleteBlueprintRequestAuditEvent.java > PRE-CREATION > > ambari-server/src/main/java/org/apache/ambari/server/audit/event/request/DeleteGroupRequestAuditEvent.java > PRE-CREATION > > ambari-server/src/main/java/org/apache/ambari/server/audit/event/request/DeleteHostRequestAuditEvent.java > PRE-CREATION > > ambari-server/src/main/java/org/apache/ambari/server/audit/event/request/DeleteRepositoryVersionRequestAuditEvent.java > PRE-CREATION > > ambari-server/src/main/java/org/apache/ambari/server/audit/event/request/DeleteServiceRequestAuditEvent.java > PRE-CREATION > > ambari-server/src/main/java/org/apache/ambari/server/audit/event/request/DeleteUserRequestAuditEvent.java > PRE-CREATION > > ambari-server/src/main/java/org/apache/ambari/server/audit/event/request/DeleteViewInstanceRequestAuditEvent.java > PRE-CREATION > > ambari-server/src/main/java/org/apache/ambari/server/audit/event/request/MembershipChangeRequestAuditEvent.java > PRE-CREATION > > ambari-server/src/main/java/org/apache/ambari/server/audit/event/request/PrivilegeChangeRequestAuditEvent.java > PRE-CREATION > > ambari-server/src/main/java/org/apache/ambari/server/audit/event/request/RemoveUserFromGroupRequestAuditEvent.java > PRE-CREATION > > ambari-server/src/main/java/org/apache/ambari/server/audit/event/request/StartOperationRequestAuditEvent.java > PRE-CREATION > > ambari-server/src/main/java/org/apache/ambari/server/audit/event/request/UpdateRepositoryRequestAuditEvent.java > PRE-CREATION > > ambari-server/src/main/java/org/apache/ambari/server/audit/event/request/UpdateUpgradeItemRequestAuditEvent.java > PRE-CREATION > > ambari-server/src/main/java/org/apache/ambari/server/audit/event/request/UserPasswordChangeRequestAuditEvent.java > PRE-CREATION > > ambari-server/src/main/java/org/apache/ambari/server/audit/event/request/ViewPrivilegeChangeRequestAuditEvent.java > PRE-CREATION > > ambari-server/src/main/java/org/apache/ambari/server/audit/request/RequestAuditEvent.java > PRE-CREATION > > ambari-server/src/main/java/org/apache/ambari/server/audit/request/RequestAuditEventCreator.java > PRE-CREATION > > ambari-server/src/main/java/org/apache/ambari/server/audit/request/RequestAuditLogger.java > PRE-CREATION > > ambari-server/src/main/java/org/apache/ambari/server/audit/request/RequestAuditLoggerImpl.java > PRE-CREATION > > ambari-server/src/main/java/org/apache/ambari/server/audit/request/eventcreator/AlertGroupEventCreator.java > PRE-CREATION > > ambari-server/src/main/java/org/apache/ambari/server/audit/request/eventcreator/AlertTargetEventCreator.java > PRE-CREATION > > ambari-server/src/main/java/org/apache/ambari/server/audit/request/eventcreator/BlueprintEventCreator.java > PRE-CREATION > > ambari-server/src/main/java/org/apache/ambari/server/audit/request/eventcreator/BlueprintExportEventCreator.java > PRE-CREATION > > ambari-server/src/main/java/org/apache/ambari/server/audit/request/eventcreator/ComponentEventCreator.java > PRE-CREATION > > ambari-server/src/main/java/org/apache/ambari/server/audit/request/eventcreator/ConfigurationChangeEventCreator.java > PRE-CREATION > > ambari-server/src/main/java/org/apache/ambari/server/audit/request/eventcreator/CredentialEventCreator.java > PRE-CREATION > > ambari-server/src/main/java/org/apache/ambari/server/audit/request/eventcreator/DefaultEventCreator.java > PRE-CREATION > > ambari-server/src/main/java/org/apache/ambari/server/audit/request/eventcreator/GroupEventCreator.java > PRE-CREATION > > ambari-server/src/main/java/org/apache/ambari/server/audit/request/eventcreator/HostEventCreator.java > PRE-CREATION > > ambari-server/src/main/java/org/apache/ambari/server/audit/request/eventcreator/MemberEventCreator.java > PRE-CREATION > > ambari-server/src/main/java/org/apache/ambari/server/audit/request/eventcreator/PrivilegeEventCreator.java > PRE-CREATION > > ambari-server/src/main/java/org/apache/ambari/server/audit/request/eventcreator/RecommendationIgnoreEventCreator.java > PRE-CREATION > > ambari-server/src/main/java/org/apache/ambari/server/audit/request/eventcreator/RepositoryEventCreator.java > PRE-CREATION > > ambari-server/src/main/java/org/apache/ambari/server/audit/request/eventcreator/RepositoryVersionEventCreator.java > PRE-CREATION > > ambari-server/src/main/java/org/apache/ambari/server/audit/request/eventcreator/RequestEventCreator.java > PRE-CREATION > > ambari-server/src/main/java/org/apache/ambari/server/audit/request/eventcreator/ServiceConfigDownloadEventCreator.java > PRE-CREATION > > ambari-server/src/main/java/org/apache/ambari/server/audit/request/eventcreator/ServiceEventCreator.java > PRE-CREATION > > ambari-server/src/main/java/org/apache/ambari/server/audit/request/eventcreator/UnauthorizedEventCreator.java > PRE-CREATION > > ambari-server/src/main/java/org/apache/ambari/server/audit/request/eventcreator/UpgradeEventCreator.java > PRE-CREATION > > ambari-server/src/main/java/org/apache/ambari/server/audit/request/eventcreator/UpgradeItemEventCreator.java > PRE-CREATION > > ambari-server/src/main/java/org/apache/ambari/server/audit/request/eventcreator/UserEventCreator.java > PRE-CREATION > > ambari-server/src/main/java/org/apache/ambari/server/audit/request/eventcreator/ValidationIgnoreEventCreator.java > PRE-CREATION > > ambari-server/src/main/java/org/apache/ambari/server/audit/request/eventcreator/ViewInstanceEventCreator.java > PRE-CREATION > > ambari-server/src/main/java/org/apache/ambari/server/audit/request/eventcreator/ViewPrivilegeEventCreator.java > PRE-CREATION > > ambari-server/src/main/java/org/apache/ambari/server/configuration/Configuration.java > 1d30f1c > > ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariServer.java > 4b769e3 > > ambari-server/src/main/java/org/apache/ambari/server/controller/ControllerModule.java > daca64d > > ambari-server/src/main/java/org/apache/ambari/server/controller/internal/CalculatedStatus.java > 985e9ad > > ambari-server/src/main/java/org/apache/ambari/server/security/authentication/AmbariAuthenticationFilter.java > PRE-CREATION > > ambari-server/src/main/java/org/apache/ambari/server/security/authorization/AmbariAuthorizationFilter.java > 4be804d > > ambari-server/src/main/java/org/apache/ambari/server/security/authorization/AuthorizationHelper.java > 15ef8d0 > > ambari-server/src/main/java/org/apache/ambari/server/security/authorization/PermissionHelper.java > PRE-CREATION > > ambari-server/src/main/java/org/apache/ambari/server/serveraction/AbstractServerAction.java > 33191bf > > ambari-server/src/main/java/org/apache/ambari/server/serveraction/kerberos/CreateKeytabFilesServerAction.java > 8aa816d > > ambari-server/src/main/java/org/apache/ambari/server/serveraction/kerberos/CreatePrincipalsServerAction.java > 8009ae1 > > ambari-server/src/main/java/org/apache/ambari/server/serveraction/kerberos/DestroyPrincipalsServerAction.java > 93daae8 > > ambari-server/src/main/java/org/apache/ambari/server/serveraction/kerberos/FinalizeKerberosServerAction.java > c710b8e > > ambari-server/src/main/java/org/apache/ambari/server/utils/RequestUtils.java > PRE-CREATION > ambari-server/src/main/resources/webapp/WEB-INF/spring-security.xml 8b44b94 > > ambari-server/src/test/java/org/apache/ambari/server/actionmanager/TestActionDBAccessorImpl.java > f88cf8e > > ambari-server/src/test/java/org/apache/ambari/server/actionmanager/TestActionManager.java > d7d08b1 > > ambari-server/src/test/java/org/apache/ambari/server/agent/HeartbeatProcessorTest.java > 7f3d763 > > ambari-server/src/test/java/org/apache/ambari/server/api/services/BaseServiceTest.java > 19eeffb > > ambari-server/src/test/java/org/apache/ambari/server/audit/AccessUnauthorizedAuditEventTest.java > PRE-CREATION > > ambari-server/src/test/java/org/apache/ambari/server/audit/LoginAuditEventTest.java > PRE-CREATION > > ambari-server/src/test/java/org/apache/ambari/server/audit/LogoutAuditEventTest.java > PRE-CREATION > > ambari-server/src/test/java/org/apache/ambari/server/audit/OperationStatusAuditEventTest.java > PRE-CREATION > > ambari-server/src/test/java/org/apache/ambari/server/audit/StartOperationRequestAuditEventTest.java > PRE-CREATION > > ambari-server/src/test/java/org/apache/ambari/server/audit/request/AbstractBaseCreator.java > PRE-CREATION > > ambari-server/src/test/java/org/apache/ambari/server/audit/request/AllGetCreator.java > PRE-CREATION > > ambari-server/src/test/java/org/apache/ambari/server/audit/request/AllPostAndPutCreator.java > PRE-CREATION > > ambari-server/src/test/java/org/apache/ambari/server/audit/request/DefaultEventCreatorTest.java > PRE-CREATION > > ambari-server/src/test/java/org/apache/ambari/server/audit/request/PutHostComponentCreator.java > PRE-CREATION > > ambari-server/src/test/java/org/apache/ambari/server/audit/request/RequestAuditLogModule.java > PRE-CREATION > > ambari-server/src/test/java/org/apache/ambari/server/audit/request/RequestAuditLoggerTest.java > PRE-CREATION > > ambari-server/src/test/java/org/apache/ambari/server/checks/UpgradeCheckOrderTest.java > d4ff566 > > ambari-server/src/test/java/org/apache/ambari/server/controller/AmbariManagementControllerTest.java > ca062c0 > > ambari-server/src/test/java/org/apache/ambari/server/controller/KerberosHelperTest.java > f6027f3 > > ambari-server/src/test/java/org/apache/ambari/server/notifications/DispatchFactoryTest.java > 6834d5c > > ambari-server/src/test/java/org/apache/ambari/server/orm/InMemoryDefaultTestModule.java > 3ecfe14 > > ambari-server/src/test/java/org/apache/ambari/server/orm/JdbcPropertyTest.java > c07382b > > ambari-server/src/test/java/org/apache/ambari/server/security/authentication/AmbariAuthenticationFilterTest.java > PRE-CREATION > > ambari-server/src/test/java/org/apache/ambari/server/security/authorization/AmbariAuthorizationFilterTest.java > 9db3904 > > ambari-server/src/test/java/org/apache/ambari/server/security/authorization/AmbariLdapAuthenticationProviderForDNWithSpaceTest.java > da8d9bc > > ambari-server/src/test/java/org/apache/ambari/server/security/authorization/AmbariLdapAuthenticationProviderTest.java > b26494c > > ambari-server/src/test/java/org/apache/ambari/server/security/authorization/AmbariLocalUserDetailsServiceTest.java > c410f5b > > ambari-server/src/test/java/org/apache/ambari/server/security/authorization/LdapServerPropertiesTest.java > bfb7a90 > > ambari-server/src/test/java/org/apache/ambari/server/serveraction/kerberos/KerberosServerActionTest.java > ce97f25 > > ambari-server/src/test/java/org/apache/ambari/server/serveraction/kerberos/UpdateKerberosConfigsServerActionTest.java > d6f9efe > > ambari-server/src/test/java/org/apache/ambari/server/utils/RequestUtilsTest.java > PRE-CREATION > > Diff: https://reviews.apache.org/r/44265/diff/ > > > Testing > ------- > > Unit tests are added to cover functionality. > > All tests passed on local machine. > > > Thanks, > > Daniel Gergely > >
