Fellow developers,
Currently the activation store is tightly coupled with the artifact store utilized by an OpenWhisk deployment. Meaning the controller must use the same data store, CouchDB for example, utilized by an OpenWhisk instance to store and retrieve activation details. However, an invoker can use a customized log store implementation to store user logs and activation records in a data store that is independent of the primary database. Such differences in functionality between the controller and invoker with regards to activation storage and retrieval is problematic. Activation records are special as they hold meaningful details about the execution behavior of actions, triggers, rules, and sequences. Ideally users should be able to run customized queries on these activation records in order to find out the execution time of a group of entities, if a group of entities were executed successfully, etc. By default, CouchDb does not allow such customized queries to be performed. To provide user defined queries on activations, we have already provided a log store that can be customized per-deployment via a Service Provider Interface (SPI). Currently we have log store implementations for Elasticsearch and Splunk. Both of these services allow user queries. Unfortunately these log store implementations apply mostly to the invoker. For instance, the controller can only fetch logs from the log store using the activation logs API. While the rest of the activation APIs must utilize the artifact store. Activation records must be in the artifact store in order for most of the controller activation APIs to return activation information, even though those same activation records may have also been saved in a different backing store by the log store. Consequently, activation records and logs may be duplicated in the log store and artifact store. Another problem revolves around the controller writing user logs. User logs generated by the controller for triggers and sequences are only written to the artifact store even if a non-default log store is being used. Therefore, users cannot run customized queries on user logs generated by the controller. To eliminate the duplication of activations, and controller user logs being inaccessible to the log store, an activation store SPI can be provided. When configured appropriately the activation store would be able to utilize the same data store used by the log store. This would eliminate duplication of stored activation information in separate databases, and allow the log store access to user logs generated by the controller. Work for providing the activation store that is compatible with the artifact store has already started. The related PR can be found here: https://github.com/apache/incubator-openwhisk/pull/3619. Providing an activation store for Elasticsearch will follow. Regards, James Dubee
