<!-- Thank you for suggesting an idea to make Teaclave better. Please fill in as much of the template below as you're able. -->
### Motivation & problem statement <!-- Provide a clear and concise description of what the problem is. --> Current implementation uses env_logger that outputs to std::io. It is convenient for development as the developer can see the logs immediately in the terminal. However, it is not friendly to the users. The users have no access to logs, which is not good for workflow depending on intermedia outputs, e.g., model tuning. Besides, the logging service is often required for audit in some standards of privacy-preserving platforms. It is also efficient for platform Ops to see the logs of all the distributed services through a single interface. ### Proposed solution <!-- Provide a clear and concise description of what you want to happen. --> The implementation has two parts: one is for task log and the other is for platform log. #### Task log A new logger implementation named `teaclave_logger` is added. It supports both env_logger and `task_logger`. After the execution service pulls a task successfully, `task_logger` is used. The logs will be buffered in memory and sent to storage service after the task is over or the buffer is full. The key of the log in storage is the task id with a sequence number in case the logs are sent multiple times. The task users can get the task log after they are authored through FE. This design is only for the execution service that handles tasks one by one. Concurrency would be handled when the execution service could handle tasks parallely in later design. It is out of the scope in this solution. When the execution service is not handling tasks, env_logger is used. #### Platform log A new `logging` service that aggregates logs is added. It is backed by [tantivy](https://github.com/quickwit-oss/tantivy). Tantivy is a full-text search engine library. It has powerful features and can support quick search towards the logs. The env_logger in every service is replaced by `platform_logger` in `teaclave_logger`. All the platform logs except debug level ones will be sent to `logging` service. Only platform admin can see the logs through FE. The logs can be searched in `tantivy` way. The new service topology will be as below: ``` clients => authentication <-+ +----> storage <----+ | | | | | clients => frontend ----------> management scheduler <-- execution | ---> logging | | +--> access_control | => api endpoint connections -> internal endpoint connections ``` The `debug` level logs will be output to std::io anyway. ### Describe alternatives you've considered <!-- Let us know about other solutions you've tried or researched. --> [quickwit](https://github.com/quickwit-oss/quickwit): it uses AGPL license that is not friendly to teaclave. No other mature loggers written Rust are found. Feel free to recommend here. -- Reply to this email directly or view it on GitHub: https://github.com/apache/incubator-teaclave/issues/669 You are receiving this because you are subscribed to this thread. Message ID: <apache/incubator-teaclave/issues/6...@github.com>
