roryqi opened a new issue, #11851: URL: https://github.com/apache/gravitino/issues/11851
### Describe the feature Introduce a long-running Optimizer Service for Apache Gravitino Table Maintenance Service workflows. The service should provide REST APIs for optimizer operations that are currently executed through local CLI commands, including: - `update-statistics` - `append-metrics` - `submit-strategy-jobs` - `monitor-metrics` - `list-table-metrics` - `list-job-metrics` The service should support asynchronous task execution, request IDs, task status query, cancellation, and structured results. ### Motivation The current optimizer workflow is mainly local CLI driven. This makes it difficult to support centralized task tracking, retries, concurrency control, audit logging, service-level metrics, and structured automation APIs. A long-running Optimizer Service would let users submit table maintenance work, disconnect from the CLI process, and later query task status and results through stable REST APIs. ### Describe the solution Add an independent Optimizer Service in the optimizer distribution. High-level scope: - Add `OptimizerServiceServer` as a standalone optimizer service entry point. - Add a shared task runtime for async execution and task state management. - Add REST APIs for updater, recommender, monitor, and metrics-query operations. - Keep existing CLI commands and add service mode so the CLI can call the service when configured. - Preserve local CLI execution as the default for backward compatibility. - Keep `submit-update-stats-job` integrated with the existing Gravitino job framework. A detailed design document is proposed in `design-docs/optimizer-service-design.md`. ### Additional context This is intended as an incremental evolution of the alpha Table Maintenance Service (Optimizer). The initial implementation can use an in-memory task store, with persistent task storage and stronger multi-node support added later. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
