dodjdnh opened a new issue, #1586: URL: https://github.com/apache/dubbo-admin/issues/1586
## Motivation / Problem Dubbo Admin already provides Metric and Trace observability entries, but lacks log entries associated with application and instance resources. When investigating a failure, users have to select the application and instance again in an external log system, making it difficult to view the corresponding logs within Admin. We would like to add a unified log-viewing capability supporting existing Loki or Elasticsearch log storage, together with an entry for searching related logs by Trace ID within the log page. Admin should not be responsible for deploying log storage or collectors. ## Current State The following describes the current upstream develop baseline, not the unmerged local feature branch. | Capability | Current state | | --- | --- | | Application / Instance Metric and Trace entries | Available | | Application / Instance Log tabs | Not available | | Log dashboard URL APIs and configuration | No dedicated Log configuration or APIs | | Loki / ES log dashboards shipped with the release | Not available | | Manual Trace ID lookup on the log page | Not available | | External collection integration guidance for log dashboards | Missing the corresponding field contract and examples | ## Desired Behavior - Users can open the log page from application details to view logs from multiple instances of that application, or from instance details to view logs only from the selected instance. - Deployers can choose Loki or Elasticsearch while users continue to use the same Admin log entries. - Logs are displayed as `Time | Level | Pod | Trace ID | Message`, preserving the complete original body and valid Trace IDs. Ordinary logs remain viewable without a Trace ID. - Users can enter a valid Trace ID on the log page and view related logs within T±5 minutes of the actual trace start time T. Invalid input, missing traces, Jaeger configuration or connectivity problems, and unavailable dashboards should produce distinguishable states instead of continuing to show previous results. - Dashboards can be loaded automatically with the Kubernetes release. Log collection, storage, and Grafana log datasources are still supplied by deployers, with actionable reference guidance for integration. ## Scope ### Included - Application / Instance log pages, dashboard URL APIs, and configuration. - Loki / Elasticsearch log dashboards, including variables, queries, five-column display, and Kubernetes loading. - Manual Trace ID search within the log page and the required Jaeger start-time lookup. - A raw application-name parameter, bare Pod IP identity, and a shared log field contract. - Supporting fixes and regression tests for shared dashboard URLs and iframe display. - External OTel log collection configuration, an ES index template example, and operational integration guidance. ### Not Included - Service Log, namespace Log, or cluster Log. - Automatic click-through navigation from a Trace page to Log, Log → Trace Data Links, or Trace → Metric. - dubbo-go Metric compatibility or Dubbo SDK log or metric feature development. - Direct access to Loki / ES from Admin or provider-specific backend query implementations. - Automatic deployment of Loki, ES, production Collectors, or Grafana log datasources, or installation of an ES index template. - ECS conversion or full-text searches of ES Body to find Trace IDs. ## Acceptance Criteria The following defines completion criteria, rather than serving as the current verification result list. Runtime verification for Loki and ES should be recorded separately; unit tests cannot replace a real log pipeline. - [x] Application and Instance details both provide usable log entries. Admin returns the correct dashboard URL, and the browser can load the corresponding Grafana page. - [x] Loki and ES each have two release-provisioned log dashboards loaded automatically by Kubernetes provisioning. Deployers select the Grafana datasource; environment-specific UIDs, storage endpoints, and credentials are not hardcoded. - [x] For instances A / B of the same application, Application Log returns both; Instance A does not return B, and Instance B does not return A. Instance filtering uses the corresponding bare Pod IP known to Admin, without the QoS port. - [x] Both ordinary and dotted application names, such as `shop-user` and `shop.user`, are correctly resolved. Existing Loki application-parameter semantics are preserved, and the new raw application-name parameter supports exact ES matching. - [x] Both backends consistently display five columns. Message matches the source log body, and Trace IDs are not truncated. Missing Level displays `UNKNOWN`; missing Pod names fall back to Pod IP; missing Trace ID displays `—`; ordinary logs remain visible. - [x] Manual search accepts complete 16- or 32-character hexadecimal Trace IDs and rejects invalid input. It uses the earliest valid Span time in the corresponding Jaeger trace, with a final range of T±5 minutes, without guessing the current time or expanding to all history. - [x] Loki manual search supports structured Trace IDs and Trace IDs in the body. ES performs exact matching on `trace_id.keyword` without scanning `Body`. Unrelated trace logs are not included. - [x] Manual search is not restricted by the current instance filter. Query, configuration, and connectivity failures are accurately distinguished; old results are hidden; a late initial dashboard response does not overwrite a new search result or error state. - [x] Integration documentation explains the application, Pod IP, timestamp, and original-body contract and provides adaptable Collector examples. The ES template example includes the required `.keyword` fields and explains manual installation, existing-index limitations, and template-conflict boundaries. - [x] Requests do not mutate shared URLs. Existing Metric / Trace API and query semantics are preserved, and iframe display changes are verified through browser regression checks on the corresponding pages. - [x] Relevant Go / Vue tests, builds, and YAML / JSON / configuration checks pass, with separate evidence for the real log pipeline and manual browser verification. ## Context / Design Notes ### Integration Responsibilities The pipeline is: application logs → external Collector → Loki / Elasticsearch → Grafana datasource / Dashboard → Admin log page. Admin owns resource entries, parameters, and dashboard URLs; Grafana owns storage queries and display. Automatic dashboard loading does not mean automatic datasource registration, nor automatic installation of storage and collectors. Ordinary log viewing does not depend on Jaeger; manual Trace ID lookup on the log page requires a Jaeger query endpoint. ### Fields and Parameters - The shared identity fields are `app` and `pod_ip`, corresponding to the application name and instance IP. Optional fields include `trace_id`, `level`, `pod_name`, and `logger`. - Loki uses an application index label and instance structured metadata. Trace IDs are not Loki index labels. - The first ES integration uses the OTel Elasticsearch exporter's `mapping.mode: raw`: timestamps are stored as `@timestamp`, the original body as `Body`, and identity and log fields as top-level LogRecord attributes, without ECS. - Existing `var-application` is preserved and `var-applicationRaw` is provided, allowing each dashboard to use the parameter semantics it needs without adding provider checks to Admin. - Manual Trace ID lookup reuses the Application log dashboard rather than introducing a third trace-focused dashboard. It is not click-through navigation from a Trace page to the log page. ### Prerequisites and Verification Boundaries The implementation builds on merged changes #1569 and #1573 and does not depend on the pending Instance Trace fixes. Instance lookup for logs is self-contained in the log feature commit. Reference verification versions are Grafana 12.1.0, Loki 3.4.2, Elasticsearch 8.18.0, and OTel Collector Contrib 0.112.0. These versions do not mean the release automatically deploys those components or guarantees compatibility with every version combination. The independent local branch has passed the relevant tests, builds, and static checks. The author manually verified runtime / browser acceptance and confirmed that all checks passed on 2026-10-06; the results are recorded individually in the corresponding PR. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
