songzhendong opened a new pull request, #139: URL: https://github.com/apache/skywalking-nodejs/pull/139
> **Branch:** `feature/nodejs-grpc-v1-runtime-metrics` (based on upstream `master` @ `1d3ba73`) > **Commit:** `481b290` (`481b290beadfc530e4b0adef05dbbbb211af0a66`) — single commit, 26 files changed ### CI status (fork validation, commit `481b290`) All checks **passed** on fork before upstream submission (2026-06-26, 34/34): | Workflow | Result | Run | |----------|--------|-----| | **Build** (Node 20 / 22 / 24) | ✅ | https://github.com/songzhendong/skywalking-nodejs/actions/runs/28235029528 | | **Test** (Lint / TestLib / TestPlugins) | ✅ | https://github.com/songzhendong/skywalking-nodejs/actions/runs/28235029569 | | **License** | ✅ | https://github.com/songzhendong/skywalking-nodejs/actions/runs/28235029547 | Key job (runtime meter E2E): - **TestPlugins (20, express)** — `meterItems` asserts server + client × 6 meters each: https://github.com/songzhendong/skywalking-nodejs/actions/runs/28235029569/job/83647759105 Fork PR checks: https://github.com/songzhendong/skywalking-nodejs/pull/2/checks ### OAP dependency (not in this PR) Runtime meters are reported via `MeterReportService`, but **OAP consumption requires a separate upstream PR** adding the `nodejs-runtime` analyzer (`nodejs-runtime.yaml`, 6 MAL rules). Agent-side reporting and plugin E2E `meterItems` are covered here. --- ### Summary This PR does two things: 1. **gRPC architecture v1** — Refactor Node remote reporting to match Java **`agent.core.*`**: single Channel, unified `BootService` lifecycle, Builder/Decorator chain; Trace / Heartbeat / Meter share one code path for cross-language alignment. 2. **Runtime metrics** — Six `instance_nodejs_*` process meters via `MeterReportService` (1s collect / 1s report). **Not in this PR:** release scripts, version bump, `CONTRIBUTING` / How-to-release, httpbin test infra, OAP analyzer. ### Motivation Before refactor, Node used a standalone `GrpcProtocol` with a different structure from Java `agent.core`. This PR converges remote reporting onto the same architecture as Java: one shared gRPC channel, unified lifecycle and extension points, lower cross-language maintenance cost, and a path consistent with Java for failover, TLS, Event/Log, etc. | Before refactor | After (v1) | |-----------------|------------| | `GrpcProtocol` + **two Channels** (Trace / Heartbeat each own connection) | **One Channel** via shared `GRPCChannelManager` (same as Java / Go) | | `protocol/grpc/clients/*` naming diverges from Java | `core/remote/*` **class names and roles map 1:1 to Java** | | Meter needed a parallel reporting stack | **MeterSender shares BootService → GRPCChannelManager → Client with Trace/Management** | | Segment serialized via external `SegmentObjectAdapter` | **`Segment.transform()`** — same pattern as Java `TraceSegment.transform()` | | No unified `prepare → boot → shutdown` lifecycle | **ServiceManager** orchestrates all BootServices | **v2 out of scope:** multi-address failover, full TLS CA/mTLS, Event/Log clients. ### Architecture overview **Before (Node old) — dual Channel, no central Channel manager** ``` index.ts └── GrpcProtocol ├── HeartbeatClient → own ManagementServiceClient + Channel └── TraceReportClient → own TraceSegmentReportServiceClient + Channel └── SegmentObjectAdapter └── AuthInterceptor() per RPC ``` **After (Node v1) — single Channel, Java-aligned** ``` index.ts └── ServiceManager ├── GRPCChannelManager → single shared Channel (Builder + Decorator chain) ├── TraceSegmentServiceClient → stub after CONNECTED; Segment.transform() ├── ServiceManagementClient → stub after CONNECTED; 20s heartbeat └── MeterSender (optional) → stub after CONNECTED; 6× runtime meters ``` **Java — single Channel + SPI registration** ``` ServiceManager (ServiceLoader loads BootService) ├── GRPCChannelManager → single shared Netty Channel; periodic check + multi-address rotation ├── TraceSegmentServiceClient ├── ServiceManagementClient ├── MeterSender → JVM runtime + custom meters (MeterService API) ├── EventReportServiceClient └── LogReportServiceClient └── TraceSegment.transform() ``` ```mermaid flowchart TB subgraph nodeBefore["Node before refactor"] GP[GrpcProtocol] --> HB[HeartbeatClient Ch1] GP --> TR[TraceReportClient Ch2] end subgraph nodeV1["Node v1"] SM[ServiceManager] --> CM[GRPCChannelManager x1] CM --> TC[TraceSegmentServiceClient] CM --> MC[ServiceManagementClient] CM --> MS[MeterSender] end subgraph javaAgent["Java agent"] JSM[ServiceManager SPI] --> JCM[GRPCChannelManager x1] JCM --> JTC[TraceSegmentServiceClient] JCM --> JMC[ServiceManagementClient] JCM --> JMS[MeterSender] JCM --> JEV[EventReportServiceClient] JCM --> JLG[LogReportServiceClient] end nodeBefore ~~~ nodeV1 ~~~ javaAgent ``` > **Reference baseline** > - **This PR (Node v1):** `feature/nodejs-grpc-v1-runtime-metrics` (`src/agent/core/`) > - **Before refactor (upstream master):** `1d3ba73` (`src/agent/protocol/`) > - **Java agent:** `apm-agent-core` (`org.apache.skywalking.apm.agent.core.{boot,remote,meter}`) ### Alignment with Java Agent | Category | Aspect | This PR (Node v1) | Java agent | Status | |----------|--------|-------------------|------------|--------| | **Architecture** | Package layout | `agent/core/{boot,remote,meter}/` | `agent.core.{boot,remote,meter}/` | ✅ | | | Bootstrap | `ServiceManager.INSTANCE.boot()` | `ServiceManager.INSTANCE.boot()` | ✅ | | | Lifecycle | `BootService` (prepare/boot/shutdown/priority) | Same | ✅ | | | Service registration | Hard-coded 4 services | `ServiceLoader` + SPI | ⚠️ no SPI | | | Segment serialization | `Segment.transform()` | `TraceSegment.transform()` | ✅ | | | flush / shutdown | `ServiceManager.flush/shutdown` | `ServiceManager` unified dispatch | ✅ | | **gRPC Channel** | Channel count | **1** shared `GRPCChannelManager` | **1** shared | ✅ | | | Channel build | Builder chain: Standard → TLS → Decorators | Same | ✅ | | | Connectivity notify | `GRPCChannelListener` CONNECTED/DISCONNECT | Same | ✅ | | | Stub creation | After CONNECTED callback | After CONNECTED callback | ✅ | | | Message size limit | 50 MB | 50 MB | ✅ | | | Multi-address failover | First comma-separated address only | List rotation + periodic reconnect | ❌ v2 | | | Error-driven reconnect | `reportError()` stub (debug log) | `reportError → reconnect` | ❌ v2 | | | Periodic Channel check | None | `GRPC_CHANNEL_CHECK_INTERVAL` | ❌ v2 | | | Force reconnect / DNS re-resolve | None | `FORCE_RECONNECTION_PERIOD`, etc. | ❌ v2 | | **TLS** | `SW_AGENT_SECURE` | `TLSChannelBuilder` → `createSsl()` | Full `TLSChannelBuilder` chain | ⚠️ minimal TLS | | | Custom CA / mTLS | Not implemented | `agent/ca/` + cert path config | ❌ v2 | | | TLS E2E CI | None | `simple/ssl`, `simple/mtls` | ❌ v2 | | **Remote clients** | Trace reporting | `TraceSegmentServiceClient` | `TraceSegmentServiceClient` | ✅ | | | Heartbeat / instance | `ServiceManagementClient` (20s) | `ServiceManagementClient` | ✅ | | | Auth / Agent-Version headers | `AuthenticationDecorator` + `AgentIDDecorator` | Same | ✅ | | | Event / Log | Not implemented | `EventReportServiceClient` / `LogReportServiceClient` | ❌ v2 | | | Stream completion wait | `GRPCStreamServiceStatus` ported, not wired | Used for stream RPCs | ❌ v2 | | **Runtime meter** | Reporting component | `MeterSender` (BootService + shared Channel) | `MeterSender` (BootService + shared Channel) | ✅ | | | Collect / report period | Default **1s / 1s** (configurable) | JVM runtime default **1s / 1s** | ✅ | | | RPC service | `MeterReportService` (generic Meter protobuf) | `JVMMetricReportService` (JVM protobuf) | ⚠️ different RPC / message schema (both gRPC) | | | Metric coverage | 6× `instance_nodejs_*` (CPU + memory) | heap / GC / thread / classloader / … | ⚠️ different coverage | | | Meter types | `MeterSingleValue` gauge | Counter / Gauge / Histogram | ⚠️ no Counter/Histogram | | | Custom business meters | No public API (README suggests OTel) | `MeterFactory` / `MeterService` (≤500) | ❌ v2 | | | Buffer limit | `SW_AGENT_NODEJS_RUNTIME_METRICS_BUFFER_SIZE` = 600 | `Config.Jvm.BUFFER_SIZE` = 600 | ✅ | | | Per-metric timestamp | One shared `timestamp` per stream batch | Each `JVMMetric` has its own time | ⚠️ precision difference | | | Retry on failure | Drop oldest when full; discard batch on send failure | `offer` full → drop oldest; send failure not re-queued | ✅ | | | CPU calculation | `process.cpuUsage()` ÷ logical cores | JVM MXBean multi-dimensional | ⚠️ different algorithm | | | OAP consumption | `nodejs-runtime.yaml` (6 MAL rules) | Built-in JVM analyzer | ⚠️ needs OAP analyzer | | **Tests** | Runtime meter unit tests | ✅ `RuntimeMetricsCollector.test.ts` | — | ✅ | | | Plugin E2E meter assertions | express `meterItems` (server + client, 6 meters each) | — | ✅ | | | TLS E2E CI | None (plaintext gRPC CI) | Yes | ❌ v2 | **Design note:** Node runtime metrics use `MeterReportService` (same as Go/Python), not Java `JVMMetricReportService`. **Six meters:** `instance_nodejs_process_cpu`, `instance_nodejs_heap_used`, `instance_nodejs_heap_total`, `instance_nodejs_heap_limit`, `instance_nodejs_rss`, `instance_nodejs_external_memory` ### File / class mapping (before → v1 → Java) | Role | Node before | Node v1 | Java | |------|-------------|---------|------| | Bootstrap | `GrpcProtocol.ts` | `ServiceManager.ts` | `ServiceManager.java` | | Lifecycle contract | `Protocol.ts` | `BootService.ts` | `BootService.java` | | Channel manager | ❌ (scattered in clients) | `GRPCChannelManager.ts` | `GRPCChannelManager.java` | | Channel build | ❌ | `GRPCChannel.ts` | `GRPCChannel.java` | | Plaintext builder | ❌ | `StandardChannelBuilder.ts` | `StandardChannelBuilder.java` | | TLS builder | ❌ (inline in client) | `TLSChannelBuilder.ts` | `TLSChannelBuilder.java` | | Auth decorator | `AuthInterceptor.ts` | `AuthenticationDecorator.ts` | `AuthenticationDecorator.java` | | Agent-Version decorator | ❌ | `AgentIDDecorator.ts` | `AgentIDDecorator.java` | | Connection status | `Client.ts` | `GRPCChannelStatus.ts` + Listener | `GRPCChannelStatus.java` + Listener | | Trace client | `TraceReportClient.ts` | `TraceSegmentServiceClient.ts` | `TraceSegmentServiceClient.java` | | Heartbeat client | `HeartbeatClient.ts` | `ServiceManagementClient.ts` | `ServiceManagementClient.java` | | Segment transform | `SegmentObjectAdapter.ts` | `Segment.transform()` | `TraceSegment.transform()` | | Meter | ❌ | `meter/{RuntimeSampler,RuntimeMetricsCollector,MeterSender}.ts` | `meter/{MeterSender,MeterService,...}.java` | | Event client | ❌ | ❌ | `EventReportServiceClient.java` | | Log client | ❌ | ❌ | `LogReportServiceClient.java` | ### gRPC v1 implemented #### Boot lifecycle | Module | Path | Notes | |--------|------|-------| | Boot contract | `src/agent/core/boot/BootService.ts` | Aligns with Java: `prepare()` → `boot()` → `onComplete()` → `shutdown()`, `priority()` ordering | | Bootstrap | `src/agent/core/boot/ServiceManager.ts` | Singleton manages all `BootService`; unified `boot()` / `shutdown()` / `flush()` | | Entry | `src/index.ts` | Replaces old `GrpcProtocol().heartbeat().report()` with `ServiceManager` | **Services registered by `ServiceManager` (hard-coded, no Java SPI):** | Service | Priority / condition | Role | |---------|----------------------|------| | `GRPCChannelManager` | Always | Shared gRPC Channel, connectivity listener | | `TraceSegmentServiceClient` | Always | Trace segment streaming | | `ServiceManagementClient` | Always | Instance properties + keepAlive heartbeat (20s) | | `MeterSender` | `runtimeMetricsReporterActive=true` | Runtime meter collect + report | #### gRPC Channel stack (`remote` package) ``` GRPCChannelManager └── GRPCChannel.newBuilder(host, port) ├── StandardChannelBuilder → insecure creds + 50MB message limit ├── TLSChannelBuilder → SW_AGENT_SECURE=true → createSsl() ├── AgentIDDecorator → Agent-Version metadata └── AuthenticationDecorator → SW_AGENT_AUTHENTICATION header ``` | File | Java counterpart | v1 behavior | |------|------------------|-------------| | `GRPCChannelManager.ts` | `GRPCChannelManager.java` | Single-address parse; `watchConnectivityState` notifies listeners; `reportError()` debug log only | | `GRPCChannel.ts` | `GRPCChannel.java` | Builder chain + `channelOverride` / interceptors | | `ChannelBuilder.ts` | `ChannelBuilder.java` | credentials / options context | | `StandardChannelBuilder.ts` | `StandardChannelBuilder.java` | Plaintext + message size limit | | `TLSChannelBuilder.ts` | `TLSChannelBuilder.java` | **Minimal:** `SW_AGENT_SECURE=true` → `grpc.credentials.createSsl()` | | `ChannelDecorator.ts` | `ChannelDecorator.java` | grpc-js interceptor decorator interface | | `AgentIDDecorator.ts` | `AgentIDDecorator.java` | Agent version header | | `AuthenticationDecorator.ts` | `AuthenticationDecorator.java` | OAP auth token | | `GRPCChannelListener.ts` | `GRPCChannelListener.java` | CONNECTED / DISCONNECT callbacks | | `GRPCChannelStatus.ts` | (from Node `Client.ts` enum) | Connection status enum | #### Remote clients (listener-driven stubs) | File | Old Node name | gRPC service | v1 behavior | |------|---------------|--------------|-------------| | `TraceSegmentServiceClient.ts` | `TraceReportClient` | `TraceSegmentReportService` | Stub on CONNECTED; buffer + periodic flush; `reportError()` on failure | | `ServiceManagementClient.ts` | `HeartbeatClient` | `ManagementService` | Stub on CONNECTED; 20s instance properties + keepAlive | **Segment serialization:** new `Segment.transform(): SegmentObject` in `src/trace/context/Segment.ts`; removed `SegmentObjectAdapter.ts` (same pattern as Java `TraceSegment.transform()`). #### Runtime meter (shared Channel) | File | Notes | |------|-------| | `RuntimeSampler.ts` | Single sample: `memoryUsage` + `v8.getHeapStatistics()` + `cpuUsage` | | `RuntimeMetricsCollector.ts` | Maps to 6 meters | | `MeterSender.ts` | `BootService` + `GRPCChannelListener`; 1s collect / **1s report** (configurable); `MeterReportService.collect` streaming | | Meter name | Meaning | Notes | |------------|---------|-------| | `instance_nodejs_process_cpu` | Process CPU | **%** (user+system, ÷ logical cores; see README) | | `instance_nodejs_heap_used` | V8 heap used | bytes | | `instance_nodejs_heap_total` | V8 heap total | bytes | | `instance_nodejs_heap_limit` | V8 heap limit | bytes | | `instance_nodejs_rss` | RSS | bytes | | `instance_nodejs_external_memory` | External memory | bytes | #### Configuration **New environment variables:** | Variable | Default | Notes | |----------|---------|-------| | `SW_AGENT_NODEJS_RUNTIME_METRICS_REPORTER_ACTIVE` | `true` (explicit `false` disables) | Enable runtime meter | | `SW_AGENT_NODEJS_RUNTIME_METRICS_COLLECT_PERIOD` | `1000` ms | Sample interval (local buffer, default 1s) | | `SW_AGENT_NODEJS_RUNTIME_METRICS_REPORT_PERIOD` | **`1000` ms (1s)** | Report interval, **aligned with Java JVM runtime (1s)** | | `SW_AGENT_NODEJS_RUNTIME_METRICS_BUFFER_SIZE` | `600` | Buffer size | **Deprecated aliases (still readable):** `SW_AGENT_RUNTIME_METRICS_*`, `SW_AGENT_NVM_METRICS_*`, `SW_AGENT_NVM_JVM_*`. **Existing gRPC settings (semantics unchanged):** - `SW_AGENT_COLLECTOR_BACKEND_SERVICES` — v1 uses **first** comma-separated address only - `SW_AGENT_SECURE` — enables TLS ChannelBuilder - `SW_AGENT_AUTHENTICATION` — OAP authentication #### Removed / replaced Node-specific abstractions | Removed | Replaced by | |---------|-------------| | `src/agent/protocol/grpc/GrpcProtocol.ts` | `ServiceManager` + clients | | `src/agent/protocol/grpc/SegmentObjectAdapter.ts` | `Segment.transform()` | | `src/agent/protocol/grpc/clients/*` | `src/agent/core/remote/*` (renamed + refactored) | | `src/agent/protocol/Protocol.ts` | `src/agent/core/boot/BootService.ts` | Entire `src/agent/protocol/` directory removed. #### Tests and docs | Item | Status | |------|--------| | Unit tests | `tests/runtime/RuntimeMetricsCollector.test.ts` — 6 meter names and non-negative values | | Plugin E2E | `tests/plugins/express/expected.data.yaml` — **only** added `meterItems` (one block with server + client entries, 6 meters each); httpbin / docker-compose unchanged | | CI | Build / Lint / TestLib / TestPlugins, Linux Node 20/22/24 (**plaintext gRPC**) | | README | Node.js ≥ 20, runtime metrics table and env docs | #### TLS v1 scope - `SW_AGENT_SECURE=true` → `TLSChannelBuilder` → `grpc.credentials.createSsl()` (system default trust store) - Suitable for **public CA / enterprise PKI** OAP certificates - **CI does not cover TLS** (same as Python / Go / PHP agents, plaintext `collector:11800`); self-signed CA / mTLS need v2 `SW_AGENT_CA_PATH` (`NODE_EXTRA_CA_CERTS` ineffective for `@grpc/grpc-js`) **Manual smoke test (2026-06-26, maintainer-local — not in CI):** | Check | Method | Result | |-------|--------|--------| | Setup | Docker OAP (BanyanDB) + Node Agent; OAP gRPC TLS on host `:11811` | — | | OAP | `SW_CORE_GRPC_SSL_ENABLED=true`, certs from `tls_key_generate.sh` (`SERVER_CN=localhost`) | ✅ | | Agent | `SW_AGENT_SECURE=true`, `SW_AGENT_COLLECTOR_BACKEND_SERVICES=localhost:11811` | ✅ | | TLS handshake | `openssl s_client -connect 127.0.0.1:11811 -servername localhost`, ALPN `h2` | ✅ | | Service registration | OAP GraphQL `getAllServices` | ✅ | | Instance heartbeat | GraphQL `getServiceInstances` (Agent running ≥25s, UTC window) | ✅ | Procedure: start TLS OAP → Agent (`secure: true`) → local HTTP request → `agent.flush()` → GraphQL check services and instances. Self-signed CA required loading root CA into `createSsl(rootCerts)` (**temporary patch, not in this PR**); public/enterprise CA works with `SW_AGENT_SECURE=true` alone. TLS verification should be covered by CI E2E in a follow-up (v2). ### Out of scope (v2) #### Channel failover and reconnect Per `GRPCChannelManager.ts` comments: multi-address backend rotation, `reportError()`-driven failover, periodic Channel check, forced reconnect, periodic DNS re-resolve. #### TLS / mTLS Relative to Java `TLSChannelBuilder`: custom CA path, mTLS client certs, `SW_AGENT_CA_PATH` (or equivalent), self-signed TLS E2E CI. Note: `NODE_EXTRA_CA_CERTS` is ineffective for `@grpc/grpc-js`; must pass CA explicitly to `createSsl(ca)`. #### Remote clients not ported - `EventReportServiceClient` (proto exists, no agent client) - `LogReportServiceClient` (proto exists, no agent client) #### Skeleton / unwired code - `GRPCStreamServiceStatus.ts`: ported from Java, not referenced by Trace/Meter clients yet - Java `ServiceLoader` SPI: Node hard-codes 4 services, no plugin discovery #### Test and API gaps - TLS / mTLS E2E CI - Multi-address failover tests - Custom business metrics public API (README suggests OpenTelemetry) - `instance_nodejs_active_handles` not in the current 6-meter set #### Runtime requirements - Node **>= 20** required; CI matrix 20/22/24 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
