This is an automated email from the ASF dual-hosted git repository.
wusheng pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/skywalking-graalvm-distro.git
The following commit(s) were added to refs/heads/main by this push:
new b7e4c31 Add compiling guide, pre-built Docker image doc, and track
existing docs
b7e4c31 is described below
commit b7e4c31fbb0f7237a26b22d8d872b44aa3ad6806
Author: Wu Sheng <[email protected]>
AuthorDate: Fri Feb 27 08:55:25 2026 +0800
Add compiling guide, pre-built Docker image doc, and track existing docs
- Add docs/compiling.md: step-by-step build guide from source to native
image
- Add docs/docker-image.md: GHCR pre-built image usage, tags, and run
examples
- Add Setup section to docs/menu.yml grouping compiling, Docker image, and
config
- Track existing docs/ files that were previously untracked (distro-policy,
oal/mal/lal-immigration, config-init-immigration)
---
docs/README.md | 2 +
docs/compiling.md | 169 +++++++++
docs/config-init-immigration.md | 253 +++++++++++++
docs/distro-policy.md | 322 ++++++++++++++++
docs/docker-image.md | 108 ++++++
docs/lal-immigration.md | 352 +++++++++++++++++
docs/mal-immigration.md | 817 ++++++++++++++++++++++++++++++++++++++++
docs/menu.yml | 10 +-
docs/oal-immigration.md | 162 ++++++++
9 files changed, 2193 insertions(+), 2 deletions(-)
diff --git a/docs/README.md b/docs/README.md
index eec7fff..9748460 100644
--- a/docs/README.md
+++ b/docs/README.md
@@ -104,6 +104,8 @@ and asserts identical results. Tests require actual data
flow — no vacuous emp
## Further Reading
- [Distribution Policy](distro-policy.md) — full module table, architecture
constraints, build workflow
+- [Compiling from Source](compiling.md) — build JVM distro, native image, and
Docker image step by step
+- [Pre-Built Docker Images](docker-image.md) — pull and run the CI-built
native image from GHCR
- [Configuration](configuration.md) — all available settings, environment
variables, and differences from upstream
- [OAL Pre-Compilation](oal-immigration.md) — Javassist class export,
annotation scan manifests
- [MAL Transpilation](mal-immigration.md) — Groovy-to-Java transpiler,
combination pattern, functional interfaces
diff --git a/docs/compiling.md b/docs/compiling.md
new file mode 100644
index 0000000..9777ab7
--- /dev/null
+++ b/docs/compiling.md
@@ -0,0 +1,169 @@
+# Compiling from Source
+
+This guide covers building the SkyWalking GraalVM Distro from source,
producing both the
+JVM distribution and the GraalVM native image binary.
+
+## Prerequisites
+
+- **GraalVM JDK 25** with `native-image` installed
+- **Maven 3.9+** (the Maven wrapper `./mvnw` is included)
+- **Docker** (for macOS cross-compilation and container packaging)
+- **Git** (with submodule support)
+
+Set `JAVA_HOME` to your GraalVM installation for all commands below:
+
+```bash
+export JAVA_HOME=/path/to/graalvm-jdk-25
+```
+
+## Step 1: Clone the Repository
+
+```bash
+git clone --recurse-submodules
https://github.com/apache/skywalking-graalvm-distro.git
+cd skywalking-graalvm-distro
+```
+
+If you already cloned without `--recurse-submodules`:
+
+```bash
+git submodule update --init --recursive
+```
+
+## Step 2: Install SkyWalking Submodule (First Time Only)
+
+The upstream SkyWalking artifacts must be installed into your local Maven
repository before
+the distro modules can compile against them:
+
+```bash
+make init-skywalking
+```
+
+This runs `mvn install` on the `skywalking/` submodule. The result is cached
in `~/.m2/repository`
+and only needs to be re-run when the submodule is updated.
+
+## Step 3: Build the JVM Distribution
+
+```bash
+make build-distro
+```
+
+This runs the complete build pipeline:
+
+1. Compiles and installs the repackaged `*-for-graalvm` libraries
+2. Runs the build-time precompiler (OAL class export, MAL/LAL transpilation,
reflection metadata)
+3. Packages the JVM distribution with assembly
+
+Output:
+
+```
+oap-graalvm-server/target/oap-graalvm-jvm-distro/oap-graalvm-jvm-distro/ #
exploded
+oap-graalvm-server/target/oap-graalvm-jvm-distro.tar.gz #
tarball
+```
+
+### Running the JVM Distribution Locally
+
+Start BanyanDB and boot the OAP server:
+
+```bash
+make boot
+```
+
+This starts BanyanDB via Docker Compose and launches the JVM distribution with
+`SW_STORAGE_BANYANDB_TARGETS=localhost:17912`.
+
+To stop:
+
+```bash
+make shutdown
+make docker-down
+```
+
+## Step 4: Build the Native Image
+
+After `make build-distro` (or `make compile`), build the native binary:
+
+```bash
+make native-image
+```
+
+This invokes `native-maven-plugin` with the `-Pnative` profile. The native
image build
+takes several minutes and requires significant memory (~8GB RAM recommended).
+
+Output:
+
+```
+oap-graalvm-native/target/oap-graalvm-native-*-native-dist/oap-native/ #
exploded
+oap-graalvm-native/target/oap-graalvm-native-*-native-dist.tar.gz #
tarball
+```
+
+The `oap-native/` directory contains:
+
+```
+oap-native/
+├── oap-server # ~203MB native binary
+├── config/ # application.yml, alarm-settings.yml, ui-templates,
etc.
+└── log4j2.xml # Console-only logging (no RollingFile)
+```
+
+### Cross-Compiling for Linux on macOS
+
+If you are on macOS and need a Linux native binary (e.g., for Docker):
+
+```bash
+make native-image-macos
+```
+
+This runs the `native-image` step inside a
`ghcr.io/graalvm/native-image-community:25`
+container, producing a Linux binary while reusing your host's Maven cache and
compiled classes.
+
+## Step 5: Package as Docker Image
+
+After building the native image:
+
+```bash
+make docker-native
+```
+
+This builds a Docker image `skywalking-oap-native:latest` based on
`debian:bookworm-slim`
+containing only the native binary and config files.
+
+### Running with Docker Compose
+
+```bash
+docker compose -f docker/docker-compose.yml up
+```
+
+This starts BanyanDB and the OAP native image together. Ports exposed:
+
+| Port | Service |
+|------|---------|
+| 12800 | REST API (GraphQL, health check) |
+| 11800 | gRPC (agent data collection) |
+| 17912 | BanyanDB |
+
+## Other Make Targets
+
+| Target | Description |
+|--------|-------------|
+| `make compile` | Compile and install to local repo (no tests) |
+| `make test` | Run all tests (includes MAL/LAL comparison tests) |
+| `make javadoc` | Validate javadoc correctness |
+| `make dist` | Build JVM distro and print output path |
+| `make native-dist` | Build native image and print output path |
+| `make clean` | Clean all build artifacts |
+| `make docker-up` | Start BanyanDB only (for local dev) |
+| `make docker-down` | Stop BanyanDB |
+| `make boot` | Build distro + start BanyanDB + boot OAP |
+| `make shutdown` | Stop a running OAP server |
+
+## CI Pipeline
+
+The GitHub Actions CI (`.github/workflows/ci.yml`) automates the full pipeline
on every push to `main`:
+
+1. **License header check** — Apache SkyWalking Eyes
+2. **Build & Test** — compile, javadoc, test, build-distro
+3. **Native image** — builds on both `amd64` and `arm64` runners
+4. **Docker image** — pushes multi-arch manifest to GHCR
+5. **E2E tests** — runs end-to-end tests against the native image
+
+See [Pre-Built Docker Images](docker-image.md) for using the CI-built images
directly.
diff --git a/docs/config-init-immigration.md b/docs/config-init-immigration.md
new file mode 100644
index 0000000..c9f3a25
--- /dev/null
+++ b/docs/config-init-immigration.md
@@ -0,0 +1,253 @@
+# Config Initialization — Eliminate Reflection in Config Loading
+
+## Context
+
+`YamlConfigLoaderUtils.copyProperties()` uses `Field.setAccessible(true)` +
+`field.set()` to populate `ModuleConfig` objects from YAML properties. This
+reflection pattern is problematic for GraalVM native image — every config field
+would require `reflect-config.json` entries, and `setAccessible()` is
restricted.
+
+Since our module/provider set is fixed (37 modules, see distro-policy.md), we
+can generate hardcoded field-setting code at build time that eliminates all
+config-related reflection.
+
+---
+
+## Problem
+
+In `ModuleDefine.prepare()` and in `BanyanDBConfigLoader`,
+`copyProperties()` iterates property names, looks up fields by name via
+reflection, and sets them:
+
+```java
+Field field = getDeclaredField(destClass, propertyName);
+field.setAccessible(true); // restricted in native image
+field.set(dest, value); // needs reflect-config.json
+```
+
+This requires:
+- `getDeclaredField()` with class hierarchy walk
+- `field.setAccessible(true)` to bypass private access
+- `field.set()` for every property
+
+---
+
+## Solution: Same-FQCN Replacement of YamlConfigLoaderUtils
+
+Generate a replacement `YamlConfigLoaderUtils.java` with the same FQCN
+(`org.apache.skywalking.oap.server.library.util.YamlConfigLoaderUtils`) that
+dispatches by config object type and sets fields directly — no
+`Field.setAccessible()`, no `getDeclaredField()` scan.
+
+This is one of 23 same-FQCN replacement classes in the distro (see
distro-policy.md for full list).
+
+### Field Access Strategy (per field)
+
+| Strategy | Condition | Example |
+|---|---|---|
+| **Lombok setter** | All non-final fields (class-level `@Setter` added via
`-for-graalvm` modules) | `cfg.setRole((String) value)` |
+| **Getter + clear + addAll** | Final collection field |
`cfg.getDownsampling().clear(); cfg.getDownsampling().addAll((List) value)` |
+| **Error** | Unknown config type | `throw new
IllegalArgumentException("Unknown config type: ...")` |
+
+All config classes that previously lacked `@Setter` now have it added via
+same-FQCN replacement classes in the `-for-graalvm` modules. No VarHandle, no
+reflection fallback. The generator fails at build time if any non-final field
+lacks a setter.
+
+### Generated Code Structure
+
+```java
+package org.apache.skywalking.oap.server.library.util;
+
+public class YamlConfigLoaderUtils {
+ // No VarHandle, no reflection. Pure setter-based.
+
+ // Type-dispatch: check instanceof, delegate to type-specific method
+ public static void copyProperties(Object dest, Properties src,
+ String moduleName, String providerName)
+ throws IllegalAccessException {
+ if (dest instanceof CoreModuleConfig) {
+ copyToCoreModuleConfig((CoreModuleConfig) dest, src, moduleName,
providerName);
+ } else if (dest instanceof ClusterModuleKubernetesConfig) {
+
copyToClusterModuleKubernetesConfig((ClusterModuleKubernetesConfig) dest, src,
moduleName, providerName);
+ }
+ // ... all config types ...
+ else {
+ throw new IllegalArgumentException("Unknown config type: " +
dest.getClass().getName());
+ }
+ }
+
+ // Per-type: switch on property name, set via Lombok setter
+ private static void copyToCoreModuleConfig(
+ CoreModuleConfig cfg, Properties src,
+ String moduleName, String providerName) {
+ // iterate properties
+ switch (key) {
+ case "role":
+ cfg.setRole((String) value);
+ break;
+ case "persistentPeriod":
+ cfg.setPersistentPeriod((int) value);
+ break;
+ // ... all fields via setters ...
+ default:
+ log.warn("{} setting is not supported in {} provider of {}
module",
+ key, providerName, moduleName);
+ break;
+ }
+ }
+ // ... one method per config type ...
+}
+```
+
+---
+
+## Build Tool
+
+**Module**: `build-tools/config-generator`
+
+**Main class**: `ConfigInitializerGenerator.java`
+
+### Input
+- Classpath with all SkyWalking module JARs (same dependencies as
`oap-graalvm-server`)
+- Provider class list — derived from the fixed module table in
`GraalVMOAPServerStartUp`
+- Extra config class list — BanyanDB nested configs used by
`BanyanDBConfigLoader`
+
+### Process
+1. For each provider, call `newConfigCreator().type()` to discover the config
class
+2. For each config class, use Java reflection to scan all declared fields
+ (walking up to `ModuleConfig` superclass)
+3. For each field, check if a setter method exists (`set` + capitalize(name))
+4. **Fail if any non-final field lacks a setter** (all config classes must
have `@Setter`)
+5. Generate `YamlConfigLoaderUtils.java` with type-dispatch + switch-based
field assignment
+
+The generator depends on `-for-graalvm` modules (not upstream JARs) so it sees
+config classes with `@Setter` added. Original upstream JARs are forced to
+`provided` scope to prevent classpath shadowing.
+
+### Output
+-
`oap-graalvm-server/src/main/java/org/apache/skywalking/oap/server/library/util/YamlConfigLoaderUtils.java`
+ (same-FQCN replacement)
+
+### Running the generator
+```bash
+JAVA_HOME=/Users/wusheng/.sdkman/candidates/java/25-graal \
+ mvn -pl build-tools/config-generator exec:java \
+
-Dexec.args="oap-graalvm-server/src/main/java/org/apache/skywalking/oap/server/library/util/YamlConfigLoaderUtils.java"
+```
+
+---
+
+## Config Classes Inventory
+
+37 modules registered in `GraalVMOAPServerStartUp`. 1 (`AlarmModule`) has null
+ConfigCreator. The rest need coverage.
+
+### Provider Config Classes
+
+| Provider | Config Class | Fields | Setter Status |
+|---|---|---|---|
+| CoreModuleProvider | `CoreModuleConfig` | ~40 | `@Getter` class-level,
`@Setter` on ~12 fields, rest need VarHandle |
+| BanyanDBStorageProvider | `BanyanDBStorageConfig` | ~5 | `@Getter @Setter` |
+| ClusterModuleStandaloneProvider | (empty config) | 0 | — |
+| ClusterModuleKubernetesProvider | `ClusterModuleKubernetesConfig` | 3 |
`@Setter` |
+| ConfigmapConfigurationProvider | `ConfigmapConfigurationSettings` | 3 |
`@Setter` |
+| PrometheusTelemetryProvider | `PrometheusConfig` | 5 | `@Setter` |
+| AnalyzerModuleProvider | `AnalyzerModuleConfig` | ~10 | `@Getter @Setter` |
+| LogAnalyzerModuleProvider | `LogAnalyzerModuleConfig` | 2 | `@Setter` |
+| SharingServerModuleProvider | `SharingServerConfig` | ~15 | Needs check |
+| EnvoyMetricReceiverProvider | `EnvoyMetricReceiverConfig` | ~15 | Needs
check |
+| KafkaFetcherProvider | `KafkaFetcherConfig` | ~20 | `@Data` (all setters) |
+| ZipkinReceiverProvider | `ZipkinReceiverConfig` | ~20 | Needs check |
+| GraphQLQueryProvider | `GraphQLQueryConfig` | 4 | `@Setter` |
+| HealthCheckerProvider | `HealthCheckerConfig` | 1 | `@Setter` |
+| + others | (simple configs) | 0-5 | Various |
+
+### BanyanDB Config Loading (BanyanDBConfigLoader)
+
+`BanyanDBConfigLoader` (in `storage-banyandb-plugin`) loads config from
`bydb.yml`
+and `bydb-topn.yml` independently of the standard YAML config loading path. It
+calls `copyProperties()` on nested config objects, so the generated
+`YamlConfigLoaderUtils` must handle all BanyanDB inner classes.
+
+**`loadBaseConfig()`** — reads `bydb.yml`, calls `copyProperties()` 11 times:
+
+| Call target | Type | Handler | Fields |
+|---|---|---|---|
+| `config.getGlobal()` | `Global` | `copyToGlobal()` | 18 (targets,
maxBulkSize, flushInterval, ...) |
+| `config.getRecordsNormal()` | `RecordsNormal` | `copyToRecordsNormal()` | 8
(inherited from GroupResource) |
+| `config.getRecordsLog()` | `RecordsLog` | `copyToRecordsLog()` | 8 |
+| `config.getTrace()` | `Trace` | `copyToTrace()` | 8 |
+| `config.getZipkinTrace()` | `ZipkinTrace` | `copyToZipkinTrace()` | 8 |
+| `config.getRecordsBrowserErrorLog()` | `RecordsBrowserErrorLog` |
`copyToRecordsBrowserErrorLog()` | 8 |
+| `config.getMetricsMin()` | `MetricsMin` | `copyToMetricsMin()` | 8 |
+| `config.getMetricsHour()` | `MetricsHour` | `copyToMetricsHour()` | 8 |
+| `config.getMetricsDay()` | `MetricsDay` | `copyToMetricsDay()` | 8 |
+| `config.getMetadata()` | `Metadata` | `copyToMetadata()` | 8 |
+| `config.getProperty()` | `Property` | `copyToProperty()` | 8 |
+
+**`copyStages()`** — creates warm/cold `Stage` objects, calls
`copyProperties()` on each:
+
+| Call target | Type | Handler | Fields |
+|---|---|---|---|
+| warm Stage | `Stage` | `copyToStage()` | 7 (name, nodeSelector, shardNum,
segmentInterval, ttl, replicas, close) |
+| cold Stage | `Stage` | `copyToStage()` | 7 |
+
+**`loadTopNConfig()`** — reads `bydb-topn.yml`, populates `TopN` objects via
direct
+setter calls (`topN.setName()`, `topN.setGroupByTagNames()`, etc.). Does NOT
call
+`copyProperties()` — no handler needed.
+
+**Class hierarchy note**: All group config classes (`RecordsNormal`, `Trace`,
+`MetricsMin`, etc.) extend `GroupResource`. The generator walks the class
hierarchy
+from each subclass up to `GroupResource` and generates setter calls for all 8
+inherited fields (`shardNum`, `segmentInterval`, `ttl`, `replicas`,
+`enableWarmStage`, `enableColdStage`, `defaultQueryStages`,
+`additionalLifecycleStages`). `Trace` doesn't have its own `@Getter @Setter`
but
+inherits all setters from `GroupResource`.
+
+**`instanceof` ordering**: Inner classes (`Global`, `RecordsNormal`, `Trace`,
etc.)
+are static inner classes that do NOT extend `BanyanDBStorageConfig`, so the
+`instanceof BanyanDBStorageConfig` check matches only the top-level config.
Inner
+class checks come after and work independently.
+
+Two extra inner classes (`RecordsTrace`, `RecordsZipkinTrace`) are included in
+EXTRA_CONFIG_CLASSES for completeness, even though `BanyanDBConfigLoader`
doesn't
+currently call `copyProperties()` on them.
+
+---
+
+## Same-FQCN Replacements (Config Initialization)
+
+| Upstream Class | Upstream Location | Replacement Location | What Changed |
+|---|---|---|---|
+| `YamlConfigLoaderUtils` |
`server-library/library-util/.../util/YamlConfigLoaderUtils.java` |
`oap-graalvm-server/` (not in library-util-for-graalvm due to 30+ cross-module
imports) | Complete rewrite. Uses type-dispatch with Lombok `@Setter` methods
instead of `Field.setAccessible()` + `field.set()`. |
+| `CoreModuleConfig` | `server-core/.../core/CoreModuleConfig.java` |
`oap-libs-for-graalvm/server-core-for-graalvm/` | Added `@Setter` at class
level. Upstream only has `@Getter`. |
+| `AnalyzerModuleConfig` |
`analyzer/agent-analyzer/.../provider/AnalyzerModuleConfig.java` |
`oap-libs-for-graalvm/agent-analyzer-for-graalvm/` | Added `@Setter` at class
level. |
+| `LogAnalyzerModuleConfig` |
`analyzer/log-analyzer/.../provider/LogAnalyzerModuleConfig.java` |
`oap-libs-for-graalvm/log-analyzer-for-graalvm/` | Added `@Setter` at class
level. |
+| `EnvoyMetricReceiverConfig` |
`server-receiver-plugin/envoy-metrics-receiver-plugin/.../EnvoyMetricReceiverConfig.java`
| `oap-libs-for-graalvm/envoy-metrics-receiver-for-graalvm/` | Added `@Setter`
at class level. |
+| `OtelMetricReceiverConfig` |
`server-receiver-plugin/otel-receiver-plugin/.../OtelMetricReceiverConfig.java`
| `oap-libs-for-graalvm/otel-receiver-for-graalvm/` | Added `@Setter` at class
level. |
+| `EBPFReceiverModuleConfig` |
`server-receiver-plugin/skywalking-ebpf-receiver-plugin/.../EBPFReceiverModuleConfig.java`
| `oap-libs-for-graalvm/ebpf-receiver-for-graalvm/` | Added `@Setter` at class
level. |
+| `AWSFirehoseReceiverModuleConfig` |
`server-receiver-plugin/aws-firehose-receiver/.../AWSFirehoseReceiverModuleConfig.java`
| `oap-libs-for-graalvm/aws-firehose-receiver-for-graalvm/` | Added `@Setter`
at class level. |
+| `CiliumFetcherConfig` |
`server-fetcher-plugin/cilium-fetcher-plugin/.../CiliumFetcherConfig.java` |
`oap-libs-for-graalvm/cilium-fetcher-for-graalvm/` | Added `@Setter` at class
level. |
+| `StatusQueryConfig` |
`server-query-plugin/status-query-plugin/.../StatusQueryConfig.java` |
`oap-libs-for-graalvm/status-query-for-graalvm/` | Added `@Setter` at class
level. |
+| `HealthCheckerConfig` | `server-health-checker/.../HealthCheckerConfig.java`
| `oap-libs-for-graalvm/health-checker-for-graalvm/` | Added `@Setter` at class
level. |
+
+Config replacements (except `YamlConfigLoaderUtils`) are repackaged into their
respective `-for-graalvm` modules via `maven-shade-plugin`.
`YamlConfigLoaderUtils` lives in `oap-graalvm-server` because it imports types
from 30+ modules; the original `.class` is excluded from
`library-util-for-graalvm` via shade filter.
+
+## Same-FQCN Packaging
+
+The original `YamlConfigLoaderUtils.class` is excluded from the
`library-util-for-graalvm` shaded JAR. The replacement in
`oap-graalvm-server.jar` is the only copy on the classpath.
+
+---
+
+## Verification
+
+```bash
+# Run generator
+JAVA_HOME=/Users/wusheng/.sdkman/candidates/java/25-graal \
+ mvn -pl build-tools/config-generator exec:java \
+
-Dexec.args="oap-graalvm-server/src/main/java/org/apache/skywalking/oap/server/library/util/YamlConfigLoaderUtils.java"
+
+# Full build (compile + test + package)
+JAVA_HOME=/Users/wusheng/.sdkman/candidates/java/25-graal make build-distro
+```
diff --git a/docs/distro-policy.md b/docs/distro-policy.md
new file mode 100644
index 0000000..eefadca
--- /dev/null
+++ b/docs/distro-policy.md
@@ -0,0 +1,322 @@
+# SkyWalking GraalVM Distro - Distribution Policy
+
+## Goal
+Build and package Apache SkyWalking OAP server as a GraalVM native image on
JDK 25.
+
+## Architecture Constraints
+- **Submodule**: `skywalking/` is a git submodule of `apache/skywalking.git`.
All SkyWalking source changes go through upstream PRs. **Minimize upstream
changes.**
+- **This repo**: Maven + Makefile to orchestrate building on top of the
submodule. Pre-compilation, GraalVM config, native-image wiring, and the fixed
module manager live here.
+- **JDK 25**: Already compiles and runs. Not an issue.
+
+## Module Selection (Fixed at Build Time)
+
+| Category | Module | Provider |
+|----------|--------|----------|
+| **Core** | CoreModule | default |
+| **Storage** | StorageModule | BanyanDB |
+| **Cluster** | ClusterModule | Standalone, Kubernetes |
+| **Configuration** | ConfigurationModule | Kubernetes |
+| **Receivers** | SharingServerModule, TraceModule, JVMModule,
MeterReceiverModule, LogModule, RegisterModule, ProfileModule, BrowserModule,
EventModule, OtelMetricReceiverModule, MeshReceiverModule,
EnvoyMetricReceiverModule, ZipkinReceiverModule, ZabbixReceiverModule,
TelegrafReceiverModule, AWSFirehoseReceiverModule, CiliumFetcherModule,
EBPFReceiverModule, AsyncProfilerModule, PprofModule, CLRModule,
ConfigurationDiscoveryModule, KafkaFetcherModule | default providers |
+| **Analyzers** | AnalyzerModule, LogAnalyzerModule, EventAnalyzerModule |
default providers |
+| **Query** | QueryModule (GraphQL), PromQLModule, LogQLModule,
ZipkinQueryModule, StatusQueryModule | default providers |
+| **Alarm** | AlarmModule | default |
+| **Telemetry** | TelemetryModule | Prometheus |
+| **Other** | ExporterModule, HealthCheckerModule, AIPipelineModule | default
providers |
+
+**Full feature set.** Work around issues as they arise.
+
+---
+
+## Core Strategy
+
+1. **Build-Time Class Export**: All runtime code generation (OAL via
Javassist, MAL/LAL via Groovy) runs at build time. Export `.class` files and
package into native-image classpath. Classpath scanning also runs here as a
verification gate.
+
+2. **Fixed Module Wiring**: Module/provider selection is hardcoded in this
distro (no SPI discovery). Simplified config file for selected providers only.
+
+3. **Separation**: SkyWalking upstream changes tracked separately, go through
upstream PRs.
+
+---
+
+## OAL Runtime Class Generation (Javassist)
+
+### What Happens
+OAL V2 generates metrics/builder/dispatcher classes at startup via Javassist
(`ClassPool.makeClass()` → `CtClass.toClass()`). Already has
`writeGeneratedFile()` for debug export.
+
+### Approach (this repo)
+All `.oal` scripts are known. Run OAL engine at build time, export `.class`
files, load them directly at runtime from manifests.
+
+**Details**: [oal-immigration.md](oal-immigration.md)
+
+### What Was Built
+- `OALClassExporter` processes all 9 OAL defines, exports ~620 metrics
classes, ~620 builder classes, ~45 dispatchers
+- 3 manifest files: `oal-metrics-classes.txt`, `oal-dispatcher-classes.txt`,
`oal-disabled-sources.txt`
+- Same-FQCN replacement `OALEngineLoaderService` loads pre-compiled classes
from manifests instead of running Javassist
+
+### Upstream Changes Needed
+- None. Build-time class export works via existing debug API
(`setOpenEngineDebug(true)` + `setGeneratedFilePath()`)
+
+---
+
+## MAL and LAL (Groovy + Javassist)
+
+### What Happens
+- MAL uses `GroovyShell` + `DelegatingScript` for meter rule expressions
(~1250 rules across 71 YAML files). Also, `MeterSystem.create()` uses Javassist
to dynamically generate one meter subclass per metric rule.
+- LAL uses `GroovyShell` + `@CompileStatic` + `LALPrecompiledExtension` for
log analysis scripts (10 rules).
+
+### Approach (this repo)
+Run full MAL/LAL initialization at build time via `build-tools/precompiler`
(unified tool). Export Javassist-generated `.class` files. Transpile all Groovy
expressions to pure Java at build time — zero Groovy at runtime.
+
+**Details**: [mal-immigration.md](mal-immigration.md) |
[lal-immigration.md](lal-immigration.md)
+
+### What Was Built
+- **Unified precompiler** (`build-tools/precompiler`): Replaced separate
`oal-exporter` and `mal-compiler` modules. Compiles all 71 MAL YAML rule files
(meter-analyzer-config, otel-rules, log-mal-rules, envoy-metrics-rules,
telegraf-rules, zabbix-rules) producing 1209 meter classes.
+- **MAL-to-Java transpiler**: 1250+ MAL expressions transpiled from Groovy AST
to pure Java `MalExpression` implementations. 29 filter expressions transpiled
to `MalFilter` implementations. Zero Groovy at runtime.
+- **LAL-to-Java transpiler**: 10 LAL scripts (6 unique) transpiled to pure
Java `LalExpression` implementations. Spec classes enhanced with `Consumer`
overloads for transpiled code.
+- **Groovy stubs module**: Minimal `groovy.lang.*` types (Binding, Closure,
etc.) for class loading. No `org.codehaus.groovy.*` — prevents GraalVM
`GroovyIndyInterfaceFeature` from activating.
+- **Manifests**: `META-INF/mal-expressions.txt` (transpiled Java classes),
`META-INF/mal-groovy-expression-hashes.txt` (SHA-256 for combination pattern
resolution), `META-INF/mal-meter-classes.txt` (Javassist-generated classes),
`META-INF/lal-expressions.txt` (transpiled LAL classes),
`META-INF/annotation-scan/MeterFunction.txt` (16 function classes).
+- **Combination pattern**: Multiple YAML files from different data sources
(otel, telegraf, zabbix) may define metrics with the same name. Deterministic
suffixes (`_1`, `_2`) with expression hash tracking enable unambiguous
resolution.
+- **Same-FQCN replacements**: `DSL.java` (MAL), `DSL.java` (LAL),
`FilterExpression.java`, `MeterSystem.java`, `Expression.java`,
`SampleFamily.java` — all use pure Java, no Groovy.
+- **Comparison test suite**: 73 MAL test classes (1281 assertions) + 5 LAL
test classes (19 assertions) covering all 79 YAML files. Tests require data
flow through full pipeline (no vacuous agreements). Dual-path: fresh Groovy
compilation (Path A) vs transpiled Java (Path B).
+
+### Groovy Elimination
+- MAL: `MalExpression` interface replaces `DelegatingScript`. `SampleFamily`
uses Java functional interfaces (`TagFunction`, `SampleFilter`,
`ForEachFunction`, `DecorateFunction`, `PropertiesExtractor`) instead of
`groovy.lang.Closure`.
+- LAL: `LalExpression` interface replaces `DelegatingScript`. Spec classes
have `Consumer` overloads.
+- No `groovy.lang.Closure` in any production source code. Groovy is test-only
dependency.
+
+---
+
+## Classpath Scanning (Guava ClassPath)
+
+### What Happens
+`ClassPath.from()` used in `SourceReceiverImpl.scan()`, `AnnotationScan`,
`MeterSystem`, `DefaultMetricsFunctionRegistry`, `FilterMatchers`,
`MetricsHolder`.
+
+### What Was Solved
+- `AnnotationScan` and `SourceReceiverImpl` replaced with same-FQCN classes
that read from build-time manifests. 6 annotation/interface manifests under
`META-INF/annotation-scan/`: `ScopeDeclaration`, `Stream`, `Disable`,
`MultipleDisable`, `SourceDispatcher`, `ISourceDecorator`.
+- `DefaultMetricsFunctionRegistry`, `FilterMatchers`, `MetricsHolder` — these
only run inside the OAL engine at build time, not at runtime. Automatically
solved.
+- `MeterSystem` replaced with same-FQCN class that reads from
`META-INF/annotation-scan/MeterFunction.txt` manifest (16 meter function
classes). Solved as part of MAL immigration.
+
+---
+
+## Module System & Configuration
+
+### Current Behavior
+`ModuleManager` uses `ServiceLoader` (SPI). `application.yml` selects
providers. Config loaded via reflection (`Field.setAccessible` + `field.set` in
`YamlConfigLoaderUtils.copyProperties`).
+
+### Approach (this repo)
+1. **New module manager**: Directly constructs chosen
`ModuleDefine`/`ModuleProvider` — no SPI
+2. **Simplified config file**: Only knobs for selected providers
+3. **Config loading**: **No reflection.** Build-time tool scans all
`ModuleConfig` subclass fields → generates same-FQCN replacement of
`YamlConfigLoaderUtils` that uses Lombok setters and VarHandle to set config
fields directly. Eliminates `Field.setAccessible`/`field.set` and the need for
`reflect-config.json` for config classes.
+
+**Details**: [config-init-immigration.md](config-init-immigration.md)
+
+### What Was Built
+- `FixedModuleManager` — direct module/provider construction via
`ModuleDefine.prepare()` overload, no SPI
+- `GraalVMOAPServerStartUp` — entry point with `configuration.has()` guards
for 6 optional modules
+- `application.yml` — simplified config for selected providers
+- `ConfigInitializerGenerator` — build-time tool that scans config classes and
generates `YamlConfigLoaderUtils` replacement
+- `YamlConfigLoaderUtils` — same-FQCN replacement using type-dispatch +
setter/VarHandle instead of reflection
+- `ModuleDefine` — same-FQCN replacement (`library-module-for-graalvm`) adding
`prepare(ModuleManager, ModuleProvider, ...)` overload for direct provider
wiring without ServiceLoader
+
+---
+
+## Same-FQCN Packaging (Repackaged Modules)
+
+### Problem
+
+Same-FQCN replacement classes need to shadow upstream originals. Classpath
ordering tricks confuse developers and AI tools.
+
+### Solution: Per-JAR Repackaged Modules (`oap-libs-for-graalvm`)
+
+Each upstream JAR that has replacement classes gets a corresponding
`*-for-graalvm` module under `oap-libs-for-graalvm/`. The module uses
`maven-shade-plugin` to:
+1. Include only the upstream JAR in the shade
+2. Exclude the specific `.class` files being replaced
+3. Produce a JAR containing: all upstream classes MINUS replaced ones PLUS our
replacements
+
+`oap-graalvm-server` depends on `*-for-graalvm` JARs instead of originals.
Original upstream JARs are forced to `provided` scope via
`<dependencyManagement>` to prevent transitive leakage.
+
+### 23 Same-FQCN Replacement Classes Across 13 Modules
+
+**Non-trivial replacements (load pre-compiled assets from manifests):**
+
+| Module | Replacement Classes | Purpose |
+|---|---|---|
+| `library-module-for-graalvm` | `ModuleDefine` | Add `prepare()` overload for
direct provider wiring (bypasses ServiceLoader) |
+| `server-core-for-graalvm` | `OALEngineLoaderService`, `AnnotationScan`,
`SourceReceiverImpl`, `MeterSystem`, `CoreModuleConfig`,
`HierarchyDefinitionService` | Load from manifests instead of
Javassist/ClassPath; config with @Setter; Java-backed closures instead of
GroovyShell |
+| `library-util-for-graalvm` | `YamlConfigLoaderUtils` | Set config fields via
setter instead of reflection |
+| `meter-analyzer-for-graalvm` | `DSL`, `FilterExpression`, `Rules` | Load
pre-compiled MAL Groovy scripts from manifest; load rule data from JSON
config-data manifests |
+| `log-analyzer-for-graalvm` | `DSL`, `LogAnalyzerModuleConfig`, `LALConfigs`
| Load pre-compiled LAL scripts; config with @Setter; load LAL config data from
JSON config-data manifests |
+| `agent-analyzer-for-graalvm` | `AnalyzerModuleConfig`, `MeterConfigs` |
Config with @Setter; load meter config data from JSON config-data manifests |
+
+**Config-only replacements (add `@Setter` for reflection-free config):**
+
+| Module | Replacement Class |
+|---|---|
+| `envoy-metrics-receiver-for-graalvm` | `EnvoyMetricReceiverConfig` |
+| `otel-receiver-for-graalvm` | `OtelMetricReceiverConfig` |
+| `ebpf-receiver-for-graalvm` | `EBPFReceiverModuleConfig` |
+| `aws-firehose-receiver-for-graalvm` | `AWSFirehoseReceiverModuleConfig` |
+| `cilium-fetcher-for-graalvm` | `CiliumFetcherConfig` |
+| `status-query-for-graalvm` | `StatusQueryConfig` |
+| `health-checker-for-graalvm` | `HealthCheckerConfig` |
+
+### No Classpath Ordering Required
+
+No duplicate FQCNs on the classpath. The startup script (`oapService.sh`) uses
a simple flat classpath. The `oap-graalvm-native` uber JAR also has no FQCN
conflicts.
+
+### Adding New Replacements
+
+To add a new same-FQCN replacement:
+1. Create a new `*-for-graalvm` module under `oap-libs-for-graalvm/` (or add
to existing one)
+2. Add the replacement `.java` file with the same FQCN
+3. Configure shade plugin to exclude the original `.class` from the upstream
JAR
+4. Add the `-for-graalvm` artifact to root `pom.xml` `<dependencyManagement>`
+5. In `oap-graalvm-server/pom.xml`: add the original JAR to
`<dependencyManagement>` as `provided`, add `-for-graalvm` to `<dependencies>`
+6. Add the original JAR to `distribution.xml` `<excludes>`
+
+---
+
+## Additional GraalVM Risks
+
+| Risk | Status | Mitigation |
+|------|--------|------------|
+| **Reflection** (annotations, OAL enricher, HTTP handlers, GraphQL types) |
SOLVED | Auto-generated by precompiler from manifests;
`log4j2-reflect-config.json` for Log4j2 plugins |
+| **gRPC / Netty / Armeria** | SOLVED | GraalVM reachability metadata repo
handles these automatically |
+| **Resource loading** (`ResourceUtils`, config files) | SOLVED |
`resource-config.json` via tracing agent |
+| **Log4j2** | SOLVED | Console-only `log4j2.xml` avoids RollingFile
reflection chain; Log4j2 plugin classes in `log4j2-reflect-config.json` |
+| **Kafka client** (for Kafka fetcher) | Untested | Known GraalVM support, may
need config |
+| **Kubernetes client 6.7.1** (for cluster + config) | Untested | Has GraalVM
support, may need config at runtime |
+
+---
+
+## Distro Resource Files
+
+Upstream `server-starter/src/main/resources/` contains 236 files. They fall
into
+two categories: files included directly in the distro `config/` directory
(loaded
+at runtime via file I/O), and files consumed by the precompiler at build time
+(not needed at runtime — their logic is baked into pre-compiled `.class`
files).
+
+### Directly Included in Distro (`config/`)
+
+These files are loaded at runtime via `ResourceUtils.read()`, `Files.walk()`,
or
+YAML parsing. No reflection involved — safe for GraalVM native image as-is.
+
+| File / Directory | Count | Loaded By | Purpose |
+|---|---|---|---|
+| `application.yml` | 1 | Custom (distro's own, not upstream) |
Module/provider config |
+| `bydb.yml` | 1 | `BanyanDBConfigLoader` | BanyanDB storage base config |
+| `bydb-topn.yml` | 1 | `BanyanDBConfigLoader` | BanyanDB TopN aggregation
config |
+| `log4j2.xml` | 1 | Log4j2 framework | Logging configuration |
+| `alarm-settings.yml` | 1 | `AlarmModuleProvider` via `ResourceUtils.read()`
| Alarm rules |
+| `component-libraries.yml` | 1 | `ComponentLibraryCatalogService` via
`ResourceUtils.read()` | Component ID mapping |
+| `endpoint-name-grouping.yml` | 1 | `EndpointNameGroupingRuleWatcher` via
`ResourceUtils.read()` | Endpoint grouping rules |
+| `gateways.yml` | 1 | `UninstrumentedGatewaysConfig` via
`ResourceUtils.read()` | Gateway definitions |
+| `hierarchy-definition.yml` | 1 | `HierarchyDefinitionService` via
`ResourceUtils.read()` | Layer hierarchy |
+| `metadata-service-mapping.yaml` | 1 | `ResourceUtils.read()` | Metadata
service mapping |
+| `service-apdex-threshold.yml` | 1 | `ApdexThresholdConfig` via
`ResourceUtils.read()` | APDEX thresholds |
+| `trace-sampling-policy-settings.yml` | 1 | `TraceSamplingPolicyWatcher` via
`ResourceUtils.read()` | Trace sampling |
+| `ui-initialized-templates/**` | 131 | `UITemplateInitializer` via
`Files.walk()` | UI dashboard JSON templates |
+| `cilium-rules/**` | 2 | `CiliumFetcherProvider` via
`ResourceUtils.getPathFiles()` | Cilium flow rules |
+| `openapi-definitions/**` | 1 | `EndpointNameGrouping` via
`ResourceUtils.getPathFiles()` | OpenAPI grouping definitions |
+
+**Total: 146 files** included in the distro `config/` directory.
+
+### Pre-compiled at Build Time (NOT in distro)
+
+These files are consumed by `build-tools/precompiler` during the build. Their
+expressions, scripts, and metric definitions are compiled into `.class` files
+packaged in JARs. The YAML source files are not needed at runtime.
+
+| Category | Count | Pre-compiled Into | Tool |
+|---|---|---|---|
+| `oal/*.oal` | 9 | ~620 metrics classes + ~620 builders + ~45 dispatchers
(Javassist) | `OALClassExporter` |
+| `meter-analyzer-config/*.yaml` | 11 | 147 Groovy scripts + Javassist meter
classes | `MALPrecompiler` |
+| `otel-rules/**/*.yaml` | 55 | 1044 Groovy scripts + Javassist meter classes
| `MALPrecompiler` |
+| `log-mal-rules/*.yaml` | 2 | 2 Groovy scripts | `MALPrecompiler` |
+| `envoy-metrics-rules/*.yaml` | 2 | 26 Groovy scripts + Javassist meter
classes | `MALPrecompiler` |
+| `telegraf-rules/*.yaml` | 1 | 20 Groovy scripts + Javassist meter classes |
`MALPrecompiler` |
+| `zabbix-rules/*.yaml` | 1 | 15 Groovy scripts + Javassist meter classes |
`MALPrecompiler` |
+| `lal/*.yaml` | 8 | 6 unique `@CompileStatic` Groovy classes |
`LALPrecompiler` |
+
+**Total: 89 files** consumed at build time, producing ~1285 pre-compiled
classes
+and ~1254 Groovy scripts stored in JARs.
+
+Additionally, the precompiler serializes parsed config POJOs as JSON manifests
in
+`META-INF/config-data/` (7 JSON files for meter-analyzer-config, otel-rules,
+envoy-metrics-rules, log-mal-rules, telegraf-rules, zabbix-rules, and lal).
These
+provide the runtime "wiring" data (metric prefixes, rule names, expression
lookup
+keys) that replacement loader classes use instead of filesystem YAML access.
+
+### Not Included (upstream-only)
+
+| File | Reason |
+|---|---|
+| `application.yml` (upstream) | Replaced by distro's own simplified
`application.yml` |
+
+---
+
+## Build Workflow
+
+### Build System
+- Maven + Makefile orchestrates building on top of the skywalking submodule
+- GraalVM JDK 25 in CI (`.github/workflows/ci.yml`)
+- JVM-mode starter with fixed module wiring (`FixedModuleManager` +
`GraalVMOAPServerStartUp`)
+- Simplified config file for selected modules (`application.yml`)
+
+### Build-Time Pre-Compilation
+
+**OAL**: OAL engine exports `.class` files (9 defines, ~620 metrics, ~620
builders, ~45 dispatchers). 7 annotation/interface manifests. 3 same-FQCN
replacement classes (`OALEngineLoaderService`, `AnnotationScan`,
`SourceReceiverImpl`).
+
+**MAL**: Unified precompiler (`build-tools/precompiler`) processes 71 YAML
files → 1250 expressions transpiled to pure Java `MalExpression` + 1209
Javassist meter classes. Combination pattern with deterministic suffixes +
expression hash tracking. Same-FQCN replacements: `DSL.java`,
`FilterExpression.java`, `MeterSystem.java`, `Expression.java`,
`SampleFamily.java`. 73 comparison test classes, 1281 assertions (100% YAML
coverage).
+
+**LAL**: 8 YAML files → 10 rules → 6 unique transpiled Java `LalExpression`
classes. Same-FQCN `DSL.java` loads via SHA-256 hash lookup. 5 comparison test
classes, 19 assertions (100% branch coverage).
+
+**Config initialization**: `ConfigInitializerGenerator` generates same-FQCN
`YamlConfigLoaderUtils` using Lombok setters — zero `Field.setAccessible` at
runtime.
+
+**Config data serialization**: Precompiler serializes parsed config POJOs to
`META-INF/config-data/*.json` (7 JSON files). 3 same-FQCN replacement loaders
(`MeterConfigs`, `Rules`, `LALConfigs`) deserialize from JSON instead of
filesystem YAML.
+
+**Module system**: `ModuleDefine` replacement with direct `prepare()` overload
(bypasses ServiceLoader). `GraalVMOAPServerStartUp` with `configuration.has()`
guards for 6 optional modules.
+
+**Distro resource packaging**: 146 runtime files → distro `config/`, 89
pre-compiled files → JARs. Assembly descriptor (`distribution.xml`) packages
runtime config files from upstream.
+
+### Groovy Elimination
+
+- MAL-to-Java transpiler: 1250+ expressions → pure Java `MalExpression` (no
Groovy MOP/ExpandoMetaClass)
+- LAL-to-Java transpiler: 10 scripts → pure Java `LalExpression` (no
DelegatingScript)
+- `SampleFamily` Closure parameters → Java functional interfaces (zero
`groovy.lang.Closure` in production)
+- Groovy stubs module for class loading (no `org.codehaus.groovy.*`)
+- HierarchyDefinitionService: same-FQCN replacement with Java-backed closures
+- Real Groovy (`groovy-5.0.3.jar`) is test-only;
`groovy-stubs-1.0.0-SNAPSHOT.jar` on runtime classpath
+- 1303 tests require actual data flow (no vacuous empty-result agreements)
+
+### Native Image Build
+
+- `native-maven-plugin` (GraalVM buildtools 0.10.4) in `oap-graalvm-native`
with `-Pnative` profile
+- `reflect-config.json` auto-generated by precompiler from manifests (OAL,
MAL, LAL, meter, HTTP handlers, GraphQL types)
+- `log4j2-reflect-config.json` for Log4j2 plugin classes; console-only
`log4j2.xml` with `SW_LOG_LEVEL` env var
+- gRPC/Netty/Protobuf/Armeria via GraalVM reachability metadata repository
+- Auto-scanned reflection metadata: Armeria HTTP handlers (~19), GraphQL
resolvers (~32), GraphQL types (~182), config POJOs (8)
+- Native binary: ~203MB, boots to full module init with all HTTP endpoints
functional
+
+### Native Distro Packaging
+
+- Assembly descriptor (`native-distribution.xml`) packages native binary +
config files
+- `Dockerfile.native` packages native distro into `debian:bookworm-slim`
+- `docker-compose.yml` with BanyanDB + OAP native services
+- CI pipeline: multi-arch native build (amd64 + arm64) with Docker manifest
push to GHCR
+
+### Remaining Verification
+- Verify all receiver plugins work (gRPC + HTTP endpoints)
+- Verify all query APIs work (GraphQL, PromQL, LogQL, Zipkin)
+- Verify cluster mode (K8s)
+- Verify alarm module
+- Performance benchmarking vs JVM
+
+---
+
+## Upstream Changes Tracker
+
+No upstream changes needed. All GraalVM incompatibilities are resolved in this
distro via same-FQCN replacement and build-time pre-compilation:
+- OAL: build-time class export works via existing debug API
+- MAL: transpiled to pure Java, bypasses Groovy entirely
+- LAL: transpiled to pure Java, bypasses Groovy entirely
+- Dynamic Groovy MOP: transpiled to pure Java, no ExpandoMetaClass/MOP at
runtime
diff --git a/docs/docker-image.md b/docs/docker-image.md
new file mode 100644
index 0000000..ecb75b4
--- /dev/null
+++ b/docs/docker-image.md
@@ -0,0 +1,108 @@
+# Pre-Built Docker Images
+
+Pre-built multi-architecture Docker images are published to **GitHub Container
Registry (GHCR)**
+on every push to the `main` branch via CI.
+
+## Image Repository
+
+```
+ghcr.io/apache/skywalking-graalvm-distro
+```
+
+GitHub Packages page:
[https://github.com/apache/skywalking-graalvm-distro/pkgs/container/skywalking-graalvm-distro](https://github.com/apache/skywalking-graalvm-distro/pkgs/container/skywalking-graalvm-distro)
+
+## Tags
+
+| Tag | Description |
+|-----|-------------|
+| `latest` | Most recent build from the `main` branch |
+| `<short-sha>` | Pinned to a specific commit (first 7 characters of the
commit SHA) |
+
+## Supported Architectures
+
+Each tag is a multi-architecture manifest supporting:
+
+- `linux/amd64`
+- `linux/arm64`
+
+Docker automatically selects the correct architecture for your platform.
+
+## Pull the Image
+
+```bash
+docker pull ghcr.io/apache/skywalking-graalvm-distro:latest
+```
+
+Or pin to a specific commit:
+
+```bash
+docker pull ghcr.io/apache/skywalking-graalvm-distro:<short-sha>
+```
+
+## Run with Docker
+
+### Standalone with BanyanDB
+
+The simplest way is to use the included `docker-compose.yml`, replacing the
locally built image
+with the pre-built one:
+
+```bash
+# Clone the repo (for docker-compose.yml and config files only)
+git clone https://github.com/apache/skywalking-graalvm-distro.git
+cd skywalking-graalvm-distro
+
+# Run with pre-built image
+docker compose -f docker/docker-compose.yml up
+```
+
+Edit `docker/docker-compose.yml` to use the GHCR image instead of the local
build:
+
+```yaml
+services:
+ oap:
+ image: ghcr.io/apache/skywalking-graalvm-distro:latest
+ # ... rest of config unchanged
+```
+
+### Manual Docker Run
+
+```bash
+docker run -d \
+ -p 12800:12800 \
+ -p 11800:11800 \
+ -e SW_STORAGE_BANYANDB_TARGETS=<banyandb-host>:17912 \
+ -e SW_CLUSTER=standalone \
+ -e SW_CONFIGURATION=none \
+ ghcr.io/apache/skywalking-graalvm-distro:latest
+```
+
+## Environment Variables
+
+All settings from [Configuration](configuration.md) are configurable via
environment variables.
+Common ones for Docker deployment:
+
+| Variable | Description | Default |
+|----------|-------------|---------|
+| `SW_STORAGE_BANYANDB_TARGETS` | BanyanDB server address | `127.0.0.1:17912` |
+| `SW_CLUSTER` | Cluster mode (`standalone` or `kubernetes`) | `standalone` |
+| `SW_CONFIGURATION` | Dynamic config provider (`none` or `k8s-configmap`) |
`k8s-configmap` |
+| `SW_CORE_REST_PORT` | REST API port | `12800` |
+| `SW_CORE_GRPC_PORT` | gRPC port | `11800` |
+| `SW_TELEMETRY_PROMETHEUS_PORT` | Prometheus metrics port | `1234` |
+| `SW_LOG_LEVEL` | Log4j2 log level | `INFO` |
+| `JAVA_OPTS` | Additional flags passed to the native binary (e.g.,
`-Dmode=init`) | *(empty)* |
+
+## Exposed Ports
+
+| Port | Protocol | Service |
+|------|----------|---------|
+| 12800 | HTTP | REST API — GraphQL query, health check, HTTP data receivers |
+| 11800 | gRPC | Agent data collection — traces, metrics, logs, profiling |
+| 1234 | HTTP | Prometheus self-telemetry metrics |
+
+## Image Details
+
+- **Base image**: `debian:bookworm-slim`
+- **Contents**: Native binary (`oap-server`, ~203MB) + `config/` directory
+- **Entrypoint**: `./oap-server` with `JAVA_OPTS` forwarded as command-line
arguments
+- **No JVM**: The image does not contain a JVM — the binary is a
self-contained GraalVM native image
diff --git a/docs/lal-immigration.md b/docs/lal-immigration.md
new file mode 100644
index 0000000..d22704f
--- /dev/null
+++ b/docs/lal-immigration.md
@@ -0,0 +1,352 @@
+# LAL Immigration: Build-Time Pre-Compilation + Groovy Elimination
+
+## Context
+
+LAL (Log Analysis Language) uses Groovy with `@CompileStatic` +
`LALPrecompiledExtension` for log analysis scripts. At startup,
`GroovyShell.parse()` compiles each LAL DSL script into a
`LALDelegatingScript`. GraalVM native image cannot compile Groovy at runtime.
+
+LAL also enforces security constraints via `SecureASTCustomizer` — `while`,
`do-while`, and `for` loops are disallowed. Branch coverage focuses on
`if`/`else if`/`else` chains.
+
+**Solution**: Compile all LAL scripts at build time via the unified
precompiler. Export `.class` files + manifests. At runtime, load pre-compiled
classes via SHA-256 hash lookup — no Groovy compilation.
+
+---
+
+## Rule File Inventory
+
+**8 LAL YAML files, 10 rules, 6 unique pre-compiled classes:**
+
+| File | Rule Name | DSL Features |
+|---|---|---|
+| `default.yaml` | default | Empty sink (trivial passthrough) |
+| `nginx.yaml` | nginx-access-log | `tag()` guard, `text { regexp }`,
conditional tag extraction |
+| `nginx.yaml` | nginx-error-log | `tag()` guard, `text { regexp }`, timestamp
with format, `metrics {}` |
+| `mysql-slowsql.yaml` | mysql-slowsql | `json {}`, conditional `slowSql {}` |
+| `pgsql-slowsql.yaml` | pgsql-slowsql | Identical DSL to mysql-slowsql |
+| `redis-slowsql.yaml` | redis-slowsql | Identical DSL to mysql-slowsql |
+| `envoy-als.yaml` | envoy-als | `parsed?.` navigation, conditional `abort
{}`, tag extraction, `rateLimit` sampler |
+| `envoy-als.yaml` | network-profiling-slow-trace | `json {}`, `tag()` guard,
`sampledTrace {}` with 3-way if/else chains |
+| `mesh-dp.yaml` | network-profiling-slow-trace | Identical DSL to envoy-als's
2nd rule |
+| `k8s-service.yaml` | network-profiling-slow-trace | Identical DSL to
envoy-als's 2nd rule |
+
+**SHA-256 deduplication**: mysql/pgsql/redis share identical DSL (1 class).
The 3 network-profiling rules share identical DSL (1 class). Total unique
pre-compiled classes: **6**.
+
+---
+
+## Build-Time Compilation
+
+The unified precompiler (`build-tools/precompiler`) handles LAL alongside OAL
and MAL:
+
+1. Loads all 8 LAL YAML files via `LALConfigs.load()`
+2. For each rule's DSL string, compiles with the same `CompilerConfiguration`
as upstream:
+ - `@CompileStatic` with `LALPrecompiledExtension` for type checking
+ - `SecureASTCustomizer` disallowing loops
+ - `ImportCustomizer` for `ProcessRegistry`
+ - Script base class: `LALDelegatingScript`
+3. Exports compiled `.class` files to the output directory
+4. Writes three manifest files:
+
+**`META-INF/lal-expressions.txt`** — SHA-256 hash → transpiled Java class
(used at runtime):
+```
+a1b2c3d4...=org.apache.skywalking.oap.server.core.source.oal.rt.lal.LalExpr_0
+e5f6a7b8...=org.apache.skywalking.oap.server.core.source.oal.rt.lal.LalExpr_1
+...
+```
+
+**`META-INF/lal-scripts-by-hash.txt`** — SHA-256 hash → Groovy class
(build-time artifact):
+```
+a1b2c3d4...=network_profiling_slow_trace
+...
+```
+
+**`META-INF/lal-scripts.txt`** — rule name → Groovy class (build-time
artifact):
+```
+default=default
+nginx-access-log=nginx_access_log
+...
+```
+
+The `lal-expressions.txt` manifest is the runtime manifest for the transpiled
Java expressions.
+The other two manifests are build-time artifacts for verification and
debugging.
+
+---
+
+## Runtime Replacement
+
+**Same-FQCN class**: `oap-graalvm-server/.../log/analyzer/dsl/DSL.java`
+
+Same FQCN as upstream `org.apache.skywalking.oap.log.analyzer.dsl.DSL`. The
`of()` method:
+
+1. Computes SHA-256 of the DSL string
+2. Loads `META-INF/lal-scripts-by-hash.txt` manifest (lazy, thread-safe,
cached)
+3. Looks up the pre-compiled class name by hash
+4. `Class.forName(className)` → `newInstance()` → cast to `DelegatingScript`
+5. Creates `FilterSpec`, sets delegate, returns `DSL` instance
+
+No `GroovyShell`, no compilation. The pre-compiled class already contains the
statically-compiled bytecode with all type checking baked in.
+
+---
+
+## Key Difference from MAL
+
+| Aspect | MAL | LAL |
+|---|---|---|
+| Groovy mode | Dynamic (MOP, `propertyMissing`, `ExpandoMetaClass`) |
`@CompileStatic` with extension |
+| Loop support | No restriction | Loops disallowed (`SecureASTCustomizer`) |
+| Script base class | `DelegatingScript` | `LALDelegatingScript` |
+| Manifest lookup | By metric name | By SHA-256 hash of DSL content |
+| GraalVM risk | High (dynamic Groovy MOP) | Low (statically compiled) |
+
+LAL's `@CompileStatic` compilation means the pre-compiled classes are fully
statically typed — no runtime metaclass manipulation needed. This makes LAL
significantly more native-image-friendly than MAL.
+
+---
+
+## Comparison Test Suite
+
+**5 test classes, 19 assertions** covering all 8 YAML files and 10 rules.
+
+Each test runs both paths and asserts identical `Binding` state:
+
+```
+ LAL YAML file
+ |
+ Load rules (name, dsl)
+ |
+ For each rule's DSL:
+ / \
+ Path A (Fresh Groovy) Path B (Pre-compiled)
+ GroovyShell.parse() SHA-256 → manifest lookup
+ same CompilerConfig Class.forName() → newInstance()
+ \ /
+ Create FilterSpec (mocked ModuleManager)
+ script.setDelegate(filterSpec)
+ filterSpec.bind(binding)
+ script.run()
+ \ /
+ Assert identical Binding state
+```
+
+**What is compared after evaluation:**
+- `binding.shouldAbort()` — did `abort {}` fire?
+- `binding.shouldSave()` — log persistence flag
+- `binding.log()` — LogData.Builder state (service, serviceInstance, endpoint,
layer, timestamp, tags)
+- `binding.metricsContainer()` — SampleFamily objects (for nginx-error-log
`metrics {}`)
+- `binding.databaseSlowStatement()` — builder state (for slowSql rules)
+- `binding.sampledTraceBuilder()` — builder state (for network-profiling
sampledTrace)
+
+### Test Classes
+
+| Test Class | YAML File(s) | Tests | Coverage |
+|---|---|---|---|
+| `LALDefaultTest` | default.yaml | 2 | Trivial passthrough + manifest
verification |
+| `LALNginxTest` | nginx.yaml | 5 | Access log: matching/non-matching tag +
non-matching regex. Error log: matching/non-matching tag |
+| `LALSlowSqlTest` | mysql/pgsql/redis-slowsql.yaml | 3 | SLOW_SQL tag guard
(match/skip) + 3-file manifest verification |
+| `LALEnvoyAlsTest` | envoy-als.yaml (1st rule) | 3 | Abort path (low code, no
flags), non-abort with flags, non-abort with high code |
+| `LALNetworkProfilingTest` | envoy-als/mesh-dp/k8s-service.yaml | 6 | 4
componentId branches (http/tcp x ssl/no-ssl), LOG_KIND guard, 3-file manifest
verification |
+
+### Branch Coverage
+
+- **`tag()` guard**: All rules with `if (tag("LOG_KIND") == ...)` tested with
matching and non-matching values
+- **`abort {}`**: envoy-als tested with conditions that trigger and skip abort
+- **`slowSql {}`**: Tested with SLOW_SQL tag match (block executed) and
non-match (block skipped)
+- **`sampledTrace {}`**: componentId 4-way if/else chain fully covered
(HTTP=49, HTTPS=129, TLS=130, TCP=110)
+- **`text { regexp }`**: nginx access log tested with matching and
non-matching text patterns
+- **`json {}`**: Tested via slowSql and network-profiling rules
+- **`rateLimit` sampler**: envoy-als tested with responseFlags present/absent
(if/else branch)
+- **`parsed?.` navigation**: envoy-als tested with nested Map traversal
+
+---
+
+## Config Data Serialization
+
+At build time, the precompiler serializes parsed LAL config POJOs to a JSON
manifest at
+`META-INF/config-data/lal.json`. This provides the runtime config data (rule
names, DSL
+strings, layers) for `LogFilterListener` to create `DSL` instances — without
requiring
+filesystem access to the original YAML files.
+
+| JSON Manifest | Source Directory | Serialized Type |
+|---|---|---|
+| `lal.json` | `lal/` | `Map<String, LALConfigs>` (filename → configs) |
+
+At runtime, the replacement `LALConfigs.load()` deserializes from this JSON
file instead
+of reading YAML from the filesystem.
+
+---
+
+## Same-FQCN Replacements (LAL)
+
+| Upstream Class | Upstream Location | Replacement Location | What Changed |
+|---|---|---|---|
+| `DSL` (LAL) | `analyzer/log-analyzer/.../dsl/DSL.java` |
`oap-libs-for-graalvm/log-analyzer-for-graalvm/` | Complete rewrite. Loads
transpiled `LalExpression` from `META-INF/lal-expressions.txt` manifest (keyed
by SHA-256 hash). No Groovy runtime. |
+| `LogAnalyzerModuleConfig` |
`analyzer/log-analyzer/.../provider/LogAnalyzerModuleConfig.java` |
`oap-libs-for-graalvm/log-analyzer-for-graalvm/` | Added `@Setter` at class
level. Enables reflection-free config loading via Lombok setters. |
+| `LALConfigs` | `analyzer/log-analyzer/.../provider/LALConfigs.java` |
`oap-libs-for-graalvm/log-analyzer-for-graalvm/` | Complete rewrite of static
`load()` method. Loads pre-compiled LAL config data from
`META-INF/config-data/{path}.json` instead of filesystem YAML files via
`ResourceUtils.getPathFiles()`. |
+| `AbstractSpec` | `analyzer/log-analyzer/.../dsl/spec/AbstractSpec.java` |
`oap-libs-for-graalvm/log-analyzer-for-graalvm/` | Added `abort()` no-arg
overload for transpiled code. |
+| `FilterSpec` | `analyzer/log-analyzer/.../dsl/spec/filter/FilterSpec.java` |
`oap-libs-for-graalvm/log-analyzer-for-graalvm/` | Added `Consumer` overloads:
`json()`, `text(Consumer)`, `extractor(Consumer)`, `sink(Consumer)`,
`filter(Runnable)`. |
+| `ExtractorSpec` |
`analyzer/log-analyzer/.../dsl/spec/extractor/ExtractorSpec.java` |
`oap-libs-for-graalvm/log-analyzer-for-graalvm/` | Added `Consumer` overloads:
`metrics(Consumer)`, `slowSql(Consumer)`, `sampledTrace(Consumer)`. |
+| `SinkSpec` | `analyzer/log-analyzer/.../dsl/spec/sink/SinkSpec.java` |
`oap-libs-for-graalvm/log-analyzer-for-graalvm/` | Added `Consumer` overloads:
`sampler(Consumer)`, `enforcer()`, `dropper()`. |
+| `SamplerSpec` | `analyzer/log-analyzer/.../dsl/spec/sink/SamplerSpec.java` |
`oap-libs-for-graalvm/log-analyzer-for-graalvm/` | Added `rateLimit(String,
Consumer)`, `possibility(int, Consumer)` for String-keyed samplers. |
+
+All replacements are repackaged into `log-analyzer-for-graalvm` via
`maven-shade-plugin` — the original `.class` files are excluded from the shaded
JAR.
+
+---
+
+## Files Created
+
+1.
**`oap-libs-for-graalvm/log-analyzer-for-graalvm/src/main/java/.../log/analyzer/dsl/DSL.java`**
+ Same-FQCN replacement: loads pre-compiled LAL scripts from manifest via
SHA-256 hash
+
+2.
**`oap-graalvm-server/src/test/java/.../graalvm/lal/LALScriptComparisonBase.java`**
+ Abstract base class: ModuleManager mock setup, dual-path compilation,
Binding state comparison
+
+3. **`oap-graalvm-server/src/test/java/.../graalvm/lal/LALDefaultTest.java`**
+ Tests for default.yaml (2 tests)
+
+4. **`oap-graalvm-server/src/test/java/.../graalvm/lal/LALNginxTest.java`**
+ Tests for nginx.yaml access-log and error-log rules (5 tests)
+
+5. **`oap-graalvm-server/src/test/java/.../graalvm/lal/LALSlowSqlTest.java`**
+ Tests for mysql/pgsql/redis-slowsql.yaml (3 tests)
+
+6. **`oap-graalvm-server/src/test/java/.../graalvm/lal/LALEnvoyAlsTest.java`**
+ Tests for envoy-als.yaml envoy-als rule (3 tests)
+
+7.
**`oap-graalvm-server/src/test/java/.../graalvm/lal/LALNetworkProfilingTest.java`**
+ Tests for network-profiling-slow-trace rule across 3 files (6 tests)
+
+---
+
+## Verification
+
+```bash
+# Build precompiler first (generates LAL classes + manifests)
+JAVA_HOME=/Users/wusheng/.sdkman/candidates/java/25-graal \
+ mvn -pl build-tools/precompiler install -DskipTests
+
+# Run LAL tests only
+JAVA_HOME=/Users/wusheng/.sdkman/candidates/java/25-graal \
+ mvn -pl oap-graalvm-server test \
+ -Dtest="org.apache.skywalking.oap.server.graalvm.lal.*"
+
+# Full build (all tests)
+JAVA_HOME=/Users/wusheng/.sdkman/candidates/java/25-graal make build-distro
+```
+
+Expected: 19 comparison tests across 5 test classes, all passing.
+
+---
+
+## Pure Java LAL Transpiler
+
+LAL transpilation is complete. All 10 LAL scripts (6 unique after SHA-256
dedup)
+are transpiled from Groovy AST to pure Java source at build time, compiled to
+`.class` files, and loaded at runtime via `LalExpression` interface — no Groovy
+runtime needed.
+
+**Approach**: Both options from the plan were combined:
+- **Option A (Transpiler)**: `LalToJavaTranspiler` converts Groovy AST to Java
source
+- **Option B (Groovy Stubs)**: `groovy-stubs` module provides minimal
`groovy.lang.*`
+ types for class loading (no `org.codehaus.groovy.*`)
+
+---
+
+## What Was Built
+
+### 1. LalExpression Interface
+`oap-libs-for-graalvm/log-analyzer-for-graalvm/.../dsl/LalExpression.java`
+
+```java
+@FunctionalInterface
+public interface LalExpression {
+ void execute(FilterSpec filterSpec, Binding binding);
+}
+```
+
+### 2. Groovy Stubs Module
+`oap-libs-for-graalvm/groovy-stubs/` — Minimal stub classes:
+- `groovy.lang.Binding`, `Closure`, `GString`, `GroovyObject`,
`GroovyObjectSupport`
+- `groovy.lang.Script`, `groovy.util.DelegatingScript`
+- `groovy.lang.DelegatesTo`, `MetaClass`, `MissingPropertyException`,
`GroovyRuntimeException`
+
+Key: **No `org.codehaus.groovy.*` packages** — prevents GraalVM
`GroovyIndyInterfaceFeature` from activating.
+
+### 3. Spec Class Consumer Overloads
+Same-FQCN replacements in `oap-libs-for-graalvm/log-analyzer-for-graalvm/`:
+- `AbstractSpec` — `abort()` no-arg
+- `FilterSpec` — `json()` no-arg, `text(Consumer)`, `extractor(Consumer)`,
`sink(Consumer)`, `filter(Runnable)`
+- `ExtractorSpec` — `metrics(Consumer)`, `slowSql(Consumer)`,
`sampledTrace(Consumer)`
+- `SinkSpec` — `sampler(Consumer)`, `enforcer()`, `dropper()`
+- `SamplerSpec` — `rateLimit(String, Consumer)`, `possibility(int, Consumer)`
+
+### 4. LalToJavaTranspiler
+`build-tools/precompiler/.../LalToJavaTranspiler.java` (~650 lines)
+
+Groovy AST → Java source transpilation:
+- Statement-based emission with delegation context tracking
+- If/else if/else chains
+- Property access via `getAt()`
+- Cast handling (`as String/Long/Boolean/Integer`)
+- GString interpolation → string concatenation
+- Null-safe navigation (`?.` → ternary null checks)
+- Static method calls (`ProcessRegistry`)
+- Map expression handling (named args)
+- Embedded helper methods (`getAt`, `toLong`, `toInt`, `toBoolean`, `isTruthy`)
+- JavaCompiler batch compilation
+- Manifest writing (`META-INF/lal-expressions.txt`)
+
+### 5. Runtime DSL.java
+Updated `oap-libs-for-graalvm/log-analyzer-for-graalvm/.../dsl/DSL.java`:
+- Loads `LalExpression` from `META-INF/lal-expressions.txt` (not
`DelegatingScript`)
+- `evaluate()` calls `expression.execute(filterSpec, binding)` (not
`script.run()`)
+
+### 6. Dual-Path Tests Updated
+`LALScriptComparisonBase.java` updated:
+- Path A: Fresh Groovy compilation → `DelegatingScript.run()`
+- Path B: Transpiled `LalExpression` from manifest →
`expression.execute(filterSpec, binding)`
+- 19 tests across 5 classes, all passing
+
+---
+
+## Generated Code Example
+
+Input (network-profiling-slow-trace, Groovy):
+```groovy
+filter {
+ json{}
+ extractor{
+ if (tag("LOG_KIND") == "NET_PROFILING_SAMPLED_TRACE") {
+ sampledTrace {
+ latency parsed.latency as Long
+ componentId 49 // simplified
+ }
+ }
+ }
+}
+```
+
+Output (LalExpr_0.java):
+```java
+public class LalExpr_0 implements LalExpression {
+ private static Object getAt(Object obj, String key) { ... }
+ private static long toLong(Object obj) { ... }
+
+ @Override
+ public void execute(FilterSpec filterSpec, Binding binding) {
+ filterSpec.json();
+ filterSpec.extractor(ext -> {
+ if
("NET_PROFILING_SAMPLED_TRACE".equals(filterSpec.tag("LOG_KIND"))) {
+ ext.sampledTrace(st -> {
+ st.latency(toLong(getAt(binding.parsed(), "latency")));
+ st.componentId(49);
+ });
+ }
+ });
+ }
+}
+```
+
+---
+
+## Groovy Runtime Removal
+
+- `groovy-stubs` wired as runtime dependency
+- Real Groovy (`groovy-5.0.3.jar`) moved to test-only scope
+- Native image builds and boots without `org.codehaus.groovy.*` on classpath
+- `GroovyIndyInterfaceFeature` stays dormant (no `org.codehaus.groovy`
packages)
diff --git a/docs/mal-immigration.md b/docs/mal-immigration.md
new file mode 100644
index 0000000..5506e39
--- /dev/null
+++ b/docs/mal-immigration.md
@@ -0,0 +1,817 @@
+# MAL Immigration: Build-Time Pre-Compilation + Groovy Elimination
+
+## Context
+
+MAL (Meter Analysis Language) and LAL (Log Analysis Language) have three
GraalVM-incompatible runtime patterns:
+
+1. **MeterSystem ClassPath scanning**: `MeterSystem` constructor uses Guava
`ClassPath.from()` to discover `@MeterFunction`-annotated classes (16 meter
functions).
+2. **MeterSystem Javassist dynamic class generation**: `MeterSystem.create()`
uses Javassist `ClassPool.makeClass()` to create one dynamic meter subclass per
metric rule at runtime (~1188 rules across all MAL sources). Classes live in
`org.apache.skywalking.oap.server.core.analysis.meter.dynamic.*`.
+3. **Groovy runtime compilation**: MAL uses `GroovyShell.parse()` with dynamic
Groovy features. LAL uses `GroovyShell.parse()` with `@CompileStatic`. Both
compile scripts at startup.
+
+**Key architectural difference from OAL**: MAL initialization is a tightly
coupled pipeline where Groovy compilation, static analysis, and Javassist
generation are interleaved:
+
+```
+Rule YAML → MetricConvert constructor → Analyzer.build()
+ → DSL.parse() [Groovy compilation]
+ → e.parse() [static analysis → scopeType, functionName,
metricType]
+ → meterSystem.create() [Javassist class generation +
MetricsStreamProcessor registration]
+```
+
+The build tool must execute this entire chain — Groovy pre-compilation and
Javassist pre-generation cannot be separated.
+
+**Solution**: Run the full MAL/LAL initialization at build time. Export
Javassist-generated `.class` files + compiled Groovy script bytecode. At
runtime, load pre-generated classes from manifests — no ClassPath scanning, no
Javassist, no Groovy compilation.
+
+---
+
+## Rule file inventory
+
+| Source | Path | Loader | Files | Metric Rules |
+|--------|------|--------|-------|--------------|
+| Agent meter | `meter-analyzer-config/` | `MeterConfigs.loadConfig()` | 11 |
~147 |
+| OTel metrics | `otel-rules/` | `Rules.loadRules()` | 55 | ~1039 |
+| Log MAL | `log-mal-rules/` | `Rules.loadRules()` | 2 | ~2 |
+| LAL scripts | `lal/` | `LALConfigs.load()` | 8 | 10 |
+| **Total** | | | **76** | **~1198** |
+
+Each MAL metric rule generates one Groovy script + one Javassist dynamic meter
class. Each LAL rule generates one Groovy script.
+
+---
+
+## MeterFunction Manifest + Same-FQCN MeterSystem
+
+### Problem
+
+`MeterSystem` constructor (`MeterSystem.java:75-96`) scans the classpath:
+
+```java
+ClassPath classpath = ClassPath.from(MeterSystem.class.getClassLoader());
+ImmutableSet<ClassPath.ClassInfo> classes =
classpath.getTopLevelClassesRecursive("org.apache.skywalking");
+for (ClassPath.ClassInfo classInfo : classes) {
+ Class<?> functionClass = classInfo.load();
+ if (functionClass.isAnnotationPresent(MeterFunction.class)) {
+ functionRegister.put(metricsFunction.functionName(), functionClass);
+ }
+}
+```
+
+### 16 Meter Function classes
+
+| Function Name | Class | Accept Type |
+|---|---|---|
+| `avg` | `AvgFunction` | `Long` |
+| `avgLabeled` | `AvgLabeledFunction` | `DataTable` |
+| `avgHistogram` | `AvgHistogramFunction` | `BucketedValues` |
+| `avgHistogramPercentile` | `AvgHistogramPercentileFunction` |
`PercentileArgument` |
+| `latest` | `LatestFunction` | `Long` |
+| `latestLabeled` | `LatestLabeledFunction` | `DataTable` |
+| `max` | `MaxFunction` | `Long` |
+| `maxLabeled` | `MaxLabeledFunction` | `DataTable` |
+| `min` | `MinFunction` | `Long` |
+| `minLabeled` | `MinLabeledFunction` | `DataTable` |
+| `sum` | `SumFunction` | `Long` |
+| `sumLabeled` | `SumLabeledFunction` | `DataTable` |
+| `sumHistogram` | `HistogramFunction` | `BucketedValues` |
+| `sumHistogramPercentile` | `SumHistogramPercentileFunction` |
`PercentileArgument` |
+| `sumPerMin` | `SumPerMinFunction` | `Long` |
+| `sumPerMinLabeled` | `SumPerMinLabeledFunction` | `DataTable` |
+
+### Approach
+
+1. Extend `OALClassExporter` to scan `@MeterFunction` annotation at build
time. Write `META-INF/annotation-scan/MeterFunction.txt` with format
`functionName=FQCN` (one entry per line).
+
+2. Create same-FQCN replacement `MeterSystem` in `oap-graalvm-server` that
reads function registry from the manifest instead of `ClassPath.from()`:
+
+```java
+// Pseudocode for replacement MeterSystem constructor
+public MeterSystem(ModuleManager manager) {
+ this.manager = manager;
+ this.classPool = ClassPool.getDefault();
+
+ // Read from manifest instead of ClassPath.from()
+ for (String line :
readManifest("META-INF/annotation-scan/MeterFunction.txt")) {
+ String[] parts = line.split("=", 2);
+ String functionName = parts[0];
+ Class<?> functionClass = Class.forName(parts[1]);
+ functionRegister.put(functionName, (Class<? extends AcceptableValue>)
functionClass);
+ }
+}
+```
+
+---
+
+## Build-Time MAL/LAL Pre-Compilation + MeterSystem Class Generation
+
+**Module**: `build-tools/precompiler` (unified tool)
+
+### MALCompiler.java — main build tool
+
+The tool executes the full initialization pipeline at build time:
+
+#### Initialize infrastructure
+
+```java
+// 1. Initialize scope registry (same as OALClassExporter)
+DefaultScopeDefine.reset();
+AnnotationScan scopeScan = new AnnotationScan();
+scopeScan.registerListener(new DefaultScopeDefine.Listener());
+scopeScan.scan();
+
+// 2. Create ExportingMeterSystem — intercepts Javassist to export .class files
+ExportingMeterSystem meterSystem = new ExportingMeterSystem(outputDir);
+```
+
+#### Load and compile all MAL rules
+
+```java
+// 3. Agent meter rules (meter-analyzer-config/)
+List<MeterConfig> agentConfigs =
MeterConfigs.loadConfig("meter-analyzer-config", activeFiles);
+for (MeterConfig config : agentConfigs) {
+ new MetricConvert(config, meterSystem); // triggers: Groovy compile +
parse + Javassist
+}
+
+// 4. OTel rules (otel-rules/)
+List<Rule> otelRules = Rules.loadRules("otel-rules", enabledOtelRules);
+for (Rule rule : otelRules) {
+ new MetricConvert(rule, meterSystem);
+}
+
+// 5. Log MAL rules (log-mal-rules/)
+List<Rule> logMalRules = Rules.loadRules("log-mal-rules", enabledLogMalRules);
+for (Rule rule : logMalRules) {
+ new MetricConvert(rule, meterSystem);
+}
+```
+
+#### Load and compile all LAL rules
+
+```java
+// 6. LAL rules (lal/)
+List<LALConfigs> lalConfigs = LALConfigs.load("lal", lalFiles);
+for (LALConfig config : flattenedConfigs) {
+ DSL.of(moduleManager, logConfig, config.getDsl()); // Groovy compile with
@CompileStatic
+}
+```
+
+#### Export bytecode + write manifests
+
+The tool intercepts both Groovy and Javassist class generation to capture
bytecode:
+
+**Javassist interception** (`ExportingMeterSystem`): Override `create()` to
call `ctClass.toBytecode()` and write `.class` files to the output directory.
Also records metadata for the manifest.
+
+**Groovy script capture**: Use Groovy's `CompilationUnit` API or intercept
`GroovyShell.parse()` to extract compiled script bytecode. Each MAL expression
compiles to a unique script class (name based on hash or sequential index).
+
+**Manifest files**:
+
+`META-INF/mal-meter-classes.txt` — Javassist-generated meter classes:
+```
+# format: metricsName|scopeId|functionName|dataType|FQCN
+meter_java_agent_created_tracing_context_count|1|sum|java.lang.Long|org.apache.skywalking.oap.server.core.analysis.meter.dynamic.meter_java_agent_created_tracing_context_count
+...
+```
+
+`META-INF/mal-groovy-scripts.txt` — Pre-compiled MAL Groovy scripts:
+```
+# format: metricName=scriptClassName
+meter_java_agent_created_tracing_context_count=org.apache.skywalking.oap.server.core.analysis.meter.script.Script0001
+...
+```
+
+`META-INF/lal-scripts.txt` — Pre-compiled LAL Groovy scripts:
+```
+# format: layer:ruleName=scriptClassName
+GENERAL:default=org.apache.skywalking.oap.server.core.analysis.lal.script.LALScript0001
+NGINX:nginx-access-log=org.apache.skywalking.oap.server.core.analysis.lal.script.LALScript0002
+...
+```
+
+### ExportingMeterSystem — Javassist interception
+
+This is a build-time variant of `MeterSystem` that exports bytecode instead of
loading classes:
+
+```java
+// Pseudocode
+public class ExportingMeterSystem extends MeterSystem {
+ private final Path outputDir;
+ private final List<MeterClassMetadata> metadata = new ArrayList<>();
+
+ @Override
+ public synchronized <T> void create(String metricsName, String
functionName,
+ ScopeType type, Class<T> dataType) {
+ // Same Javassist logic as upstream MeterSystem.create()
+ CtClass metricsClass = classPool.makeClass(METER_CLASS_PACKAGE +
className, parentClass);
+ // ... add constructor and createNew() method ...
+
+ // Instead of toClass(), write bytecode to disk
+ byte[] bytecode = metricsClass.toBytecode();
+ Path classFile = outputDir.resolve(packageToPath(METER_CLASS_PACKAGE +
className) + ".class");
+ Files.createDirectories(classFile.getParent());
+ Files.write(classFile, bytecode);
+
+ // Record metadata for manifest
+ metadata.add(new MeterClassMetadata(metricsName, type.getScopeId(),
+ functionName, dataType.getName(), METER_CLASS_PACKAGE +
className));
+ }
+}
+```
+
+### Same-FQCN replacement classes
+
+**1. MAL `DSL`** (`oap-graalvm-server/.../meter/analyzer/dsl/DSL.java`)
+
+Same FQCN as `org.apache.skywalking.oap.meter.analyzer.dsl.DSL`. `parse()`
loads a pre-compiled Groovy script class instead of calling
`GroovyShell.parse()`:
+
+```java
+// Pseudocode
+public static Expression parse(String metricName, String expression) {
+ // Look up pre-compiled script class from manifest
+ String scriptClassName = lookupScript(metricName);
+ Class<?> scriptClass = Class.forName(scriptClassName);
+ DelegatingScript script = (DelegatingScript)
scriptClass.getDeclaredConstructor().newInstance();
+
+ // Same CompilerConfiguration setup (imports, security) is baked into the
pre-compiled class
+ return new Expression(metricName, expression, script);
+}
+```
+
+The `Expression.empower()` method still runs at runtime — it calls
`setDelegate()` and `ExpandoMetaClass` registration. These are Java API calls
on the Groovy runtime, not compilation.
+
+**2. `FilterExpression`**
(`oap-graalvm-server/.../meter/analyzer/dsl/FilterExpression.java`)
+
+Same FQCN. Loads pre-compiled filter closure class instead of
`GroovyShell.evaluate()`.
+
+**3. LAL `DSL`** (`oap-graalvm-server/.../log/analyzer/dsl/DSL.java`)
+
+Same FQCN as `org.apache.skywalking.oap.log.analyzer.dsl.DSL`. `of()` loads
pre-compiled LAL script class (already `@CompileStatic`) instead of calling
`GroovyShell.parse()`:
+
+```java
+// Pseudocode
+public static DSL of(ModuleManager moduleManager, LogAnalyzerModuleConfig
config, String dsl) {
+ String scriptClassName = lookupLALScript(layer, ruleName);
+ Class<?> scriptClass = Class.forName(scriptClassName);
+ DelegatingScript script = (DelegatingScript)
scriptClass.getDeclaredConstructor().newInstance();
+ FilterSpec filterSpec = new FilterSpec(moduleManager, config);
+ script.setDelegate(filterSpec);
+ return new DSL(script, filterSpec);
+}
+```
+
+**4. `MeterSystem`** (enhanced from Step 1)
+
+The `create()` method becomes a manifest lookup + `MetricsStreamProcessor`
registration:
+
+```java
+// Pseudocode
+public synchronized <T> void create(String metricsName, String functionName,
+ ScopeType type, Class<T> dataType) {
+ if (meterPrototypes.containsKey(metricsName)) {
+ return; // already registered
+ }
+
+ // Load pre-generated class from classpath
+ MeterClassMetadata meta = lookupMeterClass(metricsName);
+ Class<?> targetClass = Class.forName(meta.fqcn);
+ AcceptableValue prototype = (AcceptableValue)
targetClass.getDeclaredConstructor().newInstance();
+ meterPrototypes.put(metricsName, new MeterDefinition(type, prototype,
dataType));
+
+ // Register with stream processor (same as upstream)
+ MetricsStreamProcessor.getInstance().create(
+ manager,
+ new StreamDefinition(metricsName, type.getScopeId(),
prototype.builder(),
+ MetricsStreamProcessor.class),
+ targetClass
+ );
+}
+```
+
+`buildMetrics()` and `doStreamingCalculation()` work unchanged — they use the
prototype map.
+
+### MAL Groovy: Why @CompileStatic is NOT possible
+
+MAL expressions rely on three dynamic Groovy features:
+
+1. **`propertyMissing(String)`** (`Expression.java:126`): When an expression
references `counter` or `jvm_memory_bytes_used`, Groovy calls
`ExpressionDelegate.propertyMissing(sampleName)` which looks up the sample
family from a `ThreadLocal<Map<String, SampleFamily>>`. Static compilation
cannot resolve these properties.
+
+2. **`ExpandoMetaClass` on `Number`** (`Expression.java:104-111`): The
`empower()` method registers `plus`, `minus`, `multiply`, `div` closures on
`Number.class` to allow expressions like `100 * server_cpu_seconds`. This is
runtime metaclass manipulation.
+
+3. **Closure arguments** in DSL methods: Expressions like `.tag({tags ->
tags.gc = 'young_gc'})` and `.filter({tags -> tags.job_name == 'mysql'})` pass
Groovy closures with dynamic property access.
+
+**Approach**: Pre-compile using standard dynamic Groovy (same
`CompilerConfiguration` as upstream — `DelegatingScript` base class,
`SecureASTCustomizer`, `ImportCustomizer`). The compiled `.class` files contain
the same bytecode that `GroovyShell.parse()` would produce. At runtime,
`Class.forName()` + `newInstance()` loads the pre-compiled script, and
`Expression.empower()` sets up the delegate and `ExpandoMetaClass`.
+
+**Native image solution**: Dynamic Groovy was eliminated entirely via the
MAL-to-Java transpiler (see below). All MAL expressions are transpiled to pure
Java at build time — no Groovy MOP, no `ExpandoMetaClass`, no `invokedynamic`
at runtime.
+
+### LAL Groovy: Already @CompileStatic
+
+LAL already uses `@CompileStatic` with `LALPrecompiledExtension` for type
checking. The `CompilationUnit` approach works directly. Compiled scripts
extend `LALDelegatingScript` and are fully statically typed.
+
+---
+
+## Verification Tests
+
+### MALCompilerTest (in `build-tools/mal-compiler/src/test/`)
+
+```
+- allRuleYamlFilesLoadable:
+ Verify all 76 rule files exist on classpath
+
+- fullCompilationGeneratesExpectedCounts:
+ Run MALCompiler.main(), verify:
+ - Generated meter .class count > 0 for each rule source
+ - Groovy script .class count matches rule count
+ - All 3 manifest files exist and are non-empty
+
+- manifestEntriesAreWellFormed:
+ Parse manifest files, verify format and field count per line
+```
+
+### MALPrecompiledRegistrationTest (in `oap-graalvm-server/src/test/`)
+
+```
+- meterFunctionManifestMatchesClasspath:
+ Compare manifest against Guava ClassPath scan of @MeterFunction
+ (same pattern as PrecompiledRegistrationTest for OAL)
+
+- all16MeterFunctionsInManifest:
+ Verify all 16 known function names appear
+
+- precompiledMeterClassesLoadable:
+ For each entry in mal-meter-classes.txt:
+ - Class.forName() succeeds
+ - Class extends the correct meter function parent
+
+- precompiledGroovyScriptsLoadable:
+ For each entry in mal-groovy-scripts.txt:
+ - Class.forName() succeeds
+ - Class is assignable to DelegatingScript
+
+- precompiledLALScriptsLoadable:
+ For each entry in lal-scripts.txt:
+ - Class.forName() succeeds
+ - Class is assignable to LALDelegatingScript
+```
+
+---
+
+## Config Data Serialization
+
+At build time, the precompiler serializes parsed config POJOs to JSON
manifests in
+`META-INF/config-data/`. This provides the runtime "wiring" data (metric
prefixes,
+rule names, expression lookup keys) that connects pre-compiled Groovy scripts
to
+incoming metrics — without requiring filesystem access to the original YAML
files.
+
+| JSON Manifest | Source Directory | Serialized Type |
+|---|---|---|
+| `meter-analyzer-config.json` | `meter-analyzer-config/` | `Map<String,
MeterConfig>` (filename → config) |
+| `otel-rules.json` | `otel-rules/` | `List<Rule>` |
+| `envoy-metrics-rules.json` | `envoy-metrics-rules/` | `List<Rule>` |
+| `log-mal-rules.json` | `log-mal-rules/` | `List<Rule>` |
+| `telegraf-rules.json` | `telegraf-rules/` | `List<Rule>` |
+| `zabbix-rules.json` | `zabbix-rules/` | `List<Rule>` |
+
+At runtime, replacement loader classes (`MeterConfigs`, `Rules`) deserialize
from
+these JSON files instead of reading YAML from the filesystem. Each logs that
configs
+are loaded from the pre-compiled distro.
+
+---
+
+## Same-FQCN Replacements (MAL)
+
+| Upstream Class | Upstream Location | Replacement Location | What Changed |
+|---|---|---|---|
+| `MeterSystem` | `server-core/.../analysis/meter/MeterSystem.java` |
`oap-libs-for-graalvm/server-core-for-graalvm/` | Complete rewrite. Reads
`@MeterFunction` classes from `META-INF/annotation-scan/MeterFunction.txt`
manifest instead of Guava `ClassPath.from()`. Loads pre-generated Javassist
meter classes from classpath instead of runtime `ClassPool.makeClass()`. |
+| `DSL` (MAL) | `analyzer/meter-analyzer/.../dsl/DSL.java` |
`oap-libs-for-graalvm/meter-analyzer-for-graalvm/` | Complete rewrite. Loads
pre-compiled Groovy `DelegatingScript` classes from
`META-INF/mal-groovy-scripts.txt` manifest instead of `GroovyShell.parse()`
runtime compilation. |
+| `FilterExpression` | `analyzer/meter-analyzer/.../dsl/FilterExpression.java`
| `oap-libs-for-graalvm/meter-analyzer-for-graalvm/` | Complete rewrite. Loads
pre-compiled Groovy filter closure classes from
`META-INF/mal-filter-scripts.properties` manifest instead of
`GroovyShell.evaluate()` runtime compilation. |
+| `Rules` | `analyzer/meter-analyzer/.../prometheus/rule/Rules.java` |
`oap-libs-for-graalvm/meter-analyzer-for-graalvm/` | Complete rewrite. Loads
pre-compiled rule data from `META-INF/config-data/{path}.json` instead of
filesystem YAML files via `ResourceUtils.getPath()` + `Files.walk()`. |
+| `MeterConfigs` |
`analyzer/agent-analyzer/.../meter/config/MeterConfigs.java` |
`oap-libs-for-graalvm/agent-analyzer-for-graalvm/` | Complete rewrite. Loads
pre-compiled meter config data from `META-INF/config-data/{path}.json` instead
of filesystem YAML files via `ResourceUtils.getPathFiles()`. |
+
+All replacements are repackaged into their respective `-for-graalvm` modules
via `maven-shade-plugin` — the original `.class` files are excluded from the
shaded JARs.
+
+---
+
+## Files Created
+
+1. **`build-tools/precompiler/`** (unified, replaces separate `oal-exporter` +
`mal-compiler`)
+ - Main build tool: loads all MAL/LAL rules, runs full initialization
pipeline, exports .class files + manifests
+
+2.
**`oap-libs-for-graalvm/server-core-for-graalvm/src/main/java/.../core/analysis/meter/MeterSystem.java`**
+ - Same-FQCN replacement: reads function registry from manifest, loads
pre-generated classes in `create()`
+
+3.
**`oap-libs-for-graalvm/meter-analyzer-for-graalvm/src/main/java/.../meter/analyzer/dsl/DSL.java`**
+ - Same-FQCN replacement: loads pre-compiled MAL Groovy scripts from manifest
+
+4.
**`oap-libs-for-graalvm/meter-analyzer-for-graalvm/src/main/java/.../meter/analyzer/dsl/FilterExpression.java`**
+ - Same-FQCN replacement: loads pre-compiled filter closures from manifest
+
+5.
**`oap-graalvm-server/src/test/java/.../graalvm/PrecompiledMALExecutionTest.java`**
+ - Runtime registration and loading tests
+
+6. **`oap-graalvm-server/src/test/java/.../graalvm/mal/`** — 73 comparison
test classes covering all 71 MAL YAML files
+
+## Key Upstream Files (read-only)
+
+- `MeterSystem.java` — ClassPath scan (`constructor:75-96`) + Javassist class
gen (`create():180-259`)
+- `DSL.java` (meter-analyzer) — MAL Groovy compilation: `GroovyShell.parse()`
with `DelegatingScript`, `SecureASTCustomizer`
+- `Expression.java` — Script execution: `DelegatingScript.run()`,
`ExpandoMetaClass` on Number, `ExpressionDelegate.propertyMissing()`
+- `FilterExpression.java` — Filter closure compilation:
`GroovyShell.evaluate()`
+- `Analyzer.java` — Initialization chain: `build()` → `DSL.parse()` →
`e.parse()` → `meterSystem.create()`
+- `MetricConvert.java` — Rule → Analyzer creation: `new MetricConvert(rule,
meterSystem)` triggers full pipeline
+- `DSL.java` (log-analyzer) — LAL Groovy compilation: `@CompileStatic` +
`LALPrecompiledExtension`
+- `LALDelegatingScript.java` — LAL script base class: `filter()`, `json()`,
`text()`, `extractor()`, `sink()`
+- `LogFilterListener.java` — LAL DSL factory: `DSL.of()` for each rule, stores
in `Map<Layer, Map<String, DSL>>`
+- `Rules.java` — OTel/log-MAL rule loading: `loadRules(path, enabledRules)`
+- `MeterConfigs.java` — Agent meter rule loading: `loadConfig(path, fileNames)`
+- `LALConfigs.java` — LAL rule loading: `load(path, files)`
+- `MeterFunction.java` — Annotation: `@MeterFunction(functionName = "...")` on
meter function classes
+- `AcceptableValue.java` — Interface all meter functions implement
+- `MeterClassPackageHolder.java` — Package anchor for Javassist-generated
classes
+
+---
+
+## Verification
+
+```bash
+# 1. Build everything
+make build-distro
+
+# 2. Check generated meter classes exist
+ls
build-tools/precompiler/target/generated-classes/org/apache/skywalking/oap/server/core/analysis/meter/dynamic/
+
+# 3. Check transpiled Java expression classes exist
+ls
build-tools/precompiler/target/generated-classes/org/apache/skywalking/oap/server/core/source/oal/rt/mal/
+
+# 4. Check manifest files
+wc -l
build-tools/precompiler/target/generated-classes/META-INF/mal-meter-classes.txt
+wc -l
build-tools/precompiler/target/generated-classes/META-INF/mal-expressions.txt
+wc -l
build-tools/precompiler/target/generated-classes/META-INF/lal-expressions.txt
+
+# 5. Check MeterFunction manifest
+cat
build-tools/precompiler/target/generated-classes/META-INF/annotation-scan/MeterFunction.txt
+
+# 6. Verify tests pass
+make build-distro # runs all comparison tests (1300+ assertions)
+```
+
+---
+
+## Pure Java MAL Transpiler
+
+### Why: Groovy + GraalVM Native Image is Incompatible
+
+The first native-image build (`make native-image`) failed with:
+
+```
+Error: Could not find target method:
+ protected static void com.oracle.svm.polyglot.groovy
+
.Target_org_codehaus_groovy_vmplugin_v7_IndyInterface_invalidateSwitchPoints
+ .invalidateSwitchPoints()
+```
+
+**Root cause**: GraalVM's built-in `GroovyIndyInterfaceFeature` (in
`library-support.jar`)
+targets `IndyInterface.invalidateSwitchPoints()` — a method removed in Groovy
5.0.3.
+The Groovy 5.x runtime still uses `org.codehaus.groovy` packages and
`invokedynamic`
+bootstrapping, but the internal API changed. GraalVM's substitution layer
cannot find
+the target method.
+
+**Why fixing Groovy is not worth it**: Even if the substitution were patched,
Groovy's
+dynamic runtime (ExpandoMetaClass, MOP, IndyInterface, MetaClassRegistry)
requires
+extensive GraalVM reflection/substitution configuration. The MAL pre-compiled
scripts
+use `invokedynamic` + Groovy MOP for method resolution. Making this work
reliably in
+native-image would be fragile and complex.
+
+**Solution**: Eliminate Groovy from runtime entirely. Generate pure Java code
at build
+time that implements the same MAL expression logic. The existing 1303 UTs
validate
+that the generated Java code produces identical results to the Groovy-compiled
scripts.
+
+---
+
+### Approach: MAL-to-Java Transpiler
+
+**Key insight**: MAL expressions are method chains on `SampleFamily` with a
well-defined
+API. The precompiler already parses all 1250+ MAL rules. Instead of compiling
them to
+Groovy bytecode (which needs Groovy runtime), we parse the Groovy AST at build
time and
+generate equivalent Java source code. Zero Groovy at runtime.
+
+### What the Transpiler Changes
+
+| Aspect | Before (Groovy pre-compilation) | After (Java transpiler) |
+|--------|----------------------------------|--------------------------|
+| Build output | Groovy `.class` bytecode | Pure Java `.class` files |
+| Runtime dependency | Groovy runtime (MOP, ExpandoMetaClass) | None |
+| Expression execution | `DelegatingScript.run()` + `ExpandoMetaClass` |
`MalExpression.run(Map<String, SampleFamily>)` |
+| Closure parameters | `groovy.lang.Closure` | Java functional interfaces |
+| `propertyMissing()` | Groovy MOP resolves metric names |
`samples.get("metricName")` |
+| `Number * SampleFamily` | `ExpandoMetaClass` on `Number` |
`sf.multiply(number)` (transpiler swaps operands) |
+| Filter expressions | Groovy `Script.run()` → `Closure<Boolean>` |
`SampleFilter.test(Map<String, String>)` |
+
+### What Stays the Same
+
+- **MeterSystem Javassist generation**: Pre-compiled at build time, unchanged
+- **Config data serialization**: JSON manifests for rule configs, unchanged
+- **Manifest-based loading**: Same pattern, new manifest format for Java
expressions
+- **MetricConvert pipeline**: Still runs at build time for Javassist class
generation
+
+---
+
+### Java Functional Interfaces for Closure Replacements
+
+5 `SampleFamily` methods accept `groovy.lang.Closure` parameters. Replace with
Java
+functional interfaces:
+
+| Method | Closure Type | Java Interface |
+|--------|-------------|----------------|
+| `tag(Closure<?> cl)` | `Map<String,String> → Map<String,String>` |
`TagFunction extends Function<Map, Map>` |
+| `filter(Closure<Boolean> f)` | `Map<String,String> → Boolean` |
`SampleFilter extends Predicate<Map>` |
+| `forEach(List, Closure each)` | `(String, Map) → void` | `ForEachFunction
extends BiConsumer<String, Map>` |
+| `decorate(Closure c)` | `MeterEntity → void` | `DecorateFunction extends
Consumer<MeterEntity>` |
+| `instance(..., Closure extractor)` | `Map<String,String> →
Map<String,String>` | `PropertiesExtractor extends Function<Map, Map>` |
+
+**Files created**:
+- `oap-libs-for-graalvm/meter-analyzer-for-graalvm/.../dsl/MalExpression.java`
+-
`oap-libs-for-graalvm/meter-analyzer-for-graalvm/.../dsl/SampleFamilyFunctions.java`
+
+### MalExpression Interface
+
+```java
+public interface MalExpression {
+ SampleFamily run(Map<String, SampleFamily> samples);
+}
+```
+
+Each generated MAL expression class implements this interface. The `samples`
map
+replaces Groovy's `propertyMissing()` delegate — metric names are resolved via
+`samples.get("metricName")`.
+
+### MalFilter Interface
+
+```java
+public interface MalFilter {
+ boolean test(Map<String, String> tags);
+}
+```
+
+Each generated filter expression class implements this interface.
+
+---
+
+### Same-FQCN SampleFamily Replacement
+
+**File**:
`oap-libs-for-graalvm/meter-analyzer-for-graalvm/.../dsl/SampleFamily.java`
+
+Copy upstream `SampleFamily.java` and change ONLY the 5 Closure-based method
signatures:
+
+```java
+// Before (upstream)
+public SampleFamily tag(Closure<?> cl) { ... cl.rehydrate(...).call(arg) ... }
+public SampleFamily filter(Closure<Boolean> filter) { ...
filter.call(it.labels) ... }
+public SampleFamily forEach(List<String> array, Closure<Void> each) { ...
each.call(element, labels) ... }
+public SampleFamily decorate(Closure<Void> c) { ... c.call(meterEntity) ... }
+public SampleFamily instance(..., Closure<Map<String,String>>
propertiesExtractor) { ... }
+
+// After (replacement)
+public SampleFamily tag(TagFunction fn) { ... fn.apply(arg) ... }
+public SampleFamily filter(SampleFilter f) { ... f.test(it.labels) ... }
+public SampleFamily forEach(List<String> array, ForEachFunction each) { ...
each.accept(element, labels) ... }
+public SampleFamily decorate(DecorateFunction fn) { ... fn.accept(meterEntity)
... }
+public SampleFamily instance(..., PropertiesExtractor extractor) { ... }
+```
+
+All other methods (tagEqual, sum, avg, histogram, service, endpoint, etc.)
remain
+identical — they don't use Groovy types.
+
+**Shade config**: Add `SampleFamily.class` and `SampleFamily$*.class` to shade
excludes.
+
+---
+
+### MAL-to-Java Transpiler Implementation
+
+**File**: `build-tools/precompiler/.../MalToJavaTranspiler.java`
+
+Parses MAL expression strings using Groovy's parser (available at build time,
not needed
+at runtime) and generates pure Java source code.
+
+### Input → Output Example
+
+**Input** (MAL expression string from apisix.yaml):
+```
+apisix_http_status.tagNotEqual('route','').tag({tags->tags.route='route/'+tags['route']})
+
.sum(['code','service_name','route','node']).rate('PT1M').endpoint(['service_name'],['route'],
Layer.APISIX)
+```
+
+**Output** (generated Java class):
+```java
+package org.apache.skywalking.oap.server.core.source.oal.rt.mal;
+
+import java.util.*;
+import org.apache.skywalking.oap.meter.analyzer.dsl.*;
+import org.apache.skywalking.oap.server.core.analysis.Layer;
+
+public class MalExpr_meter_apisix_endpoint_http_status implements
MalExpression {
+ @Override
+ public SampleFamily run(Map<String, SampleFamily> samples) {
+ SampleFamily v0 = samples.get("apisix_http_status");
+ if (v0 == null) return SampleFamily.EMPTY;
+ return v0
+ .tagNotEqual("route", "")
+ .tag(tags -> { tags.put("route", "route/" + tags.get("route"));
return tags; })
+ .sum(List.of("code", "service_name", "route", "node"))
+ .rate("PT1M")
+ .endpoint(List.of("service_name"), List.of("route"), Layer.APISIX);
+ }
+}
+```
+
+### AST Node → Java Output Mapping
+
+| Groovy AST Node | Java Output |
+|-----------------|-------------|
+| Bare property access (`metric_name`) | `samples.get("metric_name")` |
+| Method call on SampleFamily | Direct method call (same name) |
+| `Closure` in `.tag()` | Lambda: `tags -> { tags.put(...); return tags; }` |
+| `Closure` in `.filter()` | Lambda: `labels -> booleanExpr` |
+| `Closure` in `.forEach()` | Lambda: `(element, labels) -> { ... }` |
+| `tags.name` inside closure | `tags.get("name")` |
+| `tags.name = val` inside closure | `tags.put("name", val)` |
+| `tags['key']` inside closure | `tags.get("key")` |
+| Binary `SF + SF` | `left.plus(right)` |
+| Binary `SF - SF` | `left.minus(right)` |
+| Binary `SF * SF` | `left.multiply(right)` |
+| Binary `SF / SF` | `left.div(right)` |
+| Binary `Number * SF` | `sf.multiply(number)` (swap operands) |
+| Binary `Number + SF` | `sf.plus(number)` (swap operands) |
+| Binary `Number - SF` | `sf.minus(number).negative()` (swap operands) |
+| Binary `Number / SF` | `sf.newValue(v -> number / v)` (swap operands) |
+| Unary `-SF` | `sf.negative()` |
+| `['a','b','c']` (list literal) | `List.of("a", "b", "c")` |
+| `[50,75,90,95,99]` (int list) | `List.of(50, 75, 90, 95, 99)` |
+| String literal `'foo'` | `"foo"` |
+| Enum constant `Layer.APISIX` | `Layer.APISIX` |
+| Enum constant `DetectPoint.SERVER` | `DetectPoint.SERVER` |
+| `DownsamplingType.LATEST` / `LATEST` | `DownsamplingType.LATEST` |
+| `time()` function | `java.time.Instant.now().getEpochSecond()` |
+| `a?.b` (safe navigation) | `a != null ? a.b : null` |
+| `a ?: b` (elvis) | `a != null ? a : b` |
+
+### Filter Expression Transpilation
+
+Filter expressions are simpler — they are boolean closures:
+
+**Input**: `{ tags -> tags.job_name == 'apisix-monitoring' }`
+
+**Output**:
+```java
+public class MalFilter_apisix implements MalFilter {
+ @Override
+ public boolean test(Map<String, String> tags) {
+ return "apisix-monitoring".equals(tags.get("job_name"));
+ }
+}
+```
+
+---
+
+### Same-FQCN Expression.java Replacement
+
+**File**:
`oap-libs-for-graalvm/meter-analyzer-for-graalvm/.../dsl/Expression.java`
+
+Simplified — no DelegatingScript, no ExpandoMetaClass, no ExpressionDelegate:
+
+```java
+public class Expression {
+ private final String metricName;
+ private final String literal;
+ private final MalExpression expression; // Pure Java
+
+ public Result run(Map<String, SampleFamily> sampleFamilies) {
+ SampleFamily sf = expression.run(sampleFamilies);
+ if (sf == SampleFamily.EMPTY) return Result.fail("EMPTY");
+ return Result.success(sf);
+ }
+
+ // parse() still provides ExpressionParsingContext for static analysis
+ // (scopeType, functionName, etc.) — this is unchanged
+}
+```
+
+---
+
+### Updated DSL.java and FilterExpression.java
+
+### DSL.java (already exists, update)
+
+Change from loading `DelegatingScript` to loading `MalExpression`:
+
+```java
+public static Expression parse(String metricName, String expression) {
+ MalExpression malExpr = loadFromManifest(metricName); // Class.forName +
newInstance
+ return new Expression(metricName, expression, malExpr);
+}
+```
+
+Manifest: `META-INF/mal-expressions.txt` (replaces `mal-groovy-scripts.txt`):
+```
+meter_apisix_endpoint_http_status=org.apache.skywalking.oap.server.core.source.oal.rt.mal.MalExpr_meter_apisix_endpoint_http_status
+```
+
+### FilterExpression.java (already exists, update)
+
+Change from loading Groovy `Script` → `Closure<Boolean>` to loading
`MalFilter`:
+
+```java
+public class FilterExpression {
+ private final MalFilter filter;
+
+ public FilterExpression(String literal) {
+ this.filter = loadFromManifest(literal); // Class.forName +
newInstance
+ }
+
+ public Map<String, SampleFamily> filter(Map<String, SampleFamily>
sampleFamilies) {
+ // Apply filter.test(labels) to each sample family
+ }
+}
+```
+
+---
+
+### Precompiler Integration
+
+Replace `compileMAL()` in `Precompiler.java` to use the transpiler instead of
Groovy
+compilation:
+
+```java
+private static void compileMAL(String outputDir, ...) {
+ MalToJavaTranspiler transpiler = new MalToJavaTranspiler(outputDir);
+
+ for (Rule rule : allRules) {
+ for (MetricsRule mr : rule.getMetricsRules()) {
+ String metricName = formatMetricName(rule, mr.getName());
+ String expression = formatExp(rule.getExpPrefix(),
rule.getExpSuffix(), mr.getExp());
+ transpiler.transpile(metricName, expression);
+ }
+ if (rule.getFilter() != null) {
+ transpiler.transpileFilter(rule.getName(), rule.getFilter());
+ }
+ }
+
+ transpiler.writeManifest(); // mal-expressions.txt
+ transpiler.compileAll(); // javac on generated .java files
+}
+```
+
+The Javassist meter class generation is still run via `MetricConvert` pipeline
(unchanged
+from Phase 2). Only Groovy compilation is replaced by transpilation.
+
+---
+
+### Same-FQCN Replacements (Transpiler)
+
+| Upstream Class | Replacement Location | Change |
+|---|---|---|
+| `SampleFamily` | `meter-analyzer-for-graalvm/` | **New.** Closure parameters
→ functional interfaces |
+| `Expression` | `meter-analyzer-for-graalvm/` | **New.** No Groovy: uses
`MalExpression` instead of `DelegatingScript` |
+| `DSL` (MAL) | `meter-analyzer-for-graalvm/` | **Updated.** Loads
`MalExpression` instead of Groovy `DelegatingScript` |
+| `FilterExpression` | `meter-analyzer-for-graalvm/` | **Updated.** Loads
`MalFilter` instead of Groovy `Closure<Boolean>` |
+
+---
+
+### Files Created (Transpiler)
+
+| File | Purpose |
+|------|---------|
+| `.../dsl/MalExpression.java` | Interface for generated MAL expression
classes |
+| `.../dsl/MalFilter.java` | Interface for generated MAL filter classes |
+| `.../dsl/SampleFamilyFunctions.java` | Functional interfaces: TagFunction,
SampleFilter, ForEachFunction, DecorateFunction, PropertiesExtractor |
+| `.../meter-analyzer-for-graalvm/SampleFamily.java` | Same-FQCN: Closure →
functional interfaces |
+| `.../meter-analyzer-for-graalvm/Expression.java` | Same-FQCN: no Groovy,
uses MalExpression |
+| `.../precompiler/MalToJavaTranspiler.java` | AST-walking transpiler: Groovy
AST → Java source |
+| Generated `MalExpr_*.java` files | ~1250 pure Java expression classes |
+| Generated `MalFilter_*.java` files | ~29 pure Java filter classes |
+
+### Files Modified (Transpiler)
+
+| File | Change |
+|------|--------|
+| `Precompiler.java` | `compileMAL()` uses transpiler instead of Groovy
compilation |
+| `meter-analyzer-for-graalvm/pom.xml` | Add SampleFamily, Expression to shade
excludes |
+| `meter-analyzer-for-graalvm/DSL.java` | Load MalExpression instead of
DelegatingScript |
+| `meter-analyzer-for-graalvm/FilterExpression.java` | Load MalFilter instead
of Groovy Closure |
+
+---
+
+### Transpiler Verification
+
+```bash
+# 1. Rebuild with transpiler
+make compile
+
+# 2. Run ALL existing tests (1303 UTs validate identical MAL behavior)
+make test
+
+# 3. Boot JVM distro
+make shutdown && make boot
+
+# 4. Verify no Groovy at runtime (after LAL is also transpiled)
+jar tf
oap-graalvm-server/target/oap-graalvm-jvm-distro/oap-graalvm-jvm-distro/libs/*.jar
| grep groovy
+# Should find NOTHING
+
+# 5. Attempt native-image build (no more Groovy substitution error)
+make native-image
+```
diff --git a/docs/menu.yml b/docs/menu.yml
index 4b0af8b..a979768 100644
--- a/docs/menu.yml
+++ b/docs/menu.yml
@@ -20,8 +20,14 @@ catalog:
path: "/readme"
- name: "Distribution Policy"
path: "/distro-policy"
- - name: "Configuration"
- path: "/configuration"
+ - name: "Setup"
+ catalog:
+ - name: "Compiling from Source"
+ path: "/compiling"
+ - name: "Pre-Built Docker Images"
+ path: "/docker-image"
+ - name: "Configuration"
+ path: "/configuration"
- name: "Build-Time Immigration"
catalog:
- name: "OAL Pre-Compilation"
diff --git a/docs/oal-immigration.md b/docs/oal-immigration.md
new file mode 100644
index 0000000..dc40f99
--- /dev/null
+++ b/docs/oal-immigration.md
@@ -0,0 +1,162 @@
+# OAL Build-Time Pre-Compilation
+
+## Context
+
+OAL engine generates metrics, builder, and dispatcher classes at runtime via
Javassist (`ClassPool.makeClass()` → `CtClass.toClass()`). GraalVM native image
doesn't support runtime bytecode generation. Additionally, Guava's
`ClassPath.from()` — used by `AnnotationScan.scan()` and
`SourceReceiverImpl.scan()` — doesn't work in native image (no JAR-based
classpath).
+
+**Solution**: Run OAL engine at build time, export `.class` files + manifests.
Replace upstream classes with same-FQCN versions that load from manifests
instead of scanning or generating code.
+
+---
+
+## Build-Time OAL Class Export Tool
+
+**Module**: `build-tools/precompiler` (originally `build-tools/oal-exporter`,
merged into unified precompiler)
+
+**Created**: `OALClassExporter.java` — main class that:
+
+1. Validates all 9 OAL script files are on the classpath
+2. Initializes `DefaultScopeDefine` by scanning `@ScopeDeclaration`
annotations (OAL enricher needs scope metadata)
+3. For each of the 9 `OALDefine` configs: instantiates `OALEngineV2`, enables
debug output (`setOpenEngineDebug(true)` + `setGeneratedFilePath()`), calls
`engine.start()` which parses OAL → enriches → generates `.class` files via
Javassist
+4. Scans the output directory for generated `.class` files and writes OAL
manifests:
+ - `META-INF/oal-metrics-classes.txt` — ~620 fully-qualified class names
+ - `META-INF/oal-dispatcher-classes.txt` — ~45 fully-qualified class names
+ - `META-INF/oal-disabled-sources.txt` — disabled source names from
`disable.oal`
+5. Runs Guava `ClassPath.from()` scan at build time to produce 6
annotation/interface manifests under `META-INF/annotation-scan/`:
+ - `ScopeDeclaration.txt` — classes annotated with `@ScopeDeclaration`
+ - `Stream.txt` — classes annotated with `@Stream` (hardcoded only, not
OAL-generated)
+ - `Disable.txt` — classes annotated with `@Disable`
+ - `MultipleDisable.txt` — classes annotated with `@MultipleDisable`
+ - `SourceDispatcher.txt` — concrete implementations of `SourceDispatcher`
interface (hardcoded only)
+ - `ISourceDecorator.txt` — concrete implementations of `ISourceDecorator`
interface
+
+**Key difference from original plan**: No "collecting listeners" needed.
`engine.start()` generates `.class` files directly to disk via the debug API.
We scan the output directory for class files rather than hooking into engine
callbacks.
+
+### 9 OAL Defines processed
+
+| Define | Config File | Source Package | Catalog |
+|--------|-------------|----------------|---------|
+| `DisableOALDefine` | `oal/disable.oal` | `core.source` | — |
+| `CoreOALDefine` | `oal/core.oal` | `core.source` | — |
+| `JVMOALDefine` | `oal/java-agent.oal` | `core.source` | — |
+| `CLROALDefine` | `oal/dotnet-agent.oal` | `core.source` | — |
+| `BrowserOALDefine` | `oal/browser.oal` | `core.browser.source` | — |
+| `MeshOALDefine` | `oal/mesh.oal` | `core.source` | `ServiceMesh` |
+| `EBPFOALDefine` | `oal/ebpf.oal` | `core.source` | — |
+| `TCPOALDefine` | `oal/tcp.oal` | `core.source` | `EnvoyTCP` |
+| `CiliumOALDefine` | `oal/cilium.oal` | `core.source` | — |
+
+### Generated class packages
+
+- Metrics:
`org.apache.skywalking.oap.server.core.source.oal.rt.metrics.*Metrics`
+- Builders:
`org.apache.skywalking.oap.server.core.source.oal.rt.metrics.builder.*MetricsBuilder`
+- Dispatchers:
`org.apache.skywalking.oap.server.core.source.oal.rt.dispatcher.[catalog].*Dispatcher`
+
+---
+
+## Runtime Registration via Same-FQCN Replacement Classes
+
+Instead of extending upstream classes or hooking via `ModuleWiringBridge`, we
use **same-FQCN replacement**: create classes in `oap-graalvm-server` with the
exact same fully-qualified class name as the upstream class. Maven classpath
precedence ensures our version is loaded instead of the upstream version.
+
+### 3 replacement classes created:
+
+**1. `OALEngineLoaderService`**
(`oap-graalvm-server/.../core/oal/rt/OALEngineLoaderService.java`)
+
+Same FQCN as upstream
`org.apache.skywalking.oap.server.core.oal.rt.OALEngineLoaderService`. On first
`load()` call:
+- Reads `META-INF/oal-disabled-sources.txt` → registers with `DisableRegister`
+- Reads `META-INF/oal-metrics-classes.txt` → `Class.forName()` →
`StreamAnnotationListener.notify()`
+- Reads `META-INF/oal-dispatcher-classes.txt` → `Class.forName()` →
`DispatcherDetectorListener.addIfAsSourceDispatcher()`
+- All subsequent `load()` calls are no-ops (all classes registered on first
call regardless of which `OALDefine` triggered it)
+
+**2. `AnnotationScan`**
(`oap-graalvm-server/.../core/annotation/AnnotationScan.java`)
+
+Same FQCN as upstream
`org.apache.skywalking.oap.server.core.annotation.AnnotationScan`. Instead of
Guava `ClassPath.from()` scanning, reads manifest files from
`META-INF/annotation-scan/{AnnotationSimpleName}.txt`. Each registered
`AnnotationListener` is matched against its corresponding manifest.
+
+**3. `SourceReceiverImpl`**
(`oap-graalvm-server/.../core/source/SourceReceiverImpl.java`)
+
+Same FQCN as upstream
`org.apache.skywalking.oap.server.core.source.SourceReceiverImpl`. `scan()`
reads from `META-INF/annotation-scan/SourceDispatcher.txt` and
`META-INF/annotation-scan/ISourceDecorator.txt` instead of Guava classpath
scanning.
+
+### Key differences from original plan:
+- **No extending** — same-FQCN replacement instead of subclassing
+- **No `ModuleWiringBridge` changes** — classpath precedence handles the swap
automatically
+- **3 replacement classes, not 1** — `AnnotationScan` and `SourceReceiverImpl`
also needed replacement
+- **Classpath scanning fully eliminated** — annotation manifests replace Guava
scanning
+
+---
+
+## Class Loading and Remaining Scans
+
+### `Class.forName()` in native image
+`Class.forName()` is supported in GraalVM native image when classes are
registered in `reflect-config.json`. Since all pre-generated classes are on the
classpath at native-image build time, the GraalVM compiler includes them in the
binary. The `reflect-config.json` entries enable runtime `Class.forName()`
lookup.
+
+### OAL-internal scans — build-time only
+The 3 OAL-internal scans (`MetricsHolder`, `DefaultMetricsFunctionRegistry`,
`FilterMatchers`) only run inside the OAL engine during `engine.start()`. They
happen at **build time** in `OALClassExporter`, not at runtime. Automatically
solved.
+
+### `MeterSystem` — solved in MAL immigration
+`MeterSystem` uses Guava `ClassPath.from()` to discover meter function classes
at runtime. Replaced with manifest-based loading. See
[mal-immigration.md](mal-immigration.md).
+
+### `reflect-config.json`
+GraalVM reflection configuration for `Class.forName()` calls on OAL-generated
and manifest-listed classes is auto-generated by the precompiler from manifests.
+
+---
+
+## Same-FQCN Replacements (OAL)
+
+| Upstream Class | Upstream Location | Replacement Location | What Changed |
+|---|---|---|---|
+| `OALEngineLoaderService` |
`server-core/.../oal/rt/OALEngineLoaderService.java` |
`oap-libs-for-graalvm/server-core-for-graalvm/` | Complete rewrite. Loads
pre-compiled OAL classes from build-time manifests instead of running ANTLR4 +
FreeMarker + Javassist at runtime. |
+| `AnnotationScan` | `server-core/.../annotation/AnnotationScan.java` |
`oap-libs-for-graalvm/server-core-for-graalvm/` | Complete rewrite. Reads
`META-INF/annotation-scan/{name}.txt` manifests instead of Guava
`ClassPath.from()` scanning. |
+| `SourceReceiverImpl` | `server-core/.../source/SourceReceiverImpl.java` |
`oap-libs-for-graalvm/server-core-for-graalvm/` | Complete rewrite. Reads
dispatcher/decorator manifests instead of Guava `ClassPath.from()` scanning. |
+
+All three replacements are repackaged into `server-core-for-graalvm` via
`maven-shade-plugin` — the original `.class` files are excluded from the shaded
JAR.
+
+---
+
+## Files Created
+
+1. **`build-tools/precompiler/src/main/java/.../Precompiler.java`** (unified,
originally `oal-exporter`)
+ - Build-time tool: runs 9 OAL defines, exports `.class` files, writes OAL
manifests + annotation/interface manifests + reflection metadata
+
+2.
**`oap-libs-for-graalvm/server-core-for-graalvm/src/main/java/.../core/oal/rt/OALEngineLoaderService.java`**
+ - Same-FQCN replacement: loads pre-compiled OAL classes from manifests
+
+3.
**`oap-libs-for-graalvm/server-core-for-graalvm/src/main/java/.../core/annotation/AnnotationScan.java`**
+ - Same-FQCN replacement: reads annotation manifests instead of Guava
classpath scanning
+
+4.
**`oap-libs-for-graalvm/server-core-for-graalvm/src/main/java/.../core/source/SourceReceiverImpl.java`**
+ - Same-FQCN replacement: reads dispatcher/decorator manifests instead of
Guava classpath scanning
+
+5. **`oap-graalvm-server/src/test/java/.../PrecompiledRegistrationTest.java`**
+ - 12 tests: manifest vs Guava scan comparison, OAL class loading, scope
registration, source→dispatcher→metrics chain consistency
+
+## Key Upstream Files (read-only)
+
+- `OALEngineV2.java` — `start()` (parse → enrich → generate),
`notifyAllListeners()` (register)
+- `OALClassGeneratorV2.java` — `setOpenEngineDebug(true)`,
`setGeneratedFilePath()`, `writeGeneratedFile()` exports via
`ctClass.toBytecode()`
+- `OALEngineLoaderService.java` (upstream) — `load()` creates engine, sets
listeners, calls `start()`+`notifyAllListeners()`
+- `StorageBuilderFactory.java:67-78` — `Default` impl uses `metrics-builder`
template path
+- `StreamAnnotationListener.java` — `notify(Class)` reads `@Stream`, routes to
`MetricsStreamProcessor.create()`
+- `CoreModuleProvider.java:356-357` — registers `OALEngineLoaderService` in
`prepare()`
+- `CoreModuleProvider.java:417-421` — `start()` calls `load(DisableOALDefine)`
then `scan()`
+
+---
+
+## Verification
+
+```bash
+# 1. Build everything
+make build-distro
+
+# 2. Check generated classes exist
+ls
build-tools/precompiler/target/generated-classes/org/apache/skywalking/oap/server/core/source/oal/rt/metrics/
+ls
build-tools/precompiler/target/generated-classes/org/apache/skywalking/oap/server/core/source/oal/rt/dispatcher/
+
+# 3. Check manifest files
+cat
build-tools/precompiler/target/generated-classes/META-INF/oal-metrics-classes.txt
+cat
build-tools/precompiler/target/generated-classes/META-INF/oal-dispatcher-classes.txt
+
+# 4. Check annotation scan manifests
+ls build-tools/precompiler/target/generated-classes/META-INF/annotation-scan/
+
+# 5. Verify tests pass
+make build-distro # runs PrecompiledRegistrationTest
+```
