This is an automated email from the ASF dual-hosted git repository. wusheng pushed a commit to branch groovy-replace in repository https://gitbox.apache.org/repos/asf/skywalking.git
commit efbcb16177d1bfdd1e09814f46c88aa97fc08d49 Author: Wu Sheng <[email protected]> AuthorDate: Sun Mar 1 23:05:58 2026 +0800 Update DSL compiler docs with MAL runtime execution comparison and LAL typed signature Co-Authored-By: Claude Opus 4.6 <[email protected]> --- docs/en/academy/dsl-compiler-design.md | 20 ++++++++++----- oap-server/analyzer/log-analyzer/CLAUDE.md | 38 ++++++++++++++++++++++------ oap-server/analyzer/meter-analyzer/CLAUDE.md | 13 +++++++--- 3 files changed, 53 insertions(+), 18 deletions(-) diff --git a/docs/en/academy/dsl-compiler-design.md b/docs/en/academy/dsl-compiler-design.md index 7e243754b8..1101605841 100644 --- a/docs/en/academy/dsl-compiler-design.md +++ b/docs/en/academy/dsl-compiler-design.md @@ -42,7 +42,7 @@ Phase 3: Javassist Bytecode Generation | OAL | Extends metrics function class (e.g., `LongAvgMetrics`) | `id()`, `serialize()`, `deserialize()`, plus dispatcher `dispatch(source)` | | MAL metric | `MalExpression` | `SampleFamily run(Map<String, SampleFamily> samples)` | | MAL filter | `Predicate<Map<String, String>>` | `boolean test(Map<String, String> tags)` | -| LAL | `LalExpression` | `void execute(Object filterSpec, Object binding)` | +| LAL | `LalExpression` | `void execute(FilterSpec filterSpec, Binding binding)` | | Hierarchy | `BiFunction<Service, Service, Boolean>` | `Boolean apply(Service upper, Service lower)` | OAL is the most complex -- it generates **three classes per metric** (metrics class with storage annotations, @@ -144,10 +144,16 @@ The checker mechanism: 1. Loads all test copies of production YAML config files from `test/script-cases/scripts/` 2. For each DSL expression, compiles with **both** v1 (Groovy) and v2 (ANTLR4 + Javassist) 3. Compares the results: - - **MAL**: Compare extracted metadata (sample names, aggregation labels, - downsampling type, percentile config) - - **LAL**: Runtime execution comparison — both v1 and v2 execute with mock LogData, - then compare Binding state (service, layer, tags, abort/save flags) + - **MAL**: Two-level comparison for each expression: + 1. **Metadata comparison** -- sample names, aggregation labels, downsampling type, percentile config + 2. **Runtime execution comparison** -- builds mock `SampleFamily` input data from `ExpressionMetadata`, + executes with both v1 and v2, compares output samples (count, labels, values with epsilon). + For `increase()`/`rate()` expressions, the `CounterWindow` is primed with an initial run before + comparing the second run's output. + - **LAL**: Runtime execution comparison -- both v1 and v2 execute with mock LogData, + then compare Binding state (service, layer, tags, abort/save flags). + Test scripts include both copies of production configs (`oap-cases/`) and + dedicated feature-coverage rules (`feature-cases/`). - **Hierarchy**: Compare `BiFunction` evaluation with test Service pairs This ensures 100% behavioral parity. The Groovy v1 modules are **test-only dependencies** -- they are not @@ -157,7 +163,7 @@ included in the OAP distribution. | Checker | Expressions Tested | Status | |---------|-------------------|--------| -| MAL metric expressions | 1,187 | All pass | +| MAL metric expressions | 1,187 | All pass (metadata + runtime execution) | | MAL filter expressions | 29 | All pass | -| LAL scripts | 10 | All pass | +| LAL scripts | 29 | All pass | | Hierarchy rules | 22 | All pass | diff --git a/oap-server/analyzer/log-analyzer/CLAUDE.md b/oap-server/analyzer/log-analyzer/CLAUDE.md index a7b60c4a8e..38078c0657 100644 --- a/oap-server/analyzer/log-analyzer/CLAUDE.md +++ b/oap-server/analyzer/log-analyzer/CLAUDE.md @@ -19,8 +19,7 @@ LAL DSL string The generated class implements: ```java -void execute(Object filterSpec, Object binding) - // cast internally to FilterSpec and Binding +void execute(FilterSpec filterSpec, Binding binding) ``` ## File Structure @@ -39,8 +38,9 @@ oap-server/analyzer/log-analyzer/ BindingAware.java — Interface for consumers needing Binding access src/test/java/.../compiler/ - LALScriptParserTest.java — 8 parser tests - LALClassGeneratorTest.java — 6 generator tests + LALScriptParserTest.java — 20 parser tests + LALClassGeneratorTest.java — 35 generator tests + LALExpressionExecutionTest.java — 27 data-driven execution tests (from YAML + .input.data) ``` ## Package & Class Naming @@ -93,7 +93,7 @@ Three classes are generated: // implements Consumer, BindingAware public void accept(Object arg) { ExtractorSpec _t = (ExtractorSpec) arg; - _t.service(String.valueOf(getAt(binding.parsed(), "service"))); + _t.service(toStr(getAt(binding.parsed(), "service"))); } ``` @@ -101,9 +101,7 @@ Three classes are generated: ```java public Consumer _consumer0; // wired after toClass() - public void execute(Object arg0, Object arg1) { - FilterSpec filterSpec = (FilterSpec) arg0; - Binding binding = (Binding) arg1; + public void execute(FilterSpec filterSpec, Binding binding) { filterSpec.json(); ((BindingAware) this._consumer0).setBinding(binding); filterSpec.extractor(this._consumer0); @@ -120,6 +118,30 @@ Three classes are generated: - `sink {}` empty → no consumer, emits `filterSpec.sink()` - `sink { enforcer {} }` → allocates a consumer +## Null-Safe String Conversion + +Generated code uses `toStr()` instead of `String.valueOf()` for casting parsed values to String: +```java +private static String toStr(Object obj) { return obj == null ? null : String.valueOf(obj); } +``` +This preserves Java `null` for missing fields (matching Groovy's `null as String` → `null` behavior), +whereas `String.valueOf(null)` would produce the string `"null"`. + +## Data-Driven Execution Tests + +`LALExpressionExecutionTest` loads LAL rules from YAML and mock input from `.input.data` files: + +``` +test/script-cases/scripts/lal/test-lal/ + oap-cases/ — copies of shipped LAL configs (each with .input.data) + feature-cases/ + execution-basic.yaml — 17 LAL feature-coverage rules + execution-basic.input.data — mock input + expected output per rule +``` + +Each `.input.data` entry specifies `body-type`, `body`, optional `tags`, and `expect` assertions +(service, instance, endpoint, layer, tags, abort, save, timestamp, sampledTrace fields). + ## Dependencies All within this module (grammar, compiler, and runtime are merged): diff --git a/oap-server/analyzer/meter-analyzer/CLAUDE.md b/oap-server/analyzer/meter-analyzer/CLAUDE.md index 2cdf9e720b..1d6ed11c9d 100644 --- a/oap-server/analyzer/meter-analyzer/CLAUDE.md +++ b/oap-server/analyzer/meter-analyzer/CLAUDE.md @@ -37,10 +37,11 @@ oap-server/analyzer/meter-analyzer/ MALClassGenerator.java — Javassist code generator rt/ MalExpressionPackageHolder.java — Class loading anchor (empty marker) + MalRuntimeHelper.java — Static helpers called by generated code (e.g., divReverse) src/test/java/.../compiler/ - MALScriptParserTest.java — 14 parser tests - MALClassGeneratorTest.java — 9 generator tests + MALScriptParserTest.java — 20 parser tests + MALClassGeneratorTest.java — 28 generator tests ``` ## Package & Class Naming @@ -51,16 +52,22 @@ oap-server/analyzer/meter-analyzer/ | Generated classes | `org.apache.skywalking.oap.meter.analyzer.compiler.rt.MalExpr_<N>` | | Closure classes | `org.apache.skywalking.oap.meter.analyzer.compiler.rt.MalExpr_<N>_Closure<M>` | | Package holder | `org.apache.skywalking.oap.meter.analyzer.compiler.rt.MalExpressionPackageHolder` | +| Runtime helper | `org.apache.skywalking.oap.meter.analyzer.compiler.rt.MalRuntimeHelper` | | Functional interface | `org.apache.skywalking.oap.meter.analyzer.dsl.MalExpression` (in meter-analyzer) | `<N>` is a global `AtomicInteger` counter. `<M>` is the closure index within the expression. ## Javassist Constraints -- **No anonymous inner classes**: Javassist cannot compile `new Consumer() { ... }` or `new Function() { ... }` in method bodies. Closures are pre-compiled as separate `CtClass` instances implementing `SampleFamilyFunctions$TagFunction`, stored as fields (`_closure0`, `_closure1`, ...) on the main class, and wired via reflection after `toClass()`. +- **No anonymous inner classes**: Javassist cannot compile `new Consumer() { ... }` or `new Function() { ... }` in method bodies. Closures are pre-compiled as separate `CtClass` instances, stored as fields (`_closure0`, `_closure1`, ...) on the main class, and wired via reflection after `toClass()`. - **No lambda expressions**: Use the separate-class approach above. - **Inner class notation**: Use `$` not `.` for nested classes (e.g., `SampleFamilyFunctions$TagFunction`). - **`isPresent()`/`get()` instead of `ifPresent()`**: `ifPresent(Consumer)` would require an anonymous class. Use `Optional.isPresent()` + `Optional.get()` pattern. +- **Closure interface dispatch**: Different closure call sites use different functional interfaces: + - `tag({ ... })` → `SampleFamilyFunctions$TagFunction` + - `forEach(closure)` / `serviceRelation(closure)` etc. → `SampleFamilyFunctions$ForEachFunction` + - `instance(closure)` → `SampleFamilyFunctions$PropertiesExtractor` +- **No new v2 code in shared DSL classes**: New runtime behavior used by generated code goes in `MalRuntimeHelper` (in the `compiler.rt` package) to avoid FQCN conflicts with the v1 Groovy module which shares the same `dsl` package. ## Example
