This is an automated email from the ASF dual-hosted git repository. wusheng pushed a commit to branch oal-v2 in repository https://gitbox.apache.org/repos/asf/skywalking.git
commit a1fd4af66ddc52ea36d1d06517c74d8a4ca03a9c Author: Wu Sheng <[email protected]> AuthorDate: Tue Feb 10 16:00:33 2026 +0800 Move JDK 11 compatibility to CLAUDE.md and update OAL V2 README - Add JDK 11 compatibility section to CLAUDE.md as project-level standard - Document prohibited Java features (switch expressions, Stream.toList(), etc.) - Document allowed features (List.of(), lambdas, Lombok) - Add verification commands - Remove JDK11_COMPATIBILITY.md (content moved to CLAUDE.md) - Simplify OAL V2 README.md: - Update package structure to reflect current state - Add pipeline diagram - Remove verbose examples (documented in source files) - Add generated source files section Co-Authored-By: Claude Opus 4.5 <[email protected]> --- CLAUDE.md | 34 ++ .../skywalking/oal/v2/JDK11_COMPATIBILITY.md | 152 -------- .../java/org/apache/skywalking/oal/v2/README.md | 415 +++------------------ 3 files changed, 96 insertions(+), 505 deletions(-) diff --git a/CLAUDE.md b/CLAUDE.md index e8e492e9bd..a8ef6f7a16 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -182,6 +182,40 @@ public class XxxModuleProvider extends ModuleProvider { Java, XML, and YAML/YML files must include the Apache 2.0 license header (see `HEADER` file). JSON and Markdown files are excluded (JSON doesn't support comments, see `.licenserc.yaml`). +### JDK 11 Compatibility + +All code must be compatible with JDK 11 (LTS). The project supports JDK 11, 17, and 21. + +**Prohibited Java features (post-JDK 11):** + +| Feature | JDK Version | Use Instead | +|---------|-------------|-------------| +| Switch expressions (`->`) | 14+ | Traditional `switch` with `case:` and `break` | +| `Stream.toList()` | 16+ | `.collect(Collectors.toList())` | +| Text blocks (`"""..."""`) | 15+ | String concatenation or `+` | +| Records | 14+ | Regular classes with Lombok `@Data` | +| Pattern matching for `instanceof` | 14+ | Traditional cast after `instanceof` | +| Sealed classes/interfaces | 15+ | Regular classes/interfaces | + +**Allowed Java features (JDK 11 compatible):** +- `List.of()`, `Set.of()`, `Map.of()` - Immutable collections (Java 9+) +- `Optional` methods - `orElseThrow()`, `ifPresentOrElse()` (Java 9+) +- Lambda expressions and method references (Java 8+) +- Stream API (Java 8+) +- Lombok annotations (`@Getter`, `@Builder`, `@Data`, `@Slf4j`) + +**Verification commands:** +```bash +# Check for switch expressions (should return no matches) +grep -r "switch.*->" src/ --include="*.java" + +# Check for Stream.toList() (should return no matches) +grep -r "\.toList()" src/ --include="*.java" + +# Check for text blocks (should return no matches) +grep -r '"""' src/ --include="*.java" +``` + ## Testing ### Test Frameworks diff --git a/oap-server/oal-rt/src/main/java/org/apache/skywalking/oal/v2/JDK11_COMPATIBILITY.md b/oap-server/oal-rt/src/main/java/org/apache/skywalking/oal/v2/JDK11_COMPATIBILITY.md deleted file mode 100644 index 1f2b0f7c87..0000000000 --- a/oap-server/oal-rt/src/main/java/org/apache/skywalking/oal/v2/JDK11_COMPATIBILITY.md +++ /dev/null @@ -1,152 +0,0 @@ -# JDK 11 Compatibility - -All V2 code is compatible with JDK 11 (LTS). - -## Fixed Issues - -### ✅ Switch Expressions (Java 14+) → Traditional Switch - -**Fixed in:** -- `FilterValue.java` (line 172) -- `FunctionArgument.java` (line 127) - -**Before (Java 14+):** -```java -return switch (type) { - case NUMBER -> String.valueOf(value); - case STRING -> "\"" + value + "\""; - case BOOLEAN -> String.valueOf(value); - case NULL -> "null"; - case ARRAY -> value.toString(); -}; -``` - -**After (Java 11 compatible):** -```java -switch (type) { - case NUMBER: - return String.valueOf(value); - case STRING: - return "\"" + value + "\""; - case BOOLEAN: - return String.valueOf(value); - case NULL: - return "null"; - case ARRAY: - return value.toString(); - default: - throw new IllegalStateException("Unknown type: " + type); -} -``` - -### ✅ Stream.toList() (Java 16+) → collect(Collectors.toList()) - -**Fixed in:** -- `MetricsFunctionRegistry.java` (line 73) - -**Before (Java 16+):** -```java -return getAllFunctions().stream() - .map(MetricsFunctionDescriptor::getName) - .toList(); -``` - -**After (Java 11 compatible):** -```java -return getAllFunctions().stream() - .map(MetricsFunctionDescriptor::getName) - .collect(Collectors.toList()); -``` - -## Java Features Used (All JDK 11 Compatible) - -### ✅ Java 9+ -- `List.of()` - Immutable list creation -- `Optional.ofNullable()` - Optional handling -- Interface private methods - Not used - -### ✅ Java 11 -- Local variable type inference (`var`) - Not used (for clarity) -- String methods (`isBlank()`, `strip()`) - Not used -- Files methods - Not used - -### ✅ Always Compatible -- Enums with methods -- Builder pattern -- Lombok annotations (`@Getter`, `@EqualsAndHashCode`) -- Immutable collections (`Collections.unmodifiableList()`) -- Generics and type inference -- Lambda expressions -- Method references -- Stream API - -## Features NOT Used (Post-JDK 11) - -### ❌ Java 12+ -- ❌ Switch expressions (arrow syntax) - **FIXED** -- ❌ `String.indent()`, `String.transform()` - -### ❌ Java 13+ -- ❌ Text blocks (`"""..."""`) - -### ❌ Java 14+ -- ❌ Records -- ❌ Pattern matching for `instanceof` - -### ❌ Java 15+ -- ❌ Sealed classes - -### ❌ Java 16+ -- ❌ `Stream.toList()` - **FIXED** -- ❌ Pattern matching for `switch` - -### ❌ Java 17+ -- ❌ Enhanced switch -- ❌ Sealed interfaces - -## Verification - -All V2 code uses only JDK 11 compatible features: - -```bash -# Verify no Java 14+ switch expressions -grep -r "switch.*->" src/ --include="*.java" -# Result: No matches ✓ - -# Verify no text blocks -grep -r '"""' src/ --include="*.java" -# Result: No matches ✓ - -# Verify no records -grep -r "record " src/ --include="*.java" -# Result: No matches ✓ - -# Verify no Stream.toList() -grep -r "\.toList()" src/ --include="*.java" -# Result: No matches ✓ -``` - -## Build Requirements - -- **JDK**: 11, 17, or 21 (LTS versions) -- **Maven**: 3.6+ -- **Lombok**: Compatible with all JDK versions - -## IDE Configuration - -For IntelliJ IDEA: -- Set Project SDK: Java 11 -- Set Language Level: 11 - Local variable syntax for lambda parameters -- Enable Lombok plugin - -For Eclipse: -- Set Compiler Compliance Level: 11 -- Install Lombok plugin - -## Summary - -✅ All V2 code is **fully compatible with JDK 11** -✅ No Java 12+ features used -✅ Build succeeds on JDK 11, 17, and 21 -✅ All lombok annotations are JDK 11 compatible -✅ Code follows SkyWalking coding standards diff --git a/oap-server/oal-rt/src/main/java/org/apache/skywalking/oal/v2/README.md b/oap-server/oal-rt/src/main/java/org/apache/skywalking/oal/v2/README.md index 81bb2d6787..1cbe288e83 100644 --- a/oap-server/oal-rt/src/main/java/org/apache/skywalking/oal/v2/README.md +++ b/oap-server/oal-rt/src/main/java/org/apache/skywalking/oal/v2/README.md @@ -1,175 +1,67 @@ +# OAL V2 Engine -# OAL V2 Architecture - -This package contains the OAL (Observability Analysis Language) engine implementation. V2 is the only implementation after V1 has been completely removed. +The OAL (Observability Analysis Language) engine for Apache SkyWalking. This is the only OAL implementation. ## Package Structure ``` org.apache.skywalking.oal.v2/ -├── model/ # Immutable data models -│ ├── SourceLocation.java # Location in source file -│ ├── SourceReference.java # from(Service.latency) -│ ├── FunctionCall.java # longAvg(), percentile2(10) -│ ├── FunctionArgument.java # Typed function arguments -│ ├── FilterOperator.java # Enum: ==, !=, >, <, like, in -│ ├── FilterExpression.java # latency > 100 -│ ├── FilterValue.java # Typed filter values -│ └── MetricDefinition.java # Complete metric definition -├── registry/ # Service registries -│ ├── MetricsFunctionRegistry.java -│ └── MetricsFunctionDescriptor.java +├── model/ # Immutable data models (parser output) +│ ├── SourceLocation # Location in source file for error reporting +│ ├── SourceReference # from(Service.latency) +│ ├── FunctionCall # longAvg(), percentile2(10) +│ ├── FunctionArgument # Typed function arguments +│ ├── FilterOperator # Enum: ==, !=, >, <, like, in +│ ├── FilterExpression # latency > 100 +│ ├── FilterValue # Typed filter values +│ └── MetricDefinition # Complete parsed metric ├── parser/ # OAL script parsing -│ ├── OALListenerV2.java # ANTLR parse tree listener -│ └── OALScriptParserV2.java # Parser facade +│ ├── OALListenerV2 # ANTLR parse tree listener +│ └── OALScriptParserV2 # Parser facade ├── generator/ # Code generation -│ ├── CodeGenModel.java # Code generation data model -│ ├── MetricDefinitionEnricher.java # Metadata enrichment -│ └── OALClassGeneratorV2.java # Javassist code generator -├── metadata/ # Metadata utilities -│ ├── SourceColumnsFactory.java # Source field lookup -│ ├── SourceColumn.java # Source field representation -│ ├── FilterMatchers.java # Filter expression matchers -│ └── MetricsHolder.java # Metrics function registry +│ ├── CodeGenModel # Code generation data model +│ ├── MetricDefinitionEnricher # Metadata enrichment +│ ├── OALClassGeneratorV2 # Javassist bytecode generator +│ └── OALSourceGenerator # Source file generator (for debugging) +├── metadata/ # Source/metrics metadata utilities +│ ├── SourceColumnsFactory +│ ├── SourceColumn +│ ├── FilterMatchers +│ └── MetricsHolder ├── util/ # Code generation utilities -│ ├── ClassMethodUtil.java # Method name utilities -│ └── TypeCastUtil.java # Type casting helpers -└── OALEngineV2.java # Main V2 engine +│ ├── ClassMethodUtil +│ └── TypeCastUtil +└── OALEngineV2 # Main engine entry point ``` -## Design Principles - -1. **Immutable Data Models**: All model classes are immutable and thread-safe -2. **Builder Pattern**: Complex objects use fluent builders -3. **Separation of Concerns**: Data ≠ Logic -4. **Dependency Injection**: No static singletons, registries are injectable -5. **Type Safety**: Use enums and types instead of strings where possible - -## Model Classes - -### SourceLocation - -Represents a location in OAL source file for error reporting. +## Pipeline -```java -SourceLocation location = SourceLocation.of("core.oal", 25, 10); -System.out.println(location); // "core.oal:25:10" ``` - -### SourceReference - -Immutable representation of a source reference in OAL. - -```java -// from(Service.latency) -SourceReference source = SourceReference.builder() - .name("Service") - .addAttribute("latency") - .build(); - -// from((long)Service.tag["key"]) -SourceReference tagged = SourceReference.builder() - .name("Service") - .addAttribute("tag[key]") - .castType("long") - .build(); - -// from(Service.*) -SourceReference wildcard = SourceReference.builder() - .name("Service") - .wildcard(true) - .build(); +┌─────────────┐ ┌─────────────┐ ┌─────────────┐ ┌─────────────┐ +│ .oal file │───▶│ Parser │───▶│ Enricher │───▶│ Generator │ +│ │ │ │ │ │ │ │ +│ OAL script │ │ MetricDef │ │ CodeGenModel│ │ Bytecode/ │ +│ │ │ (immutable) │ │ (metadata) │ │ Source │ +└─────────────┘ └─────────────┘ └─────────────┘ └─────────────┘ ``` -### FilterExpression & FilterValue - -Immutable filter expression with typed values. +1. **Parser** (`OALScriptParserV2`): Parses `.oal` files into immutable `MetricDefinition` objects +2. **Enricher** (`MetricDefinitionEnricher`): Adds metadata via reflection (source columns, persistent fields) +3. **Generator** (`OALClassGeneratorV2`): Generates Java classes using Javassist and FreeMarker templates -```java -// latency > 100 (typed as NUMBER) -FilterExpression filter = FilterExpression.builder() - .fieldName("latency") - .operator(FilterOperator.GREATER) - .numberValue(100L) - .build(); - -// status == true (typed as BOOLEAN) -FilterExpression filter2 = FilterExpression.builder() - .fieldName("status") - .operator(FilterOperator.EQUAL) - .booleanValue(true) - .build(); - -// name like "serv%" (typed as STRING) -FilterExpression filter3 = FilterExpression.builder() - .fieldName("name") - .operator(FilterOperator.LIKE) - .stringValue("serv%") - .build(); - -// code in [404, 500, 503] (typed as ARRAY) -FilterExpression filter4 = FilterExpression.builder() - .fieldName("code") - .operator(FilterOperator.IN) - .arrayValue(List.of(404L, 500L, 503L)) - .build(); - -// Or use shorthand with auto-type detection: -FilterExpression filter5 = FilterExpression.of("latency", ">", 100L); - -// Access typed value: -FilterValue value = filter.getValue(); -if (value.isNumber()) { - long num = value.asLong(); -} -``` - -### FunctionCall & FunctionArgument - -Represents an aggregation function call with typed arguments. - -```java -// longAvg() - no arguments -FunctionCall avgFunc = FunctionCall.of("longAvg"); - -// percentile2(10) - literal argument -FunctionCall percentile = FunctionCall.builder() - .name("percentile2") - .addLiteral(10) - .build(); - -// apdex(name, status) - attribute arguments -FunctionCall apdex = FunctionCall.builder() - .name("apdex") - .addAttribute("name") - .addAttribute("status") - .build(); - -// rate(status == true, count) - expression and attribute -FunctionCall rate = FunctionCall.builder() - .name("rate") - .addExpression(FilterExpression.of("status", "==", true)) - .addAttribute("count") - .build(); +## Design Principles -// Or use shorthand for simple literals: -FunctionCall histogram = FunctionCall.ofLiterals("histogram", 100, 20); +1. **Immutable Models**: All parser output classes are immutable and thread-safe +2. **Type Safety**: Use enums and typed values instead of strings +3. **Builder Pattern**: Complex objects use fluent builders +4. **Separation of Concerns**: Parser models ≠ Code generation models +5. **Testability**: Models can be constructed without parsing for unit tests -// Access typed arguments: -for (FunctionArgument arg : percentile.getArguments()) { - if (arg.isLiteral()) { - Object value = arg.asLiteral(); - } else if (arg.isAttribute()) { - String fieldName = arg.asAttribute(); - } else if (arg.isExpression()) { - FilterExpression expr = arg.asExpression(); - } -} -``` +## Key Classes -### MetricDefinition +### MetricDefinition (Parser Output) -Complete metric definition combining all components. +Immutable representation of a parsed OAL metric: ```java // service_resp_time = from(Service.latency).filter(latency > 0).longAvg() @@ -178,220 +70,37 @@ MetricDefinition metric = MetricDefinition.builder() .source(SourceReference.of("Service", "latency")) .addFilter(FilterExpression.of("latency", ">", 0L)) .aggregationFunction(FunctionCall.of("longAvg")) - .location(SourceLocation.of("core.oal", 20, 1)) .build(); - -System.out.println(metric.getName()); // "service_resp_time" -System.out.println(metric.getTableName()); // "service_resp_time" -System.out.println(metric.getSource()); // "Service.latency" -System.out.println(metric.getFilters()); // [latency > 0] ``` -## Registry Interfaces - -### MetricsFunctionRegistry +### CodeGenModel (Generator Input) -Service interface for looking up metrics functions. +Enriched model with metadata for code generation: ```java -// Usage (will be injected, not created directly in production) -MetricsFunctionRegistry registry = ...; - -Optional<MetricsFunctionDescriptor> longAvg = registry.findFunction("longAvg"); -if (longAvg.isPresent()) { - Class<? extends Metrics> metricsClass = longAvg.get().getMetricsClass(); - Method entranceMethod = longAvg.get().getEntranceMethod(); -} +// Created by MetricDefinitionEnricher +CodeGenModel model = enricher.enrich(metricDefinition); -// List all functions -List<String> functionNames = registry.getFunctionNames(); -// ["longAvg", "count", "cpm", "percentile2", ...] +// Contains: source columns, persistent fields, metrics class info, etc. +model.getFieldsFromSource(); // Fields copied from source +model.getPersistentFields(); // Fields for storage +model.getMetricsClassName(); // e.g., "LongAvgMetrics" ``` -## Example: Building Complete Metrics +## Generated Source Files -### Example 1: Simple Count with Filter - -```java -// OAL: endpoint_error_count = from(Endpoint.*).filter(status == false).count() +During tests, source files are generated to `target/generated-test-sources/` for inspection: -MetricDefinition errorCount = MetricDefinition.builder() - .name("endpoint_error_count") - .source(SourceReference.builder() - .name("Endpoint") - .wildcard(true) - .build()) - .addFilter(FilterExpression.builder() - .fieldName("status") - .operator(FilterOperator.EQUAL) - .booleanValue(false) - .build()) - .aggregationFunction(FunctionCall.of("count")) - .location(SourceLocation.of("core.oal", 42, 1)) - .build(); - -// Access typed filter value -FilterValue value = errorCount.getFilters().get(0).getValue(); -assert value.isBoolean(); -assert value.asBoolean() == false; -``` - -### Example 2: Percentile with Numeric Filter - -```java -// OAL: service_slow_percentile = from(Service.latency).filter(latency > 1000).percentile2(10) - -MetricDefinition slowPercentile = MetricDefinition.builder() - .name("service_slow_percentile") - .source(SourceReference.of("Service", "latency")) - .addFilter(FilterExpression.builder() - .fieldName("latency") - .operator(FilterOperator.GREATER) - .numberValue(1000L) - .build()) - .aggregationFunction(FunctionCall.builder() - .name("percentile2") - .addLiteral(10) - .build()) - .build(); - -// Access typed argument -FunctionArgument arg = slowPercentile.getAggregationFunction().getArguments().get(0); -assert arg.isLiteral(); -assert arg.asLiteral().equals(10); -``` - -### Example 3: Apdex with Attribute Arguments - -```java -// OAL: service_apdex = from(Service.latency).apdex(name, status) - -MetricDefinition apdexMetric = MetricDefinition.builder() - .name("service_apdex") - .source(SourceReference.of("Service", "latency")) - .aggregationFunction(FunctionCall.builder() - .name("apdex") - .addAttribute("name") - .addAttribute("status") - .build()) - .build(); - -// Access typed arguments -List<FunctionArgument> args = apdexMetric.getAggregationFunction().getArguments(); -assert args.get(0).isAttribute(); -assert args.get(0).asAttribute().equals("name"); -assert args.get(1).isAttribute(); -assert args.get(1).asAttribute().equals("status"); ``` - -These immutable, type-safe objects can be passed through the pipeline: -**Parse → Semantic Analysis → Validation → Code Generation** - -## Key Features - -1. **Type Safety**: Strongly typed arguments and values - - `FunctionArgument` distinguishes LITERAL vs ATTRIBUTE vs EXPRESSION - - `FilterValue` distinguishes NUMBER vs STRING vs BOOLEAN vs NULL vs ARRAY - - Compile-time safety instead of runtime casting errors - -2. **Testability**: Models can be constructed without parsing - - Build test data with fluent builders - - No dependency on ANTLR parser - - Easy to mock and verify - -3. **Immutability**: Thread-safe, no hidden state changes - - All fields are `final` - - Collections are unmodifiable - - No setters, only builders - -4. **Clarity**: Explicit builders, no magic - - Clear method names (`addLiteral()` vs `addAttribute()`) - - Type-specific accessors (`asLong()`, `asString()`) - - Self-documenting code - -5. **Debuggability**: Clear toString() representations - - Human-readable output - - Type information included - - Easy to inspect in debugger - -6. **Validation**: Type checking at construction time - - Invalid types rejected early - - Clear error messages - - Fail fast, not at code generation - -## Usage - -V2 is now the only OAL implementation. The engine processes OAL scripts through the following pipeline: - -1. **Parse**: OALScriptParserV2 parses .oal files into immutable MetricDefinition objects -2. **Enrich**: MetricDefinitionEnricher adds metadata (source columns, persistent fields, etc.) -3. **Generate**: OALClassGeneratorV2 generates Java classes using Javassist and FreeMarker templates - -## Implementation Status - -✅ **All phases completed**: - -1. ✅ **Immutable Model Classes** - - Type-safe data models with builder patterns - - Source location tracking for error reporting - -2. ✅ **Parser Implementation** - - Full OAL grammar support via OALListenerV2 - - Converts parse tree to immutable V2 models - - Validates all production OAL scripts - -3. ✅ **Metadata Enrichment** - - MetricDefinitionEnricher extracts source metadata - - Reflects on metrics function classes - - Builds code generation models - -4. ✅ **Code Generation** - - OALClassGeneratorV2 with V2-specific FreeMarker templates - - Generates metrics, builder, and dispatcher classes - - Uses Javassist for bytecode generation - -5. ✅ **V1 Removal** - - All V1 implementation code removed - - V2 is the only OAL engine implementation - - Utilities reorganized under v2 package structure - -## Testing - -All V2 models are designed for easy unit testing: - -```java -@Test -public void testMetricDefinitionBuilder() { - MetricDefinition metric = MetricDefinition.builder() - .name("test_metric") - .source(SourceReference.of("Service")) - .aggregationFunction(FunctionCall.of("count")) - .build(); - - assertEquals("test_metric", metric.getName()); - assertEquals("Service", metric.getSource().getName()); - assertEquals("count", metric.getAggregationFunction().getName()); -} - -@Test -public void testFilterExpression() { - FilterExpression filter = FilterExpression.of("latency", ">", 100L); - - assertEquals("latency", filter.getFieldName()); - assertEquals(FilterOperator.GREATER, filter.getOperator()); - assertEquals(100L, filter.getValue()); - assertEquals("latency > 100", filter.toString()); -} +target/generated-test-sources/ +└── org/apache/skywalking/oap/server/core/source/oal/rt/ + ├── metrics/ - Generated metrics classes + │ └── builder/ - Generated builder classes + └── dispatcher/ - Generated dispatcher classes ``` -## Architecture - -The OAL engine follows a clean pipeline architecture: +These files are 100% consistent with the Javassist bytecode loaded into JVM (verified by `SourceBytecodeConsistencyTest`). -1. **OALEngineV2** - Main engine orchestrator (implements OALEngine interface) -2. **OALScriptParserV2** - Parses .oal scripts into immutable MetricDefinition objects -3. **MetricDefinitionEnricher** - Enriches definitions with metadata via reflection -4. **OALClassGeneratorV2** - Generates Java classes using Javassist and FreeMarker -5. **Runtime Integration** - Generated classes integrate with SkyWalking's stream processing +## Runtime Integration -The engine is loaded via reflection in OALEngineLoaderService because server-core compiles before oal-rt in the Maven reactor. +The engine is loaded via reflection in `OALEngineLoaderService` because `server-core` compiles before `oal-rt` in the Maven reactor. Generated classes integrate with SkyWalking's stream processing pipeline.
