xiangfu0 opened a new pull request, #19235:
URL: https://github.com/apache/pinot/pull/19235
## Summary
- Centralize ingestion Groovy policy enforcement in
`FunctionEvaluatorFactory`, with explicit per-context policy propagation and
fail-closed defaults.
- Apply the same policy to schema FieldSpec transforms and table-config
transforms during controller validation.
- Enforce the policy before evaluator construction in realtime server
ingestion, offline/minion segment generation, and standalone, Spark, and Hadoop
batch ingestion.
- Preserve explicitly enabled Groovy, built-in transforms, and legacy
implicit map transform behavior.
- Move authorization ahead of semantic validation for the affected schema
and combined table-config validation endpoints.
## Root cause
`controller.disable.ingestion.groovy` was applied only to table-config
validation through `TableConfigUtils`. Schema FieldSpec validation and runtime
transform pipelines called the common evaluator factory directly, and the
factory constructed Groovy evaluators without consulting the ingestion policy.
## How to reproduce
1. Start a controller with the default configuration, leaving
`controller.disable.ingestion.groovy` unset.
2. Submit a schema through `POST /schemas` or `POST /schemas/validate`
containing a FieldSpec transform such as:
```json
{
"schemaName": "groovy_field_spec",
"dimensionFieldSpecs": [
{"name": "source", "dataType": "STRING"},
{
"name": "derived",
"dataType": "STRING",
"transformFunction": "Groovy({source.reverse()}, source)"
}
]
}
```
Before this change, schema validation could construct and compile the Groovy
evaluator even though ingestion Groovy was disabled. Persisted schemas could
also reach runtime evaluator construction on server or segment-generation paths.
After this change, the request fails with a configuration error before
Groovy compilation. Setting `controller.disable.ingestion.groovy=false`
explicitly preserves the existing enabled behavior.
## Behavior and compatibility
- Missing, blank, or invalid policy values remain disabled.
- Explicit `false` enables Groovy consistently across validation and runtime
roles.
- Non-Groovy built-in transforms are unchanged.
- Pre-existing serialized segment-generation configs fail closed, while new
direct callers retain explicit opt-in behavior.
- Query-time Groovy policy is unchanged.
## Validation
- 192 targeted common, schema, transform-pipeline, realtime, and offline
segment-generation tests.
- 41 controller tests covering schema POST, PUT, validate, combined
tableConfigs, and authorization ordering.
- 55 built-in minion task tests.
- 4 Spark and Hadoop segment-generation tests.
- `spotless:apply`, `license:format`, `checkstyle:check`, and
`license:check` passed for all affected modules.
- JDK 25 affected-module `test-compile` passed with
`-Xlint:all,-deprecation`.
The exact JDK 25 `-Xlint:all` invocation remains blocked on current master
by `zstd-jni` referencing its provided JetBrains `@NotNull` dependency from the
unchanged `ZstandardDecompressor` source.
## Scope
This change does not alter schema authorization actions or query-time Groovy
behavior. The PR is intentionally draft for coordinated review.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
