Ma77Ball opened a new pull request, #5048:
URL: https://github.com/apache/texera/pull/5048
### What changes were proposed in this PR?
`URLFetcherOpDesc.decodingMethod` defaults to `null` (from `var
decodingMethod: DecodingMethod = _`). `sourceSchema()` branched on `if
(decodingMethod == DecodingMethod.UTF_8) STRING
else ANY`, so a null silently produced an ANY column even though the JSON
schema marks the field `required = true`. This PR adds a
`require(decodingMethod != null, ...)` precondition
at the top of `sourceSchema()`, surfacing the misconfiguration as a clear
`IllegalArgumentException` instead of a silent ANY-typed output.
### Any related issues, documentation, or discussions?
Closes: #4815
### How was this PR tested?
Updated `URLFetcherOpDescSpec`: flipped the previously pinned "default to
ANY when decodingMethod is left unset" case to assert that `sourceSchema()` now
throws
`IllegalArgumentException`. Existing UTF-8 / RAW_BYTES / physical-op /
schema-propagation specs continue to pass. Ran `sbt "project WorkflowOperator"
"testOnly
org.apache.texera.amber.operator.source.fetcher.URLFetcherOpDescSpec"`,
all 7 tests pass.
### Was this PR authored or co-authored using generative AI tooling?
Co-authored with Claude Opus 4.7 in compliance with ASF
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
