andygrove opened a new issue, #192:
URL: https://github.com/apache/datafusion-spark/issues/192
Part of #173 and apache/datafusion-comet#7300.
### What the function does
`raise_error(expr)` throws a runtime error whose message is `expr` every
time it is evaluated. In SQL it
takes exactly one argument; the argument is implicitly cast to `STRING`
(`ImplicitCastInputTypes`), so
`raise_error(1)` raises with message `1`. Its declared return type is `VOID`
(`NullType`), it is nullable,
and it is never foldable (`foldable = false`), so it only fires for rows
that actually evaluate it.
Available since Spark 3.1.0. No aliases. `assert_true` lowers to `If(cond,
NULL, raise_error(msg))`.
In 4.0+ the expression is internally `RaiseError(errorClass, errorParms)`.
The SQL form builds
`errorClass = 'USER_RAISED_EXCEPTION'` and `errorParms = map('errorMessage',
expr)`, giving a
`SparkRuntimeException` with error condition `USER_RAISED_EXCEPTION`
(SQLSTATE `P0001`, message template
`<errorMessage>`), so the rendered message is `[USER_RAISED_EXCEPTION]
<expr> SQLSTATE: P0001`.
In 3.4/3.5 it is a plain `java.lang.RuntimeException(expr)` with no error
class.
### Examples
```sql
SELECT raise_error('custom error message');
-- [USER_RAISED_EXCEPTION] custom error message
-- tbl_misc(v) = 1, 8, 2: only the row that takes the branch raises
SELECT if(v > 5, raise_error('too big: ' || v), v + 1) FROM tbl_misc;
-- SparkRuntimeException USER_RAISED_EXCEPTION, errorMessage "too big: 8"
SELECT raise_error(NULL);
-- USER_RAISED_EXCEPTION, errorMessage "null"
SELECT raise_error(1);
-- USER_RAISED_EXCEPTION, errorMessage "1"
SELECT raise_error('error message', Map());
-- AnalysisException WRONG_NUM_ARGS.WITHOUT_SUGGESTION (expectedNum 1,
actualNum 2)
SELECT raise_error();
-- AnalysisException WRONG_NUM_ARGS.WITHOUT_SUGGESTION (expectedNum 1,
actualNum 0)
-- spark.sql.legacy.raiseErrorWithoutErrorClass=true
SELECT raise_error('hello');
-- java.lang.RuntimeException: hello
```
### Edge cases to test
- NULL message: 4.x formats the parameter as the string `null`; 3.4/3.5
throw `RuntimeException()` with no
message.
- Non-string message (`INT`, `DOUBLE`, `DATE`, ...): cast to `STRING` with
Spark's cast rules first.
- Message from a column (per-row messages), and empty string message.
- Laziness: must not raise during planning or constant folding, even with a
literal argument, and must not
raise for rows where an enclosing `CASE`/`IF`/`COALESCE`/`AND`/`OR` does
not evaluate it.
- Zero input rows: no error (`SELECT raise_error('x') FROM t WHERE false`).
- Result type `VOID`/`Null`, so it can sit in a `CASE` branch alongside any
other type.
- Wrong arity raises `WRONG_NUM_ARGS.WITHOUT_SUGGESTION` (analysis time, not
runtime).
- `spark.sql.legacy.raiseErrorWithoutErrorClass` (internal, since 4.0,
default `false`): when `true` the
error class becomes `_LEGACY_ERROR_USER_RAISED_EXCEPTION` and a bare
`RuntimeException(message)` is
thrown without the `[USER_RAISED_EXCEPTION]` prefix.
- Collated string input is accepted in 4.x
(`StringTypeWithCollation(supportsTrimCollation = true)`).
### Differences across Spark versions
- 3.4, 3.5: `RaiseError(child: STRING)`, throws
`java.lang.RuntimeException(message)`; NULL message gives a
`RuntimeException` with a null message.
- 4.0+: `RaiseError(errorClass, errorParms)`; SQL form raises
`USER_RAISED_EXCEPTION` (`P0001`) with
`errorMessage` set to the message, or `"null"` for NULL. Unknown classes /
missing parameters in the
internal two-argument form raise
`USER_RAISED_EXCEPTION_UNKNOWN_ERROR_CLASS` /
`USER_RAISED_EXCEPTION_PARAMETER_MISMATCH`. Adds
`spark.sql.legacy.raiseErrorWithoutErrorClass`.
- 4.1, 4.2: no behavior change found for `raise_error` itself.
### Implementation notes
- Spark:
`sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/misc.scala`,
classes
`RaiseError` and `RaiseErrorExpressionBuilder`; error construction in
`sql/catalyst/src/main/scala/org/apache/spark/sql/errors/QueryExecutionErrors.scala`
(`raiseError`).
- DataFusion core has no equivalent.
- The UDF must be `Volatility::Volatile` (or otherwise excluded from
simplification) so DataFusion's
constant folder does not evaluate `raise_error('x')` at plan time.
- Produce the error in the same `[CONDITION] message SQLSTATE: xxxxx` style
used elsewhere in the crate
(e.g. `crates/spark/src/function/math/abs.rs`), and decide how the legacy
flag is passed (e.g. a
constructor option). Belongs in the `misc` module next to `equal_null`.
- Tests: add `crates/sqllogictest/test_files/misc/raise_error.slt` (use
`statement error` /
`query error` with the expected message), and document with `#[user_doc]`.
---
_This issue was generated by an LLM (Claude) from the Apache Spark source
code and tests. Examples come from Spark's `ExpressionDescription` examples,
test suites and `.sql.out` golden files, but please verify them against Spark
before relying on them._
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]