Yang Jie created SPARK-58506:
--------------------------------
Summary: Assert Spark's own error contract in codegen compile
failure tests
Key: SPARK-58506
URL: https://issues.apache.org/jira/browse/SPARK-58506
Project: Spark
Issue Type: Test
Components: SQL
Affects Versions: 5.0.0
Reporter: Yang Jie
Two tests around codegen compile failures assert on the compiler's own
diagnostic
wording instead of what Spark itself produces.
*{{CodeGeneratorWithInterpretedFallbackSuite}}*, test "codegen failures in the
CODEGEN_ONLY
mode":
{code:scala}
assert(errMsg.contains("Failed to compile:
org.codehaus.commons.compiler.CompileException:"))
{code}
The class name here is Janino's. What Spark owns on this path is the wrapping -
a
source-level failure goes through {{QueryExecutionErrors.compilerError}}, whose
checked
{{CompileException}} the compile cache wraps in an {{ExecutionException}}
(SPARK-23711 /
SPARK-25140 established exactly this) - and the {{"Failed to compile: "}}
prefix that
{{failedToCompileMsg}} prepends. Asserting the cause's type states that
contract directly;
asserting the class name inside a message states it by accident.
*{{ObjectExpressionsSuite}}*, test "SPARK-23593: InitializeJavaBean should
support
interpreted execution":
{code:scala}
checkExceptionInExpression[Exception](initializeWithNonexistingMethod,
"""A method named "nonexistent" is not declared in any enclosing class """ +
"nor any supertype")
{code}
{{checkExceptionInExpression}} applies one substring to both the interpreted
and the codegen
path, so this string has to match both. It does, but only because the two
happen to overlap:
interpreted execution raises Spark's own {{INTERNAL_ERROR}} from
{{QueryExecutionErrors.methodNotDeclaredError}}, whose text is
{noformat}
[INTERNAL_ERROR] A method named "nonexistent" is not declared in any enclosing
class nor any supertype SQLSTATE: XX000
{noformat}
while the codegen path fails in the Java compiler, which reports
{noformat}
org.codehaus.commons.compiler.CompileException: File 'generated.java', Line 41,
Column 12: Failed to compile: org.codehaus.commons.compiler.CompileException:
File 'generated.java', Line 41, Column 12: A method named "nonexistent" is not
declared in any enclosing class nor any supertype, nor through a static import
{noformat}
Janino's diagnostic is a superstring of Spark's sentence, so one assertion
covers both legs.
Asserting each path against what raises it is more direct: the interpreted leg
has an error
class, parameters and a sqlState that {{checkError}} can pin exactly, which is
stronger than
the substring match it replaces.
The same test also has an assertion that never ran - the {{contains}} result is
discarded:
{code:scala}
intercept[Exception] {
evaluateWithoutCodegen(initializeWithWrongParamType,
InternalRow.fromSeq(Seq()))
}.getMessage.contains(
"""A method named "setX" is not declared in any enclosing class """ +
"nor any supertype")
{code}
It has been a bare expression since SPARK-23593 introduced it in 2018, so only
"an exception
was thrown" was ever verified. That leg raises the same
{{methodNotDeclaredError}}, so it can
use {{checkError}} as well.
Not in scope, found while auditing for similar sites:
{{DataFrameSuite.scala}}'s ignored test
"SPARK-19372: Filter can be executed w/o generated code due to JVM code size
limit" asserts
{{e.contains("grows beyond 64 KiB")}}, but Janino's string is {{Code grows
beyond 64 KB}} - no
released version spells it {{KiB}}. The assertion is already stale and only
survives because
the test is {{ignore}}d.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
