JoshRosen commented on a change in pull request #25745: [SPARK-29033][SQL][WIP]
Always use UnsafeRow-based version of CreateNamedStruct
URL: https://github.com/apache/spark/pull/25745#discussion_r322824766
##########
File path:
sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/expressions/ExpressionEvalHelper.scala
##########
@@ -138,6 +138,10 @@ trait ExpressionEvalHelper extends
GeneratorDrivenPropertyChecks with PlanTestBa
case (result: Float, expected: Float) =>
if (expected.isNaN) result.isNaN else expected == result
case (result: Row, expected: InternalRow) => result.toSeq ==
expected.toSeq(result.schema)
+ case (result: Seq[InternalRow], expected: Seq[InternalRow]) =>
Review comment:
This change was needed because we can't do direct `equals()` comparison
between UnsafeRow and other row classes. After this PR's changes, the
_"SPARK-14793: split wide struct creation into blocks due to JVM code size
limit"_ test case in `CodeGenerationSuite` was failing because the new code was
producing `UnsafeRow` but the test code was comparing against
`GenericInternalRow`. In the old code, this comparison between sequences of
rows was happening in the default `case _ =>` below, but that case doesn't work
when the `InternalRow` implementations are mismatched.
I'm not sure whether this change-of-internal-row-format will have adverse
consequences in other parts of the code.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
