andygrove opened a new issue, #5579:
URL: https://github.com/apache/datafusion-comet/issues/5579
Part of #5572.
`lpad` and `rpad` decline two shapes, and both fall the whole projection
back to Spark:
```scala
if (expr.str.isInstanceOf[Literal]) {
return Unsupported(Some(PadReasons.literalStrReason))
}
if (!expr.pad.isInstanceOf[Literal]) {
return Unsupported(Some(PadReasons.nonLiteralPadReason))
}
```
`spark/src/main/scala/org/apache/comet/serde/strings.scala:413-419`
(`CometStringRPad`) and `:441-447` (`CometStringLPad`)
The non-literal `pad` restriction is the one that matters. `lpad(name, 10,
pad_col)` — padding with a value that varies per row — is a perfectly ordinary
query shape, and there is nothing about it that Spark's generated code cannot
do. `StringRPad` and `StringLPad` are plain codegen expressions over
`StringType` and `IntegerType`, all of which
`CometBatchKernelCodegen.isSupportedDataType` admits.
The literal-`str` restriction is lower value (`ConstantFolding` handles most
of it) but costs nothing extra to cover.
Fix: mix `CodegenDispatchFallback` into both serdes.
Worth checking while in here: `docs/source/user-guide/latest/expressions.md`
renders `lpad` and `rpad` with an em-dash in the Implementation column, meaning
`GenerateDocs` found no serde registered under those function names. Either the
registry name differs from the function name or the rows need fixing — either
way the doc currently says nothing useful about how these execute.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
