rshkv opened a new pull request, #40794:
URL: https://github.com/apache/spark/pull/40794
### What changes were proposed in this pull request?
This PR fixes DSL expressions on attributes with special characters by
making `DslAttr.attr` and `DslAttr.expr` return the implicitly wrapped
attribute instead of creating a new one.
### Why are the changes needed?
DSL expressions on attributes with special characters don't work even if the
attribute names are quoted:
```scala
scala> import org.apache.spark.sql.catalyst.dsl.expressions._
import org.apache.spark.sql.catalyst.dsl.expressions._
scala> "`slashed/col`".attr
res0: org.apache.spark.sql.catalyst.analysis.UnresolvedAttribute =
'slashed/col
scala> "`slashed/col`".attr.asc
org.apache.spark.sql.catalyst.parser.ParseException:
mismatched input '/' expecting {<EOF>, '.', '-'}(line 1, pos 7)
== SQL ==
slashed/col
-------^^^
```
DSL expressions rely on a call to `expr` to get child of the new expression
[(e.g.)](https://github.com/apache/spark/blob/87a5442f7ed96b11051d8a9333476d080054e5a0/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/dsl/package.scala#L149).
`expr` here is a call on implicit class `DslAttr` that's wrapping the
`UnresolvedAttribute` returned by `"...".attr` is wrapped by the implicit class
`DslAttr`.
`DslAttr` and its super class implement `DslAttr.expr` such that a new
`UnresolvedAttribute` is created from `UnresolvedAttribute.name` of the wrapped
attribute
[(here)](https://github.com/apache/spark/blob/87a5442f7ed96b11051d8a9333476d080054e5a0/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/dsl/package.scala#L273-L280).
But that's no good, because `UnresolvedAttribute.name` drops the quotes and
thus the newly created `UnresolvedAttribute` parses an identifier that should
be quoted but isn't:
```scala
scala> "`col/slash`".attr.name
res5: String = col/slash
```
### Does this PR introduce _any_ user-facing change?
DSL expressions on attributes with special characters no longer fail.
### How was this patch tested?
I couldn't find a suite testing the implicit classes in the DSL package, but
the DSL package seems used widely enough that I'm confident this doesn't break
existing behavior.
Locally, I was able to reproduce with this test; it was failing before and
passes now:
```scala
test("chained DSL expressions on attributes with special characters") {
$"`slashed/col`".asc
}
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
