Dennis Du created SPARK-38592:
---------------------------------
Summary: Column name contains back tick `
Key: SPARK-38592
URL: https://issues.apache.org/jira/browse/SPARK-38592
Project: Spark
Issue Type: Bug
Components: SQL
Affects Versions: 3.0.1
Reporter: Dennis Du
Try to modify the data frame to ensure column names have no special characters.
{code:java}
val newdf = df.select(
df.columns.map { columnName =>
df.col(surroundingBackTickedName(columnName)).as(normalizeName(columnName))
}: _*
){code}
col() kept having issue with column name contains back tick
{code:java}
org.apache.spark.sql.AnalysisException: Cannot resolve column name
"`COLNAME`2`" among (COLID, COLNAME!4, COLNAME#6, COLNAME$7, COLNAME%8,
COLNAME'25, COLNAME(11, COLNAME)12, COLNAME*10, COLNAME+16, COLNAME,26,
COLNAME-13, COLNAME/30, COLNAME:24, COLNAME;23, COLNAME<27, COLNAME=15,
COLNAME>29, COLNAME?31, COLNAME@5, COLNAME`2){code}
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
