[ https://issues.apache.org/jira/browse/SPARK-38592?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17508536#comment-17508536 ]
Hyukjin Kwon commented on SPARK-38592: -------------------------------------- [~JuzDDM] mind showing the fully self-contained reproducer? > Column name contains back tick ` > -------------------------------- > > Key: SPARK-38592 > URL: https://issues.apache.org/jira/browse/SPARK-38592 > Project: Spark > Issue Type: Bug > Components: SQL > Affects Versions: 3.0.1 > Reporter: Dennis Du > Priority: Major > > Try to modify the data frame to ensure column names have no special > characters. > {code:java} > df.columns.map { columnName => > df.col(surroundingBackTickedName(columnName)).as(normalizeName(columnName)) > } > {code} > *surroundingBackTickedName()* will enclose column name with backticks. > However, col() kept having issue with column name contains back tick becauseĀ > {*}parseAttributeName{*}() only takes backticks that appear in pair. I am > wondering if there is a workaround > {code:java} > org.apache.spark.sql.AnalysisException: Cannot resolve column name > "`COLNAME`2`" among (COLID, COLNAME!4, COLNAME#6, COLNAME$7, COLNAME%8, > COLNAME'25, COLNAME(11, COLNAME)12, COLNAME*10, COLNAME+16, COLNAME,26, > COLNAME-13, COLNAME/30, COLNAME:24, COLNAME;23, COLNAME<27, COLNAME=15, > COLNAME>29, COLNAME?31, COLNAME@5, COLNAME`2){code} -- This message was sent by Atlassian Jira (v8.20.1#820001) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org