Github user jinxing64 commented on a diff in the pull request:
https://github.com/apache/spark/pull/19652#discussion_r148748292
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/SparkSqlParser.scala ---
@@ -1454,22 +1454,24 @@ class SparkSqlAstBuilder(conf: SQLConf) extends
AstBuilder(conf) {
def format(
fmt: RowFormatContext,
configKey: String,
- defaultConfigValue: String): Format = fmt match {
+ defaultConfigValue: String,
+ isInFormat: Boolean): Format = fmt match {
case c: RowFormatDelimitedContext =>
// TODO we should use the visitRowFormatDelimited function here.
However HiveScriptIOSchema
// expects a seq of pairs in which the old parsers' token names
are used as keys.
// Transforming the result of visitRowFormatDelimited would be
quite a bit messier than
// retrieving the key value pairs ourselves.
def entry(key: String, value: Token): Seq[(String, String)] = {
- Option(value).map(t => key -> t.getText).toSeq
+ Option(value).toSeq.map(x => key -> string(x))
--- End diff --
`t.getText` doesn't work, we need to process the token. e.g. remove the
quote
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
