brkyvz commented on code in PR #59305:
URL: https://github.com/apache/spark/pull/59305#discussion_r4309199434
##########
sql/api/src/main/scala/org/apache/spark/sql/types/DataType.scala:
##########
@@ -133,6 +133,7 @@ object DataType {
private val STRING_WITH_COLLATION = """string\s+collate\s+(\w+)""".r
private val TIMESTAMP_LTZ_NANOS_TYPE = """timestamp_ltz\(\s*(\d+)\s*\)""".r
private val TIMESTAMP_NTZ_NANOS_TYPE = """timestamp_ntz\(\s*(\d+)\s*\)""".r
+ private val FILE_TYPE = """file(?:\s+(external|unknown))?""".r
Review Comment:
why the `unknown` addition into ddl?
`file` in the context of a table should be FileType.external, `file` in the
context of a query/udf should be considered FileType.unknown. Is there a way to
make that distinction?
##########
sql/api/src/main/scala/org/apache/spark/sql/types/DataType.scala:
##########
@@ -267,7 +268,7 @@ object DataType {
}
/** Given the string representation of a type, return its DataType */
- private def nameToType(name: String): DataType = {
+ private def nameToType(name: String, inStructField: Boolean = false):
DataType = {
Review Comment:
is there a way to get the distinction of `inTableSchema` instead of
`inStructField`?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
