juliuszsompolski commented on a change in pull request #25694:
[SPARK-28982][SQL] Implementation Spark's own GetTypeInfoOperation
URL: https://github.com/apache/spark/pull/25694#discussion_r322403704
##########
File path:
sql/hive-thriftserver/v2.3.5/src/main/scala/org/apache/spark/sql/hive/thriftserver/ThriftserverShimUtils.scala
##########
@@ -56,6 +56,12 @@ private[thriftserver] object ThriftserverShimUtils {
private[thriftserver] def toJavaSQLType(s: String): Int =
Type.getType(s).toJavaSQLType
+ private[thriftserver] def supportedType(): Seq[Type] = {
+ Array(Type.NULL_TYPE, Type.BOOLEAN_TYPE, Type.TINYINT_TYPE,
Type.SMALLINT_TYPE, Type.INT_TYPE,
+ Type.BIGINT_TYPE, Type.FLOAT_TYPE, Type.DOUBLE_TYPE, Type.STRING_TYPE,
Type.DATE_TYPE,
+ Type.TIMESTAMP_TYPE, Type.DECIMAL_TYPE, Type.BINARY_TYPE)
+ }
Review comment:
Actually, thanks for explaining it @AngersZhuuuu, and you convinced me that
ARRAY, MAP and STRUCT must be included.
```
scala> md.getColumnType(1)
res11: Int = 2003 (== java.sql.Types.ARRAY)
```
but then
```
scala> md.getColumnClassName(1)
res10: String = java.lang.String
```
so that tells to the client that it is actually returned as String, and I
should retrieve it as such, either with `rs.getObject(1).asInstance[String]` or
as convenient shorthand with `rs.getString(1)`.
It would actually be incorrect to not include Array, Map, Struct, because we
do return them in ResultSet schema (through
`SparkExecuteStatement.getTableSchema`), so the client can get these type
returned, and for any type that can be returned to the client there should be
an entry in GetTypeInfo.
We therefore should not include INTERVAL (because we explicitly turn it to
String return type after #25277), and not include UNIONTYPE or USER_DEFINED
because they don't have any Spark equivalent, but ARRAY, MAP and STRUCT should
be there.
Thank you for the explanation :+1:
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
