Github user liancheng commented on the pull request:
https://github.com/apache/spark/pull/9840#issuecomment-160622335
A little bit off topic but related. Currently, our casting rules (either
implicit or explicit casting) are defined as full functions, thus it's hard to
programmatically check whether data type `A` is implicitly/explicitly
convertible to data type `B`. This makes us hard coded type conversion rules
here and there, potentially in an inconsistent way.
I think we can improve this situation by leveraging partial functions. Take
`IntegerType` as an example, here is a prototype:
```scala
object Cast {
private type CastBuilder = PartialFunction[DataType, Any => Any]
private val asInt = (_: Any) match { case v: Int => v }
private val implicitlyFromInt: CastBuilder = {
case BooleanType => asInt andThen (_ == 0)
case LongType => asInt andThen (_.toLong)
case FloatType => asInt andThen (_.toFloat)
case DoubleType => asInt andThen (_.toDouble)
case StringType => _.toString
}
private val explicitlyFromInt: CastBuilder = {
case ByteType => asInt andThen (_.toByte)
case ShortType => asInt andThen (_.toShort)
}
private val buildImplicitCast: PartialFunction[DataType, CastBuilder] = {
// ...
case IntType => implicitlyFromInt
// ...
}
private val buildExplicitCast: PartialFunction[DataType, CastBuilder] = {
// ...
case IntType => explicitlyFromInt
// ...
}
private val buildCast: PartialFunction[DataType, CastBuilder] = {
buildImplicitCast orElse buildExplicitCast
}
}
```
Then we can define `Cast.nullSafeEval` as:
```scala
case class Cast(child: Expression, dataType: DataType) {
def nullSafeEval(input: Any): Any =
buildCast(child.dataType)(dataType)(input)
}
```
With these at hand, it would be trivial to write some APIs for checking
conversion between types:
```scala
```
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
