Github user AndreSchumacher commented on a diff in the pull request:
https://github.com/apache/spark/pull/360#discussion_r11986278
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/types/dataTypes.scala
---
@@ -29,25 +31,36 @@ abstract class DataType {
case e: Expression if e.dataType == this => true
case _ => false
}
+
+ def isPrimitive(): Boolean = false
}
case object NullType extends DataType
+trait PrimitiveType extends DataType {
--- End diff --
@marmbrus `PrimitiveType` is maybe a misnomer. It's the same term that
Parquet uses. Basically a `PrimitiveType` is a type that is not contained
inside another type (so non-nested). You can argue that a String is a Char
array and therefore not primitive but in terms of constructing nested rows it
means that a primitive type is a leaf inside the tree that produces a record.
It would help to somehow distinguish between nested and non-nested types.
`NativeType` comes close but for example there is `BinaryType` which is
primitive but not native.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
