Github user kiszk commented on a diff in the pull request:
https://github.com/apache/spark/pull/20637#discussion_r209692496
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/codegen/GenerateUnsafeProjection.scala
---
@@ -43,25 +43,29 @@ object GenerateUnsafeProjection extends
CodeGenerator[Seq[Expression], UnsafePro
case _ => false
}
- // TODO: if the nullability of field is correct, we can use it to save
null check.
private def writeStructToBuffer(
ctx: CodegenContext,
input: String,
index: String,
- fieldTypes: Seq[DataType],
+ fieldTypeAndNullables: Seq[(DataType, Boolean)],
--- End diff --
@cloud-fan I found this case class `case class Schema(dataType: DataType,
nullable: Boolean)` at two places.
1. `ScalaReflection.Schema`
1. `SchemaConverters.SchemaType`
Do we use one of them? Or do we define `org.apache.spark.sql.types.Schema`?
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
