HeartSaVioR commented on code in PR #49751:
URL: https://github.com/apache/spark/pull/49751#discussion_r1946054194
##########
sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/StatefulProcessorHandleImpl.scala:
##########
@@ -363,18 +363,36 @@ class DriverStatefulProcessorHandleImpl(timeMode:
TimeMode, keyExprEnc: Expressi
addTimerColFamily()
}
+ /**
+ * This method returns all column family schemas, and checks and enforces
nullability
+ * if need be. The nullability check and set is only set to true when Avro
is enabled.
+ * @param shouldCheckNullable Whether we need to check the nullability. This
is set to
+ * true when using Python, as this is the only
avenue through
+ * which users can set nullability
+ * @param shouldSetNullable Whether we need to set the fields as nullable.
This is set to
+ * true when using Scala, as case classes are set to
Review Comment:
I found this, `def nullable: Boolean = !isPrimitive` from AgnosticEncoder
trait, so it's intended and not a bug.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
