kiszk commented on pull request #29067:
URL: https://github.com/apache/spark/pull/29067#issuecomment-660593980
Can we add additional API for saying whether a given type is cached or not
to `CachedBatchSerializer`? It depends on the implementation of
`CachedBatchSerializer`. This addresses `These data types may expand in the
future. If they do we can add in a new API with a default value that says which
data types this serializer supports.`
In the following code, this part can be exposed as an API.
```
relation.schema.fields.forall(f => f.dataType match {
case BooleanType | ByteType | ShortType | IntegerType | LongType |
FloatType | DoubleType => true
case _ => false
})
```
https://github.com/apache/spark/blob/026b0b926dfd40038f2cee932f38b917eb25b77e/sql/core/src/main/scala/org/apache/spark/sql/execution/columnar/InMemoryTableScanExec.scala#L72-L80
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
