cloud-fan commented on code in PR #38397:
URL: https://github.com/apache/spark/pull/38397#discussion_r1006329856
##########
sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/FileFormat.scala:
##########
@@ -61,6 +61,11 @@ trait FileFormat {
/**
* Returns whether this format supports returning columnar batch or not.
+ * If columnar batch output is requested, users shall supply
+ * FileFormat.OPTION_RETURNING_BATCH -> true
+ * in relation options when calling buildReaderWithPartitionValues.
+ * This should only be passed as true if it can actually be supported.
+ * For ParquetFileFormat, passing this option is required.
Review Comment:
I think it's always required, but only parquet added the check for now. We
definitely should do the same for ORC.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
