shrirangmhalgi commented on code in PR #56319:
URL: https://github.com/apache/spark/pull/56319#discussion_r3532427832
##########
sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/v2/state/metadata/StateMetadataSource.scala:
##########
@@ -78,9 +78,18 @@ object StateMetadataTableEntry {
}
}
-class StateMetadataSource extends TableProvider with DataSourceRegister {
+class StateMetadataSource extends TableProvider with DataSourceRegister
+ with CreatableRelationProvider {
override def shortName(): String = "state-metadata"
+ override def createRelation(
+ sqlContext: org.apache.spark.sql.SQLContext,
+ mode: org.apache.spark.sql.SaveMode,
+ parameters: Map[String, String],
+ data: org.apache.spark.sql.DataFrame): BaseRelation = {
+ throw StateDataSourceErrors.writeUnsupported("state-metadata")
Review Comment:
Apologies for the late reply. The original error when writing to statestore
was:
```
[INTERNAL_ERROR] The operation "planForWriting" is not supported by the data
source org.apache.spark.sql.execution.datasources.v2.state.StateDataSource does
not allow create table as select.
```
This comes from the `case _` branch in `DataSource.planForWriting` - a
generic catch-all that doesn't indicate the source is intentionally read-only.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
