Ximo Guanter created SPARK-29248:
------------------------------------
Summary: Pass in number of partitions to BuildWriter
Key: SPARK-29248
URL: https://issues.apache.org/jira/browse/SPARK-29248
Project: Spark
Issue Type: Sub-task
Components: SQL
Affects Versions: 3.0.0
Reporter: Ximo Guanter
When implementing a ScanBuilder, we require the implementor to provide the
schema of the data and the number of partitions.
However, when someone is implementing WriteBuilder we only pass them the
schema, but not the number of partitions. This is an asymetrical developer
experience. Passing in the number of partitions on the WriteBuilder would
enable data sources to provision their write targets before starting to write.
For example, it could be used to provision a Kafka topic with a specific number
of partitions.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
