aokolnychyi commented on a change in pull request #30806:
URL: https://github.com/apache/spark/pull/30806#discussion_r544398504
##########
File path:
sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/v2/DataSourceV2Strategy.scala
##########
@@ -188,15 +189,20 @@ class DataSourceV2Strategy(session: SparkSession) extends
Strategy with Predicat
orCreate = orCreate) :: Nil
}
- case AppendData(r: DataSourceV2Relation, query, writeOptions, _) =>
+ case AppendData(r: DataSourceV2Relation, query, writeOptions, _, write) =>
r.table.asWritable match {
case v1 if v1.supports(TableCapability.V1_BATCH_WRITE) =>
- AppendDataExecV1(v1, writeOptions.asOptions, query, refreshCache(r))
:: Nil
+ AppendDataExecV1(
+ v1, writeOptions.asOptions, query,
+ refreshCache(r), write.map(_.asInstanceOf[V1Write])) :: Nil
Review comment:
I think there is one open point we need to discuss: do we want to always
apply the new logic or should we expose a feature flag and construct logical
writes only if the flag is enabled? I'd vote for always constructing writes
using the new logic as it feels quite same and it does not have the burden of
maintaining one more config. In addition, this will allow us to simply this PR
a bit and get rid of optional writes in exec nodes.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
