giucris commented on issue #3665:
URL: https://github.com/apache/iceberg/issues/3665#issuecomment-1455092781
Hi @RussellSpitzer @huaxingao
If this functionality is still required and Spark side is still pending, I
might try to implement it (as a Scala developer I would really like to have the
merge API).
I just need to have a chat with someone who knows the Scala code base to
clarify some missing pieces in my analysis.
I see two possible implementations, both following the implicit approach.
Option 1:
* implicits that extends DataSet[T] with a new method
mergeTo(table,condition)
* implicits that extends DataFrameWriterV2[T] with the following method:
* whenMatched(condition:Option[String],actions:Seq[String])
* whenNotMatched(condition:Option[String],actions:Seq[String])
* merge() that will calls the final operations
```scala
ds
.mergeTo(table,condition)
.whenMatched(something here)
.whenNotMatched(something here)
.merge()
```
Option 2:
* implicits that extends DataFrameWriterV2[T] with the following method:
* mergeWhen(condition:Option[String])
* whenMatched(condition:Option[String],actions:Seq[String])
* whenNotMatched(condition:Option[String],actions:Seq[String])
* merge() that will calls the final operations
```scala
ds
.writeTo(table)
.mergeWhen(condition)
.whenMatched(something here)
.whenNotMatched(something here)
.merge()
```
The difference between the two approaches is in the syntax of the API: on
the first one we generate a new 'mergeTo' method, on the other we refer to the
already existing 'writeTo'.
Looking at the actual code base I see that a MergeInto operation has already
been resolved in a Spark LogicalPlan, probably we could reuse this logic and
just call the executePlan on the final merge call
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
