Will Berkeley has uploaded a new change for review.

  http://gerrit.cloudera.org:8080/3871

Change subject: KUDU-1533 Spark Kudu Rdd/Dataframe upsert
......................................................................

KUDU-1533 Spark Kudu Rdd/Dataframe upsert

This patch adds support for upsert in Kudu dataframes. The SaveMode
parameter to createRelation does not map to Kudu very well: Append
is fine, but Overwrite is supposed to mean truncate and then insert.
Kudu does not (currently) support truncation of tables, but it does
support updates and upserts, so Overwrite is taken to mean "upsert
new rows".

Circumventing the limitations of the datasource API, users can still
restore the old behavior of mode Overwrite (update, no insert) by
setting kudu.upsert = false in the options. I think users
prefer to have upsert semantics by default.

Additionally, Ignore was previously given the same meaning as Append,
contrary to its intended meaning. It's been re-categorized as
unsupported along with ErrorIfExisting.

Change-Id: Ib8e0d50fb74dc2ce5e757e8a56fc1e863f699822
---
M java/kudu-spark/src/main/scala/org/apache/kudu/spark/kudu/DefaultSource.scala
M java/kudu-spark/src/main/scala/org/apache/kudu/spark/kudu/KuduContext.scala
M 
java/kudu-spark/src/test/scala/org/apache/kudu/spark/kudu/DefaultSourceTest.scala
3 files changed, 62 insertions(+), 26 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/71/3871/1
-- 
To view, visit http://gerrit.cloudera.org:8080/3871
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: Ib8e0d50fb74dc2ce5e757e8a56fc1e863f699822
Gerrit-PatchSet: 1
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-Owner: Will Berkeley <wdberke...@gmail.com>

Reply via email to