Will Berkeley has posted comments on this change. ( http://gerrit.cloudera.org:8080/9834 )
Change subject: [spark]KUDU-2371: Add KuduWriteOptions class and ignoreNull option ...................................................................... Patch Set 4: (4 comments) Looking pretty good on the KuduContext side. I think we want support for this in the DefaultSource as well, though. http://gerrit.cloudera.org:8080/#/c/9834/4//COMMIT_MSG Commit Message: http://gerrit.cloudera.org:8080/#/c/9834/4//COMMIT_MSG@7 PS4, Line 7: [spark]KUDU-2371 nit: Separate with a space: [spark] KUDU-2371 http://gerrit.cloudera.org:8080/#/c/9834/4/java/kudu-spark/src/main/scala/org/apache/kudu/spark/kudu/KuduContext.scala File java/kudu-spark/src/main/scala/org/apache/kudu/spark/kudu/KuduContext.scala: http://gerrit.cloudera.org:8080/#/c/9834/4/java/kudu-spark/src/main/scala/org/apache/kudu/spark/kudu/KuduContext.scala@247 PS4, Line 247: ignoring any new : * rows that have a primary key conflict with existing rows. Please add a second bit to the docs here stating that insertIgnoreRows(df, tableName) is equivalent to val wo = new KuduWriteOptions KuduWriteOptions.ignoreDuplicateRowErrors = true insertRows(data, tableName, wo) http://gerrit.cloudera.org:8080/#/c/9834/4/java/kudu-spark/src/main/scala/org/apache/kudu/spark/kudu/KuduContext.scala@253 PS4, Line 253: insertIgnoreRows I think we also want to mark this as deprecated in favor of using KuduWriteOptions. Dan, do you agree? http://gerrit.cloudera.org:8080/#/c/9834/4/java/kudu-spark/src/test/scala/org/apache/kudu/spark/kudu/DefaultSourceTest.scala File java/kudu-spark/src/test/scala/org/apache/kudu/spark/kudu/DefaultSourceTest.scala: http://gerrit.cloudera.org:8080/#/c/9834/4/java/kudu-spark/src/test/scala/org/apache/kudu/spark/kudu/DefaultSourceTest.scala@644 PS4, Line 644: val kuduWriteOptions = new KuduWriteOptions : kuduWriteOptions.ignoreDuplicateRowErrors = true Isn't this unnecessary? -- To view, visit http://gerrit.cloudera.org:8080/9834 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ide908ea29f572849eca0ba850ee197c1b22a07c8 Gerrit-Change-Number: 9834 Gerrit-PatchSet: 4 Gerrit-Owner: Fengling Wang <fw...@cloudera.com> Gerrit-Reviewer: Dan Burkert <danburk...@apache.org> Gerrit-Reviewer: Fengling Wang <fw...@cloudera.com> Gerrit-Reviewer: Hao Hao <hao....@cloudera.com> Gerrit-Reviewer: Kudu Jenkins Gerrit-Reviewer: Todd Lipcon <t...@apache.org> Gerrit-Reviewer: Will Berkeley <wdberke...@gmail.com> Gerrit-Comment-Date: Wed, 04 Apr 2018 20:37:09 +0000 Gerrit-HasComments: Yes
