Hello Kudu Jenkins, Grant Henke,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/11199
to look at the new patch set (#5).
Change subject: Supporting Spark streaming DataFrame in KuduContext.
......................................................................
Supporting Spark streaming DataFrame in KuduContext.
KUDU-2539: Supporting Spark streaming DataFrame in KuduContext.
This solution follows the way how other sinks ie. KafkaSink
is implemented, for details see
https://github.com/apache/spark/blob/master/external/kafka-0-10-sql/src/main/scala/org/apache/spark/sql/kafka010/KafkaWriter.scala#L87
Where on the DataFrame a queryExecution.toRdd.foreachPartition
is called to access the InternalRows which mapped to Rows by Catalyst
converters.
Change-Id: Iead04539d3514920a5d6803c34715e5686124572
---
M java/gradle/dependencies.gradle
M java/kudu-spark/build.gradle
M java/kudu-spark/pom.xml
M java/kudu-spark/src/main/scala/org/apache/kudu/spark/kudu/KuduContext.scala
A java/kudu-spark/src/test/scala/org/apache/kudu/spark/kudu/StreamingTest.scala
5 files changed, 126 insertions(+), 6 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/99/11199/5
--
To view, visit http://gerrit.cloudera.org:8080/11199
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: Iead04539d3514920a5d6803c34715e5686124572
Gerrit-Change-Number: 11199
Gerrit-PatchSet: 5
Gerrit-Owner: Attila Piros <[email protected]>
Gerrit-Reviewer: Attila Piros <[email protected]>
Gerrit-Reviewer: Grant Henke <[email protected]>
Gerrit-Reviewer: Kudu Jenkins
