Grant Henke has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/11199 )
Change subject: Supporting Spark streaming DataFrame in KuduContext. ...................................................................... Supporting Spark streaming DataFrame in KuduContext. KUDU-2539: Supporting Spark streaming DataFrame in KuduContext. This solution follows the way how other sinks ie. KafkaSink is implemented, for details see https://github.com/apache/spark/blob/master/external/kafka-0-10-sql/src/main/scala/org/apache/spark/sql/kafka010/KafkaWriter.scala#L87 Where on the DataFrame a queryExecution.toRdd.foreachPartition is called to access the InternalRows which mapped to Rows by Catalyst converters. Change-Id: Iead04539d3514920a5d6803c34715e5686124572 Reviewed-on: http://gerrit.cloudera.org:8080/11199 Tested-by: Kudu Jenkins Reviewed-by: Attila Bukor <[email protected]> Reviewed-by: Grant Henke <[email protected]> --- M java/gradle/dependencies.gradle M java/kudu-spark/build.gradle M java/kudu-spark/pom.xml M java/kudu-spark/src/main/scala/org/apache/kudu/spark/kudu/KuduContext.scala A java/kudu-spark/src/test/scala/org/apache/kudu/spark/kudu/StreamingTest.scala 5 files changed, 126 insertions(+), 6 deletions(-) Approvals: Kudu Jenkins: Verified Attila Bukor: Looks good to me, but someone else must approve Grant Henke: Looks good to me, approved -- To view, visit http://gerrit.cloudera.org:8080/11199 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: merged Gerrit-Change-Id: Iead04539d3514920a5d6803c34715e5686124572 Gerrit-Change-Number: 11199 Gerrit-PatchSet: 6 Gerrit-Owner: Attila Piros <[email protected]> Gerrit-Reviewer: Attila Bukor <[email protected]> Gerrit-Reviewer: Attila Piros <[email protected]> Gerrit-Reviewer: Grant Henke <[email protected]> Gerrit-Reviewer: Kudu Jenkins
