Grant Henke has posted comments on this change. ( http://gerrit.cloudera.org:8080/11199 )
Change subject: Supporting Spark streaming DataFrame in KuduContext. ...................................................................... Patch Set 5: (1 comment) http://gerrit.cloudera.org:8080/#/c/11199/5/java/kudu-spark/src/main/scala/org/apache/kudu/spark/kudu/KuduContext.scala File java/kudu-spark/src/main/scala/org/apache/kudu/spark/kudu/KuduContext.scala: http://gerrit.cloudera.org:8080/#/c/11199/5/java/kudu-spark/src/main/scala/org/apache/kudu/spark/kudu/KuduContext.scala@305 PS5, Line 305: data.queryExecution.toRdd.foreachPartition(iterator => { I don't fully understand why this is needed. Especially given that we convert back from and InternalRow to a Row below in writePartitionRows. My Spark internal background isn't the strongest. Could you explain what this conversion fixes? -- To view, visit http://gerrit.cloudera.org:8080/11199 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Iead04539d3514920a5d6803c34715e5686124572 Gerrit-Change-Number: 11199 Gerrit-PatchSet: 5 Gerrit-Owner: Attila Piros <[email protected]> Gerrit-Reviewer: Attila Bukor <[email protected]> Gerrit-Reviewer: Attila Piros <[email protected]> Gerrit-Reviewer: Grant Henke <[email protected]> Gerrit-Reviewer: Kudu Jenkins Gerrit-Comment-Date: Tue, 25 Sep 2018 18:42:12 +0000 Gerrit-HasComments: Yes
