Dan Burkert has posted comments on this change. Change subject: spark: support serializing authentication data to tasks ......................................................................
Patch Set 2: (3 comments) We've been discussing some of these notes offline, but wanted to dump them here for posterity even if we don't end up incorporating them. http://gerrit.cloudera.org:8080/#/c/6081/2/java/kudu-spark/src/main/scala/org/apache/kudu/spark/kudu/KuduContext.scala File java/kudu-spark/src/main/scala/org/apache/kudu/spark/kudu/KuduContext.scala: Line 57: @throws(classOf[java.io.IOException]) Spark doesn't always use Java serialization, so this is incorrect. I think what needs to happen is to make the security token an internal field of KuduContext which gets set whenever a client is created, and used for creating subsequent clients. http://gerrit.cloudera.org:8080/#/c/6081/2/java/kudu-spark/src/main/scala/org/apache/kudu/spark/kudu/KuduRDD.scala File java/kudu-spark/src/main/scala/org/apache/kudu/spark/kudu/KuduRDD.scala: Line 33: class KuduRDD private[kudu] (val kuduContext: KuduContext, It's a bit unfortunate that this wasn't done pre-1.0, but considering that KuduContext is marked Unstable, I don't think this ctor ever should have been considered stable. Line 34: @transient val batchSize: Integer, indent -- To view, visit http://gerrit.cloudera.org:8080/6081 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-MessageType: comment Gerrit-Change-Id: I7be2c72eb801c3ee10e576ce8705c3b5997dffd9 Gerrit-PatchSet: 2 Gerrit-Project: kudu Gerrit-Branch: master Gerrit-Owner: Todd Lipcon <[email protected]> Gerrit-Reviewer: Alexey Serbin <[email protected]> Gerrit-Reviewer: Chris George <[email protected]> Gerrit-Reviewer: Dan Burkert <[email protected]> Gerrit-Reviewer: Kudu Jenkins Gerrit-HasComments: Yes
