Hello Dan Burkert, Alexey Serbin, Kudu Jenkins,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/6081
to look at the new patch set (#2).
Change subject: spark: support serializing authentication data to tasks
......................................................................
spark: support serializing authentication data to tasks
This changes the serialization of KuduContext to export the
authentication data and then re-import it upon deserialization.
Note that this changes the signature of the KuduRDD constructor to just
take in a KuduContext instead of a list of masters and a KuduContext. I
believe this class has always been meant to be constructed from a
KuduContext, so the constructor signature change isn't incompatible. I
changed the constructor to package-private to ensure that this is the
only supported construction mechanism going forward.
I tested this in a secure cluster and was able to run Spark jobs after
the change.
Adding automated testing of this beyond the included unit test would be
a good idea, but I think doing it in a follow-on would be nice to
unblock cluster testing in the meantime.
Change-Id: I7be2c72eb801c3ee10e576ce8705c3b5997dffd9
---
M java/kudu-spark/src/main/scala/org/apache/kudu/spark/kudu/DefaultSource.scala
M java/kudu-spark/src/main/scala/org/apache/kudu/spark/kudu/KuduContext.scala
M java/kudu-spark/src/main/scala/org/apache/kudu/spark/kudu/KuduRDD.scala
M
java/kudu-spark/src/test/scala/org/apache/kudu/spark/kudu/KuduContextTest.scala
4 files changed, 57 insertions(+), 21 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/81/6081/2
--
To view, visit http://gerrit.cloudera.org:8080/6081
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I7be2c72eb801c3ee10e576ce8705c3b5997dffd9
Gerrit-PatchSet: 2
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-Owner: Todd Lipcon <[email protected]>
Gerrit-Reviewer: Alexey Serbin <[email protected]>
Gerrit-Reviewer: Dan Burkert <[email protected]>
Gerrit-Reviewer: Kudu Jenkins
