Hao Hao has uploaded this change for review. ( http://gerrit.cloudera.org:8080/8552
Change subject: KUDU-1454 [part 1]: update propagated timestamp on the driver ...................................................................... KUDU-1454 [part 1]: update propagated timestamp on the driver Currently, Saprk uses multiple clients simultaneously, possibly on different JVMs, for reads and writes. Each client may have the propagated timestamp properly updated after each read/write. However, the KuduClient on the driver does not have the correct propagated timestamp. This is problematic if the user wants to use READ_AT_SNAPSHOT ReadMode to get a consistent snapshot. This patchs use Accumulator in Spark to properly update the propagated timestamp on the driver through comparing the maximum timestamp on each executor. Test is added accordingly and ran in both 'local' and 'local-cluster' mode. Change-Id: Id0a078ae8ebaa6a859be75c822879291192c5842 --- M java/kudu-spark/src/main/scala/org/apache/kudu/spark/kudu/KuduContext.scala M java/kudu-spark/src/main/scala/org/apache/kudu/spark/kudu/KuduRDD.scala M java/kudu-spark/src/test/scala/org/apache/kudu/spark/kudu/DefaultSourceTest.scala 3 files changed, 84 insertions(+), 4 deletions(-) git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/52/8552/1 -- To view, visit http://gerrit.cloudera.org:8080/8552 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: newchange Gerrit-Change-Id: Id0a078ae8ebaa6a859be75c822879291192c5842 Gerrit-Change-Number: 8552 Gerrit-PatchSet: 1 Gerrit-Owner: Hao Hao <[email protected]>
