Xu Yao has uploaded this change for review. ( http://gerrit.cloudera.org:8080/10406
Change subject: KUDU-2437 Generate ScanToken from small chunks in tablet ...................................................................... KUDU-2437 Generate ScanToken from small chunks in tablet When reading data in a kudu table using spark, if there is a large amount of data in the tablet, reading the data takes a long time. The reason is that KuduRDD uses a tablet to generate the scanToken, so a spark task needs to process all the data in a tablet. So: 1. Split the Tablet into many small chunks by some primary keys 2. Report the primary keys to Master 3. Client get the primary keys from Master, and generate the scanToken Change-Id: I9ec4395919f4b54102e458ef5154334c08412e8a --- M java/kudu-client/src/main/java/org/apache/kudu/client/KuduScanToken.java M java/kudu-client/src/main/java/org/apache/kudu/client/LocatedTablet.java M java/kudu-client/src/main/java/org/apache/kudu/client/RemoteTablet.java M java/kudu-spark/src/main/scala/org/apache/kudu/spark/kudu/KuduRDD.scala M src/kudu/master/catalog_manager.cc M src/kudu/master/catalog_manager.h M src/kudu/master/master.proto M src/kudu/master/master_service.cc M src/kudu/tablet/rowset_tree-test.cc M src/kudu/tablet/rowset_tree.cc M src/kudu/tablet/rowset_tree.h M src/kudu/tablet/tablet.cc M src/kudu/tablet/tablet.h M src/kudu/tserver/heartbeater.cc M src/kudu/tserver/ts_tablet_manager.cc M src/kudu/tserver/ts_tablet_manager.h 16 files changed, 305 insertions(+), 8 deletions(-) git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/06/10406/1 -- To view, visit http://gerrit.cloudera.org:8080/10406 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: newchange Gerrit-Change-Id: I9ec4395919f4b54102e458ef5154334c08412e8a Gerrit-Change-Number: 10406 Gerrit-PatchSet: 1 Gerrit-Owner: Xu Yao <ocla...@gmail.com>
