Grant Henke has posted comments on this change. ( http://gerrit.cloudera.org:8080/12101 )
Change subject: Create parallelized loader Spark job ...................................................................... Patch Set 2: (3 comments) http://gerrit.cloudera.org:8080/#/c/12101/1/java/kudu-spark-tools/src/main/scala/org/apache/kudu/spark/tools/DistributedDataGenerator.scala File java/kudu-spark-tools/src/main/scala/org/apache/kudu/spark/tools/DistributedDataGenerator.scala: http://gerrit.cloudera.org:8080/#/c/12101/1/java/kudu-spark-tools/src/main/scala/org/apache/kudu/spark/tools/DistributedDataGenerator.scala@46 PS1, Line 46: GeneratorMetrics(sc.longAccumulator("rows_written"), sc.longAccumulator("row_collisions")) > And using the client directly via the context means things like authn token Yes, the context is the thing that calls/manages importAuthenticationCredentials. http://gerrit.cloudera.org:8080/#/c/12101/2/java/kudu-spark-tools/src/test/scala/org/apache/kudu/spark/tools/DistributedDataGeneratorTest.scala File java/kudu-spark-tools/src/test/scala/org/apache/kudu/spark/tools/DistributedDataGeneratorTest.scala: http://gerrit.cloudera.org:8080/#/c/12101/2/java/kudu-spark-tools/src/test/scala/org/apache/kudu/spark/tools/DistributedDataGeneratorTest.scala@37 PS2, Line 37: .excludeTypes(Type.BOOL, Type.INT8) > Should we also exclude INT16? They only have 64k values. Both tests only use 100 rows so this should be okay. http://gerrit.cloudera.org:8080/#/c/12101/2/java/kudu-spark-tools/src/test/scala/org/apache/kudu/spark/tools/DistributedDataGeneratorTest.scala@42 PS2, Line 42: private val randomTableName: String = "random-table" > That's not exactly a "random" name. Did you intend to randomize something h No, I meant that the contents are random. It's not worth testing table names here. -- To view, visit http://gerrit.cloudera.org:8080/12101 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ibdfd41a21a7f80d22125c7f4e5ca4ed62c31709d Gerrit-Change-Number: 12101 Gerrit-PatchSet: 2 Gerrit-Owner: Grant Henke <[email protected]> Gerrit-Reviewer: Adar Dembo <[email protected]> Gerrit-Reviewer: Grant Henke <[email protected]> Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Reviewer: Mike Percy <[email protected]> Gerrit-Comment-Date: Thu, 20 Dec 2018 19:24:31 +0000 Gerrit-HasComments: Yes
