Adar Dembo has posted comments on this change. ( http://gerrit.cloudera.org:8080/12101 )
Change subject: Create parallelized loader Spark job ...................................................................... Patch Set 2: Code-Review+2 (3 comments) http://gerrit.cloudera.org:8080/#/c/12101/1/java/kudu-spark-tools/src/main/scala/org/apache/kudu/spark/tools/DistributedDataGenerator.scala File java/kudu-spark-tools/src/main/scala/org/apache/kudu/spark/tools/DistributedDataGenerator.scala: http://gerrit.cloudera.org:8080/#/c/12101/1/java/kudu-spark-tools/src/main/scala/org/apache/kudu/spark/tools/DistributedDataGenerator.scala@46 PS1, Line 46: GeneratorMetrics(sc.longAccumulator("rows_written"), sc.longAccumulator("row_collisions")) > I am using the client API directly because we are inserting PartialRows ins And using the client directly via the context means things like authn tokens are propagated properly? http://gerrit.cloudera.org:8080/#/c/12101/2/java/kudu-spark-tools/src/test/scala/org/apache/kudu/spark/tools/DistributedDataGeneratorTest.scala File java/kudu-spark-tools/src/test/scala/org/apache/kudu/spark/tools/DistributedDataGeneratorTest.scala: http://gerrit.cloudera.org:8080/#/c/12101/2/java/kudu-spark-tools/src/test/scala/org/apache/kudu/spark/tools/DistributedDataGeneratorTest.scala@37 PS2, Line 37: .excludeTypes(Type.BOOL, Type.INT8) Should we also exclude INT16? They only have 64k values. http://gerrit.cloudera.org:8080/#/c/12101/2/java/kudu-spark-tools/src/test/scala/org/apache/kudu/spark/tools/DistributedDataGeneratorTest.scala@42 PS2, Line 42: private val randomTableName: String = "random-table" That's not exactly a "random" name. Did you intend to randomize something here? -- To view, visit http://gerrit.cloudera.org:8080/12101 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ibdfd41a21a7f80d22125c7f4e5ca4ed62c31709d Gerrit-Change-Number: 12101 Gerrit-PatchSet: 2 Gerrit-Owner: Grant Henke <[email protected]> Gerrit-Reviewer: Adar Dembo <[email protected]> Gerrit-Reviewer: Grant Henke <[email protected]> Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Reviewer: Mike Percy <[email protected]> Gerrit-Comment-Date: Thu, 20 Dec 2018 19:14:28 +0000 Gerrit-HasComments: Yes
