Adar Dembo has posted comments on this change. ( http://gerrit.cloudera.org:8080/12211 )
Change subject: [spark-tools] Fix DistributedDataGenerator num-tasks ...................................................................... Patch Set 1: (2 comments) http://gerrit.cloudera.org:8080/#/c/12211/1//COMMIT_MSG Commit Message: http://gerrit.cloudera.org:8080/#/c/12211/1//COMMIT_MSG@9 PS1, Line 9: bum-tasks bum tasks indeed. http://gerrit.cloudera.org:8080/#/c/12211/1/java/kudu-spark-tools/src/main/scala/org/apache/kudu/spark/tools/DistributedDataGenerator.scala File java/kudu-spark-tools/src/main/scala/org/apache/kudu/spark/tools/DistributedDataGenerator.scala: http://gerrit.cloudera.org:8080/#/c/12211/1/java/kudu-spark-tools/src/main/scala/org/apache/kudu/spark/tools/DistributedDataGenerator.scala@151 PS1, Line 151: sc.parallelize(0 until options.numTasks, numSlices = options.numTasks) : .foreachPartition(taskNum => generateRows(context, options, taskNum.next(), metrics)) Could you explain what the old code did, what the new code does, and why it's more correct? Doesn't have to be in a code comment; I trust that this is correct and intuitive for someone more familiar with Spark/Scala, I'm just trying to make sense of it myself without wading through a ton of docs. -- To view, visit http://gerrit.cloudera.org:8080/12211 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I7fa560e71b2f84a75002e9e776011e1a11c5a1ff Gerrit-Change-Number: 12211 Gerrit-PatchSet: 1 Gerrit-Owner: Grant Henke <[email protected]> Gerrit-Reviewer: Adar Dembo <[email protected]> Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Reviewer: Mike Percy <[email protected]> Gerrit-Comment-Date: Thu, 10 Jan 2019 21:57:21 +0000 Gerrit-HasComments: Yes
