Will Berkeley has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/13477 )
Change subject: [spark tools] KUDU-2831: Deflake DistributedDataGeneratorTest.testGenerateRandomData ...................................................................... [spark tools] KUDU-2831: Deflake DistributedDataGeneratorTest.testGenerateRandomData Oops. It seems it's not possible to retrieve an accumulator by name. We were creating an new accumulator for collisions after the old one was cleaned up because the job finished and all references to it ended. This meant we never actually counted collisions, so when they happened our expectation of the number of rows written was off. This fixes the problem by plumbing a reference to the rows written and collisions accumulators back to the test code. Before this patch, I saw 12/1000 failures. After, 0/1000. Change-Id: Ie2218328e400d76dbeab50cdb0d0241dfe56d20c Reviewed-on: http://gerrit.cloudera.org:8080/13477 Reviewed-by: Grant Henke <[email protected]> Tested-by: Kudu Jenkins Reviewed-by: Adar Dembo <[email protected]> --- M java/kudu-spark-tools/src/main/scala/org/apache/kudu/spark/tools/DistributedDataGenerator.scala M java/kudu-spark-tools/src/test/scala/org/apache/kudu/spark/tools/DistributedDataGeneratorTest.scala 2 files changed, 22 insertions(+), 14 deletions(-) Approvals: Grant Henke: Looks good to me, but someone else must approve Kudu Jenkins: Verified Adar Dembo: Looks good to me, approved -- To view, visit http://gerrit.cloudera.org:8080/13477 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: merged Gerrit-Change-Id: Ie2218328e400d76dbeab50cdb0d0241dfe56d20c Gerrit-Change-Number: 13477 Gerrit-PatchSet: 3 Gerrit-Owner: Will Berkeley <[email protected]> Gerrit-Reviewer: Adar Dembo <[email protected]> Gerrit-Reviewer: Grant Henke <[email protected]> Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Reviewer: Will Berkeley <[email protected]>
