Hello Kudu Jenkins, Adar Dembo, Grant Henke,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/13477
to look at the new patch set (#2).
Change subject: [spark tools] KUDU-2831: Deflake
DistributedDataGeneratorTest.testGenerateRandomData
......................................................................
[spark tools] KUDU-2831: Deflake
DistributedDataGeneratorTest.testGenerateRandomData
Oops. It seems it's not possible to retrieve an accumulator by name. We
were creating an new accumulator for collisions after the old one was
cleaned up because the job finished and all references to it ended. This
meant we never actually counted collisions, so when they happened our
expectation of the number of rows written was off. This fixes the
problem by plumbing a reference to the rows written and collisions
accumulators back to the test code.
Before this patch, I saw 12/1000 failures. After, 0/1000.
Change-Id: Ie2218328e400d76dbeab50cdb0d0241dfe56d20c
---
M
java/kudu-spark-tools/src/main/scala/org/apache/kudu/spark/tools/DistributedDataGenerator.scala
M
java/kudu-spark-tools/src/test/scala/org/apache/kudu/spark/tools/DistributedDataGeneratorTest.scala
2 files changed, 22 insertions(+), 14 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/77/13477/2
--
To view, visit http://gerrit.cloudera.org:8080/13477
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: Ie2218328e400d76dbeab50cdb0d0241dfe56d20c
Gerrit-Change-Number: 13477
Gerrit-PatchSet: 2
Gerrit-Owner: Will Berkeley <[email protected]>
Gerrit-Reviewer: Adar Dembo <[email protected]>
Gerrit-Reviewer: Grant Henke <[email protected]>
Gerrit-Reviewer: Kudu Jenkins (120)
