Alexey Serbin has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/24532 )
Change subject: KUDU-3790: Fix NPE in HdrHistogramAccumulator.isZero() ...................................................................... KUDU-3790: Fix NPE in HdrHistogramAccumulator.isZero() Spark's executor-heartbeater thread periodically calls isZero() on every running task's accumulators. The executor-side copy of KuduContext's "kudu.write_duration" accumulator can have its internal histogram field observed as null before the task records its first write (unsafe publication across threads, or constructor-bypassing deserialization such as Kryo). isZero() then threw a NullPointerException on the heartbeater thread, which Spark treats as fatal, killing the executor. In this change, isZero() (and toString()) was made null-tolerant: a null/absent histogram is logically zero. isZero() is also now lock-free so it cannot depend on any field being published. Additionally HistogramWrapper now synchronizes its mutators on `this` rather than on the reassignable `innerHistogram` var, which previously changed the monitor object over time and provided no real mutual exclusion. Adds HdrHistogramAccumulatorTest covering the zero semantics, a serialization round-trip, and regression tests that force a null field and assert isZero() does not throw. JAVA8_OVERRIDE Change-Id: Ia3d556aaf1051464303f35281d1d698b20d5ef20 Reviewed-on: http://gerrit.cloudera.org:8080/24527 Reviewed-by: Marton Greber <[email protected]> Tested-by: Marton Greber <[email protected]> Reviewed-by: Zoltan Martonka <[email protected]> (cherry picked from commit 3a4068a26bd919021345b4adf2e89ab7d6da531b) Reviewed-on: http://gerrit.cloudera.org:8080/24532 Reviewed-by: Gabriella Lotz <[email protected]> Tested-by: Alexey Serbin <[email protected]> --- M java/kudu-spark/src/main/scala/org/apache/kudu/spark/kudu/HdrHistogramAccumulator.scala A java/kudu-spark/src/test/scala/org/apache/kudu/spark/kudu/HdrHistogramAccumulatorTest.scala 2 files changed, 178 insertions(+), 12 deletions(-) Approvals: Marton Greber: Looks good to me, approved Gabriella Lotz: Looks good to me, but someone else must approve Alexey Serbin: Verified -- To view, visit http://gerrit.cloudera.org:8080/24532 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: branch-1.18.x Gerrit-MessageType: merged Gerrit-Change-Id: Ia3d556aaf1051464303f35281d1d698b20d5ef20 Gerrit-Change-Number: 24532 Gerrit-PatchSet: 4 Gerrit-Owner: Alexey Serbin <[email protected]> Gerrit-Reviewer: Alexey Serbin <[email protected]> Gerrit-Reviewer: Gabriella Lotz <[email protected]> Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Reviewer: Marton Greber <[email protected]> Gerrit-Reviewer: Zoltan Chovan <[email protected]>
