Will Berkeley has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/12251 )
Change subject: [spark] Add metrics to kudu-spark ...................................................................... [spark] Add metrics to kudu-spark This adds some basic accumulator metrics to kudu-spark. These accumulators appear for each stage involving a KuduContext, with values broken down by executor. Follow-ups will add some more sophisticated metrics and use metrics to add additional logging. In addition to the unit tests, I tested this on a 3-node cluster and confirmed I was able to see the accumulator values for stages and individual tasks on the Spark application's web UI, and that were correct. Change-Id: Ied81c890b3d4510f767d8f023963ff878f398140 Reviewed-on: http://gerrit.cloudera.org:8080/12251 Reviewed-by: Grant Henke <[email protected]> Tested-by: Will Berkeley <[email protected]> --- M java/kudu-client/src/main/java/org/apache/kudu/client/AsyncKuduClient.java M java/kudu-spark/src/main/scala/org/apache/kudu/spark/kudu/DefaultSource.scala M java/kudu-spark/src/main/scala/org/apache/kudu/spark/kudu/KuduContext.scala M java/kudu-spark/src/main/scala/org/apache/kudu/spark/kudu/KuduRDD.scala M java/kudu-spark/src/test/scala/org/apache/kudu/spark/kudu/DefaultSourceTest.scala M java/kudu-spark/src/test/scala/org/apache/kudu/spark/kudu/KuduRDDTest.scala 6 files changed, 74 insertions(+), 23 deletions(-) Approvals: Grant Henke: Looks good to me, approved Will Berkeley: Verified -- To view, visit http://gerrit.cloudera.org:8080/12251 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: merged Gerrit-Change-Id: Ied81c890b3d4510f767d8f023963ff878f398140 Gerrit-Change-Number: 12251 Gerrit-PatchSet: 3 Gerrit-Owner: Will Berkeley <[email protected]> Gerrit-Reviewer: Alexey Serbin <[email protected]> Gerrit-Reviewer: Grant Henke <[email protected]> Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Reviewer: Will Berkeley <[email protected]>
