----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/41912/ -----------------------------------------------------------
Review request for samza, Boris Shkolnik, Navina Ramesh, and Yi Pan (Data Infrastructure). Repository: samza Description ------- We've seen a strong need for validating running samza jobs programmatically. Consider the following scenarios: 1) running samza as a service (as in SAMZA-849), which requires automated job validation when upgrading the framework for large number of jobs; 2) production monitoring, automated validation can run periodically and trigger alerts; 3) integration testing, which can use validation tool to check the job status. So a tool is needed to provide the checkout of job health. Minimally it will check: - job submission - app running - container count It should also be able to extend and validate the job metrics through metrics system, such as JMX. The user will be able to validate against important metrics such as process_count or message_behind_high_watermark. Diffs ----- checkstyle/import-control.xml 53cb8b447240fea08d98ccfb12ed24bec6cbf67c samza-api/src/main/java/org/apache/samza/metrics/MetricsAccessor.java PRE-CREATION samza-api/src/main/java/org/apache/samza/metrics/MetricsValidationFailureException.java PRE-CREATION samza-api/src/main/java/org/apache/samza/metrics/MetricsValidator.java PRE-CREATION samza-core/src/main/java/org/apache/samza/job/model/JobModel.java 9445a30c9e605a3623873ed09eedfd5140af98f7 samza-core/src/main/java/org/apache/samza/metrics/JmxMetricsAccessor.java PRE-CREATION samza-core/src/main/java/org/apache/samza/metrics/JmxUtil.java PRE-CREATION samza-core/src/main/scala/org/apache/samza/metrics/reporter/JmxReporter.scala e9661023a04f39d059d879fea2140cb57af3b546 samza-core/src/test/java/org/apache/samza/metrics/TestJmxMetricsAccessor.java PRE-CREATION samza-shell/src/main/bash/validate-yarn-job.sh PRE-CREATION samza-yarn/src/main/java/org/apache/samza/validation/YarnJobValidationTool.java PRE-CREATION samza-yarn/src/test/java/org/apache/samza/validation/MockMetricsValidator.java PRE-CREATION samza-yarn/src/test/java/org/apache/samza/validation/TestYarnJobValidationTool.java PRE-CREATION Diff: https://reviews.apache.org/r/41912/diff/ Testing ------- Unit tests added to cover all the new methods/classes. Thanks, Xinyu Liu
