Apache Otava 0.8.0-incubating has been released Apache Otava is a command-line tool that detects and alerts about statistically significant changes in performance test results (more generally, time-series data) stored in CSV files or a number of supported databases.
A typical use-case of Otava is as follows: 1. A set of performance tests is scheduled repeatedly, such as after each commit is pushed. 2. The resulting metrics of the test runs are stored in a time series database (Graphite) or appended to CSV files. 3. Otava is launched by a Jenkins/Cron job (or an operator) to analyze the recorded metrics regularly. 4. Otava notifies about significant changes in recorded metrics by outputting text reports or sending Slack notifications. 5. Otava is capable of finding even small, but persistent shifts in metric values, despite noise in data. It adapts automatically to the level of noise in data and tries to notify only about persistent, statistically significant changes, be it in the system under test or in the environment. *Highlights of this release* This is our first release under the Apache Incubator program that contains significant updates and new features. Coincidentally we wish to congratulate Denis Schepakin for becoming the first committer to have been voted into the project committer team via the ASF process. Note that unlike all our prior releases, this release breaks backward compatibility in various ways, including in names of command line options, and that the algorithm will find change points that previously may have been missed (for the same input parameters). We expect to continue making breaking changes in future 0.8.x and 0.9.x versions. Users that wish to have minimal changes in their production use of Otava, are advised to continue using 0.7 until we release a version 1.0. - Otava now supports python versions (3.10 - 3.14). We aim to support all active python versions going forward. - The core algorithm at the heart of Otava change point detection was completely rewritten, clean slate, directly from the Matteson & James 2014 paper. #96 <https://github.com/apache/otava/pull/96> - A thorough and layperson understandable explanation of the algorithm and our implementation of it was added #126 <https://github.com/apache/otava/pull/126> - The independent rewrite directly from primary source detected a bug that has been in all versions of Otava all the way back to the first public release by MongoDB performance team in 2020. The bug has caused the algorithm to miss certain classes of change points as discussed in #96 <https://github.com/apache/otava/pull/96> and #154 <https://github.com/apache/otava/pull/154> On the other hand this is somewhat compensated for by either 1) using larger max_pvalue, or 2) the split-merge and weak change points modifications, that have been on and remain on by default in this release. - Note that future Otava releases will likely make further tweaks to the algorithm or its paremeter(s) as we continue to explore the practical effects of fixing this bug. For example, we may remove some of the "hunter enhancements" that in retrospect seem to mostly have been workaround fixes addressing the symptoms of the bug that is now fixed. We invite the user community to provide input on this topic as GitHub issues or on the mailing list. - Support for AMD and ARM Docker images - E2e tests for CSV, PostgreSQL and Grafite data sources - Fixes and more consistent options in ConfigArgParse *Full Changelog* 0.7.0-incubating...0.8.0-incubating <https://github.com/apache/otava/compare/0.7.0-incubating...0.8.0-incubating> *Downloads* - Source release https://dist.apache.org/repos/dist/dev/incubator/otava/0.8.0-incubating/ - Github release https://github.com/apache/otava/releases/tag0.8.0-incubating - PyPI https://pypi.org/project/apache-otava/ - Docker image https://hub.docker.com/r/apache/otava *Otava resources* Website: https://otava.apache.org/ Issues: https://github.com/apache/otava/issues Mailing list: [email protected] Henrik Ingo On behalf of Apache Otava (incubating) Team *License* Licensed to the Apache Software Foundation (ASF) under one or more contributor license agreements. See the NOTICE file distributed with this work for additional information regarding copyright ownership. The ASF licenses this file to you under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at http://www.apache.org/licenses/LICENSE-2.0 Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License. *Disclaimer* Apache Otava (incubating) is an effort undergoing incubation at the Apache Software Foundation (ASF), sponsored by the Apache Incubator PMC. Incubation is required of all newly accepted projects until a further review indicates that the infrastructure, communications, and decision making process have stabilized in a manner consistent with other successful ASF projects. While incubation status is not necessarily a reflection of the completeness or stability of the code, it does indicate that the project has yet to be fully endorsed by the ASF.
