On Mon, Oct 6, 2014 at 9:06 AM, Ivan Kelly <[email protected]> wrote:
> I've downloaded, built and tested the src release. It looks fine. > > The -bin releases contain jna.jar, which isn't mentioned in the NOTICE > file, so -1 for releasing the binary tarballs. You can still release the > source, just leave out the binaries. I think this is ok, since 4.3.0 > isn't GA. > Created https://issues.apache.org/jira/browse/BOOKKEEPER-790 to address NOTICE file issue. > > I ran a few benchmarks to compare against 4.2.2. The bookie benchmark > didn't run for 4.3.0. It just hangs on "Benchmarking latency". For a > single ledger on 3 bookies, performance is acceptable, but latency is > higher. For 1000 ledgers on 3 bookies, tpt and latency is much better, > but tpt is very variable. I did see a lot of timeouts in some runs > also. > > single ledger, 3 bookies > | 4.2.2 | | 4.3.0 | > | tpt | lat p99 | lat p95 | | tpt | lat p99 | lat p95 | > | 9995 | 87.98 | 77.26 | | 9988 | 132 | 117 | > | 9997 | 97.55 | 80.70 | | 9991 | 111 | 107 | > | 9996 | 105.80 | 86.20 | | 9989 | 103 | 100 | > > 1000 ledgers, 3 bookies > | 4.2.2 | | 4.3.0 | > | tpt | lat p99 | lat p95 | | tpt | lat p99 | lat p95 | > | 14588 | 668 | 662 | | 23728 | 229 | 219 | > | 13571 | 701 | 649 | | 39417 | 230 | 219 | > | 14062 | 697 | 688 | | 40654 | 231 | 222 | > | | | | | | | | > > Overall, I'm not sure this release is fit to go without having -alpha or > -beta attached to its version, to make it clear that it shouldn't be > used as is in production. In particular I'm worried about the bookie > benchmark not working [1], and the variability of the 1000 ledger > benchmark. The latency increase for a single ledger can be attributed to > the added batching in the journal. > > -Ivan > > [1] bookkeeper-benchmark/bin/benchmark bookie -zookeeper <zk> -host > <bookie> > > >
