Hi
While testing “[0ab208fa5] pgbench: Add --continue-on-error option”, I got a
question about the doc:
```
<para>
This option is useful when your custom script may raise errors
such as unique constraint violations, but you want the benchmark
to continue and measure performance including those failures.
</para>
```
The phrase “measure performance including those failures” gives the impression
that failed transactions might be counted in TPS, but they are not.
In my test, all transactions failed, and TPS was reported as 0:
```
% pgbench -d evantest -n -t 5 --continue-on-error --failures-detailed -f
pgbench-coe.sql
pgbench (19beta1)
transaction type: pgbench-coe.sql
scaling factor: 1
query mode: simple
number of clients: 1
number of threads: 1
maximum number of tries: 1
number of transactions per client: 5
number of transactions actually processed: 0/5
number of failed transactions: 5 (100.000%)
number of serialization failures: 0 (0.000%)
number of deadlock failures: 0 (0.000%)
number of other failures: 5 (100.000%)
latency average = 0.389 ms (including failures)
initial connection time = 4.172 ms
tps = 0.000000 (without initial connection time)
```
If this behavior is intended, I wonder if we should clarify the TPS counting
logic more explicitly. For example:
```
<para>
This option is useful when your custom script may raise errors
such as unique constraint violations, but you want the benchmark
to continue despite individual statement failures. Failed transactions
are reported separately and included in latency statistics, but are not
counted as transactions actually processed for TPS.
</para>
```
Best regards,
--
Chao Li (Evan)
HighGo Software Co., Ltd.
https://www.highgo.com/
