syahra712 opened a new pull request, #152:
URL: https://github.com/apache/solr-orbit/pull/152
### Description
`raw-request` reports every request as successful, whatever Solr answers.
`SolrClient.raw_request` returns the `requests.Response` without checking
the status. `SolrSearch` calls `resp.raise_for_status()` itself for this
reason, but `RawRequest` doesn't. It also isn't a `SolrRunner`, so it has no
error translation. A 400, 404 or 500 comes back as `{"weight": 1,
"http-status": 500, ...}` with no `success` key, and `execute_single` then
records it as a success. The error rate stays at 0 and the failing request's
latency is reported as normal.
On `solr:9`, `GET /api/collections/nope` and `POST /api/collections` with an
invalid body both return 500, and both were reported as successful operations.
This also affects converted workloads. `convert-workload` keeps
`raw-request` operations as they are, so the converted `elb_logs` workload
still sends `POST /elb_logs/_flush` and `POST /_plugins/_ppl` (the `t1-*` PPL
tasks). Solr answers those with 404, and today they are timed as successful
requests.
The fix:
- `RawRequest` now extends `SolrRunner` and calls `resp.raise_for_status()`.
- The existing decorator turns the `HTTPError` into a
`BenchmarkTransportError`, or `BenchmarkNotFoundError` for a 404.
- The sample is then recorded as `success: False` with `http-status` and an
error description, the same as the other runners. With the default `--on-error
continue` the run carries on.
Successful requests are unchanged. One behaviour change to note: workloads
that send OpenSearch-only paths through `raw-request` will now show those tasks
as errors instead of hiding them.
### Issues Resolved
None filed. I found it while reading the runner.
### Testing
- [x] New functionality includes testing
New `TestRawRequest` in `tests/unit/solr/test_runner.py`:
- A 200 still returns `http-status` and `weight`.
- A 500 raises `BenchmarkTransportError` with `status_code` 500.
- A 404 raises `BenchmarkNotFoundError`.
The 500 and 404 tests fail on `main`.
I also ran it against `solr:9` through `execute_single` with
`on_error="continue"`:
- `GET /api/collections/nope` now gives `{'success': False, 'error-type':
'transport', 'http-status': 500, ...}` with 0 ops, where it used to be a
success.
- `GET /api/collections` is still `success: True`.
The full `tests/` suite passes locally on Python 3.13 (1178 passed, 5
skipped), and so does `ruff check .`.
---
By submitting this pull request, I confirm that my contribution is made
under the terms of the Apache 2.0 license.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
