cheeeee commented on PR #23928:
URL: https://github.com/apache/kafka/pull/23928#issuecomment-6388174389
Test notes for b7708148e2 (the four commits for the review), on trunk
d18f93f985.
Unit tests:
* Checkstyle, spotless and spotbugs for `clients` and `storage` pass.
* A full run of `:clients:test` (13757 tests), `:storage:test` (1355)
and `LogDirFailureTest`, `AlterReplicaLogDirsRequestTest` and
`ReplicaManagerTest` in `core` (144) passed.
* Three other full runs of `:storage:test` (two on be606d42c7, one on
b7708148e2) each had one test time out after 60 s, every time a
different one, in tiered storage integration tests:
`TopicBasedRemoteLogMetadataManagerTest.testTopicDoesNotExist`,
`TransactionsWithTieredStoreTest.testRecoveryFromEpochOverflow` and
`.testClassicFencingOnSendOffsets`. Both classes pass alone 3 times
out of 3 on b7708148e2 and on trunk, and a full run on trunk timed
out in `TransactionsWithTieredStoreTest.testClassicFencingOnCommit`
(see KAFKA-15772).
* The new `LogDirFailureChannelTest` fails if the error is logged on
every call again, and `testFailedReadCheckDoesNotBlockTheNextOne`
fails if the read check is not finished in a `finally`.
End to end, the scenarios of the earlier notes were rerun on
b7708148e2 on the same four systems, with the jars checked against the
build on each: Linux 6.8 (3 brokers in Docker, `dm-dust` and `strace`),
macOS 26 (3 brokers, an unreadable band of a disk image and an
interposer), OpenBSD 8.0 and Windows 10 (one node in QEMU, `blkdebug`).
| | TLS | PLAINTEXT |
|---|---|---|
| persistent media error, Linux and macOS | read fails on the network
thread, no re-read, dir offline | `sendfile` fails, the re-read fails, dir
offline |
| persistent media error, OpenBSD and Windows | the same | the transfer
fails (EFAULT, WSAEINVAL), the re-read fails, dir offline |
| `EIO` on the first read only, Linux and macOS | the first failure offlines
the dir, no re-read | 3 failures, each re-read (8 MiB) succeeds, dir online |
| `EIO` on the first read only, OpenBSD | the first failure offlines the
dir, no re-read | not run |
| `EIO` on the first read only, Windows | the disk driver retries the read,
which succeeds: no failure reaches Kafka, dir online | not run |
Every run that offlined the directory logged exactly one ERROR from
`LogDirFailureChannel`. On Linux and macOS the consumer read all 20000
records, from broker 2 once the directory was offline; the single
nodes on OpenBSD and Windows have no other replica. In the five-minute
TLS disconnect stress with a healthy disk (177 consumers started and
killed after 2 to 8 seconds on Linux, 178 on macOS), the scheduler of
broker 1 read no segment file and no directory went offline. All of
this matches the runs of 73fc46a880; a known read failure is now
reported inline, and an offline directory no longer logs an ERROR for
later failures.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
