[
https://issues.apache.org/jira/browse/IGNITE-29165?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Maksim Davydov updated IGNITE-29165:
------------------------------------
Description:
Ducktests can't run a node on a size-limited disk, so nothing reproduces what a
node does when one of its volumes fills up. This issue adds that support and
one reproducer per volume.
h3. Reproducers
{{disk_full_test.py}} fills one volume of a persistent server node under a
thin-client load. The node runs with {{IgniteNodeSpecFileOutput}}.
||Full volume||What the node does||
|logs: the log4j2 log, {{console.log}}, {{gc.log}}|keeps serving puts,
{{control.sh --baseline}} answers ACTIVE; the log, {{console.log}} and
{{gc.log}} lose the lines written while full and grow again once space is freed|
|logs, full when the node restarts|starts and serves puts, ACTIVE, with not a
line in its new log; the logs grow once space is freed|
|logs, filled by a service on the node that writes to {{System.out}}|keeps
serving puts, {{control.sh --state}} answers ACTIVE; {{System.out}} drops the
lines once the volume is full, and the service keeps writing|
|JFR repository|the JVM exits: "An irrecoverable error in Jfr"|
|one volume for the logs and all the data|the failure handler stops the node|
|WAL|keeps serving puts: segments are preallocated|
|WAL archive|the failure handler stops the node: SYSTEM_WORKER_TERMINATION,
"Failed to archive WAL segment"|
|partitions|the failure handler stops the node: CRITICAL_ERROR,
StorageException|
|work, with the binary metadata|the failure handler stops the node:
CRITICAL_ERROR, BinaryMetadataFileStore|
|WAL archive, filled by CDC with no consumer and no
{{cdcWalDirectoryMaxSize}}|the failure handler stops the node:
SYSTEM_WORKER_TERMINATION|
The tests are ignored by default. To run them:
{code}
./docker/run_tests.sh -n 4 -gj '{"disk_full_tests": true}' -t
./ignitetest/tests/disk_full_test.py
{code}
was:
Ducktests can't run a node on a size-limited disk, so nothing reproduces what a
node does when one of its volumes fills up. This issue adds that support and
one reproducer per volume.
h3. Reproducers
{{disk_full_test.py}} fills one volume of a persistent server node under a
thin-client load. The node runs with {{IgniteNodeSpecFileOutput}}.
||Full volume||What the node does||
|logs: the log4j2 log, {{console.log}}, {{gc.log}}|keeps serving puts,
{{control.sh --state}} answers ACTIVE; the log and {{console.log}} lose the
lines written while full and grow again once space is freed|
|logs, filled by a service on the node that writes to {{System.out}}|keeps
serving puts, {{control.sh --state}} answers ACTIVE; {{System.out}} drops the
lines once the volume is full, and the service keeps writing|
|JFR repository|the JVM exits: "An irrecoverable error in Jfr"|
|one volume for the logs and all the data|the failure handler stops the node|
|WAL|keeps serving puts: segments are preallocated|
|WAL archive|the failure handler stops the node: SYSTEM_WORKER_TERMINATION,
"Failed to archive WAL segment"|
|partitions|the failure handler stops the node: CRITICAL_ERROR,
StorageException|
|work, with the binary metadata|the failure handler stops the node:
CRITICAL_ERROR, BinaryMetadataFileStore|
|WAL archive, filled by CDC with no consumer and no
{{cdcWalDirectoryMaxSize}}|the failure handler stops the node:
SYSTEM_WORKER_TERMINATION|
The tests are ignored by default. To run them:
{code}
./docker/run_tests.sh -n 4 -gj '{"disk_full_tests": true}' -t
./ignitetest/tests/disk_full_test.py
{code}
> [ducktests] Add reproducers of a full disk volume under a server node
> ---------------------------------------------------------------------
>
> Key: IGNITE-29165
> URL: https://issues.apache.org/jira/browse/IGNITE-29165
> Project: Ignite
> Issue Type: Test
> Reporter: Maksim Davydov
> Assignee: Maksim Davydov
> Priority: Minor
>
> Ducktests can't run a node on a size-limited disk, so nothing reproduces what
> a node does when one of its volumes fills up. This issue adds that support
> and one reproducer per volume.
> h3. Reproducers
> {{disk_full_test.py}} fills one volume of a persistent server node under a
> thin-client load. The node runs with {{IgniteNodeSpecFileOutput}}.
> ||Full volume||What the node does||
> |logs: the log4j2 log, {{console.log}}, {{gc.log}}|keeps serving puts,
> {{control.sh --baseline}} answers ACTIVE; the log, {{console.log}} and
> {{gc.log}} lose the lines written while full and grow again once space is
> freed|
> |logs, full when the node restarts|starts and serves puts, ACTIVE, with not a
> line in its new log; the logs grow once space is freed|
> |logs, filled by a service on the node that writes to {{System.out}}|keeps
> serving puts, {{control.sh --state}} answers ACTIVE; {{System.out}} drops the
> lines once the volume is full, and the service keeps writing|
> |JFR repository|the JVM exits: "An irrecoverable error in Jfr"|
> |one volume for the logs and all the data|the failure handler stops the node|
> |WAL|keeps serving puts: segments are preallocated|
> |WAL archive|the failure handler stops the node: SYSTEM_WORKER_TERMINATION,
> "Failed to archive WAL segment"|
> |partitions|the failure handler stops the node: CRITICAL_ERROR,
> StorageException|
> |work, with the binary metadata|the failure handler stops the node:
> CRITICAL_ERROR, BinaryMetadataFileStore|
> |WAL archive, filled by CDC with no consumer and no
> {{cdcWalDirectoryMaxSize}}|the failure handler stops the node:
> SYSTEM_WORKER_TERMINATION|
> The tests are ignored by default. To run them:
> {code}
> ./docker/run_tests.sh -n 4 -gj '{"disk_full_tests": true}' -t
> ./ignitetest/tests/disk_full_test.py
> {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
