This is an automated email from the ASF dual-hosted git repository.

zstan pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/ignite.git


The following commit(s) were added to refs/heads/master by this push:
     new 8bd62d4eb54 IGNITE-29171 Document the recommended file system layout 
for Ignite server nodes (#13741)
8bd62d4eb54 is described below

commit 8bd62d4eb54451001ab80a26c586f6e0946cfb69
Author: didar shayarov <[email protected]>
AuthorDate: Tue Nov 3 16:32:32 2026 +0300

    IGNITE-29171 Document the recommended file system layout for Ignite server 
nodes (#13741)
---
 docs/_data/toc.yaml                                |   2 +
 .../apache/ignite/snippets/PersistenceTuning.java  |  44 ++++
 docs/_docs/logging.adoc                            |  13 +-
 .../persistence-tuning.adoc                        |   2 +
 .../perf-and-troubleshooting/troubleshooting.adoc  |   2 +
 docs/_docs/persistence/change-data-capture.adoc    |   4 +
 docs/_docs/persistence/native-persistence.adoc     |  49 +++-
 docs/_docs/persistence/persistence-tuning.adoc     |   2 +
 docs/_docs/persistence/snapshot-directory.adoc     |   6 +
 docs/_docs/server-node-file-system-layout.adoc     | 259 +++++++++++++++++++++
 docs/_docs/setup.adoc                              |  11 +-
 docs/_docs/snapshots/snapshots.adoc                |   4 +
 12 files changed, 387 insertions(+), 11 deletions(-)

diff --git a/docs/_data/toc.yaml b/docs/_data/toc.yaml
index cb50da29a88..8771e735699 100644
--- a/docs/_data/toc.yaml
+++ b/docs/_data/toc.yaml
@@ -59,6 +59,8 @@
       url: understanding-configuration
     - title: Setting Up
       url: setup
+    - title: Server Node File System Layout
+      url: server-node-file-system-layout
     - title: Configuring Logging
       url: logging
     - title: Resources Injection
diff --git 
a/docs/_docs/code-snippets/java/src/main/java/org/apache/ignite/snippets/PersistenceTuning.java
 
b/docs/_docs/code-snippets/java/src/main/java/org/apache/ignite/snippets/PersistenceTuning.java
index 4371a95d868..dbbe3d93216 100644
--- 
a/docs/_docs/code-snippets/java/src/main/java/org/apache/ignite/snippets/PersistenceTuning.java
+++ 
b/docs/_docs/code-snippets/java/src/main/java/org/apache/ignite/snippets/PersistenceTuning.java
@@ -106,4 +106,48 @@ public class PersistenceTuning {
         ignite.close();
     }
 
+    void fileSystemLayout() {
+        // tag::layout-paths[]
+        IgniteConfiguration cfg = new IgniteConfiguration();
+
+        // Work directory: node metadata, marshaller mappings, and other node 
files.
+        cfg.setWorkDirectory("/mnt/node-meta/work");
+
+        // Destination of full snapshots.
+        cfg.setSnapshotPath("/mnt/snapshots");
+
+        DataStorageConfiguration storeCfg = new DataStorageConfiguration();
+
+        // Enabling native persistence for the default data region.
+        
storeCfg.getDefaultDataRegionConfiguration().setPersistenceEnabled(true);
+
+        // Data and indexes.
+        storeCfg.setStoragePath("/mnt/ignite-data/db");
+
+        // Active WAL segments.
+        storeCfg.setWalPath("/mnt/wal/db/wal");
+
+        // Retained WAL segments.
+        storeCfg.setWalArchivePath("/mnt/wal-archive/db/wal/archive");
+
+        // The CDC directory has to be on the same file system as the WAL 
archive.
+        storeCfg.setCdcWalPath("/mnt/wal-archive/db/wal/cdc");
+
+        cfg.setDataStorageConfiguration(storeCfg);
+        // end::layout-paths[]
+    }
+
+    void extraSnapshotPaths() {
+        // tag::extra-snapshot-paths[]
+        DataStorageConfiguration storeCfg = new DataStorageConfiguration();
+
+        // Two data directories.
+        storeCfg.setExtraStoragePaths("/mnt/ignite-data-1/db", 
"/mnt/ignite-data-2/db");
+
+        // A snapshot location for each data directory. The arrays must have 
the same length,
+        // and the entries at the same index correspond to each other.
+        storeCfg.setExtraSnapshotPaths("/mnt/snapshots-1", "/mnt/snapshots-2");
+        // end::extra-snapshot-paths[]
+    }
+
 }
diff --git a/docs/_docs/logging.adoc b/docs/_docs/logging.adoc
index 2f0fdccf976..164d3c0ccd6 100644
--- a/docs/_docs/logging.adoc
+++ b/docs/_docs/logging.adoc
@@ -35,7 +35,7 @@ When a node starts, it outputs start-up information to the 
console, including th
 | `IGNITE_LOG_INSTANCE_NAME` | If the property is set, Ignite includes its 
instance name in log messages. |  Not set
 | `IGNITE_QUIET` | Set to `false` to disable the quiet mode and enable the 
verbose mode.
 In the verbose mode, the node logs a lot more information. | `true`
-| `IGNITE_LOG_DIR` | The directory where Ignite writes log files. | 
`$IGNITE_HOME/
+| `IGNITE_LOG_DIR` | The directory where the built-in JUL logger writes log 
files. It is read as an environment variable of the node process and has no 
effect on Log4j2 appenders. | `$IGNITE_HOME/
 work/log`
 | `IGNITE_DUMP_THREADS_ON_FAILURE` | Set to `true` to output thread dumps to 
the log when a critical error is caught. | `true`
 |===
@@ -44,7 +44,11 @@ work/log`
 == Default Logging
 By default, Ignite uses the java.util.logging (JUL) framework.
 If you start Ignite using the `ignite.sh|bat` script from the distribution 
package, Ignite uses `$IGNITE_HOME/config/java.util.logging.properties` as the 
default logging configuration file and outputs all messages to log files in the 
`$IGNITE_HOME/work/log` directory.
-You can override the default logging directory by specifying the 
`IGNITE_LOG_DIR` system property.
+You can override the default logging directory by setting the `IGNITE_LOG_DIR` 
environment variable of the node process. This variable is read once at startup 
and applies to the built-in JUL logger only; if you use Log4j2, change the file 
paths in the Log4j2 configuration instead.
+
+NOTE: The default log directory is located inside the work directory, which 
Ignite also uses to store metadata and, by default, data files.
+For production server nodes, bound total log retention and, where practical, 
use a separate file system, so that log growth cannot exhaust the space 
required by checkpoints, WAL, and node startup.
+See link:server-node-file-system-layout[Server Node File System Layout].
 
 If you use Ignite as a library in your application, the default logging 
configuration includes only console handler at INFO level.
 You can provide a custom configuration file via the 
`java.util.logging.config.file` system property.
@@ -76,6 +80,11 @@ tab:C++[unsupported]
 
 In the above example, the path to `log4j2-config.xml` can be either an 
absolute path, a local path relative to META-INF in classpath or to 
`IGNITE_HOME`. An example log4j2 configuration file can be found in the 
distribution package (`$IGNITE_HOME/config/ignite-log4j.xml`).
 
+NOTE: The configuration shipped in the distribution package writes to 
`$IGNITE_HOME/work/log` and rolls individual files by time and by size.
+Its file name contains both an index and a date, so the implicit default 
rollover strategy keeps a limited number of indexed archives of the current 
period, but archives left from previous dates are never removed, and its 
`consistency.log` appender is not rolled at all.
+Add a `Delete` action to `DefaultRolloverStrategy` to bound the directory as a 
whole and bound `consistency.log` separately.
+See link:server-node-file-system-layout#logs-and-diagnostic-output[Server Node 
File System Layout] for the recommended log placement and retention.
+
 NOTE: Log4j2 supports runtime reconfiguration, i.e. changes in the 
configuration file is applied without the need to restart the application.
 
 NOTE: `log4j2-config.xml` enables colorization of console output by default. 
This feature works out of the box on many operating systems, except for 
Windows. If you want to enable this feature on Windows, please follow the 
https://logging.apache.org/log4j/2.x/manual/layouts.html#ANSI_Styling_on_Windows[official
 log4j2 instructions].
diff --git a/docs/_docs/perf-and-troubleshooting/persistence-tuning.adoc 
b/docs/_docs/perf-and-troubleshooting/persistence-tuning.adoc
index 54cd57d63c8..bee1d429e1c 100644
--- a/docs/_docs/perf-and-troubleshooting/persistence-tuning.adoc
+++ b/docs/_docs/perf-and-troubleshooting/persistence-tuning.adoc
@@ -65,6 +65,8 @@ tab:C++[unsupported]
 
 Consider using separate drives for data files and 
link:persistence/native-persistence#write-ahead-log[Write-Ahead-Logging (WAL)].
 Ignite actively writes to both the data and WAL files.
+Keeping the active WAL on its own device reduces the competition between 
checkpoint page writes and WAL writes, which helps keep write latency stable 
while a checkpoint is in progress.
+See link:server-node-file-system-layout[Server Node File System Layout] for 
the recommended placement of the node files.
 
 The example below shows how to configure separate paths for the data storage, 
WAL, and WAL archive:
 
diff --git a/docs/_docs/perf-and-troubleshooting/troubleshooting.adoc 
b/docs/_docs/perf-and-troubleshooting/troubleshooting.adoc
index 2ed39567109..6afe0151e10 100644
--- a/docs/_docs/perf-and-troubleshooting/troubleshooting.adoc
+++ b/docs/_docs/perf-and-troubleshooting/troubleshooting.adoc
@@ -29,6 +29,8 @@ On some systems, the default location for Ignite persistence 
files might be unde
 * Change the location of all persistence files using the 
`DataStorageConfiguration` APIs, such as `setStoragePath(...)`,
 `setWalPath(...)`, and `setWalArchivePath(...)`
 
+See link:server-node-file-system-layout[Server Node File System Layout] for 
the recommended placement of persistence files, logs, and diagnostic output on 
production server nodes.
+
 == Cluster Doesn't Start After Field Type Changes
 
 When developing your application, you may need to change the type of a custom
diff --git a/docs/_docs/persistence/change-data-capture.adoc 
b/docs/_docs/persistence/change-data-capture.adoc
index 2f2464b0267..a83e1eda249 100644
--- a/docs/_docs/persistence/change-data-capture.adoc
+++ b/docs/_docs/persistence/change-data-capture.adoc
@@ -49,9 +49,13 @@ On startup event processing will be continued from the last 
saved state.
 |Name |Description | Default value
 | `DataRegionConfiguration#cdcEnabled` | Flag to enable CDC on the server 
node. | `false`
 | `DataStorageConfiguration#cdcWalPath` | Path to the CDC directory | 
`"db/wal/cdc"`
+| `DataStorageConfiguration#cdcWalDirectoryMaxSize` | Maximum size of the CDC 
directory in bytes. If adding a segment would exceed the limit, the CDC link is 
skipped and the changes in that segment are lost. | `0` (no limit)
 | `DataStorageConfiguration#walForceArchiveTimeout` | Timeout to forcefully 
archive the WAL segment even it is not complete. | `-1` (disabled)
 |===
 
+NOTE: CDC links are hard links to WAL archive segments, so `cdcWalPath` must 
be on the same file system as `walArchivePath`, and the space retained by CDC 
is not bounded by `maxWalArchiveSize`.
+See link:server-node-file-system-layout#account-for-hard-links[Server Node 
File System Layout] for the capacity planning rules.
+
 === CDC Application
 
 CDC is configured in the same way as the Ignite node - via the spring XML file:
diff --git a/docs/_docs/persistence/native-persistence.adoc 
b/docs/_docs/persistence/native-persistence.adoc
index 5ca6e86d623..6362fdd84f4 100644
--- a/docs/_docs/persistence/native-persistence.adoc
+++ b/docs/_docs/persistence/native-persistence.adoc
@@ -156,6 +156,10 @@ tab:C++[unsupported]
 
 You can also change the WAL and WAL archive paths to point to directories 
outside of the storage directory. Refer to the next section for details.
 
+NOTE: If the Ignite paths are left unset, data, the active WAL, the WAL 
archive, and CDC all live in the work directory and therefore on one file 
system.
+The CDC directory and the snapshot root of an incremental snapshot chain have 
to be on the same file system as the WAL archive, because both use hard links 
to archived segments.
+See link:server-node-file-system-layout[Server Node File System Layout] for 
the recommended placement of the node directories, the capacity they need, and 
how the write load affects the layout.
+
 == Write-Ahead Log
 
 The write-ahead log is a log of all data modifying operations (including 
deletes) that happen on a node. When a page is updated in RAM, the update is 
not directly written to the partition file but is appended to the tail of the 
WAL.
@@ -207,11 +211,21 @@ If a node is terminated abruptly during update 
operations, it is very likely tha
 
 
 === WAL Archive
-The WAL archive is used to store WAL segments that may be needed to recover 
the node after a crash. The number of segments kept in the archive is such that 
the total size of all segments does not exceed the specified size of the WAL 
archive.
+The WAL archive is used to store WAL segments that may be needed to recover 
the node after a crash. The number of segments kept in the archive is defined 
by the cleanup rules described below.
+
+By default, the maximum size of the WAL archive is `maxWalArchiveSize`, whose 
default value is 1 GiB. You can change that value in the <<Configuration 
Properties,configuration>>.
+
+Cleanup starts when the archive exceeds `minWalArchiveSize`, not 
`maxWalArchiveSize`. By default, `minWalArchiveSize` is half of 
`maxWalArchiveSize`, that is 512 MiB.
+The cleanup runs asynchronously in the background. It removes the oldest 
segments, stopping at the last completed checkpoint and at the segments that 
are reserved by a running checkpoint or by a reader. It removes whole segments 
while the archive stays above the threshold, so the remaining size can end up 
below it.
+
+`maxWalArchiveSize` is the upper threshold, and it is not a strict disk-space 
quota. Cleanup runs asynchronously and preserves the WAL needed for recovery, 
so the archive can exceed this value. When the archive reaches it, Ignite 
forcibly releases the reservations of the segments that the last completed 
checkpoint already covers. This allows cleanup of older segments.
 
-By default, the maximum size of the WAL archive (total space it occupies on 
disk) is defined as 4 times the size of the 
link:persistence/persistence-tuning#adjusting-checkpointing-buffer-size[checkpointing
 buffer]. You can change that value in the <<Configuration 
Properties,configuration>>.
+Removing an archived segment does not release its disk blocks while a CDC or 
snapshot hard link still points to it, so the space taken on the file system 
can be larger than the archive size.
 
-CAUTION: Setting the WAL archive size to a value lower than the default may 
impact performance and should be tested before being used in production.
+CAUTION: Setting these thresholds to values lower than the defaults should be 
tested before being used in production. The typical amount of retained history 
follows `minWalArchiveSize`, so lowering it shortens the WAL history available 
for recovery, incremental snapshots, and historical rebalancing.
+CDC is not affected: it keeps its own hard link to each captured segment, so 
its history is retained independently of the archive cleanup and is bounded by 
`cdcWalDirectoryMaxSize` instead.
+
+NOTE: `walHistorySize` is deprecated and ignored. The archive is controlled by 
`maxWalArchiveSize` and `minWalArchiveSize` only.
 
 :walXmlFile: code-snippets/xml/wal.xml
 
@@ -245,6 +259,8 @@ tab:C++[unsupported]
 WARNING: Disabling or enabling WAL should only be done on stable topology: all 
baseline nodes should be present, no nodes should join or leave the cluster for 
the duration of this operation. Otherwise, cache may be stuck in inconsistent 
state. If that happens, it is advised to destroy the affected cache(s).
 
 There are situations when it is reasonable to have the WAL disabled to get 
better performance. For instance, it is useful to disable WAL during initial 
data loading and enable it after the pre-loading is complete.
+Disabling the WAL is a planned administrative operation with recovery 
implications, not a way to reclaim disk space: follow the procedure in this 
section and keep the files that the recovery needs.
+Do not remove WAL segments, checkpoint files, or other files of the persistent 
store manually outside a documented maintenance or recovery procedure. Manual 
changes to the internal persistent structure are not supported, and correct 
cluster operation is not guaranteed after such an intervention.
 
 ////
 todo: add c++ examples
@@ -275,13 +291,18 @@ ALTER TABLE Person LOGGING
 tab:C++[unsupported]
 --
 
-WARNING: If WAL is disabled and you restart a node, all data is removed from 
the persistent storage on that node. This is implemented because without WAL 
data consistency cannot be guaranteed in case of node crash or restart.
+WARNING: Disabling WAL through the runtime API removes local crash-recovery 
guarantees for the affected cache groups.
+If a node stops while WAL is disabled, recovery may require maintenance and 
cleanup of affected persistent files.
+`WALMode.NONE` has different shutdown and recovery behavior; see <<WAL Modes>>.
 
 === WAL Archive Compaction
+WAL archiving and WAL archive compaction are different mechanisms: archiving 
copies completed segments to the archive directory without compressing them, 
while compaction is a separate step that compresses segments that are already 
archived.
+
 You can enable WAL Archive compaction to reduce the space occupied by the WAL 
Archive.
-By default, WAL Archive contains segments for the last 20 checkpoints (this 
number is configurable).
+Compaction applies to the segments that are already archived; how much history 
is kept is defined by `minWalArchiveSize` and `maxWalArchiveSize`, not by a 
checkpoint count.
 If compaction is enabled, all archived segments that are 1 checkpoint old are 
compressed in ZIP format.
 If the segments are needed (for example, to re-balance data between nodes), 
they are uncompressed to RAW format.
+Compaction reduces the space the archive occupies, but it uses CPU and adds 
I/O, so the actual cost depends on the load and on the compression level.
 
 See the <<Configuration Properties>> section below to learn how to enable WAL 
archive compaction.
 
@@ -329,6 +350,7 @@ It is safe to disable WAL archiving because a cluster 
without the WAL archive pr
 
 To disable archiving, set the WAL path and the WAL archive path to the same 
value.
 In this case, Ignite does not copy segments to the archive; instead, it 
creates new segments in the WAL folder.
+WAL recording itself stays enabled: only the separate archive directory and 
the copying of segments to it are disabled.
 Old segments are deleted as the WAL grows, based on the WAL Archive size 
setting.
 
 
@@ -342,7 +364,17 @@ Checkpointing and write-ahead logging are designed to 
ensure durability of data
 
 image:images/checkpointing-persistence.png[]
 
-This process helps to utilize disk space frugally by keeping pages in the most 
up-to-date state on disk. After a checkpoint is passed, you can delete the WAL 
segments that were created before that point in time.
+This process helps to utilize disk space frugally by keeping pages in the most 
up-to-date state on disk. After a checkpoint completes, Ignite can clean up 
older WAL segments when they are no longer needed.
+
+[[checkpoint-marker-retention]]
+=== Checkpoint Marker Retention
+
+Checkpoint progress is recorded in START and END marker files in the `cp` 
directory of the node storage directory.
+With a finite `maxWalArchiveSize`, obsolete checkpoint marker files are 
removed as the WAL cleanup boundary advances.
+If the WAL is written slowly, a large `walSegmentSize` can hold the records of 
many checkpoints before the boundary moves to the next segment.
+Frequent checkpoints with changes can therefore leave many marker files in 
`cp`.
+Consider `checkpointFrequency`, `walSegmentSize`, the WAL write rate, and the 
archive retention together.
+When automatic WAL archive cleanup is disabled, Ignite attempts to trim 
checkpoint history after each checkpoint according to the 
`IGNITE_PDS_MAX_CHECKPOINT_MEMORY_HISTORY_SIZE` system property. WAL 
reservations can delay this cleanup.
 
 See the following related documentation:
 
@@ -368,5 +400,8 @@ The following table describes some properties of 
link:{javadoc_base_url}/org/apa
 |`walMode` | <<WAL Modes,Write-ahead logging mode>>. | `LOG_ONLY`
 
 | `walCompactionLevel` | WAL archive compression level. `1` indicates the 
fastest speed, and `9` indicates the best compression. | `1`
-|`maxWalArchiveSize`  | The maximum size (in bytes) the WAL archive can occupy 
on the file system. | Four times the size of the 
link:persistence/persistence-tuning#adjusting-checkpointing-buffer-size[checkpointing
 buffer].
+|`maxWalArchiveSize`  | The upper threshold (in bytes) of the WAL archive. 
When the archive reaches it, Ignite forcibly releases the reservations of the 
segments that the last completed checkpoint already covers, so that the cleanup 
can delete them. It is not a strict disk-space quota: the cleanup runs 
asynchronously and preserves the WAL needed for recovery, so the archive can 
exceed this value. | 1 GiB
+|`minWalArchiveSize` | The threshold (in bytes) that makes segments eligible 
for cleanup. The cleanup runs in the background and removes the oldest segments 
while the archive stays above this value. Segments are removed as whole files, 
so the remaining size can be below it. | Half of `maxWalArchiveSize`
+|`walSegments` | The number of fixed-size segments used for active WAL 
rotation when archiving is enabled. Plan space for `walSegments` × 
`walSegmentSize` bytes plus a rotation reserve. Size the WAL archive 
separately. | `10`
+|`checkpointFrequency` | The interval between scheduled checkpoints, in 
milliseconds. | `180000` (3 minutes)
 |=======================================================================
diff --git a/docs/_docs/persistence/persistence-tuning.adoc 
b/docs/_docs/persistence/persistence-tuning.adoc
index 293d9de110e..e581ad7f67e 100644
--- a/docs/_docs/persistence/persistence-tuning.adoc
+++ b/docs/_docs/persistence/persistence-tuning.adoc
@@ -63,6 +63,8 @@ tab:C++[unsupported]
 
 Consider using separate drives for data files and 
link:persistence/native-persistence#write-ahead-log[Write-Ahead-Logging (WAL)].
 Ignite actively writes to both the data and WAL files.
+Keeping the active WAL on its own device reduces the competition between 
checkpoint page writes and WAL writes, which helps keep write latency stable 
while a checkpoint is in progress.
+See link:server-node-file-system-layout[Server Node File System Layout] for 
the recommended placement of the node files.
 
 The example below shows how to configure separate paths for the data storage, 
WAL, and WAL archive:
 
diff --git a/docs/_docs/persistence/snapshot-directory.adoc 
b/docs/_docs/persistence/snapshot-directory.adoc
index a046fabb31a..10143b702a9 100644
--- a/docs/_docs/persistence/snapshot-directory.adoc
+++ b/docs/_docs/persistence/snapshot-directory.adoc
@@ -24,6 +24,12 @@ Ignite Persistence routines, it's suggested to store the 
snapshot and persistenc
 You can avoid this interference between Ignite Native persistence and 
snapshotting
 by either changing 
link:persistence/native-persistence#configuring-persistent-storage-directory[storage
 directories of the persistence files]
 or overriding the default snapshots' location as shown below:
+
+NOTE: For full snapshots without increments, prefer a separate file system 
where the available storage allows it, and separate storage where available: 
the snapshot adds a second read and write pass over the data and competes with 
the traffic the cluster is serving.
+If you create incremental snapshots, the snapshot root of the base full 
snapshot and of its increments has to be on the *same file system as the WAL 
archive*, because increments are hard links to archived WAL segments.
+Different partitions of one physical disk do not satisfy this requirement, and 
the retained chains occupy space after the archive cleanup.
+See link:server-node-file-system-layout[Server Node File System Layout] for 
the complete placement and capacity plan.
+
 [tabs]
 --
 tab:XML[]
diff --git a/docs/_docs/server-node-file-system-layout.adoc 
b/docs/_docs/server-node-file-system-layout.adoc
new file mode 100644
index 00000000000..9d7c83e48d6
--- /dev/null
+++ b/docs/_docs/server-node-file-system-layout.adoc
@@ -0,0 +1,259 @@
+// Licensed to the Apache Software Foundation (ASF) under one or more
+// contributor license agreements.  See the NOTICE file distributed with
+// this work for additional information regarding copyright ownership.
+// The ASF licenses this file to You under the Apache License, Version 2.0
+// (the "License"); you may not use this file except in compliance with
+// the License.  You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+= Server Node File System Layout
+:javaFile: {javaCodeDir}/PersistenceTuning.java
+
+This page brings together recommendations for placing the directories used by 
the Ignite server node: data and indexes, the active WAL, the WAL archive, CDC, 
the work directory, snapshots, logs, and JVM diagnostics.
+Use it to plan disk space, manage retained files, and keep log and diagnostic 
growth away from the space that the node needs.
+
+These recommendations are intended for production server nodes.
+On nodes without native persistence, the guidance for the work directory, 
logs, and JVM diagnostics still applies.
+
+== Separate File Systems
+
+Node directories differ in how they grow and how they are written.
+For write-heavy workloads, place the data files and the active WAL on separate 
devices: they are written in different patterns and otherwise compete for the 
same device.
+Other node directories can share a file system. If they do, size the file 
system for their combined volume, bound the retained logs and diagnostic 
output, and leave free space for checkpoints and WAL rotation.
+A dedicated file system keeps unrelated growth away from the space the node 
needs, and it separates the node from the other software on the host.
+
+Running out of disk space can stop a node and prevent it from restarting.
+
+Start with the split between the data files and the active WAL, and add 
volumes for the other node directories only where their growth, retention, or 
I/O patterns require it.
+The <<Recommended Placement>> section lists the preferred placement for each 
node directory and the capacity it needs.
+
+== Recommended Placement
+
+[cols="1,1,3", options="header"]
+|===
+|Node directories |Configuration |Placement
+
+|Data and indexes
+|`DataStorageConfiguration.storagePath` and 
`DataStorageConfiguration.extraStoragePaths`
+|Durable data file systems. On write-heavy workloads, give them a device of 
their own, see <<Separate File Systems>>. Keep logs, diagnostic files, and 
retained backups off them, so that unrelated growth does not take the space 
needed by checkpoint writes and recovery.
+
+|Active WAL
+|`DataStorageConfiguration.walPath`
+|Its own file system, and on write-heavy workloads its own device, with room 
for the active segments and a rotation reserve, see <<Sizing the WAL Volume>>. 
See <<Configuration Example>> for the path setting. A separate device for the 
active WAL and the data files reduces the competition between checkpoint page 
writes and WAL writes.
+
+|WAL archive
+|`DataStorageConfiguration.walArchivePath`
+|A separate file system where the available storage allows it, because the 
archive grows independently of the data volume. Otherwise plan its capacity 
together with the other node directories that share the file system. Size it 
for the retained segments, see <<Sizing the WAL Archive>>, and for the features 
that share it: CDC and incremental snapshots, see <<Account for Hard Links>>.
+
+|CDC
+|`DataStorageConfiguration.cdcWalPath`
+|The *same file system as the WAL archive*, because CDC uses hard links to 
archived WAL segments. A separate CDC directory does not isolate its disk-space 
usage. Bound its retention with `cdcWalDirectoryMaxSize`.
+
+|Work directory
+|`IgniteConfiguration.workDirectory`
+|Durable storage with free space to spare. The directory keeps binary 
metadata, marshaller mappings, and other node files even when data and WAL are 
configured elsewhere. Do not treat it as disposable temporary storage, and keep 
logs and diagnostic output out of the space it needs.
+
+|Snapshots
+|`IgniteConfiguration.snapshotPath` and 
`DataStorageConfiguration.extraSnapshotPaths`
+|For full snapshots without increments, a separate file system where the 
available storage allows it, and, where available, separate storage: the 
snapshot adds a second read and write pass over the data. For incremental 
snapshots, the snapshot root must be on the same file system as the WAL 
archive, see <<Account for Hard Links>>. Keep independent backup copies outside 
the node storage.
+
+|Ignite logs
+|Log4j2 appenders in the logging configuration; `java.util.logging.properties` 
and the `IGNITE_LOG_DIR` environment variable for the built-in JUL logger
+|Storage where the logs cannot take the space that data, WAL, and checkpoints 
need: a separate file system where the available storage allows it, otherwise a 
bounded retained volume. Bound the total retained volume rather than individual 
files. `IGNITE_LOG_DIR` affects only the built-in JUL logger, not Log4j2 
appenders.
+
+|JVM and OS diagnostics
+|GC logging options; JFR repository and output settings; heap and core dump 
paths; monitoring-tool output
+|Storage that is not needed by data, WAL, and the work directory, with 
retained output bounded and space reserved for large dumps, see <<Logs and 
Diagnostic Output>>.
+|===
+
+Keep Ignite files and diagnostic output off the operating system's root file 
system.
+
+Keep persistent data, the active WAL, the WAL archive, and node metadata on 
durable storage.
+Do not place them in temporary directories such as `/tmp`, which may be 
cleaned automatically, or on volatile file systems such as `tmpfs`.
+A temporary directory is not necessarily a `tmpfs` mount, but both are 
unsuitable for files that have to survive a host reboot.
+
+Logs and JFR data may use temporary storage if losing them on cleanup or a 
host reboot is acceptable.
+Limit retained output, and account for memory use when using `tmpfs`.
+See <<Logs and Diagnostic Output>> for the JFR repository requirements.
+
+The Ignite-managed defaults are all relative to the work directory, so leaving 
the paths unset puts data, the active WAL, and the WAL archive on one file 
system.
+By default, `storagePath` is `${IGNITE_HOME}/work/db/node{IDX}-{UUID}`, 
`walPath` is `${IGNITE_HOME}/work/db/wal/`, and `walArchivePath` is 
`${IGNITE_HOME}/work/db/wal/archive/`.
+See 
link:persistence/native-persistence#configuring-persistent-storage-directory[Configuring
 Persistent Storage Directory].
+
+If several nodes share a host, give each node its own work directory and its 
own set of node directories, and plan the file systems for their combined use.
+See link:perf-and-troubleshooting/general-perf-tips[General Performance Tips] 
for the recommended number of server nodes per host.
+
+== Configuration Example
+
+The example below shows one possible layout.
+
+[source,java]
+----
+include::{javaFile}[tags=layout-paths,indent=0]
+----
+
+The `/mnt/...` locations represent mount points prepared in advance; different 
directory names on one file system do not isolate anything.
+The WAL archive and CDC directories share `/mnt/wal-archive`.
+
+See the following sections for the individual paths:
+
+* link:setup#configuring-work-directory[Configuring Work Directory]
+* link:persistence/persistence-tuning#keep-wals-separately[Keep WALs 
Separately]
+* 
link:persistence/snapshot-directory#configuring-snapshot-directory[Configuring 
Snapshot Directory]
+
+=== Additional Data and Snapshot Directories
+
+If the data is spread over several directories with 
`DataStorageConfiguration.extraStoragePaths`, configure the snapshot locations 
for them as well:
+For full snapshots of data stored in `extraStoragePaths`, use 
`DataStorageConfiguration.extraSnapshotPaths` to set additional snapshot 
locations.
+When configured, the two arrays must have the same length; entries at the same 
index correspond to each other.
+
+[source,java]
+----
+include::{javaFile}[tags=extra-snapshot-paths,indent=0]
+----
+
+Additional snapshot paths are ordinary snapshot destinations: they do not have 
to be on the WAL archive file system.
+Do not pass an explicit snapshot path together with them: when a path is 
specified for the snapshot itself, all snapshot files are stored inside it and 
the additional locations are not used.
+
+NOTE: Additional snapshot locations currently apply to full snapshots only. 
They are not supported for incremental snapshots.
+
+== Sizing the WAL Volume
+
+The active WAL volume has to hold the segments that exist at the same time 
plus a reserve for rotation.
+A node keeps 10 active segments by default (`walSegments`), and they are 
filled in a cyclical order, so the volume has to hold that many segments of 
`walSegmentSize` plus the rotation reserve.
+Changing `walSegmentSize` changes that footprint as well as the number of 
rotations.
+Large segments combined with frequent checkpoints and a low WAL write rate can 
increase the number of retained checkpoint marker files, see 
link:persistence/native-persistence#checkpoint-marker-retention[Checkpoint 
Marker Retention].
+Monitor the volume like the other volumes, and keep the reserve for the next 
segment: a write can fail inside an already allocated segment as well.
+
+The WAL mode defines the durability guarantees of a commit rather than the 
size of the volume, see link:persistence/native-persistence#wal-modes[WAL 
Modes].
+
+Do not disable the WAL to stop unexpected volume growth.
+Disabling the WAL is a planned operation, such as initial data loading, 
performed on a stable topology, and it changes what the node can recover: 
follow the documented procedure and keep the files that the recovery needs.
+Do not remove WAL segments, checkpoint files, or other files of the persistent 
store manually outside a documented maintenance or recovery procedure, because 
correct cluster operation is not guaranteed after such an intervention.
+See link:persistence/native-persistence#disabling-wal[Disabling WAL].
+
+Setting `walPath` and `walArchivePath` to the same directory disables copying 
segments to a separate archive directory, not WAL recording.
+In that mode segments are written to the WAL directory under their absolute 
indexes instead of being copied into a separate directory, and the retained 
segments are removed according to `minWalArchiveSize` and `maxWalArchiveSize`.
+Plan the capacity of that file system like the capacity of a data volume, 
because it holds both the active and the retained segments.
+Incremental snapshots require the WAL archive to be enabled, and CDC receives 
its links from the archiver, so standalone CDC does not capture new changes 
without an archive.
+See link:persistence/native-persistence#disabling-wal-archive[Disabling WAL 
Archive].
+
+== Sizing the WAL Archive
+
+Cleanup starts when the archive reaches `minWalArchiveSize`. By default, this 
threshold is half of `maxWalArchiveSize`: 512 MiB with the default maximum of 1 
GiB.
+See link:persistence/native-persistence#wal-archive[WAL Archive] for the 
cleanup rules.
+
+The cleanup runs in the background and deletes whole segments, so the 
remaining size can end up below the threshold.
+It does not remove the segments that the recovery still needs.
+`maxWalArchiveSize` is not a strict disk-space quota: cleanup runs 
asynchronously and preserves the WAL that the recovery needs, so the archive 
can exceed this value.
+When the archive reaches `maxWalArchiveSize`, Ignite releases the reservations 
that the last completed checkpoint already covers, which allows cleanup of 
older segments.
+
+Removing an archived segment does not release its disk blocks while a CDC or 
snapshot hard link still points to it, so plan the capacity of the file system 
from the retained links as well, see <<Account for Hard Links>>.
+
+Size the file system for the archive retention, the expected CDC backlog, and 
the retained incremental snapshot chains.
+Measure the retained WAL volume under representative load, account for the 
compression ratio when WAL archive compaction is enabled, and leave free space 
for temporary growth.
+
+Lowering the thresholds below their defaults is allowed but should be tested: 
the typical amount of retained history follows `minWalArchiveSize`, so lowering 
it shortens the WAL history available for recovery, incremental snapshots, and 
historical rebalancing.
+CDC is not affected: it keeps its own hard link to each captured segment, so 
its history is bounded by `cdcWalDirectoryMaxSize` instead.
+Retention is not controlled by `walHistorySize`: the parameter is deprecated 
and ignored.
+
+WAL archiving copies completed segments to the archive directory; it does not 
compress them.
+WAL archive compaction is a separate option, disabled by default, that 
compresses eligible segments into ZIP files: it reduces disk space but uses CPU 
and adds I/O.
+See link:persistence/native-persistence#wal-archive-compaction[WAL Archive 
Compaction].
+
+Capacity is not the only requirement: the archiver has to write at the peak 
rate at which the active WAL produces segments, and the same device may also 
serve compaction and the consumers that read the archive.
+If the archiver falls behind, the working segments run out and the write waits 
for space that only archiving can free.
+Provision sequential throughput for the peak WAL rate plus compaction, and 
verify it on the archive device itself.
+
+Archived segments are written once, mostly sequentially, and are read again 
only for recovery, CDC, incremental snapshots, historical rebalancing, and 
archive compaction.
+This makes the archive a candidate for cheaper, higher-capacity, slower media, 
such as HDDs, while fast devices stay dedicated to the data and the active WAL.
+Slower media also extend the time to replay archived segments during recovery, 
to catch up CDC, and to create an incremental snapshot, so take these 
objectives into account when choosing the device.
+
+== Account for Hard Links
+
+Keep the CDC directory and the snapshot root used for incremental snapshots on 
the same file system as the WAL archive.
+Both features use hard links to archived WAL segments.
+
+The links are created on segment files, not on the CDC or snapshot 
directories, so a separate directory on the same file system does not separate 
its free space.
+
+For CDC, `maxWalArchiveSize` does not bound the retained space.
+Configure a positive `cdcWalDirectoryMaxSize` appropriate for the expected lag 
and for the capacity shared with the archive; the default is unlimited.
+This setting does not reserve free space, so leave room for the archive and 
the other users of the file system.
+If adding a segment would exceed the limit, Ignite skips its CDC link and the 
corresponding changes are unavailable to CDC.
+The skip is reported in the node log, and `ignite-cdc.sh` notices the gap only 
when it reaches it, which can happen long after the event.
+Treat the first skipped segment as a reason to re-sync CDC, because this limit 
trades CDC continuity for bounded retention.
+
+For incremental snapshots, the snapshot root of the base full snapshot and of 
its increments has to be on the same file system as the WAL archive.
+Snapshot links can retain disk blocks after the archive cleanup, so include 
the retained snapshot chains in the capacity plan.
+Additional snapshot locations configured with `extraSnapshotPaths` do not 
change this requirement and are not used for incremental snapshots.
+
+Even when a full snapshot destination is on a separate file system, 
concurrent-change tracking can require temporary space on the live data file 
systems, so reserve space on both the source and the destination.
+If the destination has its own file system, a destination that fills stops the 
snapshot operation while the node keeps serving; if the snapshot shares a file 
system with data or WAL, the same exhaustion can stop the node.
+Store independent backup copies outside the node storage.
+
+== Logs and Diagnostic Output
+
+Ignite does not bound the log directory as a whole.
+Each logger bounds something slightly different, and the defaults are weaker 
than they look, so check the effective configuration instead of assuming that 
rotation is enough.
+
+* *Log4j2.* The default `config/ignite-log4j.xml` writes to 
`${IGNITE_HOME}/work/log` and rolls files by time and by size (10 MB).
+Its file name contains both an index and a date, so the implicit default 
rollover strategy keeps a limited number of indexed archives of the current 
period, but nothing removes the archives left from previous dates.
+Add a `Delete` action to `DefaultRolloverStrategy`, by age or by total size, 
so that the directory as a whole stays bounded.
+The `consistency.log` appender is a plain file appender and is not rolled at 
all; bound it separately.
+* *JUL.* `config/java.util.logging.properties` sets both `limit` and `count`.
+The file name contains the node ID, so `count` bounds the file set of one node 
ID only, and the sets produced by earlier node IDs stay on disk.
+Clean them up separately or size the volume for several generations.
+The `IGNITE_LOG_DIR` environment variable relocates this directory; it has no 
effect on the Log4j2 appenders.
+* *GC logs.* For JVM unified logging, rotation parameters belong to the 
output-options section and go after the decorator list, for example 
`-Xlog:gc*:file=<path>::filecount=N,filesize=M`.
+* *stdout and stderr.* Ignite does not rotate a redirected stdout file.
+Prefer capturing and rotating the output with the service manager or the 
container runtime.
+If you redirect it to a file yourself, rotate that file externally and be 
aware of two limits: deleting the file does not release its space while the JVM 
keeps it open, and `copytruncate` needs space for the copy and can lose the 
lines written between the copy and the truncation.
+Enable console appenders only if you intend to capture their output, otherwise 
redirecting stdout duplicates the log.
+* *Diagnostics.* Heap dumps, core dumps, saved recordings, and monitoring 
output are not covered by any of the above and must be managed separately.
+
+Size the log volume for the total retention that the configured policies can 
leave behind, plus the captured stdout.
+
+For JFR, configure both the disk repository and the recording output locations.
+By default the repository is created in the temporary directory of the 
operating system, and setting a recording filename alone does not relocate it.
+`maxsize` and `maxage` are per-recording options: they bound the data that 
each individual recording keeps in the repository, not the size of the 
repository as a whole and not the age of the `.jfr` files already written out.
+Some JDK versions can terminate the JVM if the JFR repository runs out of disk 
space. This bypasses the Ignite failure handler, so a separate diagnostic file 
system protects the Ignite data volumes from JFR growth but does not prevent 
this JVM failure. Set retention limits for each recording, reserve free space 
for concurrent recordings and dumps, and monitor repository usage.
+
+The Control Script writes through the same logger configuration and opens its 
log file before the command runs, so a full or unwritable log directory can 
prevent the utility from starting.
+Run it from a host where the log directory is writable and has free space.
+
+== Plan Free Space
+
+Size every backing file system with free space to spare, including those used 
by additional data paths and by diagnostic tools.
+Monitor the space that the logs and diagnostic files take.
+On Linux, monitor both available bytes and free inodes, taking account of 
quotas and the space available to the Ignite operating-system user.
+
+Free space is not the only property of storage that matters.
+Check storage performance under representative load: use checkpoint duration, 
page-write time, and fsync time to assess whether the storage keeps up with the 
updates, see link:monitoring-metrics/new-metrics#data-storage[Data Storage].
+For performance checks, monitor device IOPS, throughput, and I/O latency 
alongside these metrics and the WAL write rate (`WalWritingRate`).
+
+There is no universal safe reserve.
+It depends on the write rate, the size of the data region, the checkpoint 
interval, and the time needed to react, so derive it from measurements on your 
own workload: the growth of the file system under the peak expected load and 
the time needed to react and free space.
+Leave enough free space for that growth and for checkpoints and WAL rotation 
to finish.
+
+Stopping application writes does not stop disk growth: the checkpoint that 
follows still has to flush dirty pages.
+That is why the reaction belongs at a threshold rather than at zero free space.
+
+The reserve also has a hardware side: random writes on a half-filled SSD are 
substantially faster than on a nearly full one, because of over-provisioning.
+Choosing devices with a higher over-provisioning rate, and leaving room to use 
it, is part of the same decision.
+See link:persistence/persistence-tuning#ssd-over-provisioning[SSD 
Over-provisioning].
+
+== Related Configuration
+
+* link:setup#configuring-work-directory[Configuring Work Directory]
+* link:persistence/persistence-tuning#keep-wals-separately[Keep WALs 
Separately]
+* 
link:persistence/snapshot-directory#configuring-snapshot-directory[Configuring 
Snapshot Directory]
+* link:persistence/native-persistence[Ignite Persistence]
+* link:persistence/change-data-capture[Change Data Capture]
+* link:snapshots/snapshots[Cluster Snapshots]
+* link:logging[Configuring Logging]
diff --git a/docs/_docs/setup.adoc b/docs/_docs/setup.adoc
index 0c758942c6a..79e234180f9 100644
--- a/docs/_docs/setup.adoc
+++ b/docs/_docs/setup.adoc
@@ -146,6 +146,10 @@ 
include::code-snippets/cpp/src/setting_work_directory.cpp[tag=setting-work-direc
 ----
 --
 
+The work directory contains binary metadata and marshaller mappings, so treat 
it as durable storage even when data and WAL are configured elsewhere.
+Keep the work directory on persistent storage with enough free space.
+See link:server-node-file-system-layout[Server Node File System Layout] for 
the recommended disk placement of the work directory, data, WAL, logs, and 
diagnostic files.
+
 
 
 == Enabling Modules
@@ -345,6 +349,8 @@ If you are going to use either binary distribution or 
Maven, you are
 encouraged to set up the work directory for Ignite.
 The work directory is used to store metadata information, index files, your 
application data (if you use the link:persistence/native-persistence[Native 
Persistence] feature), logs, and other files.
 We recommend you always set up the work directory.
+When data, WAL, and logs are configured to separate locations, keep the work 
directory on persistent storage with enough free space.
+See link:server-node-file-system-layout[Server Node File System Layout].
 
 
 === Recommended Logging Configuration
@@ -354,6 +360,7 @@ Logs play an important role when it comes to 
troubleshooting and finding what we
 * Start Ignite in verbose mode:
    - If you use `ignite.sh`, specify the `-v` option.
    - If you start Ignite from Java code, set the following system variable: 
`IGNITE_QUIET=false`.
-* Do not store log files in the `/tmp` folder. This folder is cleared up every 
time the server is restarted.
-* Make sure that there is enough space available on the storage where the log 
files are stored.
+* Use durable storage for logs that you need after a host reboot. Temporary 
storage is suitable only for logs that can be discarded.
+* Make sure that there is enough space available on the storage where the log 
files are stored, and bound the total retained volume, not only the size of 
individual files.
 * Archive old log files periodically to save on storage space.
+* Keep log growth away from the space that data, WAL, and node metadata need. 
See link:server-node-file-system-layout[Server Node File System Layout].
diff --git a/docs/_docs/snapshots/snapshots.adoc 
b/docs/_docs/snapshots/snapshots.adoc
index 8a18cc5cbdb..78cc0caa84a 100644
--- a/docs/_docs/snapshots/snapshots.adoc
+++ b/docs/_docs/snapshots/snapshots.adoc
@@ -115,6 +115,10 @@ Ignite Persistence routines, it's suggested to store the 
snapshot and persistenc
 See the 
link:persistence/snapshot-directory#configuring-snapshot-directory[Configuring 
Snapshot Directory] page for
 configuration examples.
 
+NOTE: A full snapshot reads all the data and writes a separate stream of 
files, so on a shared device it adds a second full read and write pass on top 
of the traffic the cluster is serving.
+The snapshot root used for incremental snapshots has to be on the same file 
system as the WAL archive, because increments are hard links to archived WAL 
segments.
+See link:server-node-file-system-layout[Server Node File System Layout] for 
capacity and placement recommendations.
+
 === Snapshot Execution Pool
 
 By default, the snapshot thread pool size has a value of `4`. Decreasing the 
number of threads involved in the snapshot creation process 

Reply via email to