nicusX commented on code in PR #28863:
URL: https://github.com/apache/flink/pull/28863#discussion_r3913069494
##########
docs/content/docs/concepts/glossary.md:
##########
@@ -46,137 +48,331 @@ Cluster](#flink-cluster) is bound to the lifetime of the
Flink Application.
#### ApplicationResultStore
The ApplicationResultStore is a Flink component that persists the results of
terminated
-(i.e. finished, cancelled or failed) applications to a filesystem, allowing
the results to outlive
-a terminated application. Each result contains the application's identifier,
final state, name,
-etc. These results are then used by Flink to determine whether applications
should
-be subject to recovery in highly-available clusters.
+(i.e. finished, cancelled or failed) Applications to a filesystem, allowing
the results to outlive
+a terminated Application. Each result contains the Application's identifier,
final state, name,
+etc. These results are then used by Flink to determine whether Applications
should
+be subject to recovery in highly-available Clusters.
+
+#### Channel
+
+Also called *Stream Partitions*.
+
+A Channel is the physical link between a [Sub-Task](#sub-task) and a
downstream Sub-Task, and the
+edge of a [Physical Graph](#physical-graph). Parts of the documentation refer
to Channels as *Stream
+Partitions*, in the sense of internal, physical Partitions.
+
+Channels carry data records as well as signals such as
[Watermarks](#watermark), Watermark Status
+updates and Checkpoint barriers. Transmission over a Channel is always
unidirectional (upstream to
+downstream) and asynchronous.
+
+A Sub-Task may have one or more input Channels and one or more output
Channels. Source Sub-Tasks have
+no input Channels, since they begin the graph, and Sink Sub-Tasks have no
output Channels, since they
+end it.
+
+A Sub-Task routes each record to one of its output Channels according to the
[Physical
+Partitioning](#partition) of the stream. Hash partitioning (`keyBy()` in the
DataStream API, `GROUP
+BY` in SQL) routes a record to the Channel connected to the downstream
Sub-Task that handles the
+record's key, whereas `rebalance()` or `rescale()` may round-robin records
across output Channels.
+
+A Channel is *local* when both Sub-Tasks run in the same [Flink
+TaskManager](#flink-taskmanager), in which case records are handed over
through an in-memory buffer,
+or *remote* when the Sub-Tasks run in different TaskManagers, in which case
the data crosses the
+network.
+
+#### Checkpoint
+
+A consistent snapshot of the State of a [Flink Job](#flink-job) at a logical
point in time, taken
+with a variant of the Chandy-Lamport algorithm and written to [Checkpoint
+Storage](#checkpoint-storage).
+
+A Checkpoint contains the [State](#managed-state) of all stateful
[Operators](#operator). This also
+includes source positions (for example Kafka partition offsets), assignment of
[Source Splits](#source-split)
+to [Sub-Tasks](#sub-task), and Sink transaction metadata. Async I/O in-flight
data and buffered data
+of some asynchronous Sink connectors are also part of the
[Operator](#operator) [State](#managed-state)
+and are saved in the Checkpoint.
+When [Unaligned Checkpoints]({{< ref
"docs/concepts/stateful-stream-processing" >}}#unaligned-checkpointing)
+are enabled, it may also contain data in flight between Sub-Tasks.
+
+Checkpoints are triggered automatically and periodically while the Job is
running, and are used to
+recover from failures such as a TaskManager crash or a network problem: the
Job restarts from the
+latest completed Checkpoint. They are designed for low overhead and run mostly
asynchronously,
+without blocking record processing, apart from a synchronous phase in each
Sub-Task.
+Transactional [Sources and Sinks](#operator) tie their transactions to the
Checkpoint; the
+Kafka Sink, for instance, commits its Kafka transactions when a Checkpoint
completes.
+
+Checkpoints are only used in the `STREAMING` [Execution
Mode](#runtime-execution-mode). In `BATCH`
Review Comment:
The level of detail is always arbitrary.
Mentioning that Checkpoints are only used in streaming mode but not in batch
mode is important. The rest is a brief explanation of how Flink recovers in
batch mode, not using Checkpoints.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
