gianm commented on a change in pull request #10149:
URL: https://github.com/apache/druid/pull/10149#discussion_r451154818
##########
File path: docs/design/architecture.md
##########
@@ -213,6 +213,75 @@ this will generally start off `true` and then become
`false` as the segment is p
published segments. Generally this is a transient state, and segments in this
state will soon have their `used` flag
automatically set to false.
+### Availability and consistency
+
+Druid has an architectural separation between ingestion and querying, as
described above in
+[Indexing and handoff](#indexing-and-handoff). This means that when
understanding Druid's availability and
+consistency properties, we must look at each function separately.
+
+On the **ingestion side**, Druid's primary [ingestion
methods](../ingestion/index.md#ingestion-methods) are all
+pull-based and offer transactional guarantees. This means that you are
guaranteed that ingestion using these will
+publish in an all-or-nothing manner:
+
+- Supervised "seekable-stream" ingestion methods like
[Kafka](../development/extensions-core/kafka-ingestion.md) and
+[Kinesis](../development/extensions-core/kinesis-ingestion.md). With these
methods, Druid commits stream offsets to its
+[metadata store](#metadata-storage) alongside segment metadata, in the same
transaction. Note that ingestion of data
+that has not yet been published can be rolled back if ingestion tasks fail. In
this case, partially-ingested data
+discarded, and Druid will resume ingestion from the last committed set of
stream offsets. This ensures exactly-once
Review comment:
Thanks, fixed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
