techdocsmith commented on a change in pull request #11541:
URL: https://github.com/apache/druid/pull/11541#discussion_r683786173
##########
File path: docs/ingestion/index.md
##########
@@ -22,33 +22,24 @@ title: "Ingestion"
~ under the License.
-->
-All data in Druid is organized into _segments_, which are data files each of
which may have up to a few million rows.
-Loading data in Druid is called _ingestion_ or _indexing_, and consists of
reading data from a source system and creating
-segments based on that data.
+Loading data in Druid is called _ingestion_ or _indexing_. When you ingest
data into Druid, Druid reads the data from your source system and stores it in
data files called _segments_. In general, segment files contain a few million
rows.
-In most ingestion methods, the Druid
[MiddleManager](../design/middlemanager.md) processes
-(or the [Indexer](../design/indexer.md) processes) load your source data. One
exception is
-Hadoop-based ingestion, where this work is instead done using a Hadoop
MapReduce job on YARN (although MiddleManager or Indexer
-processes are still involved in starting and monitoring the Hadoop jobs).
+For most ingestion methods, the Druid
[MiddleManager](../design/middlemanager.md) processes or the
[Indexer](../design/indexer.md) processes load your source data. One exception
is
+Hadoop-based ingestion, which uses a Hadoop MapReduce job on YARN
MiddleManager or Indexer processes to start and monitor Hadoop jobs.
-Once segments have been generated and stored in [deep
storage](../dependencies/deep-storage.md), they are loaded by Historical
processes.
-For more details on how this works, see the [Storage
design](../design/architecture.md#storage-design) section
-of Druid's design documentation.
+After Druid creates segments and stores them in [deep
storage](../dependencies/deep-storage.md), Historical processes load them to
respond to queries. See the [Storage
design](../design/architecture.md#storage-design) section of the Druid design
documentation for more information.
-## How to use this documentation
+This topic and the following topics describe ingestion concepts and
information that apply to all [ingestion methods](#ingestion-methods):
+- [Druid data model](./data-model.md) introduces concepts of datasources,
primary timestamp, dimensions, and metrics.
+- [Data rollup](./rollup.md) describes rollup as a concept and provides
suggestions to maximize the benefits of rollup.
+- [Partitioning](./partitioning.md) describes time chunk and secondary
partitioning in Druid.
+- [Ingestion spec reference](./ingestion-spec.md) provides a reference for the
configuration options in the ingestion spec.
-This **page you are currently reading** provides information about universal
Druid ingestion concepts, and about
-configurations that are common to all [ingestion methods](#ingestion-methods).
-
-The **individual pages for each ingestion method** provide additional
information about concepts and configurations
-that are unique to each ingestion method.
-
-We recommend reading (or at least skimming) this universal page first, and
then referring to the page for the
-ingestion method or methods that you have chosen.
+For additional information about concepts and configurations that are unique
to each ingestion method, see the topic for the ingestion method.
## Ingestion methods
-The table below lists Druid's most common data ingestion methods, along with
comparisons to help you choose
+The tables below list Druid's most common data ingestion methods, along with
comparisons to help you choose
Review comment:
There is nowhere in the docs that describes a supervisor 😞 . I added my
own brief definition.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
