This is an automated email from the ASF dual-hosted git repository.
fjy pushed a commit to branch 0.14.0-incubating
in repository https://gitbox.apache.org/repos/asf/incubator-druid.git
The following commit(s) were added to refs/heads/0.14.0-incubating by this push:
new aae0f10 Fix and improve doc for partitioning of local index (#7064)
(#7104)
aae0f10 is described below
commit aae0f104589b429d61cf48e874e2997d820644bb
Author: Jihoon Son <[email protected]>
AuthorDate: Tue Feb 19 21:08:55 2019 -0800
Fix and improve doc for partitioning of local index (#7064) (#7104)
---
docs/content/ingestion/index.md | 4 ++--
docs/content/ingestion/native_tasks.md | 2 +-
2 files changed, 3 insertions(+), 3 deletions(-)
diff --git a/docs/content/ingestion/index.md b/docs/content/ingestion/index.md
index db1edfa..e4b6b70 100644
--- a/docs/content/ingestion/index.md
+++ b/docs/content/ingestion/index.md
@@ -178,7 +178,7 @@ the best one for your situation.
|Method|How it works|Can append and overwrite?|Can handle late
data?|Exactly-once ingestion?|Real-time queries?|
|------|------------|-------------------------|---------------------|-----------------------|------------------|
|[Native batch](native_tasks.html)|Druid loads data directly from S3, HTTP,
NFS, or other networked storage.|Append or overwrite|Yes|Yes|No|
-|[Hadoop](hadoop.html)|Druid launches Hadoop Map/Reduce jobs to load data
files.|Append or overwrite|Yes|Yes|No|
+|[Hadoop](hadoop.html)|Druid launches Hadoop Map/Reduce jobs to load data
files.|Overwrite|Yes|Yes|No|
|[Kafka indexing
service](../development/extensions-core/kafka-ingestion.html)|Druid reads
directly from Kafka.|Append only|Yes|Yes|Yes|
|[Tranquility](stream-push.html)|You use Tranquility, a client side library,
to push individual records into Druid.|Append only|No - late data is dropped|No
- may drop or duplicate data|Yes|
@@ -191,7 +191,7 @@ a _time chunk_, and each time chunk contains one or more
[segments](../design/se
particular time chunk may be partitioned further using options that vary based
on the ingestion method you have chosen.
* With [Hadoop](hadoop.html) you can do hash- or range-based partitioning on
one or more columns.
- * With [Native batch](native_tasks.html) you can partition on a hash of all
dimension columns. This is useful when
+ * With [Native batch](native_tasks.html) you can partition on a hash of
dimension columns. This is useful when
rollup is enabled, since it maximizes your space savings.
* With [Kafka indexing](../development/extensions-core/kafka-ingestion.html),
partitioning is based on Kafka
partitions, and is not configurable through Druid. You can configure it on
the Kafka side by using the partitioning
diff --git a/docs/content/ingestion/native_tasks.md
b/docs/content/ingestion/native_tasks.md
index b9657d1..963adea 100644
--- a/docs/content/ingestion/native_tasks.md
+++ b/docs/content/ingestion/native_tasks.md
@@ -502,7 +502,7 @@ The tuningConfig is optional and default parameters will be
used if no tuningCon
|indexSpec|defines segment storage format options to be used at indexing time,
see [IndexSpec](#indexspec)|null|no|
|maxPendingPersists|Maximum number of persists that can be pending but not
started. If this limit would be exceeded by a new intermediate persist,
ingestion will block until the currently-running persist finishes. Maximum heap
memory usage for indexing scales with maxRowsInMemory * (2 +
maxPendingPersists).|0 (meaning one persist can be running concurrently with
ingestion, and none can be queued up)|no|
|forceExtendableShardSpecs|Forces use of extendable shardSpecs. Experimental
feature intended for use with the [Kafka indexing service
extension](../development/extensions-core/kafka-ingestion.html).|false|no|
-|forceGuaranteedRollup|Forces guaranteeing the [perfect
rollup](../ingestion/index.html#roll-up-modes). The perfect rollup optimizes
the total size of generated segments and querying time while indexing time will
be increased. This flag cannot be used with either `appendToExisting` of
IOConfig or `forceExtendableShardSpecs`. For more details, see the below
__Segment pushing modes__ section.|false|no|
+|forceGuaranteedRollup|Forces guaranteeing the [perfect
rollup](../ingestion/index.html#roll-up-modes). The perfect rollup optimizes
the total size of generated segments and querying time while indexing time will
be increased. If this is set to true, the index task will read the entire input
data twice: one for finding the optimal number of partitions per time chunk and
one for generating segments. Note that the result segments would be
hash-partitioned. You can set `forceExtendableShard [...]
|reportParseExceptions|DEPRECATED. If true, exceptions encountered during
parsing will be thrown and will halt ingestion; if false, unparseable rows and
fields will be skipped. Setting `reportParseExceptions` to true will override
existing configurations for `maxParseExceptions` and `maxSavedParseExceptions`,
setting `maxParseExceptions` to 0 and limiting `maxSavedParseExceptions` to no
more than 1.|false|no|
|pushTimeout|Milliseconds to wait for pushing segments. It must be >= 0, where
0 means to wait forever.|0|no|
|segmentWriteOutMediumFactory|Segment write-out medium to use when creating
segments. See
[SegmentWriteOutMediumFactory](#segmentWriteOutMediumFactory).|Not specified,
the value from `druid.peon.defaultSegmentWriteOutMediumFactory.type` is used|no|
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
