This is an automated email from the ASF dual-hosted git repository.
yihua pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/hudi.git
The following commit(s) were added to refs/heads/master by this push:
new fcc508c8c0a [MINOR] Fix minor issues in HoodieMetadataTableValidator
docs (#7518)
fcc508c8c0a is described below
commit fcc508c8c0a549a1e14ff1dcb2a66c30c99ad421
Author: Y Ethan Guo <[email protected]>
AuthorDate: Fri Jan 13 15:14:12 2023 -0800
[MINOR] Fix minor issues in HoodieMetadataTableValidator docs (#7518)
---
.../utilities/HoodieMetadataTableValidator.java | 49 +++++++++++-----------
1 file changed, 24 insertions(+), 25 deletions(-)
diff --git
a/hudi-utilities/src/main/java/org/apache/hudi/utilities/HoodieMetadataTableValidator.java
b/hudi-utilities/src/main/java/org/apache/hudi/utilities/HoodieMetadataTableValidator.java
index 0b3c072c92a..77e39af38ff 100644
---
a/hudi-utilities/src/main/java/org/apache/hudi/utilities/HoodieMetadataTableValidator.java
+++
b/hudi-utilities/src/main/java/org/apache/hudi/utilities/HoodieMetadataTableValidator.java
@@ -99,12 +99,12 @@ import static
org.apache.hudi.hadoop.CachingPath.getPathWithoutSchemeAndAuthorit
* between metadata table and filesystem.
* <p>
* There are five validation tasks, that can be enabled independently through
the following CLI options:
- * - `--validate-latest-file-slices`: validate latest file slices for all
partitions.
- * - `--validate-latest-base-files`: validate latest base files for all
partitions.
+ * - `--validate-latest-file-slices`: validate the latest file slices for all
partitions.
+ * - `--validate-latest-base-files`: validate the latest base files for all
partitions.
* - `--validate-all-file-groups`: validate all file groups, and all file
slices within file groups.
* - `--validate-all-column-stats`: validate column stats for all columns in
the schema
* - `--validate-bloom-filters`: validate bloom filters of base files
- *
+ * <p>
* If the Hudi table is on the local file system, the base path passed to
`--base-path` must have
* "file:" prefix to avoid validation failure.
* <p>
@@ -113,37 +113,36 @@ import static
org.apache.hudi.hadoop.CachingPath.getPathWithoutSchemeAndAuthorit
* Example command:
* ```
* spark-submit \
- * --class org.apache.hudi.utilities.HoodieMetadataTableValidator \
- * --master spark://xxxx:7077 \
- * --driver-memory 1g \
- * --executor-memory 1g \
- *
$HUDI_DIR/hudi/packaging/hudi-utilities-bundle/target/hudi-utilities-bundle_2.11-0.11.0-SNAPSHOT.jar
\
- * --base-path basePath \
- * --validate-latest-file-slices \
- * --validate-latest-base-files \
- * --validate-all-file-groups
+ * --class org.apache.hudi.utilities.HoodieMetadataTableValidator \
+ * --master spark://xxxx:7077 \
+ * --driver-memory 1g \
+ * --executor-memory 1g \
+ *
$HUDI_DIR/packaging/hudi-utilities-bundle/target/hudi-utilities-bundle_2.11-0.13.0-SNAPSHOT.jar
\
+ * --base-path basePath \
+ * --validate-latest-file-slices \
+ * --validate-latest-base-files \
+ * --validate-all-file-groups
* ```
*
* <p>
- * Also You can set `--continuous` for long running this validator.
+ * Also, You can set `--continuous` for long running this validator.
* And use `--min-validate-interval-seconds` to control the validation
frequency, default is 10 minutes.
* <p>
* Example command:
* ```
* spark-submit \
- * --class org.apache.hudi.utilities.HoodieMetadataTableValidator \
- * --master spark://xxxx:7077 \
- * --driver-memory 1g \
- * --executor-memory 1g \
- *
$HUDI_DIR/hudi/packaging/hudi-utilities-bundle/target/hudi-utilities-bundle_2.11-0.11.0-SNAPSHOT.jar
\
- * --base-path basePath \
- * --validate-latest-file-slices \
- * --validate-latest-base-files \
- * --validate-all-file-groups \
- * --continuous \
- * --min-validate-interval-seconds 60
+ * --class org.apache.hudi.utilities.HoodieMetadataTableValidator \
+ * --master spark://xxxx:7077 \
+ * --driver-memory 1g \
+ * --executor-memory 1g \
+ *
$HUDI_DIR/packaging/hudi-utilities-bundle/target/hudi-utilities-bundle_2.11-0.13.0-SNAPSHOT.jar
\
+ * --base-path basePath \
+ * --validate-latest-file-slices \
+ * --validate-latest-base-files \
+ * --validate-all-file-groups \
+ * --continuous \
+ * --min-validate-interval-seconds 60
* ```
- *
*/
public class HoodieMetadataTableValidator implements Serializable {
