This is an automated email from the ASF dual-hosted git repository.
vbalaji pushed a commit to branch asf-site
in repository https://gitbox.apache.org/repos/asf/hudi.git
The following commit(s) were added to refs/heads/asf-site by this push:
new ac1a2231cbe [HUDI-5856][DOCS] Update SQL in procedures.md (#8004)
ac1a2231cbe is described below
commit ac1a2231cbe5931ea84f65c11606c2bc52a3d2c4
Author: Kunni <[email protected]>
AuthorDate: Tue Mar 28 09:47:12 2023 +0800
[HUDI-5856][DOCS] Update SQL in procedures.md (#8004)
---
website/docs/procedures.md | 1555 ++++++++++++++++++++++++++++++++++++++++++--
1 file changed, 1497 insertions(+), 58 deletions(-)
diff --git a/website/docs/procedures.md b/website/docs/procedures.md
index eb88fa2ccc7..1144efc8d52 100644
--- a/website/docs/procedures.md
+++ b/website/docs/procedures.md
@@ -24,6 +24,32 @@ CALL system.procedure_name(arg_1, arg_2, ... arg_n)
```
*note:* The system here has no practical meaning, the complete procedure name
is system.procedure_name.
+### Help Procedure
+
+Show parameters and outputTypes of a procedure.
+
+**Input**
+
+| Parameter Name | Type | Required | Default Value | Description |
+|----------------|--------|----------|---------------|---------------------|
+| cmd | String | N | None | name of a procedure |
+
+**Output**
+
+| Output Name | Type |
+|--------------|--------|
+| result | String |
+
+**Example**x
+
+```
+call help(cmd => 'show_commits');
+```
+
+| result
[...]
+|-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
[...]
+| parameters: <br/>param type_name
default_value required <br/>table
string None true
<br/>limit integer 10
false <br/>outputType: <br/>name
type_name nullable metadata
<br/>commit_time [...]
+
## Commit management
### show_commits
@@ -110,118 +136,1100 @@ call show_commits_metadata(table => 'test_hudi_table');
|20220109191648181 | commit | dt=2021-05-02 |
b3b32bac-8a44-4c4d-b433-0cb1bf620f23-0 | null | 1 | 1
| 0 | 0 | 0 | 0 | 0
| 0 | 0 | 0
| 435341 |
|20220109191631015 | commit | dt=2021-05-01 |
0d7298b3-6b55-4cff-8d7d-b0772358b78a-0 | null | 1 | 1
| 0 | 0 | 0 | 0 | 0
| 0 | 0 | 0
| 435341 |
-### rollback_to_instant
+### show_commit_extra_metadata
+
+Show commits' extra metadata.
+
+**Input**
+
+| Parameter Name | Type | Required | Default Value | Description
|
+|----------------|--------|----------|---------------|--------------------------------------|
+| table | String | Y | None | Hudi table name
|
+| limit | Int | N | 10 | Max number of records
to be returned |
+| instant_time | String | N | None | Instant time
|
+| metadata_key | String | N | None | Key of metadata
|
+
+**Output**
+
+| Output Name | Type |
+|----------------|--------|
+| instant_time | String |
+| action | String |
+| metadata_key | String |
+| metadata_value | String |
+
+**Example**
+
+```
+call show_commit_extra_metadata(table => 'test_hudi_table');
+```
+
+| instant_time | action | metadata_key | metadata_value
[...]
+|-------------------|-------------|---------------|---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
[...]
+| 20230206174349556 | deltacommit | schema |
{"type":"record","name":"hudi_mor_tbl","fields":[{"name":"_hoodie_commit_time","type":["null","string"],"doc":"","default":null},{"name":"_hoodie_commit_seqno","type":["null","string"],"doc":"","default":null},{"name":"_hoodie_record_key","type":["null","string"],"doc":"","default":null},{"name":"_hoodie_partition_path","type":["null","string"],"doc":"","default":null},{"name":"_hoodie_file_name","type":["null","string"],"doc":"","defau
[...]
+| 20230206174349556 | deltacommit | latest_schema |
{"max_column_id":8,"version_id":20230206174349556,"type":"record","fields":[{"id":0,"name":"_hoodie_commit_time","optional":true,"type":"string","doc":""},{"id":1,"name":"_hoodie_commit_seqno","optional":true,"type":"string","doc":""},{"id":2,"name":"_hoodie_record_key","optional":true,"type":"string","doc":""},{"id":3,"name":"_hoodie_partition_path","optional":true,"type":"string","doc":""},{"id":4,"name":"_hoodie_file_name","optional"
[...]
+
+### show_archived_commits
+
+Show archived commits.
+
+**Input**
+
+| Parameter Name | Type | Required | Default Value | Description
|
+|----------------|--------|----------|---------------|------------------------------------------------|
+| table | String | Y | None | Hudi table name
|
+| limit | Int | N | 10 | Max number of records
to be returned |
+| start_ts | String | N | "" | Start time for commits,
default: now - 10 days |
+| end_ts | String | N | "" | End time for commits,
default: now - 1 day |
+
+**Output**
+
+| Output Name | Type |
+|------------------------------|--------|
+| commit_time | String |
+| total_bytes_written | Long |
+| total_files_added | Long |
+| total_files_updated | Long |
+| total_partitions_written | Long |
+| total_records_written | Long |
+| total_update_records_written | Long |
+| total_errors | Long |
+
+**Example**
+
+```
+call show_archived_commits(table => 'test_hudi_table');
+```
+
+| commit_time | total_bytes_written | total_files_added |
total_files_updated | total_partitions_written | total_records_written |
total_update_records_written | total_errors |
+|-------------------|--------------------------|-------------------|---------------------|--------------------------|-----------------------|------------------------------|--------------|
+| 20220216171049652 | 432653 | 0 | 1
| 1 | 0 | 0
| 0 |
+| 20220216171027021 | 435346 | 1 | 0
| 1 | 1 | 0
| 0 |
+| 20220216171019361 | 435349 | 1 | 0
| 1 | 1 | 0
| 0 |
+
+
+### show_archived_commits_metadata
+
+Show archived commits' metadata.
+
+**Input**
+
+| Parameter Name | Type | Required | Default Value | Description
|
+|----------------|--------|----------|---------------|------------------------------------------------|
+| table | String | Y | None | Hudi table name
|
+| limit | Int | N | 10 | Max number of records
to be returned |
+| start_ts | String | N | "" | Start time for commits,
default: now - 10 days |
+| end_ts | String | N | "" | End time for commits,
default: now - 1 day |
+
+**Output**
+
+| Output Name | Type |
+|---------------------------------|--------|
+| commit_time | String |
+| action | String |
+| partition | String |
+| file_id | String |
+| previous_commit | String |
+| num_writes | Long |
+| num_inserts | Long |
+| num_deletes | Long |
+| num_update_writes | String |
+| total_errors | Long |
+| total_log_blocks | Long |
+| total_corrupt_logblocks | Long |
+| total_rollback_blocks | Long |
+| total_log_records | Long |
+| total_updated_records_compacted | Long |
+| total_bytes_written | Long |
+
+**Example**
+
+```
+call show_archived_commits_metadata(table => 'test_hudi_table');
+```
+
+| commit_time | action | partition | file_id
| previous_commit | num_writes | num_inserts | num_deletes |
num_update_writes | total_errors | total_log_blocks | total_corrupt_logblocks |
total_rollback_blocks | total_log_records | total_updated_records_compacted |
total_bytes_written|
+|-----------------
|---------|---------------|----------------------------------------|-------------------|------------|-------------|-------------|-------------------|--------------|------------------|-------------------------|-----------------------|-------------------|---------------------------------|-------------------
|
+|20220109225319449 | commit | dt=2021-05-03 |
d0073a12-085d-4f49-83e9-402947e7e90a-0 | null | 1 | 1
| 0 | 0 | 0 | 0 |
0 | 0 | 0 | 0
| 435349 |
+|20220109225311742 | commit | dt=2021-05-02 |
b3b32bac-8a44-4c4d-b433-0cb1bf620f23-0 | 20220109214830592 | 1 | 1
| 0 | 0 | 0 | 0 | 0
| 0 | 0 | 0
| 435340 |
+|20220109225301429 | commit | dt=2021-05-01 |
0d7298b3-6b55-4cff-8d7d-b0772358b78a-0 | 20220109214830592 | 1 | 1
| 0 | 0 | 0 | 0 |
0 | 0 | 0 | 0
| 435340 |
+|20220109214830592 | commit | dt=2021-05-01 |
0d7298b3-6b55-4cff-8d7d-b0772358b78a-0 | 20220109191631015 | 0 | 0
| 1 | 0 | 0 | 0 | 0
| 0 | 0 | 0
| 432653 |
+|20220109214830592 | commit | dt=2021-05-02 |
b3b32bac-8a44-4c4d-b433-0cb1bf620f23-0 | 20220109191648181 | 0 | 0
| 1 | 0 | 0 | 0 |
0 | 0 | 0 | 0
| 432653 |
+|20220109191648181 | commit | dt=2021-05-02 |
b3b32bac-8a44-4c4d-b433-0cb1bf620f23-0 | null | 1 | 1
| 0 | 0 | 0 | 0 | 0
| 0 | 0 | 0
| 435341 |
+|20220109191631015 | commit | dt=2021-05-01 |
0d7298b3-6b55-4cff-8d7d-b0772358b78a-0 | null | 1 | 1
| 0 | 0 | 0 | 0 | 0
| 0 | 0 | 0
| 435341 |
+
+
+```
+call show_archived_commits(table => 'test_hudi_table');
+```
+
+| commit_time | total_bytes_written | total_files_added |
total_files_updated | total_partitions_written | total_records_written |
total_update_records_written | total_errors |
+|-------------------|--------------------------|-------------------|---------------------|--------------------------|-----------------------|------------------------------|--------------|
+| 20220216171049652 | 432653 | 0 | 1
| 1 | 0 | 0
| 0 |
+| 20220216171027021 | 435346 | 1 | 0
| 1 | 1 | 0
| 0 |
+| 20220216171019361 | 435349 | 1 | 0
| 1 | 1 | 0
| 0 |
+
+### show_commit_files
+
+Show files of a commit.
+
+**Input**
+
+| Parameter Name | Type | Required | Default Value | Description
|
+|----------------|--------|----------|---------------|--------------------------------------|
+| table | String | Y | None | Hudi table name
|
+| limit | Int | N | 10 | Max number of records
to be returned |
+| instant_time | String | Y | None | Instant time
|
+
+**Output**
+
+| Output Name | Type |
+|-----------------------|--------|
+| action | String |
+| partition_path | String |
+| file_id | String |
+| previous_commit | String |
+| total_records_updated | Long |
+| total_records_written | Long |
+| total_bytes_written | Long |
+| total_errors | Long |
+| file_size | Long |
+
+**Example**
+
+```
+call show_commit_files(table => 'test_hudi_table', instant_time =>
'20230206174349556');
+```
+
+| action | partition_path | file_id
| previous_commit | total_records_updated | total_records_written |
total_bytes_written | total_errors | file_size |
+|-------------|-----------------|----------------------------------------|-----------------|-----------------------|-----------------------|---------------------|--------------|-----------|
+| deltacommit | dt=2021-05-03 | 7fb52523-c7f6-41aa-84a6-629041477aeb-0 |
null | 0 | 1 | 434768
| 0 | 434768 |
+
+### show_commit_partitions
+
+Show partitions of a commit.
+
+**Input**
+
+| Parameter Name | Type | Required | Default Value | Description
|
+|----------------|--------|----------|---------------|--------------------------------------|
+| table | String | Y | None | Hudi table name
|
+| limit | Int | N | 10 | Max number of records
to be returned |
+| instant_time | String | Y | None | Instant time
|
+
+**Output**
+
+| Output Name | Type |
+|------------------------|--------|
+| action | String |
+| partition_path | String |
+| total_files_added | Long |
+| total_files_updated | Long |
+| total_records_inserted | Long |
+| total_records_updated | Long |
+| total_bytes_written | Long |
+| total_errors | Long |
+
+**Example**
+
+```
+call show_commit_partitions(table => 'test_hudi_table', instant_time =>
'20230206174349556');
+```
+
+| action | partition_path | total_files_added
| total_files_updated | total_records_inserted | total_records_updated |
total_bytes_written | total_errors |
+|-------------|-----------------|----------------------------------------|---------------------|------------------------|-----------------------|---------------------|--------------|
+| deltacommit | dt=2021-05-03 | 7fb52523-c7f6-41aa-84a6-629041477aeb-0 | 0
| 1 | 434768 | 0
| 0 |
+
+### show_commit_write_stats
+
+Show write statistics of a commit.
+
+**Input**
+
+| Parameter Name | Type | Required | Default Value | Description
|
+|----------------|--------|----------|---------------|--------------------------------------|
+| table | String | Y | None | Hudi table name
|
+| limit | Int | N | 10 | Max number of records
to be returned |
+| instant_time | String | Y | None | Instant time
|
+
+**Output**
+
+| Output Name | Type |
+|-----------------------|--------|
+| action | String |
+| total_bytes_written | Long |
+| total_records_written | Long |
+| avg_record_size | Long |
+
+**Example**
+
+```
+call show_commit_write_stats(table => 'test_hudi_table', instant_time =>
'20230206174349556');
+```
+
+| action | total_bytes_written | total_records_written | avg_record_size |
+|-------------|---------------------|-----------------------|-----------------|
+| deltacommit | 434768 | 1 | 434768 |
+
+### show_rollbacks
+
+Show rollback commits.
+
+**Input**
+
+| Parameter Name | Type | Required | Default Value | Description
|
+|----------------|--------|----------|---------------|--------------------------------------|
+| table | String | Y | None | Hudi table name
|
+| limit | Int | N | 10 | Max number of records
to be returned |
+
+**Output**
+
+| Output Name | Type |
+|----------------------|--------|
+| instant | String |
+| rollback_instant | String |
+| total_files_deleted | Int |
+| time_taken_in_millis | Long |
+| total_partitions | Int |
+
+**Example**
+
+```
+call show_rollbacks(table => 'test_hudi_table');
+```
+
+| instant | rollback_instant | total_files_deleted | time_taken_in_millis
| time_taken_in_millis |
+|-------------|------------------|---------------------|----------------------|----------------------|
+| deltacommit | 434768 | 1 | 434768
| 2 |
+
+
+### show_rollback_detail
+
+Show details of a rollback commit.
+
+**Input**
+
+| Parameter Name | Type | Required | Default Value | Description
|
+|----------------|--------|----------|---------------|--------------------------------------|
+| table | String | Y | None | Hudi table name
|
+| limit | Int | N | 10 | Max number of records
to be returned |
+| instant_time | String | Y | None | Instant time
|
+
+**Output**
+
+| Output Name | Type |
+|------------------|--------|
+| instant | String |
+| rollback_instant | String |
+| partition | String |
+| deleted_file | String |
+| succeeded | Int |
+
+**Example**
+
+```
+call show_rollback_detail(table => 'test_hudi_table', instant_time =>
'20230206174349556');
+```
+
+| instant | rollback_instant | partition | deleted_file | succeeded |
+|-------------|------------------|-----------|--------------|-----------|
+| deltacommit | 434768 | 1 | 434768 | 2 |
+
+### commits_compare
+
+Compare commit with another path.
+
+**Input**
+
+| Parameter Name | Type | Required | Default Value | Description |
+|----------------|--------|----------|---------------|-----------------|
+| table | String | Y | None | Hudi table name |
+| path | String | Y | None | Path of table |
+
+**Output**
+
+| Output Name | Type |
+|----------------|--------|
+| compare_detail | String |
+
+**Example**
+
+```
+call commits_compare(table => 'test_hudi_table', path =>
'hdfs://ns1/hive/warehouse/hudi.db/test_hudi_table');
+```
+
+| compare_detail |
+|------------------------------------------------------------------------|
+| Source test_hudi_table is ahead by 0 commits. Commits to catch up - [] |
+
+### archive_commits
+
+archive commits.
+
+**Input**
+
+| Parameter Name | Type | Required | Default Value | Description
|
+|-----------------|---------|----------|---------------|--------------------------------------------------|
+| table | String | N | None | Hudi table name
|
+| path | String | N | None | Path of table
|
+| min_commits | Int | N | 20 | Configuration as
'hoodie.keep.min.commits' |
+| max_commits | Int | N | 30 | Configuration as
'hoodie.keep.max.commits' |
+| retain_commits | Int | N | 10 | Configuration as
'hoodie.commits.archival.batch' |
+| enable_metadata | Boolean | N | false | Enable the internal
metadata table |
+
+**Output**
+
+| Output Name | Type |
+|-------------|------|
+| result | Int |
+
+**Example**
+
+```
+call archive_commits(table => 'test_hudi_table');
+```
+
+| result |
+|--------|
+| 0 |
+
+### export_instants
+
+extract instants to local folder.
+
+**Input**
+
+| Parameter Name | Type | Required | Default Value
| Description |
+|----------------|---------|----------|-----------------------------------------------------|-----------------------------------|
+| table | String | Y | None
| Hudi table name |
+| local_folder | String | Y | None
| Local folder |
+| limit | Int | N | -1
| Number of instants to be exported |
+| actions | String | N |
clean,commit,deltacommit,rollback,savepoint,restore | Commit action
|
+| desc | Boolean | N | false
| Descending order |
+
+**Output**
+
+| Output Name | Type |
+|---------------|--------|
+| export_detail | String |
+
+**Example**
+
+```
+call export_instants(table => 'test_hudi_table', local_folder =>
'/tmp/folder');
+```
+
+| export_detail |
+|:-----------------------------------|
+| Exported 6 Instants to /tmp/folder |
+
+### rollback_to_instant
+
+Rollback a table to the commit that was current at some time.
+
+**Input**
+
+| Parameter Name | Type | Required | Default Value | Description |
+|----------------|--------|----------|---------------|-----------------|
+| table | String | Y | None | Hudi table name |
+
+**Output**
+
+| Output Name | Type |
+|-----------------|---------|
+| rollback_result | Boolean |
+
+**Example**
+
+Roll back test_hudi_table to one instant
+```
+call rollback_to_instant(table => 'test_hudi_table', instant_time =>
'20220109225319449');
+```
+
+| rollback_result |
+|:----------------|
+| true |
+
+### create_savepoint
+
+Create a savepoint to hudi's table.
+
+**Input**
+
+| Parameter Name | Type | Required | Default Value | Description |
+|----------------|--------|----------|---------------|-----------------|
+| table | String | Y | None | Hudi table name |
+| commit_time | String | Y | None | Commit time |
+| user | String | N | "" | User name |
+| comments | String | N | "" | Comments |
+
+**Output**
+
+| Output Name | Type |
+|-------------------------|---------|
+| create_savepoint_result | Boolean |
+
+**Example**
+
+Roll back test_hudi_table to one instant
+```
+call create_savepoint(table => 'test_hudi_table', commit_time =>
'20220109225319449');
+```
+
+| create_savepoint_result |
+|:------------------------|
+| true |
+
+### show_savepoints
+
+Show savepoints.
+
+**Input**
+
+| Parameter Name | Type | Required | Default Value | Description |
+|----------------|--------|----------|---------------|-----------------|
+| table | String | Y | None | Hudi table name |
+
+**Output**
+
+| Output Name | Type |
+|----------------|--------|
+| savepoint_time | String |
+
+**Example**
+
+```
+call show_savepoints(table => 'test_hudi_table');
+```
+
+| savepoint_time |
+|:------------------|
+| 20220109225319449 |
+| 20220109225311742 |
+| 20220109225301429 |
+
+### delete_savepoint
+
+Delete a savepoint to hudi's table.
+
+**Input**
+
+| Parameter Name | Type | Required | Default Value | Description |
+|----------------|--------|----------|---------------|-----------------|
+| table | String | Y | None | Hudi table name |
+| instant_time | String | Y | None | Instant time |
+
+**Output**
+
+| Output Name | Type |
+|-------------------------|---------|
+| delete_savepoint_result | Boolean |
+
+**Example**
+
+Delete a savepoint to test_hudi_table
+```
+call delete_savepoint(table => 'test_hudi_table', instant_time =>
'20220109225319449');
+```
+
+| delete_savepoint_result |
+|:------------------------|
+| true |
+
+### rollback_to_savepoint
+
+Rollback a table to the commit that was current at some time.
+
+**Input**
+
+| Parameter Name | Type | Required | Default Value | Description |
+|----------------|--------|----------|---------------|-----------------|
+| table | String | Y | None | Hudi table name |
+| instant_time | String | Y | None | Instant time |
+
+**Output**
+
+| Output Name | Type |
+|---------------------------|---------|
+| rollback_savepoint_result | Boolean |
+
+**Example**
+
+Rollback test_hudi_table to one savepoint
+```
+call rollback_to_savepoint(table => 'test_hudi_table', instant_time =>
'20220109225319449');
+```
+
+| rollback_savepoint_result |
+|:--------------------------|
+| true |
+
+### copy_to_temp_view
+
+copy table to a temporary view.
+
+**Input**
+
+| Parameter Name | Type | Required | Default Value | Description
|
+|---------------------|---------|----------|---------------|-------------------------------------------------|
+| table | String | Y | None | Hudi table name
|
+| query_type | String | N | "snapshot" | Configuration as
'hoodie.datasource.query.type' |
+| view_name | String | Y | None | Name of view
|
+| begin_instance_time | String | N | "" | Begin instance
time |
+| end_instance_time | String | N | "" | End instance time
|
+| as_of_instant | String | N | "" | As of instant
time |
+| replace | Boolean | N | false | Replace an
existed view |
+| global | Boolean | N | false | Global view
|
+
+**Output**
+
+| Output Name | Type |
+|-------------|---------|
+| status | Boolean |
+
+**Example**
+
+```
+call copy_to_temp_view(table => 'test_hudi_table', view_name =>
'copy_view_test_hudi_table');
+```
+
+| status |
+|--------|
+| 0 |
+
+### copy_to_table
+
+copy table to a new table.
+
+**Input**
+
+| Parameter Name | Type | Required | Default Value | Description
|
+|---------------------|--------|----------|---------------|-------------------------------------------------|
+| table | String | Y | None | Hudi table name
|
+| query_type | String | N | "snapshot" | Configuration as
'hoodie.datasource.query.type' |
+| new_table | String | Y | None | Name of new table
|
+| begin_instance_time | String | N | "" | Begin instance
time |
+| end_instance_time | String | N | "" | End instance time
|
+| as_of_instant | String | N | "" | As of instant time
|
+| save_mode | String | N | "overwrite" | Save mode
|
+
+**Output**
+
+| Output Name | Type |
+|-------------|---------|
+| status | Boolean |
+
+**Example**
+
+```
+call copy_to_table(table => 'test_hudi_table', new_table =>
'copy_table_test_hudi_table');
+```
+
+| status |
+|--------|
+| 0 |
+
+## Metadata Table management
+
+### create_metadata_table
+
+Create metadata table of a hudi table.
+
+**Input**
+
+| Parameter Name | Type | Required | Default Value | Description |
+|----------------|--------|----------|---------------|-----------------|
+| table | String | Y | None | Hudi table name |
+
+**Output**
+
+| Output Name | Type |
+|-------------|--------|
+| result | String |
+
+**Example**
+
+```
+call create_metadata_table(table => 'test_hudi_table');
+```
+
+| result
|
+|:------------------------------------------------------------------------------------------------------------------|
+| Created Metadata Table in
hdfs://ns1/hive/warehouse/hudi.db/test_hudi_table/.hoodie/metadata
(duration=2.777secs) |
+
+### init_metadata_table
+
+Init metadata table of a hudi table.
+
+**Input**
+
+| Parameter Name | Type | Required | Default Value | Description |
+|----------------|---------|----------|---------------|-----------------|
+| table | String | Y | None | Hudi table name |
+| read_only | Boolean | N | false | Read only |
+
+**Output**
+
+| Output Name | Type |
+|-------------|---------|
+| result | String |
+
+**Example**
+
+```
+call init_metadata_table(table => 'test_hudi_table');
+```
+
+| result
|
+|:---------------------------------------------------------------------------------------------------------------------|
+| Initialized Metadata Table in
hdfs://ns1/hive/warehouse/hudi.db/test_hudi_table/.hoodie/metadata
(duration=0.023sec) |
+
+### delete_metadata_table
+
+Delete metadata table of a hudi table.
+
+**Input**
+
+| Parameter Name | Type | Required | Default Value | Description |
+|----------------|---------|----------|---------------|-----------------|
+| table | String | Y | None | Hudi table name |
+
+**Output**
+
+| Output Name | Type |
+|-------------|---------|
+| result | String |
+
+**Example**
+
+```
+call delete_metadata_table(table => 'test_hudi_table');
+```
+
+| result
|
+|:-----------------------------------------------------------------------------------------------|
+| Removed Metadata Table from
hdfs://ns1/hive/warehouse/hudi.db/test_hudi_table/.hoodie/metadata |
+
+### show_metadata_table_partitions
+
+Show partition of a hudi table.
+
+**Input**
+
+| Parameter Name | Type | Required | Default Value | Description |
+|----------------|--------|----------|---------------|-----------------|
+| table | String | Y | None | Hudi table name |
+
+**Output**
+
+| Output Name | Type |
+|-------------|---------|
+| partition | String |
+
+**Example**
+
+```
+call show_metadata_table_partitions(table => 'test_hudi_table');
+```
+
+| partition |
+|:--------------|
+| dt=2021-05-01 |
+| dt=2021-05-02 |
+| dt=2021-05-03 |
+
+### show_metadata_table_files
+
+Show files of a hudi table.
+
+**Input**
+
+| Parameter Name | Type | Required | Default Value | Description |
+|----------------|--------|----------|---------------|-----------------|
+| table | String | Y | None | Hudi table name |
+| partition | String | N | "" | Partition name |
+
+**Output**
+
+| Output Name | Type |
+|-------------|---------|
+| file_path | String |
+
+**Example**
+
+Show files of a hudi table under one partition.
+```
+call show_metadata_table_files(table => 'test_hudi_table', partition =>
'dt=20230220');
+```
+
+| file_path |
+|:--------------------------------------------------------------------------|
+| .d3cdf6ff-250a-4cee-9af4-ab179fdb9bfb-0_20230220190948086.log.1_0-111-123 |
+| d3cdf6ff-250a-4cee-9af4-ab179fdb9bfb-0_0-78-81_20230220190948086.parquet |
+
+### show_metadata_table_stats
+
+Show metadata table stats of a hudi table.
+
+**Input**
+
+| Parameter Name | Type | Required | Default Value | Description |
+|----------------|--------|----------|---------------|-----------------|
+| table | String | Y | None | Hudi table name |
+
+**Output**
+
+| Output Name | Type |
+|-------------|--------|
+| stat_key | String |
+| stat_value | String |
+
+**Example**
+
+```
+call show_metadata_table_stats(table => 'test_hudi_table');
+```
+
+| stat_key | stat_value |
+|----------------------------------------|------------|
+| dt=2021-05-03.totalBaseFileSizeInBytes | 23142 |
+
+### validate_metadata_table_files
+
+Validate metadata table files of a hudi table.
+
+**Input**
+
+| Parameter Name | Type | Required | Default Value | Description
|
+|----------------|---------|----------|---------------|----------------------------|
+| table | String | Y | None | Hudi table name
|
+| verbose | Boolean | N | False | If verbose print all
files |
+
+
+**Output**
+
+| Output Name | Type |
+|------------------------|---------|
+| partition | String |
+| file_name | String |
+| is_present_in_fs | Boolean |
+| is_present_in_metadata | Boolean |
+| fs_size | Long |
+| metadata_size | Long |
+
+**Example**
+
+```
+call validate_metadata_table_files(table => 'test_hudi_table');
+```
+
+| partition | file_name
| is_present_in_fs | is_present_in_metadata | fs_size | metadata_size |
+|---------------|---------------------------------------------------------------------|------------------|------------------------|---------|---------------|
+| dt=2021-05-03 |
ad1e5a3f-532f-4a13-9f60-223676798bf3-0_0-4-4_00000000000002.parquet | true
| true | 43523 | 43523 |
+
+## Table information
+
+### show_table_properties
+
+Show hudi properties of a table.
+
+**Input**
+
+| Parameter Name | Type | Required | Default Value | Description
|
+|----------------|--------|----------|---------------|--------------------------------------|
+| table | String | Y | None | Hudi table name
|
+| path | String | N | None | Path of table
|
+| limit | Int | N | 10 | Max number of records
to be returned |
+
+**Output**
+
+| Output Name | Type |
+|-------------|--------|
+| key | String |
+| value | String |
+
+**Example**
+
+```
+call show_table_properties(table => 'test_hudi_table', limit => 10);
+```
+
+| key | value |
+|-------------------------------|-------|
+| hoodie.table.precombine.field | ts |
+| hoodie.table.partition.fields | dt |
+
+### show_fs_path_detail
+
+Show detail of a path.
+
+**Input**
+
+| Parameter Name | Type | Required | Default Value | Description
|
+|----------------|---------|----------|---------------|------------------------|
+| path | String | Y | None | Hudi table name
|
+| is_sub | Boolean | N | false | Whether to list files
|
+| sort | Boolean | N | true | Sorted by storage_size
|
+
+**Output**
+
+| Output Name | Type |
+|--------------------|--------|
+| path_num | Long |
+| file_num | Long |
+| storage_size | Long |
+| storage_size(unit) | String |
+| storage_path | String |
+| space_consumed | Long |
+| quota | Long |
+| space_quota | Long |
+
+**Example**
+
+```
+call show_fs_path_detail(path =>
'hdfs://ns1/hive/warehouse/hudi.db/test_hudi_table');
+```
+
+| path_num | file_num | storage_size | storage_size(unit) | storage_path
| space_consumed | quota | space_quota |
+|----------|----------|--------------|--------------------|---------------------------------------------------|----------------|---------|-------------|
+| 22 | 58 | 2065612 | 1.97MB |
hdfs://ns1/hive/warehouse/hudi.db/test_hudi_table | -1 | 6196836 |
-1 |
+
+### stats_file_sizes
+
+Show file sizes of a table.
+
+**Input**
+
+| Parameter Name | Type | Required | Default Value | Description
|
+|----------------|--------|----------|---------------|--------------------------------------|
+| table | String | Y | None | Hudi table name
|
+| partition_path | String | N | "" | Partition path
|
+| limit | Int | N | 10 | Max number of records
to be returned |
+
+**Output**
+
+| Output Name | Type |
+|-------------|--------|
+| commit_time | String |
+| min | Long |
+| 10th | Double |
+| 50th | Double |
+| avg | Double |
+| 95th | Double |
+| max | Long |
+| num_files | Int |
+| std_dev | Double |
+
+**Example**
+
+```
+call stats_file_sizes(table => 'test_hudi_table');
+```
+
+| commit_time | min | 10th | 50th | avg | 95th | max
| num_files | std_dev |
+|-------------------|--------|----------|----------|----------|----------|--------|-----------|---------|
+| 20230205134149455 | 435000 | 435000.0 | 435000.0 | 435000.0 | 435000.0 |
435000 | 1 | 0.0 |
+
+### stats_wa
+
+Show write stats and amplification of a table.
+
+**Input**
+
+| Parameter Name | Type | Required | Default Value | Description
|
+|----------------|--------|----------|---------------|--------------------------------------|
+| table | String | Y | None | Hudi table name
|
+| limit | Int | N | 10 | Max number of records
to be returned |
+
+**Output**
+
+| Output Name | Type |
+|----------------------------|--------|
+| commit_time | String |
+| total_upserted | Long |
+| total_written | Long |
+| write_amplification_factor | String |
+
+**Example**
+
+```
+call stats_wa(table => 'test_hudi_table');
+```
+
+| commit_time | total_upserted | total_written | write_amplification_factor |
+|-------------|----------------|---------------|----------------------------|
+| Total | 0 | 0 | 0 |
+
+### show_logfile_records
+
+Show records in logfile of a table.
+
+**Input**
+
+| Parameter Name | Type | Required | Default Value | Description
|
+|-----------------------|---------|----------|---------------|--------------------------------------|
+| table | String | Y | None | Hudi table name
|
+| log_file_path_pattern | String | Y | 10 | Pattern of
logfile |
+| merge | Boolean | N | false | Merge results
|
+| limit | Int | N | 10 | Max number of
records to be returned |
+
+**Output**
+
+| Output Name | Type |
+|-------------|--------|
+| records | String |
+
+**Example**
+
+```
+call show_logfile_records(table => 'test_hudi_table', log_file_path_pattern =>
'hdfs://ns1/hive/warehouse/hudi.db/test_hudi_table/*.log*');
+```
-Rollback a table to the commit that was current at some time.
+| records
|
+|--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
+| {"_hoodie_commit_time": "20230205133427059", "_hoodie_commit_seqno":
"20230205133427059_0_10", "_hoodie_record_key": "1", "_hoodie_partition_path":
"", "_hoodie_file_name": "3438e233-7b50-4eff-adbb-70b1cd76f518-0", "id": 1,
"name": "a1", "price": 40.0, "ts": 1111} |
+
+### show_logfile_metadata
+
+Show metadatas in logfile of a table.
**Input**
-| Parameter Name | Type | Required | Default Value | Description |
-|----------------|--------|----------|---------------|-----------------|
-| table | String | Y | None | Hudi table name |
+| Parameter Name | Type | Required | Default Value | Description
|
+|-----------------------|---------|----------|---------------|--------------------------------------|
+| table | String | Y | None | Hudi table name
|
+| log_file_path_pattern | String | Y | 10 | Pattern of
logfile |
+| merge | Boolean | N | false | Merge results
|
+| limit | Int | N | 10 | Max number of
records to be returned |
**Output**
-| Output Name | Type |
-|-----------------|---------|
-| rollback_result | Boolean |
+| Output Name | Type |
+|-----------------|--------|
+| instant_time | String |
+| record_count | Int |
+| block_type | String |
+| header_metadata | String |
+| footer_metadata | String |
**Example**
-Roll back test_hudi_table to one instant
```
-call rollback_to_instant(table => 'test_hudi_table', instant_time =>
'20220109225319449');
+call show_logfile_metadata(table => 'hudi_mor_tbl', log_file_path_pattern =>
'hdfs://ns1/hive/warehouse/hudi.db/hudi_mor_tbl/*.log*');
```
-| rollback_result |
-| :---------------|
-| true |
+| instant_time | record_count | block_type | header_metadata
[...]
+|-------------------|--------------|-----------------|------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
[...]
+| 20230205133427059 | 1 | AVRO_DATA_BLOCK |
{"INSTANT_TIME":"20230205133427059","SCHEMA":"{\"type\":\"record\",\"name\":\"hudi_mor_tbl_record\",\"namespace\":\"hoodie.hudi_mor_tbl\",\"fields\":[{\"name\":\"_hoodie_commit_time\",\"type\":[\"null\",\"string\"],\"doc\":\"\",\"default\":null},{\"name\":\"_hoodie_commit_seqno\",\"type\":[\"null\",\"string\"],\"doc\":\"\",\"default\":null},{\"name\":\"_hoodie_record_key\",\"type\":[\"null\",\"string\"],\"doc\":\"\",\"default\":null}
[...]
-### create_savepoints
+### show_invalid_parquet
-Create a savepoint to hudi's table.
+Show invalid parquet files of a table.
**Input**
-| Parameter Name | Type | Required | Default Value | Description |
-|----------------|--------|----------|---------------|-----------------|
-| table | String | Y | None | Hudi table name |
-| commit_Time | String | Y | None | Commit time |
-| user | String | N | "" | User name |
-| comments | String | N | "" | Comments |
+| Parameter Name | Type | Required | Default Value | Description
|
+|----------------|---------|----------|---------------|--------------------------------------|
+| Path | String | Y | None | Hudi table name
|
**Output**
-| Output Name | Type |
-|-------------------------|---------|
-| create_savepoint_result | Boolean |
+| Output Name | Type |
+|-------------|--------|
+| Path | String |
**Example**
-Roll back test_hudi_table to one instant
```
-call create_savepoints(table => 'test_hudi_table', instant_time =>
'20220109225319449');
+call show_invalid_parquet(path =>
'hdfs://ns1/hive/warehouse/hudi.db/test_hudi_table');
```
-| create_savepoint_result |
-|:------------------------|
-| true |
+| Path
|
+|----------------------------------------------------------------------------------------------------------------------------|
+|
hdfs://ns1/hive/warehouse/hudi.db/test_hudi_table/7fb52523-c7f6-41aa-84a6-629041477aeb-0_0-92-99_20230205133532199.parquet
|
-### delete_savepoints
+### show_fsview_all
-Delete a savepoint to hudi's table.
+Show file system views of a table.
**Input**
-| Parameter Name | Type | Required | Default Value | Description |
-|----------------|--------|----------|---------------|-----------------|
-| table | String | Y | None | Hudi table name |
-| instant_time | String | Y | None | Instant time |
+| Parameter Name | Type | Required | Default Value | Description
|
+|--------------------|---------|----------|---------------|--------------------------------------|
+| table | String | Y | None | Hudi table name
|
+| max_instant | String | N | "" | Max instant time
|
+| include_max | Boolean | N | false | Include max
instant |
+| include_in_flight | Boolean | N | false | Include in flight
|
+| exclude_compaction | Boolean | N | false | Exclude compaction
|
+| limit | Int | N | 10 | Max number of
records to be returned |
+| path_regex | String | N | "*/*/*" | Pattern of path
|
**Output**
-| Output Name | Type |
-|-------------------------|---------|
-| delete_savepoint_result | Boolean |
+| Output Name | Type |
+|-----------------------|--------|
+| partition | String |
+| file_id | String |
+| base_instant | String |
+| data_file | String |
+| data_file_size | Long |
+| num_delta_files | Long |
+| total_delta_file_size | Long |
+| delta_files | String |
**Example**
-Delete a savepoint to test_hudi_table
```
-call delete_savepoints(table => 'test_hudi_table', instant_time =>
'20220109225319449');
+call show_fsview_all(table => 'test_hudi_table');
```
-| delete_savepoint_result |
-|:------------------------|
-| true |
+| partition | file_id | base_instant |
data_file |
data_file_size | num_delta_files | total_delta_file_size | delta_files
|
+|---------------|----------------------------------------|-------------------|--------------------------------------------------------------------------|----------------|-----------------|-----------------------|-------------------------------------------------------------------------|
+| dt=2021-05-03 | d0073a12-085d-4f49-83e9-402947e7e90a-0 | 20220109225319449 |
7fb52523-c7f6-41aa-84a6-629041477aeb-0_0-92-99_20220109225319449.parquet |
5319449 | 1 | 213193 |
.7fb52523-c7f6-41aa-84a6-629041477aeb-0_20230205133217210.log.1_0-60-63 |
-### rollback_savepoints
+### show_fsview_latest
-Rollback a table to the commit that was current at some time.
+Show latest file system view of a table.
**Input**
-| Parameter Name | Type | Required | Default Value | Description |
-|----------------|--------|----------|---------------|-----------------|
-| table | String | Y | None | Hudi table name |
-| instant_time | String | Y | None | Instant time |
+| Parameter Name | Type | Required | Default Value | Description
|
+|--------------------|---------|----------|---------------|--------------------------------------|
+| table | String | Y | None | Hudi table name
|
+| max_instant | String | N | "" | Max instant time
|
+| include_max | Boolean | N | false | Include max
instant |
+| include_in_flight | Boolean | N | false | Include in flight
|
+| exclude_compaction | Boolean | N | false | Exclude compaction
|
+| partition_path | String | Y | "" | Partition path
|
+| merge | Boolean | N | false | Merge results
|
**Output**
-| Output Name | Type |
-|---------------------------|---------|
-| rollback_savepoint_result | Boolean |
+| Output Name | Type |
+|--------------------------------------------|--------|
+| partition | String |
+| file_id | String |
+| base_instant | String |
+| data_file | String |
+| data_file_size | Long |
+| num_delta_files | Long |
+| total_delta_file_size | Long |
+| delta_size_compaction_scheduled | Long |
+| delta_size_compaction_unscheduled | Long |
+| delta_to_base_radio_compaction_scheduled | Double |
+| delta_to_base_radio_compaction_unscheduled | Double |
+| delta_files_compaction_scheduled | String |
+| delta_files_compaction_unscheduled | String |
**Example**
-Rollback test_hudi_table to one savepoint
```
-call rollback_savepoints(table => 'test_hudi_table', instant_time =>
'20220109225319449');
+call show_fsview_latest(table => 'test_hudi_table');
```
-| rollback_savepoint_result |
-|:--------------------------|
-| true |
+| partition | file_id | base_instant |
data_file |
data_file_size | num_delta_files | total_delta_file_size | delta_files
|
+|---------------|----------------------------------------|-------------------|--------------------------------------------------------------------------|----------------|-----------------|-----------------------|-------------------------------------------------------------------------|
+| dt=2021-05-03 | d0073a12-085d-4f49-83e9-402947e7e90a-0 | 20220109225319449 |
7fb52523-c7f6-41aa-84a6-629041477aeb-0_0-92-99_20220109225319449.parquet |
5319449 | 1 | 213193 |
.7fb52523-c7f6-41aa-84a6-629041477aeb-0_20230205133217210.log.1_0-60-63 |
## Optimization table
@@ -449,4 +1457,435 @@ call show_compaction(table => 'test_hudi_table', limit
=> 1)
```
| timestamp | action | size |
|-------------------|------------|---------|
-| 20220408153707928 | compaction | 10 |
\ No newline at end of file
+| 20220408153707928 | compaction | 10 |
+
+### delete_marker
+
+Delete marker files of a hudi table.
+
+**Input**
+
+| Parameter Name | Type | Required | Default Value | Description |
+|----------------|--------|----------|---------------|-----------------|
+| table | String | Y | None | Hudi table name |
+| instant_time | String | Y | None | Instant name |
+
+**Output**
+
+| Output Name | Type |
+|----------------------|---------|
+| delete_marker_result | Boolean |
+
+**Example**
+
+```
+call delete_marker(table => 'test_hudi_table', instant_time =>
'20230206174349556');
+```
+
+| delete_marker_result |
+|:---------------------|
+| true |
+
+### sync_validate
+
+Validate sync procedure.
+
+**Input**
+
+| Parameter Name | Type | Required | Default Value | Description |
+|-----------------|--------|----------|---------------|-------------------|
+| src_table | String | Y | None | Source table name |
+| dst_table | String | Y | None | Target table name |
+| mode | String | Y | "complete" | Mode |
+| hive_server_url | String | Y | None | Hive server url |
+| hive_pass | String | Y | None | Hive password |
+| src_db | String | N | "rawdata" | Source database |
+| target_db | String | N | dwh_hoodie" | Target database |
+| partition_cnt | Int | N | 5 | Partition count |
+| hive_user | String | N | "" | Hive user name |
+
+**Output**
+
+| Output Name | Type |
+|-------------|--------|
+| result | String |
+
+**Example**
+
+```
+ call sync_validate(hive_server_url=>'jdbc:hive2://localhost:10000/default',
src_table => 'test_hudi_table_src', dst_table=> 'test_hudi_table_dst',
mode=>'complete', hive_pass=>'', src_db=> 'default', target_db=>'default');
+```
+
+### hive_sync
+
+Sync the table's latest schema to Hive metastore.
+
+**Input**
+
+| Parameter Name | Type | Required | Default Value | Description
|
+|---------------------------|--------|----------|---------------|--------------------------------------------------------------------------|
+| table | String | Y | None | Hudi table
name |
+| metastore_uri | String | N | "" |
Metastore_uri |
+| username | String | N | "" | User name
|
+| password | String | N | "" | Password
|
+| use_jdbc | String | N | "" | Configration
as 'hoodie.datasource.hive_sync.use_jdbc' |
+| mode | String | N | "" |
Configuration as 'hoodie.datasource.hive_sync.mode' |
+| partition_fields | String | N | "" |
Configuration as 'hoodie.datasource.hive_sync.partition_fields' |
|
+| partition_extractor_class | String | N | "" |
Configuration as 'hoodie.datasource.hive_sync.partition_extractor_class' |
+| strategy | String | N | "" |
Configuration as 'hoodie.datasource.hive_sync.table.strategy' |
+
+**Output**
+
+| Output Name | Type |
+|-------------|--------|
+| result | String |
+
+**Example**
+
+```
+call hive_sync(table => 'test_hudi_table');
+```
+
+| result |
+|:-------|
+| true |
+
+### hdfs_parquet_import
+
+add parquet files to a hudi table.
+
+**Input**
+
+| Parameter Name | Type | Required | Default Value | Description
|
+|------------------|--------|----------|---------------|--------------------------------------------|
+| table | String | Y | None | Hudi table name
|
+| table_type | String | Y | "" | Table type,
MERGE_ON_READ or COPY_ON_WRITE |
+| src_path | String | Y | "" | Source path
|
+| target_path | String | Y | "" | target path
|
+| row_key | String | Y | "" | Primary key
|
+| partition_key | String | Y | "" | Partition key
|
+| schema_file_path | String | Y | "" | Path of Schema file
|
+| format | String | N | "parquet" | File format
|
+| command | String | N | "insert" | Import command
|
+| retry | Int | N | 0 | Retry times
|
+| parallelism | Int | N | None | Parallelism
|
+| props_file_path | String | N | "" | Path of properties
file |
+
+**Output**
+
+| Output Name | Type |
+|---------------|------|
+| import_result | Int |
+
+**Example**
+
+```
+call hdfs_parquet_import(table => 'test_hudi_table', table_type =>
'COPY_ON_WRITE', src_path => '', target_path => '', row_key => 'id',
partition_key => 'dt', schema_file_path => '');
+```
+
+| import_result |
+|:--------------|
+| 0 |
+
+
+### repair_add_partition_meta
+
+Repair add partition for a hudi table.
+
+**Input**
+
+| Parameter Name | Type | Required | Default Value | Description |
+|----------------|---------|----------|---------------|-----------------|
+| table | String | Y | None | Hudi table name |
+| dry_run | Boolean | N | true | Dry run |
+
+**Output**
+
+| Output Name | Type |
+|---------------------|--------|
+| partition_path | String |
+| metadata_is_present | String |
+| action | String |
+
+**Example**
+
+```
+call repair_add_partition_meta(table => 'test_hudi_table');
+```
+
+| partition_path | metadata_is_present | action |
+|----------------|---------------------|--------|
+| dt=2021-05-03 | Yes | None |
+
+### repair_corrupted_clean_files
+
+Repair corrupted clean files for a hudi table.
+
+**Input**
+
+| Parameter Name | Type | Required | Default Value | Description
|
+|---------------------|---------|----------|---------------|--------------------|
+| table | String | Y | None | Hudi table name
|
+
+**Output**
+
+| Output Name | Type |
+|-------------|---------|
+| result | Boolean |
+
+**Example**
+
+```
+call repair_corrupted_clean_files(table => 'test_hudi_table');
+```
+
+| result |
+|--------|
+| true |
+
+### repair_deduplicate
+
+Repair deduplicate records for a hudi table.
+
+**Input**
+
+| Parameter Name | Type | Required | Default Value | Description
|
+|---------------------------|---------|----------|---------------|---------------------------|
+| table | String | Y | None | Hudi table
name |
+| duplicated_partition_path | String | Y | None | Duplicated
partition path |
+| repaired_output_path | String | Y | None | Repaired
output path |
+| dry_run | Boolean | N | true | Dry run
|
+| dedupe_type | String | N | "insert_type" | Dedupe type
|
+
+**Output**
+
+| Output Name | Type |
+|-------------|--------|
+| result | String |
+
+**Example**
+
+```
+call repair_deduplicate(table => 'test_hudi_table', duplicated_partition_path
=> 'dt=2021-05-03', repaired_output_path => 'dt=2021-05-04');
+```
+
+| result |
+|----------------------------------------------|
+| Reduplicated files placed in: dt=2021-05-04. |
+
+### repair_migrate_partition_meta
+
+downgrade a hudi table.
+
+**Input**
+
+| Parameter Name | Type | Required | Default Value | Description |
+|----------------|---------|----------|---------------|-----------------|
+| table | String | Y | None | Hudi table name |
+| dry_run | Boolean | N | true | Dry run |
+
+**Output**
+
+| Output Name | Type |
+|-----------------------|--------|
+| partition_path | String |
+| text_metafile_present | String |
+| base_metafile_present | String |
+| action | String |
+
+**Example**
+
+```
+call repair_migrate_partition_meta(table => 'test_hudi_table');
+```
+
+### repair_overwrite_hoodie_props
+
+overwrite a hudi table properties.
+
+**Input**
+
+| Parameter Name | Type | Required | Default Value | Description
|
+|---------------------|--------|----------|---------------|------------------------|
+| table | String | Y | None | Hudi table name
|
+| new_props_file_path | String | Y | None | Path of new
properties |
+
+**Output**
+
+| Output Name | Type |
+|-------------|--------|
+| property | String |
+| old_value | String |
+| new_value | String |
+
+**Example**
+
+```
+call repair_overwrite_hoodie_props(table => 'test_hudi_table',
new_props_file_path = > '/tmp/props');
+```
+
+| property | old_value | new_value |
+|--------------------------|-----------|-----------|
+| hoodie.file.index.enable | true | false |
+
+## Bootstrap
+
+### run_bootstrap
+
+Convert an existing table to Hudi.
+
+**Input**
+
+| Parameter Name | Type | Required | Default Value
| Description
|
+|-------------------------------|---------|----------|-------------------------------------------------------------------------------|--------------------------------------------|
+| table | String | Y | None
| Name of table to be
clustered |
+| table_type | String | Y | None
| Table type,
MERGE_ON_READ or COPY_ON_WRITE |
+| bootstrap_path | String | Y | None
| Bootstrap path
|
+| base_path | String | Y | None
| Base path
|
+| rowKey_field | String | Y | None
| Primary key field
|
+| base_file_format | String | N | "PARQUET"
| Format of base file
|
+| partition_path_field | String | N | ""
| Partitioned column
field |
+| bootstrap_index_class | String | N |
"org.apache.hudi.common.bootstrap.index.HFileBootstrapIndex" |
Class of bootstrap index |
+| selector_class | String | N |
"org.apache.hudi.client.bootstrap.selector.MetadataOnlyBootstrapModeSelector" |
Class of selector |
+| key_generator_class | String | N |
"org.apache.hudi.keygen.SimpleKeyGenerator" |
Class of key generator |
+| full_bootstrap_input_provider | String | N |
"org.apache.hudi.bootstrap.SparkParquetBootstrapDataProvider" |
Class of full bootstrap input provider |
+| schema_provider_class | String | N | ""
| Class of schema
provider |
+| payload_class | String | N |
"org.apache.hudi.common.model.OverwriteWithLatestAvroPayload" |
Class of payload |
+| parallelism | Int | N | 1500
| Parallelism
|
+| enable_hive_sync | Boolean | N | false
| Whether to enable hive
sync |
+| props_file_path | String | N | ""
| Path of properties file
|
+| bootstrap_overwrite | Boolean | N | false
| Overwrite bootstrap
path |
+
+**Output**
+
+| Output Name | Type |
+|-------------|---------|
+| status | Boolean |
+
+**Example**
+
+```
+call run_bootstrap(table => 'test_hudi_table', table_type => 'COPY_ON_WRITE',
bootstrap_path => 'hdfs://ns1/hive/warehouse/hudi.db/test_hudi_table',
base_path => 'hdfs://ns1//tmp/hoodie/test_hudi_table', rowKey_field => 'id',
partition_path_field => 'dt',bootstrap_overwrite => true);
+```
+
+| status |
+|--------|
+| 0 |
+
+### show_bootstrap_mapping
+
+Show mapping files of a bootstrap table.
+
+**Input**
+
+| Parameter Name | Type | Required | Default Value | Description
|
+|----------------|---------|----------|---------------|--------------------------------------|
+| table | String | Y | None | Name of table to be
clustered |
+| partition_path | String | N | "" | Partition path
|
+| file_ids | String | N | "" | File ids
|
+| limit | Int | N | 10 | Max number of records
to be returned |
+| sort_by | String | N | "partition" | Sort by columns
|
+| desc | Boolean | N | false | Descending order
|
+
+**Output**
+
+| Parameter Name | Type |
+|------------------|--------|
+| partition | String |
+| file_id | Int |
+| source_base_path | String |
+| source_partition | Int |
+| source_file | String |
+
+**Example**
+
+```
+call show_bootstrap_mapping(table => 'test_hudi_table')
+```
+
+| partition | file_id | source_base_path
| source_partition | source_file
|
+|---------------|----------------------------------------|-------------------------------------------------------------------------------------------------------------------------------------|------------------|--------------------------------------------|
+| dt=2021-05-03 | d0073a12-085d-4f49-83e9-402947e7e90a-0 |
hdfs://ns1/hive/warehouse/hudi.db/test_hudi_table/dt=2021-05-03/d0073a12-085d-4f49-83e9-402947e7e90a-0_0-2-2_00000000000002.parquet
| dt=2021-05-03 | hdfs://ns1/tmp/dt=2021-05-03/00001.parquet |
+
+
+### show_bootstrap_partitions
+
+Show partitions of a bootstrap table.
+
+**Input**
+
+| Parameter Name | Type | Required | Default Value | Description
|
+|----------------|--------|----------|---------------|--------------------------------------|
+| table | String | Y | None | Name of table to be
clustered |
+
+**Output**
+
+| Parameter Name | Type |
+|--------------------|--------|
+| indexed_partitions | String |
+
+**Example**
+
+```
+call show_bootstrap_partitions(table => 'test_hudi_table')
+```
+
+| indexed_partitions |
+|--------------------|
+| dt=2021-05-03 |
+
+## Version management
+
+### upgrade_table
+
+upgrade a hudi table to a specific version.
+
+**Input**
+
+| Parameter Name | Type | Required | Default Value | Description
|
+|----------------|--------|----------|---------------|-------------------------|
+| table | String | Y | None | Hudi table name
|
+| to_version | String | Y | None | Version of hoodie table
|
+
+**Output**
+
+| Output Name | Type |
+|-------------|---------|
+| result | Boolean |
+
+**Example**
+
+```
+call upgrade_table(table => 'test_hudi_table', to_version => 'FIVE');
+```
+
+| result |
+|--------|
+| true |
+
+### downgrade_table
+
+downgrade a hudi table to a specific version.
+
+**Input**
+
+| Parameter Name | Type | Required | Default Value | Description
|
+|----------------|--------|----------|---------------|-------------------------|
+| table | String | Y | None | Hudi table name
|
+| to_version | String | Y | None | Version of hoodie table
|
+
+**Output**
+
+| Output Name | Type |
+|-------------|---------|
+| result | Boolean |
+
+**Example**
+
+```
+call downgrade_table(table => 'test_hudi_table', to_version => 'FOUR');
+```
+
+| result |
+|--------|
+| true |
\ No newline at end of file
