nsivabalan opened a new pull request #3073:
URL: https://github.com/apache/hudi/pull/3073
## What is the purpose of the pull request
Added more yamls to test suite framework.
Default: sanity.yml
Optional yamls
medium_test_suite.yaml: medium sized test suite which validates entire input
for N no of rounds
long_test_suite.yaml: long running test suite which validates input after
every round and deletes the input data. So that we can scale this test for
larger iterations if required.
clustering.yaml : tests clustering.
## Brief change log
- Added more yamls to test suite framework.
## Verify this pull request
```
./generate_test_suite.sh --execute_test_suite false
--include_medium_test_suite_yaml true --include_long_test_suite_yaml true
--include_cluster_yaml true
Include Medium test suite true
Medium test suite iterations = 20
Include Long test suite true
Long test suite iterations = 50
Intermittent delay in mins = 1
Table type = COPY_ON_WRITE
Include cluster yaml true
Cluster total itr count 30
Cluster delay mins 1
Cluster exec itr count 15
Jar name hudi-integ-test-bundle-0.9.0-SNAPSHOT.jar
Input path \/user\/hive\/warehouse\/hudi-integ-test-suite\/input\/
Output path \/user\/hive\/warehouse\/hudi-integ-test-suite\/output\/
Cleaning up staging dir
Creating staging dir
```
Once executed above command, generated files can be found in staging folder.
```
ls demo/config/test-suite/staging
clustering.yaml
clustering_spark_command.sh
long_test_suite.yaml
long_test_suite_spark_command.sh
medium_test_suite.yaml
medium_test_suite_spark_command.sh
sanity.yaml
sanity_spark_command.sh
test.properties
```
You can run the same command w/o "--execute_test_suite false" which will go
ahead and execute the yamls included.
## Committer checklist
- [ ] Has a corresponding JIRA in PR title & commit
- [ ] Commit message is descriptive of the change
- [ ] CI is green
- [ ] Necessary doc changes done or have another open PR
- [ ] For large changes, please consider breaking it into sub-tasks under
an umbrella JIRA.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
