Adamyuanyuan opened a new pull request, #2:
URL: https://github.com/apache/seatunnel-tools/pull/2
<!--
Thank you for contributing to SeaTunnel! Please make sure that your code
changes
are covered with tests. And in case of new features or big changes
remember to adjust the documentation.
Feel free to ping committers for the review!
## Contribution Checklist
- Make sure that the pull request corresponds to a [GITHUB
issue](https://github.com/apache/seatunnel/issues).
- Name the pull request in the form "[Feature] [component] Title of the
pull request", where *Feature* can be replaced by `Hotfix`, `Bug`, etc.
- Minor fixes should be named following this pattern: `[hotfix] [docs] Fix
typo in README.md doc`.
-->
### Purpose of this pull request
X2SeaTunnel is a tool for converting DataX and other configuration files to
SeaTunnel configuration files, designed to help users quickly migrate from
other data integration platforms to SeaTunnel.
This is the implementation of the first version.
### Does this PR introduce _any_ user-facing change?
Added a new tool with the following functions:
- ✅ **Standard Configuration Conversion**: DataX → SeaTunnel configuration
file conversion
- ✅ **Custom Template Conversion**: Support for user-defined conversion
templates
- ✅ **Detailed Conversion Reports**: Generate Markdown format conversion
reports
- ✅ **Regular Expression Variable Extraction**: Extract variables from
configuration using regex, supporting custom scenarios
- ✅ **Batch Conversion Mode**: Support directory and file wildcard batch
conversion, automatic report and summary report generation
### Basic Usage
```bash
# Standard conversion: Use default template system with built-in common
Sources and Sinks
./bin/x2seatunnel.sh -s examples/source/datax-mysql2hdfs.json -t
examples/target/mysql2hdfs-result.conf -r examples/report/mysql2hdfs-report.md
# Custom task: Implement customized conversion requirements through custom
templates
# Scenario: MySQL → Hive (DataX doesn't have HiveWriter)
# DataX configuration: MySQL → HDFS Custom task: Convert to MySQL → Hive
./bin/x2seatunnel.sh -s examples/source/datax-mysql2hdfs2hive.json -t
examples/target/mysql2hive-result.conf -r examples/report/mysql2hive-report.md
-T templates/datax/custom/mysql-to-hive.conf
# YAML configuration method (equivalent to above command line parameters)
./bin/x2seatunnel.sh -c examples/yaml/datax-mysql2hdfs2hive.yaml
# Batch conversion mode: Process by directory
./bin/x2seatunnel.sh -d examples/source -o examples/target2 -R
examples/report2
# Batch mode supports wildcard filtering
./bin/x2seatunnel.sh -d examples/source -o examples/target3 -R
examples/report3 --pattern "*-full.json" --verbose
# View help
./bin/x2seatunnel.sh --help
```
### Conversion Report
After conversion is completed, view the generated Markdown report file,
which includes:
- **Basic Information**: Conversion time, source/target file paths,
connector types, conversion status, etc.
- **Conversion Statistics**: Counts and percentages of direct mappings,
smart transformations, default values used, and unmapped fields
- **Detailed Field Mapping Relationships**: Source values, target values,
filters used for each field
- **Default Value Usage**: List of all fields using default values
- **Unmapped Fields**: Fields present in DataX but not converted
- **Possible Error and Warning Information**: Issue prompts during
conversion process
For batch conversions, a batch summary report `summary.md` will be generated
in the batch report directory, including:
- **Conversion Overview**: Overall statistics, success rate, duration, etc.
- **Successful Conversion List**: Complete list of successfully converted
files
- **Failed Conversion List**: Failed files and error messages (if any)
### How was this patch tested?
Yes, including unit tests, local tests, and production tests.
### Check list
* [x] If any new Jar binary package adding in your PR, please add License
Notice according
[New License
Guide](https://github.com/apache/seatunnel/blob/dev/docs/en/contribution/new-license.md)
* [ ] If necessary, please update the documentation to describe the new
feature. https://github.com/apache/seatunnel/tree/dev/docs
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
