mattcasters opened a new pull request, #8118:
URL: https://github.com/apache/hop/pull/8118
Adds a **Data set output** transform so a pipeline can write rows into a
unit-test data set (metadata + CSV) from Hop GUI and `hop-run`, without
pointing a Text file output at the data set path.
This addresses #8035. The original request also asked for a “Current rows /
All rows” choice on **Create data set**; that stays out of scope. Canvas
**Create data set** and **Write rows to data set** already cover interactive
capture of the full pipeline stream.
### Transform
- Data set name, optional folder (`'${HOP_DATASETS_FOLDER}'` when empty),
and CSV filename
- **(Re)create data set** from incoming row metadata (default on)
- **Validate** incoming field count, names (same order, case-insensitive),
and types against an existing data set
- Pass-through output; CSV is overwritten each run; multiple copies are
rejected
- Empty streams still write a header-only CSV
The dialog is generated from `@GuiWidgetElement` annotations
(`GuiCompositeWidgets.addScrolledComposite`) with a grouped layout so the
fields sit in a fill-and-scroll container above the button bar.
### Internals
- New streaming `DataSetCsvWriter` (Apache VFS);
`DataSetCsvUtil.writeDataSetData` and **Write rows to data set** use it instead
of buffering the whole stream
- Schema helpers on `DataSet`: `createFieldsFromRowMeta` / `validateRowMeta`
### Tests and docs
- Unit tests for serialization, schema validation, and CSV round-trip
(`./mvnw -pl plugins/misc/testing test`, `apache-rat:check` on that module)
- Integration test `main-0108-data-set-output` (generate then read back
against a golden set)
- User-manual page plus nav / data-set / unit-testing docs
`GuiWidgetGroupType.BOXES` is annotated on the dialog fields; it currently
still renders as tabs until #8117.
------------------------
Thank you for your contribution! Follow this checklist to help us
incorporate your contribution quickly and easily:
- [x] Run `mvn clean install apache-rat:check` to make sure basic checks
pass. A more thorough check will be performed on your pull request
automatically.
- [x] If you have a group of commits related to the same change, please
squash your commits into one and force push your branch using `git rebase -i`.
- [x] Mention the appropriate issue in your description (for example:
`addresses #123`), if applicable.
To make clear that you license your contribution under the [Apache License
Version 2.0, January 2004](http://www.apache.org/licenses/LICENSE-2.0)
you have to acknowledge this by using the following check-box.
- [x] I hereby declare this contribution to be licensed under the [Apache
License Version 2.0, January 2004](http://www.apache.org/licenses/LICENSE-2.0)
- [ ] In any other case, please file an [Apache Individual Contributor
License Agreement](https://www.apache.org/licenses/icla.pdf).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
