bamaer opened a new pull request, #9340:
URL: https://github.com/apache/hop/pull/9340
Builds on #9339; the last commit is the new one.
- Write-ahead spool on disk (`HOP_LINEAGE_OPENLINEAGE_SPOOL_DIR`,
`_SPOOL_MAX_BYTES`):
events survive a backend outage and a restart.
- Batched sends; each send is sent, retried or rejected, and 401/403/404/407
are retried
instead of dropped.
- Extension hooks so another sink can reuse the delivery code (the DataHub
sink, #9034).
- Files always use OpenLineage spec naming.
`HOP_LINEAGE_OPENLINEAGE_FILE_SPEC_NAMING`
(2.20.0, off by default) is removed, so existing file datasets get new
names.
- Column types on schemas, copy/move pairs, renames only when a dataset's
identity changes,
and a Table Input query no longer overwrites the table's schema.
- `HOP_LINEAGE_OPENLINEAGE_TRANSFORM_JOBS` (off): a job per transform/action
with a `hop`
job facet. `HOP_LINEAGE_OPENLINEAGE_PARENT_RUNS`: parent run facets for
nested runs.
114 OpenLineage tests pass.
------------------------
Thank you for your contribution! Follow this checklist to help us
incorporate your contribution quickly and easily:
- [x] Run `mvn clean install apache-rat:check` to make sure basic checks
pass. A more thorough check will be performed on your pull request
automatically.
- [x] If you have a group of commits related to the same change, please
squash your commits into one and force push your branch using `git rebase -i`.
- [x] Mention the appropriate issue in your description (for example:
`addresses #123`), if applicable.
To make clear that you license your contribution under the [Apache License
Version 2.0, January 2004](http://www.apache.org/licenses/LICENSE-2.0)
you have to acknowledge this by using the following check-box.
- [x] I hereby declare this contribution to be licensed under the [Apache
License Version 2.0, January 2004](http://www.apache.org/licenses/LICENSE-2.0)
- [ ] In any other case, please file an [Apache Individual Contributor
License Agreement](https://www.apache.org/licenses/icla.pdf).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
