Blizzke opened a new issue, #37006:
URL: https://github.com/apache/airflow/issues/37006
### Description
I would like to know if it is possible to have the behavior of the
`register_dataset_change` function to copy the `extra` from the specified
dataset if the actual `extra` argument is empty?
### Use case/motivation
We have a lot of datasets where we, within the dataset, have versions of the
data.
Because - for lineage etc - we want to keep the URI for all of those the
same (it is the same dataset after all), we use the `extra` argument to pass
along the version number etc. Unfortunately, after a change is registered, we
lose that information.
It seems logical to us that when no `extra` is specified (which is *actually
impossible* as far as we can see, since the current calling locations don't
even allow it), that you fall back to the extra of the dataset. Our override is
as simple as
```python
class DatasetManager(OriginalDatasetManager):
def register_dataset_change(self, *, task_instance: TaskInstance,
dataset: Dataset, extra=None, session: Session,
**kwargs) -> None:
if extra is None and dataset.extra:
extra = dataset.extra
super().register_dataset_change(task_instance=task_instance,
dataset=dataset, extra=extra, session=session,
**kwargs)
```
### Related issues
_No response_
### Are you willing to submit a PR?
- [X] Yes I am willing to submit a PR!
### Code of Conduct
- [X] I agree to follow this project's [Code of
Conduct](https://github.com/apache/airflow/blob/main/CODE_OF_CONDUCT.md)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]