Blizzke opened a new issue, #37006:
URL: https://github.com/apache/airflow/issues/37006

   ### Description
   
   I would like to know if it is possible to have the behavior of the 
`register_dataset_change` function to copy the `extra` from the specified 
dataset if the actual `extra` argument is empty?
   
   
   ### Use case/motivation
   
   We have a lot of datasets where we, within the dataset, have versions of the 
data. 
   Because - for lineage etc - we want to keep the URI for all of those the 
same (it is the same dataset after all), we use the `extra` argument to pass 
along the version number etc. Unfortunately, after a change is registered, we 
lose that information. 
   
   It seems logical to us that when no `extra` is specified (which is *actually 
impossible* as far as we can see, since the current calling locations don't 
even allow it), that you fall back to the extra of the dataset. Our override is 
as simple as
   
   ```python
   class DatasetManager(OriginalDatasetManager):
       def register_dataset_change(self, *, task_instance: TaskInstance, 
dataset: Dataset, extra=None, session: Session,
                                   **kwargs) -> None:
           if extra is None and dataset.extra:
               extra = dataset.extra
   
           super().register_dataset_change(task_instance=task_instance, 
dataset=dataset, extra=extra, session=session,
                                           **kwargs)
   ```
   
   
   ### Related issues
   
   _No response_
   
   ### Are you willing to submit a PR?
   
   - [X] Yes I am willing to submit a PR!
   
   ### Code of Conduct
   
   - [X] I agree to follow this project's [Code of 
Conduct](https://github.com/apache/airflow/blob/main/CODE_OF_CONDUCT.md)
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Reply via email to