potiuk commented on PR #39606:
URL: https://github.com/apache/airflow/pull/39606#issuecomment-2119366778
The change for tests for Providers with Airflow 2.8 compatibility should be
ready. That was fairly easy - there is only one case where FileTransfer
operator looks like to be broken when openlineage is enabled. I disabled it for
2.8, but @mobuchowski @kacpermuda - maybe you would be able to fix it in the
provider?
Here is the error when I re-enable the test:
```python
_________________________________________________________________________________
test_get_openlineage_facets_on_start
_________________________________________________________________________________
def test_get_openlineage_facets_on_start():
src_bucket = "src-bucket"
src_key = "src-key"
dst_bucket = "dst-bucket"
dst_key = "dst-key"
expected_input = Dataset(namespace=f"s3://{src_bucket}",
name=src_key)
expected_output = Dataset(namespace=f"s3://{dst_bucket}",
name=dst_key)
op = FileTransferOperator(
task_id="test",
src=f"s3://{src_bucket}/{src_key}",
dst=f"s3://{dst_bucket}/{dst_key}",
)
> lineage = op.get_openlineage_facets_on_start()
tests/providers/common/io/operators/test_file_transfer.py:69:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
/usr/local/lib/python3.8/site-packages/airflow/providers/common/io/operators/file_transfer.py:85:
in get_openlineage_facets_on_start
input_dataset = Dataset(namespace=src.namespace, name=src.key)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
self = ObjectStoragePath('s3://src-bucket/src-key'), item = 'namespace'
def __getattr__(self, item: str) -> Any:
if item == "_accessor":
# cache the _accessor attribute on first access
kwargs = self._kwargs.copy()
self._accessor = _accessor = self._default_accessor(self._url,
**kwargs)
return _accessor
else:
> raise AttributeError(item)
E AttributeError: namespace
/usr/local/lib/python3.8/site-packages/upath/core.py:267: AttributeError
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
