[
https://issues.apache.org/jira/browse/AIRFLOW-3720?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16744957#comment-16744957
]
ASF GitHub Bot commented on AIRFLOW-3720:
-----------------------------------------
chaimt commented on pull request #4546: AIRFLOW-3720
URL: https://github.com/apache/airflow/pull/4546
Make sure you have checked _all_ steps below.
### Jira
- [ ] My PR addresses the following [Airflow Jira]
- https://issues.apache.org/jira/browse/AIRFLOW-3720
### Description
- [ ] When comparing google storage to s3 buckets, folder names are not
compared correctly
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
> GoogleCloudStorageToS3Operator - incorrect folder compare
> ----------------------------------------------------------
>
> Key: AIRFLOW-3720
> URL: https://issues.apache.org/jira/browse/AIRFLOW-3720
> Project: Apache Airflow
> Issue Type: Bug
> Components: boto3
> Affects Versions: 1.10.0
> Reporter: Chaim
> Assignee: Chaim
> Priority: Major
> Fix For: 1.10.2
>
>
> the code that compares folders from gcp to s3 is incorrect.
> the code is:
> files = set(files) - set(existing_files)
> but the list from gcp has a "/" to the name, for example: "myfolder/", while
> in s3 it does not have "/" so the folder is "myfolder"
> the result is that the code tries to recopy the folder name but fails since
> it already exists
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
