Ke Zhu created AIRFLOW-5627:
-------------------------------
Summary: transformation function should be optional for
s3_file_tranformation_operator
Key: AIRFLOW-5627
URL: https://issues.apache.org/jira/browse/AIRFLOW-5627
Project: Apache Airflow
Issue Type: Improvement
Components: operators
Affects Versions: 2.0.0
Reporter: Ke Zhu
h3. What happened
After AIRFLOW-2299, it asks people to choose either {{transform_expression}} or
{{transform_script}} when using S3FileTransformOperator. According to user case
like moving objects only without any content transformation, it has to use some
hack like {{transform_script='/bin/cp'}}, which simply copy a temp file to
another temp file.
If you use neither parameter, it will throw exception saying {{Either
transform_script or select_expression must be specified}}. See
[https://github.com/apache/airflow/blob/d719e1fd6705a93a0dfefef4b46478ade5e006ea/airflow/operators/s3_file_transform_operator.py#L110-L112]
h3. Expected outcome
enhancement like -AIRFLOW-2299- should not force user to use new added feature
like transform_expression/transform_script or choose hacking path to workaround
it. these two parameters should just be optional.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
