Syar123 opened a new issue #21714:
URL: https://github.com/apache/airflow/issues/21714
### Description
in airflow 1: we used to pass parameters like below to external dag trigger
def trigger_gcs_to_bq_dag(context, dag_run_obj) :
dag_run_obj.payload = {
'gcs_bucket" : context['ti' ].xcom_pull (key='gcs_bucket'
, task_ids='parse_params' ), "gcs_objects_list"
context['ti'].xcom_pull(key='gcs_objects_list'
task_ids='parse_params')
, task_ids='filter_gcs_files'),"bq_schema"
:context['ti'].xcom_pull(key='bq_schema
}
return dag_run_obi
trigger_gcs_to _ba_dag = TriggerDagRunoperator(
task_id='trigger_gcs_to_bq_dag'
trigger_dag_id="gcs_to_bg"
python_callable=trigger_gcs_to_bq_dag
)
in airflow 2.1.4 as python_callable is deprecated we need to pass parameters
through conf
trigger_gcs_to_bq_dag = TriggerDagRunOperator (
task_id='trigger_gcs_to_bq_dag'
trigger_dag_id="gcs_to_bq"
conf= {"gcs_bucket":
"{{ti.xcom_pull(key='gcs_bucket', task_ids='parse_params') }}",
"gcs_objects_list":
“{{ti.xcom_pull(key='gcs_objects_list', task_ids='parse_params') }}",
"bq_schema":
"{{ xcom_pull (key=' bq_schema',
task_ids='parse_params') }}" },
)
when passing through conf the parameter values are going as strings instead
of their own values (None is passing as 'None')
in the external dag we are using logic like
if params['bq-schema' ] is not None:
schema_fields=[
{
"name' : f[0],
'type': f[11
} for f in params['bq_schema']]
and other values which arent strings.
how can we eradicate this issue in airflow 2.1.4 when the previous modules
are deprecated?
### Use case/motivation
_No response_
### Related issues
_No response_
### Are you willing to submit a PR?
- [ ] Yes I am willing to submit a PR!
### Code of Conduct
- [X] I agree to follow this project's [Code of
Conduct](https://github.com/apache/airflow/blob/main/CODE_OF_CONDUCT.md)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
