[
https://issues.apache.org/jira/browse/AIRFLOW-1252?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sergio Herrera updated AIRFLOW-1252:
------------------------------------
Description:
When someones calls to the endpoint _POST
<host>:<port>/api/experimental/dags/<dag_id>/dag_runs {}_, Airflow never run
that request if the body of that contains _conf_.
This occurs due to a mismatch between types when calling function
_trigger_dag()_, which is also used by *CLI*. That function perform a
_json.loads(conf)_ because from CLI the type of conf is _string_, but, in the
other side, from *experimental API*, that type is _dict_ (because _Json_ is
processed before to get all data, such as execution_date).
There are two possibilities:
1. Look for every use of _trigger_dag()_ function and put _Json_ formatting
from outside the function.
2. In the *experimental API*, put the conf in a string (with _json.dumps()_)
to allow _trigger_dag()_ transform into _dict_.
I have implemented the second option, so I can make a PR with that if you want.
Thank you a lot
EDIT: Also, there are no tests which uses conf in the Json passed through
request currently.
Examples:
- Before fix (escaped json):
{noformat}
POST /api/experimental/dags/test_conf/dag_runs HTTP/1.1
Content-Type: application/json
{
"conf": "{
\"k1\": \"v1\",
\"k2\": \"v2\",
\"k3\": [\"av1\", \"av2\", \"av3\"],
\"k4\": {
\"sk1\": \"sv1\",
\"sk2\": \"sv2\"
}
}"
}
{noformat}
- After fix (pure json):
{noformat}
POST /api/experimental/dags/test_conf/dag_runs HTTP/1.1
Content-Type: application/json
{
"conf": {
"k1": "v1",
"k2": "v2",
"k3": ["av1", "av2", "av3"],
"k4": {
"sk1": "sv1",
"sk2": "sv2"
}
}
}
{noformat}
was:
When someones calls to the endpoint _POST
<host>:<port>/api/experimental/dags/<dag_id>/dag_runs {}_, Airflow never run
that request if the body of that contains _conf_.
This occurs due to a mismatch between types when calling function
_trigger_dag()_, which is also used by *CLI*. That function perform a
_json.loads(conf)_ because from CLI the type of conf is _string_, but, in the
other side, from *experimental API*, that type is _dict_ (because _Json_ is
processed before to get all data, such as execution_date).
There are two possibilities:
1. Look for every use of _trigger_dag()_ function and put _Json_ formatting
from outside the function.
2. In the *experimental API*, put the conf in a string (with _json.dumps()_)
to allow _trigger_dag()_ transform into _dict_.
I have implemented the second option, so I can make a PR with that if you want.
Thank you a lot
EDIT: Also, there are no tests which uses conf in the Json passed through
request currently.
Examples:
- Before fix:
{{POST /api/experimental/dags/test_conf/dag_runs HTTP/1.1}}
{{Content-Type: application/json}}
{{{}}
{{ "conf": {}}
{{ \"k1\": \"v1\",}}
{{ \"k2\": \"v2\",}}{{ }}
{{ \"k3\": {{[\"av1\", \"av2\", \"av3\"]}},}}{{ }}
{{ \"k4\": }}{{{ }}
{{ \"sk1\": \"sv1\",}}
{{ \"sk2\": \"sv2\" }}
{{ }}}
{{ }}}
{{ }}}
- After fix:
{{POST /api/experimental/dags/test_conf/dag_runs HTTP/1.1}}
{{Content-Type: application/json}}{{ }}
{{{}}{{ }}
{{ "conf": {}}
{{ "k1": "v1",}}
{{ "k2": "v2",}}{{ }}
{{ "k3": {{["av1", "av2", "av3"]}},}}{{ }}
{{ "k4": }}{{{ }}
{{ "sk1": "sv1", }}
{{ "sk2": "sv2" }}
{{ }}}
{{ }}}
{{}}}
> Experimental API - exception when conf is present in JSON body
> --------------------------------------------------------------
>
> Key: AIRFLOW-1252
> URL: https://issues.apache.org/jira/browse/AIRFLOW-1252
> Project: Apache Airflow
> Issue Type: Bug
> Components: api
> Affects Versions: Airflow 1.8, 1.8.1, 1.9.0, 1.10.0, 2.0.0
> Reporter: Sergio Herrera
> Assignee: Sergio Herrera
> Priority: Major
> Labels: api
>
> When someones calls to the endpoint _POST
> <host>:<port>/api/experimental/dags/<dag_id>/dag_runs {}_, Airflow never run
> that request if the body of that contains _conf_.
> This occurs due to a mismatch between types when calling function
> _trigger_dag()_, which is also used by *CLI*. That function perform a
> _json.loads(conf)_ because from CLI the type of conf is _string_, but, in the
> other side, from *experimental API*, that type is _dict_ (because _Json_ is
> processed before to get all data, such as execution_date).
> There are two possibilities:
> 1. Look for every use of _trigger_dag()_ function and put _Json_ formatting
> from outside the function.
> 2. In the *experimental API*, put the conf in a string (with _json.dumps()_)
> to allow _trigger_dag()_ transform into _dict_.
> I have implemented the second option, so I can make a PR with that if you
> want.
> Thank you a lot
> EDIT: Also, there are no tests which uses conf in the Json passed through
> request currently.
> Examples:
> - Before fix (escaped json):
> {noformat}
> POST /api/experimental/dags/test_conf/dag_runs HTTP/1.1
> Content-Type: application/json
> {
> "conf": "{
> \"k1\": \"v1\",
> \"k2\": \"v2\",
> \"k3\": [\"av1\", \"av2\", \"av3\"],
> \"k4\": {
> \"sk1\": \"sv1\",
> \"sk2\": \"sv2\"
> }
> }"
> }
> {noformat}
> - After fix (pure json):
> {noformat}
> POST /api/experimental/dags/test_conf/dag_runs HTTP/1.1
> Content-Type: application/json
> {
> "conf": {
> "k1": "v1",
> "k2": "v2",
> "k3": ["av1", "av2", "av3"],
> "k4": {
> "sk1": "sv1",
> "sk2": "sv2"
> }
> }
> }
> {noformat}
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
