[
https://issues.apache.org/jira/browse/AIRFLOW-3616?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ash Berlin-Taylor updated AIRFLOW-3616:
---------------------------------------
Fix Version/s: (was: 2.0.0)
1.10.3
Issue Type: Improvement (was: Bug)
> Connection parsed from URI - unacceptable underscore in schema part
> --------------------------------------------------------------------
>
> Key: AIRFLOW-3616
> URL: https://issues.apache.org/jira/browse/AIRFLOW-3616
> Project: Apache Airflow
> Issue Type: Improvement
> Reporter: Kamil Bregula
> Assignee: Kamil Bregula
> Priority: Major
> Fix For: 1.10.3
>
>
> There is a problem with creating a new connection based on the state of the
> environment variable if the connection type contains the underscore character.
> If we want to configure a connection based on an environment variable, we
> must create an environment variable. The name of the environment must be
> given according to the scheme:
> {{AIRFLOW_CONN_[CONN_ID]}}
> where {{[CONN_ID]}} is the connection identifier to be used, written in
> upper case.
> The content of the variable defines the connection and is saved in the form
> of URI.
> Defining a URI is complex, but the key is that the connection type is given
> as the schema. There are many possible values to give. but the sample values
> are {{mysql}}, {{postgresql}} or {{google_cloud_platform}}. Unfortunately,
> the last case is not properly handled.
> This is caused by using {{urllib.parse}} to process the value. Unfortunately,
> this module does not support the uppercase character in schema port of URI -
> see below snippets showing the behaviour.
> Since urllib.parse is really there to parse URLs and it is not good for
> parsing non-URL URIs - we should likely use different parser which handles
> more generic URIs.
> Especially that it also creates other problems:
> https://issues.apache.org/jira/browse/AIRFLOW-3615
> Another solution is to create aliases for each connection type with a variant
> that does not contain an unacceptable character. For example
> {{google_cloud_platform => gcp}}. It is worth noting that one alias is
> currently defined - {{postgresql => postgres}}.
> Snippet showing urrlib.parse behaviour:
> Python 3.6.5 (default, Oct 3 2018, 10:03:09)
> Type 'copyright', 'credits' or 'license' for more information
> IPython 7.0.1 – An enhanced Interactive Python. Type '?' for help
> In [1]: from urllib.parse import urlparse
> In [2]:
> urlparse("google_cloud_platform://user:pass@hostname/path?extra_param=extra_param_value")
> Out[2]: ParseResult(scheme='', netloc='',
> path='google_cloud_platform://user:pass@hostname/path', params='',
> query='extra_param=extra_param_value', fragment='')
> In [3]:
> urlparse("gcp://user:pass@hostname/path?extra_param=extra_param_value")
> Out[3]: ParseResult(scheme='gcp', netloc='user:pass@hostname', path='/path',
> params='', query='extra_param=extra_param_value', fragment='')
> Connection parsed from URI - unacceptable underscore in schema part
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
