[
https://issues.apache.org/jira/browse/AIRFLOW-2910?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16833471#comment-16833471
]
Kamil Bregula commented on AIRFLOW-2910:
----------------------------------------
[~higrys], you have touched many issues. This ticket concerns the creation of
connections to the HTTP type using environment variables.
I think it's worth thinking about creating a connection, but it will be a more
complex process. I would like the JSON field to be an implementation detail,
not a way to configure some connection options.
Method get_hook is used by generic operators for DB.
[https://github.com/apache/airflow/blob/e62ad5333cbb56ae0f2001f0f79008a21c41a983/airflow/operators/generic_transfer.py
]
[https://github.com/apache/airflow/blob/e62ad5333cbb56ae0f2001f0f79008a21c41a983/airflow/operators/check_operator.py#L95
]However, I think, that this is a piece of code that is worth refactoring.
I think the problem is that we mix two matters:
* Determine the location of the resource on the Internet. Then we use the
address in HTTP / HTTPS / FTP format
* Determine the configuration of the connect. Then we use a custom Airflow
format.
We can add support for the first formats when the right schema is equals to
"HTTP" or "HTTPS". I will prepare a PR to present my solution.
> models.Connection cannot use https
> ----------------------------------
>
> Key: AIRFLOW-2910
> URL: https://issues.apache.org/jira/browse/AIRFLOW-2910
> Project: Apache Airflow
> Issue Type: Bug
> Reporter: isaac martin
> Priority: Major
>
> The SimpleHttpOperator, and anything else relying on
> airlfow.models.Connection, cannot make use of https due to what appears to be
> a bug in the way it parses user-provided urls. The bug ends up replacing any
> https uri with an http uri.
> To reproduce:
> * Create a new airflow implementation.
> * Set a connection environment var:
> AIRFLOW_CONN_ETL_API=[https://yourdomain.com|https://yourdomain.com/]
> * Instantiate a SimpleHttpOperator which uses the above for its http_conn_id
> argument.
> * Notice with horror that your requests are made to http://yourdomain.com
> To fix:
> Proposal 1
> Line 590 of airflow.models.py assigns nothing to Connection.schema.
> Change:
> self.schema = temp_uri.path[1:]
> to
> self.schema = temp_uri[0]
>
> Proposal 2:
> Line 40 or airflow.hooks.http_hook.py starts a block which tries to set the
> base_url. We could add a new elif which checks self.conn_type, as
> self.conn_type is correctly populated with 'https'.
> For example:
> elif conn.conn_type:
> self.base_url = conn.conn_type + "://" + conn.host
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
