[
https://issues.apache.org/jira/browse/AIRFLOW-6372?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Yuen-Kuei Hsueh updated AIRFLOW-6372:
-------------------------------------
Description:
The `remote_base_log_folder` for S3 and GCS are URI Schemes, but Azure Blob
Storage is an ugly format `wasb-<whatever you want here>` and [requires
additional
settings|https://airflow.apache.org/docs/stable/howto/write-logs.html#writing-logs-to-azure-blob-storage].
For Hadoop ecosystem, the [URI Scheme for Azure Blob
Storage|https://hadoop.apache.org/docs/current/hadoop-azure/index.html#Accessing_wasb_URLs]
was `wasb[s]://<containername>@<accountname>.blob.core.windows.net/<path>`. So
I think we can use Hadoop URI Scheme for Azure Blob Storage to align the
configuration and omit some parameter, such as `container_name`.
The changes should be also compatible with legacy configuration.
was:
The `remote_base_log_folder` for S3 and GCS are URI Schemes, but Azure Blob
Storage is an ugly format `wasb-<whatever you want here>` and [requires
additional
settings|https://airflow.apache.org/docs/stable/howto/write-logs.html#writing-logs-to-azure-blob-storage].
For Hadoop ecosystem, the [URI Scheme for Azure Blob
Storage|https://hadoop.apache.org/docs/current/hadoop-azure/index.html#Accessing_wasb_URLs]
was `wasb[s]://<containername>@<accountname>.blob.core.windows.net/<path>`. So
I think we can use Hadoop URI Scheme for Azure Blob Storage to align the
configuration and omit some parameter, such as `container_name`.
The changes should be also compatible with legacy configuration.
> Align Azure Blob Storage remote logging URI scheme with S3 and GCS
> ------------------------------------------------------------------
>
> Key: AIRFLOW-6372
> URL: https://issues.apache.org/jira/browse/AIRFLOW-6372
> Project: Apache Airflow
> Issue Type: Improvement
> Components: logging
> Affects Versions: 1.10.6
> Reporter: Yuen-Kuei Hsueh
> Assignee: Yuen-Kuei Hsueh
> Priority: Minor
>
> The `remote_base_log_folder` for S3 and GCS are URI Schemes, but Azure Blob
> Storage is an ugly format `wasb-<whatever you want here>` and [requires
> additional
> settings|https://airflow.apache.org/docs/stable/howto/write-logs.html#writing-logs-to-azure-blob-storage].
> For Hadoop ecosystem, the [URI Scheme for Azure Blob
> Storage|https://hadoop.apache.org/docs/current/hadoop-azure/index.html#Accessing_wasb_URLs]
> was `wasb[s]://<containername>@<accountname>.blob.core.windows.net/<path>`.
> So I think we can use Hadoop URI Scheme for Azure Blob Storage to align the
> configuration and omit some parameter, such as `container_name`.
>
> The changes should be also compatible with legacy configuration.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
