[jira] [Created] (AIRFLOW-2655) Default Kubernetes worker configurations are inconsistent
Shintaro Murakami created AIRFLOW-2655: -- Summary: Default Kubernetes worker configurations are inconsistent Key: AIRFLOW-2655 URL: https://issues.apache.org/jira/browse/AIRFLOW-2655 Project: Apache Airflow Issue Type: Bug Components: executor Affects Versions: 1.10.0 Reporter: Shintaro Murakami if optional config `airflow_configmap` is not set, the worker configured with `LocalExecutor` and sql_alchemy_conn starts with `sqlite`. This combination is not allowed. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (AIRFLOW-2567) Allow Kubernetes Pod Operator to extract result from the container as Xcom
Shintaro Murakami created AIRFLOW-2567: -- Summary: Allow Kubernetes Pod Operator to extract result from the container as Xcom Key: AIRFLOW-2567 URL: https://issues.apache.org/jira/browse/AIRFLOW-2567 Project: Apache Airflow Issue Type: Improvement Components: operators Affects Versions: 1.10.0 Reporter: Shintaro Murakami In order to simplify the construction of DAG, it is useful to be able to extract the execution result of the container as XCom. Users will be able to do this by writing the result as the `result.json` in `/airflow/xcom` directory mounted on the container automatically. This function is realized by Pod's sidecar container. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (AIRFLOW-2530) KubernetesOperator supports multiple cluster
Shintaro Murakami created AIRFLOW-2530: -- Summary: KubernetesOperator supports multiple cluster Key: AIRFLOW-2530 URL: https://issues.apache.org/jira/browse/AIRFLOW-2530 Project: Apache Airflow Issue Type: Improvement Components: operators Affects Versions: 1.10.0 Reporter: Shintaro Murakami Now, kubernetesOperator supports only in-cluster or current context. It's useful to support multiple clusters with specifying the kubernetes context. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (AIRFLOW-1730) The value of XCom that queried from DB is not unpickled.
Shintaro Murakami created AIRFLOW-1730: -- Summary: The value of XCom that queried from DB is not unpickled. Key: AIRFLOW-1730 URL: https://issues.apache.org/jira/browse/AIRFLOW-1730 Project: Apache Airflow Issue Type: Bug Components: models, xcom Affects Versions: 1.9.0 Reporter: Shintaro Murakami Attachments: xcoms_by_example_xcom.png If enable_xcom_pickling is True, the value of XCom that queried from DB is not unpickled. The list of XComs not rendered correctly. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (AIRFLOW-1704) If the subdag is defined as a local variable, its variable is_subdag is False.
[ https://issues.apache.org/jira/browse/AIRFLOW-1704?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shintaro Murakami updated AIRFLOW-1704: --- Attachment: dags.png tree.png > If the subdag is defined as a local variable, its variable is_subdag is False. > -- > > Key: AIRFLOW-1704 > URL: https://issues.apache.org/jira/browse/AIRFLOW-1704 > Project: Apache Airflow > Issue Type: Bug > Components: models >Reporter: Shintaro Murakami > Attachments: dags.png, test_subdag_local_variable.py, tree.png > > > If the subdag is defined as a local variable in DAG definition code, its > variable is_subdag is False. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (AIRFLOW-1704) If the subdag is defined as a local variable, its variable is_subdag is False.
[ https://issues.apache.org/jira/browse/AIRFLOW-1704?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shintaro Murakami updated AIRFLOW-1704: --- Attachment: test_subdag_local_variable.py DAG definition for reproducing > If the subdag is defined as a local variable, its variable is_subdag is False. > -- > > Key: AIRFLOW-1704 > URL: https://issues.apache.org/jira/browse/AIRFLOW-1704 > Project: Apache Airflow > Issue Type: Bug > Components: models >Reporter: Shintaro Murakami > Attachments: test_subdag_local_variable.py > > > If the subdag is defined as a local variable in DAG definition code, its > variable is_subdag is False. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (AIRFLOW-1704) If the subdag is defined as a local variable, its variable is_subdag is False.
Shintaro Murakami created AIRFLOW-1704: -- Summary: If the subdag is defined as a local variable, its variable is_subdag is False. Key: AIRFLOW-1704 URL: https://issues.apache.org/jira/browse/AIRFLOW-1704 Project: Apache Airflow Issue Type: Bug Components: models Reporter: Shintaro Murakami If the subdag is defined as a local variable in DAG definition code, its variable is_subdag is False. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (AIRFLOW-1654) Show tooltips for link icons in DAGs view
[ https://issues.apache.org/jira/browse/AIRFLOW-1654?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shintaro Murakami updated AIRFLOW-1654: --- Attachment: screenshot-1.png > Show tooltips for link icons in DAGs view > - > > Key: AIRFLOW-1654 > URL: https://issues.apache.org/jira/browse/AIRFLOW-1654 > Project: Apache Airflow > Issue Type: Improvement >Reporter: Shintaro Murakami >Priority: Minor > Attachments: screenshot-1.png > > > It is hard to me to remember what icons mean. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (AIRFLOW-1654) Show tooltips for link icons in DAGs view
[ https://issues.apache.org/jira/browse/AIRFLOW-1654?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shintaro Murakami updated AIRFLOW-1654: --- Attachment: (was: スクリーンショット 2017-09-28 17.48.18.png) > Show tooltips for link icons in DAGs view > - > > Key: AIRFLOW-1654 > URL: https://issues.apache.org/jira/browse/AIRFLOW-1654 > Project: Apache Airflow > Issue Type: Improvement >Reporter: Shintaro Murakami >Priority: Minor > > It is hard to me to remember what icons mean. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (AIRFLOW-1654) Show tooltips for link icons in DAGs view
Shintaro Murakami created AIRFLOW-1654: -- Summary: Show tooltips for link icons in DAGs view Key: AIRFLOW-1654 URL: https://issues.apache.org/jira/browse/AIRFLOW-1654 Project: Apache Airflow Issue Type: Improvement Reporter: Shintaro Murakami Priority: Minor Attachments: スクリーンショット 2017-09-28 17.48.18.png It is hard to me to remember what icons mean. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (AIRFLOW-1595) SqliteHook is broken
[ https://issues.apache.org/jira/browse/AIRFLOW-1595?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16183854#comment-16183854 ] Shintaro Murakami commented on AIRFLOW-1595: I created PR(https://github.com/apache/incubator-airflow/pull/2598) > SqliteHook is broken > > > Key: AIRFLOW-1595 > URL: https://issues.apache.org/jira/browse/AIRFLOW-1595 > Project: Apache Airflow > Issue Type: Bug > Components: hooks >Affects Versions: 1.8.1 >Reporter: Shintaro Murakami > > SqliteHook is built using the host attribute of connection, but correctly we > should use the schema attribute. The path to the DB parsed from the URI is > set as schema. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (AIRFLOW-1628) Docstring of sqlsensor is incorrect
[
https://issues.apache.org/jira/browse/AIRFLOW-1628?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Shintaro Murakami updated AIRFLOW-1628:
---
Description: The description of keep trying condition for sqlsensor is
strange. The condition should be {{while}} not {{until}}. (was: The
description of keep trying condition for sqlsensor is strange.)
> Docstring of sqlsensor is incorrect
> ---
>
> Key: AIRFLOW-1628
> URL: https://issues.apache.org/jira/browse/AIRFLOW-1628
> Project: Apache Airflow
> Issue Type: Bug
> Components: docs, hooks
>Reporter: Shintaro Murakami
>Priority: Trivial
>
> The description of keep trying condition for sqlsensor is strange. The
> condition should be {{while}} not {{until}}.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Created] (AIRFLOW-1628) Docstring of sqlsensor is incorrect
Shintaro Murakami created AIRFLOW-1628: -- Summary: Docstring of sqlsensor is incorrect Key: AIRFLOW-1628 URL: https://issues.apache.org/jira/browse/AIRFLOW-1628 Project: Apache Airflow Issue Type: Bug Components: docs, hooks Reporter: Shintaro Murakami Priority: Trivial The description of keep trying condition for sqlsensor is strange. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (AIRFLOW-1330) Connection.parse_from_uri doesn't work for google_cloud_platform and so on
[
https://issues.apache.org/jira/browse/AIRFLOW-1330?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16174120#comment-16174120
]
Shintaro Murakami commented on AIRFLOW-1330:
Fixed it with the option (2). In addition, made it possible to parse even the
scheme containing underscores. Since we did not want to add all setters to
Connection class, made {{conn_uri}} and {{conn_type}} be exclusive.
> Connection.parse_from_uri doesn't work for google_cloud_platform and so on
> --
>
> Key: AIRFLOW-1330
> URL: https://issues.apache.org/jira/browse/AIRFLOW-1330
> Project: Apache Airflow
> Issue Type: Bug
> Components: cli
>Reporter: Yu Ishikawa
>Assignee: Shintaro Murakami
>
> h2. Overview
> {{Connection.parse_from_uri}} doesn't work for some types like
> {{google_cloud_platform}} whose type name includes under scores. Since
> `urllib.parse.urlparse()` which is used in {{Connection.parse_from_url}}
> doesn't support a schema name which include under scores.
> So, airflow's CLI doesn't work when a given connection URI includes under
> scores like {{google_cloud_platform://X}}.
> h3. Workaround
> https://medium.com/@yuu.ishikawa/apache-airflow-how-to-add-a-connection-to-google-cloud-with-cli-af2cc8df138d
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (AIRFLOW-1595) SqliteHook is broken
[ https://issues.apache.org/jira/browse/AIRFLOW-1595?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shintaro Murakami updated AIRFLOW-1595: --- Description: SqliteHook is built using the host attribute of connection, but correctly we should use the schema attribute. The path to the DB parsed from the URI is set as schema. (was: SqliteHook is built using the host attribute of connection, but correctly we should use the schema attribute.) > SqliteHook is broken > > > Key: AIRFLOW-1595 > URL: https://issues.apache.org/jira/browse/AIRFLOW-1595 > Project: Apache Airflow > Issue Type: Bug > Components: hooks >Affects Versions: 1.8.1 >Reporter: Shintaro Murakami > > SqliteHook is built using the host attribute of connection, but correctly we > should use the schema attribute. The path to the DB parsed from the URI is > set as schema. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (AIRFLOW-1542) X-axis labels of charts display only numbers correctly
[
https://issues.apache.org/jira/browse/AIRFLOW-1542?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Shintaro Murakami updated AIRFLOW-1542:
---
Description:
Since the chart's x-axis label is formatted as {{.2f}}, only numbers (and date
time) are displayed correctly.
!label_nan.png!
was:
Since the chart's x-axis label is formatted as `.2f`, only numbers (and date
time) are displayed correctly.
!label_nan.png!
> X-axis labels of charts display only numbers correctly
> --
>
> Key: AIRFLOW-1542
> URL: https://issues.apache.org/jira/browse/AIRFLOW-1542
> Project: Apache Airflow
> Issue Type: Bug
> Components: webapp
>Reporter: Shintaro Murakami
> Attachments: label_nan.png
>
>
> Since the chart's x-axis label is formatted as {{.2f}}, only numbers (and
> date time) are displayed correctly.
> !label_nan.png!
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (AIRFLOW-1542) X-axis labels of charts display only numbers correctly
[ https://issues.apache.org/jira/browse/AIRFLOW-1542?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shintaro Murakami updated AIRFLOW-1542: --- Description: Since the chart's x-axis label is formatted as `.2f`, only numbers (and date time) are displayed correctly. !label_nan.png! was: Since the chart's x-axis label is formatted as `.2f`, only numerical (and date time) are displayed correctly. !label_nan.png! > X-axis labels of charts display only numbers correctly > -- > > Key: AIRFLOW-1542 > URL: https://issues.apache.org/jira/browse/AIRFLOW-1542 > Project: Apache Airflow > Issue Type: Bug > Components: webapp >Reporter: Shintaro Murakami > Attachments: label_nan.png > > > Since the chart's x-axis label is formatted as `.2f`, only numbers (and date > time) are displayed correctly. > !label_nan.png! -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (AIRFLOW-1542) X-axis labels of charts display only numbers correctly
[ https://issues.apache.org/jira/browse/AIRFLOW-1542?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shintaro Murakami updated AIRFLOW-1542: --- Summary: X-axis labels of charts display only numbers correctly (was: X-axis labels of charts display only numericals correctly) > X-axis labels of charts display only numbers correctly > -- > > Key: AIRFLOW-1542 > URL: https://issues.apache.org/jira/browse/AIRFLOW-1542 > Project: Apache Airflow > Issue Type: Bug > Components: webapp >Reporter: Shintaro Murakami > Attachments: label_nan.png > > > Since the chart's x-axis label is formatted as `.2f`, only numerical (and > date time) are displayed correctly. > !label_nan.png! -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (AIRFLOW-1542) X-axis labels of charts display only numericals correctly
[ https://issues.apache.org/jira/browse/AIRFLOW-1542?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shintaro Murakami updated AIRFLOW-1542: --- Attachment: label_nan.png > X-axis labels of charts display only numericals correctly > - > > Key: AIRFLOW-1542 > URL: https://issues.apache.org/jira/browse/AIRFLOW-1542 > Project: Apache Airflow > Issue Type: Bug > Components: webapp >Reporter: Shintaro Murakami > Attachments: label_nan.png > > > Since the chart's x-axis label is formatted as `.2f`, only numerical (and > date time) are displayed correctly. > !label_nan.png! -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (AIRFLOW-1542) X-axis labels of charts display only numericals correctly
Shintaro Murakami created AIRFLOW-1542: -- Summary: X-axis labels of charts display only numericals correctly Key: AIRFLOW-1542 URL: https://issues.apache.org/jira/browse/AIRFLOW-1542 Project: Apache Airflow Issue Type: Bug Components: webapp Reporter: Shintaro Murakami Attachments: label_nan.png Since the chart's x-axis label is formatted as `.2f`, only numerical (and date time) are displayed correctly. !label_nan.png|thumbnail! -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Comment Edited] (AIRFLOW-1330) Connection.parse_from_uri doesn't work for google_cloud_platform and so on
[
https://issues.apache.org/jira/browse/AIRFLOW-1330?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16144995#comment-16144995
]
Shintaro Murakami edited comment on AIRFLOW-1330 at 8/29/17 9:34 AM:
-
It would be better to drop {{--conn_url}}, or specify either only one of
{{--conn_url}} or other new arguments. If we set both {{--conn_url}} and other
new arguments, it's confusing as to which one takes precedence.
After deciding which plan to go with, I will fix this
PR([https://github.com/apache/incubator-airflow/pull/2525]).
was (Author: mrkm4ntr):
It would be better to drop '--conn_url'{{monospaced text}}, or specify either
only one of `--conn_url` or other new arguments. If we set both `--conn_url`
and other new arguments, it's confusing as to which one takes precedence.
After deciding which plan to go with, I will fix this
PR([https://github.com/apache/incubator-airflow/pull/2525]).
> Connection.parse_from_uri doesn't work for google_cloud_platform and so on
> --
>
> Key: AIRFLOW-1330
> URL: https://issues.apache.org/jira/browse/AIRFLOW-1330
> Project: Apache Airflow
> Issue Type: Bug
> Components: cli
>Reporter: Yu Ishikawa
>
> h2. Overview
> {{Connection.parse_from_uri}} doesn't work for some types like
> {{google_cloud_platform}} whose type name includes under scores. Since
> `urllib.parse.urlparse()` which is used in {{Connection.parse_from_url}}
> doesn't support a schema name which include under scores.
> So, airflow's CLI doesn't work when a given connection URI includes under
> scores like {{google_cloud_platform://X}}.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Comment Edited] (AIRFLOW-1330) Connection.parse_from_uri doesn't work for google_cloud_platform and so on
[
https://issues.apache.org/jira/browse/AIRFLOW-1330?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16144995#comment-16144995
]
Shintaro Murakami edited comment on AIRFLOW-1330 at 8/29/17 9:33 AM:
-
It would be better to drop {noformat}--conn_url{noformat}, or specify either
only one of `--conn_url` or other new arguments. If we set both `--conn_url`
and other new arguments, it's confusing as to which one takes precedence.
After deciding which plan to go with, I will fix this
PR([https://github.com/apache/incubator-airflow/pull/2525]).
was (Author: mrkm4ntr):
It would be better to drop `--conn_url`, or specify either only one of
`--conn_url` or other new arguments. If we set both `--conn_url` and other new
arguments, it's confusing as to which one takes precedence.
After deciding which plan to go with, I will fix this
PR([https://github.com/apache/incubator-airflow/pull/2525]).
> Connection.parse_from_uri doesn't work for google_cloud_platform and so on
> --
>
> Key: AIRFLOW-1330
> URL: https://issues.apache.org/jira/browse/AIRFLOW-1330
> Project: Apache Airflow
> Issue Type: Bug
> Components: cli
>Reporter: Yu Ishikawa
>
> h2. Overview
> {{Connection.parse_from_uri}} doesn't work for some types like
> {{google_cloud_platform}} whose type name includes under scores. Since
> `urllib.parse.urlparse()` which is used in {{Connection.parse_from_url}}
> doesn't support a schema name which include under scores.
> So, airflow's CLI doesn't work when a given connection URI includes under
> scores like {{google_cloud_platform://X}}.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Created] (AIRFLOW-1497) Hidden fields in connection form aren't reset when changing connection type
Shintaro Murakami created AIRFLOW-1497: -- Summary: Hidden fields in connection form aren't reset when changing connection type Key: AIRFLOW-1497 URL: https://issues.apache.org/jira/browse/AIRFLOW-1497 Project: Apache Airflow Issue Type: Bug Components: ui Reporter: Shintaro Murakami Priority: Minor Some fields in connection form are hidden when selecting specific connection type. The fields once hidden are cannot be shown again by changing connection type. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (AIRFLOW-1489) Docs: Typo in BigQueryCheckOperator
Shintaro Murakami created AIRFLOW-1489: -- Summary: Docs: Typo in BigQueryCheckOperator Key: AIRFLOW-1489 URL: https://issues.apache.org/jira/browse/AIRFLOW-1489 Project: Apache Airflow Issue Type: Improvement Components: docs Reporter: Shintaro Murakami Priority: Trivial In 'airflow/contrib/operators/bigquery_check_operator.py', some descriptions remain as 'Presto'. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
