[jira] [Resolved] (AIRFLOW-7005) add exec option for breeze
[ https://issues.apache.org/jira/browse/AIRFLOW-7005?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jarek Potiuk resolved AIRFLOW-7005. --- Fix Version/s: 1.10.10 Resolution: Fixed > add exec option for breeze > -- > > Key: AIRFLOW-7005 > URL: https://issues.apache.org/jira/browse/AIRFLOW-7005 > Project: Apache Airflow > Issue Type: Improvement > Components: breeze >Affects Versions: 2.0.0, 1.10.9 >Reporter: Jarek Potiuk >Priority: Major > Fix For: 1.10.10 > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (AIRFLOW-7002) Get rid of Yaml "parser" in bash
[ https://issues.apache.org/jira/browse/AIRFLOW-7002?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jarek Potiuk resolved AIRFLOW-7002. --- Fix Version/s: 1.10.10 Resolution: Fixed > Get rid of Yaml "parser" in bash > > > Key: AIRFLOW-7002 > URL: https://issues.apache.org/jira/browse/AIRFLOW-7002 > Project: Apache Airflow > Issue Type: Improvement > Components: breeze, ci >Affects Versions: 2.0.0, 1.10.9 >Reporter: Jarek Potiuk >Priority: Major > Fix For: 1.10.10 > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (AIRFLOW-7006) Missing +e in the breeze script
[ https://issues.apache.org/jira/browse/AIRFLOW-7006?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jarek Potiuk resolved AIRFLOW-7006. --- Fix Version/s: 1.10.10 Resolution: Fixed > Missing +e in the breeze script > --- > > Key: AIRFLOW-7006 > URL: https://issues.apache.org/jira/browse/AIRFLOW-7006 > Project: Apache Airflow > Issue Type: Bug > Components: breeze >Affects Versions: 2.0.0, 1.10.9 >Reporter: Jarek Potiuk >Priority: Major > Fix For: 1.10.10 > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (AIRFLOW-7006) Missing +e in the breeze script
Jarek Potiuk created AIRFLOW-7006: - Summary: Missing +e in the breeze script Key: AIRFLOW-7006 URL: https://issues.apache.org/jira/browse/AIRFLOW-7006 Project: Apache Airflow Issue Type: Bug Components: breeze Affects Versions: 1.10.9, 2.0.0 Reporter: Jarek Potiuk -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (AIRFLOW-7005) add exec option for breeze
Jarek Potiuk created AIRFLOW-7005: - Summary: add exec option for breeze Key: AIRFLOW-7005 URL: https://issues.apache.org/jira/browse/AIRFLOW-7005 Project: Apache Airflow Issue Type: Improvement Components: breeze Affects Versions: 1.10.9, 2.0.0 Reporter: Jarek Potiuk -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (AIRFLOW-5828) Build scripts should be simplified
[ https://issues.apache.org/jira/browse/AIRFLOW-5828?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jarek Potiuk resolved AIRFLOW-5828. --- Fix Version/s: 1.10.10 Resolution: Fixed > Build scripts should be simplified > -- > > Key: AIRFLOW-5828 > URL: https://issues.apache.org/jira/browse/AIRFLOW-5828 > Project: Apache Airflow > Issue Type: Sub-task > Components: breeze >Affects Versions: 2.0.0 >Reporter: Jarek Potiuk >Assignee: Jarek Potiuk >Priority: Major > Fix For: 1.10.10 > > > Builds scripts are unncessarily complex because of hooks/builds were > initially done for Dockerhub only and then adapted to Travis CI. We can > simplify the scripts by moving everything to the scripts/ci to separate bash > functions and make hooks/build use the functions. > Separating everything to functions will make the scripts easy to maintain as > well. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (AIRFLOW-7001) Mysql 5.7 handles timezone-aware timestamps differently than 5.6
[
https://issues.apache.org/jira/browse/AIRFLOW-7001?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jarek Potiuk resolved AIRFLOW-7001.
---
Fix Version/s: 1.10.10
Resolution: Fixed
> Mysql 5.7 handles timezone-aware timestamps differently than 5.6
>
>
> Key: AIRFLOW-7001
> URL: https://issues.apache.org/jira/browse/AIRFLOW-7001
> Project: Apache Airflow
> Issue Type: Improvement
> Components: database, mysql
>Affects Versions: 2.0.0, 1.10.9
>Reporter: Jarek Potiuk
>Priority: Major
> Fix For: 1.10.10
>
>
> In Airflow when UtcDateTime is used, always Timezone is required and added if
> missing.
> For example when utcnow() function is used to get timezone we get the
> timestamp in the form of '2020-03-07 07:32:34.121705+00:00'
> When such value - with timezone - is used in MySQL 5.6 the timezone part is
> IGNORED:
>
> {code:java}
> mysql> create table test (a timestamp(6));
> Query OK, 0 rows affected (0.01 sec)
> mysql> insert into test values ('2020-03-07 07:32:34.121705+00:00');
> Query OK, 1 row affected, 1 warning (0.00 sec)
> mysql> insert into test values ('2020-03-07 07:32:34.121705+01:00');
> Query OK, 1 row affected, 1 warning (0.00 sec)
> mysql> select * from test;
> ++
> | a |
> ++
> | 2020-03-07 07:32:34.121705 |
> | 2020-03-07 07:32:34.121705 |
> ++
> 2 rows in set (0.00 sec)
> mysql> SHOW VARIABLES LIKE "%version%";
> +-+--+
> | Variable_name | Value |
> +-+--+
> | innodb_version | 5.6.47 |
> | protocol_version | 10 |
> | slave_type_conversions | |
> | version | 5.6.47 |
> | version_comment | MySQL Community Server (GPL) |
> | version_compile_machine | x86_64 |
> | version_compile_os | Linux |
> +-+--+
> 7 rows in set (0.00 sec)
> {code}
>
> The same insert in 5.7 results in error:
>
> {code:java}
> mysql> create table test(a TIMESTAMP(6));
> Query OK, 0 rows affected (0.00 sec)
> mysql> insert into test values ('2020-03-07 07:32:34.121705+01:00');
> ERROR 1292 (22007): Incorrect datetime value: '2020-03-07
> 07:32:34.121705+01:00' for column 'a' at row 1
> mysql> SHOW VARIABLES LIKE "%version%";
> +-+--+
> | Variable_name | Value |
> +-+--+
> | innodb_version | 5.7.29 |
> | protocol_version | 10 |
> | slave_type_conversions | |
> | tls_version | TLSv1,TLSv1.1,TLSv1.2 |
> | version | 5.7.29 |
> | version_comment | MySQL Community Server (GPL) |
> | version_compile_machine | x86_64 |
> | version_compile_os | Linux |
> +-+--+
> 8 rows in set (0.00 sec)
> {code}
> Seems that for MySQL - neither 5.6 (ignore timezone) nor 5.7 (crashes) works
> properly.
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Created] (AIRFLOW-7002) Get rid of Yaml "parser" in bash
Jarek Potiuk created AIRFLOW-7002: - Summary: Get rid of Yaml "parser" in bash Key: AIRFLOW-7002 URL: https://issues.apache.org/jira/browse/AIRFLOW-7002 Project: Apache Airflow Issue Type: Improvement Components: breeze, ci Affects Versions: 1.10.9, 2.0.0 Reporter: Jarek Potiuk -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (AIRFLOW-7001) Mysql 5.7 handles timezone-aware timestamps differently than 5.6
Jarek Potiuk created AIRFLOW-7001:
-
Summary: Mysql 5.7 handles timezone-aware timestamps differently
than 5.6
Key: AIRFLOW-7001
URL: https://issues.apache.org/jira/browse/AIRFLOW-7001
Project: Apache Airflow
Issue Type: Improvement
Components: database, mysql
Affects Versions: 1.10.9, 2.0.0
Reporter: Jarek Potiuk
In Airflow when UtcDateTime is used, always Timezone is required and added if
missing.
For example when utcnow() function is used to get timezone we get the timestamp
in the form of '2020-03-07 07:32:34.121705+00:00'
When such value - with timezone - is used in MySQL 5.6 the timezone part is
IGNORED:
{code:java}
mysql> create table test (a timestamp(6));
Query OK, 0 rows affected (0.01 sec)
mysql> insert into test values ('2020-03-07 07:32:34.121705+00:00');
Query OK, 1 row affected, 1 warning (0.00 sec)
mysql> insert into test values ('2020-03-07 07:32:34.121705+01:00');
Query OK, 1 row affected, 1 warning (0.00 sec)
mysql> select * from test;
++
| a |
++
| 2020-03-07 07:32:34.121705 |
| 2020-03-07 07:32:34.121705 |
++
2 rows in set (0.00 sec)
mysql> SHOW VARIABLES LIKE "%version%";
+-+--+
| Variable_name | Value |
+-+--+
| innodb_version | 5.6.47 |
| protocol_version | 10 |
| slave_type_conversions | |
| version | 5.6.47 |
| version_comment | MySQL Community Server (GPL) |
| version_compile_machine | x86_64 |
| version_compile_os | Linux |
+-+--+
7 rows in set (0.00 sec)
{code}
The same insert in 5.7 results in error:
{code:java}
mysql> create table test(a TIMESTAMP(6));
Query OK, 0 rows affected (0.00 sec)
mysql> insert into test values ('2020-03-07 07:32:34.121705+01:00');
ERROR 1292 (22007): Incorrect datetime value: '2020-03-07
07:32:34.121705+01:00' for column 'a' at row 1
mysql> SHOW VARIABLES LIKE "%version%";
+-+--+
| Variable_name | Value |
+-+--+
| innodb_version | 5.7.29 |
| protocol_version | 10 |
| slave_type_conversions | |
| tls_version | TLSv1,TLSv1.1,TLSv1.2 |
| version | 5.7.29 |
| version_comment | MySQL Community Server (GPL) |
| version_compile_machine | x86_64 |
| version_compile_os | Linux |
+-+--+
8 rows in set (0.00 sec)
{code}
Seems that for MySQL - neither 5.6 (ignore timezone) nor 5.7 (crashes) works
properly.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Created] (AIRFLOW-6998) System test is failing in Travis
Jarek Potiuk created AIRFLOW-6998: - Summary: System test is failing in Travis Key: AIRFLOW-6998 URL: https://issues.apache.org/jira/browse/AIRFLOW-6998 Project: Apache Airflow Issue Type: Improvement Components: ci Affects Versions: 2.0.0 Reporter: Jarek Potiuk -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (AIRFLOW-6973) Make GCSCreateBucketOperator idempotent
[ https://issues.apache.org/jira/browse/AIRFLOW-6973?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jarek Potiuk resolved AIRFLOW-6973. --- Fix Version/s: 2.0.0 Resolution: Fixed > Make GCSCreateBucketOperator idempotent > --- > > Key: AIRFLOW-6973 > URL: https://issues.apache.org/jira/browse/AIRFLOW-6973 > Project: Apache Airflow > Issue Type: Improvement > Components: gcp, operators >Affects Versions: 2.0.0 >Reporter: Tomasz Urbaszek >Priority: Minor > Fix For: 2.0.0 > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (AIRFLOW-6977) Fix BigQuery DTS example DAG
[ https://issues.apache.org/jira/browse/AIRFLOW-6977?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jarek Potiuk resolved AIRFLOW-6977. --- Fix Version/s: 2.0.0 Resolution: Fixed > Fix BigQuery DTS example DAG > > > Key: AIRFLOW-6977 > URL: https://issues.apache.org/jira/browse/AIRFLOW-6977 > Project: Apache Airflow > Issue Type: Improvement > Components: examples >Affects Versions: 2.0.0 >Reporter: Tomasz Urbaszek >Priority: Minor > Fix For: 2.0.0 > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (AIRFLOW-6926) Fix Google Tasks operators return types and idempotency
[ https://issues.apache.org/jira/browse/AIRFLOW-6926?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jarek Potiuk resolved AIRFLOW-6926. --- Fix Version/s: 2.0.0 Resolution: Fixed > Fix Google Tasks operators return types and idempotency > --- > > Key: AIRFLOW-6926 > URL: https://issues.apache.org/jira/browse/AIRFLOW-6926 > Project: Apache Airflow > Issue Type: Improvement > Components: gcp, operators >Affects Versions: 2.0.0 >Reporter: Tomasz Urbaszek >Priority: Major > Fix For: 2.0.0 > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (AIRFLOW-6970) Improve GCP Video Intelligence system tests
[ https://issues.apache.org/jira/browse/AIRFLOW-6970?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jarek Potiuk resolved AIRFLOW-6970. --- Fix Version/s: 2.0.0 Resolution: Fixed > Improve GCP Video Intelligence system tests > --- > > Key: AIRFLOW-6970 > URL: https://issues.apache.org/jira/browse/AIRFLOW-6970 > Project: Apache Airflow > Issue Type: Improvement > Components: gcp, tests >Affects Versions: 2.0.0 >Reporter: Tomasz Urbaszek >Priority: Minor > Fix For: 2.0.0 > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (AIRFLOW-6949) SparkSubitOperator ignores explicit spark.kubernetes.namespace config option
[
https://issues.apache.org/jira/browse/AIRFLOW-6949?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17050618#comment-17050618
]
Jarek Potiuk commented on AIRFLOW-6949:
---
I removed the cherry-picked commit as it was failing with missing conn_data:
commit f8a83a7188c83113c69b9711b9a42e69d2fc6ba7 (polidea/v1-10-test,
origin/v1-10-test)
Author: Ash Berlin-Taylor
Date: Fri Feb 28 11:14:21 2020 +
[AIRFLOW-6949] Respect explicit `spark.kubernetes.namespace` conf to
SparkSubmitOperator (#7575)
This means the value from the Operator/dag file takes precedence over
the connection
The previous behaviour was to emit one line from the conf arg, but then
a later one from the connection:
```
--conf spark.kubernetes.namespace=airflow \
--conf spark.kubernetes.namespace=default \
```
(cherry picked from commit b59042b5ab083c77ba08ba804df76b7c728815dc)
> SparkSubitOperator ignores explicit spark.kubernetes.namespace config option
>
>
> Key: AIRFLOW-6949
> URL: https://issues.apache.org/jira/browse/AIRFLOW-6949
> Project: Apache Airflow
> Issue Type: Bug
> Components: hooks
>Affects Versions: 1.10.9
>Reporter: Ash Berlin-Taylor
>Assignee: Ash Berlin-Taylor
>Priority: Minor
> Fix For: 1.10.10
>
>
> If a user explicitly passes {{spark.kubernetes.namespace}} in the config
> attribute, we should respect that over what is given in the connection.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (AIRFLOW-6979) Fix breeze test-target specify individual tests not work
[
https://issues.apache.org/jira/browse/AIRFLOW-6979?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jarek Potiuk updated AIRFLOW-6979:
--
Fix Version/s: (was: 2.0.0)
1.10.10
> Fix breeze test-target specify individual tests not work
>
>
> Key: AIRFLOW-6979
> URL: https://issues.apache.org/jira/browse/AIRFLOW-6979
> Project: Apache Airflow
> Issue Type: Bug
> Components: breeze
>Affects Versions: 1.10.9
>Reporter: zhongjiajie
>Assignee: zhongjiajie
>Priority: Major
> Fix For: 1.10.10
>
>
> We miss a test-target arg when running
> {code:java}
> ./breeze test-target tests/test_core.py::TestCore{code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Created] (AIRFLOW-6969) Move backport packages to test
Jarek Potiuk created AIRFLOW-6969: - Summary: Move backport packages to test Key: AIRFLOW-6969 URL: https://issues.apache.org/jira/browse/AIRFLOW-6969 Project: Apache Airflow Issue Type: Improvement Components: ci Affects Versions: 2.0.0 Reporter: Jarek Potiuk When one of the test builds fails even restarting it does not help to succeed the whole job if there is "post-test" stage. The post-test job will remain in cancelled state after the job is restarted We should move prepare-backport-packages to test phase (it is very fast). -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (AIRFLOW-6472) Some of the options we have are long options with single -
[
https://issues.apache.org/jira/browse/AIRFLOW-6472?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jarek Potiuk resolved AIRFLOW-6472.
---
Fix Version/s: 2.0.0
Resolution: Fixed
> Some of the options we have are long options with single -
> --
>
> Key: AIRFLOW-6472
> URL: https://issues.apache.org/jira/browse/AIRFLOW-6472
> Project: Apache Airflow
> Issue Type: Improvement
> Components: cli
>Affects Versions: 2.0.0, 1.10.7
>Reporter: Jarek Potiuk
>Assignee: zhongjiajie
>Priority: Major
> Fix For: 2.0.0
>
>
> We have some "short" options that are really "long" ones: namely -int and -sd
> in the run task. This is against the idea of short and long options in Unix
> (and argparse follows that). The main reason to have short options is that
> you can combine short options:
> {{airflow task run -iAlm}}
> When you have more than one letter "short" options this might become quickly
> ambiguous. We do not have -s yet but if we add few more options this might
> become a problem.
> Also in the argparse documentation
> [https://docs.python.org/2/library/argparse.html] it's mentioned that short
> options should be single character only.\
> This should be fixed.
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (AIRFLOW-6947) UTF8mb4 encoding for mysql does not work in Airflow 2.0
[ https://issues.apache.org/jira/browse/AIRFLOW-6947?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jarek Potiuk updated AIRFLOW-6947: -- Description: The problem is with how MySQL handles different encodings. Especially UTF8. UTF8 in Mysql - default utf8 encoding - does not handle all UTF8 characters (only those encoded in 3 bytes) - the 4-bytes one are not working (there is an error - "Incorrect string value: ' xF0' for column 'description' at row 1") when you try to insert DAG with 4-bytes character unicode. This a problem mainly with DAG description that is stored in the database. One of our customers had this very issue with it's database and there database encoding is utf8. Current utf8 behaviour - is that it is an alias to utf8mb3 [https://dev.mysql.com/doc/refman/5.7/en/charset-unicode-utf8.html] which means it does not handle all characters (mostly Emojis but also some chinese characters [https://stackoverflow.com/questions/17680237/mysql-four-byte-chinese-characters-support] ) . In some future versions of mysql - UTF8 will become alias for utf8mb4 ([https://dev.mysql.com/doc/refman/8.0/en/charset-unicode-utf8.html]) which supports full range of UTF-encoded characters. It is strongly advised to use utf8mb4 directly as default encoding. I decided to see how it works with utf8mb4 encoding and - unfortunately it turns out that in case we switch to it, migration scripts for Airflow fails because row size for at least one of the indexes exceeds maximum row size: ‘'Specified key was too long; max key length is 3072 bytes'” when XCOM key is created. ALTER TABLE xcom ADD CONSTRAINT pk_xcom PRIMARY KEY (dag_id, task_id, `key`, execution_date)] In Airflow 1.10 the primary key for an xcom was an integer and in 2.0 it is a compound index with dag_id, task_id, execution_date and key - they together make the row too big for utf8mb4 (in utf8mb4 encoding the text fields take 4x number of characters). In our CI we had so far the default mysql encoding (which for the uninitiated is latin1_swedish_ci (!), I switched it to utf8mb4 so that you can see the behaviour - and created PR here [https://github.com/apache/airflow/pull/7570] and failed test here: [https://travis-ci.org/apache/airflow/jobs/655733996?utm_medium=notification_source=github_status] Note similar problem occurs in 1.10 with MySQL 5.6 - if I change the charset to utf8mb4 and choose 5.6 mysql, it will fail because there the max key length was half the size (1536 characters). There is even an issue for it in our JIRA https://issues.apache.org/jira/browse/AIRFLOW-3786 - for different index. The workaround was to use the UTF8 (UTF8mb3) or switching to MySQL 5.7. was: The problem is with how MySQL handles different encodings. Especially UTF8. UTF8 in Mysql - default utf8 encoding - does not handle all UTF8 characters (only those encoded in 3 bytes) - the 4-bytes one are not working (there is an error - "Incorrect string value: ' xF0' for column 'description' at row 1") when you try to insert DAG with 4-bytes character unicode. This a problem mainly with DAG description that is stored in the database. One of our customers had this very issue with it's database and there database encoding is utf8. Current utf8 behaviour - is that it is an alias to utf8mb3 [https://dev.mysql.com/doc/refman/5.7/en/charset-unicode-utf8.html] which means it does not handle all characters (mostly Emojis but also some chinese characters [https://stackoverflow.com/questions/17680237/mysql-four-byte-chinese-characters-support] ) . In some future versions of mysql - UTF8 will become alias for utf8mb4 ([https://dev.mysql.com/doc/refman/8.0/en/charset-unicode-utf8.html]) which supports full range of UTF-encoded characters. It is strongly advised to use utf8mb4 directly as default encoding. I decided to see how it works with utf8mb4 encoding and - unfortunately it turns out that in case we switch to it, migration scripts for Airflow fails because row size for at least one of the indexes exceeds maximum row size: ‘'Specified key was too long; max key length is 3072 bytes'” when XCOM key is created. ALTER TABLE xcom ADD CONSTRAINT pk_xcom PRIMARY KEY (dag_id, task_id, `key`, execution_date)] In Airflow 1.10 the primary key was an integer and in 2.0 it is a compound index with dag_id, task_id, execution_date and key - they together make the row too big for utf8mb4 (in utf8mb4 encoding the text fields take 4x number of characters). In our CI we had so far the default mysql encoding (which for the uninitiated is latin1_swedish_ci (!), I switched it to utf8mb4 so that you can see the behaviour - and created PR here [https://github.com/apache/airflow/pull/7570] and failed test here: [https://travis-ci.org/apache/airflow/jobs/655733996?utm_medium=notification_source=github_status] Note similar problem occurs in 1.10 with MySQL 5.6 - if I
[jira] [Updated] (AIRFLOW-6947) UTF8mb4 encoding for mysql does not work in Airflow 2.0
[ https://issues.apache.org/jira/browse/AIRFLOW-6947?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jarek Potiuk updated AIRFLOW-6947: -- Description: The problem is with how MySQL handles different encodings. Especially UTF8. UTF8 in Mysql - default utf8 encoding - does not handle all UTF8 characters (only those encoded in 3 bytes) - the 4-bytes one are not working (there is an error - "Incorrect string value: ' xF0' for column 'description' at row 1") when you try to insert DAG with 4-bytes character unicode. This a problem mainly with DAG description that is stored in the database. One of our customers had this very issue with it's database and there database encoding is utf8. Current utf8 behaviour - is that it is an alias to utf8mb3 [https://dev.mysql.com/doc/refman/5.7/en/charset-unicode-utf8.html] which means it does not handle all characters (mostly Emojis but also some chinese characters [https://stackoverflow.com/questions/17680237/mysql-four-byte-chinese-characters-support] ) . In some future versions of mysql - UTF8 will become alias for utf8mb4 ([https://dev.mysql.com/doc/refman/8.0/en/charset-unicode-utf8.html]) which supports full range of UTF-encoded characters. It is strongly advised to use utf8mb4 directly as default encoding. I decided to see how it works with utf8mb4 encoding and - unfortunately it turns out that in case we switch to it, migration scripts for Airflow fails because row size for at least one of the indexes exceeds maximum row size: ‘'Specified key was too long; max key length is 3072 bytes'” when XCOM key is created. ALTER TABLE xcom ADD CONSTRAINT pk_xcom PRIMARY KEY (dag_id, task_id, `key`, execution_date)] In Airflow 1.10 the primary key was an integer and in 2.0 it is a compound index with dag_id, task_id, execution_date and key - they together make the row too big for utf8mb4 (in utf8mb4 encoding the text fields take 4x number of characters). In our CI we had so far the default mysql encoding (which for the uninitiated is latin1_swedish_ci (!), I switched it to utf8mb4 so that you can see the behaviour - and created PR here [https://github.com/apache/airflow/pull/7570] and failed test here: [https://travis-ci.org/apache/airflow/jobs/655733996?utm_medium=notification_source=github_status] Note similar problem occurs in 1.10 with MySQL 5.6 - if I change the charset to utf8mb4 and choose 5.6 mysql, it will fail because there the max key length was half the size (1536 characters). There is even an issue for it in our JIRA https://issues.apache.org/jira/browse/AIRFLOW-3786 - for different index. The workaround was to use the UTF8 (UTF8mb3) or switching to MySQL 5.7. was: The problem is with how MySQL handles different encodings. Especially UTF8. UTF8 in Mysql - default utf8 encoding - does not handle all UTF8 characters (only those encoded in 3 bytes) - the 4-bytes one are not working (there is an error - "Incorrect string value: ' xF0' for column 'description' at row 1") when you try to insert DAG with 4-bytes character unicode. This a problem mainly with DAG description that is stored in the database. One of our customers had this very issue with it's database and there database encoding is utf8. Current utf8 behaviour - is that it is an alias to utf8mb3 [https://dev.mysql.com/doc/refman/5.7/en/charset-unicode-utf8.html] which means it does not handle all characters (mostly Emojis but also some chinese characters [https://stackoverflow.com/questions/17680237/mysql-four-byte-chinese-characters-support] ) . In some future versions of mysql - UTF8 will become alias for utf8mb4 ([https://dev.mysql.com/doc/refman/8.0/en/charset-unicode-utf8.html]) which supports full range of UTF-encoded characters. It is strongly advised to use utf8mb4 directly as default encoding. I decided to see how it works with utf8mb4 encoding and - unfortunately it turns out that in case we switch to it, migration scripts for Airflow fails because row size for at least one of the indexes exceeds maximum row size: ‘'Specified key was too long; max key length is 3072 bytes'” when XCOM key is created. ALTER TABLE xcom ADD CONSTRAINT pk_xcom PRIMARY KEY (dag_id, task_id, `key`, execution_date)] Apparently increased size of some columns (key?) make the row too big for utf8mb4 (in utf8mb4 encoding the text fields take 4x number of characters). In our CI we had so far the default mysql encoding (which for the uninitiated is latin1_swedish_ci (!), I switched it to utf8mb4 so that you can see the behaviour - and created PR here [https://github.com/apache/airflow/pull/7570] and failed test here: [https://travis-ci.org/apache/airflow/jobs/655733996?utm_medium=notification_source=github_status] Note similar problem occurs in 1.10 with MySQL 5.6 - if I change the charset to utf8mb4 and choose 5.6 mysql, it will fail because there the max key length was half
[jira] [Updated] (AIRFLOW-6947) UTF8mb4 encoding for mysql does not work in Airflow 2.0
[ https://issues.apache.org/jira/browse/AIRFLOW-6947?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jarek Potiuk updated AIRFLOW-6947: -- Description: The problem is with how MySQL handles different encodings. Especially UTF8. UTF8 in Mysql - default utf8 encoding - does not handle all UTF8 characters (only those encoded in 3 bytes) - the 4-bytes one are not working (there is an error - "Incorrect string value: ' xF0' for column 'description' at row 1") when you try to insert DAG with 4-bytes character unicode. This a problem mainly with DAG description that is stored in the database. One of our customers had this very issue with it's database and there database encoding is utf8. Current utf8 behaviour - is that it is an alias to utf8mb3 [https://dev.mysql.com/doc/refman/5.7/en/charset-unicode-utf8.html] which means it does not handle all characters (mostly Emojis but also some chinese characters [https://stackoverflow.com/questions/17680237/mysql-four-byte-chinese-characters-support] ) . In some future versions of mysql - UTF8 will become alias for utf8mb4 ([https://dev.mysql.com/doc/refman/8.0/en/charset-unicode-utf8.html]) which supports full range of UTF-encoded characters. It is strongly advised to use utf8mb4 directly as default encoding. I decided to see how it works with utf8mb4 encoding and - unfortunately it turns out that in case we switch to it, migration scripts for Airflow fails because row size for at least one of the indexes exceeds maximum row size: ‘'Specified key was too long; max key length is 3072 bytes'” when XCOM key is created. ALTER TABLE xcom ADD CONSTRAINT pk_xcom PRIMARY KEY (dag_id, task_id, `key`, execution_date)] Apparently increased size of some columns (key?) make the row too big for utf8mb4 (in utf8mb4 encoding the text fields take 4x number of characters). In our CI we had so far the default mysql encoding (which for the uninitiated is latin1_swedish_ci (!), I switched it to utf8mb4 so that you can see the behaviour - and created PR here [https://github.com/apache/airflow/pull/7570] and failed test here: [https://travis-ci.org/apache/airflow/jobs/655733996?utm_medium=notification_source=github_status] Note similar problem occurs in 1.10 with MySQL 5.6 - if I change the charset to utf8mb4 and choose 5.6 mysql, it will fail because there the max key length was half the size (1536 characters). There is even an issue for it in our JIRA https://issues.apache.org/jira/browse/AIRFLOW-3786. The workaround was to use the UTF8 (UTF8mb3) or switching to MySQL 5.7. was: The problem is with how MySQL handles different encodings. Especially UTF8. UTF8 in Mysql - default utf8 encoding - does not handle all UTF8 characters (only those encoded in 3 bytes) - the 4-bytes one are not working (there is an error - "Incorrect string value: '\\xF0' for column 'description' at row 1") when you try to insert DAG with 4-bytes character unicode. This a problem for example with DAG description that is stored in the database. One of our customers had this very issue with it's database and there database encoding is utf8. Current utf8 behaviour - is that it is an alias to utf8mb3 https://dev.mysql.com/doc/refman/5.7/en/charset-unicode-utf8.html which means it does not handle all characters (mostly Emojis) . In some future versions of mysql - UTF8 will become alias for utf8mb4 (https://dev.mysql.com/doc/refman/8.0/en/charset-unicode-utf8.html) which supports full range of UTF-encoded characters. It is strongly advised to use utf8mb4 directly as default encoding. I decided to see how it works with utf8mb4 encoding and - unfortunately it turns out that in case we switch to it, migration scripts for Airflow fails because row size for at least one of the indexes exceeds maximum row size: ‘'Specified key was too long; max key length is 3072 bytes'” when XCOM key is created. ALTER TABLE xcom ADD CONSTRAINT pk_xcom PRIMARY KEY (dag_id, task_id, `key`, execution_date)] Apparently increased size of some columns (key?) make the row too big for utf8mb4 (in utf8mb4 encoding the text fields take 4x number of characters). In our CI we had so far the default mysql encoding (which for the uninitiated is latin1_swedish_ci (!), I switched it to utf8mb4 so that you can see the behaviour - and created PR here https://github.com/apache/airflow/pull/7570 and failed test here: https://travis-ci.org/apache/airflow/jobs/655733996?utm_medium=notification_source=github_status Note similar problem occurs in 1.10 with MySQL 5.6 - if I change the charset to utf8mb4 and choose 5.6 mysql, it will fail because there the max key length was half the size (1536 characters). There is even an issue for it in our JIRA https://issues.apache.org/jira/browse/AIRFLOW-3786. The workaround was to use the UTF8 (UTF8mb3) or switching to MySQL 5.7. > UTF8mb4 encoding for mysql does not work in
[jira] [Created] (AIRFLOW-6947) UTF8mb4 encoding for mysql does not work in Airflow 2.0
Jarek Potiuk created AIRFLOW-6947: - Summary: UTF8mb4 encoding for mysql does not work in Airflow 2.0 Key: AIRFLOW-6947 URL: https://issues.apache.org/jira/browse/AIRFLOW-6947 Project: Apache Airflow Issue Type: Improvement Components: mysql, database Affects Versions: 2.0.0 Reporter: Jarek Potiuk The problem is with how MySQL handles different encodings. Especially UTF8. UTF8 in Mysql - default utf8 encoding - does not handle all UTF8 characters (only those encoded in 3 bytes) - the 4-bytes one are not working (there is an error - "Incorrect string value: '\\xF0' for column 'description' at row 1") when you try to insert DAG with 4-bytes character unicode. This a problem for example with DAG description that is stored in the database. One of our customers had this very issue with it's database and there database encoding is utf8. Current utf8 behaviour - is that it is an alias to utf8mb3 https://dev.mysql.com/doc/refman/5.7/en/charset-unicode-utf8.html which means it does not handle all characters (mostly Emojis) . In some future versions of mysql - UTF8 will become alias for utf8mb4 (https://dev.mysql.com/doc/refman/8.0/en/charset-unicode-utf8.html) which supports full range of UTF-encoded characters. It is strongly advised to use utf8mb4 directly as default encoding. I decided to see how it works with utf8mb4 encoding and - unfortunately it turns out that in case we switch to it, migration scripts for Airflow fails because row size for at least one of the indexes exceeds maximum row size: ‘'Specified key was too long; max key length is 3072 bytes'” when XCOM key is created. ALTER TABLE xcom ADD CONSTRAINT pk_xcom PRIMARY KEY (dag_id, task_id, `key`, execution_date)] Apparently increased size of some columns (key?) make the row too big for utf8mb4 (in utf8mb4 encoding the text fields take 4x number of characters). In our CI we had so far the default mysql encoding (which for the uninitiated is latin1_swedish_ci (!), I switched it to utf8mb4 so that you can see the behaviour - and created PR here https://github.com/apache/airflow/pull/7570 and failed test here: https://travis-ci.org/apache/airflow/jobs/655733996?utm_medium=notification_source=github_status Note similar problem occurs in 1.10 with MySQL 5.6 - if I change the charset to utf8mb4 and choose 5.6 mysql, it will fail because there the max key length was half the size (1536 characters). There is even an issue for it in our JIRA https://issues.apache.org/jira/browse/AIRFLOW-3786. The workaround was to use the UTF8 (UTF8mb3) or switching to MySQL 5.7. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (AIRFLOW-6946) Switch to MySQL 5.7 in 2.0 as base
[ https://issues.apache.org/jira/browse/AIRFLOW-6946?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jarek Potiuk updated AIRFLOW-6946: -- Description: Switch to MySQL 5.7 in tests. Also test utf8mb4 encoding was:Switch to MySQL 5.7 in tests. > Switch to MySQL 5.7 in 2.0 as base > -- > > Key: AIRFLOW-6946 > URL: https://issues.apache.org/jira/browse/AIRFLOW-6946 > Project: Apache Airflow > Issue Type: Improvement > Components: ci >Affects Versions: 2.0.0 >Reporter: Jarek Potiuk >Priority: Major > > Switch to MySQL 5.7 in tests. > Also test utf8mb4 encoding -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (AIRFLOW-6946) Switch to MySQL 5.7 in 2.0 as base
Jarek Potiuk created AIRFLOW-6946: - Summary: Switch to MySQL 5.7 in 2.0 as base Key: AIRFLOW-6946 URL: https://issues.apache.org/jira/browse/AIRFLOW-6946 Project: Apache Airflow Issue Type: Improvement Components: ci Affects Versions: 2.0.0 Reporter: Jarek Potiuk Switch to MySQL 5.7 in tests. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (AIRFLOW-6945) Switch to MySQL 5.7 for airflow 1.10
Jarek Potiuk created AIRFLOW-6945: - Summary: Switch to MySQL 5.7 for airflow 1.10 Key: AIRFLOW-6945 URL: https://issues.apache.org/jira/browse/AIRFLOW-6945 Project: Apache Airflow Issue Type: Improvement Components: ci Affects Versions: 1.10.9 Reporter: Jarek Potiuk We decided to switch to 5.7 as default as it is already 4 years old. 5.6 is 7 years old. We are also testing that utf8mb4 db encoding works for this version. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (AIRFLOW-6932) Add restart-environment command to Breeze
[ https://issues.apache.org/jira/browse/AIRFLOW-6932?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jarek Potiuk resolved AIRFLOW-6932. --- Fix Version/s: 1.10.9 Resolution: Fixed > Add restart-environment command to Breeze > - > > Key: AIRFLOW-6932 > URL: https://issues.apache.org/jira/browse/AIRFLOW-6932 > Project: Apache Airflow > Issue Type: Bug > Components: breeze >Affects Versions: 2.0.0, 1.10.9 >Reporter: Jarek Potiuk >Priority: Major > Fix For: 1.10.9 > > > When you switch between versions of Aiflow installed, you want to delete the > database so that the scripts for resetdb work -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (AIRFLOW-6943) Dag view fails to render when DAG description contains utf-8 character
Jarek Potiuk created AIRFLOW-6943: - Summary: Dag view fails to render when DAG description contains utf-8 character Key: AIRFLOW-6943 URL: https://issues.apache.org/jira/browse/AIRFLOW-6943 Project: Apache Airflow Issue Type: Bug Components: ui Affects Versions: 1.10.9 Reporter: Jarek Potiuk For python 2 airflow when DAG description contains UTF-8 character UI produces mushroom cloud with errors that it can't decode the string: For example if you put 礪 in the description you get: UnicodeDecodeError: 'ascii' codec can't decode byte 0xf0 in position 0: ordinal not in range(128) -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (AIRFLOW-3786) mysql initdb failes because the primary key is too large.
[ https://issues.apache.org/jira/browse/AIRFLOW-3786?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17046267#comment-17046267 ] Jarek Potiuk commented on AIRFLOW-3786: --- NOTE! For Airflow 1.10 this problem does not exist in MySQL 5.7 . Maximum size of the index has been increased for mysql 5.7 and is enough to handle the indexes. > mysql initdb failes because the primary key is too large. > -- > > Key: AIRFLOW-3786 > URL: https://issues.apache.org/jira/browse/AIRFLOW-3786 > Project: Apache Airflow > Issue Type: Bug > Components: database >Affects Versions: 1.10.2 >Reporter: Thayne McCombs >Priority: Major > > When running `airflow initdb` using a mysql server using the utf8mb character > set, I get this error: > ``` > sqlalchemy.exc.OperationalError: (MySQLdb._exceptions.OperationalError) > (1071, 'Specified key was too long; max key length is 767 bytes') [SQL: > '\nCREATE TABLE dag (\n\tdag_id VARCHAR(250) NOT NULL, \n\tis_paused BOOL, > \n\tis_subdag BOOL, \n\tis_active BOOL, \n\tlast_scheduler_run DATETIME, > \n\tlast_pickled DATETIME, \n\tlast_expired DATETIME, \n\tscheduler_lock > BOOL, \n\tpickle_id INTEGER, \n\tfileloc VARCHAR(2000), \n\towners > VARCHAR(2000), \n\tPRIMARY KEY (dag_id), \n\tCHECK (is_paused IN (0, 1)), > \n\tCHECK (is_subdag IN (0, 1)), \n\tCHECK (is_active IN (0, 1)), \n\tCHECK > (scheduler_lock IN (0, 1))\n)\n\n'] (Background on this error at: > [http://sqlalche.me/e/e3q8)] > ``` > I've found a few stack overflow questions from other users that have run into > this problem, the workarounds given are to either enable innodb_large_prefix, > (which I can't do) or use utf8 or ascii encoding for the database (not > desirable). Ideally, this should just work, or at the very least, have well > documented workarounds for this problem. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (AIRFLOW-6932) Add restart-environment command to Breeze
[ https://issues.apache.org/jira/browse/AIRFLOW-6932?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jarek Potiuk updated AIRFLOW-6932: -- Description: When you switch between versions of Aiflow installed, you want to delete the database so that the scripts for resetdb work > Add restart-environment command to Breeze > - > > Key: AIRFLOW-6932 > URL: https://issues.apache.org/jira/browse/AIRFLOW-6932 > Project: Apache Airflow > Issue Type: Bug > Components: breeze >Affects Versions: 2.0.0, 1.10.9 >Reporter: Jarek Potiuk >Priority: Major > > When you switch between versions of Aiflow installed, you want to delete the > database so that the scripts for resetdb work -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (AIRFLOW-6932) Add restart-environment command to Breeze
Jarek Potiuk created AIRFLOW-6932: - Summary: Add restart-environment command to Breeze Key: AIRFLOW-6932 URL: https://issues.apache.org/jira/browse/AIRFLOW-6932 Project: Apache Airflow Issue Type: Bug Components: breeze Affects Versions: 1.10.9, 2.0.0 Reporter: Jarek Potiuk -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (AIRFLOW-6763) Make system test ready for backport tests
[ https://issues.apache.org/jira/browse/AIRFLOW-6763?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jarek Potiuk updated AIRFLOW-6763: -- Fix Version/s: (was: 2.0.0) 1.10.10 > Make system test ready for backport tests > - > > Key: AIRFLOW-6763 > URL: https://issues.apache.org/jira/browse/AIRFLOW-6763 > Project: Apache Airflow > Issue Type: Bug > Components: ci >Affects Versions: 2.0.0 >Reporter: Jarek Potiuk >Priority: Major > Fix For: 1.10.10 > > > We will run system test on back-ported operators for 1.10* series of airflow > and for that we need to have support for running system tests using pytest's > markers and reading environment variables passed from HOST machine (to pass > credentials). > This is the first step to automate system test execution. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (AIRFLOW-6866) export EXTRA_DC_OPTIONS+= fails on mac
[ https://issues.apache.org/jira/browse/AIRFLOW-6866?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jarek Potiuk updated AIRFLOW-6866: -- Fix Version/s: (was: 2.0.0) 1.10.10 > export EXTRA_DC_OPTIONS+= fails on mac > -- > > Key: AIRFLOW-6866 > URL: https://issues.apache.org/jira/browse/AIRFLOW-6866 > Project: Apache Airflow > Issue Type: Bug > Components: breeze >Affects Versions: 2.0.0, 1.10.9 >Reporter: Jarek Potiuk >Priority: Major > Fix For: 1.10.10 > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (AIRFLOW-6919) Make Breeeze more Dag-test friendly
Jarek Potiuk created AIRFLOW-6919: - Summary: Make Breeeze more Dag-test friendly Key: AIRFLOW-6919 URL: https://issues.apache.org/jira/browse/AIRFLOW-6919 Project: Apache Airflow Issue Type: Bug Components: breeze Affects Versions: 1.10.9, 2.0.0 Reporter: Jarek Potiuk Originally Breeze was used to run unit and integration tests, recently system tests and finally we make it a bit more friendly to test your DAGs there. You can now install any older airflow version in Breeze via --install-airflow-version switch and "files/dags" folder is mounted to "/files/dags" and this folder is used to read the dags from. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (AIRFLOW-6906) Flaky test_kubernetes_executor.py
[ https://issues.apache.org/jira/browse/AIRFLOW-6906?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jarek Potiuk resolved AIRFLOW-6906. --- Fix Version/s: 2.0.0 Resolution: Fixed > Flaky test_kubernetes_executor.py > -- > > Key: AIRFLOW-6906 > URL: https://issues.apache.org/jira/browse/AIRFLOW-6906 > Project: Apache Airflow > Issue Type: Bug > Components: ci >Affects Versions: 2.0.0 >Reporter: Jarek Potiuk >Priority: Major > Fix For: 2.0.0 > > > Kubernetes executors are flaky - seems this is a race where (quite often) > monitor_task gets 404 instead of "connection error" or success. It seems we > should treat 404 same way as connection error as it indicates that the > webserver is running but the task was not yet submitted to the DB -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (AIRFLOW-6838) Introduce real subcommands for Breeze
[ https://issues.apache.org/jira/browse/AIRFLOW-6838?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jarek Potiuk resolved AIRFLOW-6838. --- Fix Version/s: 1.10.10 Resolution: Fixed > Introduce real subcommands for Breeze > - > > Key: AIRFLOW-6838 > URL: https://issues.apache.org/jira/browse/AIRFLOW-6838 > Project: Apache Airflow > Issue Type: Bug > Components: core >Affects Versions: 2.0.0, 1.10.9 >Reporter: Kamil Bregula >Priority: Major > Fix For: 1.10.10 > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (AIRFLOW-6838) Introduce real subcommands for Breeze
[ https://issues.apache.org/jira/browse/AIRFLOW-6838?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jarek Potiuk updated AIRFLOW-6838: -- Affects Version/s: 2.0.0 > Introduce real subcommands for Breeze > - > > Key: AIRFLOW-6838 > URL: https://issues.apache.org/jira/browse/AIRFLOW-6838 > Project: Apache Airflow > Issue Type: Bug > Components: core >Affects Versions: 2.0.0, 1.10.9 >Reporter: Kamil Bregula >Priority: Major > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (AIRFLOW-6873) mypyd is failing again with new type errors
[ https://issues.apache.org/jira/browse/AIRFLOW-6873?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jarek Potiuk resolved AIRFLOW-6873. --- Fix Version/s: 2.0.0 Resolution: Fixed > mypyd is failing again with new type errors > --- > > Key: AIRFLOW-6873 > URL: https://issues.apache.org/jira/browse/AIRFLOW-6873 > Project: Apache Airflow > Issue Type: Improvement > Components: build >Affects Versions: 2.0.0 >Reporter: QP Hou >Assignee: QP Hou >Priority: Trivial > Fix For: 2.0.0 > > > New version of mypy is throwing new type errors, we should get that fixed. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (AIRFLOW-6906) Flaky test_kubernetes_executor.py
Jarek Potiuk created AIRFLOW-6906: - Summary: Flaky test_kubernetes_executor.py Key: AIRFLOW-6906 URL: https://issues.apache.org/jira/browse/AIRFLOW-6906 Project: Apache Airflow Issue Type: Bug Components: ci Affects Versions: 2.0.0 Reporter: Jarek Potiuk Kubernetes executors are flaky - seems this is a race where (quite often) monitor_task gets 404 instead of "connection error" or success. It seems we should treat 404 same way as connection error as it indicates that the webserver is running but the task was not yet submitted to the DB -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (AIRFLOW-6904) Airflow 1.10.9 suppresses Operator logs
[ https://issues.apache.org/jira/browse/AIRFLOW-6904?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jarek Potiuk updated AIRFLOW-6904: -- Affects Version/s: (was: 1.10.7) > Airflow 1.10.9 suppresses Operator logs > --- > > Key: AIRFLOW-6904 > URL: https://issues.apache.org/jira/browse/AIRFLOW-6904 > Project: Apache Airflow > Issue Type: Bug > Components: core >Affects Versions: 1.10.8, 1.10.9 >Reporter: RAHUL JAIN >Priority: Critical > Attachments: 1.10.2.png, 1.10.9.png > > > After upgrading from 1.10.2 to 1.10.9, we noticed that the Operator logs are > no longer printed. See the attachments for comparison. There is also a slack > channel discussion pointing to a recen change that may have broken this - > [https://apache-airflow.slack.com/archives/CCQ7EGB1P/p1582548602014200] > > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (AIRFLOW-6904) Airflow 1.10.9 suppresses Operator logs
[ https://issues.apache.org/jira/browse/AIRFLOW-6904?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jarek Potiuk updated AIRFLOW-6904: -- Affects Version/s: 1.10.7 > Airflow 1.10.9 suppresses Operator logs > --- > > Key: AIRFLOW-6904 > URL: https://issues.apache.org/jira/browse/AIRFLOW-6904 > Project: Apache Airflow > Issue Type: Bug > Components: core >Affects Versions: 1.10.7, 1.10.8, 1.10.9 >Reporter: RAHUL JAIN >Priority: Critical > Attachments: 1.10.2.png, 1.10.9.png > > > After upgrading from 1.10.2 to 1.10.9, we noticed that the Operator logs are > no longer printed. See the attachments for comparison. There is also a slack > channel discussion pointing to a recen change that may have broken this - > [https://apache-airflow.slack.com/archives/CCQ7EGB1P/p1582548602014200] > > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (AIRFLOW-5986) Change import paths for "samba" modules
[ https://issues.apache.org/jira/browse/AIRFLOW-5986?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jarek Potiuk resolved AIRFLOW-5986. --- Fix Version/s: 2.0.0 Resolution: Duplicate Those all issues have been resolved by a number of other JIRA issues: AIRFLOW-6644 AIRFLOW-6646 AIRFLOW-6656 AIRFLOW-6674 AIRFLOW-6689 ARIFLOW-6763 > Change import paths for "samba" modules > --- > > Key: AIRFLOW-5986 > URL: https://issues.apache.org/jira/browse/AIRFLOW-5986 > Project: Apache Airflow > Issue Type: Sub-task > Components: operators >Affects Versions: 2.0.0 >Reporter: Jarek Potiuk >Priority: Major > Fix For: 2.0.0 > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (AIRFLOW-6674) Move example_dags in accordance with AIP-21
[ https://issues.apache.org/jira/browse/AIRFLOW-6674?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jarek Potiuk resolved AIRFLOW-6674. --- Fix Version/s: 2.0.0 Resolution: Fixed > Move example_dags in accordance with AIP-21 > --- > > Key: AIRFLOW-6674 > URL: https://issues.apache.org/jira/browse/AIRFLOW-6674 > Project: Apache Airflow > Issue Type: Improvement > Components: hooks, operators >Affects Versions: 1.10.7 >Reporter: Kamil Bregula >Priority: Major > Fix For: 2.0.0 > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (AIRFLOW-5949) Change import paths for "microsoft" modules
[ https://issues.apache.org/jira/browse/AIRFLOW-5949?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jarek Potiuk resolved AIRFLOW-5949. --- Fix Version/s: 2.0.0 Resolution: Duplicate Those all issues have been resolved by a number of other JIRA issues: AIRFLOW-6644 AIRFLOW-6646 AIRFLOW-6656 AIRFLOW-6674 AIRFLOW-6689 ARIFLOW-6763 > Change import paths for "microsoft" modules > --- > > Key: AIRFLOW-5949 > URL: https://issues.apache.org/jira/browse/AIRFLOW-5949 > Project: Apache Airflow > Issue Type: Sub-task > Components: operators >Affects Versions: 2.0.0 >Reporter: Jarek Potiuk >Priority: Major > Fix For: 2.0.0 > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (AIRFLOW-5985) Change import paths for "redis" modules
[ https://issues.apache.org/jira/browse/AIRFLOW-5985?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jarek Potiuk resolved AIRFLOW-5985. --- Fix Version/s: 2.0.0 Resolution: Duplicate Those all issues have been resolved by a number of other JIRA issues: AIRFLOW-6644 AIRFLOW-6646 AIRFLOW-6656 AIRFLOW-6674 AIRFLOW-6689 ARIFLOW-6763 > Change import paths for "redis" modules > --- > > Key: AIRFLOW-5985 > URL: https://issues.apache.org/jira/browse/AIRFLOW-5985 > Project: Apache Airflow > Issue Type: Sub-task > Components: operators >Affects Versions: 2.0.0 >Reporter: Jarek Potiuk >Priority: Major > Fix For: 2.0.0 > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (AIRFLOW-5995) Change import paths for "grpc" modules
[ https://issues.apache.org/jira/browse/AIRFLOW-5995?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jarek Potiuk resolved AIRFLOW-5995. --- Fix Version/s: 2.0.0 Resolution: Duplicate Those all issues have been resolved by a number of other JIRA issues: AIRFLOW-6644 AIRFLOW-6646 AIRFLOW-6656 AIRFLOW-6674 AIRFLOW-6689 ARIFLOW-6763 > Change import paths for "grpc" modules > -- > > Key: AIRFLOW-5995 > URL: https://issues.apache.org/jira/browse/AIRFLOW-5995 > Project: Apache Airflow > Issue Type: Sub-task > Components: operators >Affects Versions: 2.0.0 >Reporter: Jarek Potiuk >Priority: Major > Fix For: 2.0.0 > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (AIRFLOW-5968) Change import paths for "salesforce" modules
[ https://issues.apache.org/jira/browse/AIRFLOW-5968?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jarek Potiuk resolved AIRFLOW-5968. --- Fix Version/s: 2.0.0 Resolution: Duplicate Those all issues have been resolved by a number of other JIRA issues: AIRFLOW-6644 AIRFLOW-6646 AIRFLOW-6656 AIRFLOW-6674 AIRFLOW-6689 ARIFLOW-6763 > Change import paths for "salesforce" modules > > > Key: AIRFLOW-5968 > URL: https://issues.apache.org/jira/browse/AIRFLOW-5968 > Project: Apache Airflow > Issue Type: Sub-task > Components: operators >Affects Versions: 2.0.0 >Reporter: Jarek Potiuk >Priority: Major > Fix For: 2.0.0 > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (AIRFLOW-5951) Change import paths for "apache/druid" modules
[ https://issues.apache.org/jira/browse/AIRFLOW-5951?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jarek Potiuk resolved AIRFLOW-5951. --- Fix Version/s: 2.0.0 Resolution: Duplicate Those all issues have been resolved by a number of other JIRA issues: AIRFLOW-6644 AIRFLOW-6646 AIRFLOW-6656 AIRFLOW-6674 AIRFLOW-6689 ARIFLOW-6763 > Change import paths for "apache/druid" modules > -- > > Key: AIRFLOW-5951 > URL: https://issues.apache.org/jira/browse/AIRFLOW-5951 > Project: Apache Airflow > Issue Type: Sub-task > Components: operators >Affects Versions: 2.0.0 >Reporter: Jarek Potiuk >Priority: Major > Fix For: 2.0.0 > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (AIRFLOW-5954) Change import paths for "apache/pig" modules
[ https://issues.apache.org/jira/browse/AIRFLOW-5954?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jarek Potiuk resolved AIRFLOW-5954. --- Fix Version/s: 2.0.0 Resolution: Duplicate Those all issues have been resolved by a number of other JIRA issues: AIRFLOW-6644 AIRFLOW-6646 AIRFLOW-6656 AIRFLOW-6674 AIRFLOW-6689 ARIFLOW-6763 > Change import paths for "apache/pig" modules > > > Key: AIRFLOW-5954 > URL: https://issues.apache.org/jira/browse/AIRFLOW-5954 > Project: Apache Airflow > Issue Type: Sub-task > Components: operators >Affects Versions: 2.0.0 >Reporter: Jarek Potiuk >Priority: Major > Fix For: 2.0.0 > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (AIRFLOW-5952) Change import paths for "apache/hadoop" modules
[ https://issues.apache.org/jira/browse/AIRFLOW-5952?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jarek Potiuk resolved AIRFLOW-5952. --- Fix Version/s: 2.0.0 Resolution: Duplicate Those all issues have been resolved by a number of other JIRA issues: AIRFLOW-6644 AIRFLOW-6646 AIRFLOW-6656 AIRFLOW-6674 AIRFLOW-6689 ARIFLOW-6763 > Change import paths for "apache/hadoop" modules > --- > > Key: AIRFLOW-5952 > URL: https://issues.apache.org/jira/browse/AIRFLOW-5952 > Project: Apache Airflow > Issue Type: Sub-task > Components: operators >Affects Versions: 2.0.0 >Reporter: Jarek Potiuk >Priority: Major > Fix For: 2.0.0 > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (AIRFLOW-5961) Change import paths for "datadog" modules
[ https://issues.apache.org/jira/browse/AIRFLOW-5961?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jarek Potiuk resolved AIRFLOW-5961. --- Fix Version/s: 2.0.0 Resolution: Duplicate Those all issues have been resolved by a number of other JIRA issues: AIRFLOW-6644 AIRFLOW-6646 AIRFLOW-6656 AIRFLOW-6674 AIRFLOW-6689 ARIFLOW-6763 > Change import paths for "datadog" modules > - > > Key: AIRFLOW-5961 > URL: https://issues.apache.org/jira/browse/AIRFLOW-5961 > Project: Apache Airflow > Issue Type: Sub-task > Components: operators >Affects Versions: 2.0.0 >Reporter: Jarek Potiuk >Priority: Major > Fix For: 2.0.0 > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (AIRFLOW-5972) Change import paths for "vertica" modules
[ https://issues.apache.org/jira/browse/AIRFLOW-5972?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jarek Potiuk resolved AIRFLOW-5972. --- Fix Version/s: 2.0.0 Resolution: Duplicate Those all issues have been resolved by a number of other JIRA issues: AIRFLOW-6644 AIRFLOW-6646 AIRFLOW-6656 AIRFLOW-6674 AIRFLOW-6689 ARIFLOW-6763 > Change import paths for "vertica" modules > - > > Key: AIRFLOW-5972 > URL: https://issues.apache.org/jira/browse/AIRFLOW-5972 > Project: Apache Airflow > Issue Type: Sub-task > Components: operators >Affects Versions: 2.0.0 >Reporter: Jarek Potiuk >Priority: Major > Fix For: 2.0.0 > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (AIRFLOW-5956) Change import paths for "apache/spark" modules
[ https://issues.apache.org/jira/browse/AIRFLOW-5956?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jarek Potiuk resolved AIRFLOW-5956. --- Fix Version/s: 2.0.0 Resolution: Duplicate Those all issues have been resolved by a number of other JIRA issues: AIRFLOW-6644 AIRFLOW-6646 AIRFLOW-6656 AIRFLOW-6674 AIRFLOW-6689 ARIFLOW-6763 > Change import paths for "apache/spark" modules > -- > > Key: AIRFLOW-5956 > URL: https://issues.apache.org/jira/browse/AIRFLOW-5956 > Project: Apache Airflow > Issue Type: Sub-task > Components: operators >Affects Versions: 2.0.0 >Reporter: Jarek Potiuk >Priority: Major > Fix For: 2.0.0 > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (AIRFLOW-5969) Change import paths for "segment" modules
[ https://issues.apache.org/jira/browse/AIRFLOW-5969?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jarek Potiuk resolved AIRFLOW-5969. --- Fix Version/s: 2.0.0 Resolution: Duplicate Those all issues have been resolved by a number of other JIRA issues: AIRFLOW-6644 AIRFLOW-6646 AIRFLOW-6656 AIRFLOW-6674 AIRFLOW-6689 ARIFLOW-6763 > Change import paths for "segment" modules > - > > Key: AIRFLOW-5969 > URL: https://issues.apache.org/jira/browse/AIRFLOW-5969 > Project: Apache Airflow > Issue Type: Sub-task > Components: operators >Affects Versions: 2.0.0 >Reporter: Jarek Potiuk >Priority: Major > Fix For: 2.0.0 > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (AIRFLOW-5984) Change import paths for "presto" modules
[ https://issues.apache.org/jira/browse/AIRFLOW-5984?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jarek Potiuk resolved AIRFLOW-5984. --- Fix Version/s: 2.0.0 Resolution: Duplicate Those all issues have been resolved by a number of other JIRA issues: AIRFLOW-6644 AIRFLOW-6646 AIRFLOW-6656 AIRFLOW-6674 AIRFLOW-6689 ARIFLOW-6763 > Change import paths for "presto" modules > > > Key: AIRFLOW-5984 > URL: https://issues.apache.org/jira/browse/AIRFLOW-5984 > Project: Apache Airflow > Issue Type: Sub-task > Components: operators >Affects Versions: 2.0.0 >Reporter: Jarek Potiuk >Priority: Major > Fix For: 2.0.0 > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (AIRFLOW-5974) Change import paths for "celery" modules
[ https://issues.apache.org/jira/browse/AIRFLOW-5974?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jarek Potiuk resolved AIRFLOW-5974. --- Fix Version/s: 2.0.0 Resolution: Duplicate Those all issues have been resolved by a number of other JIRA issues: AIRFLOW-6644 AIRFLOW-6646 AIRFLOW-6656 AIRFLOW-6674 AIRFLOW-6689 ARIFLOW-6763 > Change import paths for "celery" modules > > > Key: AIRFLOW-5974 > URL: https://issues.apache.org/jira/browse/AIRFLOW-5974 > Project: Apache Airflow > Issue Type: Sub-task > Components: operators >Affects Versions: 2.0.0 >Reporter: Jarek Potiuk >Priority: Major > Fix For: 2.0.0 > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (AIRFLOW-5980) Change import paths for "openfaas" modules
[ https://issues.apache.org/jira/browse/AIRFLOW-5980?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jarek Potiuk resolved AIRFLOW-5980. --- Fix Version/s: 2.0.0 Resolution: Duplicate Those all issues have been resolved by a number of other JIRA issues: AIRFLOW-6644 AIRFLOW-6646 AIRFLOW-6656 AIRFLOW-6674 AIRFLOW-6689 ARIFLOW-6763 > Change import paths for "openfaas" modules > -- > > Key: AIRFLOW-5980 > URL: https://issues.apache.org/jira/browse/AIRFLOW-5980 > Project: Apache Airflow > Issue Type: Sub-task > Components: operators >Affects Versions: 2.0.0 >Reporter: Jarek Potiuk >Priority: Major > Fix For: 2.0.0 > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (AIRFLOW-5981) Change import paths for "oracle" modules
[ https://issues.apache.org/jira/browse/AIRFLOW-5981?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jarek Potiuk resolved AIRFLOW-5981. --- Fix Version/s: 2.0.0 Resolution: Duplicate Those all issues have been resolved by a number of other JIRA issues: AIRFLOW-6644 AIRFLOW-6646 AIRFLOW-6656 AIRFLOW-6674 AIRFLOW-6689 ARIFLOW-6763 > Change import paths for "oracle" modules > > > Key: AIRFLOW-5981 > URL: https://issues.apache.org/jira/browse/AIRFLOW-5981 > Project: Apache Airflow > Issue Type: Sub-task > Components: operators >Affects Versions: 2.0.0 >Reporter: Jarek Potiuk >Priority: Major > Fix For: 2.0.0 > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (AIRFLOW-5997) Change import paths for "jdbc" modules
[ https://issues.apache.org/jira/browse/AIRFLOW-5997?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jarek Potiuk resolved AIRFLOW-5997. --- Fix Version/s: 2.0.0 Resolution: Duplicate Those all issues have been resolved by a number of other JIRA issues: AIRFLOW-6644 AIRFLOW-6646 AIRFLOW-6656 AIRFLOW-6674 AIRFLOW-6689 ARIFLOW-6763 > Change import paths for "jdbc" modules > -- > > Key: AIRFLOW-5997 > URL: https://issues.apache.org/jira/browse/AIRFLOW-5997 > Project: Apache Airflow > Issue Type: Sub-task > Components: operators >Affects Versions: 2.0.0 >Reporter: Jarek Potiuk >Priority: Major > Fix For: 2.0.0 > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (AIRFLOW-5994) Change import paths for "http" modules
[ https://issues.apache.org/jira/browse/AIRFLOW-5994?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jarek Potiuk resolved AIRFLOW-5994. --- Fix Version/s: 2.0.0 Resolution: Duplicate Those all issues have been resolved by a number of other JIRA issues: AIRFLOW-6644 AIRFLOW-6646 AIRFLOW-6656 AIRFLOW-6674 AIRFLOW-6689 ARIFLOW-6763 > Change import paths for "http" modules > -- > > Key: AIRFLOW-5994 > URL: https://issues.apache.org/jira/browse/AIRFLOW-5994 > Project: Apache Airflow > Issue Type: Sub-task > Components: operators >Affects Versions: 2.0.0 >Reporter: Jarek Potiuk >Priority: Major > Fix For: 2.0.0 > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (AIRFLOW-5962) Change import paths for "dingding" modules
[ https://issues.apache.org/jira/browse/AIRFLOW-5962?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jarek Potiuk resolved AIRFLOW-5962. --- Fix Version/s: 2.0.0 Resolution: Duplicate Those all issues have been resolved by a number of other JIRA issues: AIRFLOW-6644 AIRFLOW-6646 AIRFLOW-6656 AIRFLOW-6674 AIRFLOW-6689 ARIFLOW-6763 > Change import paths for "dingding" modules > -- > > Key: AIRFLOW-5962 > URL: https://issues.apache.org/jira/browse/AIRFLOW-5962 > Project: Apache Airflow > Issue Type: Sub-task > Components: operators >Affects Versions: 2.0.0 >Reporter: Jarek Potiuk >Priority: Major > Fix For: 2.0.0 > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (AIRFLOW-5964) Change import paths for "cloudant" modules
[ https://issues.apache.org/jira/browse/AIRFLOW-5964?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jarek Potiuk resolved AIRFLOW-5964. --- Fix Version/s: 2.0.0 Resolution: Duplicate Those all issues have been resolved by a number of other JIRA issues: AIRFLOW-6644 AIRFLOW-6646 AIRFLOW-6656 AIRFLOW-6674 AIRFLOW-6689 ARIFLOW-6763 > Change import paths for "cloudant" modules > -- > > Key: AIRFLOW-5964 > URL: https://issues.apache.org/jira/browse/AIRFLOW-5964 > Project: Apache Airflow > Issue Type: Sub-task > Components: operators >Affects Versions: 2.0.0 >Reporter: Jarek Potiuk >Priority: Major > Fix For: 2.0.0 > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (AIRFLOW-5982) Change import paths for "papermill" modules
[ https://issues.apache.org/jira/browse/AIRFLOW-5982?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jarek Potiuk resolved AIRFLOW-5982. --- Fix Version/s: 2.0.0 Resolution: Duplicate Those all issues have been resolved by a number of other JIRA issues: AIRFLOW-6644 AIRFLOW-6646 AIRFLOW-6656 AIRFLOW-6674 AIRFLOW-6689 ARIFLOW-6763 > Change import paths for "papermill" modules > --- > > Key: AIRFLOW-5982 > URL: https://issues.apache.org/jira/browse/AIRFLOW-5982 > Project: Apache Airflow > Issue Type: Sub-task > Components: operators >Affects Versions: 2.0.0 >Reporter: Jarek Potiuk >Priority: Major > Fix For: 2.0.0 > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (AIRFLOW-5977) Change import paths for "mssql" modules
[ https://issues.apache.org/jira/browse/AIRFLOW-5977?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jarek Potiuk resolved AIRFLOW-5977. --- Fix Version/s: 2.0.0 Resolution: Duplicate Those all issues have been resolved by a number of other JIRA issues: AIRFLOW-6644 AIRFLOW-6646 AIRFLOW-6656 AIRFLOW-6674 AIRFLOW-6689 ARIFLOW-6763 > Change import paths for "mssql" modules > --- > > Key: AIRFLOW-5977 > URL: https://issues.apache.org/jira/browse/AIRFLOW-5977 > Project: Apache Airflow > Issue Type: Sub-task > Components: operators >Affects Versions: 2.0.0 >Reporter: Jarek Potiuk >Priority: Major > Fix For: 2.0.0 > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (AIRFLOW-5953) Change import paths for "apache/hive" modules
[ https://issues.apache.org/jira/browse/AIRFLOW-5953?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jarek Potiuk resolved AIRFLOW-5953. --- Fix Version/s: 2.0.0 Resolution: Duplicate Those all issues have been resolved by a number of other JIRA issues: AIRFLOW-6644 AIRFLOW-6646 AIRFLOW-6656 AIRFLOW-6674 AIRFLOW-6689 ARIFLOW-6763 > Change import paths for "apache/hive" modules > - > > Key: AIRFLOW-5953 > URL: https://issues.apache.org/jira/browse/AIRFLOW-5953 > Project: Apache Airflow > Issue Type: Sub-task > Components: operators >Affects Versions: 2.0.0 >Reporter: Jarek Potiuk >Priority: Major > Fix For: 2.0.0 > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (AIRFLOW-5996) Change import paths for "smtp" modules
[ https://issues.apache.org/jira/browse/AIRFLOW-5996?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jarek Potiuk resolved AIRFLOW-5996. --- Fix Version/s: 2.0.0 Resolution: Duplicate Those all issues have been resolved by a number of other JIRA issues: AIRFLOW-6644 AIRFLOW-6646 AIRFLOW-6656 AIRFLOW-6674 AIRFLOW-6689 ARIFLOW-6763 > Change import paths for "smtp" modules > -- > > Key: AIRFLOW-5996 > URL: https://issues.apache.org/jira/browse/AIRFLOW-5996 > Project: Apache Airflow > Issue Type: Sub-task > Components: operators >Affects Versions: 2.0.0 >Reporter: Jarek Potiuk >Priority: Major > Fix For: 2.0.0 > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (AIRFLOW-5966) Change import paths for "opsgenie" modules
[ https://issues.apache.org/jira/browse/AIRFLOW-5966?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jarek Potiuk resolved AIRFLOW-5966. --- Fix Version/s: 2.0.0 Resolution: Duplicate Those all issues have been resolved by a number of other JIRA issues: AIRFLOW-6644 AIRFLOW-6646 AIRFLOW-6656 AIRFLOW-6674 AIRFLOW-6689 ARIFLOW-6763 > Change import paths for "opsgenie" modules > -- > > Key: AIRFLOW-5966 > URL: https://issues.apache.org/jira/browse/AIRFLOW-5966 > Project: Apache Airflow > Issue Type: Sub-task > Components: operators >Affects Versions: 2.0.0 >Reporter: Jarek Potiuk >Priority: Major > Fix For: 2.0.0 > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (AIRFLOW-5988) Change import paths for "sqlite" modules
[ https://issues.apache.org/jira/browse/AIRFLOW-5988?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jarek Potiuk resolved AIRFLOW-5988. --- Fix Version/s: 2.0.0 Resolution: Duplicate Those all issues have been resolved by a number of other JIRA issues: AIRFLOW-6644 AIRFLOW-6646 AIRFLOW-6656 AIRFLOW-6674 AIRFLOW-6689 ARIFLOW-6763 > Change import paths for "sqlite" modules > > > Key: AIRFLOW-5988 > URL: https://issues.apache.org/jira/browse/AIRFLOW-5988 > Project: Apache Airflow > Issue Type: Sub-task > Components: operators >Affects Versions: 2.0.0 >Reporter: Jarek Potiuk >Priority: Major > Fix For: 2.0.0 > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (AIRFLOW-5999) Add mechanism for backporting "providers" to 1.10.*
[ https://issues.apache.org/jira/browse/AIRFLOW-5999?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jarek Potiuk resolved AIRFLOW-5999. --- Fix Version/s: 2.0.0 Resolution: Duplicate Those all issues have been resolved by a number of other JIRA issues: AIRFLOW-6644 AIRFLOW-6646 AIRFLOW-6656 AIRFLOW-6674 AIRFLOW-6689 ARIFLOW-6763 > Add mechanism for backporting "providers" to 1.10.* > --- > > Key: AIRFLOW-5999 > URL: https://issues.apache.org/jira/browse/AIRFLOW-5999 > Project: Apache Airflow > Issue Type: Sub-task > Components: core, dependencies >Affects Versions: 2.0.0 >Reporter: Jarek Potiuk >Priority: Major > Fix For: 2.0.0 > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (AIRFLOW-5975) Change import paths for "docker" modules
[ https://issues.apache.org/jira/browse/AIRFLOW-5975?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jarek Potiuk resolved AIRFLOW-5975. --- Fix Version/s: 2.0.0 Resolution: Duplicate Those all issues have been resolved by a number of other JIRA issues: AIRFLOW-6644 AIRFLOW-6646 AIRFLOW-6656 AIRFLOW-6674 AIRFLOW-6689 ARIFLOW-6763 > Change import paths for "docker" modules > > > Key: AIRFLOW-5975 > URL: https://issues.apache.org/jira/browse/AIRFLOW-5975 > Project: Apache Airflow > Issue Type: Sub-task > Components: operators >Affects Versions: 2.0.0 >Reporter: Jarek Potiuk >Priority: Major > Fix For: 2.0.0 > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (AIRFLOW-5965) Change import paths for "jenkins" modules
[ https://issues.apache.org/jira/browse/AIRFLOW-5965?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jarek Potiuk resolved AIRFLOW-5965. --- Fix Version/s: 2.0.0 Resolution: Duplicate Those all issues have been resolved by a number of other JIRA issues: AIRFLOW-6644 AIRFLOW-6646 AIRFLOW-6656 AIRFLOW-6674 AIRFLOW-6689 ARIFLOW-6763 > Change import paths for "jenkins" modules > - > > Key: AIRFLOW-5965 > URL: https://issues.apache.org/jira/browse/AIRFLOW-5965 > Project: Apache Airflow > Issue Type: Sub-task > Components: operators >Affects Versions: 2.0.0 >Reporter: Jarek Potiuk >Priority: Major > Fix For: 2.0.0 > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (AIRFLOW-5987) Change import paths for "samba" modules
[ https://issues.apache.org/jira/browse/AIRFLOW-5987?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jarek Potiuk resolved AIRFLOW-5987. --- Fix Version/s: 2.0.0 Resolution: Duplicate Those all issues have been resolved by a number of other JIRA issues: AIRFLOW-6644 AIRFLOW-6646 AIRFLOW-6656 AIRFLOW-6674 AIRFLOW-6689 ARIFLOW-6763 > Change import paths for "samba" modules > --- > > Key: AIRFLOW-5987 > URL: https://issues.apache.org/jira/browse/AIRFLOW-5987 > Project: Apache Airflow > Issue Type: Sub-task > Components: operators >Affects Versions: 2.0.0 >Reporter: Jarek Potiuk >Priority: Major > Fix For: 2.0.0 > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (AIRFLOW-5983) Change import paths for "postgres" modules
[ https://issues.apache.org/jira/browse/AIRFLOW-5983?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jarek Potiuk resolved AIRFLOW-5983. --- Fix Version/s: 2.0.0 Resolution: Duplicate Those all issues have been resolved by a number of other JIRA issues: AIRFLOW-6644 AIRFLOW-6646 AIRFLOW-6656 AIRFLOW-6674 AIRFLOW-6689 ARIFLOW-6763 > Change import paths for "postgres" modules > -- > > Key: AIRFLOW-5983 > URL: https://issues.apache.org/jira/browse/AIRFLOW-5983 > Project: Apache Airflow > Issue Type: Sub-task > Components: operators >Affects Versions: 2.0.0 >Reporter: Jarek Potiuk >Priority: Major > Fix For: 2.0.0 > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (AIRFLOW-5963) Change import paths for "discord" modules
[ https://issues.apache.org/jira/browse/AIRFLOW-5963?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jarek Potiuk resolved AIRFLOW-5963. --- Fix Version/s: 2.0.0 Resolution: Duplicate Those all issues have been resolved by a number of other JIRA issues: AIRFLOW-6644 AIRFLOW-6646 AIRFLOW-6656 AIRFLOW-6674 AIRFLOW-6689 ARIFLOW-6763 > Change import paths for "discord" modules > - > > Key: AIRFLOW-5963 > URL: https://issues.apache.org/jira/browse/AIRFLOW-5963 > Project: Apache Airflow > Issue Type: Sub-task > Components: operators >Affects Versions: 2.0.0 >Reporter: Jarek Potiuk >Priority: Major > Fix For: 2.0.0 > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (AIRFLOW-5998) Change import paths for "winrm" modules
[ https://issues.apache.org/jira/browse/AIRFLOW-5998?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jarek Potiuk resolved AIRFLOW-5998. --- Fix Version/s: 2.0.0 Resolution: Duplicate Those all issues have been resolved by a number of other JIRA issues: AIRFLOW-6644 AIRFLOW-6646 AIRFLOW-6656 AIRFLOW-6674 AIRFLOW-6689 ARIFLOW-6763 > Change import paths for "winrm" modules > --- > > Key: AIRFLOW-5998 > URL: https://issues.apache.org/jira/browse/AIRFLOW-5998 > Project: Apache Airflow > Issue Type: Sub-task > Components: operators >Affects Versions: 2.0.0 >Reporter: Jarek Potiuk >Priority: Major > Fix For: 2.0.0 > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (AIRFLOW-5993) Change import paths for "ftp" modules
[ https://issues.apache.org/jira/browse/AIRFLOW-5993?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jarek Potiuk resolved AIRFLOW-5993. --- Fix Version/s: 2.0.0 Resolution: Duplicate Those all issues have been resolved by a number of other JIRA issues: AIRFLOW-6644 AIRFLOW-6646 AIRFLOW-6656 AIRFLOW-6674 AIRFLOW-6689 ARIFLOW-6763 > Change import paths for "ftp" modules > - > > Key: AIRFLOW-5993 > URL: https://issues.apache.org/jira/browse/AIRFLOW-5993 > Project: Apache Airflow > Issue Type: Sub-task > Components: operators >Affects Versions: 2.0.0 >Reporter: Jarek Potiuk >Priority: Major > Fix For: 2.0.0 > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (AIRFLOW-5989) Change import paths for "imap" modules
[ https://issues.apache.org/jira/browse/AIRFLOW-5989?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jarek Potiuk resolved AIRFLOW-5989. --- Fix Version/s: 2.0.0 Resolution: Duplicate Those all issues have been resolved by a number of other JIRA issues: AIRFLOW-6644 AIRFLOW-6646 AIRFLOW-6656 AIRFLOW-6674 AIRFLOW-6689 ARIFLOW-6763 > Change import paths for "imap" modules > -- > > Key: AIRFLOW-5989 > URL: https://issues.apache.org/jira/browse/AIRFLOW-5989 > Project: Apache Airflow > Issue Type: Sub-task > Components: operators >Affects Versions: 2.0.0 >Reporter: Jarek Potiuk >Priority: Major > Fix For: 2.0.0 > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (AIRFLOW-5955) Change import paths for "apache/pinot" modules
[ https://issues.apache.org/jira/browse/AIRFLOW-5955?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jarek Potiuk resolved AIRFLOW-5955. --- Fix Version/s: 2.0.0 Resolution: Duplicate Those all issues have been resolved by a number of other JIRA issues: AIRFLOW-6644 AIRFLOW-6646 AIRFLOW-6656 AIRFLOW-6674 AIRFLOW-6689 ARIFLOW-6763 > Change import paths for "apache/pinot" modules > -- > > Key: AIRFLOW-5955 > URL: https://issues.apache.org/jira/browse/AIRFLOW-5955 > Project: Apache Airflow > Issue Type: Sub-task > Components: operators >Affects Versions: 2.0.0 >Reporter: Jarek Potiuk >Priority: Major > Fix For: 2.0.0 > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (AIRFLOW-5973) Change import paths for "zendesk" modules
[ https://issues.apache.org/jira/browse/AIRFLOW-5973?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jarek Potiuk resolved AIRFLOW-5973. --- Fix Version/s: 2.0.0 Resolution: Duplicate Those all issues have been resolved by a number of other JIRA issues: AIRFLOW-6644 AIRFLOW-6646 AIRFLOW-6656 AIRFLOW-6674 AIRFLOW-6689 ARIFLOW-6763 > Change import paths for "zendesk" modules > - > > Key: AIRFLOW-5973 > URL: https://issues.apache.org/jira/browse/AIRFLOW-5973 > Project: Apache Airflow > Issue Type: Sub-task > Components: operators >Affects Versions: 2.0.0 >Reporter: Jarek Potiuk >Priority: Major > Fix For: 2.0.0 > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (AIRFLOW-5788) Change import paths for AWS modules
[ https://issues.apache.org/jira/browse/AIRFLOW-5788?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jarek Potiuk resolved AIRFLOW-5788. --- Fix Version/s: 2.0.0 Resolution: Duplicate Those all issues have been resolved by a number of other JIRA issues: AIRFLOW-6644 AIRFLOW-6646 AIRFLOW-6656 AIRFLOW-6674 AIRFLOW-6689 ARIFLOW-6763 > Change import paths for AWS modules > --- > > Key: AIRFLOW-5788 > URL: https://issues.apache.org/jira/browse/AIRFLOW-5788 > Project: Apache Airflow > Issue Type: Sub-task > Components: pylint >Affects Versions: 2.0.0 >Reporter: Bas Harenslak >Assignee: Kamil Bregula >Priority: Major > Fix For: 2.0.0 > > > When migrating, think of the following: > * UPDATING.MD > * pylint_todo.txt > * docs (docs/operators-and-hooks-ref.rst) > * deprecation warning for old paths > * migrating tests > * tests/test_core_to_contrib.py -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (AIRFLOW-5166) Change import paths for GCP modules
[ https://issues.apache.org/jira/browse/AIRFLOW-5166?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jarek Potiuk resolved AIRFLOW-5166. --- Fix Version/s: 2.0.0 Resolution: Duplicate Those all issues have been resolved by a number of other JIRA issues: AIRFLOW-6644 AIRFLOW-6646 AIRFLOW-6656 AIRFLOW-6674 AIRFLOW-6689 ARIFLOW-6763 > Change import paths for GCP modules > --- > > Key: AIRFLOW-5166 > URL: https://issues.apache.org/jira/browse/AIRFLOW-5166 > Project: Apache Airflow > Issue Type: Sub-task > Components: gcp >Affects Versions: 2.0.0 >Reporter: Tomasz Urbaszek >Priority: Major > Fix For: 2.0.0 > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (AIRFLOW-5957) Change import paths for "apache/sqoop" modules
[ https://issues.apache.org/jira/browse/AIRFLOW-5957?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jarek Potiuk resolved AIRFLOW-5957. --- Fix Version/s: 2.0.0 Resolution: Duplicate Those all issues have been resolved by a number of other JIRA issues: AIRFLOW-6644 AIRFLOW-6646 AIRFLOW-6656 AIRFLOW-6674 AIRFLOW-6689 ARIFLOW-6763 > Change import paths for "apache/sqoop" modules > -- > > Key: AIRFLOW-5957 > URL: https://issues.apache.org/jira/browse/AIRFLOW-5957 > Project: Apache Airflow > Issue Type: Sub-task > Components: operators >Affects Versions: 2.0.0 >Reporter: Jarek Potiuk >Priority: Major > Fix For: 2.0.0 > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (AIRFLOW-5958) Change import paths for "mysql" modules
[ https://issues.apache.org/jira/browse/AIRFLOW-5958?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jarek Potiuk resolved AIRFLOW-5958. --- Fix Version/s: 2.0.0 Resolution: Duplicate Those all issues have been resolved by a number of other JIRA issues: AIRFLOW-6644 AIRFLOW-6646 AIRFLOW-6656 AIRFLOW-6674 AIRFLOW-6689 ARIFLOW-6763 > Change import paths for "mysql" modules > --- > > Key: AIRFLOW-5958 > URL: https://issues.apache.org/jira/browse/AIRFLOW-5958 > Project: Apache Airflow > Issue Type: Sub-task > Components: operators >Affects Versions: 2.0.0 >Reporter: Jarek Potiuk >Assignee: Rich Dean >Priority: Major > Fix For: 2.0.0 > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (AIRFLOW-5992) Change import paths for "sftp" modules
[ https://issues.apache.org/jira/browse/AIRFLOW-5992?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jarek Potiuk resolved AIRFLOW-5992. --- Fix Version/s: 2.0.0 Resolution: Duplicate Those all issues have been resolved by a number of other JIRA issues: AIRFLOW-6644 AIRFLOW-6646 AIRFLOW-6656 AIRFLOW-6674 AIRFLOW-6689 ARIFLOW-6763 > Change import paths for "sftp" modules > -- > > Key: AIRFLOW-5992 > URL: https://issues.apache.org/jira/browse/AIRFLOW-5992 > Project: Apache Airflow > Issue Type: Sub-task > Components: operators >Affects Versions: 2.0.0 >Reporter: Jarek Potiuk >Priority: Major > Fix For: 2.0.0 > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (AIRFLOW-5990) Change import paths for "ssh" modules
[ https://issues.apache.org/jira/browse/AIRFLOW-5990?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jarek Potiuk resolved AIRFLOW-5990. --- Fix Version/s: 2.0.0 Resolution: Duplicate Those all issues have been resolved by a number of other JIRA issues: AIRFLOW-6644 AIRFLOW-6646 AIRFLOW-6656 AIRFLOW-6674 AIRFLOW-6689 ARIFLOW-6763 > Change import paths for "ssh" modules > - > > Key: AIRFLOW-5990 > URL: https://issues.apache.org/jira/browse/AIRFLOW-5990 > Project: Apache Airflow > Issue Type: Sub-task > Components: operators >Affects Versions: 2.0.0 >Reporter: Jarek Potiuk >Priority: Major > Fix For: 2.0.0 > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (AIRFLOW-5960) Change import paths for "databricks" module
[ https://issues.apache.org/jira/browse/AIRFLOW-5960?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jarek Potiuk resolved AIRFLOW-5960. --- Fix Version/s: 2.0.0 Resolution: Duplicate Those all issues have been resolved by a number of other JIRA issues: AIRFLOW-6644 AIRFLOW-6646 AIRFLOW-6656 AIRFLOW-6674 AIRFLOW-6689 ARIFLOW-6763 > Change import paths for "databricks" module > --- > > Key: AIRFLOW-5960 > URL: https://issues.apache.org/jira/browse/AIRFLOW-5960 > Project: Apache Airflow > Issue Type: Sub-task > Components: operators >Affects Versions: 2.0.0 >Reporter: Jarek Potiuk >Priority: Major > Fix For: 2.0.0 > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (AIRFLOW-5991) Change import paths for "filesystem" modules
[ https://issues.apache.org/jira/browse/AIRFLOW-5991?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jarek Potiuk resolved AIRFLOW-5991. --- Fix Version/s: 2.0.0 Resolution: Duplicate Those all issues have been resolved by a number of other JIRA issues: AIRFLOW-6644 AIRFLOW-6646 AIRFLOW-6656 AIRFLOW-6674 AIRFLOW-6689 ARIFLOW-6763 > Change import paths for "filesystem" modules > > > Key: AIRFLOW-5991 > URL: https://issues.apache.org/jira/browse/AIRFLOW-5991 > Project: Apache Airflow > Issue Type: Sub-task > Components: operators >Affects Versions: 2.0.0 >Reporter: Jarek Potiuk >Priority: Major > Fix For: 2.0.0 > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (AIRFLOW-5979) Change import paths for "mysql" modules
[ https://issues.apache.org/jira/browse/AIRFLOW-5979?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jarek Potiuk resolved AIRFLOW-5979. --- Fix Version/s: 2.0.0 Resolution: Duplicate Those all issues have been resolved by a number of other JIRA issues: AIRFLOW-6644 AIRFLOW-6646 AIRFLOW-6656 AIRFLOW-6674 AIRFLOW-6689 ARIFLOW-6763 > Change import paths for "mysql" modules > --- > > Key: AIRFLOW-5979 > URL: https://issues.apache.org/jira/browse/AIRFLOW-5979 > Project: Apache Airflow > Issue Type: Sub-task > Components: operators >Affects Versions: 2.0.0 >Reporter: Jarek Potiuk >Priority: Major > Fix For: 2.0.0 > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (AIRFLOW-5967) Change import paths for "qubole" modules
[ https://issues.apache.org/jira/browse/AIRFLOW-5967?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jarek Potiuk resolved AIRFLOW-5967. --- Fix Version/s: 2.0.0 Resolution: Duplicate Those all issues have been resolved by a number of other JIRA issues: AIRFLOW-6644 AIRFLOW-6646 AIRFLOW-6656 AIRFLOW-6674 AIRFLOW-6689 ARIFLOW-6763 > Change import paths for "qubole" modules > > > Key: AIRFLOW-5967 > URL: https://issues.apache.org/jira/browse/AIRFLOW-5967 > Project: Apache Airflow > Issue Type: Sub-task > Components: operators >Affects Versions: 2.0.0 >Reporter: Jarek Potiuk >Priority: Major > Fix For: 2.0.0 > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (AIRFLOW-5971) Change import paths for "snowflake" modules
[ https://issues.apache.org/jira/browse/AIRFLOW-5971?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jarek Potiuk resolved AIRFLOW-5971. --- Fix Version/s: 2.0.0 Resolution: Duplicate Those all issues have been resolved by a number of other JIRA issues: AIRFLOW-6644 AIRFLOW-6646 AIRFLOW-6656 AIRFLOW-6674 AIRFLOW-6689 ARIFLOW-6763 > Change import paths for "snowflake" modules > --- > > Key: AIRFLOW-5971 > URL: https://issues.apache.org/jira/browse/AIRFLOW-5971 > Project: Apache Airflow > Issue Type: Sub-task > Components: operators >Affects Versions: 2.0.0 >Reporter: Jarek Potiuk >Priority: Major > Fix For: 2.0.0 > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (AIRFLOW-5978) Change import paths for "mongodb" modules
[ https://issues.apache.org/jira/browse/AIRFLOW-5978?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jarek Potiuk resolved AIRFLOW-5978. --- Fix Version/s: 2.0.0 Resolution: Duplicate Those all issues have been resolved by a number of other JIRA issues: AIRFLOW-6644 AIRFLOW-6646 AIRFLOW-6656 AIRFLOW-6674 AIRFLOW-6689 ARIFLOW-6763 > Change import paths for "mongodb" modules > - > > Key: AIRFLOW-5978 > URL: https://issues.apache.org/jira/browse/AIRFLOW-5978 > Project: Apache Airflow > Issue Type: Sub-task > Components: operators >Affects Versions: 2.0.0 >Reporter: Jarek Potiuk >Priority: Major > Fix For: 2.0.0 > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (AIRFLOW-5970) Change import paths for "slack" modules
[ https://issues.apache.org/jira/browse/AIRFLOW-5970?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jarek Potiuk resolved AIRFLOW-5970. --- Fix Version/s: 2.0.0 Resolution: Duplicate Those all issues have been resolved by a number of other JIRA issues: AIRFLOW-6644 AIRFLOW-6646 AIRFLOW-6656 AIRFLOW-6674 AIRFLOW-6689 ARIFLOW-6763 > Change import paths for "slack" modules > --- > > Key: AIRFLOW-5970 > URL: https://issues.apache.org/jira/browse/AIRFLOW-5970 > Project: Apache Airflow > Issue Type: Sub-task > Components: operators >Affects Versions: 2.0.0 >Reporter: Jarek Potiuk >Priority: Major > Fix For: 2.0.0 > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (AIRFLOW-5976) Change import paths for "kubernetes" modules
[ https://issues.apache.org/jira/browse/AIRFLOW-5976?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jarek Potiuk resolved AIRFLOW-5976. --- Fix Version/s: 2.0.0 Resolution: Duplicate Those all issues have been resolved by a number of other JIRA issues: AIRFLOW-6644 AIRFLOW-6646 AIRFLOW-6656 AIRFLOW-6674 AIRFLOW-6689 ARIFLOW-6763 > Change import paths for "kubernetes" modules > > > Key: AIRFLOW-5976 > URL: https://issues.apache.org/jira/browse/AIRFLOW-5976 > Project: Apache Airflow > Issue Type: Sub-task > Components: operators >Affects Versions: 2.0.0 >Reporter: Jarek Potiuk >Priority: Major > Fix For: 2.0.0 > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (AIRFLOW-4733) [AIP-21] Changes in airflow module names
[ https://issues.apache.org/jira/browse/AIRFLOW-4733?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jarek Potiuk resolved AIRFLOW-4733. --- Fix Version/s: 2.0.0 Resolution: Fixed > [AIP-21] Changes in airflow module names > > > Key: AIRFLOW-4733 > URL: https://issues.apache.org/jira/browse/AIRFLOW-4733 > Project: Apache Airflow > Issue Type: New Feature > Components: core >Affects Versions: 2.0.0 >Reporter: Kamil Bregula >Priority: Major > Fix For: 2.0.0 > > > I suggested some changes in the import paths. > > Discussion: > [https://lists.apache.org/thread.html/c880ef89f8cb4a0240c404f9372615b998c4a4eeca342651927d596c@%3Cdev.airflow.apache.org%3E] > AIP: > [https://cwiki.apache.org/confluence/display/AIRFLOW/AIP-21%3A+Changes+in+import+paths] -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (AIRFLOW-6663) Backportable packages for AIP-21
[ https://issues.apache.org/jira/browse/AIRFLOW-6663?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jarek Potiuk resolved AIRFLOW-6663. --- Fix Version/s: 2.0.0 Resolution: Fixed > Backportable packages for AIP-21 > > > Key: AIRFLOW-6663 > URL: https://issues.apache.org/jira/browse/AIRFLOW-6663 > Project: Apache Airflow > Issue Type: Improvement > Components: ci, dependencies, packages >Affects Versions: 2.0.0 >Reporter: Jarek Potiuk >Priority: Major > Fix For: 2.0.0 > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (AIRFLOW-4030) Add Singularity Container Operator
[ https://issues.apache.org/jira/browse/AIRFLOW-4030?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jarek Potiuk resolved AIRFLOW-4030. --- Fix Version/s: 2.0.0 Resolution: Fixed > Add Singularity Container Operator > -- > > Key: AIRFLOW-4030 > URL: https://issues.apache.org/jira/browse/AIRFLOW-4030 > Project: Apache Airflow > Issue Type: Improvement > Components: operators >Reporter: Vanessa >Assignee: Vanessa >Priority: Minor > Fix For: 2.0.0 > > > Singularity containers are comparable to Docker in the level of operation - > they provide an encapsulated environment with an OS, libraries, and custom > software for the user to run. The key difference is that Docker is not > optimized for scientific compute because it could never be installed on a > shared research cluster. Singularity, on the other hand, does not have these > issues and is installed across HCP centers internationally. > This issue is to add Singularity containers as an operator to Apache Airflow, > so that we can start to explore using airflow in an HPC environment. I work > with Encode DCC at Stanford, and am hopeful to explore Airflow as an > alternative to the workflow manager(s) we are using. I am one of the > [original Singularity developers see > |https://journals.plos.org/plosone/article?id=10.1371/journal.pone.0177459] > that manages the Singularity Python client (spython), Singularity Hub and > Singularity Registry Server, and have started working on this issue here: > [https://github.com/apache/airflow/pull/4846.] Looking forward to working > with you! -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (AIRFLOW-6877) Add backport packages dependency warnings
Jarek Potiuk created AIRFLOW-6877: - Summary: Add backport packages dependency warnings Key: AIRFLOW-6877 URL: https://issues.apache.org/jira/browse/AIRFLOW-6877 Project: Apache Airflow Issue Type: Improvement Components: backport-packages Affects Versions: 2.0.0 Reporter: Jarek Potiuk Some of the backport packages depend on others - but this is mostly for transfer operators. other operators should work fine. We should allow those packages to be imported with a warning that you need to import the other package to get full functionality -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (AIRFLOW-6817) Move airflow __init__.py imports to sub-packages
[ https://issues.apache.org/jira/browse/AIRFLOW-6817?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jarek Potiuk resolved AIRFLOW-6817. --- Fix Version/s: 2.0.0 Resolution: Fixed > Move airflow __init__.py imports to sub-packages > > > Key: AIRFLOW-6817 > URL: https://issues.apache.org/jira/browse/AIRFLOW-6817 > Project: Apache Airflow > Issue Type: Improvement > Components: core >Affects Versions: 2.0.0 >Reporter: Jarek Potiuk >Priority: Major > Fix For: 2.0.0 > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (AIRFLOW-6870) Remove models from models.__init__.py
[ https://issues.apache.org/jira/browse/AIRFLOW-6870?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17042128#comment-17042128 ] Jarek Potiuk commented on AIRFLOW-6870: --- See the comment here: https://github.com/apache/airflow/pull/7456#discussion_r382758098 > Remove models from models.__init__.py > - > > Key: AIRFLOW-6870 > URL: https://issues.apache.org/jira/browse/AIRFLOW-6870 > Project: Apache Airflow > Issue Type: Improvement > Components: models >Affects Versions: 2.0.0 >Reporter: Jarek Potiuk >Priority: Major > > We should remove all models from __init__.py and have a separate > `all_models.py` which then will be used by resetdb to drop all tables. This > is a follow-up to AIRFLOW-6817 > > See the comment > https://github.com/apache/airflow/pull/7456#discussion_r382758098 -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (AIRFLOW-6870) Remove models from models.__init__.py
[ https://issues.apache.org/jira/browse/AIRFLOW-6870?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jarek Potiuk updated AIRFLOW-6870: -- Description: We should remove all models from __init__.py and have a separate `all_models.py` which then will be used by resetdb to drop all tables. This is a follow-up to AIRFLOW-6817 See the comment https://github.com/apache/airflow/pull/7456#discussion_r382758098 was:We should remove all models from __init__.py and have a separate `all_models.py` which then will be used by resetdb to drop all tables. This is a follow-up to AIRFLOW-6817 > Remove models from models.__init__.py > - > > Key: AIRFLOW-6870 > URL: https://issues.apache.org/jira/browse/AIRFLOW-6870 > Project: Apache Airflow > Issue Type: Improvement > Components: models >Affects Versions: 2.0.0 >Reporter: Jarek Potiuk >Priority: Major > > We should remove all models from __init__.py and have a separate > `all_models.py` which then will be used by resetdb to drop all tables. This > is a follow-up to AIRFLOW-6817 > > See the comment > https://github.com/apache/airflow/pull/7456#discussion_r382758098 -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (AIRFLOW-6870) Remove models from models.__init__.py
Jarek Potiuk created AIRFLOW-6870: - Summary: Remove models from models.__init__.py Key: AIRFLOW-6870 URL: https://issues.apache.org/jira/browse/AIRFLOW-6870 Project: Apache Airflow Issue Type: Improvement Components: models Affects Versions: 2.0.0 Reporter: Jarek Potiuk We should remove all models from __init__.py and have a separate `all_models.py` which then will be used by resetdb to drop all tables. This is a follow-up to AIRFLOW-6817 -- This message was sent by Atlassian Jira (v8.3.4#803005)
