[GitHub] [airflow] stale[bot] commented on pull request #8709: Acquire lock on db for the time of migration

2020-06-19 Thread GitBox
stale[bot] commented on pull request #8709: URL: https://github.com/apache/airflow/pull/8709#issuecomment-646931307 This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[GitHub] [airflow] stale[bot] commented on pull request #6961: [AIRFLOW-XXX] Add task execution process on CeleryExecutor diagram

2020-06-19 Thread GitBox
stale[bot] commented on pull request #6961: URL: https://github.com/apache/airflow/pull/6961#issuecomment-646931310 This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[GitHub] [airflow] stale[bot] closed pull request #8595: test pulling docker iamge from github

2020-06-19 Thread GitBox
stale[bot] closed pull request #8595: URL: https://github.com/apache/airflow/pull/8595 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go

[GitHub] [airflow] stale[bot] closed pull request #8586: Bump FAB in order to bump jQuery (resolves SNYK-JS-JQUERY-565129)

2020-06-19 Thread GitBox
stale[bot] closed pull request #8586: URL: https://github.com/apache/airflow/pull/8586 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go

[GitHub] [airflow] stale[bot] commented on pull request #8553: Add DataflowStartSQLQuery operator

2020-06-19 Thread GitBox
stale[bot] commented on pull request #8553: URL: https://github.com/apache/airflow/pull/8553#issuecomment-646931304 This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[GitHub] [airflow] stale[bot] commented on pull request #8621: Docker Compose CeleryExecutor example

2020-06-19 Thread GitBox
stale[bot] commented on pull request #8621: URL: https://github.com/apache/airflow/pull/8621#issuecomment-646931322 This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[GitHub] [airflow] stale[bot] commented on pull request #8668: Bump supported mysqlclient to <1.5

2020-06-19 Thread GitBox
stale[bot] commented on pull request #8668: URL: https://github.com/apache/airflow/pull/8668#issuecomment-646931318 This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[GitHub] [airflow] stale[bot] closed pull request #8491: add Kerberos support to PrestoHook

2020-06-19 Thread GitBox
stale[bot] closed pull request #8491: URL: https://github.com/apache/airflow/pull/8491 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go

[GitHub] [airflow] stale[bot] commented on pull request #8154: Delete cast(DR.execution_date, DateTime)

2020-06-19 Thread GitBox
stale[bot] commented on pull request #8154: URL: https://github.com/apache/airflow/pull/8154#issuecomment-646931321 This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[GitHub] [airflow] stale[bot] commented on pull request #8432: Provide GCP credentials in Bash/Python operators

2020-06-19 Thread GitBox
stale[bot] commented on pull request #8432: URL: https://github.com/apache/airflow/pull/8432#issuecomment-646931305 This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[GitHub] [airflow] ephraimbuddy commented on a change in pull request #9266: Add CRUD endpoint for connections

2020-06-19 Thread GitBox
ephraimbuddy commented on a change in pull request #9266: URL: https://github.com/apache/airflow/pull/9266#discussion_r443095821 ## File path: tests/api_connexion/endpoints/test_connection_endpoint.py ## @@ -200,14 +229,143 @@ def _create_connections(self, count): class

[GitHub] [airflow] ephraimbuddy commented on a change in pull request #9266: Add CRUD endpoint for connections

2020-06-19 Thread GitBox
ephraimbuddy commented on a change in pull request #9266: URL: https://github.com/apache/airflow/pull/9266#discussion_r443095754 ## File path: tests/api_connexion/endpoints/test_connection_endpoint.py ## @@ -200,14 +229,143 @@ def _create_connections(self, count): class

[GitHub] [airflow] kaxil opened a new pull request #9428: Remove incorrect docstrings in check_migrations

2020-06-19 Thread GitBox
kaxil opened a new pull request #9428: URL: https://github.com/apache/airflow/pull/9428 No point in having docstrings like: ``` @param timeout: ``` --- Make sure to mark the boxes below before creating PR: [x] - [x] Description above provides context of the

[airflow] branch v1-10-test updated (4d8c4e2 -> f2ee8e8)

2020-06-19 Thread dimberman
This is an automated email from the ASF dual-hosted git repository. dimberman pushed a change to branch v1-10-test in repository https://gitbox.apache.org/repos/asf/airflow.git. discard 4d8c4e2 remove core_to_contrib discard 5091334 remove core_to_contrib discard f56eeab remote

[airflow] 03/04: remove core_to_contrib

2020-06-19 Thread dimberman
This is an automated email from the ASF dual-hosted git repository. dimberman pushed a commit to branch v1-10-test in repository https://gitbox.apache.org/repos/asf/airflow.git commit db795a4e43778dd2f18ccfa3ab054e9a2ad823f9 Author: Daniel Imberman AuthorDate: Fri Jun 19 19:35:47 2020 -0700

[airflow] 04/04: flake8 tests

2020-06-19 Thread dimberman
This is an automated email from the ASF dual-hosted git repository. dimberman pushed a commit to branch v1-10-test in repository https://gitbox.apache.org/repos/asf/airflow.git commit f2ee8e868ff969860d3ba5c47f4f1c04808543ca Author: Daniel Imberman AuthorDate: Fri Jun 19 20:01:25 2020 -0700

[airflow] 01/04: flake8 pass Merging multiple sql operators (#9124)

2020-06-19 Thread dimberman
This is an automated email from the ASF dual-hosted git repository. dimberman pushed a commit to branch v1-10-test in repository https://gitbox.apache.org/repos/asf/airflow.git commit 212a4c8c80ca8fdb0e451a54d8665802406fa74a Author: Daniel Imberman AuthorDate: Fri Jun 19 15:15:50 2020 -0700

[airflow] 02/04: alembic fix

2020-06-19 Thread dimberman
This is an automated email from the ASF dual-hosted git repository. dimberman pushed a commit to branch v1-10-test in repository https://gitbox.apache.org/repos/asf/airflow.git commit 220848589b193c1a4f8216ae83e78172e53a8b81 Author: Daniel Imberman AuthorDate: Fri Jun 19 17:27:27 2020 -0700

[GitHub] [airflow] kaxil opened a new pull request #9427: Remove redundant functions in airflow/security/utils.py

2020-06-19 Thread GitBox
kaxil opened a new pull request #9427: URL: https://github.com/apache/airflow/pull/9427 https://github.com/apache/airflow/pull/8071 removed Auth Backend but didn't remove some functions used by those method. --- Make sure to mark the boxes below before creating PR: [x] -

[airflow] tag nightly-master updated (48da4b4 -> a87575f)

2020-06-19 Thread github-bot
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a change to tag nightly-master in repository https://gitbox.apache.org/repos/asf/airflow.git. *** WARNING: tag nightly-master was modified! *** from 48da4b4 (commit) to a87575f (commit) from

[airflow] branch v1-10-test updated: remove core_to_contrib

2020-06-19 Thread dimberman
This is an automated email from the ASF dual-hosted git repository. dimberman pushed a commit to branch v1-10-test in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/v1-10-test by this push: new 4d8c4e2 remove core_to_contrib

[airflow] branch v1-10-test updated: remove core_to_contrib

2020-06-19 Thread dimberman
This is an automated email from the ASF dual-hosted git repository. dimberman pushed a commit to branch v1-10-test in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/v1-10-test by this push: new 5091334 remove core_to_contrib

[airflow] branch v1-10-test updated: remote core_to_contrib

2020-06-19 Thread dimberman
This is an automated email from the ASF dual-hosted git repository. dimberman pushed a commit to branch v1-10-test in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/v1-10-test by this push: new f56eeab remote core_to_contrib

[GitHub] [airflow] kaxil opened a new pull request #9426: Remove empy file: test_hdfs_sensor.py

2020-06-19 Thread GitBox
kaxil opened a new pull request #9426: URL: https://github.com/apache/airflow/pull/9426 This contents of this file were moved to tests/providers/apache/hdfs/sensors/test_hdfs.py in https://github.com/apache/airflow/commit/0481b9a95786a62de4776a735ae80e746583ef2b --- Make sure to

[GitHub] [airflow] kaxil opened a new pull request #9425: Remove unused variables in backport_packages

2020-06-19 Thread GitBox
kaxil opened a new pull request #9425: URL: https://github.com/apache/airflow/pull/9425 There were few instances where we didn't need all the variables from the returned functions. We can remove them. --- Make sure to mark the boxes below before creating PR: [x] - [x]

[GitHub] [airflow] kaxil opened a new pull request #9424: Simplify chained SQL Queries in Connexion API Endpoints

2020-06-19 Thread GitBox
kaxil opened a new pull request #9424: URL: https://github.com/apache/airflow/pull/9424 - The SQLAlchemy Queries can be nested - The comma filters command can take multiple filters --- Make sure to mark the boxes below before creating PR: [x] - [x] Description above

[GitHub] [airflow] kaxil opened a new pull request #9423: Make airflow/models/pool.py Pylint compatible

2020-06-19 Thread GitBox
kaxil opened a new pull request #9423: URL: https://github.com/apache/airflow/pull/9423 Make airflow/models/pool.py Pylint compatible & remove "./airflow/models/dagpickle.py" as it is already Pylint compatible --- Make sure to mark the boxes below before creating PR: [x] -

[airflow] branch master updated (b23e380 -> a87575f)

2020-06-19 Thread kaxilnaik
This is an automated email from the ASF dual-hosted git repository. kaxilnaik pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/airflow.git. from b23e380 Restore airflow.www.app.csrf to avoid breaking change (#9402) add a87575f Make

[GitHub] [airflow] kaxil merged pull request #9422: Make airflow/models/variable.py Pylint compatible

2020-06-19 Thread GitBox
kaxil merged pull request #9422: URL: https://github.com/apache/airflow/pull/9422 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

[GitHub] [airflow] kaxil commented on a change in pull request #9419: Pylint fixes and deprecation of rare used methods in Connection

2020-06-19 Thread GitBox
kaxil commented on a change in pull request #9419: URL: https://github.com/apache/airflow/pull/9419#discussion_r443086868 ## File path: UPDATING.md ## @@ -62,6 +62,29 @@ https://developers.google.com/style/inclusive-documentation --> +### Deprecated method in Connection +

[GitHub] [airflow] mik-laj commented on issue #7935: scheduler gets stuck without a trace

2020-06-19 Thread GitBox
mik-laj commented on issue #7935: URL: https://github.com/apache/airflow/issues/7935#issuecomment-646910530 Could someone try to run pyspy when this incident occurs? This may bring us to a solution. Thanks to this, we will be able to check what code is currently being executed without

[GitHub] [airflow] chrismclennon commented on issue #7935: scheduler gets stuck without a trace

2020-06-19 Thread GitBox
chrismclennon commented on issue #7935: URL: https://github.com/apache/airflow/issues/7935#issuecomment-646909595 I've anecdotally noticed that once I've dropped argument `-n 25` from our scheduler invocation, I haven't seen this issue come up since. Before, it would crop up every ~10

[GitHub] [airflow] kaxil opened a new pull request #9422: Make airflow/models/variable.py Pylint compatible

2020-06-19 Thread GitBox
kaxil opened a new pull request #9422: URL: https://github.com/apache/airflow/pull/9422 Make airflow/models/variable.py Pylint compatible --- Make sure to mark the boxes below before creating PR: [x] - [x] Description above provides context of the change - [x] Unit tests

[airflow] branch v1-10-test updated: alembic fix

2020-06-19 Thread dimberman
This is an automated email from the ASF dual-hosted git repository. dimberman pushed a commit to branch v1-10-test in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/v1-10-test by this push: new 0fa10db alembic fix 0fa10db is

[GitHub] [airflow] svetli-n closed pull request #9338: Get API request limit and offset from parameters.py

2020-06-19 Thread GitBox
svetli-n closed pull request #9338: URL: https://github.com/apache/airflow/pull/9338 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go

[GitHub] [airflow] kaxil commented on a change in pull request #9409: Add extra options for slack webhook operator and slack hook

2020-06-19 Thread GitBox
kaxil commented on a change in pull request #9409: URL: https://github.com/apache/airflow/pull/9409#discussion_r443082354 ## File path: airflow/contrib/hooks/slack_webhook_hook.py ## @@ -86,6 +89,7 @@ def __init__(self, self.icon_url = icon_url

[GitHub] [airflow] kaxil commented on a change in pull request #8962: [AIRFLOW-8057] [AIP-31] Add @task decorator

2020-06-19 Thread GitBox
kaxil commented on a change in pull request #8962: URL: https://github.com/apache/airflow/pull/8962#discussion_r443078458 ## File path: docs/concepts.rst ## @@ -173,6 +213,62 @@ Each task is a node in our DAG, and there is a dependency from task_1 to task_2: We can say that

[GitHub] [airflow] kaxil commented on a change in pull request #8962: [AIRFLOW-8057] [AIP-31] Add @task decorator

2020-06-19 Thread GitBox
kaxil commented on a change in pull request #8962: URL: https://github.com/apache/airflow/pull/8962#discussion_r443078056 ## File path: docs/concepts.rst ## @@ -173,6 +213,62 @@ Each task is a node in our DAG, and there is a dependency from task_1 to task_2: We can say that

[GitHub] [airflow] mik-laj opened a new pull request #9421: Fix pylint issues in some airflow/www/* files

2020-06-19 Thread GitBox
mik-laj opened a new pull request #9421: URL: https://github.com/apache/airflow/pull/9421 --- Make sure to mark the boxes below before creating PR: [x] - [X] Description above provides context of the change - [X] Unit tests coverage for changes (not needed for documentation

[GitHub] [airflow] mik-laj commented on a change in pull request #9338: Get API request limit and offset from parameters.py

2020-06-19 Thread GitBox
mik-laj commented on a change in pull request #9338: URL: https://github.com/apache/airflow/pull/9338#discussion_r443074741 ## File path: airflow/api_connexion/openapi/v1.yaml ## @@ -2134,6 +2134,7 @@ components: schema: type: integer minimum: 1 +

[airflow] branch v1-10-test updated: ASf

2020-06-19 Thread dimberman
This is an automated email from the ASF dual-hosted git repository. dimberman pushed a commit to branch v1-10-test in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/v1-10-test by this push: new a873e85 ASf a873e85 is described

[GitHub] [airflow] ephraimbuddy commented on a change in pull request #9266: Add CRUD endpoint for connections

2020-06-19 Thread GitBox
ephraimbuddy commented on a change in pull request #9266: URL: https://github.com/apache/airflow/pull/9266#discussion_r439981380 ## File path: airflow/api_connexion/endpoints/connection_endpoint.py ## @@ -47,31 +55,53 @@ def get_connection(connection_id, session):

[airflow] branch v1-10-test updated (2e400d3 -> eb15571)

2020-06-19 Thread dimberman
This is an automated email from the ASF dual-hosted git repository. dimberman pushed a change to branch v1-10-test in repository https://gitbox.apache.org/repos/asf/airflow.git. omit 2e400d3 flake8 pass Merging multiple sql operators (#9124) new eb15571 flake8 pass Merging multiple

[airflow] 01/01: flake8 pass Merging multiple sql operators (#9124)

2020-06-19 Thread dimberman
This is an automated email from the ASF dual-hosted git repository. dimberman pushed a commit to branch v1-10-test in repository https://gitbox.apache.org/repos/asf/airflow.git commit eb1557173aee6340bd3595a341aebbcc70bc689f Author: Daniel Imberman AuthorDate: Fri Jun 19 15:15:50 2020 -0700

[airflow] 01/01: flake8 pass Merging multiple sql operators (#9124)

2020-06-19 Thread dimberman
This is an automated email from the ASF dual-hosted git repository. dimberman pushed a commit to branch v1-10-test in repository https://gitbox.apache.org/repos/asf/airflow.git commit 2e400d3a3f0063170b90bac414c40a4e84c031d2 Author: Daniel Imberman AuthorDate: Fri Jun 19 15:15:50 2020 -0700

[airflow] branch v1-10-test updated (6678ac4 -> 2e400d3)

2020-06-19 Thread dimberman
This is an automated email from the ASF dual-hosted git repository. dimberman pushed a change to branch v1-10-test in repository https://gitbox.apache.org/repos/asf/airflow.git. discard 6678ac4 flake8 fixes new 2e400d3 flake8 pass Merging multiple sql operators (#9124) This update

[GitHub] [airflow] ruoyu90 commented on issue #9321: Add Signal Based Scheduling To Airflow

2020-06-19 Thread GitBox
ruoyu90 commented on issue #9321: URL: https://github.com/apache/airflow/issues/9321#issuecomment-646884033 (Thanks @casassg for pointing us here) Hello from [TFX](https://github.com/tensorflow/tfx/) community! It is excited to see this proposal for Airflow as we are also

[airflow] branch v1-10-test updated: flake8 fixes

2020-06-19 Thread dimberman
This is an automated email from the ASF dual-hosted git repository. dimberman pushed a commit to branch v1-10-test in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/v1-10-test by this push: new 6678ac4 flake8 fixes 6678ac4 is

[GitHub] [airflow] mik-laj commented on pull request #9354: Task logging handlers can provide custom log links

2020-06-19 Thread GitBox
mik-laj commented on pull request #9354: URL: https://github.com/apache/airflow/pull/9354#issuecomment-646878976 TaskLogReader has been merged. Can you do.a rebase? This is an automated message from the Apache Git Service.

[GitHub] [airflow] mik-laj merged pull request #9402: Restore airflow.www.app.csrf to avoid breaking change

2020-06-19 Thread GitBox
mik-laj merged pull request #9402: URL: https://github.com/apache/airflow/pull/9402 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

[airflow] branch master updated (e587320 -> b23e380)

2020-06-19 Thread kamilbregula
This is an automated email from the ASF dual-hosted git repository. kamilbregula pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/airflow.git. from e587320 Warn about incompatible plugins (#9416) add b23e380 Restore airflow.www.app.csrf to avoid

[GitHub] [airflow] mik-laj commented on issue #9418: Deprecated AI Platform Operators and Runtimes in Example DAG

2020-06-19 Thread GitBox
mik-laj commented on issue #9418: URL: https://github.com/apache/airflow/issues/9418#issuecomment-646875666 I am very happy that you want to join this project. I assigned you to this ticket. I am the maintainer of this integration, so if you have questions, ask I look forward to your

[GitHub] [airflow] mik-laj opened a new issue #9420: Add note about secret backend in security.rst

2020-06-19 Thread GitBox
mik-laj opened a new issue #9420: URL: https://github.com/apache/airflow/issues/9420 Hello, A small note would be useful. Now we only have information about Fernet on this page. Best regards, Kamil CC: @kaxil

[GitHub] [airflow] takunnithan commented on issue #9112: API Endpoint - Batch

2020-06-19 Thread GitBox
takunnithan commented on issue #9112: URL: https://github.com/apache/airflow/issues/9112#issuecomment-646873195 @mik-laj Thanks! This is an automated message from the Apache Git Service. To respond to the message, please log

[GitHub] [airflow] mik-laj commented on issue #9112: API Endpoint - Batch

2020-06-19 Thread GitBox
mik-laj commented on issue #9112: URL: https://github.com/apache/airflow/issues/9112#issuecomment-646871158 ``POST /dags/~/dagRuns/~/taskInstances/list`` is still blocked, but you can start working on ``POST /dags/~/dagRuns/list``. I assigned you to this ticket.

[GitHub] [airflow] takunnithan commented on issue #9112: API Endpoint - Batch

2020-06-19 Thread GitBox
takunnithan commented on issue #9112: URL: https://github.com/apache/airflow/issues/9112#issuecomment-646870645 @mik-laj I would like to work on this. Can I take this as my first issue ? This is an automated message from the

[GitHub] [airflow] takunnithan commented on issue #9109: API Endpoint - CRUD - DAG

2020-06-19 Thread GitBox
takunnithan commented on issue #9109: URL: https://github.com/apache/airflow/issues/9109#issuecomment-646870395 @mik-laj Thanks! This is an automated message from the Apache Git Service. To respond to the message, please log

[GitHub] [airflow] mik-laj commented on issue #9109: API Endpoint - CRUD - DAG

2020-06-19 Thread GitBox
mik-laj commented on issue #9109: URL: https://github.com/apache/airflow/issues/9109#issuecomment-646869181 This ticket is blocked because it depends on read-only endpoints. - https://github.com/apache/airflow/issues/8128 These two tickets have a common scheme. You can check available

[airflow] branch v1-10-test updated: Merging multiple sql operators (#9124)

2020-06-19 Thread dimberman
This is an automated email from the ASF dual-hosted git repository. dimberman pushed a commit to branch v1-10-test in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/v1-10-test by this push: new 88eebcb Merging multiple sql

[GitHub] [airflow] mik-laj opened a new pull request #9419: Pylint fixes and deprecation of rare used methods in Connection

2020-06-19 Thread GitBox
mik-laj opened a new pull request #9419: URL: https://github.com/apache/airflow/pull/9419 I wanted to fix the pylint errors for connection, but I saw that there is a small mess here that would be worth cleaning up. All changes are backward compatible or not part of the end-user

[GitHub] [airflow] takunnithan commented on issue #9109: API Endpoint - CRUD - DAG

2020-06-19 Thread GitBox
takunnithan commented on issue #9109: URL: https://github.com/apache/airflow/issues/9109#issuecomment-646865546 @mik-laj Hi, Is this assigned to @bertosantamaria ? If not, I would like to work on this This is an automated

[GitHub] [airflow] mik-laj merged pull request #9416: Warn about incompatible plugins

2020-06-19 Thread GitBox
mik-laj merged pull request #9416: URL: https://github.com/apache/airflow/pull/9416 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

[airflow] branch master updated (05ea888 -> e587320)

2020-06-19 Thread kamilbregula
This is an automated email from the ASF dual-hosted git repository. kamilbregula pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/airflow.git. from 05ea888 add guidance re yarn build for local virtualenv development (#9411) add e587320 Warn about

[airflow] branch v1-10-test updated (a9dc0b5 -> 52f3796)

2020-06-19 Thread dimberman
This is an automated email from the ASF dual-hosted git repository. dimberman pushed a change to branch v1-10-test in repository https://gitbox.apache.org/repos/asf/airflow.git. from a9dc0b5 Add 'main' param to template_fields in DataprocSubmitPySparkJobOperator (#9154) add 52f3796

[airflow] branch v1-10-test updated (8c2f735 -> a9dc0b5)

2020-06-19 Thread dimberman
This is an automated email from the ASF dual-hosted git repository. dimberman pushed a change to branch v1-10-test in repository https://gitbox.apache.org/repos/asf/airflow.git. discard 8c2f735 Add 'main' param to template_fields in DataprocSubmitPySparkJobOperator (#9154) add a9dc0b5

[GitHub] [airflow] boring-cyborg[bot] commented on issue #9418: Deprecated AI Platform Operators and Runtimes in Example DAG

2020-06-19 Thread GitBox
boring-cyborg[bot] commented on issue #9418: URL: https://github.com/apache/airflow/issues/9418#issuecomment-646854465 Thanks for opening your first issue here! Be sure to follow the issue template! This is an automated

[GitHub] [airflow] takunnithan commented on issue #8494: TriggerDagOperator sometimes causes database to trigger UniqueViolation constraint

2020-06-19 Thread GitBox
takunnithan commented on issue #8494: URL: https://github.com/apache/airflow/issues/8494#issuecomment-646854532 @Lyalpha This is a better approach.  This is an automated message from the Apache Git Service. To respond to

[GitHub] [airflow] vuppalli opened a new issue #9418: Deprecated AI Platform Operators and Runtimes in Example DAG

2020-06-19 Thread GitBox
vuppalli opened a new issue #9418: URL: https://github.com/apache/airflow/issues/9418 **Apache Airflow version**: 1.10.10 **Kubernetes version (if you are using kubernetes)** (use `kubectl version`): **Environment**: - **Cloud provider or hardware

[GitHub] [airflow] svetli-n commented on a change in pull request #9338: Get API request limit and offset from parameters.py

2020-06-19 Thread GitBox
svetli-n commented on a change in pull request #9338: URL: https://github.com/apache/airflow/pull/9338#discussion_r443033237 ## File path: airflow/api_connexion/openapi/v1.yaml ## @@ -2134,6 +2134,7 @@ components: schema: type: integer minimum: 1 +

[GitHub] [airflow] Lyalpha commented on issue #8494: TriggerDagOperator sometimes causes database to trigger UniqueViolation constraint

2020-06-19 Thread GitBox
Lyalpha commented on issue #8494: URL: https://github.com/apache/airflow/issues/8494#issuecomment-646848561 I handled it with the following wrapper. Rather than changing the `execution_date`, just try again in a microsecond. (The micro-sleep is probably superfluous given there is an

[GitHub] [airflow] turbaszek edited a comment on issue #9415: [AIP-31] Update DAG examples to use Functional DAG style

2020-06-19 Thread GitBox
turbaszek edited a comment on issue #9415: URL: https://github.com/apache/airflow/issues/9415#issuecomment-646845554 Plus, should we update the information about XComs and the `.output` attribute? This is an automated

[GitHub] [airflow] turbaszek commented on issue #9415: [AIP-31] Update DAG examples to use Functional DAG style

2020-06-19 Thread GitBox
turbaszek commented on issue #9415: URL: https://github.com/apache/airflow/issues/9415#issuecomment-646845554 + should we update the information about XComs and the `.output` attribute? This is an automated message from the

[airflow] branch v1-10-test updated: Add 'main' param to template_fields in DataprocSubmitPySparkJobOperator (#9154)

2020-06-19 Thread dimberman
This is an automated email from the ASF dual-hosted git repository. dimberman pushed a commit to branch v1-10-test in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/v1-10-test by this push: new 8c2f735 Add 'main' param to

[airflow] branch v1-10-test updated (4950aaa -> 5b30d45)

2020-06-19 Thread dimberman
This is an automated email from the ASF dual-hosted git repository. dimberman pushed a change to branch v1-10-test in repository https://gitbox.apache.org/repos/asf/airflow.git. omit 4950aaa Add 'main' param to template_fields in DataprocSubmitPySparkJobOperator (#9154) This update

[GitHub] [airflow] mik-laj commented on pull request #9397: Add sensitive_variable_fields option to extend `DEFAULT_SENSITIVE_VARIABLE_FIELDS`

2020-06-19 Thread GitBox
mik-laj commented on pull request #9397: URL: https://github.com/apache/airflow/pull/9397#issuecomment-646835717 Can you also add docs to https://airflow.readthedocs.io/en/latest/security.html? This is an automated message

[GitHub] [airflow] mik-laj commented on a change in pull request #9397: Add sensitive_variable_fields option to extend `DEFAULT_SENSITIVE_VARIABLE_FIELDS`

2020-06-19 Thread GitBox
mik-laj commented on a change in pull request #9397: URL: https://github.com/apache/airflow/pull/9397#discussion_r443017078 ## File path: airflow/www/utils.py ## @@ -39,23 +39,28 @@ from airflow.www.forms import DateTimeWithTimezoneField from airflow.www.widgets import

[GitHub] [airflow] mik-laj commented on a change in pull request #9397: Add sensitive_variable_fields option to extend `DEFAULT_SENSITIVE_VARIABLE_FIELDS`

2020-06-19 Thread GitBox
mik-laj commented on a change in pull request #9397: URL: https://github.com/apache/airflow/pull/9397#discussion_r443016525 ## File path: airflow/config_templates/config.yml ## @@ -1678,6 +1678,13 @@ type: string example: ~ default: "True" +- name:

[GitHub] [airflow] mik-laj commented on a change in pull request #9397: Add sensitive_variable_fields option to extend `DEFAULT_SENSITIVE_VARIABLE_FIELDS`

2020-06-19 Thread GitBox
mik-laj commented on a change in pull request #9397: URL: https://github.com/apache/airflow/pull/9397#discussion_r443016525 ## File path: airflow/config_templates/config.yml ## @@ -1678,6 +1678,13 @@ type: string example: ~ default: "True" +- name:

[GitHub] [airflow] mik-laj opened a new issue #9417: YAML support in LocalFilesystemBackend

2020-06-19 Thread GitBox
mik-laj opened a new issue #9417: URL: https://github.com/apache/airflow/issues/9417 Hello, I would be happy if LocalFilesystemBackend would support YAML files. This file format is easier for human editing than JSON files, so it would be easier for developers to use. Tasks

[GitHub] [airflow] Marloxo commented on pull request #9397: Add sensitive_variable_fields option to extend `DEFAULT_SENSITIVE_VARIABLE_FIELDS`

2020-06-19 Thread GitBox
Marloxo commented on pull request #9397: URL: https://github.com/apache/airflow/pull/9397#issuecomment-646829722 @mik-laj I updated the PR with tests, would you please take a look. This is an automated message from the

[GitHub] [airflow] mik-laj commented on issue #9205: Add ability to use key files for Hooks development

2020-06-19 Thread GitBox
mik-laj commented on issue #9205: URL: https://github.com/apache/airflow/issues/9205#issuecomment-646829270 @sann05 Have you looked at how this problem was solved in SSHHook? This is an automated message from the Apache Git

[GitHub] [airflow] takunnithan commented on issue #8494: TriggerDagOperator sometimes causes database to trigger UniqueViolation constraint

2020-06-19 Thread GitBox
takunnithan commented on issue #8494: URL: https://github.com/apache/airflow/issues/8494#issuecomment-646824149 Ran into the same issue with REST API, We were trying to trigger a dag via experimental API (sometimes more than 30 DAG runs at a time). Tried setting the

[GitHub] [airflow] mik-laj opened a new pull request #9416: Warn about incompatible plugins

2020-06-19 Thread GitBox
mik-laj opened a new pull request #9416: URL: https://github.com/apache/airflow/pull/9416 I suggest you add a warning when the plugin has elements for non-RBAC UI, but at the same time, it has no elements for RBAC Web UI. --- Make sure to mark the boxes below before creating PR:

[GitHub] [airflow] casassg commented on a change in pull request #8962: [AIRFLOW-8057] [AIP-31] Add @task decorator

2020-06-19 Thread GitBox
casassg commented on a change in pull request #8962: URL: https://github.com/apache/airflow/pull/8962#discussion_r442974458 ## File path: docs/concepts.rst ## @@ -116,6 +116,46 @@ DAGs can be used as context managers to automatically assign new operators to th op.dag

[GitHub] [airflow] casassg opened a new issue #9415: [AIP-31] Update DAG examples to use Functional DAG style

2020-06-19 Thread GitBox
casassg opened a new issue #9415: URL: https://github.com/apache/airflow/issues/9415 **Description** Update Airflow documentation to use functional style DAGs instead. **Use case / motivation** As discussed in

[GitHub] [airflow] mustafagok commented on pull request #8888: Add AWS ECS system test

2020-06-19 Thread GitBox
mustafagok commented on pull request #: URL: https://github.com/apache/airflow/pull/#issuecomment-646779817 > @mustafagok yes looks good to me. Do you want to switch some of it over to `terraform`? I already integrated it (see #8877). But we can do this in another PR.

[airflow] branch v1-10-test updated: Add 'main' param to template_fields in DataprocSubmitPySparkJobOperator (#9154)

2020-06-19 Thread dimberman
This is an automated email from the ASF dual-hosted git repository. dimberman pushed a commit to branch v1-10-test in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/v1-10-test by this push: new 4950aaa Add 'main' param to

[GitHub] [airflow] aneesh-joseph commented on a change in pull request #9371: Add git sync option & helm unit tests

2020-06-19 Thread GitBox
aneesh-joseph commented on a change in pull request #9371: URL: https://github.com/apache/airflow/pull/9371#discussion_r442960495 ## File path: airflow/kubernetes/worker_configuration.py ## @@ -127,6 +127,10 @@ def _get_init_containers(self) -> List[k8s.V1Container]:

[GitHub] [airflow] potiuk commented on a change in pull request #9375: Remove redundant code from breeze initialization

2020-06-19 Thread GitBox
potiuk commented on a change in pull request #9375: URL: https://github.com/apache/airflow/pull/9375#discussion_r442964109 ## File path: scripts/ci/libraries/_initialization.sh ## @@ -59,8 +59,6 @@ function initialize_common_environment { # Create useful directories if

[GitHub] [airflow] aneesh-joseph commented on a change in pull request #9371: Add git sync option & helm unit tests

2020-06-19 Thread GitBox
aneesh-joseph commented on a change in pull request #9371: URL: https://github.com/apache/airflow/pull/9371#discussion_r442960495 ## File path: airflow/kubernetes/worker_configuration.py ## @@ -127,6 +127,10 @@ def _get_init_containers(self) -> List[k8s.V1Container]:

[jira] [Commented] (AIRFLOW-3973) `airflow initdb` logs errors when `Variable` is used in DAGs and Postgres is used for the internal database

2020-06-19 Thread ASF subversion and git services (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-3973?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17140717#comment-17140717 ] ASF subversion and git services commented on AIRFLOW-3973: -- Commit

[airflow] 02/02: Parameterized bash/python in the prod image (#9157)

2020-06-19 Thread dimberman
This is an automated email from the ASF dual-hosted git repository. dimberman pushed a commit to branch v1-10-test in repository https://gitbox.apache.org/repos/asf/airflow.git commit 5b30d4537cb39f9c6c4b9fd6acb69b498f7dc707 Author: Jarek Potiuk AuthorDate: Sat Jun 6 09:48:02 2020 +0200

[airflow] branch v1-10-test updated (ba645ec -> 5b30d45)

2020-06-19 Thread dimberman
This is an automated email from the ASF dual-hosted git repository. dimberman pushed a change to branch v1-10-test in repository https://gitbox.apache.org/repos/asf/airflow.git. from ba645ec [AIRFLOW-4472] Use json.dumps/loads for templating lineage data (#5253) new 5bc5018

[jira] [Commented] (AIRFLOW-3973) `airflow initdb` logs errors when `Variable` is used in DAGs and Postgres is used for the internal database

2020-06-19 Thread ASF subversion and git services (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-3973?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17140716#comment-17140716 ] ASF subversion and git services commented on AIRFLOW-3973: -- Commit

[airflow] 01/02: [AIRFLOW-3973] Commit after each alembic migration (#4797)

2020-06-19 Thread dimberman
This is an automated email from the ASF dual-hosted git repository. dimberman pushed a commit to branch v1-10-test in repository https://gitbox.apache.org/repos/asf/airflow.git commit 5bc50183e0934f7368d9cd991074b2b581114395 Author: Elliott Shugerman AuthorDate: Tue Jun 9 14:12:47 2020 -0600

[airflow] branch v1-10-test updated (ba645ec -> 5b30d45)

2020-06-19 Thread dimberman
This is an automated email from the ASF dual-hosted git repository. dimberman pushed a change to branch v1-10-test in repository https://gitbox.apache.org/repos/asf/airflow.git. from ba645ec [AIRFLOW-4472] Use json.dumps/loads for templating lineage data (#5253) new 5bc5018

[airflow] 02/02: Parameterized bash/python in the prod image (#9157)

2020-06-19 Thread dimberman
This is an automated email from the ASF dual-hosted git repository. dimberman pushed a commit to branch v1-10-test in repository https://gitbox.apache.org/repos/asf/airflow.git commit 5b30d4537cb39f9c6c4b9fd6acb69b498f7dc707 Author: Jarek Potiuk AuthorDate: Sat Jun 6 09:48:02 2020 +0200

[airflow] 01/02: [AIRFLOW-3973] Commit after each alembic migration (#4797)

2020-06-19 Thread dimberman
This is an automated email from the ASF dual-hosted git repository. dimberman pushed a commit to branch v1-10-test in repository https://gitbox.apache.org/repos/asf/airflow.git commit 5bc50183e0934f7368d9cd991074b2b581114395 Author: Elliott Shugerman AuthorDate: Tue Jun 9 14:12:47 2020 -0600

[GitHub] [airflow] mik-laj opened a new issue #9414: CLI environment variables reference

2020-06-19 Thread GitBox
mik-laj opened a new issue #9414: URL: https://github.com/apache/airflow/issues/9414 Hello, I think it's worth creating a list of environment variables used by Airflow. This will allow you to refer to it from other places in the documentation. We should use [envvar

[GitHub] [airflow] aneesh-joseph commented on a change in pull request #9371: Add git sync option & helm unit tests

2020-06-19 Thread GitBox
aneesh-joseph commented on a change in pull request #9371: URL: https://github.com/apache/airflow/pull/9371#discussion_r442960495 ## File path: airflow/kubernetes/worker_configuration.py ## @@ -127,6 +127,10 @@ def _get_init_containers(self) -> List[k8s.V1Container]:

  1   2   3   >