[GitHub] [airflow] github-actions[bot] commented on pull request #15022: Loosen cassandra-driver requirement to allow latest version

2021-03-26 Thread GitBox
github-actions[bot] commented on pull request #15022: URL: https://github.com/apache/airflow/pull/15022#issuecomment-808309920 [The Workflow run](https://github.com/apache/airflow/actions/runs/690376617) is cancelling this PR. It has some failed jobs matching ^Pylint$,^Static

[GitHub] [airflow] github-actions[bot] commented on pull request #15029: Merge contextlib.suppress() calls

2021-03-26 Thread GitBox
github-actions[bot] commented on pull request #15029: URL: https://github.com/apache/airflow/pull/15029#issuecomment-808310243 The PR most likely needs to run full matrix of tests because it modifies parts of the core of Airflow. However, committers might decide to merge it quickly and

[airflow] branch master updated (b1ce429 -> b5e7ada)

2021-03-26 Thread ash
This is an automated email from the ASF dual-hosted git repository. ash pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/airflow.git. from b1ce429 UI basic api (#15015) add b5e7ada Fix support for long dag_id and task_id in KubernetesExecutor

[GitHub] [airflow] ashb closed issue #14326: Task Instances stuck in "scheduled" state

2021-03-26 Thread GitBox
ashb closed issue #14326: URL: https://github.com/apache/airflow/issues/14326 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please

[GitHub] [airflow] ashb merged pull request #14703: Fix support for long dag_id and task_id in KubernetesExecutor

2021-03-26 Thread GitBox
ashb merged pull request #14703: URL: https://github.com/apache/airflow/pull/14703 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please

[airflow] branch v2-0-test updated: Fix support for long dag_id and task_id in KubernetesExecutor (#14703)

2021-03-26 Thread ash
This is an automated email from the ASF dual-hosted git repository. ash pushed a commit to branch v2-0-test in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/v2-0-test by this push: new ee1026f Fix support for long dag_id and

[GitHub] [airflow] SamWheating commented on pull request #14981: Allow override of project_id in DataprocJobBaseOperator

2021-03-26 Thread GitBox
SamWheating commented on pull request #14981: URL: https://github.com/apache/airflow/pull/14981#issuecomment-808302773 > please consider using `DataprocSubmitJobOperator`. All other operators like pig, spark etc are deprecated and will be removed one day. I'll pass this along to the

[GitHub] [airflow] ashb merged pull request #14977: Fix running "upgrade_check" command in a PTY.

2021-03-26 Thread GitBox
ashb merged pull request #14977: URL: https://github.com/apache/airflow/pull/14977 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please

[airflow] branch v1-10-stable updated: Fix running "upgrade_check" command in a PTY. (#14977)

2021-03-26 Thread ash
This is an automated email from the ASF dual-hosted git repository. ash pushed a commit to branch v1-10-stable in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/v1-10-stable by this push: new 4c3b31a Fix running "upgrade_check"

[GitHub] [airflow] uranusjr edited a comment on pull request #14978: Bump mysqlclient to support the 1.4.x series

2021-03-26 Thread GitBox
uranusjr edited a comment on pull request #14978: URL: https://github.com/apache/airflow/pull/14978#issuecomment-808239911 Updated. I should have checked and realised 2.x is available first! -- This is an automated message from the Apache Git Service. To respond to the message, please

[GitHub] [airflow] uranusjr opened a new pull request #15029: Merge contextlib.suppress() calls

2021-03-26 Thread GitBox
uranusjr opened a new pull request #15029: URL: https://github.com/apache/airflow/pull/15029 Noticed this while tracing some unrelated code. `contextlib.suppress()` can take multiple exception classes directly so there’s no need for two calls. --- **^ Add meaningful description

[GitHub] [airflow-site] gpodevijn opened a new pull request #398: Add Viewflow to the Ecosystem

2021-03-26 Thread GitBox
gpodevijn opened a new pull request #398: URL: https://github.com/apache/airflow-site/pull/398 DataCamp recently released Viewflow, a framework that allows data scientists to write data models/materialized views without writing Airflow code. See our [blog

[GitHub] [airflow] ashb closed pull request #14982: Recognize more ways to import DAG in ChainBetweenDAGAndOperatorNotAllowedRule

2021-03-26 Thread GitBox
ashb closed pull request #14982: URL: https://github.com/apache/airflow/pull/14982 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please

[GitHub] [airflow] ashb commented on pull request #14982: Recognize more ways to import DAG in ChainBetweenDAGAndOperatorNotAllowedRule

2021-03-26 Thread GitBox
ashb commented on pull request #14982: URL: https://github.com/apache/airflow/pull/14982#issuecomment-808181377 re-triggering ci. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

[GitHub] [airflow] github-actions[bot] commented on pull request #14978: Bump mysqlclient to support the 1.4.x series

2021-03-26 Thread GitBox
github-actions[bot] commented on pull request #14978: URL: https://github.com/apache/airflow/pull/14978#issuecomment-808243898 The PR most likely needs to run full matrix of tests because it modifies parts of the core of Airflow. However, committers might decide to merge it quickly and

[GitHub] [airflow] ashb commented on issue #14782: The scale-in settings are sligthly too aggressive I think

2021-03-26 Thread GitBox
ashb commented on issue #14782: URL: https://github.com/apache/airflow/issues/14782#issuecomment-808269754 Just noticed this happening a lot today, and (thanks to the extra logs) I am testing a change to when we mark a node as OkayToTerminate. -- This is an automated message from the

[GitHub] [airflow] ashb commented on a change in pull request #14978: Bump mysqlclient to support the 1.4.x and 2.x series

2021-03-26 Thread GitBox
ashb commented on a change in pull request #14978: URL: https://github.com/apache/airflow/pull/14978#discussion_r602372969 ## File path: docs/apache-airflow-providers-mysql/index.rst ## @@ -93,7 +93,7 @@ PIP requirements PIP package Version required

[GitHub] [airflow] github-actions[bot] commented on pull request #14982: Recognize more ways to import DAG in ChainBetweenDAGAndOperatorNotAllowedRule

2021-03-26 Thread GitBox
github-actions[bot] commented on pull request #14982: URL: https://github.com/apache/airflow/pull/14982#issuecomment-808184259 The PR most likely needs to run full matrix of tests because it modifies parts of the core of Airflow. However, committers might decide to merge it quickly and

[GitHub] [airflow] eladkal commented on a change in pull request #14978: Bump mysqlclient to support the 1.4.x and 2.x series

2021-03-26 Thread GitBox
eladkal commented on a change in pull request #14978: URL: https://github.com/apache/airflow/pull/14978#discussion_r602362154 ## File path: docs/apache-airflow-providers-mysql/index.rst ## @@ -93,7 +93,7 @@ PIP requirements PIP package Version required

[GitHub] [airflow] ashb closed issue #8237: dag will never start if we set a cron string with a clear day of week to the schedule_interval arg

2021-03-26 Thread GitBox
ashb closed issue #8237: URL: https://github.com/apache/airflow/issues/8237 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please

[GitHub] [airflow] ashb commented on issue #8237: dag will never start if we set a cron string with a clear day of week to the schedule_interval arg

2021-03-26 Thread GitBox
ashb commented on issue #8237: URL: https://github.com/apache/airflow/issues/8237#issuecomment-808325072 > I expect the DAG will execute twice (2020-04-11 11:52:00 and 2020-04-12 11:52:00). This is not currently how the scheduler operates -- the task is not scheduled until _after_

[GitHub] [airflow] eflemist commented on issue #14834: FileNotFoundError: [Errno 2] No such file or directory: 'airflow'

2021-03-26 Thread GitBox
eflemist commented on issue #14834: URL: https://github.com/apache/airflow/issues/14834#issuecomment-808274613 closing... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment.

[GitHub] [airflow] eflemist closed issue #14834: FileNotFoundError: [Errno 2] No such file or directory: 'airflow'

2021-03-26 Thread GitBox
eflemist closed issue #14834: URL: https://github.com/apache/airflow/issues/14834 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please

[GitHub] [airflow] github-actions[bot] commented on pull request #14968: Fix Sphinx Issues with Docstrings

2021-03-26 Thread GitBox
github-actions[bot] commented on pull request #14968: URL: https://github.com/apache/airflow/pull/14968#issuecomment-808278149 [The Workflow run](https://github.com/apache/airflow/actions/runs/690179007) is cancelling this PR. It has some failed jobs matching ^Pylint$,^Static

[GitHub] [airflow] ericpp commented on a change in pull request #14869: Fixed autocommit calls for mysql-connector-python

2021-03-26 Thread GitBox
ericpp commented on a change in pull request #14869: URL: https://github.com/apache/airflow/pull/14869#discussion_r602418206 ## File path: airflow/providers/mysql/hooks/mysql.py ## @@ -52,7 +52,10 @@ def __init__(self, *args, **kwargs) -> None: def set_autocommit(self,

[GitHub] [airflow] ericpp commented on a change in pull request #14869: Fixed autocommit calls for mysql-connector-python

2021-03-26 Thread GitBox
ericpp commented on a change in pull request #14869: URL: https://github.com/apache/airflow/pull/14869#discussion_r602418397 ## File path: airflow/providers/mysql/hooks/mysql.py ## @@ -63,7 +66,10 @@ def get_autocommit(self, conn: Connection) -> bool: # noqa: D403

[GitHub] [airflow] kapopken commented on a change in pull request #15002: Fix-wasb-read-to-return-decoded-string-instead-of-bytes

2021-03-26 Thread GitBox
kapopken commented on a change in pull request #15002: URL: https://github.com/apache/airflow/pull/15002#discussion_r602188331 ## File path: tests/providers/microsoft/azure/log/test_wasb_task_handler.py ## @@ -98,7 +98,7 @@ def test_wasb_log_exists(self, mock_hook):

[GitHub] [airflow] potiuk commented on issue #14989: Make Docs builds fallback in case external docs sources are missing

2021-03-26 Thread GitBox
potiuk commented on issue #14989: URL: https://github.com/apache/airflow/issues/14989#issuecomment-808194362 Looks great! @uranusjr ! I have another thought in the meantime. Why not using caching of GitHub Actions:

[GitHub] [airflow] potiuk edited a comment on issue #14989: Make Docs builds fallback in case external docs sources are missing

2021-03-26 Thread GitBox
potiuk edited a comment on issue #14989: URL: https://github.com/apache/airflow/issues/14989#issuecomment-808194362 Looks great! @uranusjr ! I have another thought in the meantime. Why not using caching of GitHub Actions:

[GitHub] [airflow] github-actions[bot] commented on pull request #14978: Bump mysqlclient to support the 1.4.x series

2021-03-26 Thread GitBox
github-actions[bot] commented on pull request #14978: URL: https://github.com/apache/airflow/pull/14978#issuecomment-808262639 [The Workflow run](https://github.com/apache/airflow/actions/runs/690179007) is cancelling this PR. Building images for the PR has failed. Follow the workflow

[GitHub] [airflow] ashb closed issue #8245: Add Option for BashOperator to Set State to SKIPPED

2021-03-26 Thread GitBox
ashb closed issue #8245: URL: https://github.com/apache/airflow/issues/8245 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please

[GitHub] [airflow] ashb commented on issue #8245: Add Option for BashOperator to Set State to SKIPPED

2021-03-26 Thread GitBox
ashb commented on issue #8245: URL: https://github.com/apache/airflow/issues/8245#issuecomment-808275371 Closed by https://github.com/apache/airflow/pull/13421 but exit code will be changed in https://github.com/apache/airflow/pull/14963 -- This is an automated message from the Apache

[GitHub] [airflow] ayushchauhan0811 commented on a change in pull request #13832: removing try-catch block to fix timeout exception getting ignored in aws batch operator

2021-03-26 Thread GitBox
ayushchauhan0811 commented on a change in pull request #13832: URL: https://github.com/apache/airflow/pull/13832#discussion_r602401001 ## File path: airflow/providers/amazon/aws/operators/batch.py ## @@ -177,29 +177,26 @@ def submit_job(self, context: Dict): # pylint:

[GitHub] [airflow] ashb commented on pull request #14978: Bump mysqlclient to support the 1.4.x series

2021-03-26 Thread GitBox
ashb commented on pull request #14978: URL: https://github.com/apache/airflow/pull/14978#issuecomment-808180248 @uranusjr May be worth updating the dep to `<3` now actually. The only breaking change form 1.x to 2.x was removal of of py2 support

[GitHub] [airflow] uranusjr commented on pull request #14978: Bump mysqlclient to support the 1.4.x series

2021-03-26 Thread GitBox
uranusjr commented on pull request #14978: URL: https://github.com/apache/airflow/pull/14978#issuecomment-808239911 Updated. I should have checked the version compatibility first! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [airflow] uranusjr commented on issue #14989: Make Docs builds fallback in case external docs sources are missing

2021-03-26 Thread GitBox
uranusjr commented on issue #14989: URL: https://github.com/apache/airflow/issues/14989#issuecomment-808267669 Using GitHub Action’s cache sounds like a cool idea. The only concern I have is the cache can be pretty difficult to purge if something goes wrong (maybe just me not knowing a

[GitHub] [airflow] ashb commented on a change in pull request #14883: Fix celery executor bug trying to call len on map

2021-03-26 Thread GitBox
ashb commented on a change in pull request #14883: URL: https://github.com/apache/airflow/pull/14883#discussion_r602394746 ## File path: tests/executors/test_celery_executor.py ## @@ -469,3 +470,23 @@ def test_should_support_base_backend(self): )

[airflow-client-go] branch master updated: Prepare for rename default branch to main (#8)

2021-03-26 Thread ash
This is an automated email from the ASF dual-hosted git repository. ash pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/airflow-client-go.git The following commit(s) were added to refs/heads/master by this push: new 275741a Prepare for rename default

[GitHub] [airflow-client-go] ashb merged pull request #8: Prepare for rename default branch to main

2021-03-26 Thread GitBox
ashb merged pull request #8: URL: https://github.com/apache/airflow-client-go/pull/8 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service,

[GitHub] [airflow] uranusjr commented on issue #14989: Make Docs builds fallback in case external docs sources are missing

2021-03-26 Thread GitBox
uranusjr commented on issue #14989: URL: https://github.com/apache/airflow/issues/14989#issuecomment-808280458 I just thought of another thing, does Action’s cache set file creation date etc. correctly? Because if it does not, we won’t be able to fetch in new documentation updates (say

[GitHub] [airflow] ashb commented on issue #14660: KeyError template_fields does not exists when get_template_context

2021-03-26 Thread GitBox
ashb commented on issue #14660: URL: https://github.com/apache/airflow/issues/14660#issuecomment-808358088 Oh actually, this may just be a "bug"/change needed in your tests for 2.0: ``` test_operator.prepare_for_execution().execute(ti.get_template_context()) ```

[GitHub] [airflow] ashb closed issue #14660: KeyError template_fields does not exists when get_template_context

2021-03-26 Thread GitBox
ashb closed issue #14660: URL: https://github.com/apache/airflow/issues/14660 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please

[GitHub] [airflow] jedcunningham commented on a change in pull request #15017: Faster default role syncing during webserver start

2021-03-26 Thread GitBox
jedcunningham commented on a change in pull request #15017: URL: https://github.com/apache/airflow/pull/15017#discussion_r602442775 ## File path: airflow/www/security.py ## @@ -171,12 +172,26 @@ def __init__(self, appbuilder): view.datamodel =

[GitHub] [airflow] ashb commented on pull request #15022: Loosen cassandra-driver requirement to allow latest version

2021-03-26 Thread GitBox
ashb commented on pull request #15022: URL: https://github.com/apache/airflow/pull/15022#issuecomment-808368220 @potiuk How do I make this rebuild with latest dependencies? Right now this is testing against the old cassandra-driver dep because of the Constraints. I thought there was some

[airflow] branch master updated (b5e7ada -> 7320037)

2021-03-26 Thread xddeng
This is an automated email from the ASF dual-hosted git repository. xddeng pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/airflow.git. from b5e7ada Fix support for long dag_id and task_id in KubernetesExecutor (#14703) add 7320037 Merge

[GitHub] [airflow] github-actions[bot] commented on pull request #15033: Check if we need to upgrade deps before rebuilding everything

2021-03-26 Thread GitBox
github-actions[bot] commented on pull request #15033: URL: https://github.com/apache/airflow/pull/15033#issuecomment-808377319 The PR most likely needs to run full matrix of tests because it modifies parts of the core of Airflow. However, committers might decide to merge it quickly and

[GitHub] [airflow] ashb merged pull request #15033: Check if we need to upgrade deps before rebuilding everything

2021-03-26 Thread GitBox
ashb merged pull request #15033: URL: https://github.com/apache/airflow/pull/15033 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please

[GitHub] [airflow] ashb edited a comment on issue #13934: EMR URI Parse Fails to resolve nested dict objects

2021-03-26 Thread GitBox
ashb edited a comment on issue #13934: URL: https://github.com/apache/airflow/issues/13934#issuecomment-808389284 I think generally the extra should be encoded as query-string-escaped json, so: ```

[airflow] branch master updated (c5cc018 -> fe2a70c)

2021-03-26 Thread ash
This is an automated email from the ASF dual-hosted git repository. ash pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/airflow.git. from c5cc018 Check if we need to upgrade deps before rebuilding everything (#15033) add fe2a70c Bump mysqlclient to

[GitHub] [airflow] github-actions[bot] commented on pull request #15022: Loosen cassandra-driver requirement to allow latest version

2021-03-26 Thread GitBox
github-actions[bot] commented on pull request #15022: URL: https://github.com/apache/airflow/pull/15022#issuecomment-808396778 [The Workflow run](https://github.com/apache/airflow/actions/runs/690700370) is cancelling this PR. Building images for the PR has failed. Follow the workflow

[GitHub] [airflow] dimberman commented on a change in pull request #15034: Taskgroup decorator

2021-03-26 Thread GitBox
dimberman commented on a change in pull request #15034: URL: https://github.com/apache/airflow/pull/15034#discussion_r602510578 ## File path: airflow/decorators/task_group.py ## @@ -0,0 +1,78 @@ +# +# Licensed to the Apache Software Foundation (ASF) under one +# or more

[GitHub] [airflow] ashb commented on a change in pull request #15037: Batch send to not overload multiprocessing pipe

2021-03-26 Thread GitBox
ashb commented on a change in pull request #15037: URL: https://github.com/apache/airflow/pull/15037#discussion_r602538992 ## File path: airflow/jobs/scheduler_job.py ## @@ -1245,6 +1252,9 @@ def _process_executor_events(self, session: Session = None) -> int:

[GitHub] [airflow] github-actions[bot] commented on pull request #14869: Fixed autocommit calls for mysql-connector-python

2021-03-26 Thread GitBox
github-actions[bot] commented on pull request #14869: URL: https://github.com/apache/airflow/pull/14869#issuecomment-808469194 The PR is likely OK to be merged with just subset of tests for default Python and Database versions without running the full matrix of tests, because it does not

[GitHub] [airflow] ashb edited a comment on pull request #14978: Bump mysqlclient to support the 1.4.x and 2.x series

2021-03-26 Thread GitBox
ashb edited a comment on pull request #14978: URL: https://github.com/apache/airflow/pull/14978#issuecomment-808475436 A bug in our CI might have meant that these tests were run against the _old_ dep, and as a result one of the GCS tests fails because it tried to do: ``` from

[GitHub] [airflow] mik-laj edited a comment on issue #14989: Make Docs builds fallback in case external docs sources are missing

2021-03-26 Thread GitBox
mik-laj edited a comment on issue #14989: URL: https://github.com/apache/airflow/issues/14989#issuecomment-808448643 > As I understand it the inventory we are using now simply downloads URL and only if it fails, it reads it from the cache @mik-laj might confirm it maybe? Not

[GitHub] [airflow] jedcunningham commented on a change in pull request #15017: Faster default role syncing during webserver start

2021-03-26 Thread GitBox
jedcunningham commented on a change in pull request #15017: URL: https://github.com/apache/airflow/pull/15017#discussion_r602438881 ## File path: airflow/www/security.py ## @@ -467,6 +473,34 @@ def get_all_permissions(self) -> Set[Tuple[str, str]]: .all()

[jira] [Commented] (AIRFLOW-5071) Thousand os Executor reports task instance X finished (success) although the task says its queued. Was the task killed externally?

2021-03-26 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-5071?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17309539#comment-17309539 ] ASF GitHub Bot commented on AIRFLOW-5071: - easthy-alterpost commented on issue #10790: URL:

[GitHub] [airflow] jedcunningham commented on a change in pull request #15017: Faster default role syncing during webserver start

2021-03-26 Thread GitBox
jedcunningham commented on a change in pull request #15017: URL: https://github.com/apache/airflow/pull/15017#discussion_r602438708 ## File path: airflow/www/security.py ## @@ -171,19 +171,25 @@ def __init__(self, appbuilder): view.datamodel =

[GitHub] [airflow] Jake-Gillberg commented on issue #14838: local_executor failes when password contains `&` in SQL_ALCHEMY_CONN_CMD

2021-03-26 Thread GitBox
Jake-Gillberg commented on issue #14838: URL: https://github.com/apache/airflow/issues/14838#issuecomment-808364008 It is exactly the tutorial task / dag: https://airflow.apache.org/docs/apache-airflow/stable/_modules/airflow/example_dags/tutorial.html The problem isn't with running

[GitHub] [airflow] ashb edited a comment on issue #13934: EMR URI Parse Fails to resolve nested dict objects

2021-03-26 Thread GitBox
ashb edited a comment on issue #13934: URL: https://github.com/apache/airflow/issues/13934#issuecomment-808389284 I think generally the extra should be encoded as query-string-escaped json, so: ```

[GitHub] [airflow] ashb commented on a change in pull request #15034: Taskgroup decorator

2021-03-26 Thread GitBox
ashb commented on a change in pull request #15034: URL: https://github.com/apache/airflow/pull/15034#discussion_r602497237 ## File path: airflow/decorators/task_group.py ## @@ -0,0 +1,78 @@ +# +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor

[GitHub] [airflow] boring-cyborg[bot] commented on issue #15036: Scheduler is skipping a day sometimes

2021-03-26 Thread GitBox
boring-cyborg[bot] commented on issue #15036: URL: https://github.com/apache/airflow/issues/15036#issuecomment-808429102 Thanks for opening your first issue here! Be sure to follow the issue template! -- This is an automated message from the Apache Git Service. To respond to the

[GitHub] [airflow] iameugenejo opened a new issue #15036: Scheduler is skipping a day sometimes

2021-03-26 Thread GitBox
iameugenejo opened a new issue #15036: URL: https://github.com/apache/airflow/issues/15036 I'm having an exactly same issue as this user - https://www.reddit.com/r/dataengineering/comments/lri9fv/airflow_dag_is_skipping_a_day/ The version I'm using is 2.0.1 ![Screen Shot

[GitHub] [airflow] dimberman commented on a change in pull request #15034: Taskgroup decorator

2021-03-26 Thread GitBox
dimberman commented on a change in pull request #15034: URL: https://github.com/apache/airflow/pull/15034#discussion_r602510248 ## File path: airflow/decorators/task_group.py ## @@ -0,0 +1,78 @@ +# +# Licensed to the Apache Software Foundation (ASF) under one +# or more

[GitHub] [airflow] mik-laj commented on issue #14989: Make Docs builds fallback in case external docs sources are missing

2021-03-26 Thread GitBox
mik-laj commented on issue #14989: URL: https://github.com/apache/airflow/issues/14989#issuecomment-808448643 > As I understand it the inventory we are using now simply downloads URL and only if it fails, it reads it from the cache @mik-laj might confirm it maybe? Not exactly.

[GitHub] [airflow] potiuk commented on pull request #15022: Loosen cassandra-driver requirement to allow latest version

2021-03-26 Thread GitBox
potiuk commented on pull request #15022: URL: https://github.com/apache/airflow/pull/15022#issuecomment-808462857 Just a word of warning here - this might become another blocker if we finally manage Python 3.9 working. From what I see there is no 3.9 wheel (and even the driver itself does

[GitHub] [airflow] potiuk commented on pull request #15022: Loosen cassandra-driver requirement to allow latest version

2021-03-26 Thread GitBox
potiuk commented on pull request #15022: URL: https://github.com/apache/airflow/pull/15022#issuecomment-808466017 > Looks like 3.9 support at all may be questionable for Cassandra https://datastax-oss.atlassian.net/browse/PYTHON-1246 This is not a full-blocker in general. I'd love

[GitHub] [airflow] ashb commented on issue #14955: Custom macros not working on Airflow 2.0.1 Python 3.9

2021-03-26 Thread GitBox
ashb commented on issue #14955: URL: https://github.com/apache/airflow/issues/14955#issuecomment-808342481 Macros are only designed for use in Jinja templates, not for calling directly from Python code as you are trying. In your case you to import it directly from the plugin:

[GitHub] [airflow] easthy-alterpost commented on issue #10790: Copy of [AIRFLOW-5071] JIRA: Thousands of Executor reports task instance X finished (success) although the task says its queued. Was the

2021-03-26 Thread GitBox
easthy-alterpost commented on issue #10790: URL: https://github.com/apache/airflow/issues/10790#issuecomment-808355662 Trying Airflow 2.0.1. No tasks could be executed :( `scheduler_1 | [2021-03-26 16:25:55,097] {{scheduler_job.py:941}} INFO - 1 tasks up for execution:

[GitHub] [airflow] easthy-alterpost edited a comment on issue #10790: Copy of [AIRFLOW-5071] JIRA: Thousands of Executor reports task instance X finished (success) although the task says its queued. W

2021-03-26 Thread GitBox
easthy-alterpost edited a comment on issue #10790: URL: https://github.com/apache/airflow/issues/10790#issuecomment-808355662 Trying Airflow 2.0.1. No tasks could be executed :( ``` scheduler_1 | [2021-03-26 16:25:55,097] {{scheduler_job.py:941}} INFO - 1 tasks up for execution:

[GitHub] [airflow] dstandish commented on a change in pull request #14869: Fixed autocommit calls for mysql-connector-python

2021-03-26 Thread GitBox
dstandish commented on a change in pull request #14869: URL: https://github.com/apache/airflow/pull/14869#discussion_r602459790 ## File path: airflow/providers/mysql/hooks/mysql.py ## @@ -52,7 +52,10 @@ def __init__(self, *args, **kwargs) -> None: def

[GitHub] [airflow] ashb merged pull request #14978: Bump mysqlclient to support the 1.4.x and 2.x series

2021-03-26 Thread GitBox
ashb merged pull request #14978: URL: https://github.com/apache/airflow/pull/14978 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please

[GitHub] [airflow] ashb closed issue #8292: Bump supported mysqlclient to <1.5

2021-03-26 Thread GitBox
ashb closed issue #8292: URL: https://github.com/apache/airflow/issues/8292 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please

[GitHub] [airflow] github-actions[bot] commented on pull request #15017: Faster default role syncing during webserver start

2021-03-26 Thread GitBox
github-actions[bot] commented on pull request #15017: URL: https://github.com/apache/airflow/pull/15017#issuecomment-808395503 The PR is likely OK to be merged with just subset of tests for default Python and Database versions without running the full matrix of tests, because it does not

[GitHub] [airflow] ericpp commented on pull request #14869: Fixed autocommit calls for mysql-connector-python

2021-03-26 Thread GitBox
ericpp commented on pull request #14869: URL: https://github.com/apache/airflow/pull/14869#issuecomment-808395665 Would you like me to squash my commits when this is ready to be merged? I added a bunch of development commits which are probably not useful to have in the main codebase. --

[GitHub] [airflow] ashb commented on issue #14782: The scale-in settings are sligthly too aggressive I think

2021-03-26 Thread GitBox
ashb commented on issue #14782: URL: https://github.com/apache/airflow/issues/14782#issuecomment-808395294 Let me know if you notice any more instances now. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL

[GitHub] [airflow] tpilewicz opened a new pull request #15035: Add missing comma

2021-03-26 Thread GitBox
tpilewicz opened a new pull request #15035: URL: https://github.com/apache/airflow/pull/15035 A comma seems to be missing on the page documenting the KubernetesExecutor -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use

[GitHub] [airflow] dstandish commented on pull request #14869: Fixed autocommit calls for mysql-connector-python

2021-03-26 Thread GitBox
dstandish commented on pull request #14869: URL: https://github.com/apache/airflow/pull/14869#issuecomment-808443751 > @dstandish Don't we need mysql.connector-specific tests for `test_run_with_autocommit` and ` test_run_without_autocommit` ? Should those be handled separately from this

[airflow] branch v2-0-test updated: Prepare ad-hoc release of the four previously excluded providers (#14655)

2021-03-26 Thread potiuk
This is an automated email from the ASF dual-hosted git repository. potiuk pushed a commit to branch v2-0-test in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/v2-0-test by this push: new 3554370 Prepare ad-hoc release of the

[GitHub] [airflow] mik-laj edited a comment on issue #14989: Make Docs builds fallback in case external docs sources are missing

2021-03-26 Thread GitBox
mik-laj edited a comment on issue #14989: URL: https://github.com/apache/airflow/issues/14989#issuecomment-808448643 > As I understand it the inventory we are using now simply downloads URL and only if it fails, it reads it from the cache @mik-laj might confirm it maybe? Not

[GitHub] [airflow] potiuk commented on issue #15028: Don't remove "full tests required" label when PR is created with it

2021-03-26 Thread GitBox
potiuk commented on issue #15028: URL: https://github.com/apache/airflow/issues/15028#issuecomment-808451865 It gets removed when the Image is not in "removed" state The current process is that label is not really supposed to be "manually" set - it is set automatically when we

[GitHub] [airflow] mik-laj edited a comment on issue #14989: Make Docs builds fallback in case external docs sources are missing

2021-03-26 Thread GitBox
mik-laj edited a comment on issue #14989: URL: https://github.com/apache/airflow/issues/14989#issuecomment-808448643 > As I understand it the inventory we are using now simply downloads URL and only if it fails, it reads it from the cache @mik-laj might confirm it maybe? Not

[GitHub] [airflow] mik-laj edited a comment on issue #14989: Make Docs builds fallback in case external docs sources are missing

2021-03-26 Thread GitBox
mik-laj edited a comment on issue #14989: URL: https://github.com/apache/airflow/issues/14989#issuecomment-808448643 > As I understand it the inventory we are using now simply downloads URL and only if it fails, it reads it from the cache @mik-laj might confirm it maybe? Not

[GitHub] [airflow] ashb commented on pull request #15022: Loosen cassandra-driver requirement to allow latest version

2021-03-26 Thread GitBox
ashb commented on pull request #15022: URL: https://github.com/apache/airflow/pull/15022#issuecomment-808464533 Looks like 3.9 support at all may be questionable for Cassandra https://datastax-oss.atlassian.net/browse/PYTHON-1246 -- This is an automated message from the Apache Git

[GitHub] [airflow] boring-cyborg[bot] commented on issue #15032: Probléme d'accés user et pass

2021-03-26 Thread GitBox
boring-cyborg[bot] commented on issue #15032: URL: https://github.com/apache/airflow/issues/15032#issuecomment-808345120 Thanks for opening your first issue here! Be sure to follow the issue template! -- This is an automated message from the Apache Git Service. To respond to the

[GitHub] [airflow] yacine-del opened a new issue #15032: Probléme d'accés user et pass

2021-03-26 Thread GitBox
yacine-del opened a new issue #15032: URL: https://github.com/apache/airflow/issues/15032

[GitHub] [airflow] yacine-del commented on issue #15032: Probléme d'accés user et pass

2021-03-26 Thread GitBox
yacine-del commented on issue #15032: URL: https://github.com/apache/airflow/issues/15032#issuecomment-808354081 Ci-dessous le message d'ERR sqlalchemy.orm.exc.DetachedInstanceError: Parent instance is not bound to a Session; lazy load operation of attribute 'view_menu' cannot

[GitHub] [airflow] ashb commented on a change in pull request #14946: Standardize default fab perms

2021-03-26 Thread GitBox
ashb commented on a change in pull request #14946: URL: https://github.com/apache/airflow/pull/14946#discussion_r602457448 ## File path: airflow/migrations/versions/a13f7613ad25_resource_based_permissions_for_default_.py ## @@ -0,0 +1,172 @@ +# +# Licensed to the Apache

[GitHub] [airflow] marcosmarxm commented on pull request #15012: PostgreSQL operator returns query result

2021-03-26 Thread GitBox
marcosmarxm commented on pull request #15012: URL: https://github.com/apache/airflow/pull/15012#issuecomment-808395191 The `DBApiHook` **run** hasn't any return. Also reading the documentation: ``` Runs a command or a list of commands. Pass a list of sql statements to the

[airflow] branch master updated (7320037 -> c5cc018)

2021-03-26 Thread ash
This is an automated email from the ASF dual-hosted git repository. ash pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/airflow.git. from 7320037 Merge contextlib.suppress() calls (#15029) add c5cc018 Check if we need to upgrade deps before

[GitHub] [airflow] marcosmarxm edited a comment on pull request #15012: PostgreSQL operator returns query result

2021-03-26 Thread GitBox
marcosmarxm edited a comment on pull request #15012: URL: https://github.com/apache/airflow/pull/15012#issuecomment-808395191 The `DBApiHook` **run** doesn't return anything. Reading the documentation: ``` Runs a command or a list of commands. Pass a list of sql

[GitHub] [airflow] potiuk edited a comment on issue #14989: Make Docs builds fallback in case external docs sources are missing

2021-03-26 Thread GitBox
potiuk edited a comment on issue #14989: URL: https://github.com/apache/airflow/issues/14989#issuecomment-808438815 > Using GitHub Action’s cache sounds like a cool idea. The only concern I have is the cache can be pretty difficult to purge if something goes wrong (maybe just me not

[GitHub] [airflow] uranusjr opened a new pull request #15037: Batch send to not overload multiprocessing pipe

2021-03-26 Thread GitBox
uranusjr opened a new pull request #15037: URL: https://github.com/apache/airflow/pull/15037 This breaks the loops sending callbacks to the multiprocessing pipe into size 100 batches, and call `DagFileProcessorAgent.heartbeat()` after each pipe (which would call `Pipe.recv()`) to consume

[GitHub] [airflow] potiuk edited a comment on issue #14989: Make Docs builds fallback in case external docs sources are missing

2021-03-26 Thread GitBox
potiuk edited a comment on issue #14989: URL: https://github.com/apache/airflow/issues/14989#issuecomment-808438815 > Using GitHub Action’s cache sounds like a cool idea. The only concern I have is the cache can be pretty difficult to purge if something goes wrong (maybe just me not

[GitHub] [airflow] potiuk commented on pull request #15033: Check if we need to upgrade deps before rebuilding everything

2021-03-26 Thread GitBox
potiuk commented on pull request #15033: URL: https://github.com/apache/airflow/pull/15033#issuecomment-808442543 Oh yeah :facepalm: -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

[GitHub] [airflow] mik-laj edited a comment on issue #14989: Make Docs builds fallback in case external docs sources are missing

2021-03-26 Thread GitBox
mik-laj edited a comment on issue #14989: URL: https://github.com/apache/airflow/issues/14989#issuecomment-808448643 > As I understand it the inventory we are using now simply downloads URL and only if it fails, it reads it from the cache @mik-laj might confirm it maybe? Not

[GitHub] [airflow] mik-laj edited a comment on issue #14989: Make Docs builds fallback in case external docs sources are missing

2021-03-26 Thread GitBox
mik-laj edited a comment on issue #14989: URL: https://github.com/apache/airflow/issues/14989#issuecomment-808448643 > As I understand it the inventory we are using now simply downloads URL and only if it fails, it reads it from the cache @mik-laj might confirm it maybe? Not

[GitHub] [airflow] ashb commented on a change in pull request #15037: Batch send to not overload multiprocessing pipe

2021-03-26 Thread GitBox
ashb commented on a change in pull request #15037: URL: https://github.com/apache/airflow/pull/15037#discussion_r602534318 ## File path: airflow/jobs/scheduler_job.py ## @@ -1245,6 +1252,9 @@ def _process_executor_events(self, session: Session = None) -> int:

[GitHub] [airflow] ashb edited a comment on pull request #15022: Loosen cassandra-driver requirement to allow latest version

2021-03-26 Thread GitBox
ashb edited a comment on pull request #15022: URL: https://github.com/apache/airflow/pull/15022#issuecomment-808464533 Looks like 3.9 support at all may be questionable for Cassandra https://datastax-oss.atlassian.net/browse/PYTHON-1246 There is no explicit `python_requires` in

[GitHub] [airflow] ashb commented on pull request #14978: Bump mysqlclient to support the 1.4.x and 2.x series

2021-03-26 Thread GitBox
ashb commented on pull request #14978: URL: https://github.com/apache/airflow/pull/14978#issuecomment-808475436 A bug in our CI might have meant that these tests were run against the _old_ dep, and as a result one of the GCS tests fails because it tried to do: ``` from

  1   2   3   >