[GitHub] [airflow] potiuk commented on pull request #17400: Example dag slackfile

2021-08-03 Thread GitBox
potiuk commented on pull request #17400: URL: https://github.com/apache/airflow/pull/17400#issuecomment-892378601 some docs and test failures -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

[GitHub] [airflow] github-actions[bot] commented on pull request #17400: Example dag slackfile

2021-08-03 Thread GitBox
github-actions[bot] commented on pull request #17400: URL: https://github.com/apache/airflow/pull/17400#issuecomment-892378608 The PR is likely OK to be merged with just subset of tests for default Python and Database versions without running the full matrix of tests, because it does not

[GitHub] [airflow] josh-fell commented on pull request #17399: Adding JWT, IP filtering, and direct session login support for SalesforceHook

2021-08-03 Thread GitBox
josh-fell commented on pull request #17399: URL: https://github.com/apache/airflow/pull/17399#issuecomment-892325412 @eladkal FYI if you'd like to review. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above

[GitHub] [airflow] unixfree opened a new issue #17401: users/list error

2021-08-03 Thread GitBox
unixfree opened a new issue #17401: URL: https://github.com/apache/airflow/issues/17401 **Apache Airflow version**: **Kubernetes version (if you are using kubernetes)** (use `kubectl version`): **Environment**: - **Cloud provider or hardware

[GitHub] [airflow] boring-cyborg[bot] commented on issue #17401: users/list error

2021-08-03 Thread GitBox
boring-cyborg[bot] commented on issue #17401: URL: https://github.com/apache/airflow/issues/17401#issuecomment-892325011 Thanks for opening your first issue here! Be sure to follow the issue template! -- This is an automated message from the Apache Git Service. To respond to the

[GitHub] [airflow-client-python] er1shivam commented on issue #27: get_dag_details api is broken

2021-08-03 Thread GitBox
er1shivam commented on issue #27: URL: https://github.com/apache/airflow-client-python/issues/27#issuecomment-892322524 I will share it in few hours when I log back in. But as you said regarding dag_run_timeout, I haven't defined this in my dag as it is not a REQUIRED parameter. So I

[GitHub] [airflow] github-actions[bot] commented on pull request #15042: Add REST API auth flow for the new UI

2021-08-03 Thread GitBox
github-actions[bot] commented on pull request #15042: URL: https://github.com/apache/airflow/pull/15042#issuecomment-892248347 This pull request has been automatically marked as stale because it has not had recent activity. It will be closed in 5 days if no further activity occurs. Thank

[GitHub] [airflow] mariotaddeucci edited a comment on pull request #16915: New generic tableau operator: TableauOperator

2021-08-03 Thread GitBox
mariotaddeucci edited a comment on pull request #16915: URL: https://github.com/apache/airflow/pull/16915#issuecomment-892245415 @eladkal @potiuk the unit tests have run successfully now ✅ -- This is an automated message from the Apache Git Service. To respond to the message, please log

[GitHub] [airflow] mariotaddeucci commented on pull request #16915: New generic tableau operator: TableauOperator

2021-08-03 Thread GitBox
mariotaddeucci commented on pull request #16915: URL: https://github.com/apache/airflow/pull/16915#issuecomment-892245415 @eladkal @potiuk the unit tests have run successfully now  -- This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [airflow] subkanthi opened a new pull request #17400: Example dag slackfile

2021-08-03 Thread GitBox
subkanthi opened a new pull request #17400: URL: https://github.com/apache/airflow/pull/17400 closes: #17368 - Fixed bug with SlackAPIFileOperator with uploading file. - Added example dag to demonstrate support for uploading file and file content. -- This is an automated message

[GitHub] [airflow] fmv1992 commented on issue #13637: Scheduler takes 100% of CPU without task execution

2021-08-03 Thread GitBox
fmv1992 commented on issue #13637: URL: https://github.com/apache/airflow/issues/13637#issuecomment-892225483 In our case we get n processes determined by `parsing_processes`. `top` gives: ``` airflow scheduler - DagFileProcessor /mydags/dag1.py ``` All of them with a

[GitHub] [airflow] jedcunningham commented on issue #17398: With Helm Chart, get CSRF error if setting webserver.replicas > 1

2021-08-03 Thread GitBox
jedcunningham commented on issue #17398: URL: https://github.com/apache/airflow/issues/17398#issuecomment-892200174 Thanks @smythie86. Unfortunately the naive approach of just adding annotations where necessary isn't enough to fix this (it would cause worker restarts on any `helm

[GitHub] [airflow] smythie86 commented on issue #17398: With Helm Chart, get CSRF error if setting webserver.replicas > 1

2021-08-03 Thread GitBox
smythie86 commented on issue #17398: URL: https://github.com/apache/airflow/issues/17398#issuecomment-892176505 Yes, I confirmed that I am running 1.1.0, and that the AIRFLOW__WEBSERVER__SECRET_KEY env variable is set. After more digging, I discovered that this bug doesn't happen on

[GitHub] [airflow] raphaelauv commented on issue #17035: KubernetesPodOperator 'pod_template_file' template not found, after upgrading 2.1.0 -> 2.1.2

2021-08-03 Thread GitBox
raphaelauv commented on issue #17035: URL: https://github.com/apache/airflow/issues/17035#issuecomment-892159335 check this https://github.com/apache/airflow/issues/16922#issuecomment-892153152 -- This is an automated message from the Apache Git Service. To respond to the message,

[GitHub] [airflow] raphaelauv edited a comment on issue #16922: Using the string ".json" in a dag makes KubernetesPodOperator worker unable to trigger the pod

2021-08-03 Thread GitBox
raphaelauv edited a comment on issue #16922: URL: https://github.com/apache/airflow/issues/16922#issuecomment-892153152 @louison @ericdevries ```python class MyKubernetesPodOperator(KubernetesPodOperator): template_fields = tuple(x for x in

[GitHub] [airflow] raphaelauv edited a comment on issue #16922: Using the string ".json" in a dag makes KubernetesPodOperator worker unable to trigger the pod

2021-08-03 Thread GitBox
raphaelauv edited a comment on issue #16922: URL: https://github.com/apache/airflow/issues/16922#issuecomment-892153152 @louison @ericdevries ```python class MyKubernetesPodOperator(KubernetesPodOperator): template_fields = tuple(x for x in

[GitHub] [airflow] raphaelauv edited a comment on issue #16922: Using the string ".json" in a dag makes KubernetesPodOperator worker unable to trigger the pod

2021-08-03 Thread GitBox
raphaelauv edited a comment on issue #16922: URL: https://github.com/apache/airflow/issues/16922#issuecomment-892153152 @louison @ericdevries ```python class MyKubernetesPodOperator(KubernetesPodOperator): template_fields = tuple(x for x in

[GitHub] [airflow] raphaelauv commented on issue #16922: Using the string ".json" in a dag makes KubernetesPodOperator worker unable to trigger the pod

2021-08-03 Thread GitBox
raphaelauv commented on issue #16922: URL: https://github.com/apache/airflow/issues/16922#issuecomment-892153152 @louison @ericdevries ```python class MyKubernetesPodOperator(KubernetesPodOperator): a = tuple(x for x in KubernetesPodOperator.template_fields if x !=

[GitHub] [airflow] jedcunningham commented on issue #17398: With Helm Chart, get CSRF error if setting webserver.replicas > 1

2021-08-03 Thread GitBox
jedcunningham commented on issue #17398: URL: https://github.com/apache/airflow/issues/17398#issuecomment-892152186 #17142 should have fixed this. Silly question, but are you sure you are on 1.1.0, not 1.0.0? Do you see the `valueFrom` `AIRFLOW__WEBSERVER__SECRET_KEY` env var in your

[GitHub] [airflow] potiuk commented on pull request #16904: Improve AWS SQS Sensor (#16880)

2021-08-03 Thread GitBox
potiuk commented on pull request #16904: URL: https://github.com/apache/airflow/pull/16904#issuecomment-892149704 The problem is that we can't do it differently until we have better support from GitHub for cross-workflow communication. All details are described in CI.rst but here

[GitHub] [airflow] kurtqq commented on a change in pull request #17399: Adding JWT, IP filtering, and direct session login support for SalesforceHook

2021-08-03 Thread GitBox
kurtqq commented on a change in pull request #17399: URL: https://github.com/apache/airflow/pull/17399#discussion_r68202 ## File path: airflow/providers/salesforce/hooks/salesforce.py ## @@ -74,6 +91,29 @@ def get_connection_form_widgets() -> Dict[str, Any]:

[GitHub] [airflow] natanweinberger commented on a change in pull request #15013: Enable Connection creation from Vault parameters

2021-08-03 Thread GitBox
natanweinberger commented on a change in pull request #15013: URL: https://github.com/apache/airflow/pull/15013#discussion_r682064811 ## File path: airflow/providers/hashicorp/secrets/vault.py ## @@ -188,12 +204,28 @@ def get_conn_uri(self, conn_id: str) -> Optional[str]:

[GitHub] [airflow] josh-fell opened a new pull request #17399: Adding JWT, IP filtering, and direct session login support for SalesforceHook

2021-08-03 Thread GitBox
josh-fell opened a new pull request #17399: URL: https://github.com/apache/airflow/pull/17399 Closes: #17192 - Adding login and authentication support when using the `SalesforceHook`: - JWT bearer authentication - IP filtering (allow-listing) - Direct session/instance

[GitHub] [airflow] sreenath-kamath commented on pull request #17339: Adding support for multiple task-ids in the external task sensor

2021-08-03 Thread GitBox
sreenath-kamath commented on pull request #17339: URL: https://github.com/apache/airflow/pull/17339#issuecomment-892121981 @potiuk Have made the recommended changes, Can you take another pass at it ? -- This is an automated message from the Apache Git Service. To respond to the message,

[GitHub] [airflow] smythie86 opened a new issue #17398: With Helm Chart, get CSRF error if setting webserver.replicas > 1

2021-08-03 Thread GitBox
smythie86 opened a new issue #17398: URL: https://github.com/apache/airflow/issues/17398 **Apache Airflow version**: 2.1.2 **Airflow Helm Chart version: 1.1.0 **Kubernetes version (if you are using kubernetes)** (use `kubectl version`): Client Version:

[GitHub] [airflow] boring-cyborg[bot] commented on issue #17398: With Helm Chart, get CSRF error if setting webserver.replicas > 1

2021-08-03 Thread GitBox
boring-cyborg[bot] commented on issue #17398: URL: https://github.com/apache/airflow/issues/17398#issuecomment-892113279 Thanks for opening your first issue here! Be sure to follow the issue template! -- This is an automated message from the Apache Git Service. To respond to the

[GitHub] [airflow] jedcunningham commented on issue #17394: WARNING - Recorded pid 4969 does not match the current pid 4969

2021-08-03 Thread GitBox
jedcunningham commented on issue #17394: URL: https://github.com/apache/airflow/issues/17394#issuecomment-892065842 Likely related to `run_as_user`: https://github.com/apache/airflow/blob/4dae4ec9a4148db6c32f15ccc122d42c93a2c58a/airflow/jobs/local_task_job.py#L198-L199 Do you

[GitHub] [airflow] flolas commented on pull request #17329: Split sql statements in DbApi run

2021-08-03 Thread GitBox
flolas commented on pull request #17329: URL: https://github.com/apache/airflow/pull/17329#issuecomment-892050301 @uranusjr @potiuk Okay, I think that defaulting True for split_statement sounds good, bc if someone has some edge-cases (sqlparse is not perfect) can be easily fixed changing

[GitHub] [airflow] dimon222 commented on a change in pull request #16980: Change pod id notation back to hyphen

2021-08-03 Thread GitBox
dimon222 commented on a change in pull request #16980: URL: https://github.com/apache/airflow/pull/16980#discussion_r681979565 ## File path: airflow/kubernetes/pod_generator.py ## @@ -453,7 +453,7 @@ def make_unique_pod_id(pod_id: str) -> str: # Strip trailing '-' and

[GitHub] [airflow] dimon222 commented on a change in pull request #16980: Change pod id notation back to hyphen

2021-08-03 Thread GitBox
dimon222 commented on a change in pull request #16980: URL: https://github.com/apache/airflow/pull/16980#discussion_r681979565 ## File path: airflow/kubernetes/pod_generator.py ## @@ -453,7 +453,7 @@ def make_unique_pod_id(pod_id: str) -> str: # Strip trailing '-' and

[GitHub] [airflow] denimalpaca commented on a change in pull request #17382: Add new LocalFilesystemToS3Operator under Amazon provider (#17168)

2021-08-03 Thread GitBox
denimalpaca commented on a change in pull request #17382: URL: https://github.com/apache/airflow/pull/17382#discussion_r681970662 ## File path: airflow/providers/amazon/aws/example_dags/example_local_to_s3.py ## @@ -0,0 +1,40 @@ +# Licensed to the Apache Software Foundation

[GitHub] [airflow] potiuk commented on pull request #17329: Split sql statements in DbApi run

2021-08-03 Thread GitBox
potiuk commented on pull request #17329: URL: https://github.com/apache/airflow/pull/17329#issuecomment-892032420 My thoughts exactly @uranusjr. I was a bit on a side of backwards compatibility and our SemVer approach (which would make such change wait for 3.* Release). But we

[GitHub] [airflow] jedcunningham opened a new pull request #17397: Update to Celery 5

2021-08-03 Thread GitBox
jedcunningham opened a new pull request #17397: URL: https://github.com/apache/airflow/pull/17397 Celery 4 is no longer supported as of 2021-08-01: https://docs.celeryproject.org/en/stable/history/whatsnew-5.0.html#long-term-support-policy Closes: #11301 -- This is an

[GitHub] [airflow] potiuk commented on pull request #17386: Update company after changing jobs

2021-08-03 Thread GitBox
potiuk commented on pull request #17386: URL: https://github.com/apache/airflow/pull/17386#issuecomment-892028776 And congrats Astronomer :). -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

[GitHub] [airflow] potiuk commented on pull request #17386: Update company after changing jobs

2021-08-03 Thread GitBox
potiuk commented on pull request #17386: URL: https://github.com/apache/airflow/pull/17386#issuecomment-892028180 Congrats @BasPH -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

[GitHub] [airflow-site] potiuk commented on pull request #462: Add Seniorlink to Use Cases

2021-08-03 Thread GitBox
potiuk commented on pull request #462: URL: https://github.com/apache/airflow-site/pull/462#issuecomment-892024467 I started discussion https://lists.apache.org/x/thread.html/rb567279f7ea1611ef2145c61c0c726bce540bbf06e21e0447bd7be6c@%3Cdev.airflow.apache.org%3E -- This is an automated

[GitHub] [airflow] josh-fell closed pull request #17396: Adding JWT, IP filtering, and direct session login support for SalesforceHook

2021-08-03 Thread GitBox
josh-fell closed pull request #17396: URL: https://github.com/apache/airflow/pull/17396 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[GitHub] [airflow] fmv1992 commented on issue #13637: Scheduler takes 100% of CPU without task execution

2021-08-03 Thread GitBox
fmv1992 commented on issue #13637: URL: https://github.com/apache/airflow/issues/13637#issuecomment-892019154 We upgraded from `airflow` `1.10.15` to `2.0.2` and are experiencing this. @kaxil, configs: * `[scheduler] parsing_processes`: `2`. * `[scheduler]

[GitHub] [airflow-site] potiuk commented on pull request #462: Add Seniorlink to Use Cases

2021-08-03 Thread GitBox
potiuk commented on pull request #462: URL: https://github.com/apache/airflow-site/pull/462#issuecomment-892019012 Let me start discussion -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

[GitHub] [airflow-site] potiuk commented on pull request #462: Add Seniorlink to Use Cases

2021-08-03 Thread GitBox
potiuk commented on pull request #462: URL: https://github.com/apache/airflow-site/pull/462#issuecomment-892018433 I am not sure if we have any policy around that ? Do we want to have bigger number of small use cases or smaller number of bigger ones. I think we might want to discuss it at

[GitHub] [airflow-site] potiuk commented on pull request #462: Add Seniorlink to Use Cases

2021-08-03 Thread GitBox
potiuk commented on pull request #462: URL: https://github.com/apache/airflow-site/pull/462#issuecomment-892017506 Ah sorry , no - it is in use-case rather than blog. I think we have not discussed what can go here. -- This is an automated message from the Apache Git Service. To respond

[GitHub] [airflow-site] potiuk commented on pull request #462: Add Seniorlink to Use Cases

2021-08-03 Thread GitBox
potiuk commented on pull request #462: URL: https://github.com/apache/airflow-site/pull/462#issuecomment-892016362 I think blogs on airflow site are mostly for serious announcements and some big improvements from the community. We recently added an Airflow Medium publication and it's much

[GitHub] [airflow] Raju-basava commented on issue #8208: Create guide for Cloud Tasks operators

2021-08-03 Thread GitBox
Raju-basava commented on issue #8208: URL: https://github.com/apache/airflow/issues/8208#issuecomment-892003826 hi, I'm new to airflow and interested in contributing to this task, let me know. -- This is an automated message from the Apache Git Service. To respond to the message, please

[GitHub] [airflow] andrewgodwin commented on a change in pull request #15389: AIP-40: Deferrable Tasks

2021-08-03 Thread GitBox
andrewgodwin commented on a change in pull request #15389: URL: https://github.com/apache/airflow/pull/15389#discussion_r681933392 ## File path: airflow/jobs/local_task_job.py ## @@ -241,7 +241,6 @@ def _run_mini_scheduler_on_child_tasks(self, session=None) -> None:

[airflow] branch main updated (e8e3b06 -> 4dae4ec)

2021-08-03 Thread potiuk
This is an automated email from the ASF dual-hosted git repository. potiuk pushed a change to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git. from e8e3b06 More touches on dag_maker fixture to create DagModel automatically (#17391) add 4dae4ec refactor:

[GitHub] [airflow] potiuk merged pull request #17388: refactor: fixed type annotation for 'sql' in MySqlOperator

2021-08-03 Thread GitBox
potiuk merged pull request #17388: URL: https://github.com/apache/airflow/pull/17388 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[GitHub] [airflow] potiuk commented on issue #17186: Fix template_ext processing for Kubernetes Pod Operator

2021-08-03 Thread GitBox
potiuk commented on issue #17186: URL: https://github.com/apache/airflow/issues/17186#issuecomment-891973397 I have changed the name to correctly reflect the discussion results. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [airflow] potiuk commented on issue #16922: Using the string ".json" in a dag makes KubernetesPodOperator worker unable to trigger the pod

2021-08-03 Thread GitBox
potiuk commented on issue #16922: URL: https://github.com/apache/airflow/issues/16922#issuecomment-891972732 See discussion in #17186 - it turned out really to be a bug in the implementation of recursive rendering of parameters in Kubernetes Pod Operator, You can (and Kubernetes

[airflow-site] branch gh-pages updated: Deploying to gh-pages from @ 6427952fab9f966b19166ff65a84e0b0f187449b 

2021-08-03 Thread github-bot
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a commit to branch gh-pages in repository https://gitbox.apache.org/repos/asf/airflow-site.git The following commit(s) were added to refs/heads/gh-pages by this push: new 83521ef Deploying to gh-pages

[GitHub] [airflow-site] potiuk merged pull request #461: Fixes messed-up changelog for some providers

2021-08-03 Thread GitBox
potiuk merged pull request #461: URL: https://github.com/apache/airflow-site/pull/461 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[GitHub] [airflow] subkanthi commented on a change in pull request #17382: Add new LocalFilesystemToS3Operator under Amazon provider (#17168)

2021-08-03 Thread GitBox
subkanthi commented on a change in pull request #17382: URL: https://github.com/apache/airflow/pull/17382#discussion_r681881085 ## File path: airflow/providers/amazon/aws/example_dags/example_local_to_s3.py ## @@ -0,0 +1,40 @@ +# Licensed to the Apache Software Foundation

[airflow-site] branch main updated: Fixes messed-up changelog for some providers (#461)

2021-08-03 Thread potiuk
This is an automated email from the ASF dual-hosted git repository. potiuk pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/airflow-site.git The following commit(s) were added to refs/heads/main by this push: new 6427952 Fixes messed-up changelog for some

[GitHub] [airflow] davlum commented on a change in pull request #15013: Enable Connection creation from Vault parameters

2021-08-03 Thread GitBox
davlum commented on a change in pull request #15013: URL: https://github.com/apache/airflow/pull/15013#discussion_r681881872 ## File path: airflow/providers/hashicorp/secrets/vault.py ## @@ -188,12 +204,28 @@ def get_conn_uri(self, conn_id: str) -> Optional[str]:

[GitHub] [airflow] potiuk commented on pull request #16858: Bump sshtunnel to current release

2021-08-03 Thread GitBox
potiuk commented on pull request #16858: URL: https://github.com/apache/airflow/pull/16858#issuecomment-891950891 Not really - there is one intermittent error with scheduler job, but SSH tests are still failing:

[GitHub] [airflow] potiuk commented on issue #16728: SparkKubernetesOperator to support arguments

2021-08-03 Thread GitBox
potiuk commented on issue #16728: URL: https://github.com/apache/airflow/issues/16728#issuecomment-891945673 Assigned you @goodhamgupta ! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

[GitHub] [airflow] ephraimbuddy merged pull request #17391: More touches on `dag_maker` fixture to create DagModel automatically

2021-08-03 Thread GitBox
ephraimbuddy merged pull request #17391: URL: https://github.com/apache/airflow/pull/17391 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[airflow] branch main updated (ed99eaa -> e8e3b06)

2021-08-03 Thread ephraimanierobi
This is an automated email from the ASF dual-hosted git repository. ephraimanierobi pushed a change to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git. from ed99eaa Regression on pid reset to allow task start after heartbeat (#17333) add e8e3b06 More

[GitHub] [airflow] josh-fell opened a new pull request #17396: Adding JWT, IP filtering, and direct session login support for SalesforceHook

2021-08-03 Thread GitBox
josh-fell opened a new pull request #17396: URL: https://github.com/apache/airflow/pull/17396 Closes: #17192 Adding login and authentication support when using the `SalesforceHook`: - JWT bearer authentication - IP filtering (allow-listing) - Direct session/instance access

[GitHub] [airflow] davlum commented on a change in pull request #17387: Invalidate Vault cached prop when not authenticated

2021-08-03 Thread GitBox
davlum commented on a change in pull request #17387: URL: https://github.com/apache/airflow/pull/17387#discussion_r681851621 ## File path: airflow/providers/hashicorp/_internal_client/vault_client.py ## @@ -191,8 +191,21 @@ def __init__( self.radius_secret =

[GitHub] [airflow-site] christopherpetrino commented on pull request #462: Add Seniorlink to Use Cases

2021-08-03 Thread GitBox
christopherpetrino commented on pull request #462: URL: https://github.com/apache/airflow-site/pull/462#issuecomment-891924768 @potiuk @kaxil @jhtimmins who are the appropriate people to tag on these types of PRs? -- This is an automated message from the Apache Git Service. To respond

[GitHub] [airflow-site] christopherpetrino opened a new pull request #462: Add Seniorlink to Use Cases

2021-08-03 Thread GitBox
christopherpetrino opened a new pull request #462: URL: https://github.com/apache/airflow-site/pull/462 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe,

[GitHub] [airflow] santosh-d3vpl3x closed pull request #17341: Add ExecStop to systemd service files.

2021-08-03 Thread GitBox
santosh-d3vpl3x closed pull request #17341: URL: https://github.com/apache/airflow/pull/17341 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[GitHub] [airflow] cpdean commented on pull request #15599: Mask passwords and sensitive info in task logs and UI

2021-08-03 Thread GitBox
cpdean commented on pull request #15599: URL: https://github.com/apache/airflow/pull/15599#issuecomment-891921076 I am trying to get my installation of airflow to mask passwords in the rendered template UI of a given task. It seems this pull request was merged and made available somewhere

[GitHub] [airflow] wyattshapiro commented on a change in pull request #16980: Change pod id notation back to hyphen

2021-08-03 Thread GitBox
wyattshapiro commented on a change in pull request #16980: URL: https://github.com/apache/airflow/pull/16980#discussion_r681837507 ## File path: airflow/kubernetes/pod_generator.py ## @@ -453,7 +453,7 @@ def make_unique_pod_id(pod_id: str) -> str: # Strip trailing '-'

[GitHub] [airflow] yevsh opened a new issue #17395: DAG is executed twice on submit

2021-08-03 Thread GitBox
yevsh opened a new issue #17395: URL: https://github.com/apache/airflow/issues/17395 Airflow 2.1.2 runs as pod in kubernetes cluster. python DAG is placed in airflow_dags location, airflow executes it twice! From log: ``` [2021-08-02 14:48:07,900]

[GitHub] [airflow] SamWheating commented on a change in pull request #17347: Handle and log exceptions raised during task callback

2021-08-03 Thread GitBox
SamWheating commented on a change in pull request #17347: URL: https://github.com/apache/airflow/pull/17347#discussion_r681810429 ## File path: tests/models/test_taskinstance.py ## @@ -1743,6 +1743,32 @@ def on_execute_callable(context): ti.refresh_from_db()

[GitHub] [airflow] wojsamjan commented on pull request #17285: Add info log how to fix: More than one pod running with labels

2021-08-03 Thread GitBox
wojsamjan commented on pull request #17285: URL: https://github.com/apache/airflow/pull/17285#issuecomment-891880603 > The labels should already be unique: >

[GitHub] [airflow] terryfugit opened a new issue #17394: WARNING - Recorded pid 4969 does not match the current pid 4969

2021-08-03 Thread GitBox
terryfugit opened a new issue #17394: URL: https://github.com/apache/airflow/issues/17394 **Apache Airflow version**: 2.1.2 **What happened**: Receive warning message "WARNING - Recorded pid 4969 does not match the current pid 4969", then the "INFO - Sending

[GitHub] [airflow] boring-cyborg[bot] commented on issue #17394: WARNING - Recorded pid 4969 does not match the current pid 4969

2021-08-03 Thread GitBox
boring-cyborg[bot] commented on issue #17394: URL: https://github.com/apache/airflow/issues/17394#issuecomment-891878172 Thanks for opening your first issue here! Be sure to follow the issue template! -- This is an automated message from the Apache Git Service. To respond to the

[GitHub] [airflow] dash-stefan-petrescu commented on pull request #16858: Bump sshtunnel to current release

2021-08-03 Thread GitBox
dash-stefan-petrescu commented on pull request #16858: URL: https://github.com/apache/airflow/pull/16858#issuecomment-891874964 @potiuk All looks good now -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above

[GitHub] [airflow] rodrigoechaide commented on issue #11379: Temporary failure in name resolution while running tasks using KubernetesExecutor

2021-08-03 Thread GitBox
rodrigoechaide commented on issue #11379: URL: https://github.com/apache/airflow/issues/11379#issuecomment-891865334 Thanks, @uranusjr. I will check that entry. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL

[GitHub] [airflow] github-actions[bot] commented on pull request #17391: More touches on `dag_maker` fixture to create DagModel automatically

2021-08-03 Thread GitBox
github-actions[bot] commented on pull request #17391: URL: https://github.com/apache/airflow/pull/17391#issuecomment-891860163 The PR most likely needs to run full matrix of tests because it modifies parts of the core of Airflow. However, committers might decide to merge it quickly and

[GitHub] [airflow] uranusjr edited a comment on pull request #16352: AIP-39: ``DagRun.data_interval_start`` and ``data_interval_end``

2021-08-03 Thread GitBox
uranusjr edited a comment on pull request #16352: URL: https://github.com/apache/airflow/pull/16352#issuecomment-891859321 Note: I’ve changed `schedule_date` in this PR to `logical_date` as I mentioned on the dev mailing list:

[GitHub] [airflow] uranusjr commented on pull request #16352: AIP-39: ``DagRun.data_interval_start`` and ``data_interval_end``

2021-08-03 Thread GitBox
uranusjr commented on pull request #16352: URL: https://github.com/apache/airflow/pull/16352#issuecomment-891859321 Note: I’ve changed `schedule_date` in this PR to `logical_date` as I mentioned on the dev mailing list:

[GitHub] [airflow] uranusjr commented on issue #11379: Temporary failure in name resolution while running tasks using KubernetesExecutor

2021-08-03 Thread GitBox
uranusjr commented on issue #11379: URL: https://github.com/apache/airflow/issues/11379#issuecomment-891853538 You may find this StackOverflow entry interesting: https://stackoverflow.com/questions/56188537 From what I can gather, this tends to happen when you send a ton of things

[GitHub] [airflow] sreenath-kamath commented on issue #17340: Retrieve session logs when using Livy Operator

2021-08-03 Thread GitBox
sreenath-kamath commented on issue #17340: URL: https://github.com/apache/airflow/issues/17340#issuecomment-891851776 Have raised a PR https://github.com/apache/airflow/pull/17393 for the same. -- This is an automated message from the Apache Git Service. To respond to the message,

[GitHub] [airflow] rodrigoechaide edited a comment on issue #11379: Temporary failure in name resolution while running tasks using KubernetesExecutor

2021-08-03 Thread GitBox
rodrigoechaide edited a comment on issue #11379: URL: https://github.com/apache/airflow/issues/11379#issuecomment-891850328 Hi, @Siddharthk are you still facing the same issue? Because I am facing the same issue when running a DAG that has 500 parallel tasks because I am running some

[GitHub] [airflow] sreenath-kamath opened a new pull request #17393: Fetching and logging Livy session logs for LivyOperator

2021-08-03 Thread GitBox
sreenath-kamath opened a new pull request #17393: URL: https://github.com/apache/airflow/pull/17393 Fetching and logging Livy session logs for LivyOperator closes: #17340 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [airflow] rodrigoechaide commented on issue #11379: Temporary failure in name resolution while running tasks using KubernetesExecutor

2021-08-03 Thread GitBox
rodrigoechaide commented on issue #11379: URL: https://github.com/apache/airflow/issues/11379#issuecomment-891850328 Hi, @Siddharthk are you still facing the same issue? Because I am facing the same issue when running a DAG that has 500 parallel tasks because I am running some stress

[GitHub] [airflow] ashb commented on a change in pull request #15389: AIP-40: Deferrable Tasks

2021-08-03 Thread GitBox
ashb commented on a change in pull request #15389: URL: https://github.com/apache/airflow/pull/15389#discussion_r681735890 ## File path: airflow/jobs/local_task_job.py ## @@ -241,7 +241,6 @@ def _run_mini_scheduler_on_child_tasks(self, session=None) -> None:

[GitHub] [airflow] ashb commented on issue #16618: Access specific values within an `XCom` value using Taskflow API

2021-08-03 Thread GitBox
ashb commented on issue #16618: URL: https://github.com/apache/airflow/issues/16618#issuecomment-891827633 > It would also be great if there was a way of NOT creating the dependencies between tasks automatically. > For cluster or job IDs (EMR, Databricks, etc) is very common, and having

[GitHub] [airflow] potiuk commented on pull request #17356: Switches to "/" convention in ghcr.io images

2021-08-03 Thread GitBox
potiuk commented on pull request #17356: URL: https://github.com/apache/airflow/pull/17356#issuecomment-891808959 Yeah. I also run all the scenarios in my own repos 1) 'main' from my own repo 2) "main' from a completely new repo (to test builds in forks of our users) 3) PR from another

[GitHub] [airflow] github-actions[bot] commented on pull request #17068: Influxdb Hook

2021-08-03 Thread GitBox
github-actions[bot] commented on pull request #17068: URL: https://github.com/apache/airflow/pull/17068#issuecomment-891799985 The PR most likely needs to run full matrix of tests because it modifies parts of the core of Airflow. However, committers might decide to merge it quickly and

[GitHub] [airflow] cansjt edited a comment on issue #17022: TaskInstance.previous_execution_date_success fails with: 'TaskInstance' object has no attribute 'task'

2021-08-03 Thread GitBox
cansjt edited a comment on issue #17022: URL: https://github.com/apache/airflow/issues/17022#issuecomment-891787393 With Airflow 2.1.2 we are something similar but not identical: ``` Task failed with exception", "exc_info": "Traceback (most recent call last): File

[GitHub] [airflow] cansjt commented on issue #17022: TaskInstance.previous_execution_date_success fails with: 'TaskInstance' object has no attribute 'task'

2021-08-03 Thread GitBox
cansjt commented on issue #17022: URL: https://github.com/apache/airflow/issues/17022#issuecomment-891787393 With Airflow 2.1.2 we are something similar but not identical: ``` Task failed with exception", "exc_info": "Traceback (most recent call last): File

[GitHub] [airflow] goodhamgupta commented on issue #16728: SparkKubernetesOperator to support arguments

2021-08-03 Thread GitBox
goodhamgupta commented on issue #16728: URL: https://github.com/apache/airflow/issues/16728#issuecomment-891784787 Incase this has not yet been picked up, I would like to work on this issue  -- This is an automated message from the Apache Git Service. To respond to the message, please

[GitHub] [airflow] ephraimbuddy commented on issue #16227: LocalTaskJob heartbeat race condition with finishing task causing SIGTERM

2021-08-03 Thread GitBox
ephraimbuddy commented on issue #16227: URL: https://github.com/apache/airflow/issues/16227#issuecomment-891783610 You can create an issue with a simple dag to reproduce the issue. That way we will figure it out -- This is an automated message from the Apache Git Service. To respond to

[GitHub] [airflow] rafraph commented on issue #8907: Airflow web UI is slow

2021-08-03 Thread GitBox
rafraph commented on issue #8907: URL: https://github.com/apache/airflow/issues/8907#issuecomment-891782320 I think in my case it also was DNS issue since when I use the IP instead of the URL the UI become very fast -- This is an automated message from the Apache Git Service. To respond

[GitHub] [airflow] mwaaas commented on issue #16227: LocalTaskJob heartbeat race condition with finishing task causing SIGTERM

2021-08-03 Thread GitBox
mwaaas commented on issue #16227: URL: https://github.com/apache/airflow/issues/16227#issuecomment-891781711 Yes from the scheduler logs, I was seeing logs saying cannot find the task. Why would the task go missing but its still processing. -- This is an automated message from

[GitHub] [airflow] ephraimbuddy commented on issue #16227: LocalTaskJob heartbeat race condition with finishing task causing SIGTERM

2021-08-03 Thread GitBox
ephraimbuddy commented on issue #16227: URL: https://github.com/apache/airflow/issues/16227#issuecomment-891779597 You can check the scheduler log when this happens, probably the task went missing and was removed somewhere. That happened externally and I think it might not be related to

[GitHub] [airflow] ephraimbuddy commented on pull request #17347: Handle and log exceptions raised during task callback

2021-08-03 Thread GitBox
ephraimbuddy commented on pull request #17347: URL: https://github.com/apache/airflow/pull/17347#issuecomment-891770159 Can you rebase @SamWheating -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go

[GitHub] [airflow] ephraimbuddy closed pull request #17347: Handle and log exceptions raised during task callback

2021-08-03 Thread GitBox
ephraimbuddy closed pull request #17347: URL: https://github.com/apache/airflow/pull/17347 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[GitHub] [airflow] rounakdatta commented on a change in pull request #17349: #16037 Add support for passing templated requirements.txt in PythonVirtualenvOperator

2021-08-03 Thread GitBox
rounakdatta commented on a change in pull request #17349: URL: https://github.com/apache/airflow/pull/17349#discussion_r681669098 ## File path: airflow/operators/python.py ## @@ -333,20 +337,31 @@ def __init__( templates_exts=templates_exts, **kwargs,

[GitHub] [airflow] ephraimbuddy commented on a change in pull request #17391: More touches on `dag_maker` fixture to create DagModel automatically

2021-08-03 Thread GitBox
ephraimbuddy commented on a change in pull request #17391: URL: https://github.com/apache/airflow/pull/17391#discussion_r681669038 ## File path: tests/conftest.py ## @@ -436,25 +436,27 @@ def dag_maker(request): DEFAULT_DATE = timezone.datetime(2016, 1, 1) class

[GitHub] [airflow] mwaaas commented on issue #16227: LocalTaskJob heartbeat race condition with finishing task causing SIGTERM

2021-08-03 Thread GitBox
mwaaas commented on issue #16227: URL: https://github.com/apache/airflow/issues/16227#issuecomment-891765746 > Yeah, checking the image it has the change. In your dag, how does the task state change to removed? Not sure how the task change to removed. I have added this

[GitHub] [airflow] ephraimbuddy commented on issue #16227: LocalTaskJob heartbeat race condition with finishing task causing SIGTERM

2021-08-03 Thread GitBox
ephraimbuddy commented on issue #16227: URL: https://github.com/apache/airflow/issues/16227#issuecomment-891761998 I will also simulate this: ```the publish task ( this task takes time to run up to 30min)``` -- This is an automated message from the Apache Git Service. To respond to the

[GitHub] [airflow] ephraimbuddy commented on issue #16227: LocalTaskJob heartbeat race condition with finishing task causing SIGTERM

2021-08-03 Thread GitBox
ephraimbuddy commented on issue #16227: URL: https://github.com/apache/airflow/issues/16227#issuecomment-891761170 Yeah, checking the image it has the change. In your dag, how does the task state change to removed? -- This is an automated message from the Apache Git Service. To respond

[GitHub] [airflow] ashb commented on a change in pull request #17391: More touches on `dag_maker` fixture to create DagModel automatically

2021-08-03 Thread GitBox
ashb commented on a change in pull request #17391: URL: https://github.com/apache/airflow/pull/17391#discussion_r681664002 ## File path: tests/conftest.py ## @@ -436,25 +436,27 @@ def dag_maker(request): DEFAULT_DATE = timezone.datetime(2016, 1, 1) class

[GitHub] [airflow] ephraimbuddy opened a new pull request #17391: More touches on `dag_maker` fixture to create DagModel automatically

2021-08-03 Thread GitBox
ephraimbuddy opened a new pull request #17391: URL: https://github.com/apache/airflow/pull/17391 This change improves the dag_maker fixture to create DagModel on exit --- **^ Add meaningful description above** Read the **[Pull Request

[GitHub] [airflow] alittlesliceoftom edited a comment on issue #13888: Kubernetes Launchers Log Check in Time is Not Configurable - Excess Logs Slow Airflow

2021-08-03 Thread GitBox
alittlesliceoftom edited a comment on issue #13888: URL: https://github.com/apache/airflow/issues/13888#issuecomment-891757648 Hey @kdunee a very crude one for #14259, I just replace the command I use with one that runs an echo every 30 secs in addition to the main task. So I

  1   2   >