[GitHub] [airflow] uranusjr edited a comment on pull request #18930: Add Guide for Apache Pinot

2021-10-27 Thread GitBox
uranusjr edited a comment on pull request #18930: URL: https://github.com/apache/airflow/pull/18930#issuecomment-953515482 I think this is rebased to the wrong base. Not sure what the best way to recover is; personally I usually `cherry-pick` the commits manually back: ```bash #

[GitHub] [airflow] uranusjr commented on pull request #18930: Add Guide for Apache Pinot

2021-10-27 Thread GitBox
uranusjr commented on pull request #18930: URL: https://github.com/apache/airflow/pull/18930#issuecomment-953515482 I think this is rebased to the wrong base. Not sure what the best way to recover is; personally I usually `cherry-pick` the commits manually back: ``` # Assumes

[GitHub] [airflow] sharon2719 edited a comment on pull request #18930: Add Guide for Apache Pinot

2021-10-27 Thread GitBox
sharon2719 edited a comment on pull request #18930: URL: https://github.com/apache/airflow/pull/18930#issuecomment-953508074 I have rebased it but I still dont get why it is still pushing with the unrelated changes -- This is an automated message from the Apache Git Service. To

[GitHub] [airflow] sharon2719 commented on pull request #18930: Add Guide for Apache Pinot

2021-10-27 Thread GitBox
sharon2719 commented on pull request #18930: URL: https://github.com/apache/airflow/pull/18930#issuecomment-953508074 > It seems like the merge (or rebase?) didn't went well and pulled in a ton of unrelated things... - [ ] -- This is an automated message from the Apache Git

[GitHub] [airflow] sharon2719 closed pull request #18930: Add Guide for Apache Pinot

2021-10-27 Thread GitBox
sharon2719 closed pull request #18930: URL: https://github.com/apache/airflow/pull/18930 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[GitHub] [airflow] uranusjr commented on pull request #18930: Add Guide for Apache Pinot

2021-10-27 Thread GitBox
uranusjr commented on pull request #18930: URL: https://github.com/apache/airflow/pull/18930#issuecomment-953506387 It seems like the merge (or rebase?) didn't went well and pulled in a ton of unrelated things... -- This is an automated message from the Apache Git Service. To respond to

[GitHub] [airflow] kn6405 opened a new issue #19275: Airflow Triggerer Pod has AttributeError: module 'asyncio' has no attribute 'run'

2021-10-27 Thread GitBox
kn6405 opened a new issue #19275: URL: https://github.com/apache/airflow/issues/19275 ### Apache Airflow version 2.2.0 (latest released) ### Operating System Docker Image ### Versions of Apache Airflow Providers _No response_ ### Deployment

[GitHub] [airflow] chinwobble edited a comment on issue #18999: Update databricks provider to use TriggerOperator

2021-10-27 Thread GitBox
chinwobble edited a comment on issue #18999: URL: https://github.com/apache/airflow/issues/18999#issuecomment-953359692 @eskarimov I have implemented a prototype like this: One issue i have with the above limitation is that the task duration does not count the time the task was

[GitHub] [airflow] uranusjr commented on issue #19224: Add subprocess.DEVNULL to LoggingCommandExecutor stdout

2021-10-27 Thread GitBox
uranusjr commented on issue #19224: URL: https://github.com/apache/airflow/issues/19224#issuecomment-953488539 Go ahead! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment.

[airflow] branch main updated (83eb80d -> 60fade2)

2021-10-27 Thread dstandish
This is an automated email from the ASF dual-hosted git repository. dstandish pushed a change to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git. from 83eb80d SFTP hook to prefer the SSH paramiko key over the key file path (#18988) add 60fade2 Rename

[GitHub] [airflow] dstandish merged pull request #19265: Rename variable `serialize_dag` to `serialized_dag`

2021-10-27 Thread GitBox
dstandish merged pull request #19265: URL: https://github.com/apache/airflow/pull/19265 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[GitHub] [airflow] ShakaibKhan commented on issue #19224: Add subprocess.DEVNULL to LoggingCommandExecutor stdout

2021-10-27 Thread GitBox
ShakaibKhan commented on issue #19224: URL: https://github.com/apache/airflow/issues/19224#issuecomment-95349 I would like to try to implementing this -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL

[GitHub] [airflow] harsh2911 opened a new issue #19272: Create get_partition, create_partition methods in AWS Glue Catalog Hook

2021-10-27 Thread GitBox
harsh2911 opened a new issue #19272: URL: https://github.com/apache/airflow/issues/19272 ### Description Current AWS Glue Catalog Hook does not have all the methods. If possible, we can add get_partition and create_partition methods in existing Glue Catalog Hook. ### Use

[GitHub] [airflow] boring-cyborg[bot] commented on issue #19272: Create get_partition, create_partition methods in AWS Glue Catalog Hook

2021-10-27 Thread GitBox
boring-cyborg[bot] commented on issue #19272: URL: https://github.com/apache/airflow/issues/19272#issuecomment-953467917 Thanks for opening your first issue here! Be sure to follow the issue template! -- This is an automated message from the Apache Git Service. To respond to the

[GitHub] [airflow] boring-cyborg[bot] commented on issue #19271: Airflow Docker Compose Network Issue

2021-10-27 Thread GitBox
boring-cyborg[bot] commented on issue #19271: URL: https://github.com/apache/airflow/issues/19271#issuecomment-953464767 Thanks for opening your first issue here! Be sure to follow the issue template! -- This is an automated message from the Apache Git Service. To respond to the

[GitHub] [airflow] buffaloDeveloper opened a new issue #19271: Airflow Docker Compose Network Issue

2021-10-27 Thread GitBox
buffaloDeveloper opened a new issue #19271: URL: https://github.com/apache/airflow/issues/19271 ### Apache Airflow version 2.2.0 (latest released) ### Operating System Debian 11 ### Versions of Apache Airflow Providers kafka-python==2.0.2

[GitHub] [airflow] mik-laj commented on a change in pull request #19189: Swtich both PROD and CI images to use virtual environments

2021-10-27 Thread GitBox
mik-laj commented on a change in pull request #19189: URL: https://github.com/apache/airflow/pull/19189#discussion_r737982064 ## File path: docs/docker-stack/build.rst ## @@ -203,14 +203,22 @@ You should be aware, about a few things: `best practices of Dockerfiles

[GitHub] [airflow] chandu-007 commented on issue #10026: Duplicate tasks invoked for a single task_id when manually invoked task details modal.

2021-10-27 Thread GitBox
chandu-007 commented on issue #10026: URL: https://github.com/apache/airflow/issues/10026#issuecomment-953455644 We are also facing same issue using `Composer 1.17.3` and `Airflow 2.1.2` -- This is an automated message from the Apache Git Service. To respond to the message, please log

[GitHub] [airflow] mik-laj commented on a change in pull request #19189: Swtich both PROD and CI images to use virtual environments

2021-10-27 Thread GitBox
mik-laj commented on a change in pull request #19189: URL: https://github.com/apache/airflow/pull/19189#discussion_r737970047 ## File path: Dockerfile ## @@ -183,16 +183,19 @@ ENV INSTALL_MYSQL_CLIENT=${INSTALL_MYSQL_CLIENT} \

[GitHub] [airflow] potiuk commented on a change in pull request #19189: Swtich both PROD and CI images to use virtual environments

2021-10-27 Thread GitBox
potiuk commented on a change in pull request #19189: URL: https://github.com/apache/airflow/pull/19189#discussion_r737969733 ## File path: airflow/operators/python.py ## @@ -338,6 +347,7 @@ def __init__( python_version: Optional[Union[str, int, float]] = None,

[GitHub] [airflow] potiuk commented on a change in pull request #19189: Swtich both PROD and CI images to use virtual environments

2021-10-27 Thread GitBox
potiuk commented on a change in pull request #19189: URL: https://github.com/apache/airflow/pull/19189#discussion_r737969733 ## File path: airflow/operators/python.py ## @@ -338,6 +347,7 @@ def __init__( python_version: Optional[Union[str, int, float]] = None,

[GitHub] [airflow] mik-laj commented on a change in pull request #19189: Swtich both PROD and CI images to use virtual environments

2021-10-27 Thread GitBox
mik-laj commented on a change in pull request #19189: URL: https://github.com/apache/airflow/pull/19189#discussion_r737970047 ## File path: Dockerfile ## @@ -183,16 +183,19 @@ ENV INSTALL_MYSQL_CLIENT=${INSTALL_MYSQL_CLIENT} \

[GitHub] [airflow] potiuk commented on a change in pull request #19189: Swtich both PROD and CI images to use virtual environments

2021-10-27 Thread GitBox
potiuk commented on a change in pull request #19189: URL: https://github.com/apache/airflow/pull/19189#discussion_r737969733 ## File path: airflow/operators/python.py ## @@ -338,6 +347,7 @@ def __init__( python_version: Optional[Union[str, int, float]] = None,

[GitHub] [airflow] potiuk commented on a change in pull request #19189: Swtich both PROD and CI images to use virtual environments

2021-10-27 Thread GitBox
potiuk commented on a change in pull request #19189: URL: https://github.com/apache/airflow/pull/19189#discussion_r737969733 ## File path: airflow/operators/python.py ## @@ -338,6 +347,7 @@ def __init__( python_version: Optional[Union[str, int, float]] = None,

[GitHub] [airflow] potiuk commented on a change in pull request #19189: Swtich both PROD and CI images to use virtual environments

2021-10-27 Thread GitBox
potiuk commented on a change in pull request #19189: URL: https://github.com/apache/airflow/pull/19189#discussion_r737960560 ## File path: airflow/operators/python.py ## @@ -338,6 +347,7 @@ def __init__( python_version: Optional[Union[str, int, float]] = None,

[GitHub] [airflow] potiuk commented on a change in pull request #19189: Swtich both PROD and CI images to use virtual environments

2021-10-27 Thread GitBox
potiuk commented on a change in pull request #19189: URL: https://github.com/apache/airflow/pull/19189#discussion_r737967697 ## File path: docs/docker-stack/build.rst ## @@ -203,14 +203,22 @@ You should be aware, about a few things: `best practices of Dockerfiles

[GitHub] [airflow] potiuk commented on a change in pull request #19189: Swtich both PROD and CI images to use virtual environments

2021-10-27 Thread GitBox
potiuk commented on a change in pull request #19189: URL: https://github.com/apache/airflow/pull/19189#discussion_r737967697 ## File path: docs/docker-stack/build.rst ## @@ -203,14 +203,22 @@ You should be aware, about a few things: `best practices of Dockerfiles

[GitHub] [airflow] mik-laj commented on a change in pull request #19189: Swtich both PROD and CI images to use virtual environments

2021-10-27 Thread GitBox
mik-laj commented on a change in pull request #19189: URL: https://github.com/apache/airflow/pull/19189#discussion_r737952212 ## File path: Dockerfile ## @@ -183,16 +183,19 @@ ENV INSTALL_MYSQL_CLIENT=${INSTALL_MYSQL_CLIENT} \

[GitHub] [airflow] dimon222 edited a comment on issue #18939: Upgrading from version 2.1.4 to 2.2.0 fails during mysql db upgrade with error Can't DROP 'dag_id'

2021-10-27 Thread GitBox
dimon222 edited a comment on issue #18939: URL: https://github.com/apache/airflow/issues/18939#issuecomment-953438714 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To

[GitHub] [airflow] dimon222 edited a comment on issue #18939: Upgrading from version 2.1.4 to 2.2.0 fails during mysql db upgrade with error Can't DROP 'dag_id'

2021-10-27 Thread GitBox
dimon222 edited a comment on issue #18939: URL: https://github.com/apache/airflow/issues/18939#issuecomment-953438714 @jedcunningham I can confirm that I have zoo of collations due to legacy setup of back from beginning versions of Airflow 1.10.x so I'm ready to believe in this theory.

[GitHub] [airflow] dimon222 commented on issue #18939: Upgrading from version 2.1.4 to 2.2.0 fails during mysql db upgrade with error Can't DROP 'dag_id'

2021-10-27 Thread GitBox
dimon222 commented on issue #18939: URL: https://github.com/apache/airflow/issues/18939#issuecomment-953438714 @jedcunningham I can confirm that I'm have zoo of collations due to legacy setup of back from beginning versions of Airflow 1.10.x so I'm ready to believe in this theory. Jeez,

[GitHub] [airflow] potiuk commented on a change in pull request #19189: Swtich both PROD and CI images to use virtual environments

2021-10-27 Thread GitBox
potiuk commented on a change in pull request #19189: URL: https://github.com/apache/airflow/pull/19189#discussion_r737965074 ## File path: Dockerfile ## @@ -183,16 +182,20 @@ ENV INSTALL_MYSQL_CLIENT=${INSTALL_MYSQL_CLIENT} \

[GitHub] [airflow] potiuk commented on a change in pull request #19189: Swtich both PROD and CI images to use virtual environments

2021-10-27 Thread GitBox
potiuk commented on a change in pull request #19189: URL: https://github.com/apache/airflow/pull/19189#discussion_r737962330 ## File path: scripts/ci/libraries/_verify_image.sh ## @@ -340,7 +340,7 @@ function verify_image::verify_prod_image_as_root() { function

[GitHub] [airflow] potiuk commented on a change in pull request #19189: Swtich both PROD and CI images to use virtual environments

2021-10-27 Thread GitBox
potiuk commented on a change in pull request #19189: URL: https://github.com/apache/airflow/pull/19189#discussion_r737961883 ## File path: airflow/utils/python_virtualenv.py ## @@ -27,13 +28,22 @@ from airflow.utils.process_utils import execute_in_subprocess -def

[GitHub] [airflow] potiuk commented on a change in pull request #19189: Swtich both PROD and CI images to use virtual environments

2021-10-27 Thread GitBox
potiuk commented on a change in pull request #19189: URL: https://github.com/apache/airflow/pull/19189#discussion_r737960560 ## File path: airflow/operators/python.py ## @@ -338,6 +347,7 @@ def __init__( python_version: Optional[Union[str, int, float]] = None,

[GitHub] [airflow] potiuk commented on a change in pull request #19189: Swtich both PROD and CI images to use virtual environments

2021-10-27 Thread GitBox
potiuk commented on a change in pull request #19189: URL: https://github.com/apache/airflow/pull/19189#discussion_r737960560 ## File path: airflow/operators/python.py ## @@ -338,6 +347,7 @@ def __init__( python_version: Optional[Union[str, int, float]] = None,

[GitHub] [airflow] potiuk commented on a change in pull request #19189: Swtich both PROD and CI images to use virtual environments

2021-10-27 Thread GitBox
potiuk commented on a change in pull request #19189: URL: https://github.com/apache/airflow/pull/19189#discussion_r737960560 ## File path: airflow/operators/python.py ## @@ -338,6 +347,7 @@ def __init__( python_version: Optional[Union[str, int, float]] = None,

[GitHub] [airflow] bugraoz93 opened a new issue #19269: Add support for multiple AdAccount when getting Bulk Facebook Report from Facebook Ads Hook

2021-10-27 Thread GitBox
bugraoz93 opened a new issue #19269: URL: https://github.com/apache/airflow/issues/19269 ### Description Currently, the Facebook Ads Hook only supports one `account_id` for every hook(`facebook_conn_id`). It would be great if we could pass multiple `account_id` from the Facebook

[GitHub] [airflow] boring-cyborg[bot] commented on issue #19269: Add support for multiple AdAccount when getting Bulk Facebook Report from Facebook Ads Hook

2021-10-27 Thread GitBox
boring-cyborg[bot] commented on issue #19269: URL: https://github.com/apache/airflow/issues/19269#issuecomment-953429690 Thanks for opening your first issue here! Be sure to follow the issue template! -- This is an automated message from the Apache Git Service. To respond to the

[GitHub] [airflow] asaf400 commented on pull request #19027: Fix for DockerOperator Xcoms functionality

2021-10-27 Thread GitBox
asaf400 commented on pull request #19027: URL: https://github.com/apache/airflow/pull/19027#issuecomment-953429266 Okay, I'll be able to invest more time into this PR in the next few day or two, I wasn't actually able to work on this over the last week -- This is an automated message

[GitHub] [airflow] asaf400 commented on pull request #19030: Align EcsOperator with DockerOperator capabilities

2021-10-27 Thread GitBox
asaf400 commented on pull request #19030: URL: https://github.com/apache/airflow/pull/19030#issuecomment-953428954 Okay, I'll be able to invest more time into the comments in the next few day or two, I wasn't actually able to work on this over the last week -- This is an automated

[GitHub] [airflow] mik-laj commented on a change in pull request #19189: Swtich both PROD and CI images to use virtual environments

2021-10-27 Thread GitBox
mik-laj commented on a change in pull request #19189: URL: https://github.com/apache/airflow/pull/19189#discussion_r737954160 ## File path: Dockerfile ## @@ -183,16 +183,19 @@ ENV INSTALL_MYSQL_CLIENT=${INSTALL_MYSQL_CLIENT} \

[GitHub] [airflow] mik-laj commented on a change in pull request #19189: Swtich both PROD and CI images to use virtual environments

2021-10-27 Thread GitBox
mik-laj commented on a change in pull request #19189: URL: https://github.com/apache/airflow/pull/19189#discussion_r737953527 ## File path: Dockerfile ## @@ -183,16 +182,20 @@ ENV INSTALL_MYSQL_CLIENT=${INSTALL_MYSQL_CLIENT} \

[GitHub] [airflow] mik-laj commented on a change in pull request #19189: Swtich both PROD and CI images to use virtual environments

2021-10-27 Thread GitBox
mik-laj commented on a change in pull request #19189: URL: https://github.com/apache/airflow/pull/19189#discussion_r737952212 ## File path: Dockerfile ## @@ -183,16 +183,19 @@ ENV INSTALL_MYSQL_CLIENT=${INSTALL_MYSQL_CLIENT} \

[GitHub] [airflow] mik-laj commented on pull request #19263: Chart: add resources for cleanup and createuser jobs

2021-10-27 Thread GitBox
mik-laj commented on pull request #19263: URL: https://github.com/apache/airflow/pull/19263#issuecomment-953421813 @Aakcht We have reference documentation that allows a Helm / Kubernetes expert to take advantage of this option. However, we lack a guide that will allow a less experienced

[GitHub] [airflow] potiuk commented on a change in pull request #19189: Swtich both PROD and CI images to use virtual environments

2021-10-27 Thread GitBox
potiuk commented on a change in pull request #19189: URL: https://github.com/apache/airflow/pull/19189#discussion_r737947221 ## File path: Dockerfile ## @@ -441,25 +443,53 @@ RUN chmod a+x /scripts/docker/install_mysql.sh && \ find "${AIRFLOW_HOME}" -executable -print0 |

[GitHub] [airflow] potiuk commented on a change in pull request #19189: Swtich both PROD and CI images to use virtual environments

2021-10-27 Thread GitBox
potiuk commented on a change in pull request #19189: URL: https://github.com/apache/airflow/pull/19189#discussion_r737947221 ## File path: Dockerfile ## @@ -441,25 +443,53 @@ RUN chmod a+x /scripts/docker/install_mysql.sh && \ find "${AIRFLOW_HOME}" -executable -print0 |

[GitHub] [airflow] Aakcht commented on pull request #19263: Chart: add resources for cleanup and createuser jobs

2021-10-27 Thread GitBox
Aakcht commented on pull request #19263: URL: https://github.com/apache/airflow/pull/19263#issuecomment-953412760 @mik-laj , do you want to describe it specifically on the page http://apache-airflow-docs.s3-website.eu-central-1.amazonaws.com/docs/helm-chart/latest/index.html ? From what I

[GitHub] [airflow] josh-fell commented on a change in pull request #19237: Cleanup of start_date and default arg use for Amazon example DAGs

2021-10-27 Thread GitBox
josh-fell commented on a change in pull request #19237: URL: https://github.com/apache/airflow/pull/19237#discussion_r737941955 ## File path: airflow/providers/amazon/aws/example_dags/example_dms_full_load_task.py ## @@ -50,20 +49,14 @@ with DAG(

[GitHub] [airflow] josh-fell commented on a change in pull request #19237: Cleanup of start_date and default arg use for Amazon example DAGs

2021-10-27 Thread GitBox
josh-fell commented on a change in pull request #19237: URL: https://github.com/apache/airflow/pull/19237#discussion_r737940767 ## File path: airflow/providers/amazon/aws/example_dags/example_dms_full_load_task.py ## @@ -50,20 +49,14 @@ with DAG(

[GitHub] [airflow] potiuk commented on a change in pull request #19189: Swtich both PROD and CI images to use virtual environments

2021-10-27 Thread GitBox
potiuk commented on a change in pull request #19189: URL: https://github.com/apache/airflow/pull/19189#discussion_r737939995 ## File path: Dockerfile ## @@ -21,13 +21,13 @@ # # airflow-build-image - there all airflow dependencies can be installed (and #

[GitHub] [airflow] potiuk commented on a change in pull request #19189: Swtich both PROD and CI images to use virtual environments

2021-10-27 Thread GitBox
potiuk commented on a change in pull request #19189: URL: https://github.com/apache/airflow/pull/19189#discussion_r737939807 ## File path: Dockerfile ## @@ -399,12 +405,9 @@ ENV RUNTIME_APT_DEPS=${RUNTIME_APT_DEPS} \ AIRFLOW__CORE__LOAD_EXAMPLES="false" \

[GitHub] [airflow] potiuk commented on a change in pull request #19189: Swtich both PROD and CI images to use virtual environments

2021-10-27 Thread GitBox
potiuk commented on a change in pull request #19189: URL: https://github.com/apache/airflow/pull/19189#discussion_r737939320 ## File path: Dockerfile ## @@ -382,12 +392,8 @@ ARG AIRFLOW_HOME # Having the variable in final image allows to disable providers manager warnings

[GitHub] [airflow] potiuk commented on a change in pull request #19189: Swtich both PROD and CI images to use virtual environments

2021-10-27 Thread GitBox
potiuk commented on a change in pull request #19189: URL: https://github.com/apache/airflow/pull/19189#discussion_r737939320 ## File path: Dockerfile ## @@ -382,12 +392,8 @@ ARG AIRFLOW_HOME # Having the variable in final image allows to disable providers manager warnings

[GitHub] [airflow] potiuk commented on a change in pull request #19189: Swtich both PROD and CI images to use virtual environments

2021-10-27 Thread GitBox
potiuk commented on a change in pull request #19189: URL: https://github.com/apache/airflow/pull/19189#discussion_r737936033 ## File path: Dockerfile ## @@ -183,16 +183,19 @@ ENV INSTALL_MYSQL_CLIENT=${INSTALL_MYSQL_CLIENT} \

[GitHub] [airflow] potiuk commented on a change in pull request #19189: Swtich both PROD and CI images to use virtual environments

2021-10-27 Thread GitBox
potiuk commented on a change in pull request #19189: URL: https://github.com/apache/airflow/pull/19189#discussion_r737936033 ## File path: Dockerfile ## @@ -183,16 +183,19 @@ ENV INSTALL_MYSQL_CLIENT=${INSTALL_MYSQL_CLIENT} \

[airflow] branch main updated (20847fd -> 83eb80d)

2021-10-27 Thread potiuk
This is an automated email from the ASF dual-hosted git repository. potiuk pushed a change to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git. from 20847fd Add value to "namespaceId" of query (#19163) add 83eb80d SFTP hook to prefer the SSH paramiko key over

[GitHub] [airflow] potiuk merged pull request #18988: SFTP hook to prefer the SSH paramiko key over the key file path

2021-10-27 Thread GitBox
potiuk merged pull request #18988: URL: https://github.com/apache/airflow/pull/18988 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[GitHub] [airflow] boring-cyborg[bot] commented on pull request #18988: SFTP hook to prefer the SSH paramiko key over the key file path

2021-10-27 Thread GitBox
boring-cyborg[bot] commented on pull request #18988: URL: https://github.com/apache/airflow/pull/18988#issuecomment-953401976 Awesome work, congrats on your first merged pull request! -- This is an automated message from the Apache Git Service. To respond to the message, please log

[GitHub] [airflow] jedcunningham edited a comment on issue #18939: Upgrading from version 2.1.4 to 2.2.0 fails during mysql db upgrade with error Can't DROP 'dag_id'

2021-10-27 Thread GitBox
jedcunningham edited a comment on issue #18939: URL: https://github.com/apache/airflow/issues/18939#issuecomment-953395944 I was able to reproduce this. The following traceback is the original failure: ```

[GitHub] [airflow] jedcunningham opened a new pull request #19268: Fix MySQL db migration with default encoding/collation

2021-10-27 Thread GitBox
jedcunningham opened a new pull request #19268: URL: https://github.com/apache/airflow/pull/19268 This allows the migrations to complete for MySQL db's without proper encoding/collation defaults. I almost feel that we should be blocking migrations from running if they aren't set

[GitHub] [airflow] github-actions[bot] closed pull request #17933: enable setting of pod specific annotations

2021-10-27 Thread GitBox
github-actions[bot] closed pull request #17933: URL: https://github.com/apache/airflow/pull/17933 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[GitHub] [airflow] github-actions[bot] closed pull request #16636: Added dashes to pod naming

2021-10-27 Thread GitBox
github-actions[bot] closed pull request #16636: URL: https://github.com/apache/airflow/pull/16636 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[GitHub] [airflow] mik-laj commented on pull request #19263: Chart: add resources for cleanup and createuser jobs

2021-10-27 Thread GitBox
mik-laj commented on pull request #19263: URL: https://github.com/apache/airflow/pull/19263#issuecomment-953396458 CC: @simplylizz -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

[GitHub] [airflow] mik-laj commented on pull request #19263: Chart: add resources for cleanup and createuser jobs

2021-10-27 Thread GitBox
mik-laj commented on pull request #19263: URL: https://github.com/apache/airflow/pull/19263#issuecomment-953396230 We have a few options for resource configuration, but [no documentation](http://apache-airflow-docs.s3-website.eu-central-1.amazonaws.com/docs/helm-chart/latest/index.html)

[GitHub] [airflow] potiuk commented on issue #19192: The scheduler does not appear to be running. Last heartbeat was received X minutes ago.

2021-10-27 Thread GitBox
potiuk commented on issue #19192: URL: https://github.com/apache/airflow/issues/19192#issuecomment-953395948 It looks like you are actually using SequentialExecutor. That would perfectly explain the behaviour. Are you sure you are using local executor and scheduler is running ? Can you

[GitHub] [airflow] jedcunningham commented on issue #18939: Upgrading from version 2.1.4 to 2.2.0 fails during mysql db upgrade with error Can't DROP 'dag_id'

2021-10-27 Thread GitBox
jedcunningham commented on issue #18939: URL: https://github.com/apache/airflow/issues/18939#issuecomment-953395944 I was able to reproduce this. The following traceback is the original failure: ```

[GitHub] [airflow] dstandish opened a new pull request #19267: Fix serialization of Params with set data type

2021-10-27 Thread GitBox
dstandish opened a new pull request #19267: URL: https://github.com/apache/airflow/pull/19267 OK this may be a solution for https://github.com/apache/airflow/issues/19096 Though I'll appreciate comment on the approach. Just wanted to get something working initially though. --

[GitHub] [airflow] potiuk closed issue #19188: Cannot put uploaded dag into the 'active state'. I can for a local docker-compose version.

2021-10-27 Thread GitBox
potiuk closed issue #19188: URL: https://github.com/apache/airflow/issues/19188 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[airflow] branch main updated (d51c149 -> 20847fd)

2021-10-27 Thread potiuk
This is an automated email from the ASF dual-hosted git repository. potiuk pushed a change to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git. from d51c149 Edit permalinks in OpenApi description file (#19244) add 20847fd Add value to "namespaceId" of query

[GitHub] [airflow] potiuk merged pull request #19163: Add value to "namespaceId" of query

2021-10-27 Thread GitBox
potiuk merged pull request #19163: URL: https://github.com/apache/airflow/pull/19163 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[airflow] branch main updated (bce8883 -> d51c149)

2021-10-27 Thread kamilbregula
This is an automated email from the ASF dual-hosted git repository. kamilbregula pushed a change to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git. from bce8883 sqlite_default has been hard-coded to /tmp, usegettempdir instead, (#19255) add d51c149 Edit

[GitHub] [airflow] mik-laj merged pull request #19244: Edit permalinks in OpenApi description file

2021-10-27 Thread GitBox
mik-laj merged pull request #19244: URL: https://github.com/apache/airflow/pull/19244 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[GitHub] [airflow] jsnb-devoted commented on issue #12236: Logging to S3 stops working when MLFlow import added to file

2021-10-27 Thread GitBox
jsnb-devoted commented on issue #12236: URL: https://github.com/apache/airflow/issues/12236#issuecomment-953369499 Just ran into this issue as our team is starting to use MLFlow: `airflow: 2.1.3 (kubernetes executor)` `mlflow: 1.20.0` All of our DAGs are able to send logs up to

[GitHub] [airflow] potiuk commented on pull request #19257: fix: dag_id and task_id non ascii char

2021-10-27 Thread GitBox
potiuk commented on pull request #19257: URL: https://github.com/apache/airflow/pull/19257#issuecomment-953369332 Some other static check still fails . Installing pre-commit locally usualy helps :). -- This is an automated message from the Apache Git Service. To respond to the message,

[GitHub] [airflow] potiuk commented on pull request #19257: fix: dag_id and task_id non ascii char

2021-10-27 Thread GitBox
potiuk commented on pull request #19257: URL: https://github.com/apache/airflow/pull/19257#issuecomment-953369446 Just less iterations -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

[GitHub] [airflow] chinwobble commented on issue #18999: Update databricks provider to use TriggerOperator

2021-10-27 Thread GitBox
chinwobble commented on issue #18999: URL: https://github.com/apache/airflow/issues/18999#issuecomment-953359692 @eskarimov I have implemented a prototype like this: I'm sure many improvements could be made but this should work. ```python # pylint: disable=abstract-method

[airflow] branch main updated (b49b81a -> bce8883)

2021-10-27 Thread potiuk
This is an automated email from the ASF dual-hosted git repository. potiuk pushed a change to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git. from b49b81a Fix release check script (#19238) add bce8883 sqlite_default has been hard-coded to /tmp,

[GitHub] [airflow] potiuk merged pull request #19255: Fix hard-coded /tmp dir in sqlite_default

2021-10-27 Thread GitBox
potiuk merged pull request #19255: URL: https://github.com/apache/airflow/pull/19255 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[GitHub] [airflow] boring-cyborg[bot] commented on pull request #19255: Fix hard-coded /tmp dir in sqlite_default

2021-10-27 Thread GitBox
boring-cyborg[bot] commented on pull request #19255: URL: https://github.com/apache/airflow/pull/19255#issuecomment-953349477 Awesome work, congrats on your first merged pull request! -- This is an automated message from the Apache Git Service. To respond to the message, please log

[GitHub] [airflow] o-nikolas commented on a change in pull request #19237: Cleanup of start_date and default arg use for Amazon example DAGs

2021-10-27 Thread GitBox
o-nikolas commented on a change in pull request #19237: URL: https://github.com/apache/airflow/pull/19237#discussion_r737881176 ## File path: airflow/providers/amazon/aws/example_dags/example_google_api_to_s3_transfer_advanced.py ## @@ -33,13 +33,13 @@ on a YouTube channel

[GitHub] [airflow] Melodie97 commented on pull request #19257: fix: dag_id and task_id non ascii char

2021-10-27 Thread GitBox
Melodie97 commented on pull request #19257: URL: https://github.com/apache/airflow/pull/19257#issuecomment-953345178 Hi @potiuk, @uranusjr, I've fixed the issues, pls can you review the PR -- This is an automated message from the Apache Git Service. To respond to the message, please log

[GitHub] [airflow] o-nikolas commented on a change in pull request #19237: Cleanup of start_date and default arg use for Amazon example DAGs

2021-10-27 Thread GitBox
o-nikolas commented on a change in pull request #19237: URL: https://github.com/apache/airflow/pull/19237#discussion_r737876692 ## File path: airflow/providers/amazon/aws/example_dags/example_dms_full_load_task.py ## @@ -50,20 +49,14 @@ with DAG(

[GitHub] [airflow] potiuk commented on a change in pull request #19237: Cleanup of start_date and default arg use for Amazon example DAGs

2021-10-27 Thread GitBox
potiuk commented on a change in pull request #19237: URL: https://github.com/apache/airflow/pull/19237#discussion_r737875169 ## File path: airflow/providers/amazon/aws/example_dags/example_dms_full_load_task.py ## @@ -50,20 +49,14 @@ with DAG(

[GitHub] [airflow] potiuk commented on a change in pull request #19237: Cleanup of start_date and default arg use for Amazon example DAGs

2021-10-27 Thread GitBox
potiuk commented on a change in pull request #19237: URL: https://github.com/apache/airflow/pull/19237#discussion_r737875169 ## File path: airflow/providers/amazon/aws/example_dags/example_dms_full_load_task.py ## @@ -50,20 +49,14 @@ with DAG(

[GitHub] [airflow] potiuk commented on a change in pull request #19237: Cleanup of start_date and default arg use for Amazon example DAGs

2021-10-27 Thread GitBox
potiuk commented on a change in pull request #19237: URL: https://github.com/apache/airflow/pull/19237#discussion_r737875169 ## File path: airflow/providers/amazon/aws/example_dags/example_dms_full_load_task.py ## @@ -50,20 +49,14 @@ with DAG(

[GitHub] [airflow] o-nikolas commented on issue #19264: TriggerDagRunOperator External Link doesn't link with execution date to match to triggered DAG

2021-10-27 Thread GitBox
o-nikolas commented on issue #19264: URL: https://github.com/apache/airflow/issues/19264#issuecomment-953339516 Hey @ljades, I looked into this, but couldn't replicate it on current main branch. Do you have example dag(s) code to repro the issue? -- This is an automated message

[GitHub] [airflow] potiuk commented on a change in pull request #19237: Cleanup of start_date and default arg use for Amazon example DAGs

2021-10-27 Thread GitBox
potiuk commented on a change in pull request #19237: URL: https://github.com/apache/airflow/pull/19237#discussion_r737873020 ## File path: airflow/providers/amazon/aws/example_dags/example_google_api_to_s3_transfer_advanced.py ## @@ -33,13 +33,13 @@ on a YouTube channel you

[GitHub] [airflow] josh-fell commented on a change in pull request #19237: Cleanup of start_date and default arg use for Amazon example DAGs

2021-10-27 Thread GitBox
josh-fell commented on a change in pull request #19237: URL: https://github.com/apache/airflow/pull/19237#discussion_r737829769 ## File path: airflow/providers/amazon/aws/example_dags/example_google_api_to_s3_transfer_advanced.py ## @@ -33,13 +33,13 @@ on a YouTube channel

[GitHub] [airflow] dstandish commented on issue #19096: Sets in parameters cause "TypeError: Object of type set is not JSON serializable"

2021-10-27 Thread GitBox
dstandish commented on issue #19096: URL: https://github.com/apache/airflow/issues/19096#issuecomment-953329485 OK so, I found the source of the the problem but need to look into it more to see what we need to do. Serialization blows up here because it's not json encodable:

[GitHub] [airflow] Aakcht commented on pull request #19263: Chart: add resources for cleanup and createuser jobs

2021-10-27 Thread GitBox
Aakcht commented on pull request #19263: URL: https://github.com/apache/airflow/pull/19263#issuecomment-953321166 Looks like the failing check is not related to this PR: `Error: Process completed with exit code 137.` -- This is an automated message from the Apache Git Service. To

[GitHub] [airflow] josh-fell commented on a change in pull request #19237: Cleanup of start_date and default arg use for Amazon example DAGs

2021-10-27 Thread GitBox
josh-fell commented on a change in pull request #19237: URL: https://github.com/apache/airflow/pull/19237#discussion_r737847953 ## File path: airflow/providers/amazon/aws/example_dags/example_dms_full_load_task.py ## @@ -50,20 +49,14 @@ with DAG(

[GitHub] [airflow] github-actions[bot] commented on pull request #19265: Rename variable `serialize_dag` to `serialized_dag`

2021-10-27 Thread GitBox
github-actions[bot] commented on pull request #19265: URL: https://github.com/apache/airflow/pull/19265#issuecomment-953317741 The PR most likely needs to run full matrix of tests because it modifies parts of the core of Airflow. However, committers might decide to merge it quickly and

[GitHub] [airflow] dstandish opened a new pull request #19265: Rename variable `serialize_dag` to `serialized_dag`

2021-10-27 Thread GitBox
dstandish opened a new pull request #19265: URL: https://github.com/apache/airflow/pull/19265 The expression `serialize_dag` represents an action, but the variable is actually a dag that has been _serialized_. -- This is an automated message from the Apache Git Service. To respond

[GitHub] [airflow] josh-fell commented on a change in pull request #19237: Cleanup of start_date and default arg use for Amazon example DAGs

2021-10-27 Thread GitBox
josh-fell commented on a change in pull request #19237: URL: https://github.com/apache/airflow/pull/19237#discussion_r737847953 ## File path: airflow/providers/amazon/aws/example_dags/example_dms_full_load_task.py ## @@ -50,20 +49,14 @@ with DAG(

[GitHub] [airflow] josh-fell commented on a change in pull request #19237: Cleanup of start_date and default arg use for Amazon example DAGs

2021-10-27 Thread GitBox
josh-fell commented on a change in pull request #19237: URL: https://github.com/apache/airflow/pull/19237#discussion_r737829769 ## File path: airflow/providers/amazon/aws/example_dags/example_google_api_to_s3_transfer_advanced.py ## @@ -33,13 +33,13 @@ on a YouTube channel

[GitHub] [airflow] josh-fell commented on a change in pull request #19237: Cleanup of start_date and default arg use for Amazon example DAGs

2021-10-27 Thread GitBox
josh-fell commented on a change in pull request #19237: URL: https://github.com/apache/airflow/pull/19237#discussion_r737829769 ## File path: airflow/providers/amazon/aws/example_dags/example_google_api_to_s3_transfer_advanced.py ## @@ -33,13 +33,13 @@ on a YouTube channel

[GitHub] [airflow] kaxil closed pull request #19262: Updates fernet.rst for correct documentation of CLI and nitpick around bolded environment variable

2021-10-27 Thread GitBox
kaxil closed pull request #19262: URL: https://github.com/apache/airflow/pull/19262 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[GitHub] [airflow] mehmax commented on issue #19196: Allow to turn dag run state to skipped

2021-10-27 Thread GitBox
mehmax commented on issue #19196: URL: https://github.com/apache/airflow/issues/19196#issuecomment-953255967 Would it be sufficient to add a "Mark Skipped" to the TaskInstance view?: e.g:

[GitHub] [airflow] potiuk commented on a change in pull request #19262: Updates fernet.rst for correct documentation of CLI and nitpick around bolded environment variable

2021-10-27 Thread GitBox
potiuk commented on a change in pull request #19262: URL: https://github.com/apache/airflow/pull/19262#discussion_r737770879 ## File path: docs/apache-airflow/security/secrets/fernet.rst ## @@ -59,5 +59,5 @@ the new key to the ``fernet_key`` setting, run ``fernet_key``: #.

  1   2   3   >