[GitHub] [airflow] imamdigmi commented on issue #16828: Cannot Set Index Pattern on Elasticsearch as a Log Handler

2021-07-06 Thread GitBox
imamdigmi commented on issue #16828: URL: https://github.com/apache/airflow/issues/16828#issuecomment-875284402 Hi @jedcunningham thanks for your suggestion, I have tried it, and it works ![20210707120518

[GitHub] [airflow] dstandish commented on a change in pull request #16821: Add more filter options to list_keys of S3Hook

2021-07-06 Thread GitBox
dstandish commented on a change in pull request #16821: URL: https://github.com/apache/airflow/pull/16821#discussion_r665031254 ## File path: airflow/providers/amazon/aws/hooks/s3.py ## @@ -292,18 +302,24 @@ def list_keys( 'PageSize': page_size,

[GitHub] [airflow] dstandish commented on a change in pull request #16821: Add more filter options to list_keys of S3Hook

2021-07-06 Thread GitBox
dstandish commented on a change in pull request #16821: URL: https://github.com/apache/airflow/pull/16821#discussion_r664912477 ## File path: airflow/providers/amazon/aws/hooks/s3.py ## @@ -292,18 +302,24 @@ def list_keys( 'PageSize': page_size,

[GitHub] [airflow] dstandish commented on a change in pull request #16821: Add more filter options to list_keys of S3Hook

2021-07-06 Thread GitBox
dstandish commented on a change in pull request #16821: URL: https://github.com/apache/airflow/pull/16821#discussion_r665027960 ## File path: airflow/providers/amazon/aws/hooks/s3.py ## @@ -292,18 +302,24 @@ def list_keys( 'PageSize': page_size,

[GitHub] [airflow] dstandish edited a comment on pull request #16821: Add more filter options to list_keys of S3Hook

2021-07-06 Thread GitBox
dstandish edited a comment on pull request #16821: URL: https://github.com/apache/airflow/pull/16821#issuecomment-875262841 > Let me offer a recommendation. > > Rather than adding `start_after_datetime` and `to_datetime`, just add a single param called `object_filter` that takes a

[GitHub] [airflow] dstandish commented on pull request #16821: Add more filter options to list_keys of S3Hook

2021-07-06 Thread GitBox
dstandish commented on pull request #16821: URL: https://github.com/apache/airflow/pull/16821#issuecomment-875262841 > Let me offer a recommendation. > > Rather than adding `start_after_datetime` and `to_datetime`, just add a single param called `object_filter` that takes a function

[GitHub] [airflow] edwardwang888 edited a comment on issue #16824: Initiating `mark success/failed` operation causes UI border to disappear for task instances with no status

2021-07-06 Thread GitBox
edwardwang888 edited a comment on issue #16824: URL: https://github.com/apache/airflow/issues/16824#issuecomment-875241895 A few more observations: - I think the missing borders are due to the run being a manual run - the borderless white box is meant to display when the task instance

[GitHub] [airflow] boring-cyborg[bot] commented on issue #16847: Marking task as UP_FOR_RETRY but didn't retry

2021-07-06 Thread GitBox
boring-cyborg[bot] commented on issue #16847: URL: https://github.com/apache/airflow/issues/16847#issuecomment-875256438 Thanks for opening your first issue here! Be sure to follow the issue template! -- This is an automated message from the Apache Git Service. To respond to the

[GitHub] [airflow] david30907d opened a new issue #16847: Marking task as UP_FOR_RETRY but didn't retry

2021-07-06 Thread GitBox
david30907d opened a new issue #16847: URL: https://github.com/apache/airflow/issues/16847 **Apache Airflow version**: `2.1.1rc1` **Kubernetes version (if you are using kubernetes)** (use `kubectl version`): 1.18 **Environment**: - **Cloud provider

[GitHub] [airflow] yuqian90 opened a new issue #16846: Better integration with systemd

2021-07-06 Thread GitBox
yuqian90 opened a new issue #16846: URL: https://github.com/apache/airflow/issues/16846 **Description** Airflow services (scheduler, worker and webserver) can be managed with systemd. Some small improvements will make this experience better: - systemd has a feature called

[GitHub] [airflow] edwardwang888 edited a comment on issue #16824: Initiating `mark success/failed` operation causes UI border to disappear for task instances with no status

2021-07-06 Thread GitBox
edwardwang888 edited a comment on issue #16824: URL: https://github.com/apache/airflow/issues/16824#issuecomment-875241895 A few more observations: - I think the missing borders are due to the run being a manual run - the borderless white box is meant to display when the task instance

[GitHub] [airflow] edwardwang888 edited a comment on issue #16824: Initiating `mark success/failed` operation causes UI border to disappear for task instances with no status

2021-07-06 Thread GitBox
edwardwang888 edited a comment on issue #16824: URL: https://github.com/apache/airflow/issues/16824#issuecomment-875241895 A few more observations: - I think the missing borders are due to the run being a manual run - the borderless white box is meant to display when the task instance

[GitHub] [airflow] edwardwang888 edited a comment on issue #16824: Initiating `mark success/failed` operation causes UI border to disappear for task instances with no status

2021-07-06 Thread GitBox
edwardwang888 edited a comment on issue #16824: URL: https://github.com/apache/airflow/issues/16824#issuecomment-875241895 A few more observations: - I think the missing borders are due to the run being a manual run - the borderless white box is meant to display when the task instance

[GitHub] [airflow] edwardwang888 edited a comment on issue #16824: Initiating `mark success/failed` operation causes UI border to disappear for task instances with no status

2021-07-06 Thread GitBox
edwardwang888 edited a comment on issue #16824: URL: https://github.com/apache/airflow/issues/16824#issuecomment-875241895 A few more observations: - I think the missing borders are due to the run being a manual run - the borderless white box is meant to display when the task instance

[GitHub] [airflow] edwardwang888 edited a comment on issue #16824: Initiating `mark success/failed` operation causes UI border to disappear for task instances with no status

2021-07-06 Thread GitBox
edwardwang888 edited a comment on issue #16824: URL: https://github.com/apache/airflow/issues/16824#issuecomment-875161577 I actually prefer having non-existent tasks still render with the outline, the reason being that after adding a task, there is sometimes a need to mark the

[GitHub] [airflow] edwardwang888 commented on issue #16824: Initiating `mark success/failed` operation causes UI border to disappear for task instances with no status

2021-07-06 Thread GitBox
edwardwang888 commented on issue #16824: URL: https://github.com/apache/airflow/issues/16824#issuecomment-875241895 A few more observations: - I think the missing borders are due to the run being a manual run - the borderless white box is meant to display when the task instance

[GitHub] [airflow] huozhanfeng commented on a change in pull request #16768: Fix bug that log can't be shown when task runs failed

2021-07-06 Thread GitBox
huozhanfeng commented on a change in pull request #16768: URL: https://github.com/apache/airflow/pull/16768#discussion_r664999838 ## File path: airflow/dag_processing/processor.py ## @@ -568,14 +568,11 @@ def _execute_task_callbacks(self, dagbag: DagBag, request:

[GitHub] [airflow] uranusjr commented on a change in pull request #16768: Fix bug that log can't be shown when task runs failed

2021-07-06 Thread GitBox
uranusjr commented on a change in pull request #16768: URL: https://github.com/apache/airflow/pull/16768#discussion_r664998123 ## File path: airflow/dag_processing/processor.py ## @@ -568,14 +568,11 @@ def _execute_task_callbacks(self, dagbag: DagBag, request:

[GitHub] [airflow] psg2 commented on pull request #16842: Fix/consistent get dag rest endpoint

2021-07-06 Thread GitBox
psg2 commented on pull request #16842: URL: https://github.com/apache/airflow/pull/16842#issuecomment-875217766 > When you select "All", you should see deleted and active DAG in UI. If not, then we have a bug in the Web UI. What about those recent changes on the changelog?

[GitHub] [airflow] mik-laj commented on pull request #16842: Fix/consistent get dag rest endpoint

2021-07-06 Thread GitBox
mik-laj commented on pull request #16842: URL: https://github.com/apache/airflow/pull/16842#issuecomment-875217704 I have the impression that this change caused this change, but the Airflow 1.10 had a different behavior in this situation.

[GitHub] [airflow] mik-laj commented on pull request #16842: Fix/consistent get dag rest endpoint

2021-07-06 Thread GitBox
mik-laj commented on pull request #16842: URL: https://github.com/apache/airflow/pull/16842#issuecomment-875209144 > If I try to access the DAG after file deletion the Web UI I receive the SerializedDagNotFound exception We can think of a clearer error message here, but from the

[GitHub] [airflow] mik-laj commented on pull request #16842: Fix/consistent get dag rest endpoint

2021-07-06 Thread GitBox
mik-laj commented on pull request #16842: URL: https://github.com/apache/airflow/pull/16842#issuecomment-875208525 > Web UI is now powered by Serialized Dags that table would be the source of truth. Only some views use SerializedDAG because reading the data from this table is

[GitHub] [airflow] psg2 commented on pull request #16842: Fix/consistent get dag rest endpoint

2021-07-06 Thread GitBox
psg2 commented on pull request #16842: URL: https://github.com/apache/airflow/pull/16842#issuecomment-875208447 If I try to access the DAG after file deletion the Web UI I receive the SerializedDagNotFound exception

[GitHub] [airflow] gyscos commented on a change in pull request #16845: Add state filter to dag_runs endpoint

2021-07-06 Thread GitBox
gyscos commented on a change in pull request #16845: URL: https://github.com/apache/airflow/pull/16845#discussion_r664984322 ## File path: airflow/api_connexion/parameters.py ## @@ -89,6 +89,13 @@ def wrapped_function(*args, **kwargs): return format_parameters_decorator

[GitHub] [airflow] psg2 commented on a change in pull request #16842: Fix/consistent get dag rest endpoint

2021-07-06 Thread GitBox
psg2 commented on a change in pull request #16842: URL: https://github.com/apache/airflow/pull/16842#discussion_r664984122 ## File path: airflow/api_connexion/endpoints/dag_endpoint.py ## @@ -34,14 +34,14 @@ @security.requires_access([(permissions.ACTION_CAN_READ,

[GitHub] [airflow] psg2 commented on a change in pull request #16842: Fix/consistent get dag rest endpoint

2021-07-06 Thread GitBox
psg2 commented on a change in pull request #16842: URL: https://github.com/apache/airflow/pull/16842#discussion_r664983577 ## File path: tests/api_connexion/endpoints/test_dag_endpoint.py ## @@ -137,54 +155,44 @@ def _create_deactivated_dag(self, session=None): class

[GitHub] [airflow] psg2 commented on pull request #16842: Fix/consistent get dag rest endpoint

2021-07-06 Thread GitBox
psg2 commented on pull request #16842: URL: https://github.com/apache/airflow/pull/16842#issuecomment-875206435 > > I just tested it on Airflow 2.1.1 and if I remove the file for a DAG with existing runs the SerializedDag table is cleaned up and because of this the DAG no longer shows up

[GitHub] [airflow] mik-laj commented on pull request #16842: Fix/consistent get dag rest endpoint

2021-07-06 Thread GitBox
mik-laj commented on pull request #16842: URL: https://github.com/apache/airflow/pull/16842#issuecomment-875205150 > I just tested it on Airflow 2.1.1 and if I remove the file for a DAG with existing runs the SerializedDag table is cleaned up and because of this the DAG no longer shows up

[GitHub] [airflow] gyscos commented on a change in pull request #16845: Add state filter to dag_runs endpoint

2021-07-06 Thread GitBox
gyscos commented on a change in pull request #16845: URL: https://github.com/apache/airflow/pull/16845#discussion_r664980573 ## File path: airflow/api_connexion/endpoints/task_instance_endpoint.py ## @@ -120,7 +114,7 @@ def get_task_instances( end_date_lte: Optional[str]

[GitHub] [airflow] gyscos commented on a change in pull request #16845: Add state filter to dag_runs endpoint

2021-07-06 Thread GitBox
gyscos commented on a change in pull request #16845: URL: https://github.com/apache/airflow/pull/16845#discussion_r664980430 ## File path: airflow/api_connexion/endpoints/dag_run_endpoint.py ## @@ -218,6 +226,7 @@ def get_dag_runs_batch(session):

[GitHub] [airflow] psg2 commented on pull request #16842: Fix/consistent get dag rest endpoint

2021-07-06 Thread GitBox
psg2 commented on pull request #16842: URL: https://github.com/apache/airflow/pull/16842#issuecomment-875203560 > I am not sure about this change. Web UI displays DAGs that are deleted. This allows us to access the archival DAG Runes. > > If you want to fetch a list of DAGs that

[GitHub] [airflow] mik-laj commented on a change in pull request #16842: Fix/consistent get dag rest endpoint

2021-07-06 Thread GitBox
mik-laj commented on a change in pull request #16842: URL: https://github.com/apache/airflow/pull/16842#discussion_r664979943 ## File path: tests/api_connexion/endpoints/test_dag_endpoint.py ## @@ -137,54 +155,44 @@ def _create_deactivated_dag(self, session=None): class

[GitHub] [airflow] mik-laj commented on a change in pull request #16842: Fix/consistent get dag rest endpoint

2021-07-06 Thread GitBox
mik-laj commented on a change in pull request #16842: URL: https://github.com/apache/airflow/pull/16842#discussion_r664979838 ## File path: tests/api_connexion/endpoints/test_dag_endpoint.py ## @@ -137,54 +155,44 @@ def _create_deactivated_dag(self, session=None): class

[GitHub] [airflow] mik-laj commented on a change in pull request #16842: Fix/consistent get dag rest endpoint

2021-07-06 Thread GitBox
mik-laj commented on a change in pull request #16842: URL: https://github.com/apache/airflow/pull/16842#discussion_r664978929 ## File path: airflow/api_connexion/endpoints/dag_endpoint.py ## @@ -34,14 +34,14 @@ @security.requires_access([(permissions.ACTION_CAN_READ,

[GitHub] [airflow] mik-laj commented on a change in pull request #16842: Fix/consistent get dag rest endpoint

2021-07-06 Thread GitBox
mik-laj commented on a change in pull request #16842: URL: https://github.com/apache/airflow/pull/16842#discussion_r664978929 ## File path: airflow/api_connexion/endpoints/dag_endpoint.py ## @@ -34,14 +34,14 @@ @security.requires_access([(permissions.ACTION_CAN_READ,

[GitHub] [airflow] mik-laj commented on pull request #16842: Fix/consistent get dag rest endpoint

2021-07-06 Thread GitBox
mik-laj commented on pull request #16842: URL: https://github.com/apache/airflow/pull/16842#issuecomment-875201608 I am not sure about this change. Web UI displays DAGs that are deleted. This allows us to access the archival DAG Runes. If you want to fetch a list of DAGs that have

[GitHub] [airflow] ferruzzi commented on a change in pull request #16571: Implemented Basic EKS Integration

2021-07-06 Thread GitBox
ferruzzi commented on a change in pull request #16571: URL: https://github.com/apache/airflow/pull/16571#discussion_r664171804 ## File path: airflow/providers/amazon/aws/utils/eks_kube_config.py ## @@ -0,0 +1,133 @@ +# Licensed to the Apache Software Foundation (ASF) under one

[GitHub] [airflow] uranusjr commented on a change in pull request #16732: Update `chain()` and `cross_downstream()` to support XComArgs

2021-07-06 Thread GitBox
uranusjr commented on a change in pull request #16732: URL: https://github.com/apache/airflow/pull/16732#discussion_r664974960 ## File path: airflow/models/baseoperator.py ## @@ -1595,11 +1652,14 @@ def chain(*tasks: Union[BaseOperator, Sequence[BaseOperator]]): def

[GitHub] [airflow] uranusjr commented on a change in pull request #16845: Add state filter to dag_runs endpoint

2021-07-06 Thread GitBox
uranusjr commented on a change in pull request #16845: URL: https://github.com/apache/airflow/pull/16845#discussion_r664968581 ## File path: airflow/api_connexion/parameters.py ## @@ -89,6 +89,13 @@ def wrapped_function(*args, **kwargs): return format_parameters_decorator

[GitHub] [airflow] uranusjr commented on a change in pull request #16845: Add state filter to dag_runs endpoint

2021-07-06 Thread GitBox
uranusjr commented on a change in pull request #16845: URL: https://github.com/apache/airflow/pull/16845#discussion_r664967968 ## File path: airflow/api_connexion/endpoints/dag_run_endpoint.py ## @@ -144,11 +149,14 @@ def _fetch_dag_runs( start_date_gte,

[GitHub] [airflow] github-actions[bot] commented on pull request #16842: Fix/consistent get dag rest endpoint

2021-07-06 Thread GitBox
github-actions[bot] commented on pull request #16842: URL: https://github.com/apache/airflow/pull/16842#issuecomment-875188918 The PR is likely OK to be merged with just subset of tests for default Python and Database versions without running the full matrix of tests, because it does not

[GitHub] [airflow] mik-laj edited a comment on issue #16684: Helm Chart Allow for Empty data settings

2021-07-06 Thread GitBox
mik-laj edited a comment on issue #16684: URL: https://github.com/apache/airflow/issues/16684#issuecomment-875182606 > I am not sure if we can already do it in the current chart easily, This is the currently used supported solution by Helm Chart, but we want to add support for

[GitHub] [airflow] mik-laj commented on issue #16684: Helm Chart Allow for Empty data settings

2021-07-06 Thread GitBox
mik-laj commented on issue #16684: URL: https://github.com/apache/airflow/issues/16684#issuecomment-875182606 > I am not sure if we can already do it in the current chart easily, This is the currently used supported solution by Helm Chart, but we want to add support for non-native

[GitHub] [airflow] mik-laj commented on issue #16684: Helm Chart Allow for Empty data settings

2021-07-06 Thread GitBox
mik-laj commented on issue #16684: URL: https://github.com/apache/airflow/issues/16684#issuecomment-875180534 > using valueFrom option. It uses the native secret backend provided by Kubernetes. Unfortunately, he is not considered to be very safe and is quite limited, e.g. it does

[GitHub] [airflow] gyscos commented on a change in pull request #16845: Add state filter to dag_runs endpoint

2021-07-06 Thread GitBox
gyscos commented on a change in pull request #16845: URL: https://github.com/apache/airflow/pull/16845#discussion_r664957685 ## File path: airflow/api_connexion/endpoints/dag_run_endpoint.py ## @@ -144,11 +149,14 @@ def _fetch_dag_runs( start_date_gte,

[GitHub] [airflow] gyscos commented on a change in pull request #16845: Add state filter to dag_runs endpoint

2021-07-06 Thread GitBox
gyscos commented on a change in pull request #16845: URL: https://github.com/apache/airflow/pull/16845#discussion_r664957615 ## File path: airflow/api_connexion/endpoints/dag_run_endpoint.py ## @@ -125,15 +129,16 @@ def get_dag_runs( def _fetch_dag_runs( query, -

[GitHub] [airflow] gyscos commented on issue #16844: Rest API: allow filtering DagRuns by state.

2021-07-06 Thread GitBox
gyscos commented on issue #16844: URL: https://github.com/apache/airflow/issues/16844#issuecomment-875173323 I can try? Tentatively sent #16845 for that. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above

[GitHub] [airflow] gyscos opened a new pull request #16845: Add state filter to dag_runs endpoint

2021-07-06 Thread GitBox
gyscos opened a new pull request #16845: URL: https://github.com/apache/airflow/pull/16845 This adds a new `state` filter parameter to the dagRuns endpoints. closes: #16844 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [airflow] boring-cyborg[bot] commented on pull request #16845: Add state filter to dag_runs endpoint

2021-07-06 Thread GitBox
boring-cyborg[bot] commented on pull request #16845: URL: https://github.com/apache/airflow/pull/16845#issuecomment-875173251 Congratulations on your first Pull Request and welcome to the Apache Airflow community! If you have any issues or are unsure about any anything please check our

[GitHub] [airflow] edwardwang888 edited a comment on issue #16824: Initiating `mark success/failed` operation causes UI border to disappear for task instances with no status

2021-07-06 Thread GitBox
edwardwang888 edited a comment on issue #16824: URL: https://github.com/apache/airflow/issues/16824#issuecomment-875161577 I actually prefer having non-existent tasks still render with the outline, the reason being that after adding a task, there is sometimes a need to mark the

[GitHub] [airflow] edwardwang888 edited a comment on issue #16824: Initiating `mark success/failed` operation causes UI border to disappear for task instances with no status

2021-07-06 Thread GitBox
edwardwang888 edited a comment on issue #16824: URL: https://github.com/apache/airflow/issues/16824#issuecomment-875161577 I actually prefer having non-existent tasks still render with the outline, the reason being that after adding a task, there is sometimes a need to mark the

[GitHub] [airflow] edwardwang888 edited a comment on issue #16824: Initiating `mark success/failed` operation causes UI border to disappear for task instances with no status

2021-07-06 Thread GitBox
edwardwang888 edited a comment on issue #16824: URL: https://github.com/apache/airflow/issues/16824#issuecomment-875161577 I actually prefer having non-existent tasks still render with the outline, the reason being that after adding a task, there is sometimes a need to mark the

[GitHub] [airflow] edwardwang888 edited a comment on issue #16824: Initiating `mark success/failed` operation causes UI border to disappear for task instances with no status

2021-07-06 Thread GitBox
edwardwang888 edited a comment on issue #16824: URL: https://github.com/apache/airflow/issues/16824#issuecomment-875161577 I actually prefer having non-existent tasks still render with the outline, the reason being that after adding a task, there is sometimes a need to mark the

[GitHub] [airflow] uranusjr commented on issue #16844: Rest API: allow filtering DagRuns by state.

2021-07-06 Thread GitBox
uranusjr commented on issue #16844: URL: https://github.com/apache/airflow/issues/16844#issuecomment-875163235 Hi! Would you be interested in submmitting a pull request for this? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [airflow] github-actions[bot] commented on issue #14938: docker airflow-init is stuck

2021-07-06 Thread GitBox
github-actions[bot] commented on issue #14938: URL: https://github.com/apache/airflow/issues/14938#issuecomment-875163118 This issue has been automatically marked as stale because it has been open for 30 days with no response from the author. It will be closed in next 7 days if no further

[airflow] branch v2-1-stable updated (4c1abfb -> 5b3b7d4)

2021-07-06 Thread kaxilnaik
This is an automated email from the ASF dual-hosted git repository. kaxilnaik pushed a change to branch v2-1-stable in repository https://gitbox.apache.org/repos/asf/airflow.git. from 4c1abfb Fix "Invalid JSON configuration, must be a dict" (#16648) add 7a5bb88 Only allow webserver

[GitHub] [airflow] kaxil merged pull request #16843: Sync v2 1 2 stable

2021-07-06 Thread GitBox
kaxil merged pull request #16843: URL: https://github.com/apache/airflow/pull/16843 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[GitHub] [airflow] boring-cyborg[bot] commented on issue #16844: Rest API: allow filtering DagRuns by state.

2021-07-06 Thread GitBox
boring-cyborg[bot] commented on issue #16844: URL: https://github.com/apache/airflow/issues/16844#issuecomment-875162535 Thanks for opening your first issue here! Be sure to follow the issue template! -- This is an automated message from the Apache Git Service. To respond to the

[GitHub] [airflow] gyscos opened a new issue #16844: Rest API: allow filtering DagRuns by state.

2021-07-06 Thread GitBox
gyscos opened a new issue #16844: URL: https://github.com/apache/airflow/issues/16844 **Add state filter to the dag runs list API endpoint** One feature available in the "Browse / Dag Runs" page but not in the current rest API is the ability to filter runs of a specific state(s).

[GitHub] [airflow] edwardwang888 commented on issue #16824: Initiating `mark success/failed` operation causes UI border to disappear for task instances with no status

2021-07-06 Thread GitBox
edwardwang888 commented on issue #16824: URL: https://github.com/apache/airflow/issues/16824#issuecomment-875161577 I actually prefer having non-existent tasks still render with the outline, the reason being that after adding a task, there is sometimes a need to mark the non-existent

[GitHub] [airflow] o-nikolas commented on a change in pull request #16685: reattach_prev_task ECSOperator

2021-07-06 Thread GitBox
o-nikolas commented on a change in pull request #16685: URL: https://github.com/apache/airflow/pull/16685#discussion_r664939787 ## File path: airflow/providers/amazon/aws/operators/ecs.py ## @@ -264,6 +276,21 @@ def _start_task(self): self.log.info('ECS Task started:

[airflow] branch main updated: Add State types for tasks and DAGs (#15285)

2021-07-06 Thread kaxilnaik
This is an automated email from the ASF dual-hosted git repository. kaxilnaik pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/main by this push: new 2b7c596 Add State types for tasks and DAGs

[GitHub] [airflow] kaxil merged pull request #15285: Fixed #9387: Add State types for tasks and DAGs

2021-07-06 Thread GitBox
kaxil merged pull request #15285: URL: https://github.com/apache/airflow/pull/15285 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[GitHub] [airflow] kaxil closed issue #9387: Use enum for task_states/dag_states

2021-07-06 Thread GitBox
kaxil closed issue #9387: URL: https://github.com/apache/airflow/issues/9387 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[GitHub] [airflow] j143 edited a comment on pull request #16498: gitpodify Apache Airflow - online development workspace

2021-07-06 Thread GitBox
j143 edited a comment on pull request #16498: URL: https://github.com/apache/airflow/pull/16498#issuecomment-875150114 cc @potiuk ,  Can you help me answer the following questions? - is this the intended feature. - there are some errors - are they expected? - Which

[GitHub] [airflow] j143 commented on pull request #16498: gitpodify Apache Airflow - online development workspace

2021-07-06 Thread GitBox
j143 commented on pull request #16498: URL: https://github.com/apache/airflow/pull/16498#issuecomment-875150114 cc @potiuk ,  - is this the intended feature. - there are some errors - are they expected? - Which ports are to be open? Thanks, Janardhan -- This

[GitHub] [airflow] dstandish commented on a change in pull request #16821: Add more filter options to list_keys of S3Hook

2021-07-06 Thread GitBox
dstandish commented on a change in pull request #16821: URL: https://github.com/apache/airflow/pull/16821#discussion_r664912477 ## File path: airflow/providers/amazon/aws/hooks/s3.py ## @@ -292,18 +302,24 @@ def list_keys( 'PageSize': page_size,

[airflow] branch main updated (4421023 -> f0df184)

2021-07-06 Thread kamilbregula
This is an automated email from the ASF dual-hosted git repository. kamilbregula pushed a change to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git. from 4421023 Clarify the role of commits history (#16840) add f0df184 Update AWS Base hook to use refreshable

[GitHub] [airflow] mik-laj merged pull request #16771: Update AWS Base hook to use refreshable credentials (#16770)

2021-07-06 Thread GitBox
mik-laj merged pull request #16771: URL: https://github.com/apache/airflow/pull/16771 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[GitHub] [airflow] mik-laj closed issue #16770: AWS hook should automatically refresh credentials when using temporary credentials

2021-07-06 Thread GitBox
mik-laj closed issue #16770: URL: https://github.com/apache/airflow/issues/16770 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[GitHub] [airflow] github-actions[bot] commented on pull request #16771: Update AWS Base hook to use refreshable credentials (#16770)

2021-07-06 Thread GitBox
github-actions[bot] commented on pull request #16771: URL: https://github.com/apache/airflow/pull/16771#issuecomment-875114496 The PR is likely OK to be merged with just subset of tests for default Python and Database versions without running the full matrix of tests, because it does not

[GitHub] [airflow] dacort commented on pull request #16766: Add an Amazon EMR on EKS provider package

2021-07-06 Thread GitBox
dacort commented on pull request #16766: URL: https://github.com/apache/airflow/pull/16766#issuecomment-875107879 All checks passing - will do a squash after I address any review comments. -- This is an automated message from the Apache Git Service. To respond to the message, please log

[GitHub] [airflow] jedcunningham commented on issue #16783: Airflow 2.1.0 Oauth for google Too Many Redirects b/c Google User does not have Role

2021-07-06 Thread GitBox
jedcunningham commented on issue #16783: URL: https://github.com/apache/airflow/issues/16783#issuecomment-875104038 > But it definitely erased the Role Yep, that's the behavior with `AUTH_ROLES_SYNC_AT_LOGIN` and no matching roles in `AUTH_ROLE_MAPPING`. You could also disable

[GitHub] [airflow] ephraimbuddy commented on pull request #16823: Add ability to specify XCom key for operators and task decorator

2021-07-06 Thread GitBox
ephraimbuddy commented on pull request #16823: URL: https://github.com/apache/airflow/pull/16823#issuecomment-875100326 Also, I feel the way multiple_outputs work is wrong and this can be used instead of it -- This is an automated message from the Apache Git Service. To respond to the

[GitHub] [airflow] ephraimbuddy edited a comment on pull request #16823: Add ability to specify XCom key for operators and task decorator

2021-07-06 Thread GitBox
ephraimbuddy edited a comment on pull request #16823: URL: https://github.com/apache/airflow/pull/16823#issuecomment-875098890 > > I want to be able to control the key used in pushing xcom when a value is returned from a task > > Step back further  Why? Please explain what you are

[GitHub] [airflow] ephraimbuddy edited a comment on pull request #16823: Add ability to specify XCom key for operators and task decorator

2021-07-06 Thread GitBox
ephraimbuddy edited a comment on pull request #16823: URL: https://github.com/apache/airflow/pull/16823#issuecomment-875098890 > > I want to be able to control the key used in pushing xcom when a value is returned from a task > > Step back further  Why? Please explain what you are

[GitHub] [airflow] ephraimbuddy commented on pull request #16823: Add ability to specify XCom key for operators and task decorator

2021-07-06 Thread GitBox
ephraimbuddy commented on pull request #16823: URL: https://github.com/apache/airflow/pull/16823#issuecomment-875098890 > > I want to be able to control the key used in pushing xcom when a value is returned from a task > > Step back further  Why? Please explain what you are trying

[GitHub] [airflow] warrenstephens commented on issue #15607: airflow throws error: airflow.exceptions.SerializedDagNotFound: DAG 'dns-basic-1min' not found in serialized_dag table

2021-07-06 Thread GitBox
warrenstephens commented on issue #15607: URL: https://github.com/apache/airflow/issues/15607#issuecomment-875097100 Fixed the bizarre import error, and the DAG error still occurs -- which starts only after I login to the localhost:8080 interface. ```

[GitHub] [airflow] ashb commented on pull request #16823: Add ability to specify XCom key for operators and task decorator

2021-07-06 Thread GitBox
ashb commented on pull request #16823: URL: https://github.com/apache/airflow/pull/16823#issuecomment-875089706 > I want to be able to control the key used in pushing xcom when a value is returned from a task Step back further  Why? Please explain what you are trying to achieve,

[airflow] 02/03: Set version to 2.1.2

2021-07-06 Thread ash
This is an automated email from the ASF dual-hosted git repository. ash pushed a commit to branch v2-1-test in repository https://gitbox.apache.org/repos/asf/airflow.git commit 051a78a9f1145baae570859162fa9cba9cde6fbd Author: James Timmins AuthorDate: Tue Jul 6 11:46:16 2021 -0700 Set

[airflow] 03/03: Add changelog updates for 2.1.2 (#16838)

2021-07-06 Thread ash
This is an automated email from the ASF dual-hosted git repository. ash pushed a commit to branch v2-1-test in repository https://gitbox.apache.org/repos/asf/airflow.git commit dd92d9a300778d91f0722da38db0c3feb12edcb0 Author: James Timmins AuthorDate: Tue Jul 6 11:50:48 2021 -0700 Add

[airflow] 01/03: Only allow webserver to request from the worker log server (#16754)

2021-07-06 Thread ash
This is an automated email from the ASF dual-hosted git repository. ash pushed a commit to branch v2-1-test in repository https://gitbox.apache.org/repos/asf/airflow.git commit f20d5c74933555c956350aa659308fb3e9c4b805 Author: Ash Berlin-Taylor AuthorDate: Thu Jul 1 16:20:36 2021 +0100 Only

[airflow] branch v2-1-test updated (e9500cf -> dd92d9a)

2021-07-06 Thread ash
This is an automated email from the ASF dual-hosted git repository. ash pushed a change to branch v2-1-test in repository https://gitbox.apache.org/repos/asf/airflow.git. discard e9500cf Add changelog updates for 2.1.2 (#16838) discard 51b6c33 Set version to 2.1.2 discard cf3a22b Only

[GitHub] [airflow] ashb commented on pull request #16843: Sync v2 1 2 stable

2021-07-06 Thread GitBox
ashb commented on pull request #16843: URL: https://github.com/apache/airflow/pull/16843#issuecomment-875075849 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To

[GitHub] [airflow] github-actions[bot] commented on pull request #16843: Sync v2 1 2 stable

2021-07-06 Thread GitBox
github-actions[bot] commented on pull request #16843: URL: https://github.com/apache/airflow/pull/16843#issuecomment-875075245 The PR most likely needs to run full matrix of tests because it modifies parts of the core of Airflow. However, committers might decide to merge it quickly and

[GitHub] [airflow] bbovenzi commented on issue #16824: Initiating `mark success/failed` operation causes UI border to disappear for task instances with no status

2021-07-06 Thread GitBox
bbovenzi commented on issue #16824: URL: https://github.com/apache/airflow/issues/16824#issuecomment-875070447 That makes sense to me. If anything, it should render the other way around: non-existent tasks are just blank with no outline. We can still handle this in `tree.js` of the UI.

[airflow] branch main updated (6c06c5d -> 4421023)

2021-07-06 Thread potiuk
This is an automated email from the ASF dual-hosted git repository. potiuk pushed a change to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git. from 6c06c5d Change whimsy url since Airflow is a TLP (#16841) add 4421023 Clarify the role of commits history

[GitHub] [airflow] potiuk merged pull request #16840: Clarify the role of commits history

2021-07-06 Thread GitBox
potiuk merged pull request #16840: URL: https://github.com/apache/airflow/pull/16840 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[airflow] branch main updated (f40ade4 -> 6c06c5d)

2021-07-06 Thread potiuk
This is an automated email from the ASF dual-hosted git repository. potiuk pushed a change to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git. from f40ade4 When a task instance fails with exception, log it (#16805) add 6c06c5d Change whimsy url since Airflow

[GitHub] [airflow] potiuk merged pull request #16841: Change whimsy url since Airflow is a TLP

2021-07-06 Thread GitBox
potiuk merged pull request #16841: URL: https://github.com/apache/airflow/pull/16841 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[GitHub] [airflow] ephraimbuddy edited a comment on pull request #16823: Add ability to specify XCom key for operators and task decorator

2021-07-06 Thread GitBox
ephraimbuddy edited a comment on pull request #16823: URL: https://github.com/apache/airflow/pull/16823#issuecomment-875058937 > FWIW that feels like a bug itself -- it shouldn't show up "twice" in xcom, which is especially important for custom xcom backends and larger data (i.e. DFs)

[GitHub] [airflow] ephraimbuddy commented on pull request #16823: Add ability to specify XCom key for operators and task decorator

2021-07-06 Thread GitBox
ephraimbuddy commented on pull request #16823: URL: https://github.com/apache/airflow/pull/16823#issuecomment-875058937 > FWIW that feels like a bug itself -- it shouldn't show up "twice" in xcom, which is especially important for custom xcom backends and larger data (i.e. DFs)

[GitHub] [airflow] jhtimmins opened a new pull request #16843: Sync v2 1 2 stable

2021-07-06 Thread GitBox
jhtimmins opened a new pull request #16843: URL: https://github.com/apache/airflow/pull/16843 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[GitHub] [airflow] potiuk commented on pull request #15385: Added table to view providers in airflow ui under admin tab

2021-07-06 Thread GitBox
potiuk commented on pull request #15385: URL: https://github.com/apache/airflow/pull/15385#issuecomment-875051551 Can you please rebase this one to latest main @ShakaibKhan ? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub

[GitHub] [airflow] ashb commented on pull request #16823: Add ability to specify XCom key for operators and task decorator

2021-07-06 Thread GitBox
ashb commented on pull request #16823: URL: https://github.com/apache/airflow/pull/16823#issuecomment-875051253 > Then Xcoms pushed will be > > ``` > key=mykey, value=[3,4,5,6] > key=return_value, value={'mykey':[3,4,56]} > ``` FWIW that feels like a bug itself -- it

[GitHub] [airflow] psg2 opened a new pull request #16842: Fix/consistent get dag rest endpoint

2021-07-06 Thread GitBox
psg2 opened a new pull request #16842: URL: https://github.com/apache/airflow/pull/16842 closes: https://github.com/apache/airflow/issues/16839 --- **^ Add meaningful description above** Read the **[Pull Request

[GitHub] [airflow] boring-cyborg[bot] commented on pull request #16842: Fix/consistent get dag rest endpoint

2021-07-06 Thread GitBox
boring-cyborg[bot] commented on pull request #16842: URL: https://github.com/apache/airflow/pull/16842#issuecomment-875044004 Congratulations on your first Pull Request and welcome to the Apache Airflow community! If you have any issues or are unsure about any anything please check our

[GitHub] [airflow] ephraimbuddy commented on pull request #16823: Add ability to specify XCom key for operators and task decorator

2021-07-06 Thread GitBox
ephraimbuddy commented on pull request #16823: URL: https://github.com/apache/airflow/pull/16823#issuecomment-875041504 One thing that one can do if he's able to control the xcom_key: In overriding the `orm_deserialize_value` we can do something like: ``` def

[GitHub] [airflow] ephraimbuddy commented on a change in pull request #16823: Add ability to specify XCom key for operators and task decorator

2021-07-06 Thread GitBox
ephraimbuddy commented on a change in pull request #16823: URL: https://github.com/apache/airflow/pull/16823#discussion_r664837950 ## File path: airflow/serialization/schema.json ## @@ -161,6 +161,7 @@ "weight_rule": { "type": "string" }, "executor_config": {

[GitHub] [airflow] ephraimbuddy commented on a change in pull request #16823: Add ability to specify XCom key for operators and task decorator

2021-07-06 Thread GitBox
ephraimbuddy commented on a change in pull request #16823: URL: https://github.com/apache/airflow/pull/16823#discussion_r664836811 ## File path: tests/models/test_taskinstance.py ## @@ -1163,6 +1163,33 @@ def test_xcom_push_flag(self): ti.run() assert

  1   2   3   4   >