[jira] [Commented] (AIRFLOW-3223) RBAC with GitHub Authentication
[ https://issues.apache.org/jira/browse/AIRFLOW-3223?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16720211#comment-16720211 ] Sai Phanindhra commented on AIRFLOW-3223: - [~ashb] I think airflow already supports github authentication > RBAC with GitHub Authentication > > > Key: AIRFLOW-3223 > URL: https://issues.apache.org/jira/browse/AIRFLOW-3223 > Project: Apache Airflow > Issue Type: Wish > Components: authentication >Affects Versions: 1.10.0 >Reporter: Vikram Fugro >Assignee: Sai Phanindhra >Priority: Major > > With airflow 1.10 released having RBAC support, I was wondering how I do > configure GitHub Auth with airflow's RBAC. In which case, I believe we don't > have to create any users using airflow. Are there any notes on this? -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Assigned] (AIRFLOW-3223) RBAC with GitHub Authentication
[ https://issues.apache.org/jira/browse/AIRFLOW-3223?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sai Phanindhra reassigned AIRFLOW-3223: --- Assignee: Sai Phanindhra > RBAC with GitHub Authentication > > > Key: AIRFLOW-3223 > URL: https://issues.apache.org/jira/browse/AIRFLOW-3223 > Project: Apache Airflow > Issue Type: Wish > Components: authentication >Affects Versions: 1.10.0 >Reporter: Vikram Fugro >Assignee: Sai Phanindhra >Priority: Major > > With airflow 1.10 released having RBAC support, I was wondering how I do > configure GitHub Auth with airflow's RBAC. In which case, I believe we don't > have to create any users using airflow. Are there any notes on this? -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Assigned] (AIRFLOW-3053) Airflow scheduler - high availability ?
[ https://issues.apache.org/jira/browse/AIRFLOW-3053?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sai Phanindhra reassigned AIRFLOW-3053: --- Assignee: Sai Phanindhra > Airflow scheduler - high availability ? > --- > > Key: AIRFLOW-3053 > URL: https://issues.apache.org/jira/browse/AIRFLOW-3053 > Project: Apache Airflow > Issue Type: Wish > Components: scheduler >Reporter: Damian Momot >Assignee: Sai Phanindhra >Priority: Major > > Probably more of a question than issue. > Does airflow scheduler support high availability? For example running 2 > instances of airflow-scheduler connecting to single MySQL database? > I tried to search for this question but only thing i found are conversations > from 2-3 years ago that "at that time it wasn't supported but it was planned" -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (AIRFLOW-3352) Don't Show Airflow config in rbac view based on flag
[ https://issues.apache.org/jira/browse/AIRFLOW-3352?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sai Phanindhra updated AIRFLOW-3352: Fix Version/s: 1.10.2 > Don't Show Airflow config in rbac view based on flag > > > Key: AIRFLOW-3352 > URL: https://issues.apache.org/jira/browse/AIRFLOW-3352 > Project: Apache Airflow > Issue Type: Bug > Components: webserver >Affects Versions: 1.10.0 >Reporter: Sai Phanindhra >Assignee: Sai Phanindhra >Priority: Major > Labels: configuration, rbac, webserver > Fix For: 1.10.2 > > > Earlier expose_config flags is used to toggle whether to show configuration > in UI or not. This feature is not enabled when rbac is enabled. Add provision > to toggle this feature when rbac is enabled on airflow. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (AIRFLOW-3352) Don't Show Airflow config in rbac view based on flag
[ https://issues.apache.org/jira/browse/AIRFLOW-3352?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sai Phanindhra updated AIRFLOW-3352: Labels: configuration rbac webserver (was: rbac) > Don't Show Airflow config in rbac view based on flag > > > Key: AIRFLOW-3352 > URL: https://issues.apache.org/jira/browse/AIRFLOW-3352 > Project: Apache Airflow > Issue Type: Bug > Components: webserver >Affects Versions: 1.10.0 >Reporter: Sai Phanindhra >Assignee: Sai Phanindhra >Priority: Major > Labels: configuration, rbac, webserver > Fix For: 1.10.2 > > > Earlier expose_config flags is used to toggle whether to show configuration > in UI or not. This feature is not enabled when rbac is enabled. Add provision > to toggle this feature when rbac is enabled on airflow. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (AIRFLOW-3352) Don't Show Airflow config in rbac view based on flag
[ https://issues.apache.org/jira/browse/AIRFLOW-3352?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sai Phanindhra updated AIRFLOW-3352: Component/s: webserver > Don't Show Airflow config in rbac view based on flag > > > Key: AIRFLOW-3352 > URL: https://issues.apache.org/jira/browse/AIRFLOW-3352 > Project: Apache Airflow > Issue Type: Bug > Components: webserver >Affects Versions: 1.10.0 >Reporter: Sai Phanindhra >Assignee: Sai Phanindhra >Priority: Major > Labels: rbac > > Earlier expose_config flags is used to toggle whether to show configuration > in UI or not. This feature is not enabled when rbac is enabled. Add provision > to toggle this feature when rbac is enabled on airflow. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (AIRFLOW-3352) Don't Show Airflow config in rbac view based on flag
[ https://issues.apache.org/jira/browse/AIRFLOW-3352?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sai Phanindhra updated AIRFLOW-3352: Affects Version/s: 1.10.0 > Don't Show Airflow config in rbac view based on flag > > > Key: AIRFLOW-3352 > URL: https://issues.apache.org/jira/browse/AIRFLOW-3352 > Project: Apache Airflow > Issue Type: Bug > Components: webserver >Affects Versions: 1.10.0 >Reporter: Sai Phanindhra >Assignee: Sai Phanindhra >Priority: Major > Labels: rbac > > Earlier expose_config flags is used to toggle whether to show configuration > in UI or not. This feature is not enabled when rbac is enabled. Add provision > to toggle this feature when rbac is enabled on airflow. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Work started] (AIRFLOW-3352) Don't Show Airflow config in rbac view based on flag
[ https://issues.apache.org/jira/browse/AIRFLOW-3352?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on AIRFLOW-3352 started by Sai Phanindhra. --- > Don't Show Airflow config in rbac view based on flag > > > Key: AIRFLOW-3352 > URL: https://issues.apache.org/jira/browse/AIRFLOW-3352 > Project: Apache Airflow > Issue Type: Bug >Reporter: Sai Phanindhra >Assignee: Sai Phanindhra >Priority: Major > Labels: rbac > > Earlier expose_config flags is used to toggle whether to show configuration > in UI or not. This feature is not enabled when rbac is enabled. Add provision > to toggle this feature when rbac is enabled on airflow. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (AIRFLOW-3352) Don't Show Airflow config in rbac view based on flag
Sai Phanindhra created AIRFLOW-3352: --- Summary: Don't Show Airflow config in rbac view based on flag Key: AIRFLOW-3352 URL: https://issues.apache.org/jira/browse/AIRFLOW-3352 Project: Apache Airflow Issue Type: Bug Reporter: Sai Phanindhra Assignee: Sai Phanindhra Earlier expose_config flags is used to toggle whether to show configuration in UI or not. This feature is not enabled when rbac is enabled. Add provision to toggle this feature when rbac is enabled on airflow. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (AIRFLOW-3266) AWS Athena Operator in Airflow
[ https://issues.apache.org/jira/browse/AIRFLOW-3266?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sai Phanindhra updated AIRFLOW-3266: Description: There is no official athena operator as of now airflow. Either one has do it using boto3 in python operator or using aws cli in bash operator. Either of these does not take care of total life cycle of query. Create a Athena operator and hook to submit presto query and update task based of state of submitted query. (was: These is no official athena operator as of now airflow. Either one has do it using boto3 in python operator or using aws cli in bash operator. Either of these does not take care of total life cycle of query. Create a Athena operator and hook to submit presto query and update task based of state of submitted query.) > AWS Athena Operator in Airflow > -- > > Key: AIRFLOW-3266 > URL: https://issues.apache.org/jira/browse/AIRFLOW-3266 > Project: Apache Airflow > Issue Type: New Feature > Components: aws >Affects Versions: 1.10.0 >Reporter: Sai Phanindhra >Assignee: Sai Phanindhra >Priority: Minor > > There is no official athena operator as of now airflow. Either one has do it > using boto3 in python operator or using aws cli in bash operator. Either of > these does not take care of total life cycle of query. Create a Athena > operator and hook to submit presto query and update task based of state of > submitted query. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (AIRFLOW-3266) AWS Athena Operator in Airflow
[ https://issues.apache.org/jira/browse/AIRFLOW-3266?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sai Phanindhra updated AIRFLOW-3266: Description: These is no official athena operator as of now airflow. Either one has do it using boto3 in python operator or using aws cli in bash operator. Either of these does not take care of total life cycle of query. Create a Athena operator and hook to submit presto query and update task based of state of submitted query. (was: Create an airflow operator using boto3 for running aws athena queries) > AWS Athena Operator in Airflow > -- > > Key: AIRFLOW-3266 > URL: https://issues.apache.org/jira/browse/AIRFLOW-3266 > Project: Apache Airflow > Issue Type: New Feature > Components: aws >Affects Versions: 1.10.0 >Reporter: Sai Phanindhra >Assignee: Sai Phanindhra >Priority: Minor > > These is no official athena operator as of now airflow. Either one has do it > using boto3 in python operator or using aws cli in bash operator. Either of > these does not take care of total life cycle of query. Create a Athena > operator and hook to submit presto query and update task based of state of > submitted query. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (AIRFLOW-3266) AWS Athena Operator in Airflow
Sai Phanindhra created AIRFLOW-3266: --- Summary: AWS Athena Operator in Airflow Key: AIRFLOW-3266 URL: https://issues.apache.org/jira/browse/AIRFLOW-3266 Project: Apache Airflow Issue Type: New Feature Components: aws Affects Versions: 1.10.0 Reporter: Sai Phanindhra Assignee: Sai Phanindhra Create an airflow operator using boto3 for running aws athena queries -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Comment Edited] (AIRFLOW-2064) Polish timezone implementation
[ https://issues.apache.org/jira/browse/AIRFLOW-2064?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16636261#comment-16636261 ] Sai Phanindhra edited comment on AIRFLOW-2064 at 10/3/18 12:13 AM: --- Hi, In point 2, if no timezone is specified we should ideally use UTC irrespective of timezone used by airflow scheduler. What should be changed here in that case? was (Author: phani8996): Hi, In point 2, if no timezone is created we should ideally use UTC irrespective of timezone used by airflow scheduler. What should be changed here in that case? > Polish timezone implementation > -- > > Key: AIRFLOW-2064 > URL: https://issues.apache.org/jira/browse/AIRFLOW-2064 > Project: Apache Airflow > Issue Type: Improvement >Reporter: Bolke de Bruin >Priority: Blocker > Fix For: 1.10.0 > > > Couple of things are left over after moving to time zone support: > > # End_dates within dags should be converted to UTC by using the time zone of > start_date if naive > # Task instances that are instantiated without timezone information for > their execution_date should convert those to UTC by using the DAG's timezone > or configured > # Some doc polishing > # Tests should be added that cover more of the edge cases -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (AIRFLOW-2064) Polish timezone implementation
[ https://issues.apache.org/jira/browse/AIRFLOW-2064?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16636261#comment-16636261 ] Sai Phanindhra commented on AIRFLOW-2064: - Hi, In point 2, if no timezone is created we should ideally use UTC irrespective of timezone used by airflow scheduler. What should be changed here in that case? > Polish timezone implementation > -- > > Key: AIRFLOW-2064 > URL: https://issues.apache.org/jira/browse/AIRFLOW-2064 > Project: Apache Airflow > Issue Type: Improvement >Reporter: Bolke de Bruin >Priority: Blocker > Fix For: 1.10.0 > > > Couple of things are left over after moving to time zone support: > > # End_dates within dags should be converted to UTC by using the time zone of > start_date if naive > # Task instances that are instantiated without timezone information for > their execution_date should convert those to UTC by using the DAG's timezone > or configured > # Some doc polishing > # Tests should be added that cover more of the edge cases -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Assigned] (AIRFLOW-620) Ability to refresh logs of a TI without reloading whole page
[ https://issues.apache.org/jira/browse/AIRFLOW-620?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sai Phanindhra reassigned AIRFLOW-620: -- Assignee: Sai Phanindhra (was: Sumit Maheshwari) > Ability to refresh logs of a TI without reloading whole page > > > Key: AIRFLOW-620 > URL: https://issues.apache.org/jira/browse/AIRFLOW-620 > Project: Apache Airflow > Issue Type: Improvement >Reporter: Sumit Maheshwari >Assignee: Sai Phanindhra >Priority: Major > Labels: UI, usability > > As of now to see the latest logs of a task instance one has to refresh the > whole page, which is time consuming. It'll be great if just logs can be > re-fetched from backend. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Assigned] (AIRFLOW-1945) Pass --autoscale to celery workers
[
https://issues.apache.org/jira/browse/AIRFLOW-1945?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sai Phanindhra reassigned AIRFLOW-1945:
---
Assignee: Sai Phanindhra
> Pass --autoscale to celery workers
> --
>
> Key: AIRFLOW-1945
> URL: https://issues.apache.org/jira/browse/AIRFLOW-1945
> Project: Apache Airflow
> Issue Type: Improvement
> Components: celery, cli
>Reporter: Michael O.
>Assignee: Sai Phanindhra
>Priority: Trivial
> Labels: easyfix
> Original Estimate: 0.5h
> Remaining Estimate: 0.5h
>
> Celery supports autoscaling of the worker pool size (number of tasks that can
> parallelize within one worker node). I'd like to propose to support passing
> the --autoscale parameter to {{airflow worker}}.
> Since this is a trivial change, I am not sure if there's any reason for not
> being supported already.(?)
> For example
> {{airflow worker --concurrency=4}} will set a fixed pool size of 4.
> With minimal changes in
> [https://github.com/apache/incubator-airflow/blob/4ce4faaeae7a76d97defcf9a9d3304ac9d78b9bd/airflow/bin/cli.py#L855]
> it could support
> {{airflow worker --autoscale=2,10}} to set an autoscaled pool size of 2 to 10
> Some references:
> *
> http://docs.celeryproject.org/en/latest/internals/reference/celery.worker.autoscale.html
> *
> https://github.com/apache/incubator-airflow/blob/4ce4faaeae7a76d97defcf9a9d3304ac9d78b9bd/airflow/bin/cli.py#L855
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
