Re: Deprecate legacy UI in favor of FAB RBAC

2018-12-18 Thread airflowuser
I don't think this can be merged until stopping adding fixes to 1.10.2 as there are some fixes for the old UI which this PR removes. Sent with ProtonMail Secure Email. ‐‐‐ Original Message ‐‐‐ On Tuesday, December 18, 2018 12:55 PM, Verdan Mahmood wrote: > Hi all, > > At the moment,

Re: Call for fixes for Airflow 1.10.2

2018-12-13 Thread airflowuser
Some more PRs to consider for 1.10.2 : https://issues.apache.org/jira/browse/AIRFLOW-3378 KubernetesPodOperator does not delete on timeout failure https://issues.apache.org/jira/browse/AIRFLOW-3375 Support returning multiple tasks with BranchPythonOperator

Re: Recommended backend metastore for Airflow

2018-12-10 Thread airflowuser
Definitely PostgreSQL. https://www.2ndquadrant.com/en/postgresql/postgresql-vs-mysql/ ‐‐‐ Original Message ‐‐‐ On Monday, December 10, 2018 5:10 PM, ramandu...@gmail.com wrote: > Hi All, > > It seems that Airflow supports mysql, postgresql and mssql as backend store. > Any

Re: Call for fixes for Airflow 1.10.2

2018-12-05 Thread airflowuser
Possible tickets to consider (All Prs merged and point to 2.0.0): https://issues.apache.org/jira/browse/AIRFLOW-3378 https://issues.apache.org/jira/browse/AIRFLOW-3375 https://issues.apache.org/jira/browse/AIRFLOW-3371 https://issues.apache.org/jira/browse/AIRFLOW-3332

Re: Will Airflow 2.0.0 support Python 2.7?

2018-11-29 Thread airflowuser
on 2.7 is still maintained for another year, yes, probably. > > > On 29 Nov 2018, at 08:48, airflowuser airflowu...@protonmail.com.INVALID > > wrote: > > Are there plans to drop support for Python 2.7 - if so when ?

Will Airflow 2.0.0 support Python 2.7?

2018-11-29 Thread airflowuser
Are there plans to drop support for Python 2.7 - if so when ?

Request for review PR

2018-11-22 Thread airflowuser
Can one of the committers please review this one ? https://github.com/apache/incubator-airflow/pull/2709

Re: Customised alerts/notifications and enhancements to alerting/notifications on Airflow

2018-11-14 Thread airflowuser
There is a pending PR to refactor the SLA: https://github.com/apache/incubator-airflow/pull/3584 But it requires more reviews from committers. Sent with ProtonMail Secure Email. ‐‐‐ Original Message ‐‐‐ On Wednesday, November 14, 2018 5:11 PM, Sai Phanindhra wrote: > Hello airflow

Creation of new hooks

2018-10-29 Thread airflowuser
I'm wondering if the current approach to edit models.py for each new hook is the best way to go. This issue is discussed in several Jira tickets: https://issues.apache.org/jira/browse/AIRFLOW-1307 https://issues.apache.org/jira/browse/AIRFLOW-2316 The problem is that each new hooks requires to

Re: 1.10.1 Release?

2018-10-28 Thread airflowuser
I was really hoping that https://github.com/apache/incubator-airflow/pull/4069 will be merged into 1.10.1 Deleting dags was a highly requested feature for 1.10 - this can fix the problem with it. ‐‐‐ Original Message ‐‐‐ On Friday, October 26, 2018 6:12 PM, Bolke de Bruin wrote: >

Re: How do you branch your code with BigQuery?

2018-10-15 Thread airflowuser
called airflow_1.9 that also contains the latest BigQuery hook and so works > on airflow 1.9 > > The query you run must return true for all columns - the same as for the > BigQuery check operator, so you may need to rewrite your queries to do this > > On Sun, 14 Oct 2

How do you branch your code with BigQuery?

2018-10-14 Thread airflowuser
I believe this is quite common case when working with data. If something : do A else: do B With coding PythonBranchOperator is the solution. But when working on Google Cloud there is no way to do this. All existed operators are designed to continue or fail on comparison of specific value:

Re: execution_date - can we stop the confusion?

2018-09-30 Thread airflowuser
ds > > > > > > > compatibility. Offering both views (left bound/right bound), as > > > > > > > it's > > > > > > > > been > > > > > > > > > > > > proposed before, either as an environment setting or a user >

Re: Travis CI tests failing in master

2018-09-27 Thread airflowuser
Also I see this a lot: 1) ERROR: Failure: ProgrammingError ((_mysql_exceptions.ProgrammingError) (1146, "Table 'airflow.task_instance' doesn't exist") [SQL: u'DELETE FROM task_instance WHERE task_instance.dag_id = %s'] [parameters: ('unit_tests',)]) Sent with ProtonMail Secure Email. ‐‐‐

execution_date - can we stop the confusion?

2018-09-26 Thread airflowuser
One of the most annoying, hard to understand and against all common sense is the execution_date behavior. I assume that any new Airflow user has been struggling with it. The amount of questions with answers referring to : https://airflow.apache.org/scheduler.html?scheduling-triggers is

Re: It's very hard to become a committer on the project

2018-09-20 Thread airflowuser
or vague suggestions still get > > > > > > discussed on the dev mailing list before a Jira is even opened, > > > > > > but after > > > > > > that, the discussion moves to the Jira, with a link back to the > > > > > > maili

Re: It's very hard to become a committer on the project

2018-09-20 Thread airflowuser
______ >>> > From: James Meickle >>> > Sent: Sunday, September 16, 2018 7:46:58 AM >>> > To: d...@airflow.apache.org >>> > Subject: Re: It's very hard to become a committer on the project >>> > >>> > Definitely

Fundamental change - Separate DAG name and id.

2018-09-20 Thread airflowuser
Hi, though this could have been explained on Jira I think this should be discussed first. The problem: Airflow mixes DAG name with id. It uses same filed for both purposes. I assume that most of you use the dag_id to describe what the DAG actually does. For example: dag = DAG(

It's very hard to become a committer on the project

2018-09-16 Thread airflowuser
Hello all, I'm struggling finding tickets to address and while discussing it on chat others reported they had the same problem when they began working on the project. The problem is due to: 1. It's very hard to locate tickets on Jira. The categories are a mess, versions are not enforced. each

Re: Security issue being ignored?

2018-09-06 Thread airflowuser
Another example: https://issues.apache.org/jira/projects/AIRFLOW/issues/AIRFLOW-2283 Sent with [ProtonMail](https://protonmail.com) Secure Email. ‐‐‐ Original Message ‐‐‐ On September 3, 2018 10:20 AM, airflowuser wrote: > Hi, > I noticed you opened a disccusion about the nec

Re: [VOTE] Replace with Gitter with Slack?

2018-09-06 Thread airflowuser
-1 Sent with ProtonMail Secure Email. ‐‐‐ Original Message ‐‐‐ On September 6, 2018 5:30 AM, Sid Anand wrote: > Hi Folks! > In the Apache tradition, I'd like to ask the community to vote on replacing > Gitter with Slack. > > For more information about what was recently discussed,

Re: Call for fixes for Airflow 1.10.1

2018-09-05 Thread airflowuser
f Jira issue they create. Or at least not uniformly. > > -ash > > > On 5 Sep 2018, at 14:08, airflowuser airflowu...@protonmail.com.INVALID > > wrote: > > May I suggest for future releases to show the change log as: > > Bug Fixes: > > New Features: > &

Re: Best Practice of Airflow Setting-Up & Usage

2018-09-05 Thread airflowuser
Hi, Setting up Airflow for the first time is a BIG DEAL. unlike the initial intention of the community of easy install with SQLite and SequentialExecutor - for actually working environment you need to change a lot of settings. It doesn't help much that the demo install went smoothly. The

Re: Call for fixes for Airflow 1.10.1

2018-09-05 Thread airflowuser
May I suggest for future releases to show the change log as: Bug Fixes: New Features: etc... This makes it easier to look over the list. This shouldn't be manual work.. it can be taken from the Jira ticket. Sent with ProtonMail Secure Email. ‐‐‐ Original Message ‐‐‐ On September

Security issue being ignored?

2018-09-03 Thread airflowuser
Hi, I noticed you opened a disccusion about the neccesity of Gitter... I think the main problem is that unlike other open source projects with Airflow no one is monitoring the Jira. So people tend to report many stuff on the Gitter to get assistance. Sometimes answers are given but no one answer