Security issue being ignored?

2018-09-03 Thread airflowuser
Hi, I noticed you opened a disccusion about the neccesity of Gitter... I think the main problem is that unlike other open source projects with Airflow no one is monitoring the Jira. So people tend to report many stuff on the Gitter to get assistance. Sometimes answers are given but no one answer

re run build

2018-09-05 Thread airflowuser
Can someone assist with rerun build for: https://github.com/apache/incubator-airflow/pull/2488 tried to send it to committers mailing list but for some reason I can only get mails from list but not sending to list :\ Sent with [ProtonMail](https://protonmail.com) Secure Email.

Re: Call for fixes for Airflow 1.10.1

2018-09-05 Thread airflowuser
May I suggest for future releases to show the change log as: Bug Fixes: New Features: etc... This makes it easier to look over the list. This shouldn't be manual work.. it can be taken from the Jira ticket. Sent with ProtonMail Secure Email. ‐‐‐ Original Message ‐‐‐ On September 3,

Re: Best Practice of Airflow Setting-Up & Usage

2018-09-05 Thread airflowuser
Hi, Setting up Airflow for the first time is a BIG DEAL. unlike the initial intention of the community of easy install with SQLite and SequentialExecutor - for actually working environment you need to change a lot of settings. It doesn't help much that the demo install went smoothly. The suppor

Re: Call for fixes for Airflow 1.10.1

2018-09-05 Thread airflowuser
hat type of Jira issue they create. Or at least not uniformly. > > -ash > > > On 5 Sep 2018, at 14:08, airflowuser airflowu...@protonmail.com.INVALID > > wrote: > > May I suggest for future releases to show the change log as: > > Bug Fixes: > > New Features: &g

Re: [VOTE] Replace with Gitter with Slack?

2018-09-06 Thread airflowuser
-1 Sent with ProtonMail Secure Email. ‐‐‐ Original Message ‐‐‐ On September 6, 2018 5:30 AM, Sid Anand wrote: > Hi Folks! > In the Apache tradition, I'd like to ask the community to vote on replacing > Gitter with Slack. > > For more information about what was recently discussed, refer

Re: Security issue being ignored?

2018-09-06 Thread airflowuser
Another example: https://issues.apache.org/jira/projects/AIRFLOW/issues/AIRFLOW-2283 Sent with [ProtonMail](https://protonmail.com) Secure Email. ‐‐‐ Original Message ‐‐‐ On September 3, 2018 10:20 AM, airflowuser wrote: > Hi, > I noticed you opened a disccusion about the necces

It's very hard to become a committer on the project

2018-09-16 Thread airflowuser
Hello all, I'm struggling finding tickets to address and while discussing it on chat others reported they had the same problem when they began working on the project. The problem is due to: 1. It's very hard to locate tickets on Jira. The categories are a mess, versions are not enforced. each

Fundamental change - Separate DAG name and id.

2018-09-20 Thread airflowuser
Hi, though this could have been explained on Jira I think this should be discussed first. The problem: Airflow mixes DAG name with id. It uses same filed for both purposes. I assume that most of you use the dag_id to describe what the DAG actually does. For example: dag = DAG( dag_id='cost_

Re: It's very hard to become a committer on the project

2018-09-20 Thread airflowuser
concerned with HOW the change/fix is >>> > implemented. >>> > >>> > Get Outlook for Android<https://aka.ms/ghei36> >>> > >>> > >>> > From: James Meickle >>> > Sent: Sunday, Sept

Re: It's very hard to become a committer on the project

2018-09-20 Thread airflowuser
acknowledgement by the main contributors. > > > > > > > > > > > > 3. Larger enhancements and efforts or vague suggestions still get > > > > > > discussed on the dev mailing list before a Jira is even opened, > >

execution_date - can we stop the confusion?

2018-09-26 Thread airflowuser
One of the most annoying, hard to understand and against all common sense is the execution_date behavior. I assume that any new Airflow user has been struggling with it. The amount of questions with answers referring to : https://airflow.apache.org/scheduler.html?scheduling-triggers is uncounta

Re: Travis CI tests failing in master

2018-09-27 Thread airflowuser
Also I see this a lot: 1) ERROR: Failure: ProgrammingError ((_mysql_exceptions.ProgrammingError) (1146, "Table 'airflow.task_instance' doesn't exist") [SQL: u'DELETE FROM task_instance WHERE task_instance.dag_id = %s'] [parameters: ('unit_tests',)]) Sent with ProtonMail Secure Email. ‐‐‐ O

Re: execution_date - can we stop the confusion?

2018-09-30 Thread airflowuser
gt; > > > The main thing is that whatever we do we cannot break backwards > > > > > > > compatibility. Offering both views (left bound/right bound), as > > > > > > > it's > > > > > > > > been > > > > > > > >

How do you branch your code with BigQuery?

2018-10-14 Thread airflowuser
I believe this is quite common case when working with data. If something : do A else: do B With coding PythonBranchOperator is the solution. But when working on Google Cloud there is no way to do this. All existed operators are designed to continue or fail on comparison of specific value: BigQue

Re: How do you branch your code with BigQuery?

2018-10-15 Thread airflowuser
d airflow_1.9 that also contains the latest BigQuery hook and so works > on airflow 1.9 > > The query you run must return true for all columns - the same as for the > BigQuery check operator, so you may need to rewrite your queries to do this > > On Sun, 14 Oct 2

Re: 1.10.1 Release?

2018-10-28 Thread airflowuser
I was really hoping that https://github.com/apache/incubator-airflow/pull/4069 will be merged into 1.10.1 Deleting dags was a highly requested feature for 1.10 - this can fix the problem with it. ‐‐‐ Original Message ‐‐‐ On Friday, October 26, 2018 6:12 PM, Bolke de Bruin wrote: > Hey

Creation of new hooks

2018-10-29 Thread airflowuser
I'm wondering if the current approach to edit models.py for each new hook is the best way to go. This issue is discussed in several Jira tickets: https://issues.apache.org/jira/browse/AIRFLOW-1307 https://issues.apache.org/jira/browse/AIRFLOW-2316 The problem is that each new hooks requires to "d

Re: Customised alerts/notifications and enhancements to alerting/notifications on Airflow

2018-11-14 Thread airflowuser
There is a pending PR to refactor the SLA: https://github.com/apache/incubator-airflow/pull/3584 But it requires more reviews from committers. Sent with ProtonMail Secure Email. ‐‐‐ Original Message ‐‐‐ On Wednesday, November 14, 2018 5:11 PM, Sai Phanindhra wrote: > Hello airflow co

Request for review PR

2018-11-22 Thread airflowuser
Can one of the committers please review this one ? https://github.com/apache/incubator-airflow/pull/2709

Will Airflow 2.0.0 support Python 2.7?

2018-11-29 Thread airflowuser
Are there plans to drop support for Python 2.7 - if so when ?

Re: Will Airflow 2.0.0 support Python 2.7?

2018-11-29 Thread airflowuser
Python 2.7 is still maintained for another year, yes, probably. > > > On 29 Nov 2018, at 08:48, airflowuser airflowu...@protonmail.com.INVALID > > wrote: > > Are there plans to drop support for Python 2.7 - if so when ?

Re: Call for fixes for Airflow 1.10.2

2018-12-05 Thread airflowuser
Possible tickets to consider (All Prs merged and point to 2.0.0): https://issues.apache.org/jira/browse/AIRFLOW-3378 https://issues.apache.org/jira/browse/AIRFLOW-3375 https://issues.apache.org/jira/browse/AIRFLOW-3371 https://issues.apache.org/jira/browse/AIRFLOW-3332 https://issues.apache.org/ji

Re: Recommended backend metastore for Airflow

2018-12-10 Thread airflowuser
Definitely PostgreSQL. https://www.2ndquadrant.com/en/postgresql/postgresql-vs-mysql/ ‐‐‐ Original Message ‐‐‐ On Monday, December 10, 2018 5:10 PM, ramandu...@gmail.com wrote: > Hi All, > > It seems that Airflow supports mysql, postgresql and mssql as backend store. > Any recommend

Re: Call for fixes for Airflow 1.10.2

2018-12-13 Thread airflowuser
Some more PRs to consider for 1.10.2 : https://issues.apache.org/jira/browse/AIRFLOW-3378 KubernetesPodOperator does not delete on timeout failure https://issues.apache.org/jira/browse/AIRFLOW-3375 Support returning multiple tasks with BranchPythonOperator https://issues.apache.org/jira/browse

Re: Deprecate legacy UI in favor of FAB RBAC

2018-12-18 Thread airflowuser
I don't think this can be merged until stopping adding fixes to 1.10.2 as there are some fixes for the old UI which this PR removes. Sent with ProtonMail Secure Email. ‐‐‐ Original Message ‐‐‐ On Tuesday, December 18, 2018 12:55 PM, Verdan Mahmood wrote: > Hi all, > > At the moment,