Re: [ANNOUNCE] Please welcome new Airflow committer Felix Uellendall

Congrats! On Mon, Sep 9, 2019 at 4:13 PM Kevin Yang wrote: > Congrats man :D > > On Mon, Sep 9, 2019 at 4:00 PM Chao-Han Tsai wrote: > > > Congrats. > > > > On Mon, Sep 9, 2019 at 6:15 AM Philippe Gagnon > > wrote: > > > > > Congratulations! > > > > > > On Mon, Sep 9, 2019, 09:10 Kaxil Naik w

Re: Outstanding review requests

Hey Bolke, I think you sent the wrong alias. The review should go to the atlas alias :). On Wed, Sep 11, 2019 at 12:57 PM Bolke de Bruin wrote: > Ping! > > > On 4 September 2019 at 14:47:41, Bolke de Bruin (bdbr...@gmail.com) wrote: > > Hello Team! > > I have some outstanding review requests an

Re: [DISCUSS] Donating Airflow k8s Operator

+1 as well. One question: does the original author still actively maintain or would like to continue maintaining the code base after the repo is moved to Airflow? On Wed, Sep 11, 2019 at 11:10 PM Sumit Maheshwari wrote: > Strong +1 > > On Thu, Sep 12, 2019 at 7:29 AM Aizhamal Nurmamat kyzy < > a

Re: [DISCUSS] Donating Airflow k8s Operator

mer where we will most likely use and > > extend Kubernetes Operator. > > > > > > > > > > We have 3 people that could become contributors and commiters day one, so > > we can already help building community for it. Looking forward to > > following the

Re: Drop Python 3.5 support?

+1 On Tue, Nov 12, 2019 at 3:31 PM Jarek Potiuk wrote: > Well. It seems then that my Buster change might also contain dropping 3.5 > from Travis and switching to 3.6 everywhere we use 3.5 as default version > :). Good ! > > And as soon as we get faster CI we can add 3.8 to the list instead :). >

Re: [DISCUSS] Donating Airflow k8s Operator

Potiuk > > wrote: > > > > > > +1 for aieflow-on-k8s-operator. Good enough :) > > > > > > J. > > > > > > pt., 13 wrz 2019, 19:19 użytkownik Tao Feng > > napisał: > > > > > > > +1 on `airflow-on-k8s-op

Re: [VOTE] Accept new Airflow website contribution

+1 (binding), this is super cool :) On Thu, Nov 21, 2019 at 10:15 PM Maxime Beauchemin < maximebeauche...@gmail.com> wrote: > +1 (binding) this is amazing! > > it's hard to believe that the project gained so much traction despite the > website that we had before. > > On Thu, Nov 21, 2019 at 5:01

Re: [VOTE] Accept donation of Airflow k8s Operator to Apache Airflow

+1 (binding) On Thu, Nov 21, 2019 at 3:38 PM Tomasz Urbaszek wrote: > +1 (non-binding) > > Tomek > > On Thu, 21 Nov 2019 at 19:52, Kamil Breguła > wrote: > > > +1 (non binding) > > > > On Thu, Nov 21, 2019 at 7:17 PM Jay Sen wrote: > > > > > > +1 ( non-binding ) > > > > > > On Thu, Nov 21, 201

Re: [VOTE] Allow using asserts in Airflow code

-1 (binding) I share the same with most other comments. And I personally prefer to use try,except to make it consistent across the code base while use assert in unit test . On Thu, Dec 5, 2019 at 3:09 PM Felix Uellendall wrote: > -1 (binding) > > I agree. There shouldn’t be any confusion around

Re: Scheduler Stuck when using LocalExecutor and KubernetesPodOperator

Curious to know why you want to run k8spodoperator while you have k8s executor in prod? Could it solve for a different use case that cannot solve by k8s executor? On Thu, Dec 12, 2019 at 11:13 PM Maulik Soneji wrote: > Hi Maxime, > > We have been using KubernetesExecutor for our ETL use cases. >

Re: [DISCUSS] Donating Airflow k8s Operator

ugin.system.issuetabpanels%3Aall-tabpanel > > > > > > > > > > On Fri, Nov 15, 2019 at 10:46 AM Aizhamal Nurmamat kyzy < > > > > > aizha...@apache.org> wrote: > > > > > > > > > >> Hi Tao and everyone, > >

Re: [DISCUSS] Lineage improvements and standardization of operator signatures

Thanks Bolke. For those that are not aware, my team is working with Bolke's team on Amundsen which is a data discovery and metadata project( https://github.com/lyft/amundsen) . I think although it ships with Atlas client(or it used to be), the new API per my understanding is generic enough that doe

Re: [DISCUSS] Donating Airflow k8s Operator

I don't think Airflow is using github issues. I am a committer and can't create issue on the repo as well. I assume it is disabled by Apache infra. We could always file a jira for the issue :) . On Thu, Jan 23, 2020 at 1:54 PM Barni Seetharaman wrote: > Over the last few days we merged a few cha

Re: [DISCUSS] Donating Airflow k8s Operator

he.org/jira/projects/AIRFLOW/summary > Do we need to add a new component for "Airflow-on-k8s-operator" ? > > regs > Barni > > On Thu, Jan 23, 2020 at 2:12 PM Tao Feng wrote: > >> I don't think Airflow is using github issues. I am a committer and can't >

Re: [VOTE] Release Apache Airflow 1.10.9 from 1.10.9rc1 (shorter vote)

+1 On Fri, Feb 7, 2020 at 08:36 Driesprong, Fokko wrote: > +1 (binding) > > Op vr 7 feb. 2020 om 17:17 schreef Jarek Potiuk > > > +1 approved (binding). Works for me in the same configurations as 1.10.8 > > > > On Fri, Feb 7, 2020 at 5:05 PM Kaxil Naik wrote: > > > > > +1 approved (binding) >

Re: Big performance optimization of Scheduler - 10x faster , 2000+ fewer queries count

Great work Kamil! Let us know once it is landed in one of the future releases. Would love to try it out :) Best, -Tao On Mon, Feb 24, 2020 at 12:54 PM Qingping Hou wrote: > Awesome work Kamil! Great to see us embracing query batching in the > code base. I can't wait to deploy those optimization

Re: [VOTE] Switch from using Jira to Github Issues

+1 On Mon, Mar 16, 2020 at 7:36 AM Ash Berlin-Taylor wrote: > The original discuss thread is here. > https://lists.apache.org/thread.html/r19014fac5aa2aa15a86890aa438940da263e430cb535ca451ef451a4%40%3Cdev.airflow.apache.org%3E > > Proposal is: > > - We create a few appropriate issue templates in

Re: New committer: Jiajie Zhong

Congrats! On Wed, Apr 1, 2020 at 12:56 AM Robin Edwards wrote: > Nice one Jiajie :-) > > Cheers, > > R > > On Wed, 1 Apr 2020 at 07:58, Michał Słowikowski > wrote: > > > > Congratulations! > > > > Best regards > > M. > > > > wt., 31 mar 2020, 21:00 użytkownik Xinbin Huang > > napisał: > > > >

Re: Language matters - don't use the term "whitelist" in Apache Airflow

+1 On Mon, Jun 8, 2020 at 1:55 AM Soma S Dhavala wrote: > +1 👍 > > On Mon, Jun 8, 2020 at 2:05 PM Kaxil Naik wrote: > > > 👏 > > > > On Mon, Jun 8, 2020, 08:28 Robin Edwards wrote: > > > > > Nice one Ash, > > > > > > +1 > > > > > > R > > > > > > On Mon, 8 Jun 2020 at 05:04, Jarek Potiuk > > >

Re: Apply permission to create AIP

hey Nicholas, I added you. Let me know if you still can't access it. -Tao On Mon, Jun 15, 2020 at 7:54 PM 蒋晓峰 wrote: > Hello community, > I don't have permission to create AIP for new ideas to Airflow. Anyone > could help to grant permission to me? > My WIKI ID: Nicholas Jiang > W

Re: new committer: QP Hou

Welcome QP! On Tue, Jul 14, 2020 at 1:53 PM Jarek Potiuk wrote: > The Project Management Committee (PMC) for Apache Airflow has invited > QP Hou to become a committer and we are pleased to announce that he > has accepted. > > Being a committer enables easier contribution to the > project since t

Re: Discuss: should we allow HTML emails on dev@ list

+1 On Mon, Sep 14, 2020 at 09:05 Daniel Imberman wrote: > +1 (binding) > > via Newton Mail > > [ > https://cloudmagic.com/k/d/mailapp?ct=dx&cv=10.0.50&pv=10.15.6&source=email_footer_2 > ] > > On Mon, Sep 14, 2020 at 9:03 AM, Jarek Potiuk > > wrote: > > Oh noo .. I just started to use Pl

Re: Airflow Feature Design Feedback

+1, Lyft uses the extra blob in the log table? to store UI auditing information. On Fri, Oct 16, 2020 at 9:42 AM Sumit Maheshwari wrote: > +1 for the feature. Looking at the schema of the log table, I think it's > perfect to store such kind of information. > > On Fri, Oct 16, 2020 at 9:55 PM Dan

Re: [VOTE] Release Apache Airflow 2.0.0 form 2.0.0rc3

+1 (binding) great work! On Wed, Dec 16, 2020 at 23:02 Deng Xiaodong wrote: > +1 (non-binding). > > The issues that can be confirmed in rc1/2 have been well fixed. > > Cannot wait to see the bunch of new nice features in 2.0 to be rolled out. > > > XD > > On Thu, Dec 17, 2020 at 02:00 Maxime Bea

Re: A Visual Editor for Airflow pipelines

Hey Max, I just saw https://github.com/elyra-ai/elyra . May worth check out for your use case. On Wed, Mar 17, 2021 at 10:50 AM Max Payton wrote: > We at Lyft are starting to think about developing pipelines using Jupyter, > and was wondering if there were any notes from this meeting, or follow

Re: Searchable Slack archives are now here -- if we want it.

Hey Ash, That's great! I am interested in using for my other open source project ( https://github.com/amundsen-io/amundsen) which is currently hitting the 10k message limit as well. I wonder what it takes to setup/enable for other slack workspace? Thanks, -Tao On Wed, Apr 14, 2021 at 5:40 AM Ash

Re: New Committers: Jed Cunningham & Tzu-ping Chung

Congrats！ On Tue, Jun 29, 2021 at 13:20 Aizhamal Nurmamat kyzy wrote: > Congratulations, Jed and Tzu-ping! Thank you for your contributions! > > On Tue, Jun 29, 2021 at 7:43 AM Tomasz Urbaszek > wrote: > >> Congrats Jed & Tzu-ping!! >> >> Tomek >> >> On Tue, 29 Jun 2021 at 15:45, Bas Harenslak

Re: Increasing Astronomer's sponsorship of Airflow CI by 4x

Cool! On Thu, Jan 20, 2022 at 1:28 PM Jarek Potiuk wrote: > Super cool! > > czw., 20 sty 2022, 21:53 użytkownik Aizhamal Nurmamat kyzy < > aizha...@apache.org> napisał: > >> Awesome news! Thank you, Ash. >> >> On Thu, Jan 20, 2022 at 12:17 PM Ash Berlin-Taylor >> wrote: >> >>> Hi everyone, >>>

Re: New Committer: Ping Zhang

Congrats On Tue, May 31, 2022 at 7:08 AM Elad Kalif wrote: > Congrats! > > On Tue, May 31, 2022 at 4:56 PM Kaxil Naik wrote: > >> Congratulations Ping, welcome aboard >> >> On Tue, 31 May 2022 at 14:24, Josh Fell >> wrote: >> >>> Congrats Ping! >>> >>> On Mon, May 30, 2022 at 10:43 PM morefree

Re: [DISCUSS] Deprecate KnownEvent/KnownEventType

+1, but I think we still need the migration script for the change. On Wed, Jan 2, 2019 at 1:44 PM Bas Harenslak wrote: > +1 > > > On 2 Jan 2019, at 22:42, Daniel Imberman > wrote: > > > > +1 > > > > On Wed, Jan 2, 2019 at 1:18 PM Maxime Beauchemin < > maximebeauche...@gmail.com> > > wrote: > >

Re: Travis CI reporting incorrectly

reply.com > www.reply.com > > [image: Data Reply] > -- > *From:* Deng Xiaodong > *Sent:* Saturday, January 5, 2019 3:20:57 PM > *To:* dev@airflow.apache.org > *Cc:* Tao Feng; Driesprong, Fokko > *Subject:* Re: Travis CI reporting incorrectly > > Hi

Re: Travis CI reporting incorrectly

1465 introduced in https://github.com/apache/airflow/pull/4407. Not sure if it is a flaky test or a real issue. On Sat, Jan 5, 2019 at 9:16 PM Tao Feng wrote: > I will submit a pr to fix the test for PR # 4432 . > > > > On Sat, Jan 5, 2019 at 7:45 PM Deng Xiaodong wrote: > >

Re: Requesting AIP access

James, I added you to the wiki. Please give a try and see if it works for you. Happy AIP! On Mon, Jan 14, 2019 at 9:15 AM James Meickle wrote: > Could I get AIP access for "eronarn"? Thank you! >

Re: [VOTE] Airflow 1.10.2rc3

Hi Xiaodong, I just tried with a viewer role which can't reproduce your issue. I could see all the roles without any issues. Have you reset your db with this rc? On Sun, Jan 20, 2019 at 9:50 PM Deng Xiaodong wrote: > Hi Kaxil, > > A potential bug found in 1.10.2rc3. > > > # Potential Bug: > > V

Re: [VOTE] Airflow 1.10.2rc3

+1 (binding) I install the rc3 locally and create a new role and could view it in UI without issues. On Sat, Jan 19, 2019 at 8:36 AM Kaxil Naik wrote: > Hey all, > > I have cut Airflow 1.10.2 RC3. This email is calling a vote on the release, > which will last for 72 hours. Consider this my (bin

Re: [VOTE] Airflow 1.10.2rc3

is still > there. > > Please note the issue is not “see all the roles”. The issue is after > logging in as a Viewer role, I can’t access the pages including Tree View, > Graph View, Task Duration, Gantt, Code View, etc. > > > XD > > On Mon, Jan 21, 2019 at 12:03 Tao Fe

Re: [VOTE] Airflow 1.10.2rc3

by Seelman, can you change your *vote >> back to +1 *. >> > And we can fix that in the upcoming release with other fixes >> which should >> > be quick and aim to release that in a month as well? >> > >> &g

Re: [VOTE] Airflow 1.10.2rc3

ot;-1" vote or "+1" vote? > > On Mon, Jan 21, 2019, 18:21 Tao Feng >> Create a jira for the viewer bug( >> https://issues.apache.org/jira/browse/AIRFLOW-3745). >> >> On Mon, Jan 21, 2019 at 9:12 AM Deng Xiaodong >> wrote: >> >>> Hi

Re: [VOTE] Airflow 1.10.2rc3

+1 given it is a time based release. On Tue, Jan 22, 2019 at 7:00 AM Ash Berlin-Taylor wrote: > +1 (binding) > > - Checksums match > - Signing good > - version.py doesn't contain "rc" > - UPDATING.md looks good > - Tested celery executor on redis. > - Pausing/unpausing dags on RBAC UI > - Impor

Re: [PROPOSAL] Add a landing page for Apache Airflow

Not sure how you are going to tackle this issue. But I look at Apache Samza's repo which hosts the website content(http://samza.apache.org/) under the doc folder(https://github.com/apache/samza/tree/master/docs). Just FYI. On Thu, Jan 24, 2019 at 10:28 PM Tao Feng wrote: > +1. This is

Re: [PROPOSAL] Add a landing page for Apache Airflow

+1. This is a great idea! Thanks, -Tao On Thu, Jan 24, 2019 at 4:49 PM Aizhamal Nurmamat kyzy wrote: > Hello all, > thanks for the prompt feedback! It seems like the community would like it. > I'll start working on an AIP. > > > On Thu, Jan 24, 2019 at 4:11 PM Sid Anand wrote: > > > > > > > A

Re: [PROPOSAL] Add a landing page for Apache Airflow

I grant you the permission for the wiki space. You are good to go. On Thu, Jan 24, 2019 at 4:49 PM Aizhamal Nurmamat kyzy wrote: > Hello all, > thanks for the prompt feedback! It seems like the community would like it. > I'll start working on an AIP. > > > On Thu, Jan 24, 2019 at 4:11 PM Sid Ana

Re: Extending AirflowSecurityManager

I think you could extend that method. We shouldn't call db directly unless FAB doesn't provide the api to do so. On Wed, Jan 23, 2019 at 8:18 AM Victor Noël wrote: > Hi, > > I'm upgrading from 1.10.1 to 1.10.2 and I had a custom SecurityManager that > was extending flask_appbuilder.security.sqla

Re: Custom scheduler support in Airflow

If I understand your request correctly, I don't think you need a custom scheduler but have a custom way to create dag run which is non-time based? Have you looked into TriggerDagRunOperator ( https://github.com/apache/airflow/blob/master/airflow/operators/dagrun_operator.py#L36 )? On Thu, Jan 31,

Re: API Reference - current confusion and improvement plan

+1 as well, I like the layout for the new doc. On Tue, Feb 5, 2019 at 11:57 AM Ash Berlin-Taylor wrote: > I like the API reference v2 layout a lot! Much easier to navigate and see > what classes are available, for me at least > > Documenting modules will help somewhat with a few things but, lets

better way to schedule pyspark with SparkOperator on Airflow

Hi, I wonder any suggestions on how to use SparkOperator to send pyspark file to the spark cluster. And any suggestions on how to specify the pyspark dependency ? We currently push user pyspark file and dependency to a S3 location and get picked up by our Spark cluster. And we would like to explo

Re: better way to schedule pyspark with SparkOperator on Airflow

single GIT repository: > > dir_path = os.path.dirname(os.path.realpath(__file__)) > DataProcPySparkOperator( > main=dir_path + 'my_pyspark_job.py' > ) > > Hope this helps. > > Cheers, Fokko > > > > Op do 7 feb. 2019 om 08:27 schreef Tao Feng : > > > Hi,

Re: Voting on AIPs - what's our process?

Please correct me if I am wrong. I think normally it starts with a [DISCUSS] email thread for all the technical details discussion. If everyone aligns, we could move to the vote thread. Here are some guidelines found on the apache website(https://www.apache.org/foundation/voting.html) . A few point

Re: [VOTE] Accept AIP-13: OpenAPI 3 based API definition

+1(binding) Thanks for the AIP. I think the proposal looks good to me overall. But I have a question for the authentication part. Currently, the Airflow webserver supports DAG level access control which is managed by Airflow security manager. I wonder whether the AIP authentication interface has

Re: RFC - GCP Operator Development Guide

+1 from my side. Thanks Feng. On Thu, Feb 21, 2019 at 8:23 PM Feng Lu wrote: > Hi all, > > 20-ish people from the community have reviewed this doc. I believe we have > addressed all comments and the doc is good to go. > Trying to figure out the right place to publish this GCP operator style > gu

Re: Voting on AIPs - what's our process?

spoke up on the discussion thread ;) > > -ash > > > > On 21 Feb 2019, at 20:22, Tao Feng wrote: > > > > Please correct me if I am wrong. I think normally it starts with a > > [DISCUSS] email thread for all the technical details discussion. If > > everyone alig

Re: [DISCUSS] AIP-12 Persist DAG into DB

Thanks for bringing this up. I look at the pr before and here recaps some of the previous concerns: 1. There will have a new *giant *button next to "go" button in graph view(it could be found in the screenshot in the pr) based on my last read of the pr. It even exists after DAGRun is created. I do

Re: [DISCUSS] AIP-12 Persist DAG into DB

I manage to test out the latest version of the branch which I think all my previous concerns don't exist anymore. Great job! The latest screenshot for the new graph view could be found in the pr comment. On Sat, Feb 23, 2019 at 3:30 PM Tao Feng wrote: > Thanks for bringing this up. I

Re: Voting on AIPs - what's our process?

he code or > merging > >>> the pr can happen on GitHub as usual. > >>> > >>> For example AIP-12 is an excellent Idea but there are a few questions > to > >>> answer about the design. (The author of the proposal also has an open > >> PR

Re: RFC - GCP Operator Development Guide

> Either confluence of this is fine with me. >> >> Regards, >> Kaxil >> >> On Fri, Feb 22, 2019 at 10:48 AM Tao Feng wrote: >> >> > +1 from my side. Thanks Feng. >> > >> > On Thu, Feb 21, 2019 at 8:23 PM Feng Lu >> wrote: &

Re: Change Airflow Dag Execution time timezone on Airflow UI

Have you looked into https://github.com/apache/airflow/blob/0d64fd8aac7b93f4143d8f1d3baa4a0e239aba55/docs/timezone.rst or set https://github.com/apache/airflow/blob/master/airflow/config_templates/default_airflow.cfg#L78 to Beijing timezone? Does it not work for you use case? On Thu, Feb 28, 2019

Re: Change Airflow Dag Execution time timezone on Airflow UI

n. > > > XD > > On Fri, Mar 1, 2019 at 14:04 Manu Zhang wrote: > > > IIRC, the timezone setting only changes backend schedule time but nothing > > on UI and causes even more confusion. > > > > > > Manu Zhang > > On Mar 1,

Re: Multiple Schedulers - "scheduler_lock"

Does the proposal use master-slave architecture(leader scheduler vs slave scheduler)? On Fri, Mar 1, 2019 at 5:32 PM Kevin Yang wrote: > Preventing double-triggering by separating DAG files different schedulers > parse sounds easier and more intuitive. I actually removed one of the > double-trig

Re: Welcoming Xiaodong Deng as a new commiter!

Congrats Xiaodong! On Thu, Mar 7, 2019 at 5:25 AM Ash Berlin-Taylor wrote: > Hello Airflowers, > > Please join the Apache Airflow PMC in welcoming our newest committer: > Xiaodong Deng (a.k.a. XD-DENG ). > > XD has been an active member of the community helping other

Welcome Daniel Imberman as a new committer!

Hellow Airflowers, Please join the Apache Airflow PMC in welcoming our newest committer: Daniel Imberman (a.k.a. dimberman ). Daniel has been a key contributor on the Airflow K8S integration and has been helping the community for a long time. Welcome Daniel, great t

Re: [VOTE] Accept AIP-3: Drop support for Python 2

+1 (binding) On Sat, Mar 23, 2019 at 4:19 PM Driesprong, Fokko wrote: > Dear Airflow community, > > This email calls for a vote to accept Airflow Improvement Proposal 3: Drop > support for Python 2 > > The vote will last for at least 1 week until Midnight Sunday 31th of March, > and until three

Re: Airflow 1.10.3 has been released!

Thanks Ash for the work! On Wed, Apr 10, 2019 at 11:29 AM Aizhamal Nurmamat kyzy wrote: > Thank you, Ash for all the work! > > On Wed, Apr 10, 2019 at 11:26 AM Driesprong, Fokko > wrote: > > > Thanks a lot for being our release manager Ash, great job! > > > > Big thanks to the community for ma

Re: [VOTE] AIP-6 Apply Pylint to Airflow

+1 On Thu, Apr 11, 2019 at 4:27 PM Beau Barker wrote: > +1 non binding > > Pylint is extremely strict so your may want to be selective about the > rules it applies. > > > > On 12 Apr 2019, at 7:15 am, Kamil Breguła > wrote: > > > > +! (non-binding) > > > > On Thu, Apr 11, 2019 at 11:05 PM Dries

Re: [2.0 spring cleaning] Deprecate subdags

+1 on fixing the subdags. I think one issue I recall is that subdag uses backfill scheduler which doesn't have the on same functionality as normal scheduler. E.g the tasks within subdag are not resource bound by the pool. But with Chao-han's fix on https://github.com/apache/airflow/pull/4949, I th

Re: [DISCUSS]: Remove Mesos Executor from Airflow 2.0.0?

+1 on removing mesos executor. On Fri, Apr 12, 2019 at 2:25 PM Daniel Imberman wrote: > Since we are now doing version 2.0 spring cleaning can we revisit this? > > On Wed, Feb 27, 2019 at 8:38 AM Maxime Beauchemin < > maximebeauche...@gmail.com> wrote: > > > +1 as it relies on pickles > > > > On

Re: [VOTE] AIP-10: Multi-layered and multi-stage official Airflow image

+1 binding On Mon, Apr 8, 2019 at 2:06 PM Kaxil Naik wrote: > +1 binding > > On Mon, Apr 8, 2019, 21:50 Maxime Beauchemin > wrote: > > > +1 binding > > > > Max > > > > On Sun, Apr 7, 2019 at 5:23 PM Jiajie Zhong > > wrote: > > > > > +1 non binding > > > > > > Best wish. > > > -- Jiajie > > > _

Re: Proposal: Directly tie Jira into Github for airflow

+1 On Wed, Apr 17, 2019 at 2:19 PM Ash Berlin-Taylor wrote: > +1 > > I've ask in #asfinfra (on the-asfk.slack.com) to see if we can actually > use that on the Apache Jira install. > > -ash > > > On 17 Apr 2019, at 22:16, Kevin Yang wrote: > > > > +1 > > > > Cheers, > > Kevin Y > > > > Bas Haren

Re: [ANNOUNCE] Please welcome new Airflow committer Jarek Potiuk

Congrats! On Wed, Apr 17, 2019 at 2:08 PM Jarek Potiuk wrote: > Thanks a lot! It's an honor to be part of such great community and I am > happy to take more responsibility as commiter. Looking forward to what we > can do together! > > J.. > > Principal Software Engineer > Phone: +48660796129 > >

Re: New job: I'm joining Astronomer.io to work on Airflow

Congrats Ash and the Astronomer team! On Thu, Apr 18, 2019 at 11:37 AM Ash Berlin-Taylor wrote: > Hi everyone! > > I've hinted at this to a few people in Slack, and it's now official. I'm > super excited to be able to say that I've joined Astronomer as a full time > employee, and a good portion

Re: [ANNOUNCE] Please welcome new Airflow committer Joshua Carp

Congrats Joshua! On Mon, Apr 29, 2019 at 2:44 PM Kaxil Naik wrote: > Welcome aboard Joshua > > On Mon, Apr 29, 2019, 22:28 Aizhamal Nurmamat kyzy > wrote: > > > Congratulations Joshua! Thank you for all your contributions! [image: > > 🥳][image: > > 🥳][image: 🥳] > > > > On Mon, Apr 29, 2019 at 2

Re: [ANNOUNCE] Please welcome new Airflow committer Kevin Yang

Congrats! On Tue, Apr 30, 2019 at 10:09 AM Daniel Imberman < dimberman.opensou...@gmail.com> wrote: > Congrats Kevin! > > On Tue, Apr 30, 2019 at 9:09 AM Aizhamal Nurmamat kyzy > wrote: > > > Congratulations Kevin! > > > > On Tue, Apr 30, 2019 at 02:06 Jiajie Zhong > > wrote: > > > > > Congratu

[ANNOUNCE] Please welcome new Airflow committer Kamil Breguła

All, I'm pleased to announce that the Airflow PMC has voted in Kamil Breguła to be a committer. Kamil has been actively involved in the project for some time now, contributing patches and participating in the mailing list as well as AIP discussion. He spends lots of efforts improving the documenta

Re: [DISCUSS] AIP-5: Remote DAG Fetcher

A vote thread should be initiated which AIP get accepted if it gets 3 binding vote. There are three AIPs which been accepted so far. On Wed, May 1, 2019 at 7:49 PM Chao-Han Tsai wrote: > Dear Airflow committers, > > What is the current process to get an AIP accepted/approved? I looked > through

Re: Announcement: I'm joining Astronomer!

Daniel, Congrats! Win-win for you, Airflow and Astronomer! Best, -Tao On Thu, May 30, 2019 at 3:38 PM Arthur Wiedmer wrote: > Daniel, > > Congratulations! Very exciting! > > Best, > Arthur > > On Thu, May 30, 2019 at 2:07 PM Daniel Imberman > > wrote: > > > Hello Airflow community! > > > > I'

Re: Project Genevere

Hi Gerard, Good to know :) FYI, Linkedin created https://github.com/apache/incubator-gobblin as the data ingestion framework which could potentially solve your use case. But I am not sure it provides support for sqlserver as sink. Best, -Tao On Fri, May 31, 2019 at 6:58 AM Gerard Toonstra wrote

Re: [VOTE] AIP-16: CLI: Use nested commands instead of flags

+1 (binding) On Tue, Jun 11, 2019 at 4:15 AM Ash Berlin-Taylor wrote: > Hi Airflowers, > > This email calls for a vote to introduce restructure the CLI to use nested > commands instead of flags. The vote will last for at least 1 week (June > 18th 12:00 BST), and at least three +1 (binding) votes

Re: I have joined Astronomer full-time !!

Awesome:) Congrats! On Fri, Jun 21, 2019 at 11:53 AM Kaxil Naik wrote: > Hi all, > > I have already posted this on LinkedIn but for others in the community, I > am super-excited to announce that as of 20 June I have joined Astronomer as > a full-time employee. And similar to Ash & Daniel, I will

Re: [2.0 spring cleaning] Remove `dag >> task`?

I am not aware of this feature either. And +1 on removing it. On Wed, Jul 3, 2019 at 9:36 AM Kaxil Naik wrote: > I am definitely sure that 99% of users, including me, didn't knew this > feature ever existed 😀. > > It is not a feature worth having tbh. So I am in favor of removing it. > > Regards

Re: [DISCUSS] AIP-12 Persist DAG into DB

Do we reach any consensus on this topic /AIP? I think persisting DAG is pretty important actually. -Tao On Tue, Mar 12, 2019 at 3:01 AM Kevin Yang wrote: > Hi Fokko, > > As a large cluster maintainer, I’m not a big fan of large DAG files > neither. But I’m not sure if I’ll consider this bad pra

Re: [DISCUSS] AIP-12 Persist DAG into DB

f Airflow - Piper > > https://eng.uber.com/managing-data-workflows-at-scale/ > > > > I'll create something in the wiki (probably not an AIP to start with) to > > collect the possible approaches and downsides/limitations. > > > > Watch this space. > > &g

Re: Removal of "run_duration" and its impact on orphaned tasks

Late in the game as I was't aware of `run_duration` option been removed. But just want to point out that Lyft also did very similar with James' setup that we run the scheduler in a fix internal instead of infinite loop and let the runit/supervisor to restart the scheduler process. This is to solve:

Re: Airflow DAG Serialisation

hey Zhou, Great to see this happens and make it backward compatible. I will persist DAG into DB is definitely needed. And it will make migration easier with a lightweight approach. At Lyft we sometimes observe nondeterministic increased scheduling delay once users add some dynamic generated large

Re: Blog post: Upgrading & Scaling Airflow at Robinhood

+1 Max, thanks for sharing! On Fri, Aug 9, 2019 at 2:05 PM Jarek Potiuk wrote: > +1 > > On Fri, Aug 9, 2019 at 10:54 PM Maxime Beauchemin < > maximebeauche...@gmail.com> wrote: > >> Thanks to Abhishek Ray @ Robinhood for this great post. I felt like I had >> to share it here >> >> https://robinh

[ANNOUNCE] Please welcome new Airflow committer Chao-han Tsai

All, I'm pleased to announce that the Airflow PMC has voted in Chao-han Tsai to be a committer. Chao-han has been actively involved in the project for some time now, contributing patches including backfill enhancement, stats monitoring and subdag improvement and participating in the mailing list a

Re: [IE] Re: [VOTE] Change the Airflow logo

Option 1 On Wed, Aug 21, 2019 at 8:26 AM Leah Cole wrote: > Option 1 > > On Wed, Aug 21, 2019 at 7:01 AM Sunil Varma Chiluvuri < > sunilvarma.chiluv...@equifax.com> wrote: > > > Option 1 > > > > On Wed, Aug 21, 2019 at 8:55 AM Kaxil Naik wrote: > > > > > A real tough one. > > > > > > *Option 1*