Suspend and resume Sensors

2016-10-17 Thread Michal K
Hello Everyone, our company is currently considering Airflow for the engine of a pretty big workflow system.We use Python 3 and have some HA requirements, so we plan to use Celery with multiple worker and scheduler nodes and RabbitMQ as message bus. We are also using Docker containers for deploy

Article - Get started developing workflows with Apache Airflow

2017-04-04 Thread Michal K
Hi everyone, I would just like to let you know that I wrote an introductory article for new users who are getting started with Airflow. You can find it here: https://bitly.com/2nGRgzj I would be happy to hear your comments or suggestions. Feel free to link to the article if you find it useful.

Re: Article - Get started developing workflows with Apache Airflow

2017-04-05 Thread Michal K
ted in its own virtualenv, so dependencies don't conflict. Michal On Wednesday, April 5, 2017 3:07 AM, Boris Tyukin wrote: Hi Michał, really enjoyed your post - great job! what is the reason to use virtualenv? would you use it for production setup? On Tue, Apr 4, 2017 at 4:25 PM,

Airflow talks at EuroPython 2017

2017-06-07 Thread Michal K
Hi there,  Is anyone planning to attend this year's EuroPython conference? I'll be there and I will be giving a talk on Airflow:https://ep2017.europython.eu/conference/talks/developing-elegant-workflows-in-python-code-with-apache-airflow If you're going I hope to meet you there. Let me know, or ju