jack created AIRFLOW-2975:
-----------------------------
Summary: Add support for Amazon cloudwatch (computing power at
will)
Key: AIRFLOW-2975
URL: https://issues.apache.org/jira/browse/AIRFLOW-2975
Project: Apache Airflow
Issue Type: New Feature
Affects Versions: 1.10.0
Reporter: jack
Fix For: 2.0.0, 1.10.1
Some of have one machine that runs airfow…
While we can scale up the executor to have many resources over different
servers this is consider to be expensive.
There is another solution.. using Amazon Cloudwatch:
[https://aws.amazon.com/premiumsupport/knowledge-center/start-stop-lambda-cloudwatch/]
This enable the user to create and close EC2 machines on specific intervals for
specific tasks.
Basically if I have 50 DAGS to run on 1PM-3PM and few dags to run on other
hours there is no point in paying for 2nd server 24/7.
This could be an enhancement to one of the executors that know to work with
more than one server.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
