[ https://issues.apache.org/jira/browse/AIRFLOW-2975?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16596216#comment-16596216 ]
jack commented on AIRFLOW-2975: ------------------------------- [~ashb] How can I optimize the number of workers? 1 per core? > Add support for Amazon cloudwatch (computing power at will) > ----------------------------------------------------------- > > Key: AIRFLOW-2975 > URL: https://issues.apache.org/jira/browse/AIRFLOW-2975 > Project: Apache Airflow > Issue Type: New Feature > Affects Versions: 1.10.0 > Reporter: jack > Priority: Major > Fix For: 2.0.0, 1.10.1 > > > Some of have one machine that runs airfow… > While we can scale up the executor to have many resources over different > servers this is consider to be expensive. > There is another solution.. using Amazon Cloudwatch: > [https://aws.amazon.com/premiumsupport/knowledge-center/start-stop-lambda-cloudwatch/] > > This enable the user to create and close EC2 machines on specific intervals > for specific tasks. > Basically if I have 50 DAGS to run on 1PM-3PM and few dags to run on other > hours there is no point in paying for 2nd server 24/7. > This could be an enhancement to one of the executors that know to work with > more than one server. > > > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)