This is an automated email from the ASF dual-hosted git repository. potiuk pushed a commit to branch v2-1-test in repository https://gitbox.apache.org/repos/asf/airflow.git
commit 00e8072db677df815fac9f72ebb66f3404ba1538 Author: Ash Berlin-Taylor <[email protected]> AuthorDate: Sat Jun 26 09:13:55 2021 +0100 Remove upstart from docs (#16672) No modern dist ships upstart anymore (everyone has migrated to systemd, or never used upstart) so these docs have no value anymore. (cherry picked from commit 6d6eae3851de5c5b209a31201d1430a995e525ad) --- MANIFEST.in | 1 - docs/apache-airflow/howto/index.rst | 1 - docs/apache-airflow/howto/run-with-upstart.rst | 44 -------------------------- scripts/upstart/README | 33 ------------------- scripts/upstart/airflow-flower.conf | 34 -------------------- scripts/upstart/airflow-scheduler.conf | 38 ---------------------- scripts/upstart/airflow-webserver.conf | 34 -------------------- scripts/upstart/airflow-worker.conf | 34 -------------------- 8 files changed, 219 deletions(-) diff --git a/MANIFEST.in b/MANIFEST.in index 9dcf610..4dc999f 100644 --- a/MANIFEST.in +++ b/MANIFEST.in @@ -25,7 +25,6 @@ graft airflow/www graft airflow/www/static graft airflow/www/templates graft scripts/systemd -graft scripts/upstart graft airflow/config_templates recursive-exclude airflow/www/node_modules * global-exclude __pycache__ *.pyc diff --git a/docs/apache-airflow/howto/index.rst b/docs/apache-airflow/howto/index.rst index 9a7e68b..efd5c48 100644 --- a/docs/apache-airflow/howto/index.rst +++ b/docs/apache-airflow/howto/index.rst @@ -40,7 +40,6 @@ configuring an Airflow environment. variable run-behind-proxy run-with-systemd - run-with-upstart use-test-config define_extra_link email-config diff --git a/docs/apache-airflow/howto/run-with-upstart.rst b/docs/apache-airflow/howto/run-with-upstart.rst deleted file mode 100644 index cf6bd7f..0000000 --- a/docs/apache-airflow/howto/run-with-upstart.rst +++ /dev/null @@ -1,44 +0,0 @@ - .. Licensed to the Apache Software Foundation (ASF) under one - or more contributor license agreements. See the NOTICE file - distributed with this work for additional information - regarding copyright ownership. The ASF licenses this file - to you under the Apache License, Version 2.0 (the - "License"); you may not use this file except in compliance - with the License. You may obtain a copy of the License at - - .. http://www.apache.org/licenses/LICENSE-2.0 - - .. Unless required by applicable law or agreed to in writing, - software distributed under the License is distributed on an - "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY - KIND, either express or implied. See the License for the - specific language governing permissions and limitations - under the License. - - - -Running Airflow with upstart -============================ - -Airflow can integrate with upstart based systems. Upstart automatically starts all airflow services for which you -have a corresponding ``*.conf`` file in ``/etc/init`` upon system boot. On failure, upstart automatically restarts -the process (until it reaches re-spawn limit set in a ``*.conf`` file). - -You can find sample upstart job files in the ``scripts/upstart`` directory. - -The following assumptions have been used while creating these unit files: - -1. Airflow will run as the following ``user:group`` ``airflow:airflow``. - Change ``setuid`` and ``setgid`` appropriately in ``*.conf`` if airflow runs as a different user or group -2. These files have been tested on **Ubuntu 14.04 LTS** - You may have to adjust ``start on`` and ``stop on`` stanzas to make it work on other upstart systems. - Some of the possible options are listed in ``scripts/upstart/README`` - -Modify ``*.conf`` files as needed and copy to ``/etc/init`` directory. - -You can use ``initctl`` to manually start, stop, view status of the airflow process that has been -integrated with upstart - -.. code-block:: bash - - initctl airflow-webserver status diff --git a/scripts/upstart/README b/scripts/upstart/README deleted file mode 100644 index 124c18c..0000000 --- a/scripts/upstart/README +++ /dev/null @@ -1,33 +0,0 @@ -The upstart files in this directory are tested on Ubuntu 14.04 LTS based systems running in VPC on AWS. - -Copy *.conf files to /etc/init. - -You can then start airflow services by using initctl start <service>. Where <service> is airflow-worker, -airflow-webserver, airflow-scheduler, etc. - -Upstart automatically starts all airflow services for which you have a corresponding *.conf file in /etc/init -upon system boot. If service process dies, upstart will automatically re-spawn it (until it hits re-spawn limit -set in a *.conf file) - -You may have to adjust `start on` & `stop on` stanzas to make it work on other upstart systems. Some of the possible -options are listed below - -# This should work on most Linux distributions that support upstart -start on started network-services - -# This is for Ubuntu based systems which lack generic network-services job -# Wait for a non-loopback interface before starting airflow services -start on (local-filesystems and net-device-up IFACE!=lo) - -# This should work on Ubuntu 11.10 based systems -# Start after all network interfaces are up -start on static-network-up - -# If nothing else works, use this -start on runlevel [2345] - -It is assumed that airflow will run under `airflow:airflow`. Change `setuid` and `setgid` in *.conf files -if you use other user/group - -You can use `initctl` to manually start, stop, view status of the airflow process. For example -`initctl status airflow-webserver` diff --git a/scripts/upstart/airflow-flower.conf b/scripts/upstart/airflow-flower.conf deleted file mode 100644 index 176ffb0..0000000 --- a/scripts/upstart/airflow-flower.conf +++ /dev/null @@ -1,34 +0,0 @@ -# Licensed to the Apache Software Foundation (ASF) under one -# or more contributor license agreements. See the NOTICE file -# distributed with this work for additional information -# regarding copyright ownership. The ASF licenses this file -# to you under the Apache License, Version 2.0 (the -# "License"); you may not use this file except in compliance -# with the License. You may obtain a copy of the License at -# -# http://www.apache.org/licenses/LICENSE-2.0 -# -# Unless required by applicable law or agreed to in writing, -# software distributed under the License is distributed on an -# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY -# KIND, either express or implied. See the License for the -# specific language governing permissions and limitations -# under the License. - -description "Airflow celery flower" - -start on started networking -stop on (deconfiguring-networking or runlevel [016]) - -respawn -respawn limit 5 30 - -setuid airflow -setgid airflow - -# env AIRFLOW_CONFIG= -# env AIRFLOW_HOME= -# export AIRFLOW_CONFIG -# export AIRFLOW_HOME - -exec usr/local/bin/airflow celery flower diff --git a/scripts/upstart/airflow-scheduler.conf b/scripts/upstart/airflow-scheduler.conf deleted file mode 100644 index e8bb1fe..0000000 --- a/scripts/upstart/airflow-scheduler.conf +++ /dev/null @@ -1,38 +0,0 @@ -# Licensed to the Apache Software Foundation (ASF) under one -# or more contributor license agreements. See the NOTICE file -# distributed with this work for additional information -# regarding copyright ownership. The ASF licenses this file -# to you under the Apache License, Version 2.0 (the -# "License"); you may not use this file except in compliance -# with the License. You may obtain a copy of the License at -# -# http://www.apache.org/licenses/LICENSE-2.0 -# -# Unless required by applicable law or agreed to in writing, -# software distributed under the License is distributed on an -# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY -# KIND, either express or implied. See the License for the -# specific language governing permissions and limitations -# under the License. - -description "Airflow scheduler daemon" - -start on started networking -stop on (deconfiguring-networking or runlevel [016]) - -respawn -respawn limit 5 10 - -setuid airflow -setgid airflow - -# env AIRFLOW_CONFIG= -# env AIRFLOW_HOME= -# export AIRFLOW_CONFIG -# export AIRFLOW_HOME - -# required setting, 0 sets it to unlimited. Scheduler will restart after every X runs -env SCHEDULER_RUNS=5 -export SCHEDULER_RUNS - -exec usr/local/bin/airflow scheduler -n ${SCHEDULER_RUNS} diff --git a/scripts/upstart/airflow-webserver.conf b/scripts/upstart/airflow-webserver.conf deleted file mode 100644 index 241dcd9..0000000 --- a/scripts/upstart/airflow-webserver.conf +++ /dev/null @@ -1,34 +0,0 @@ -# Licensed to the Apache Software Foundation (ASF) under one -# or more contributor license agreements. See the NOTICE file -# distributed with this work for additional information -# regarding copyright ownership. The ASF licenses this file -# to you under the Apache License, Version 2.0 (the -# "License"); you may not use this file except in compliance -# with the License. You may obtain a copy of the License at -# -# http://www.apache.org/licenses/LICENSE-2.0 -# -# Unless required by applicable law or agreed to in writing, -# software distributed under the License is distributed on an -# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY -# KIND, either express or implied. See the License for the -# specific language governing permissions and limitations -# under the License. - -description "Airflow webserver daemon" - -start on started networking -stop on (deconfiguring-networking or runlevel [016]) - -respawn -respawn limit 5 30 - -setuid airflow -setgid airflow - -# env AIRFLOW_CONFIG= -# env AIRFLOW_HOME= -# export AIRFLOW_CONFIG -# export AIRFLOW_HOME - -exec usr/local/bin/airflow webserver diff --git a/scripts/upstart/airflow-worker.conf b/scripts/upstart/airflow-worker.conf deleted file mode 100644 index 94c3a7c..0000000 --- a/scripts/upstart/airflow-worker.conf +++ /dev/null @@ -1,34 +0,0 @@ -# Licensed to the Apache Software Foundation (ASF) under one -# or more contributor license agreements. See the NOTICE file -# distributed with this work for additional information -# regarding copyright ownership. The ASF licenses this file -# to you under the Apache License, Version 2.0 (the -# "License"); you may not use this file except in compliance -# with the License. You may obtain a copy of the License at -# -# http://www.apache.org/licenses/LICENSE-2.0 -# -# Unless required by applicable law or agreed to in writing, -# software distributed under the License is distributed on an -# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY -# KIND, either express or implied. See the License for the -# specific language governing permissions and limitations -# under the License. - -description "Airflow celery worker daemon" - -start on started networking -stop on (deconfiguring-networking or runlevel [016]) - -respawn -respawn limit 5 30 - -setuid airflow -setgid airflow - -# env AIRFLOW_CONFIG= -# env AIRFLOW_HOME= -# export AIRFLOW_CONFIG -# export AIRFLOW_HOME - -exec usr/local/bin/airflow celery worker
