[GitHub] [airflow] ashb commented on issue #6274: AIRFLOW-5608: Fix bug in SchedulerJob when calling executor.end

[GitBox]

ashb commented on issue #6274: AIRFLOW-5608: Fix bug in SchedulerJob when 
calling executor.end
URL: https://github.com/apache/airflow/pull/6274#issuecomment-543176050
 
 
   @kpathak13 This PR breaks the Scheduler stopping with LocalExecutor:
   
   
   ```
   Traceback (most recent call last):
     File 
"/Users/ash/code/python/incubator-airflow/airflow/jobs/scheduler_job.py", line 
1358, in _execute
       self._execute_helper()
     File 
"/Users/ash/code/python/incubator-airflow/airflow/jobs/scheduler_job.py", line 
1485, in _execute_helper
       sleep(sleep_length)
     File "/Users/ash/code/python/incubator-airflow/airflow/bin/cli.py", line 
91, in sigint_handler
       sys.exit(0)
   SystemExit: 0
   
   During handling of the above exception, another exception occurred:
   
   Traceback (most recent call last):
     File "/Users/ash/.virtualenvs/airflow/bin/airflow", line 7, in <module>
       exec(compile(f.read(), __file__, 'exec'))
     File "/Users/ash/code/python/incubator-airflow/airflow/bin/airflow", line 
37, in <module>
       args.func(args)
     File "/Users/ash/code/python/incubator-airflow/airflow/utils/cli.py", line 
74, in wrapper
       return f(*args, **kwargs)
     File "/Users/ash/code/python/incubator-airflow/airflow/bin/cli.py", line 
1042, in scheduler
       job.run()
     File "/Users/ash/code/python/incubator-airflow/airflow/jobs/base_job.py", 
line 222, in run
       self._execute()
     File 
"/Users/ash/code/python/incubator-airflow/airflow/jobs/scheduler_job.py", line 
1363, in _execute
       self.executor.end()
     File 
"/Users/ash/code/python/incubator-airflow/airflow/executors/local_executor.py", 
line 233, in end
       self.impl.end()
     File 
"/Users/ash/code/python/incubator-airflow/airflow/executors/local_executor.py", 
line 209, in end
       self.queue.put((None, None))
     File "<string>", line 2, in put
     File 
"/Users/ash/.homebrew/Cellar/python/3.7.3/Frameworks/Python.framework/Versions/3.7/lib/python3.7/multiprocessing/managers.py",
 line 796, in _callmethod
       kind, result = conn.recv()
     File 
"/Users/ash/.homebrew/Cellar/python/3.7.3/Frameworks/Python.framework/Versions/3.7/lib/python3.7/multiprocessing/connection.py",
 line 250, in recv
       buf = self._recv_bytes()
     File 
"/Users/ash/.homebrew/Cellar/python/3.7.3/Frameworks/Python.framework/Versions/3.7/lib/python3.7/multiprocessing/connection.py",
 line 407, in _recv_bytes
       buf = self._recv(4)
     File 
"/Users/ash/.homebrew/Cellar/python/3.7.3/Frameworks/Python.framework/Versions/3.7/lib/python3.7/multiprocessing/connection.py",
 line 383, in _recv
       raise EOFError
   EOFError
   ```
   
   This makes what _was_ a clean shutdown for me an abnormal shutdown. I am 
reverting this change.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services