Actually, I saw the error today even without the broken pipe message in the
logs. Perhaps the broken pipe message is a red herring?
Airflow seems to lose track of jobs randomly, even though the underlying
process seems to be healthy and doing it's thing. No errors, nothing in the
logs. Task gets marked as failed and downstream tasks never get kicked off.
On Tue, Jul 24, 2018 at 2:11 AM, Sai Phanindhra <phani8...@gmail.com> wrote:
> Im facing the same issue. Even worker does not seem to be writing in
> filesystem.
>
> On Mon 23 Jul, 2018, 22:26 srinivas.ramabhad...@gmail.com, <
> srinivas.ramabhad...@gmail.com> wrote:
>
> > HI all -
> >
> > I am using Airflow version 1.9.0 and am seeing these random errors in
> > my logs. This causes Airflow to think that the task has failed, even
> though
> > I can find the process with a ps aux. Any ideas what is causing this?
> Don't
> > know if this is relevant, but this happens with processes that are fairly
> > long-lived (they poll every minute during the day until it's time for
> them
> > to run).
> >
> > [2018-07-23 16:31:53,891] {logging_mixin.py:84} WARNING - Traceback (most
> > recent call last):
> >
> > [2018-07-23 16:31:53,892] {logging_mixin.py:84} WARNING - File
> > "/var/software/anaconda3/envs/statar_20180624/lib/python3.6/
> logging/__init__.py",
> > line 996, in emit
> > self.flush()
> >
> > [2018-07-23 16:31:53,892] {logging_mixin.py:84} WARNING - File
> > "/var/software/anaconda3/envs/statar_20180624/lib/python3.6/
> logging/__init__.py",
> > line 976, in flush
> > self.stream.flush()
> >
> > [2018-07-23 16:31:53,892] {logging_mixin.py:84} WARNING -
> BrokenPipeError:
> > [Errno 32] Broken pipe
> >
> >
> >
> > Ram.
> >
>
