Ok, I figured out why this is happening. settings.py:35 is hard coding the
logging level to INFO. I hacked it locally to:
LOGGING_LEVEL = conf.get('core', 'logging_level')
and now all the INFO logs are gone. Is this fixed in a later version of
airflow?
On Tue, Apr 23, 2019 at 1:09 PM Frank Maritato <[email protected]>
wrote:
> Those two issues don't seem to be present in 1.10.0. I only see one
> message for "db connection invalidated" and I don't see anything about
> harvesting.
>
> I have tried using a custom logger to see if that effects the output to
> airflow-scheduler.{log|out} but so far I haven't had any luck. The logs
> that go to /var/log/airflow/scheduler are affected, but
> AIRFLOW_HOME/airflow-scheduler.log always is printing out INFO or below.
>
> Is there any documentation on how to do this? Or examples somewhere?
>
> On Tue, Apr 23, 2019 at 11:39 AM Daniel Standish <[email protected]>
> wrote:
>
>> I noticed two issues in 1.10.2, one of which has been resolved, but I am
>> not sure if they were present in 1.10.0, or if they are the messages that
>> are bothering you.
>>
>> 1. "Harvesting DAG parsing results" was printed every 2 seconds or
>> something.
>> This was resolved in commit [AIRFLOW-3911] Change Harvesting DAG parsing
>> results to DEBUG log level (#4729) which I believe is in 1.10.3 now.
>> It just changes log level to debug for that message.
>>
>> 2. Frequent "db connection invalidated" warning
>> This one is unsolved. I created a ticket here:
>> https://issues.apache.org/jira/browse/AIRFLOW-4134
>> It seems like every 5 seconds the warning "db connection invalidated" is
>> logged.
>> It happens in a connection reconnect try loop. It always seems to be able
>> to reconnect on first try, so one idea is we could just set it to only
>> warn
>> if the first retry fails (debug on first reconnect). But it would be more
>> satisfying to figure out why this connection always seems to be
>> invalidated
>> and fix the root cause. Alas I am not sure how to proceed...
>>
>>
>>
>> On Tue, Apr 23, 2019 at 9:54 AM Bolke de Bruin <[email protected]> wrote:
>>
>> > It's probably better to create a custom logging.conf and use that
>> instead.
>> >
>> > B.
>> >
>> > Op di 23 apr. 2019 18:13 schreef Frank Maritato
>> > <[email protected]>:
>> >
>> > > No one else has this issue? Or no one has a solution?
>> > >
>> > >
>> > > On Wed, Apr 17, 2019 at 5:49 PM Frank Maritato <
>> [email protected]>
>> > > wrote:
>> > >
>> > > > Hi All,
>> > > >
>> > > > We are running airflow 1.10.0 and I'm wondering how I can turn off
>> or
>> > > turn
>> > > > down the logging for the scheduler? I tried setting
>> logging_level=WARN
>> > in
>> > > > airflow.cfg and restarting the process but I'm still seeing a ton of
>> > info
>> > > > logging to .out and .log.
>> > > >
>> > > > Thanks!
>> > > > --
>> > > > Frank Maritato
>> > > >
>> > >
>> > >
>> > > --
>> > > Frank Maritato
>> > >
>> >
>>
>
>
> --
> Frank Maritato
>
--
Frank Maritato
