@Frank - It is still the case. I will fix this in master and should be fixed in 1.10.4
On Tue, Apr 23, 2019 at 9:38 PM Frank Maritato <[email protected]> wrote: > sorry, it's line 93, not 35. > > On Tue, Apr 23, 2019 at 1:34 PM Frank Maritato <[email protected]> > wrote: > > > Ok, I figured out why this is happening. settings.py:35 is hard coding > the > > logging level to INFO. I hacked it locally to: > > > > LOGGING_LEVEL = conf.get('core', 'logging_level') > > > > and now all the INFO logs are gone. Is this fixed in a later version of > > airflow? > > > > > > On Tue, Apr 23, 2019 at 1:09 PM Frank Maritato <[email protected]> > > wrote: > > > >> Those two issues don't seem to be present in 1.10.0. I only see one > >> message for "db connection invalidated" and I don't see anything about > >> harvesting. > >> > >> I have tried using a custom logger to see if that effects the output to > >> airflow-scheduler.{log|out} but so far I haven't had any luck. The logs > >> that go to /var/log/airflow/scheduler are affected, but > >> AIRFLOW_HOME/airflow-scheduler.log always is printing out INFO or below. > >> > >> Is there any documentation on how to do this? Or examples somewhere? > >> > >> On Tue, Apr 23, 2019 at 11:39 AM Daniel Standish <[email protected]> > >> wrote: > >> > >>> I noticed two issues in 1.10.2, one of which has been resolved, but I > am > >>> not sure if they were present in 1.10.0, or if they are the messages > that > >>> are bothering you. > >>> > >>> 1. "Harvesting DAG parsing results" was printed every 2 seconds or > >>> something. > >>> This was resolved in commit [AIRFLOW-3911] Change Harvesting DAG > parsing > >>> results to DEBUG log level (#4729) which I believe is in 1.10.3 now. > >>> It just changes log level to debug for that message. > >>> > >>> 2. Frequent "db connection invalidated" warning > >>> This one is unsolved. I created a ticket here: > >>> https://issues.apache.org/jira/browse/AIRFLOW-4134 > >>> It seems like every 5 seconds the warning "db connection invalidated" > is > >>> logged. > >>> It happens in a connection reconnect try loop. It always seems to be > >>> able > >>> to reconnect on first try, so one idea is we could just set it to only > >>> warn > >>> if the first retry fails (debug on first reconnect). But it would be > >>> more > >>> satisfying to figure out why this connection always seems to be > >>> invalidated > >>> and fix the root cause. Alas I am not sure how to proceed... > >>> > >>> > >>> > >>> On Tue, Apr 23, 2019 at 9:54 AM Bolke de Bruin <[email protected]> > >>> wrote: > >>> > >>> > It's probably better to create a custom logging.conf and use that > >>> instead. > >>> > > >>> > B. > >>> > > >>> > Op di 23 apr. 2019 18:13 schreef Frank Maritato > >>> > <[email protected]>: > >>> > > >>> > > No one else has this issue? Or no one has a solution? > >>> > > > >>> > > > >>> > > On Wed, Apr 17, 2019 at 5:49 PM Frank Maritato < > >>> [email protected]> > >>> > > wrote: > >>> > > > >>> > > > Hi All, > >>> > > > > >>> > > > We are running airflow 1.10.0 and I'm wondering how I can turn > off > >>> or > >>> > > turn > >>> > > > down the logging for the scheduler? I tried setting > >>> logging_level=WARN > >>> > in > >>> > > > airflow.cfg and restarting the process but I'm still seeing a ton > >>> of > >>> > info > >>> > > > logging to .out and .log. > >>> > > > > >>> > > > Thanks! > >>> > > > -- > >>> > > > Frank Maritato > >>> > > > > >>> > > > >>> > > > >>> > > -- > >>> > > Frank Maritato > >>> > > > >>> > > >>> > >> > >> > >> -- > >> Frank Maritato > >> > > > > > > -- > > Frank Maritato > > > > > -- > Frank Maritato > -- *Kaxil Naik* *Big Data Consultant *@ *Data Reply UK* *Certified *Google Cloud Data Engineer | *Certified* Apache Spark & Neo4j Developer *LinkedIn*: https://www.linkedin.com/in/kaxil
