You seem to mix master and 1-9-stable. The RedirectStdHandler is only available in master.
B. Verstuurd vanaf mijn iPad > Op 19 dec. 2017 om 23:37 heeft Kevin Lam <ke...@fathomhealth.co> het volgende > geschreven: > > Hi Feng, > > Thanks for your help! Got it, will try to push on the python based logging > config. > > I'm trying to set-up the GCS logging on airflow v1-9-stable and my > logging_config.py seems to be causing a python import error, caused by > 'from airflow import configuration' > > "Initialize database... > Unable to load the config, contains a configuration error. > Traceback (most recent call last): > File "/usr/lib/python3.5/logging/config.py", line 384, in resolve > self.importer(used) > ImportError: No module named > 'airflow.utils.log.logging_mixin.RedirectStdHandler'; > 'airflow.utils.log.logging_mixin' is not a package > > The above exception was the direct cause of the following exception: > > Traceback (most recent call last): > File "/usr/lib/python3.5/logging/config.py", line 558, in configure > handler = self.configure_handler(handlers[name]) > File "/usr/lib/python3.5/logging/config.py", line 708, in > configure_handler > klass = self.resolve(cname) > File "/usr/lib/python3.5/logging/config.py", line 391, in resolve > raise v > File "/usr/lib/python3.5/logging/config.py", line 384, in resolve > self.importer(used) > ValueError: Cannot resolve > 'airflow.utils.log.logging_mixin.RedirectStdHandler': No module named > 'airflow.utils.log.logging_mixin.RedirectStdHandler'; > 'airflow.utils.log.logging_mixin' is not a package > > During handling of the above exception, another exception occurred: > > Traceback (most recent call last): > File "/usr/local/bin/airflow", line 16, in <module> > from airflow import configuration > File "/usr/local/lib/python3.5/dist-packages/airflow/__init__.py", line > 31, in <module> > from airflow import settings > File "/usr/local/lib/python3.5/dist-packages/airflow/settings.py", line > 148, in <module> > configure_logging() > File "/usr/local/lib/python3.5/dist-packages/airflow/logging_config.py", > line 75, in configure_logging > raise e > File "/usr/local/lib/python3.5/dist-packages/airflow/logging_config.py", > line 70, in configure_logging > dictConfig(logging_config) > File "/usr/lib/python3.5/logging/config.py", line 795, in dictConfig > dictConfigClass(config).configure() > File "/usr/lib/python3.5/logging/config.py", line 566, in configure > '%r: %s' % (name, e)) > ValueError: Unable to configure handler 'console': Cannot resolve > 'airflow.utils.log.logging_mixin.RedirectStdHandler': No module named > 'airflow.utils.log.logging_mixin.RedirectStdHandler'; > 'airflow.utils.log.logging_mixin' is not a package > HTTP/1.1 200 OK > Unable to load the config, contains a configuration error. > Traceback (most recent call last): > File "/usr/lib/python3.5/logging/config.py", line 384, in resolve > self.importer(used) > ImportError: No module named > 'airflow.utils.log.logging_mixin.RedirectStdHandler'; > 'airflow.utils.log.logging_mixin' is not a package > > The above exception was the direct cause of the following exception: > > Traceback (most recent call last): > File "/usr/lib/python3.5/logging/config.py", line 558, in configure > handler = self.configure_handler(handlers[name]) > File "/usr/lib/python3.5/logging/config.py", line 708, in > configure_handler > klass = self.resolve(cname) > File "/usr/lib/python3.5/logging/config.py", line 391, in resolve > raise v > File "/usr/lib/python3.5/logging/config.py", line 384, in resolve > self.importer(used) > ValueError: Cannot resolve > 'airflow.utils.log.logging_mixin.RedirectStdHandler': No module named > 'airflow.utils.log.logging_mixin.RedirectStdHandler'; > 'airflow.utils.log.logging_mixin' is not a package > > During handling of the above exception, another exception occurred: > > Traceback (most recent call last): > File "/usr/local/bin/airflow", line 16, in <module> > from airflow import configuration > File "/usr/local/lib/python3.5/dist-packages/airflow/__init__.py", line > 31, in <module> > from airflow import settings > File "/usr/local/lib/python3.5/dist-packages/airflow/settings.py", line > 148, in <module> > configure_logging() > File "/usr/local/lib/python3.5/dist-packages/airflow/logging_config.py", > line 75, in configure_logging > raise e > File "/usr/local/lib/python3.5/dist-packages/airflow/logging_config.py", > line 70, in configure_logging > dictConfig(logging_config) > File "/usr/lib/python3.5/logging/config.py", line 795, in dictConfig > dictConfigClass(config).configure() > File "/usr/lib/python3.5/logging/config.py", line 566, in configure > '%r: %s' % (name, e)) > ValueError: Unable to configure handler 'console': Cannot resolve > 'airflow.utils.log.logging_mixin.RedirectStdHandler': No module named > 'airflow.utils.log.logging_mixin.RedirectStdHandler'; > 'airflow.utils.log.logging_mixin' is not a package" > > Have you encountered this before? > >> On Mon, Dec 18, 2017 at 8:53 PM, Feng Lu <fen...@google.com.invalid> wrote: >> >> Hi Kevin, >> >> Kindly see my reply inline: >> >>> On Mon, Dec 18, 2017 at 3:28 PM, Kevin Lam <ke...@fathomhealth.co> wrote: >>> >>> Hi, >>> >>> I'm trying to get airflow to use GCS for logging purposes and had a few >>> questions. >>> >>> We're currently using Airflow 1.9rc2, running in a Kubernetes Airflow >>> deployment (similar to https://github.com/mumoshu/kube-airflow) >>> >>> 1/ Seems like the logging code has been going through some changes in the >>> recent versions. What's the correct way to set up GCS for logging? Is it >> by >>> just specifying remote_base_log_folder and remote_log_conn_id in >>> airflow.cfg? Or by following this guide: >>> http://airflow.readthedocs.io/en/latest/integration.html#gcp, using the >>> python based logging config? Is there an Airflow version that we should >> use >>> to be most stable? >>> >> The python based logging config is the right place to make changes, in our >> test setup, we override the airflow_local_settings.py similarly to the link >> you pasted. >> You may also want to config: [core]task_log_reader = gcs.task >> >> >>> >>> 2/ Is there a way to encode the connection for GCS in a file so that one >>> doesn't have to open the webserver and create it from the admin panel? >> It'd >>> be nice if the GCS connection would be automatically created. >>> >> Unfortunately GCS connection ties to some GCP project and is impossible to >> pre-populate. >> Airflow1.9 should fix the gcp connection type issue ( >> https://github.com/apache/incubator-airflow/commit/ >> 2f107d8a30910fd025774004d5c4c95407ed55c5), >> so you can use airflow connections CLI directly. >> >> >>> >>> Thanks in advance for your help! >>> >>
