Connections are configured differently: http://airflow.readthedocs.io/en/latest/concepts.html#connections
> When referencing the connection in the Airflow pipeline, the conn_id should > be the name of the variable without the prefix. For example, if the `conn_id` > is named `postgres_master` the environment variable should be named > `AIRFLOW_CONN_POSTGRES_MASTER` (note that the environment variable must be > all uppercase). Airflow assumes the value returned from the environment > variable to be in a URI format (e.g. > `postgres://user:password@localhost:5432/master or > s3://accesskey:secretkey@S3`). If your sql alchemy conn isn't being picked up by airflow it is possible that /etc/environment isn't affecting running services, just interactive logins/shells. If you are using systemd add EnvironmentFile=/etc/environment (I think that's the right stanza) to make it load that too -ash > On 9 Nov 2017, at 08:35, Somasundaram Sekar > <[email protected]> wrote: > > Below are the env variables configured > > AIRFLOW__CORE__SQL_ALCHEMY_CONN=mysql://XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX > AIRFLOW__CELERY__BROKER_URL=redis://XXXXXXXXXXXXXXXXXXXX/1 > AIRFLOW__CELERY__CELERY_RESULT_BACKEND=db+mysql://XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXx > AIRFLOW__WEBSERVER__AUTHENTICATE=True > AIRFLOW__WEBSERVER__AUTH_BACKEND=airflow.contrib.auth.backends.google_auth > AIRFLOW__GOOGLE__CLIENT_ID=XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXx > AIRFLOW__GOOGLE__CLIENT_SECRET=XXXXXXXXXXXXXXXXXXxx > AIRFLOW__GOOGLE__OAUTH_CALLBACK_ROUTE=/ > AIRFLOW__GOOGLE__DOMAIN=XXXXXXXXXXXXXXXXX > > -----Original Message----- > From: Gerard Toonstra [mailto:[email protected]] > Sent: Thursday, November 9, 2017 1:59 PM > To: [email protected] > Subject: Re: Airflow configuration in environment variable not working > > What's the variable key you are using. Does it follow this convention? > > https://airflow.apache.org/configuration.html > > That's AIRFLOW (two underscores) configuration section (two underscores) > env var. > > > G> > > > On Thu, Nov 9, 2017 at 8:30 AM, Somasundaram Sekar < > [email protected]> wrote: > >> I tried using ENV variable to configure connection urls, I have an ami >> that is preconfigured with alchemy_conn and broker_url etc, I have >> written environment variables to /etc/environment in the instances >> being spun up from the amis to override the properties in airflow.cfg >> file. I was able to access the variables from python code as well. >> >> But it doesn't seem to take effect while running ariflow, tried >> restarting the process as well but didn't work, it still points to the >> one in airflow.cfg file >> >> Any help appreciated. >> >> Regards, >> >> Somasundaram S >>
