Hi Abhishek, Workers read connection info from its local environment variables. So if you have multiple Celery executor nodes (in distributed fashion), possibly check whether the same environement variable is exported on all of them? This may explain why the behaviour is random in your environment.
Folks please correct me if I’m wrong. Thanks. XD On Wed, Aug 8, 2018 at 01:04 Abhishek Sinha <[email protected]> wrote: > I am trying to use: MySqlToHiveTransfer Operator. > > > From the base_hook code, I see that there is a way to pass connection URL > via environment variables. The variable needs to be pre-fixed with: > AIRFLOW_CONN_ followed by the connection ID. > > I have tried exporting this variable but the behaviour seems to be random. > In some instances of the DAG Run, the variable is present in os.environ but > in some of the runs it is not. Is this an issue? > > We are on version 1.8.2 with Celery Executor. > > > > > Regards, > > Abhishek > >
