RE: Lazily load input for Airflow operators

2018-09-29 Thread Victor Jimenez
` and do the magic. Or how about a `pre_execute(context): pass` BaseOperator method as a nice hook to allow for this kind of stuff without having to call `super`. Max On Mon, Aug 27, 2018 at 2:29 PM Victor Jimenez mailto:vjime...@vistaprint.com>> wrote: > TL;DR Is there any reco

Re: Lazily load input for Airflow operators

2018-08-27 Thread Maxime Beauchemin
ok to allow for this kind of stuff without having to call `super`. Max On Mon, Aug 27, 2018 at 2:29 PM Victor Jimenez wrote: > TL;DR Is there any recommended way to lazily load input for Airflow > operators? > > > I could not found a way to do this. While I faced this limitatio

Lazily load input for Airflow operators

2018-08-27 Thread Victor Jimenez
TL;DR Is there any recommended way to lazily load input for Airflow operators? I could not found a way to do this. While I faced this limitation while using the Databricks operator, it seems other operators might potentially lack such a functionality. Please, keep reading for more details