Re: Good jinja templating in airflow examples

2017-01-19 Thread Boris Tyukin
Max, this is awesome! Here is the working example if someone needs it - you almost typed it right :) def foo(templates_dict, *args, **kwargs): return templates_dict['ds'] # All of the values in the `templates_dict` should get templated by the Airflow engine, # meaning that in this context,

Re: Good jinja templating in airflow examples

2017-01-19 Thread Maxime Beauchemin
I didn't test the code bellow, but hopefully you get the idea: -- def foo(templates_dict, *args, **kwargs): print(templates_dict['ds']) op = PythonOperator('a_task_id', python_callable=foo, templates_dict={'ds': '{{ ds }}'}) # All of the values in the `templates_dict` shou

Re: Good jinja templating in airflow examples

2017-01-19 Thread Boris Tyukin
Maxime, I have a related question. Can you explain how template_dict work with PythonOperator? Documentation is very vague about it. Example would be nice. I could not find a way to use jinja template with Python operator i.e. passing templated parameters to a pythonoperator callable. Someone told

Re: Good jinja templating in airflow examples

2017-01-18 Thread Maxime Beauchemin
Here's the list of variables and macros exposed in the jinja context: https://airflow.apache.org/code.html#macros Those are all exposed by the framework, meaning they are usable for any templated field without doing anything special. There are ways to pass your own variables and methods using the

Re: Good jinja templating in airflow examples

2017-01-18 Thread Boris Tyukin
Hi Guilherme, I guess it depends what exactly you want to do as not everything works with jinja. >From documentation: https://pythonhosted.org/airflow/concepts.html#jinja-templating You can use Jinja templating with every parameter that is marked as “templated” in the documentation. You can op

Good jinja templating in airflow examples

2017-01-18 Thread Guilherme Marthe
Hey folks! I am trying to write a dag that works well with jinja templating, since through my study of the documentation, is the best way to ensure compatibility with the back-fill function. Are you guys aware of any examples online with this functionality working? The, documentation is still a b