Try changing:
{% for key, value in task_instance.xcom_pull(task_
ids='data_validation_tests', key='tests_results') %}
to
{% for key, value in task_instance.xcom_pull(task_
ids='data_validation_tests', key='tests_results').items() %}
Maybe?
Max
On Thu, Jun 16, 2016 at 11:51 AM, Jeremiah Lowin <jlo...@apache.org> wrote:
> Or, I'm afraid my jinja-fu is sorely lacking and I'm not going to be much
> help to you here... perhaps Maxime or one of the other devs is more
> experienced with templates and can offer some advice?
>
> Apologies!
> Jeremiah
>
> On Thu, Jun 16, 2016 at 2:40 AM Or Sher <or.sh...@gmail.com> wrote:
>
>> Thanks Jeremiah!
>>
>> Looking at the code it seems like get_many can return only many xcom
>> values
>> from different task_ids and not different keys from the same task_id.
>> But it doesn't really matter as I would really like to do the iteration
>> somehow inside the template.
>> I feel like building the template inside the previous python operator is
>> kind of missing the whole point of templates as I can already create the
>> whole "already rendered" email message at the same time.
>>
>> I tried pushing one big json xcom to reflect all of the tests, but now
>> I'm struggling
>> with iterating string json inside the template.
>> The following template fails on the "for" line:
>> VALIDATION_EMAIL_CONTENT = """
>> <b>Date:</b> {{ ds }}<br>
>> <p>
>> <table>
>> {% for key, value in
>> task_instance.xcom_pull(task_ids='data_validation_tests',
>> key='tests_results') %}
>> {{ key|string() }}
>> {{ value|string() }}
>> {% endfor %}
>> ..
>> ..
>>
>> File "<template>", line 5, in top-level template code
>> ValueError: too many values to unpack
>>
>> I'm guessing it's because the xcom_pull returns a string but I didn't
>> managed to cast it to a json yet..
>>
>> On Wed, Jun 15, 2016 at 5:33 PM Jeremiah Lowin <jlo...@apache.org> wrote:
>>
>> > Hi Or,
>> >
>> > There is support for pulling multiple XComs at once (see
>> XCom.get_many())
>> > but it won't be directly available inside a jinja template, where you
>> you
>> > are typically calling ti.xcom_pull() and returning a single XCom at a
>> time,
>> > using a key you know in advance.
>> >
>> > I would suggest that you do this processing either inside your existing
>> > PythonOperator or in a subsequent PythonOperator that calls
>> XCom.get_many()
>> > and produces the final template for your email operator. It is
>> > theoretically possible for you to do this in your email template using
>> > template macros, but I only have a basic knowledge of that (and it's a
>> > convoluted process) so I don't want to give you bad advice. If you can
>> do
>> > your preprocessing in your PythonOperator I think you will be happier,
>> > especially because the result will be visible in the Airflow UI and
>> logs.
>> >
>> > Best,
>> > Jeremiah
>> >
>> > On Wed, Jun 15, 2016 at 9:49 AM Or Sher <or.sh...@gmail.com> wrote:
>> >
>> > > Hi all,
>> > >
>> > > For the purpose of data validation tests I have two adjacent
>> operators:
>> > > Python - Parsing predefined json of tests and saves the result as an
>> xcom
>> > > where each key is a test id.
>> > >
>> > > Email - I'd like to be able to iterate over all keys from the previous
>> > task
>> > > inside the predefined email template so I won't need to change it
>> every
>> > > time I change the tests json.
>> > >
>> > > It seems like it's not supported. Is that right?
>> > > What are my choices?
>> > > If I won't find anything else, I assume I can always use one key for
>> all
>> > of
>> > > the tests combined into one json.
>> > >
>> >
>>
>
