HI all
was wondering if someone can advise here as i am puzzled on what is
happening (and perhaps it's my poor understanding of beam)
Here's my usecase - very simplified -:
I have 3 collections which i want to use to populate an email
combined = ({'collection1': keyed_etoro, 'collection2': keyed_finviz,
'collection3' : keyed_llm}
| beam.CoGroupByKey())
send_email(combined, known_args.sendgridkey)
Each collection above contain a single string, as i have used a combine
function to reduce all the elements to one string
keyed_llm is an output of running an LLM (via RunInference)
BEFORE i added the RunInference pipeline, the code worked fine and
assembled the email out of the two collections
Now that i have added LLM, what is happening is that 2 mails are being sent
1 - one containing 'collection1' and 'collection2'
2 - one containing just 'collection3'
I am missing something big time...
Apologies for the poor details
Kind regards
Marco
I have the following snippet from my pipeline
