Hi Eila, Python textio currently does not support dynamic writes. So, to achieve this, you'll have to implement a custom DoFn that writes data to dynamic destinations.
Thanks, Cham On Mon, Mar 12, 2018 at 6:26 AM OrielResearch Eila Arich-Landkof < e...@orielresearch.org> wrote: > Hello all, > > I would like to print each PCollection element to a different folder on > gs://.../ > the folder name is not available in advance, but it is available on the > pCollection data (or can be calculated prior to printing). > My question are: > - Is it possible to use the folder path as a parameter to the > beam.io.WriteToText? > - Can you refer me to a code example where dynamic output (or similar) is > being used in Python? > Thanks, > Eila > > -- > Eila > www.orielresearch.org > https://www.meetup.com/Deep-Learning-In-Production/ >
