Re: [YAML] add timestamp to a bounded PCollection

2024-01-09 Thread Robert Bradshaw via dev
Just created https://github.com/apache/beam/pull/29969 On Mon, Jan 8, 2024 at 2:49 PM Robert Bradshaw wrote: > > This does appear to be a significant missing feature. I'll try to make > sure something easier gets in by the next release. See also below. > > On Mon, Jan 8, 2024 at 11:30 AM Ferran

Re: [YAML] add timestamp to a bounded PCollection

2024-01-08 Thread Robert Bradshaw via dev
This does appear to be a significant missing feature. I'll try to make sure something easier gets in by the next release. See also below. On Mon, Jan 8, 2024 at 11:30 AM Ferran Fernández Garrido wrote: > > Hi Yarden, > > Since it's a bounded source you could try with Sql transformation >

Re: [YAML] add timestamp to a bounded PCollection

2024-01-08 Thread Ferran Fernández Garrido
Hi Yarden, Since it's a bounded source you could try with Sql transformation grouping by the timestamp column. Here are some examples of grouping: https://github.com/apache/beam/tree/master/sdks/python/apache_beam/yaml However, if you want to add a timestamp column in addition to the original

[YAML] add timestamp to a bounded PCollection

2024-01-08 Thread Yarden BenMoshe
Hi all, Im quite new to using beam yaml. I am working with a CSV file and want to implement some windowing logic to it. Was wondering what is the right way to add timestamps to each element, assuming I have a column including a timestamp. I am aware of Beam Programming Guide (apache.org)