> > @Reuven Lax <[email protected]> yes I am aware of that transform, but > that’s different from the explode operation I was referring to: > https://spark.apache.org/docs/latest/api/sql/index.html#explode >
How is it different? It'd help if you could provide the signature (input and output PCollection types) of the transform you have in mind. On Tue, Jan 12, 2021 at 4:49 PM Tao Li <[email protected]> wrote: > @Reuven Lax <[email protected]> yes I am aware of that transform, but > that’s different from the explode operation I was referring to: > https://spark.apache.org/docs/latest/api/sql/index.html#explode > > > > *From: *Reuven Lax <[email protected]> > *Reply-To: *"[email protected]" <[email protected]> > *Date: *Tuesday, January 12, 2021 at 2:04 PM > *To: *user <[email protected]> > *Subject: *Re: Is there an array explode function/transform? > > > > Have you tried Flatten.iterables > > > > On Tue, Jan 12, 2021, 2:02 PM Tao Li <[email protected]> wrote: > > Hi community, > > > > Is there a beam function to explode an array (similarly to spark sql’s > explode())? I did some research but did not find anything. > > > > BTW I think we can potentially use FlatMap to implement the explode > functionality, but a Beam provided function would be very handy. > > > > Thanks a lot! > >
