Hi bluejoe,
Assuming you know that you have a very small PCollection, the way you can
do this is by reading it as a side input. See
https://beam.apache.org/documentation/programming-guide/#transforms-sideio
Here's a snippet as a teaser to read the docs I link to:
PCollection<Whatever> mySmallCollection = ...
PCollectionView<List<Whatever>> mySideInput =
mySmallCollection.apply(View.asList());
someOtherCollection.apply(ParDo.of(...).withSideInputs(mySideInput);
This won't work if your collection is actually large, where "large" means
too big for memory. And it could be slow depending on your runner and
access pattern, even if you have a medium-sized PCollection, aka fits in
memory but still a lot to read without parallelism.
Hope that helps,
Kenn
On Mon, Jul 10, 2017 at 8:35 PM, bluejoe <[email protected]> wrote:
> Hi,
>
> Can anybody tell me how to convert a PCollection to an Array in local
> memory?
> I noticed there is a Create.of() which converts a local list into a
> PCollection
> But how to do the conversion in an inverse direction?
>
> Best regards,
> bluejoe
>
>
