Do both PCollections have the same windowing type? On Sun, Sep 17, 2017 at 5:16 PM, Richard Tian <rt...@snap.com.invalid> wrote:
> Hi > > In our streaming pipeline, we have two pubsub sources. > PCollection<OurType> A and PCollection<OurType> B is created from the two > pubsub sources respectively. > > In this case, both A and B is unbounded PCollection. Can we do something > like: > PCollection<OurType> C = PCollectionList.of(A).and(B).apply(Flatten. > pCollections()) ? > > When we try to flatten them into one PCollection like above, basically the > pipeline stopped working by repeatedly throwing message "GetData failed: > status: APPLICATION_ERROR(3): Computation S6does not have state family S4 > for value read". Anyone know what this error msg mean? > > Alternatively, if we transform A and B into KV<OurType, Integer> and > flatten them into C and do a GroupByKey with a 1 second window then > transform it back to PCollection<OurType>, the pipeline seems working > correctly. > > -- > Richard Tian > > Software Engineer > Snap Inc. >
