The DoFn is associated with a PTransform and in the pipeline proto there is a unique id associated with each PTransform. You can use that to generate a composite key (ptransformid, stateid) which will be unique within the pipeline.
On Fri, Aug 21, 2020 at 11:26 AM Ke Wu <[email protected]> wrote: > Thank you Reuven for the confirmation. Do you know what is the recommended > way for underlying runners to distinguish same state id in different > DoFn(s)? > > On Aug 21, 2020, at 10:27 AM, Reuven Lax <[email protected]> wrote: > > StateId is scoped to the DoFn. You can use the same string in different > DoFns for completely different states. > > On Fri, Aug 21, 2020 at 10:21 AM Ke Wu <[email protected]> wrote: > >> Hello everyone, >> >> After reading through Stateful processing with Apache Beam >> <https://beam.apache.org/blog/stateful-processing/> and DoFn.StateId >> <https://beam.apache.org/releases/javadoc/2.23.0/org/apache/beam/sdk/transforms/DoFn.StateId.html>, >> I understand that each state id must be unique and must be the same type at >> least in the same DoFn, however, it does not explicitly mention whether or >> not it is expected and supported that the same state id to be declared in >> different DoFn(s). If Yes, is the state supposed to be a shared state or is >> supposed to completed separate, therefore it could even be different types. >> If No, it seems that the validation in Beam SDK only validates uniqueness >> in the same DoFn. >> >> Thanks, >> Ke >> > >
