egalpin commented on PR #31608: URL: https://github.com/apache/beam/pull/31608#issuecomment-2215408771
> So in this case you don't care exactly what the partitions are, as long as they are created? I would want to be able to partition based on some attribute of the data (ex. user_id) such that data for user_id `n` always ended up in partition `x`. I believe that should be fully doable with support for ordering key here; I've done that before with great effort haha (enabling dataflow experiments, building local versions of pubsubIO, etc). And similar to what @sjvanrossum mentioned, I would deal with out-of-order data using application layer logic within consumers and using data within the message payloads to decide desired order. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@beam.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
