robertwb commented on pull request #13069: URL: https://github.com/apache/beam/pull/13069#issuecomment-709463485
I don't think the runner can know, given a ShardedKey, that the shard id was one that it chose (and, related, this would impose hidden restrictions on what the SDK could choose here). However, we're not making this a magic type, rather we're making GroupIntoBatches a magic transform (which seems a more sane direction). So I'm OK with making it required, with no special meanings for empty (or other) bytes. The other question I'd like to bring up is the visibility of the ShardID. If it's not opaque, is there any reason to not just use a KV here? ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected]
