If I am incorrect, how would we know whether to use the defaultParallelism or some other value? I don't think it would be appropriate to force a SourceRDD, that may have had hundreds of partitions, into the defaultParallelism number of partitions, which may be quite small, as this may result in too much data being in each partition.
[ Full content available at: https://github.com/apache/beam/pull/6181 ] This message was relayed via gitbox.apache.org for [email protected]
