Github user squito commented on the issue:
https://github.com/apache/spark/pull/21527
I tell users 2001 partitions all the time. I haven't bothered trying to
make it configurable in the past because I dunno what else you'd set this value
to -- normally I think it would be better for them to bump up the number of
partitions anyway. Probably there are some cases where you'd rather change the
conf rather than change the number of partitions?
agree with Mark, would be nice for the jira & pr description to explain why
this is useful.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
