Github user maropu commented on the issue:
https://github.com/apache/spark/pull/19266
I though, if this limit highly depends on JVM implementations, better to
put the limit as a global variable somewhere (e.g., `ARRAY_INT_MAX` in
`spark.util.Utils` or other places)? As another option, how about adding
internal option in `SparkConf` for that?
Also, how about making a parent jira tcket to track the similar issue cuz
it seems difficult to cover all the possible place in this pr.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
