That’s the max size of a byte array in Java, limited by the length which is
defined as integer, and in most JVMS arrays can’t hold more than
Int.MaxValue - 8 elements. Other way to overcome this is to create multiple
broadcast variables
On Sunday, August 5, 2018, klrmowse wrote:
> i don't need
i don't need more, per se... i just need to watch the size of the variable;
then, if it's within the size limit, go ahead and broadcast it; if not, then
i won't broadcast...
so, that would be a yes then? (2 GB, or which is it exactly?)
--
Sent from:
I think if you need more then you should anyway think about something different
than broadcast variable ...
> On 5. Aug 2018, at 16:51, klrmowse wrote:
>
> is it currently still ~2GB (Integer.MAX_VALUE) ??
>
> or am i misinformed, since that's what google-search and scouring this
> mailing
is it currently still ~2GB (Integer.MAX_VALUE) ??
or am i misinformed, since that's what google-search and scouring this
mailing list seem to say... ?
Thanks
--
Sent from: http://apache-spark-user-list.1001560.n3.nabble.com/
