There is chance that the log message may change in future releases.
Log snooping would be broken.
FYI
On Mon, Feb 1, 2016 at 9:55 PM, Takeshi Yamamuro
wrote:
> Hi,
>
> Currently, there is no way to check the size except for snooping INFO-logs
> in a driver;
>
> 16/02/02 14:51:53 INFO BlockMana
Hi,
Currently, there is no way to check the size except for snooping INFO-logs
in a driver;
16/02/02 14:51:53 INFO BlockManagerInfo: Added rdd_2_12 in memory on
localhost:58536 (size: 40.0 B, free: 510.7 MB)
On Tue, Feb 2, 2016 at 8:20 AM, apu mishra . rr
wrote:
> How can I determine the siz
How can I determine the size (in bytes) of a broadcast variable? Do I need
to use the .dump method and then look at the size of the result, or is
there an easier way?
Using PySpark with Spark 1.6.
Thanks!
Apu
