hantiantian created SPARK-26726:
-----------------------------------
Summary: The amount of memory used by the broadcast variable is
not synchronized to the UI display
Key: SPARK-26726
URL: https://issues.apache.org/jira/browse/SPARK-26726
Project: Spark
Issue Type: Bug
Components: Spark Core
Affects Versions: 2.4.0
Reporter: hantiantian
The amount of memory used by the broadcast variable is not synchronized to the
UI display，
spark-sql> select /*+ broadcast(a)*/ a.id,b.id from a join b on a.id = b.id;
View the app's driver log：
2019-01-25 16:45:23,726 INFO org.apache.spark.storage.BlockManagerInfo: Added
broadcast_4_piece0 in memory on 10.43.xx.xx:33907 (size: 6.6 KB, free: 2.5 GB)
2019-01-25 16:45:23,727 INFO org.apache.spark.storage.BlockManagerInfo: Added
broadcast_4_piece0 in memory on 10.43.xx.xx:38399 (size: 6.6 KB, free: 2.5 GB)
2019-01-25 16:45:23,745 INFO org.apache.spark.storage.BlockManagerInfo: Added
broadcast_3_piece0 in memory on 10.43.xx.xx:33907 (size: 32.1 KB, free: 2.5 GB)
2019-01-25 16:45:23,749 INFO org.apache.spark.storage.BlockManagerInfo: Added
broadcast_3_piece0 in memory on 10.43.xx.xx:38399 (size: 32.1 KB, free: 2.5 GB)
2019-01-25 16:45:23,838 INFO org.apache.spark.storage.BlockManagerInfo: Added
broadcast_2_piece0 in memory on 10.43.xx.xx:38399 (size: 147.0 B, free: 2.5 GB)
2019-01-25 16:45:23,840 INFO org.apache.spark.storage.BlockManagerInfo: Added
broadcast_2_piece0 in memory on 10.43.xx.xx:33907 (size: 147.0 B, free: 2.5 GB)
Web UI does not have the use of memory，
!image-2019-01-25-16-48-28-435.png!
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
