Hi,
We've had some performance issues since switching to 1.1.0, and we finally found
the origin : TorrentBroadcast seems to be very slow in our setting (and it
became default with 1.1.0)
The logs of a 4MB variable with TB : (15s)
14/10/01 15:47:13 INFO storage.MemoryStore: Block broadcast_84_piece1 stored as
bytes in memory (estimated size 171.6 KB, free 7.2 GB)
14/10/01 15:47:13 INFO storage.BlockManagerMaster: Updated info of block
broadcast_84_piece1
14/10/01 15:47:23 INFO storage.MemoryStore: ensureFreeSpace(4194304) called with
curMem=1401611984, maxMem=9168696115
14/10/01 15:47:23 INFO storage.MemoryStore: Block broadcast_84_piece0 stored as
bytes in memory (estimated size 4.0 MB, free 7.2 GB)
14/10/01 15:47:23 INFO storage.BlockManagerMaster: Updated info of block
broadcast_84_piece0
14/10/01 15:47:23 INFO broadcast.TorrentBroadcast: Reading broadcast variable 84
took 15.202260006 s
14/10/01 15:47:23 INFO storage.MemoryStore: ensureFreeSpace(4371392) called with
curMem=1405806288, maxMem=9168696115
14/10/01 15:47:23 INFO storage.MemoryStore: Block broadcast_84 stored as values
in memory (estimated size 4.2 MB, free 7.2 GB)
And with HttpBroadcast (0.3s):
14/10/01 16:05:58 INFO broadcast.HttpBroadcast: Started reading broadcast
variable 147
14/10/01 16:05:58 INFO storage.MemoryStore: ensureFreeSpace(4369376) called with
curMem=1373493232, maxMem=9168696115
14/10/01 16:05:58 INFO storage.MemoryStore: Block broadcast_147 stored as values
in memory (estimated size 4.2 MB, free 7.3 GB)
14/10/01 16:05:58 INFO broadcast.HttpBroadcast: Reading broadcast variable 147
took 0.320907112 s 14/10/01 16:05:58 INFO storage.BlockManager: Found block
broadcast_147 locally
Since Torrent is supposed to perform much better than Http, we suspect a
configuration error from our side, but are unable to pin it down. Does someone
have any idea of the origin of the problem ?
For now we're sticking with the HttpBroadcast workaround.
Guillaume
--
eXenSa
*Guillaume PITEL, Président*
+33(0)626 222 431
eXenSa S.A.S. <http://www.exensa.com/>
41, rue Périer - 92120 Montrouge - FRANCE
Tel +33(0)184 163 677 / Fax +33(0)972 283 705
