Github user vincent-grosbois commented on the issue:
https://github.com/apache/spark/pull/22024
Actually I spoke too soon!
We spotted some regression in TorrentBroadcast:
https://github.com/apache/spark/blob/46110a589f4e91cd7605c5a2c34c3db6b2635830/core/src/main/scala/org/apache/spark/broadcast/TorrentBroadcast.scala#L164
it seems that the code in charge of checking the CheckSum is assuming that
we only receive 1 block in the chunkedByteBuffer (cf checksum of only
chunk(0)). Does anyone know why ?
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
