+1, Jeremy I think let's move forward with the change and revisit later if we find that the performance degrades too drastically for IPv6 datasets.
Thank you, Tadd Wood On Wed, May 6, 2020 at 12:10 PM Jeremy Nelson <[email protected]> wrote: > Greetings, > > SPOT-286 points out that in > scala-ml/src/main/scala/org/apache/spot/*/model/*.scala, there are > calls to org.apache.spark.sql.functions.broadcast(), which is a > performance optimization hint to the scala engine. The ticket > identified at least one scenario where this hint causes a crash. > > There is no harm to correctness from removing this optimization, and > it fixes the crash. I recommend we remove this optimization until the > matter is re-assessed later. > > Jeremy >
