It would seem that I have hit SPARK-10787, an OOME during ClosureCleaner#ensureSerializable(). I am trying to run LSH over a SparseVector consisting of ~4M features with no more than 3K non-zero values per vector. I am hitting this OOME before even the hashes are calculated.
I know the issue is reported and open. My question is whether there have since been any decent workarounds found or if anyone has any suggestions on how I can debug whether there are specific instances of data that are causing my issue that I might filter out prior to hitting this point. I have set spark.serializer.objectStreamReset to 1 and it did not help. I am rather new to Spark and I'm having a tough time following exactly where in the execution of the job this is occurring and whether there is anything within my control to avoid it. Any insights would be greatly appreciated. Thanks, - J.
