Dhiraj created SPARK-23427:

             Summary: spark.sql.autoBroadcastJoinThreshold causing OOM  in the 
                 Key: SPARK-23427
                 URL: https://issues.apache.org/jira/browse/SPARK-23427
             Project: Spark
          Issue Type: Bug
          Components: SQL
    Affects Versions: 2.0.0
         Environment: SPARK 2.0 version
            Reporter: Dhiraj

We are facing issue around value of spark.sql.autoBroadcastJoinThreshold.

With spark.sql.autoBroadcastJoinThreshold -1 ( disable) we seeing driver memory 
used flat.

With any other values 10MB, 5MB, 2 MB, 1MB, 10K, 1K we see driver memory used 
goes up with rate depending upon the size of the autoBroadcastThreshold and 
getting OOM exception. The problem is memory used by autoBroadcast is not being 
free up in the driver.


This message was sent by Atlassian JIRA

To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org

Reply via email to