[ https://issues.apache.org/jira/browse/SPARK-18548?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Sean Owen resolved SPARK-18548. ------------------------------- Resolution: Duplicate > OnlineLDAOptimizer reads the same broadcast data after deletion > --------------------------------------------------------------- > > Key: SPARK-18548 > URL: https://issues.apache.org/jira/browse/SPARK-18548 > Project: Spark > Issue Type: Improvement > Components: MLlib > Affects Versions: 1.6.1 > Reporter: Xiaoye Sun > Priority: Trivial > Original Estimate: 1h > Remaining Estimate: 1h > > In submitMiniBatch() called by OnlineLDAOptimizer, broadcast variable > expElogbeta is deleted before its use in the second time, which causes the > executor reads the same large broadcast data twice. I suggest to move the > broadcast data deletion (expElogbetaBc.unpersist()) later. -- This message was sent by Atlassian JIRA (v6.3.4#6332) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org