Hi Lauren, Thanks for the feedbacks. How do you identify this REPL memory issue? If it is a problem of Scala REPL, then I think we don't have pretty much solutions for it. Do you have any thought on it?
BTW, which version of Spark are you using? @Jeff, if it is a problem of Scala REPL, then I believe Zeppelin also suffers from such issue, did you see any report from Zeppelin community? Thanks Jerry 2017-12-01 11:17 GMT+08:00 Lauren Spiegel <[email protected]>: > The spark driver. I think it is related to this since a session is a spark > shell/scala REPL: https://issues.scala-lang.org/browse/SI-4331 > > > On Thu, Nov 30, 2017 at 6:11 PM, Jeff Zhang <[email protected]> wrote: > >> >> Which component's memory keeps growing ? livy server or the spark driver ? >> >> >> Lauren Spiegel <[email protected]>于2017年12月1日周五 上午8:55写道： >> >>> Hi Livyers, >>> >>> When I have an active session for a couple hours, the memory keeps >>> growing and I have to kill it. What do people do to manage ever-growing >>> memory? Are there any plans to limit the memory growth so that sessions can >>> be long living? >>> >>> Thank you >>> >>> >>> <http://storage.scality.com/report-gartner-magic-quadrant-storage.html?utm_source=ScalityEmail&utm_medium=Email&utm_campaign=MagicQuadrant> >> >> > > > <http://storage.scality.com/report-gartner-magic-quadrant-storage.html?utm_source=ScalityEmail&utm_medium=Email&utm_campaign=MagicQuadrant>
