RE: Memory usage for spark types

2016-09-20 Thread assaf.mendelson
anyone point me in the right direction? From: rxin [via Apache Spark Developers List] [mailto:ml-node+s1001551n18985...@n3.nabble.com] Sent: Monday, September 19, 2016 12:23 AM To: Mendelson, Assaf Subject: Re: Memory usage for spark types Take a look at UnsafeArrayData and UnsafeMapData. On Sun

Re: Memory usage for spark types

2016-09-18 Thread Reynold Xin
Take a look at UnsafeArrayData and UnsafeMapData. On Sun, Sep 18, 2016 at 9:06 AM, assaf.mendelson wrote: > Hi, > > I am trying to understand how spark types are kept in memory and accessed. > > I tried to look at the code at the definition of MapType and ArrayType for