Thanks,
Code is running on a single machine.
And it still doesn't answer my question.

On Fri, Jul 17, 2015 at 4:52 PM, ayan guha <[email protected]> wrote:

> You can bump up number of partitions while creating the rdd you are using
> for df
> On 17 Jul 2015 21:03, "Harit Vishwakarma" <[email protected]>
> wrote:
>
>> Hi,
>>
>> I used createDataFrame API of SqlContext in python. and getting
>> OutOfMemoryException. I am wondering if it is creating whole dataFrame in
>> memory?
>> I did not find any documentation describing memory usage of Spark APIs.
>> Documentation given is nice but little more details (specially on memory
>> usage/ data distribution etc.) will really help.
>>
>> --
>> Regards
>> Harit Vishwakarma
>>
>>


-- 
Regards
Harit Vishwakarma

Reply via email to