Re: Problem while loading saved data

2015-09-04 Thread Amila De Silva
'd guesd that HDFS isn't setup between the nodes, or for some > reason writes are defaulting to file:///path/ rather than hdfs:///path/ > > > > > -- Original message-- > > *From: *Amila De Silva > > *Date: *Thu, 3 Sep 2015 17:12 > > *To: *Ewan Le

Re: Problem while loading saved data

2015-09-02 Thread Amila De Silva
ople.parquet2. > > > > Guru Medasani > gdm...@gmail.com > > > > On Sep 2, 2015, at 8:25 PM, Amila De Silva wrote: > > Hi All, > > I have a two node spark cluster, to which I'm connecting using IPython > notebook. > To see how data saving/loading

Problem while loading saved data

2015-09-02 Thread Amila De Silva
Hi All, I have a two node spark cluster, to which I'm connecting using IPython notebook. To see how data saving/loading works, I simply created a dataframe using people.json using the Code below; df = sqlContext.read.json("examples/src/main/resources/people.json") Then called the following to sa