Re: Dataframe constructor

2015-11-23 Thread Fengdong Yu
just simple as: val df = sqlContext.sql(“select * from table”) or val df = sqlContext.read.json(“hdfs_path”) > On Nov 24, 2015, at 3:09 AM, spark_user_2015 wrote: > > Dear all, > > is the following usage of the Dataframe constructor correct or does it > trigger any s

Dataframe constructor

2015-11-23 Thread spark_user_2015
Dear all, is the following usage of the Dataframe constructor correct or does it trigger any side effects that I should be aware of? My goal is to keep track of my dataframe's state and allow custom transformations accordingly. val df: Dataframe = ...some dataframe... val newDf = n