Yes, finally it will be converted to an RDD internally. However DataFrame queries are passed through catalyst , which provides several optimizations e.g. code generation, intelligent shuffle etc , which is not the case for pure RDDs.
Regards, Rishitesh Mishra, SnappyData . (http://www.snappydata.io/) https://in.linkedin.com/in/rishiteshmishra On Thu, Jul 7, 2016 at 4:50 PM, brccosta <[email protected]> wrote: > Dear guys, > > I'm investigating the differences between RDDs and Dataframes/Datasets. I > couldn't find the answer for this question: Dataframes acts as a new layer > in the Spark stack? I mean, in the execution there is a conversion to RDD? > > For example, if I create a Dataframe and perform a query, in the final step > it will be transformed into a RDD to be executed in Spark? > > > > -- > View this message in context: > http://apache-spark-user-list.1001560.n3.nabble.com/RDD-and-Dataframes-tp27306.html > Sent from the Apache Spark User List mailing list archive at Nabble.com. > > --------------------------------------------------------------------- > To unsubscribe e-mail: [email protected] > >
