Hi Cesar,
Can you try 1.3.1 (
https://spark.apache.org/releases/spark-release-1-3-1.html) and see if it
still shows the error?
Thanks,
Yin
On Fri, Apr 17, 2015 at 1:58 PM, Reynold Xin wrote:
> This is strange. cc the dev list since it might be a bug.
>
>
>
> On Thu, Apr 16, 2015 at 3:18 PM, C
This is strange. cc the dev list since it might be a bug.
On Thu, Apr 16, 2015 at 3:18 PM, Cesar Flores wrote:
> Never mind. I found the solution:
>
> val newDataFrame = hc.createDataFrame(hiveLoadedDataFrame.rdd,
> hiveLoadedDataFrame.schema)
>
> which translate to convert the data frame to r
Never mind. I found the solution:
val newDataFrame = hc.createDataFrame(hiveLoadedDataFrame.rdd,
hiveLoadedDataFrame.schema)
which translate to convert the data frame to rdd and back again to data
frame. Not the prettiest solution, but at least it solves my problems.
Thanks,
Cesar Flores
On
I have a data frame in which I load data from a hive table. And my issue is
that the data frame is missing the columns that I need to query.
For example:
val newdataset = dataset.where(dataset("label") === 1)
gives me an error like the following:
ERROR yarn.ApplicationMaster: User class threw e
