For cast, you can use selectExpr method. For example,
df.selectExpr("cast(col1 as int) as col1", "cast(col2 as bigint) as col2").
Or, df.select(df("colA").cast("int"), ...)
On Thu, Apr 2, 2015 at 8:33 PM, Michael Armbrust
wrote:
> val df = Seq(("test", 1)).toDF("col1", "col2")
>
> You can use SQ
val df = Seq(("test", 1)).toDF("col1", "col2")
You can use SQL style expressions as a string:
df.filter("col1 IS NOT NULL").collect()
res1: Array[org.apache.spark.sql.Row] = Array([test,1])
Or you can also reference columns using df("colName") or quot;colName" or
col("colName")
df.filter(df("c
Hi folks, having some seemingly noob issues with the dataframe API.
I have a DF which came from the csv package.
1. What would be an easy way to cast a column to a given type -- my DF
columns are all typed as strings coming from a csv. I see a schema getter
but not setter on DF
2. I am trying to
