Hi
Is there any functions to find distinct count of all the variables in
dataframe.
val sc = new SparkContext(conf) // spark context
val options = Map("header" -> "true", "delimiter" -> delimiter,
"inferSchema" -> "true")
val sqlContext = new org.apache.spark.sql.SQLContext(sc) // sql context
val datasetDF =
sqlContext.read.format("com.databricks.spark.csv").options(options).load(inputFile)
we are able to get the schema, variable data type. is there any method
to get the distinct count ?
--
Thanks and Regards
Arun
