Re: Data frames or Spark sql for partitioned tables

are you referring to data frame after map? You can use the following for columns .toDF("name1", "name2") after the map. Dr Mich Talebzadeh LinkedIn * https://www.linkedin.com/profile/view?id=AAEWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw

Data frames or Spark sql for partitioned tables

Hi, I am new to spark and trying to implement the solution without using hive. We are migrating to new environment where hive is not present intead I need to use spark to output files. I look at case class and maximum number of columns I can use is 22 but I have 180 columns . In this scenario