Ok, I found a way to solve it. Just pass the schema like this:
val schema = Encoders.product[Person].schema spark.read.schema(schema).parquet(“input”)…. > On 9. May 2020, at 13:28, Jorge Machado <jom...@me.com.INVALID> wrote: > > Hello everyone, > > One question to the community. > > Imagine I have this > > Case class Person(age: int) > > spark.read.parquet(“inputPath”).as[Person] > > > After a few weeks of coding I change the class to: > Case class Person(age: int, name: Option[String] = None) > > > Then when I run the new code on the same input it fails saying that It cannot > find the name on the schema from the parquet file. > > Spark version 2.3.3 > > How is the best way to guard or fix this? Regenerating all data seems not to > be a option for us. > > Thx > --------------------------------------------------------------------- > To unsubscribe e-mail: user-unsubscr...@spark.apache.org > --------------------------------------------------------------------- To unsubscribe e-mail: user-unsubscr...@spark.apache.org
