Re: Typed datataset from Avro generated classes?

2020-12-04 Thread Nads
Same problem here. A google search shows a few related jira tickets in "Resolved" state but I am getting the same error in Spark 3.0.1. I'm pasting my `spark-shell` output below: scala> import org.apache.spark.sql.Encoders import org.apache.spark.sql.Encoders scala> val linkageBean =

Re: Typed datataset from Avro generated classes?

2020-04-20 Thread Elkhan Dadashov
Hi Spark users, Did anyone resolve this issue? Encoder encoder = Encoders.bean(AvroGenereatedClass.class); Dataset ds = sparkSession.read().parquet(filename).as(encoder); I'm also facing the same problem: "Cannot have circular references in bean class, but got the circular reference of class

Typed datataset from Avro generated classes?

2017-09-27 Thread Joaquin Tarraga
Hi all, I have an Avro generated class (e.g., AvroGenerateClass) and I am using Encoders.bean to get a typed dataset (e.g., Dataset): Encoder encoder = Encoders.bean(AvroGenereatedClass.class); Dataset ds = sparkSession.read().parquet(filename).as(encoder); I am getting an exception from the