Re: Spark 3.1 with spark AVRO

2022-03-10 Thread Yong Zhang
n for me. Yong From: Artemis User Sent: Thursday, March 10, 2022 3:13 PM To: user Subject: Re: Spark 3.1 with spark AVRO It must be some misconfiguration in your environment. Do you perhaps have a hardwired $SPARK_HOME env variable in your shell? An easy test would be to place the spark-av

Re: Spark 3.1 with spark AVRO

2022-03-10 Thread Artemis User
It must be some misconfiguration in your environment.  Do you perhaps have a hardwired $SPARK_HOME env variable in your shell?  An easy test would be to place the spark-avro jar file you downloaded in the jars directory of Spark and run spark-shell again without the packages option.  This will

Spark 3.1 with spark AVRO

2022-03-10 Thread Yong Zhang
Hi, I am puzzled with this issue of Spark 3.1 version to read avro file. Everything is done on my local mac laptop so far, and I really don't know where the issue comes from, and I googled a lot and cannot find any clue. I am always using Spark 2.4 version, as it is really mature. But for a new