Re: Unable to read the Parquet file written by NiFi through Spark when Logical Data Type is set to true.

Compare the docs for logical types: https://avro.apache.org/docs/1.7.5/spec.html https://avro.apache.org/docs/1.7.7/spec.html https://avro.apache.org/docs/1.8.2/spec.html It looks like Avro didn't support logical types period in 1.7.5, dipped their

RE: Unable to read the Parquet file written by NiFi through Spark when Logical Data Type is set to true.

Hi, Spark 2.3 has avro-1.7.7.jar. But hive 1.2 also uses avro-1.7.5.jar and I’m able to read the parquet file from hive. Not sure if this is the reason. Thanks, Mohit From: Mike Thomsen Sent: 31 May 2018 14:15 To: users@nifi.apache.org Subject: Re: Unable to read the Parquet file

Re: Unable to read the Parquet file written by NiFi through Spark when Logical Data Type is set to true.

Maybe check to see which version of Avro is bundled with your deployment of Spark? On Thu, May 31, 2018 at 3:26 AM Mohit wrote: > Hi Mike, > > > > I have created the hive external table on the top of parquet and able to > read it from hive. > > > > While querying hive from spark these are the

RE: Unable to read the Parquet file written by NiFi through Spark when Logical Data Type is set to true.

Hi Mike, I have created the hive external table on the top of parquet and able to read it from hive. While querying hive from spark these are the errors – For the decimal type, I get the following error – (In hive, data type is decimal(12,5)) Caused by:

Re: Unable to read the Parquet file written by NiFi through Spark when Logical Data Type is set to true.

What's the error from Spark? Logical data types are just a variant on existing data types in Avro 1.8. On Wed, May 30, 2018 at 7:54 AM Mohit wrote: > Hi all, > > > > I’m fetching the data from RDBMS and writing it to parquet using > PutParquet processor. I’m not able to read the data from Spark

Unable to read the Parquet file written by NiFi through Spark when Logical Data Type is set to true.

Hi all, I'm fetching the data from RDBMS and writing it to parquet using PutParquet processor. I'm not able to read the data from Spark when Logical Data Type is true. I'm able to read it from Hive. Do I have to set some specific properties in the PutParquet processor to make it readable from