Parquet-SPARK-PIG integration.

[suman bharadwaj]

Hi All,

We have written PIG Jobs which outputs the data in parquet format.

For eg:

register parquet-column-1.3.1.jar;
register parquet-common-1.3.1.jar;
register parquet-format-2.0.0.jar;
register parquet-hadoop-1.3.1.jar;
register parquet-pig-1.3.1.jar;
register parquet-encoding-1.3.1.jar;

A =load 'path' using PigStorage('\t') as (in:int,name:chararray);
store A into 'output_path' using parquet.pig.ParquetStorer();

Now how do i read this parquet file in SPARK ?

Thanks in advance.

Regards,
SB