Re: Using Spark to process JSON with gzip filed

Yes it is. You can actually use the java.util.zip.GZIPInputStream in your case. Thanks Best Regards On Sun, Dec 20, 2015 at 3:23 AM, Eran Witkon wrote: > Thanks, since it is just a snippt do you mean that Inflater is coming > from ZLIB? > Eran > > On Fri, Dec 18, 2015 at

Re: Using Spark to process JSON with gzip filed

Thanks, since it is just a snippt do you mean that Inflater is coming from ZLIB? Eran On Fri, Dec 18, 2015 at 11:37 AM Akhil Das wrote: > Something like this? This one uses the ZLIB compression, you can replace > the decompression logic with GZip one in your case. >

Re: Using Spark to process JSON with gzip filed

Something like this? This one uses the ZLIB compression, you can replace the decompression logic with GZip one in your case. compressedStream.map(x => { val inflater = new Inflater() inflater.setInput(x.getPayload) val decompressedData = new Array[Byte](x.getPayload.size * 2)

Using Spark to process JSON with gzip filed

Hi, I have a few JSON files in which one of the field is a binary filed - this field is the output of running GZIP of a JSON stream and compressing it to the binary field. Now I want to de-compress the field and get the outpur JSON. I was thinking of running map operation and passing a function