Yes. The JSON files compressed by Flume or Spark work well with Spark. But
the json files compressed by myself cannot be read by spark due to codec
problem. It seems sparking can read files compressed by hadoop snappy(
On Thu, May 17, 2018 at 5:47 PM, Victor Noagbodji <
> Hey, Sorry if I misunderstood. Are you feeding the compressed JSON file to
> Spark directly?
> On May 17, 2018, at 4:59 AM, JF Chen <darou...@gmail.com> wrote:
> I made some snappy compressed json file with normal snappy codec(
> https://github.com/xerial/snappy-java ) , which seems cannot be read by
> Spark correctly.
> So how to make existed snappy file recognized by spark? Any tools to
> convert them?
> Junfeng Chen