Hi devs, We are trying to build a pipeline to read snappy compressed text files that contain one record per line using the Java SDK.
We have tried the following to read the files:
p.apply("ReadLines",
FileIO.match().filepattern((options.getInputFilePattern())))
.apply(FileIO.readMatches())
.setCoder(SnappyCoder.of(ReadableFileCoder.of()))
.apply(TextIO.readFiles())
.apply(ParDo.of(new TransformRecord()));
Is there a recommended way to decompress and read Snappy files with Beam?
Thanks,
Chris
