[
https://issues.apache.org/jira/browse/BEAM-11658?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ismaël Mejía updated BEAM-11658:
--------------------------------
Summary: Match .snappy files into the given (de)compressor (was: Add
Snappy compression and decompression support)
> Match .snappy files into the given (de)compressor
> -------------------------------------------------
>
> Key: BEAM-11658
> URL: https://issues.apache.org/jira/browse/BEAM-11658
> Project: Beam
> Issue Type: New Feature
> Components: sdk-java-core
> Reporter: Andrei Balici
> Assignee: Andrei Balici
> Priority: P2
> Time Spent: 1h 50m
> Remaining Estimate: 0h
>
> Snappy is a compression/decompression library. It does not aim for maximum
> compression, or compatibility with any other compression library; instead, it
> aims for very high speeds and reasonable compression. For instance, compared
> to the fastest mode of zlib, Snappy is an order of magnitude faster for most
> inputs, but the resulting compressed files are anywhere from 20% to 100%
> bigger.
>
> Many data pipelines will have as input files .snappy compressed, and these
> currently have to be read by creating custom DoFn(s).
>
> It would be nice to see Beam support this out of the box, as it does
> currently for LZO. Snappy usually is faster than algorithms in the same class
> (e.g. LZO, LZF, QuickLZ, etc.) while achieving comparable compression ratios,
> so I see no reason leaving this out.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
