[
https://issues.apache.org/jira/browse/HADOOP-13578?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16166298#comment-16166298
]
Jason Lowe commented on HADOOP-13578:
-------------------------------------
bq. it finally gets us the promised land of fast, high compression and
splittability all in one file format.
To clarify, this ZStandard codec is not splittable. The ZStandard file format
is theoretically capable of supporting splits, but that's not an official
feature yet. As I understand it, split support is being added via the
"seekable" support which is in the contrib area at
https://github.com/facebook/zstd/tree/dev/contrib/seekable_format, but that's
not part of the standard zstd library yet. See
https://github.com/facebook/zstd/issues/395 for more discussion around the
support of splits in the zstd codec.
> Add Codec for ZStandard Compression
> -----------------------------------
>
> Key: HADOOP-13578
> URL: https://issues.apache.org/jira/browse/HADOOP-13578
> Project: Hadoop Common
> Issue Type: New Feature
> Reporter: churro morales
> Assignee: churro morales
> Fix For: 2.9.0, 3.0.0-alpha2
>
> Attachments: HADOOP-13578-branch-2.v9.patch, HADOOP-13578.patch,
> HADOOP-13578.v1.patch, HADOOP-13578.v2.patch, HADOOP-13578.v3.patch,
> HADOOP-13578.v4.patch, HADOOP-13578.v5.patch, HADOOP-13578.v6.patch,
> HADOOP-13578.v7.patch, HADOOP-13578.v8.patch, HADOOP-13578.v9.patch
>
>
> ZStandard: https://github.com/facebook/zstd has been used in production for 6
> months by facebook now. v1.0 was recently released. Create a codec for this
> library.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
