[
https://issues.apache.org/jira/browse/HDDS-7350?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17633767#comment-17633767
]
Stephen O'Donnell commented on HDDS-7350:
-----------------------------------------
We can change how existing things work if it results it a better solution,
especially if the changes are inside the Ozone client and we are not changing
the interfaces that end user interacts with.
You cannot compress encrypted bytes - they will not compress due to being
effectively random after encryption.
It would be good to explore options to see what is possibly to make compression
work with encryption and be seekable. If it could be enabled at the server
level and forgotten about, it would be a very powerful feature.
There are some compression codecs that are splittable, as they effectively
compress in blocks and store the relevant details to start a new decompression
stream in an index file (eg LZO). Some discussion in
https://stackoverflow.com/questions/14820450/best-splittable-compression-for-hadoop-input-bz2
Most fileystems (eg ZFS) do seem to compress at the block level perhaps to make
it transparent to anything using the filesystem.
> Ozone Transparent Data Compression Support
> ------------------------------------------
>
> Key: HDDS-7350
> URL: https://issues.apache.org/jira/browse/HDDS-7350
> Project: Apache Ozone
> Issue Type: New Feature
> Reporter: Kirill Sizov
> Assignee: Kirill Sizov
> Priority: Major
> Attachments: compression_ozone - 2022.10.1.pdf,
> compression_ozone-2022.10.2.pdf, compression_ozone-2022.11.1.pdf,
> compression_ozone-2022.11.2.pdf
>
>
> Currently Ozone stores uncompressed data, which in case of text or a similar
> format may benefit from being compressed. This may save significant amount of
> space and hence the money.
> See the attached document for the design.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
