[
https://issues.apache.org/jira/browse/JCRVLT-259?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16322229#comment-16322229
]
Dirk Rudolph edited comment on JCRVLT-259 at 1/11/18 1:51 PM:
--------------------------------------------------------------
[~tripod] you are right there was a major change in the public APIs. I reverted
that for compatibility and now its for both packages (packaging and fs.io) a
minor change. I didn't touch the JarExporter (actually AbstractExporter) so the
MANIFEST.MF is still the first entry in any exported format (tar and zip).
Though to remove the dependency to zip in the PackageManagerImpl I had to
rewrite the rewrap method, which indeed didn't guarantee the MANIFEST being the
first entry. I corrected that.
In general abstracting the archiving aspect with plain java apis is difficult
and already done by commons-compress. So thats why I choose that one. I neither
like to embed bundles but maybe we could only embed those classes necessary for
archiving and put compression into its own module. In that case vault-core
would support zip and tar only.
Specifying the export and import behaviour was also my first try. Trying that
pointed out that there are quite some places that rely on the data being in jar
format (implicitly or explicitly) and esp. passing around the import options
would have caused a much broader API change.
[~marett]
I did a simple test deploying that patch to a standard AEM 6.3 installation
running oak 1.6.1 packaging /content with about 3200 nodes mixed content and
binary using no compression with the following result:
snappy-java (jni): Package built in 21375ms. 109.6mb (min compression ratio 1/1)
default deflate: Package built in 14826ms. 122mb (Deflater.NO_COMPRESSION)
So actually deflate is still faster but resulting in a bigger package size. I
didn't investigate yet into what the problem here is, maybe its the tar being
created, maybe some wrong buffer sizes. So from my perspective that was a nice
try, but the result is not sufficient (disclaimer: I might have done something
completely wrong, or the test is not representable). At least both the zip and
the tar.sz are valid and can be extracted without issues.
was (Author: diru):
[~tripod] you are right there was a major change in the public APIs. I reverted
that for compatibility and now its for both packages (packaging and fs.io) a
minor change. I didn't touch the JarExporter (actually AbstractExporter) so the
MANIFEST.MF is still the first entry in any exported format (tar and zip).
Though to remove the dependency to zip in the PackageManagerImpl I had to
rewrite the rewrap method, which indeed didn't guarantee the MANIFEST being the
first entry. I corrected that.
In general abstracting the archiving aspect with plain java apis is difficult
and already done by commons-compress. So thats why I choose that one. I neither
like to embed bundles but maybe we could only embed those classes necessary for
archiving and put compression into its own module. In that case vault-core
would support zip and tar only.
Specifying the export and import behaviour was also my first try. Trying that
pointed out that there are quite some places that relay on the data being in
jar format (implicitly or explicitly) and esp. passing around the import
options would have caused a much broader API change.
[~marett]
I did a simple test deploying that patch to a standard AEM 6.3 installation
running oak 1.6.1 packaging /content with about 3200 nodes mixed content and
binary using no compression with the following result:
snappy-java (jni): Package built in 21375ms. 109.6mb (min compression ratio 1/1)
default deflate: Package built in 14826ms. 122mb (Deflater.NO_COMPRESSION)
So actually deflate is still faster but resulting in a bigger package size. I
didn't investigate yet into what the problem here is, maybe its the tar being
created, maybe some wrong buffer sizes. So from my perspective that was a nice
try, but the result is not sufficient (disclaimer: I might have done something
completely wrong, or the test is not representable). At least both the zip and
the tar.sz are valid and can be extracted without issues.
> Support pluggable compression formats
> -------------------------------------
>
> Key: JCRVLT-259
> URL: https://issues.apache.org/jira/browse/JCRVLT-259
> Project: Jackrabbit FileVault
> Issue Type: New Feature
> Components: Packaging
> Reporter: Dirk Rudolph
>
> The new feature to suppress compression for already compressed binaries
> introduced in JCRVLT-163, has been implemented with Apache Sling Content
> Distribution in mind to get best performance while still keep bytes on the
> wire as small as affordable [0].
> Wouldn't it make sense for such cases to support different compression
> formats as well, especially thinking about snappy or lz4?
> Currently there is a pair for compressing and decompressing implementations,
> JarExporter and ZipArchive/ZipStreamArchive. Those are used by the
> PackageManagerImpl for example to open or assemble a new package. Adding
> support for further compression formats could be done for example by
> implementing a registry holding the constructors for compression and
> decompression implementations, where the one of choice could be requested by
> the ExportOptions (defaulting to jar/zip as it is now).
> [0]
> https://github.com/apache/sling-org-apache-sling-distribution-core/blob/master/src/main/java/org/apache/sling/distribution/serialization/impl/vlt/VltUtils.java#L190
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
