[xz-devel] XZ for Java 1.10

[Lasse Collin]

XZ for Java 1.10 is available at  and
in the Maven Central (groupId = org.tukaani, artifactId = xz).

Here is an extract from the NEWS file:

  * Licensing change: From version 1.10 onwards, XZ for Java is under
the BSD Zero Clause License (0BSD). 1.9 and older are in the
public domain and obviously remain so; the change only affects
the new releases.

0BSD is an extremely permissive license which doesn't require
retaining or reproducing copyright or license notices when
distributing the code, thus in practice there is extremely
little difference to public domain.

  * Mark copyright and license information in the source package so
that it is compliant to the [REUSE Specification version 3.2](
).

  * Improve LZMAInputStream.enableRelaxedEndCondition():

  - Error detection is slightly better.

  - The input position will always be at the end of the stream
after successful decompression.

  * Support .lzma files that have both a known uncompressed size and
the end marker. Such files are uncommon but valid. The same issue
was fixed in XZ Utils 5.2.6 in 2022.

  * Add ARM64 and RISC-V BCJ filters.

  * Speed optimizations:
  - Delta filter
  - LZMA/LZMA2 decoder
  - LZMA/LZMA2 encoder (partially Java >= 9 only)
  - CRC64 (Java >= 9 only)

  * Changes that affect API/ABI compatibility:

  - Change XZOutputStream constructors to not call the method
`public void updateFilters(FilterOptions[] filterOptions)`.

  - In SeekableXZInputStream, change the method
`public void seekToBlock(int blockNumber)` to not call
the method `public long getBlockPos(int blockNumber)`.

  - Make the filter options classes `final`:
  * ARM64Options
  * ARMOptions
  * ARMThumbOptions
  * DeltaOptions
  * IA64Options
  * LZMA2Options
  * PowerPCOptions
  * RISCVOptions
  * SPARCOptions
  * X86Options

  * Add new system properties:

  - `org.tukaani.xz.ArrayCache` sets the default ArrayCache:
`Dummy` (default) or `Basic`. See the documentation of
ArrayCache and BasicArrayCache.

  - `org.tukaani.xz.MatchLengthFinder` (Java >= 9 only) sets the
byte array comparison method used for finding match lengths in
LZMA/LZMA2 encoder: `UnalignedLongLE` (default on x86-64 and
ARM64) or `Basic` (default on other systems). The former could
be worth testing on other 64-bit little endian systems that
support fast unaligned memory access.

  * Build system (Apache Ant):

  - Building the documentation no longer downloads `element-list`
or `package-list` file; the build is now fully offline. Such
files aren't needed with OpenJDK >= 16 whose `javadoc` can
auto-link to platform documentation on docs.oracle.com. With
older OpenJDK versions, links to platform documentation aren't
generated anymore.

  - Don't require editing of build.properties to build with
OpenJDK 8. Now it's enough to use `ant -Djava8only=true`.
Older OpenJDK versions are no longer supported because
the main source tree uses Java 8 features.

  - Support reproducible builds. See the notes in README.md.

  - Add a new Ant target `pom` that only creates xz.pom.

  - Change `ant dist` to use `git archive` to create a .zip file.

  * Convert the plain text documentation in the source tree to
Markdown (CommonMark).

  * The binaries of 1.10 in the Maven Central require Java 8 and
contain optimized classes for Java >= 9 as multi-release JAR.
They were built with OpenJDK 21.0.4 on GNU/Linux using the
following command:

SOURCE_DATE_EPOCH=1722262226 TZ=UTC0 ant maven

-- 
Lasse Collin

Re: [xz-devel] XZ for Java

[Lasse Collin]

On 2022-06-21 Dennis Ens wrote:
> Why not pass on maintainership for XZ for C so you can give XZ for
> Java more attention? Or pass on XZ for Java to someone else to focus
> on XZ for C? Trying to maintain both means that neither are
> maintained well.

Finding a co-maintainer or passing the projects completely to someone
else has been in my mind a long time but it's not a trivial thing to
do. For example, someone would need to have the skills, time, and enough
long-term interest specifically for this. There are many other projects
needing more maintainers too.

As I have hinted in earlier emails, Jia Tan may have a bigger role in
the project in the future. He has been helping a lot off-list and is
practically a co-maintainer already. :-) I know that not much has
happened in the git repository yet but things happen in small steps. In
any case some change in maintainership is already in progress at least
for XZ Utils.

-- 
Lasse Collin

Re: [xz-devel] XZ for Java

[Dennis Ens]

>> I haven't lost interest but my ability to care has been fairly limited
>> mostly due to longterm mental health issues but also due to some other
>> things. Recently I've worked off-list a bit with Jia Tan on XZ Utils and
>> perhaps he will have a bigger role in the future, we'll see.
>>
>> It's also good to keep in mind that this is an unpaid hobby project.
>>
>> Anyway, I assure you that I know far too well about the problem that
>> not much progress has been made. The thought of finding new maintainers
>> has existed for a long time too as the current situation is obviously
>> bad and sad for the project.

> With your current rate, I very doubt to see 5.4.0 release this year. The only 
> progress since april has been small changes to test code. You ignore the many 
> patches bit rotting away on this mailing list. Right now you choke your repo. 
> Why wait until 5.4.0 to change maintainer? Why delay what your repo needs?

I am sorry about your mental health issues, but its important to be
aware of your own limits. I get that this is a hobby project for all
contributors, but the community desires more. Why not pass on
maintainership for XZ for C so you can give XZ for Java more
attention? Or pass on XZ for Java to someone else to focus on XZ for
C? Trying to maintain both means that neither are maintained well.

--
Dennis Ens

Re: [xz-devel] XZ for Java

[Jigar Kumar]

> Anyway, I assure you that I know far too well about the problem that
> not much progress has been made. The thought of finding new maintainers
> has existed for a long time too as the current situation is obviously
> bad and sad for the project.
>
> A new XZ Utils stable branch should get released this year with
> threaded decoder etc. and a few alpha/beta releases before that.
> Perhaps the moment after the 5.4.0 release would be a convenient moment
> to make changes in the list of project maintainer(s).

With your current rate, I very doubt to see 5.4.0 release this year. The only 
progress since april has been small changes to test code. You ignore the many 
patches bit rotting away on this mailing list. Right now you choke your repo. 
Why wait until 5.4.0 to change maintainer? Why delay what your repo needs?

Re: [xz-devel] XZ for Java

[Lasse Collin]

On 2022-06-07 Jigar Kumar wrote:
> Progress will not happen until there is new maintainer. XZ for C has
> sparse commit log too. Dennis you are better off waiting until new
> maintainer happens or fork yourself. Submitting patches here has no
> purpose these days. The current maintainer lost interest or doesn't
> care to maintain anymore. It is sad to see for a repo like this.

I haven't lost interest but my ability to care has been fairly limited
mostly due to longterm mental health issues but also due to some other
things. Recently I've worked off-list a bit with Jia Tan on XZ Utils and
perhaps he will have a bigger role in the future, we'll see.

It's also good to keep in mind that this is an unpaid hobby project.

Anyway, I assure you that I know far too well about the problem that
not much progress has been made. The thought of finding new maintainers
has existed for a long time too as the current situation is obviously
bad and sad for the project.

A new XZ Utils stable branch should get released this year with
threaded decoder etc. and a few alpha/beta releases before that.
Perhaps the moment after the 5.4.0 release would be a convenient moment
to make changes in the list of project maintainer(s).

Forks are obviously another possibility and I cannot control that. If
those happen, I hope that file format changes are done so that no
silly problems occur (like using the same ID for different things in
two projects). 7-Zip supports .xz and keeping its developer Igor Pavlov
informed about format changes (including new filters) is important too.

-- 
Lasse Collin

Re: [xz-devel] XZ for Java

[Jigar Kumar]

Progress will not happen until there is new maintainer. XZ for C has sparse 
commit log too. Dennis you are better off waiting until new maintainer happens 
or fork yourself. Submitting patches here has no purpose these days. The 
current maintainer lost interest or doesn't care to maintain anymore. It is sad 
to see for a repo like this.

Re: [xz-devel] XZ for Java

[Brett Okken]

I tested the match finder changes on arm64 (aws graviton) and results are
quite positive.

On Thu, May 19, 2022 at 3:41 PM Lasse Collin 
wrote:

> On 2022-05-19 Dennis Ens wrote:
> > Is XZ for Java still maintained?
>
> Yes, by some definition at least, like if someone reports a bug it will
> get fixed. Development of new features definitely isn't very active. :-(
>
> > I asked a question here a week ago and have not heard back.
>
> I saw. I have lots of unanswered emails at the moment and obviously
> that isn't a good thing. After the latest XZ for Java release I've
> tried focus on XZ Utils (and ignored XZ for Java), although obviously
> that hasn't worked so well either even if some progress has happened
> with XZ Utils.
>
> > When I view the git log I can see it has not updated in over a year.
> > I am looking for things like multithreaded encoding / decoding and a
> > few updates that Brett Okken had submited (but are still waiting for
> > merge). Should I add these things to only my local version, or is
> > there a plan for these things in the future?
>
> Brett Okken's patches I haven't reviewed so I cannot give definite
> answers about if you should include them in your local version, sorry.
>
> The match finder optimizations are more advanced as they are somewhat
> arch-specific so it could be good to have broader testing how much they
> help on different systems (not just x86-64 but 32-bit x86, ARM64, ...)
> and if they behave well on Android too. The benefits have to be clear
> enough (and cause no problems) to make the extra code worth it.
>
> The Delta coder patch is small and relative improvement is big, so that
> likely should get included. The Delta filter is used rarely though and
> even a slow version isn't *that* slow in the big picture (there will
> also be LZMA2 and CRC32/CRC64).
>
> Threading would be nice in the Java version. Threaded decompression only
> recently got committed to XZ Utils repository.
>
> Jia Tan has helped me off-list with XZ Utils and he might have a bigger
> role in the future at least with XZ Utils. It's clear that my resources
> are too limited (thus the many emails waiting for replies) so something
> has to change in the long term.
>
> --
> Lasse Collin
>
>

Re: [xz-devel] XZ for Java

[Lasse Collin]

On 2022-05-19 Dennis Ens wrote:
> Is XZ for Java still maintained?

Yes, by some definition at least, like if someone reports a bug it will
get fixed. Development of new features definitely isn't very active. :-(

> I asked a question here a week ago and have not heard back.

I saw. I have lots of unanswered emails at the moment and obviously
that isn't a good thing. After the latest XZ for Java release I've
tried focus on XZ Utils (and ignored XZ for Java), although obviously
that hasn't worked so well either even if some progress has happened
with XZ Utils.

> When I view the git log I can see it has not updated in over a year.
> I am looking for things like multithreaded encoding / decoding and a
> few updates that Brett Okken had submited (but are still waiting for
> merge). Should I add these things to only my local version, or is
> there a plan for these things in the future?

Brett Okken's patches I haven't reviewed so I cannot give definite
answers about if you should include them in your local version, sorry.

The match finder optimizations are more advanced as they are somewhat
arch-specific so it could be good to have broader testing how much they
help on different systems (not just x86-64 but 32-bit x86, ARM64, ...)
and if they behave well on Android too. The benefits have to be clear
enough (and cause no problems) to make the extra code worth it.

The Delta coder patch is small and relative improvement is big, so that
likely should get included. The Delta filter is used rarely though and
even a slow version isn't *that* slow in the big picture (there will
also be LZMA2 and CRC32/CRC64).

Threading would be nice in the Java version. Threaded decompression only
recently got committed to XZ Utils repository.

Jia Tan has helped me off-list with XZ Utils and he might have a bigger
role in the future at least with XZ Utils. It's clear that my resources
are too limited (thus the many emails waiting for replies) so something
has to change in the long term.

-- 
Lasse Collin

[xz-devel] XZ for Java

[Dennis Ens]

Dear XZ Java Community

Is XZ for Java still maintained? I asked a question here a week ago
and have not heard back. When I view the git log I can see it has not
updated in over a year. I am looking for things like multithreaded
encoding / decoding and a few updates that Brett Okken had submited
(but are still waiting for merge). Should I add these things to only
my local version, or is there a plan for these things in the future?

--
Dennis Ens

[xz-devel] XZ for Java 1.9

[Lasse Collin]

XZ for Java 1.9 is available at  and
in the Maven Central (groupId = org.tukaani, artifactId = xz). Here is
an extract from the NEWS file:

* Add LZMAInputStream.enableRelaxedEndCondition(). It allows
  decompression of LZMA streams whose uncompressed size is known
  but it is unknown if the end of stream marker is present. This
  method is meant to be useful in Apache Commons Compress to
  support .7z files created by certain very old 7-Zip versions.
  Such files have the end of stream marker in the LZMA data even
  though the uncompressed size is known. 7-Zip supports such files
  and thus other implementations of the .7z format should support
  them too.

* Make LZMA/LZMA2 decompression faster. With files that compress
  extremely well the performance can be a lot better but with
  more typical files the improvement is minor.

* Make the CRC64 code faster.

* Add module-info.java as multi-release JAR. The attribute
  Automatic-Module-Name was removed.

* The binaries for XZ for Java 1.9 in the Maven Central now
  require Java 7. Building the package requires at least Java 9
  for module-info support but otherwise the code should still be
  Java 5 compatible (see README and comments in build.properties).

-- 
Lasse Collin  |  IRC: Larhzu @ IRCnet & Freenode

[xz-devel] XZ for Java 1.8

[Lasse Collin]

XZ for Java 1.8 is available at  and
in the Maven Central (groupId = org.tukaani, artifactId = xz). Here is
an extract from the NEWS file:

* Fix a binary compatibility regression: XZ for Java 1.7 binaries
  in the Maven Central require Java 9 which is too new. XZ for
  Java 1.8 binaries require Java 5. (XZ for Java 1.6 and older
  binaries require Java 1.4.)

  If you are using OpenJDK 9 or later, you will need to edit the
  "sourcever = 1.5" line in the file "build.properties" before
  running "ant". Set it to 1.6 or higher. The default value 1.5
  isn't supported by OpenJDK 9 or later.

* Add "Automatic-Module-Name" = "org.tukaani.xz".

-- 
Lasse Collin  |  IRC: Larhzu @ IRCnet & Freenode

[xz-devel] XZ for Java 1.7

[Lasse Collin]

XZ for Java 1.7 is available at  and
in the Maven Central (groupId = org.tukaani, artifactId = xz). Here is
an extract from the NEWS file:

* Fix LZMA2InputStream.available() which could return a too high
  value in case of uncompressed LZMA2 chunks. This incorrect
  value was visible via other available() methods too, for example,
  XZInputStream.available().

* Add the ArrayCache API. It's a pool-like API to reuse large byte
  and int arrays between compressor and decompressor instances.
  If you are (de)compressing many tiny files in a row, taking
  advantage of this API can improve performance significantly.

-- 
Lasse Collin  |  IRC: Larhzu @ IRCnet & Freenode

[xz-devel] XZ for Java 1.6

[Lasse Collin]

XZ for Java 1.6 is available at  and
in the Maven Central (groupId = org.tukaani, artifactId = xz). Here is
an extract from the NEWS file:

* Fix LZMA2Options.getInputStream to work with a preset dictionary.

* Make it possible to disable verification of integrity checks in
  XZ decompression. It should almost never be used but may be useful
  in some rare situations. This feature is available via new
  constructors in XZInputStream, SingleXZInputStream, and
  SeekableXZInputStream.

* Add LZMAOutputStream for encoding to raw LZMA (i.e. LZMA1) streams
  and to the legacy .lzma format.

-- 
Lasse Collin  |  IRC: Larhzu @ IRCnet & Freenode

[xz-devel] XZ for Java 1.5

[Lasse Collin]

XZ for Java 1.5 is available at  and
in the Maven Central (groupId = org.tukaani, artifactId = xz). Here is
an extract from the NEWS file:

* Fix a wrong assertion in BCJ decoders.

* Use a field instead of reallocating a temporary one-byte buffer
  in read() and write() implementations in several classes.

-- 
Lasse Collin  |  IRC: Larhzu @ IRCnet & Freenode

[xz-devel] XZ for Java 1.4

[Lasse Collin]

XZ for Java 1.4 is available at  and
in the Maven Central (groupId = org.tukaani, artifactId = xz). Here is
an extract from the NEWS file:

  * Add LZMAInputStream for decoding .lzma files and raw LZMA streams.

-- 
Lasse Collin  |  IRC: Larhzu @ IRCnet & Freenode

[xz-devel] XZ for Java 1.3

[Lasse Collin]

XZ for Java 1.3 is available at  and
in the Maven Central (groupId = org.tukaani, artifactId = xz). Here is
an extract from the NEWS file:

  * Fix a data corruption bug when flushing the LZMA2 encoder or
when using a preset dictionary.

  * Make information about the XZ Block positions and sizes available
in SeekableXZInputStream by adding the following public functions:
  - int getStreamCount()
  - int getBlockCount()
  - long getBlockPos(int blockNumber)
  - long getBlockSize(int blockNumber)
  - long getBlockCompPos(int blockNumber)
  - long getBlockCompSize(int blockNumber)
  - int getBlockCheckType(int blockNumber)
  - int getBlockNumber(long pos)
  - void seekToBlock(int blockNumber)

  * Minor improvements to javadoc comments were made.

-- 
Lasse Collin  |  IRC: Larhzu @ IRCnet & Freenode

[xz-devel] XZ for Java 1.2

[Lasse Collin]

XZ for Java 1.2 is available at  and
in the Maven Central (groupId = org.tukaani, artifactId = xz). Here is
an extract from the NEWS file:

  * Use fields instead of reallocating frequently-needed temporary
objects in the LZMA encoder.

  * Fix the contents of xz-${version}-sources.jar.

  * Add OSGi attributes to xz.jar.

-- 
Lasse Collin  |  IRC: Larhzu @ IRCnet & Freenode

[xz-devel] XZ for Java 1.1

[Lasse Collin]

XZ for Java 1.1 is available at  and
in the Maven Central (groupId = org.tukaani, artifactId = xz). Here is
an extract from the NEWS file:

  * The depthLimit argument in the LZMA2Options constructor is
no longer ignored.

  * LZMA2Options() can no longer throw UnsupportedOptionsException.

  * Fix bugs in the preset dictionary support in the LZMA2 encoder.

-- 
Lasse Collin  |  IRC: Larhzu @ IRCnet & Freenode

[xz-devel] XZ for Java 1.0

[Lasse Collin]

XZ for Java 1.0 was released earlier this week:

http://tukaani.org/xz/java.html

The code is available also in Maven Central. The actual Java code
is identical to the version 0.4, but I made a new release to make
it clear that the code and API should now be stable.

-- 
Lasse Collin  |  IRC: Larhzu @ IRCnet & Freenode

[xz-devel] XZ for Java 0.4

[Lasse Collin]

XZ for Java 0.4 is now available:

http://tukaani.org/xz/java.html

There are some minor fixes to the old code. Support for random access
decompression is a new feature. Threading is missing but it's easier to
add it to this code than to liblzma in XZ Utils.

This is probably the last release before 1.0. I don't have anything
planned before 1.0 but maybe someone finds something that could be
improved. :-)

-- 
Lasse Collin  |  IRC: Larhzu @ IRCnet & Freenode