Re: [compress] Java 7 to Java 8

[Stefan Bodewig]

On 2019-04-16, Gary Gregory wrote:

> Based on a recent JIRA, it seems like using Java 8 in [compress] would be
> helpful.

I'd +1 that it bumping the dependency turned out to be really
useful. I'm not convinced, yet.

> Plus, it's 2019 and Java 12 is out.

As long as using Java 7 doesn't hurt us, I don't see any reason to
update.

Stefan

-
To unsubscribe, e-mail: dev-unsubscr...@commons.apache.org
For additional commands, e-mail: dev-h...@commons.apache.org

Re: [compress] Java 7 to Java 8

[Stefan Bodewig]

On 2019-04-16, Gary Gregory wrote:

> On Tue, Apr 16, 2019 at 9:39 AM Stefan Bodewig  wrote:

>> As long as using Java 7 doesn't hurt us, I don't see any reason to
>> update.

> Using old dead versions of Java turns off potential contributors IMO.
> You have to go dig around and install an archived version of the JDK
> which Oracle has EOLed.

How so? You can build Compress with Java 13 and nobody is going to stop
you. If you happen to use a feature of something not supported in Java 7
then this may point to a reason to switch.

> Not only that but you can't use the more modern features and APIs.

This sounds as if you believed switching to more modern language
constructs and APIs would magically attract collaborators. Been there,
see the dead compress-2 branch, and don't buy this anymore.

Note I haven't cast any vote (and this is not a vote at all), I just
don't see any reason to add requirements as long as doing so doesn't
provide any benefits. As soon as it does, I'll be happy to switch.

Stefan

-
To unsubscribe, e-mail: dev-unsubscr...@commons.apache.org
For additional commands, e-mail: dev-h...@commons.apache.org

Re: Build failed in Jenkins: Commons-Compress JDK-Matrix Build » JDK 1.7 (latest) #799

[Stefan Bodewig]

After Compress has been upgraded to Parent 48 the JDK 7 build started to
fail as the new version of the bnd plugin has been compiled with
-target 8, or so it seems.

I'll downgrade the bnd plugin for compress for now.

Stefan

-
To unsubscribe, e-mail: dev-unsubscr...@commons.apache.org
For additional commands, e-mail: dev-h...@commons.apache.org

[compress] resource leak in example code

[Stefan Bodewig]

Hi all

https://issues.apache.org/jira/browse/COMPRESS-486 highlights a problem
of the code inside the example package. The methods with stream or
channel arguments create wrapper objects around said streams or channels
and never close them. They don't close them because this in turn would
close the passed in streams or channels - and that wouldn't seem right
as the methods don't own them.

One approach I came up with is to add overloads with callbacks that
receice the wrapper instance. See
https://github.com/apache/commons-compress/commit/5b43a001d151cb7251476e347e19dce480575652
in a branch for what I mean.

I toyed with the idea of returning the wrapper object from expand/create
but this would still leave the resources unclosed if an exception
occured while expanding/creating the archive.

An alternative I see is to "simply" close the wrappers ourselves and
document the methods will do so. If people need to keep their streams
open they must decorate their stream/channel with some NoClose
wrappers. Given this is example code I wouldn't feel too bad, even if it
breaks backwards compatibility in a way.

WDYT?

Stefan

-
To unsubscribe, e-mail: dev-unsubscr...@commons.apache.org
For additional commands, e-mail: dev-h...@commons.apache.org

Re: [commons-compress] branch master updated: try to get our JDK zoo working in Travis

[Stefan Bodewig]

On 2019-08-07, Gary Gregory wrote:

> I think you are missing openjdk13, openjdk-ea now maps to 14 IIRC.

If so then it hasn't been in there before my change either :-)

Stefan

-
To unsubscribe, e-mail: dev-unsubscr...@commons.apache.org
For additional commands, e-mail: dev-h...@commons.apache.org

[compress] Need Feedback for COMPRESS-479

[Stefan Bodewig]

Hi

https://issues.apache.org/jira/browse/COMPRESS-479 highlights a problem
where our zip reading code may reject an archive because it uses extra
fields that we only partially understand. In this case the archive is
not a real one, but it is easy to envision extra fields in the wild that
use more advanced versions than we currently support.

For most of our users the ZIP extra fields will be noise and they really
are only interested in the actual content of the archives.

Therefore I have decided to treat any such "not quite as we expect"
extra field the same way as any "we have no idea how this extra field
looks internally" field - i.e. just store it but don't try to parse
it.

In addition I have provided an API inside of ZipArchiveEntry that allows
users to specify in detail how strict they want the extra field parsing
to be.

Does this make sense to you? Would you recommend taking a different
approach? And it if makes sense, are the enum names I've chosen in
https://github.com/apache/commons-compress/blob/2bf678bbc6c6002569559b90ea29a031f035f067/src/main/java/org/apache/commons/compress/archivers/zip/ZipArchiveEntry.java#L1117
understandable?

Thanks

Stefan

-
To unsubscribe, e-mail: dev-unsubscr...@commons.apache.org
For additional commands, e-mail: dev-h...@commons.apache.org

Re: [compress] Need Feedback for COMPRESS-479

[Stefan Bodewig]

On 2019-08-13, Matt Sicker wrote:

> The enum makes sense. Are there any feasible ways to, say, configure
> some sort of handler class that can implement logic around unknown
> fields?

Not really. The only extension point here currently is plugging in your
own implementations of ZipExtraField via the static
ExtraFieldUtils.register - which could use some ServiceLoader magic one
day :-)

IIUC you and Gary are both saying the same thing. The enum values are
sensible defaults but it would be good to provide a way to do the same
things with custom code (callbacks or interface implementations).

It should be possible to split ExtraFieldParsingMode into a strategy
pattern interface implemented by the enum providing default
implementations. This may also reduce some other implementation quirks
(I'm not too happy with the current exception handler inside
mergeExtraFields).

Thanks!

Stefan

-
To unsubscribe, e-mail: dev-unsubscr...@commons.apache.org
For additional commands, e-mail: dev-h...@commons.apache.org

Re: [compress] Need Feedback for COMPRESS-479

[Stefan Bodewig]

On 2019-08-14, Matt Sicker wrote:

> Yes, I think you understand us. A strategy pattern with default sensible
> strategies to choose.

Done now.

I'm going to tweak the implementation a little more but the API of
ExtraFieldParsingBehavior seems to work quite well.

Thanks

Stefan

-
To unsubscribe, e-mail: dev-unsubscr...@commons.apache.org
For additional commands, e-mail: dev-h...@commons.apache.org

[compress] Closing in on the Next Release

[Stefan Bodewig]

Hi all

while I have a bit of time at hand I'd like to use that and cut a new
release of Compress. On my list of things I intend to do for the
release:

https://issues.apache.org/jira/browse/COMPRESS-485 - make the change to
ParallelScatterZipCreator#submit backwards compatible. japicmp doesn't
even see the change is incompatible because of type erasure,
unfortunately.

https://issues.apache.org/jira/browse/COMPRESS-231 - port the linked
MultiReadOnlySeekableByteChannel.scala to Java 7 inside the utils package
and update the docs.

If I find time I really want to look at
https://issues.apache.org/jira/browse/COMPRESS-478 but this is going to
take uninterrupted time inside of a hex editor :-)

@Emmanuel, you have indicated you were close to completing
https://issues.apache.org/jira/browse/COMPRESS-487 . Is this something I
should wait for?

Is there anything else anybody wants to work on before the next release?

Cheers

Stefan

-
To unsubscribe, e-mail: dev-unsubscr...@commons.apache.org
For additional commands, e-mail: dev-h...@commons.apache.org

[VOTE] Release Compress 1.19 Based on RC1

[Stefan Bodewig]

It's been more than a year since the last release of Commons Compress
and it is about time to get the fixes and enhancements out of the
door.

Compress 1.19 RC1 is available for review here:
https://dist.apache.org/repos/dist/dev/commons/compress/ (svn revision 
35357)

The tag is here:

https://gitbox.apache.org/repos/asf?p=commons-compress.git;a=tag;h=b2a7871ae59a71e7558baa2ac28af357d6fbd11f
on commit

https://gitbox.apache.org/repos/asf?p=commons-compress.git;a=commit;h=516f76ac1fe48be9a5162e53e4d0a99f23774565

Maven artifacts are here:

https://repository.apache.org/content/repositories/orgapachecommons-1459/org/apache/commons/commons-compress/1.19/

These are the Maven artifacts and their sha256 hashes

ff2d59fad74e867630fbc7daab14c432654712ac624dbee468d220677b124dd5  
commons-compress-1.19.jar
aff72b1713b4c5227f18182fb927dfaf76c0b9dabdc8ea1c101e0d981490c250  
commons-compress-1.19-javadoc.jar
9d5a690d1dea6a747e0dd8e74e64447167f395cb8f148010b8241334f58b525b  
commons-compress-1.19.pom
3952fb4e01ea5fc03ae25e0dab2f3d5b4f71a71d3750c7f03dffa6aba8075757  
commons-compress-1.19-sources.jar
a92b92e2b4c7ea4167e50bac4f55a3f03301164d0e52f33d0db357dacf0166f6  
commons-compress-1.19-tests.jar
db59bff01c65ac3b8d078d1f4271c6f4f1754a05e79decb910c2e586f4649483  
commons-compress-1.19-test-sources.jar

I have tested this with Java 8 using Maven 3.5.4.

Details of changes since 1.18 are in the release notes:
https://dist.apache.org/repos/dist/dev/commons/compress/RELEASE-NOTES.txt

https://stefan.samaflost.de/staging/commons-compress-1.19/changes-report.html

Site:
https://stefan.samaflost.de/staging/commons-compress-1.19/

Note, as usual when I cut the release this is *not* the site I'm going
to publish. I'll modify it once the release date is fixed.  As usual the
download page or the javadocs for the 1.19 release will not work.

japicmp Report (compared to 1.18):
https://stefan.samaflost.de/staging/commons-compress-1.19/japicmp.html

For some reason japicmp decided to compare 1.19 to itself, so I
rebuilt the report on master. I hope to figure out what is going on
before preparing the final site.

The binary incompatible changes to PKWareExtraHeader and its X_*
subclasses are noted inside the release notes and are extremely
unlikely to affect anybody as nobody will be using the methods with
added throws clauses directly.

RAT Report:
https://stefan.samaflost.de/staging/commons-compress-1.19/rat-report.html

KEYS:
https://www.apache.org/dist/commons/KEYS

Please review the release candidate and vote.  This vote will close no
sooner that 72 hours from now, i.e. sometime after 17:00 UTC 27-Aug 2019

  [ ] +1 Release these artifacts
  [ ] +0 OK, but...
  [ ] -0 OK, but really should fix...
  [ ] -1 I oppose this release because...

Thanks!

Stefan

-
To unsubscribe, e-mail: dev-unsubscr...@commons.apache.org
For additional commands, e-mail: dev-h...@commons.apache.org

[RESULT] Release Compress 1.19 Based on RC1

[Stefan Bodewig]

Hi all

with binding +1s by 

Gary Gregory
Bruno P. Kinoshita
Rob Tompkins

and my own implicit +1 the vote has passed. I'll proceed with publishing
the release artifacts and will announce the release after mirrors had a
bit of time to catch up.

Many thanks to those who had a look at the release candidate

 Stefan

-
To unsubscribe, e-mail: dev-unsubscr...@commons.apache.org
For additional commands, e-mail: dev-h...@commons.apache.org

[ANN] Apache Commons Compress 1.18 Released

[Stefan Bodewig]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

The Apache Commons Team is pleased to announce the release of Apache
Commons Compress 1.19.

Apache Commons Compress software defines an API for working with
compression and archive formats.  These include: bzip2, gzip, pack200,
lzma, xz, Snappy, traditional Unix Compress, DEFLATE, DEFLATE64, LZ4,
Brotli, Zstandard and ar, cpio, jar, tar, zip, dump, 7z, arj.

This release is mostly a bugfix release, for a full list of changes see
below.

ZipArchiveInputStream and ZipFile will no longer throw an exception if
an extra field generally understood by Commons Compress is malformed
but rather turn them into UnrecognizedExtraField instances.  You can
influence the way extra fields are parsed in more detail by using the
new getExtraFields(ExtraFieldParsingBehavior) method of
ZipArchiveEntry now.

Some of the ZIP extra fields related to strong encryption will now
throw ZipExceptions rather than ArrayIndexOutOfBoundsExceptions in
certain cases when used directly. There is no practical difference
when they are read via ZipArchiveInputStream or ZipFile.

Source and binary distributions are available for download from the
Apache Commons download site:

https://commons.apache.org/proper/commons-compress/download_compress.cgi

When downloading, please verify signatures using the KEYS file available
at the above location when downloading the release.

Changes in this version include:

New features:
o It is now possible to skip parsing of local file headers when
  using ZipFile which may speed up reading the archive at the
  cost of potentially missing important information. See the
  javadocs of the ZipFile class for details.
  Issue: COMPRESS-466.
o TarArchiveInputStream has a new constructor-arg lenient that
  can be used to accept certain broken archives.
  Issue: COMPRESS-469.
o ArjArchiveEntry and SevenZArchiveEntry now implement hashCode
  and equals.
  Issue: COMPRESS-475.
o Added a MultiReadOnlySeekableByteChannel class
  that can be used to concatenate the parts of a multi volume 7z
  archive so that SevenZFile can read them.
  Issue: COMPRESS-231.
  Thanks to Tim Underwood.

Fixed Bugs:
o ZipArchiveInputStream could forget the compression level has
  changed under certain circumstances.
o Fixed another potential resource leak in
  ParallelScatterZipCreator#writeTo.
  Issue: COMPRESS-470.
o ArArchiveInputStream could think it had hit EOF prematurely.
  Github Pull Request #74.
  Thanks to Alex Bertram.
o Throw IOException rather than RuntimeExceptions for certain
  malformed LZ4 or Snappy inputs.
  Issue: COMPRESS-490.
o ZipArchiveInputStream failed to read stored entries with a
  data descriptor if the data descriptor didn't use the
  signature invented by InfoZIP.
  Issue: COMPRESS-482.

Changes:
o SevenZFile now provides a way to cap memory consumption for
  LZMA(2) compressed content.
  Github Pull Request #76.
  Issue: COMPRESS-481.
  Thanks to Robin Schimpf.
o The ARJ package has been updated to contain constants for more
  recent specifications.
  Issue: COMPRESS-464.
  Thanks to Rostislav Krasny.
o Update optional library zstd-jni from 1.3.3-3 to 1.4.0-1.
  Issue: COMPRESS-484.
o ParallelScatterZipCreator now writes the entries to the
  gathered output in the same order they have been added.
  Github Pull Requests #78 and #79.
  Issue: COMPRESS-485.
  Thanks to Hervé Boutemy, Tibor Digana.
o The Expander and Archive example classes can leak resources
  they have wrapped around passed in streams or channels. The
  methods consuming streams and channels have been adapted to
  give the calling code a chance to deal with those wrapper
  resources.
  Issue: COMPRESS-486.
o ZipArchiveInputStream and ZipFile no longer assume Commons
  Compress would understand extra fields better than the writer
  of the archive and silently turn extra fields that Commons
  Compress should understand into UnrecognizedExtraFields if
  parsing said fields fails.
  It is now possible to take more control over the extra field
  parsing process with a new overload of
  ZipArchiveEntry#getExtraFields.
  Issue: COMPRESS-479.
o ZipArchiveInputStream will now throw an exception if reading a
  stored entry with a data descriptor and the data descriptor
  doesn't match what it has actually read.

  The most common case for a situation like this is a stored ZIP
  archive inside of the archive ZipArchiveInputStream currently
  reads. In such a case ZipArchiveInputStream would happily
  extract the contained archive and stop once the central
  directory of the inner archive has been hit. This is a case
  where ZipArchiveInputStream simply can not be used and only
  ZipFile is able to read the archive.

  The only other explanation is a broken archive. So the
  exception prevents users from thinking they had successfully
  read the contents of the archive.
  Issue: COMPRESS-483.
o The 7zip tools provide a default name for archive entries
  without name; SevenZFile returns a null name for such
  ent

[ANN] Apache Commons Compress 1.19 Released

[Stefan Bodewig]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

[Re-Sending with fixed subject, sorry]

The Apache Commons Team is pleased to announce the release of Apache
Commons Compress 1.19.

Apache Commons Compress software defines an API for working with
compression and archive formats.  These include: bzip2, gzip, pack200,
lzma, xz, Snappy, traditional Unix Compress, DEFLATE, DEFLATE64, LZ4,
Brotli, Zstandard and ar, cpio, jar, tar, zip, dump, 7z, arj.

This release is mostly a bugfix release, for a full list of changes see
below.

ZipArchiveInputStream and ZipFile will no longer throw an exception if
an extra field generally understood by Commons Compress is malformed
but rather turn them into UnrecognizedExtraField instances.  You can
influence the way extra fields are parsed in more detail by using the
new getExtraFields(ExtraFieldParsingBehavior) method of
ZipArchiveEntry now.

Some of the ZIP extra fields related to strong encryption will now
throw ZipExceptions rather than ArrayIndexOutOfBoundsExceptions in
certain cases when used directly. There is no practical difference
when they are read via ZipArchiveInputStream or ZipFile.

Source and binary distributions are available for download from the
Apache Commons download site:

https://commons.apache.org/proper/commons-compress/download_compress.cgi

When downloading, please verify signatures using the KEYS file available
at the above location when downloading the release.

Changes in this version include:

New features:
o It is now possible to skip parsing of local file headers when
  using ZipFile which may speed up reading the archive at the
  cost of potentially missing important information. See the
  javadocs of the ZipFile class for details.
  Issue: COMPRESS-466.
o TarArchiveInputStream has a new constructor-arg lenient that
  can be used to accept certain broken archives.
  Issue: COMPRESS-469.
o ArjArchiveEntry and SevenZArchiveEntry now implement hashCode
  and equals.
  Issue: COMPRESS-475.
o Added a MultiReadOnlySeekableByteChannel class
  that can be used to concatenate the parts of a multi volume 7z
  archive so that SevenZFile can read them.
  Issue: COMPRESS-231.
  Thanks to Tim Underwood.

Fixed Bugs:
o ZipArchiveInputStream could forget the compression level has
  changed under certain circumstances.
o Fixed another potential resource leak in
  ParallelScatterZipCreator#writeTo.
  Issue: COMPRESS-470.
o ArArchiveInputStream could think it had hit EOF prematurely.
  Github Pull Request #74.
  Thanks to Alex Bertram.
o Throw IOException rather than RuntimeExceptions for certain
  malformed LZ4 or Snappy inputs.
  Issue: COMPRESS-490.
o ZipArchiveInputStream failed to read stored entries with a
  data descriptor if the data descriptor didn't use the
  signature invented by InfoZIP.
  Issue: COMPRESS-482.

Changes:
o SevenZFile now provides a way to cap memory consumption for
  LZMA(2) compressed content.
  Github Pull Request #76.
  Issue: COMPRESS-481.
  Thanks to Robin Schimpf.
o The ARJ package has been updated to contain constants for more
  recent specifications.
  Issue: COMPRESS-464.
  Thanks to Rostislav Krasny.
o Update optional library zstd-jni from 1.3.3-3 to 1.4.0-1.
  Issue: COMPRESS-484.
o ParallelScatterZipCreator now writes the entries to the
  gathered output in the same order they have been added.
  Github Pull Requests #78 and #79.
  Issue: COMPRESS-485.
  Thanks to Hervé Boutemy, Tibor Digana.
o The Expander and Archive example classes can leak resources
  they have wrapped around passed in streams or channels. The
  methods consuming streams and channels have been adapted to
  give the calling code a chance to deal with those wrapper
  resources.
  Issue: COMPRESS-486.
o ZipArchiveInputStream and ZipFile no longer assume Commons
  Compress would understand extra fields better than the writer
  of the archive and silently turn extra fields that Commons
  Compress should understand into UnrecognizedExtraFields if
  parsing said fields fails.
  It is now possible to take more control over the extra field
  parsing process with a new overload of
  ZipArchiveEntry#getExtraFields.
  Issue: COMPRESS-479.
o ZipArchiveInputStream will now throw an exception if reading a
  stored entry with a data descriptor and the data descriptor
  doesn't match what it has actually read.

  The most common case for a situation like this is a stored ZIP
  archive inside of the archive ZipArchiveInputStream currently
  reads. In such a case ZipArchiveInputStream would happily
  extract the contained archive and stop once the central
  directory of the inner archive has been hit. This is a case
  where ZipArchiveInputStream simply can not be used and only
  ZipFile is able to read the archive.

  The only other explanation is a broken archive. So the
  exception prevents users from thinking they had successfully
  read the contents of the archive.
  Issue: COMPRESS-483.
o The 7zip tools provide a default name for archive entries
  without name; Seven

[CVE-2019-12402] Apache Commons Compress denial of service vulnerability

[Stefan Bodewig]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

Severity: Low

Vendor:
The Apache Software Foundation

Versions Affected:
Apache Commons Compress 1.15 to 1.18

Description:
The file name encoding algorithm used internally in Apache Commons
Compress can get into an infinite loop when faced with specially
crafted inputs. This can lead to a denial of service attack if an
attacker can choose the file names inside of an archive created by
Compress.

Mitigation:
Commons Compress users should upgrade to 1.19 or later.

Credit:
This issue was discovered by Masaya Suzuki of Google.

References:
https://commons.apache.org/proper/commons-compress/security-reports.html

Stefan Bodewig
-BEGIN PGP SIGNATURE-
Version: GnuPG v1

iEYEARECAAYFAl1lgKIACgkQohFa4V9ri3IsSwCg0tYlFA5WXy6EuHFtRjsbVofR
WjAAn2uNwEELGpIR2JiRO+jEAyxQJZvV
=Ds0n
-END PGP SIGNATURE-

-
To unsubscribe, e-mail: dev-unsubscr...@commons.apache.org
For additional commands, e-mail: dev-h...@commons.apache.org

Re: [bcel][all] GitHub Actions for Maven builds

[Stefan Bodewig]

On 2019-08-31, Gary Gregory wrote:

> On Sat, Aug 31, 2019 at 10:58 AM Gilles Sadowski 
> wrote:

>> Links returns "404".

> Works for me, anyone else?

Only works if you are logged in - an probably a member of the apache
organization. github returns 404 for links you are not allowed to see.

Stefan

-
To unsubscribe, e-mail: dev-unsubscr...@commons.apache.org
For additional commands, e-mail: dev-h...@commons.apache.org

[compress] COMPRESS-491 InputStream#read returning 0

[Stefan Bodewig]

Hi all

https://issues.apache.org/jira/browse/COMPRESS-491 correctly points out
that some of our InputStream implementantions violate the contract of
the read(byte[]...) pair of methods. They may return 0 instead of trying
to block and read data.

Digging deeper this really only seems to happen on purpose in
Deflate64CompressorInputStream and I've fixed that.

Many of our other stream implementations wrap other streams and may
return 0 if the underlying stream does. I don't think we should be
stricter than the stream we wrap here. Do you think we should?

A special case is where we use SeekableByteChannel in ZipFile and
SevenZFile. The read method of ReadableByteChannel is allowed to return
0 on read for non-blocking channels. I think seekable channels are
extremely unlikely to be non-blocking, so I'd stick with documenting the
fact. The alternative would be a busy loop (or retry with sleeps and
exponential backoff or something similar).

Thoughts?

Stefan

-
To unsubscribe, e-mail: dev-unsubscr...@commons.apache.org
For additional commands, e-mail: dev-h...@commons.apache.org

Re: [VOTE] Release Apache Commons Daemon 1.2.2 RC1

[Stefan Bodewig]

On 2019-09-29, sebb wrote:

> On Sun, 29 Sep 2019 at 17:21, Matt Sicker  wrote:

>> Projects that have a configure script usually include that in the source
>> distribution but not in the source repository (at least for autotools
>> users).

> But is that correct?

This is what people expect.

As others have already pointed out it is very common to not check in the
configure script but to include it with source tarballs.  Users can be
expected to have male installed but only few of them will install
autotools.

> Surely the source archive should only contain source that is in the
> source repo?

I'm not sure this is true for generated artifacts that can be verified
easily. Also I'm pretty sure some other C based projects of the ASF do
ship a generated configure script as part of the source tarball as well.

Stefan

-
To unsubscribe, e-mail: dev-unsubscr...@commons.apache.org
For additional commands, e-mail: dev-h...@commons.apache.org

Re: [VOTE] Release Apache Commons Daemon 1.2.2 RC1

[Stefan Bodewig]

On 2019-09-30, sebb wrote:

> On Mon, 30 Sep 2019 at 10:25, Stefan Bodewig  wrote:

>> On 2019-09-29, sebb wrote:

>>> On Sun, 29 Sep 2019 at 17:21, Matt Sicker  wrote:

>>>> Projects that have a configure script usually include that in the source
>>>> distribution but not in the source repository (at least for autotools
>>>> users).

>>> But is that correct?

>> This is what people expect.

> But that does not make it right.

I'm afraid you and I have different interpretations of "right" :-)

> I'll ask again: is the generated config file universally applicable?

AFAIR yes, in certain details it may depend on the version of autotools
being used but the output is universally applicable in either
case. Disclaimer: it's been a long time since I last used autotools.

> If so, put it in the source repo.

And break with decade old traditions of C-projects, is that "right"?

What are the goals of distributing tar balls and of voting on them? IMHO
(my opinion, not one that anybody has to share) (1) we want to enable
developers to use our products and (2) ensue the ASF is not put at any
risk. Personally I don't really see the risk of shipping a configure
script that is not part of the SCM repository.

Note I haven't voted and don't intend to do so as I am unable to find
time to properly vet the release. I've just been offering an opinion and
will stop adding to the noise now.

Stefan

-
To unsubscribe, e-mail: dev-unsubscr...@commons.apache.org
For additional commands, e-mail: dev-h...@commons.apache.org

Re: [Compress] BZip2 file object size?

[Stefan Bodewig]

On 2019-10-18, Gary Gregory wrote:

> BZip2FileObject does not implement doGetContentSize() and always returns
> -1, which causes VFS to blow up if you try to read. Can this kind of
> content only be streamed?

First a "I'm not an expert in the bzip2 file format" disclaimer.

>From what I can tell the file format does not contain the information
about the uncompressed size.

BZip2 files consist of a series of blocks each of which holds the result
of compressing a multiple of 10 uncompressed bytes. The multiple
(the block size, a number between 1 and 9) is part of the meta data. All
blocks except for the last have compressed the same number of original
bytes.

So you could count the blocks for an estimate and uncompress the last
block for the exact uncompressed size but in the end you have to
uncompress at least some part of the content to get the uncompressed
size.

Also I believe blocks don't need to start on byte boundaries so even
counting the blocks will be a bit more tricky. There are parallel
implementations of bzip2 in the Hadoop eco system (uncompressing blocks
in parallel) which must have solved this part, though.

Stefan

-
To unsubscribe, e-mail: dev-unsubscr...@commons.apache.org
For additional commands, e-mail: dev-h...@commons.apache.org

Re: [All] Using "SemVer"?

[Stefan Bodewig]

On 2019-10-18, Gilles Sadowski wrote:

> In another thread, the question was asked whether "Commons"
> follows "SemVer".[1]
> It seems to me that we (informally) abide by the intended goal.
> Why not state it explicitly (and make it a formal requirement for
> a release)?

To me it seems we have often avoided making decisions at a project level
that can be made at a component level.

I'm on the fence here. When a component wants to use SemVer it can say
and do so, but I don't see any reason to enforce such a policy on
components who don't want to follow SemVer.

Stefan

-
To unsubscribe, e-mail: dev-unsubscr...@commons.apache.org
For additional commands, e-mail: dev-h...@commons.apache.org

Re: [All] Using "SemVer"?

[Stefan Bodewig]

On 2019-10-18, Gilles Sadowski wrote:

> My point/question was whether we do not already follow it.

We don't, at least not in all components.

Quite a few of our components don't have a patch number at all and they
sometimes create minor releases that would be patch releases if we
followed SemVer. And sometimes we convince ourselves something that in a
release that would be a major release it is OK to not change the major
version number. Both ends of this are true for Compress which I happen
to know best.

> What would be the additional burden?

Having to obey more rules.

Don't get me wrong. I haven't got any problems with components who want
to adhere to SemVer.

It is that I just don't think we need to force all components to do
so. As little as we don't need to force all components to use Maven or
need to force all components to use an indentation of four spaces. There
is no reason to standardize this accross components IMHO.

Stefan

-
To unsubscribe, e-mail: dev-unsubscr...@commons.apache.org
For additional commands, e-mail: dev-h...@commons.apache.org

[all] Heads-Up, watch your OSGi Bundle Names

[Stefan Bodewig]

Hi all

I've started to look into
https://issues.apache.org/jira/browse/COMPRESS-498 which (correctly)
points out that Compress 1.18 has changed the OSGi coordinates over the
previous versions.

The symbolic name is defined inside the parent POM. In Parent 46 this
has been


org.apache.commons.${commons.componentid}

in parent 47 it became


org.apache.commons.${commons.packageId}

In Compress we will need to figure out how we want to deal with this
(we've now had two releases with the "new" bundle symbolic name) and see
why our unit test didn't catch that.

For the other components you may want to verify your manifest's
Bundle-SymbolicName still is what it used to be when you cut a new
release.

Stefan

-
To unsubscribe, e-mail: dev-unsubscr...@commons.apache.org
For additional commands, e-mail: dev-h...@commons.apache.org

Re: [Compress]Discussion about COMPRESS-499

[Stefan Bodewig]

On 2019-12-19, Peter Lee wrote:

> Hi all.

> A recent issue COMPRESS-499(
> https://issues.apache.org/jira/projects/COMPRESS/issues/COMPRESS-499)
> discussed about a potential problem in SeekableInMemoryByteChannel.
> Based on the java docs(
> https://docs.oracle.com/en/java/javase/11/docs/api/java.base/java/nio/channels/SeekableByteChannel.html#truncate(long
> )), I believe there are some problems in the  SeekableInMemoryByteChannel .

Agreed.

> The method repositionIfNecessary() may change the position of the channel,
> which may be a bug basing on the docs, and it's called in read and
> truncate. I'm not an expert on SeekableByteChannel, and I'm not sure if
> SeekableInMemoryByteChannel is doing this intentionally or not.

I agree with your suggestion to add tests that verify we adhere to the
SeekableByteChannel contract for (more or less) all methods.

I've looked through the commit history and - as the one who introduced
repositionIfNecessary - believe it is just wrong. Even if existing tests
fail when it gets adjusted this may just indicate I haven't read up on
the interface's contract properly when I wrote the tests. :-)

Stefan

-
To unsubscribe, e-mail: dev-unsubscr...@commons.apache.org
For additional commands, e-mail: dev-h...@commons.apache.org

[compress] What to do with the changed OSGi Bundle Name?

[Stefan Bodewig]

Hi all

due to a change in the parent POM the Bundle-SymbolicName of the
commons-compress JAR changed from org.apache.commons.compress to
org.apache.commons.commons-compress with version 1.18. So we've had a
lot of releases up to 1.17 with one name and two newer releases with a
different name in 1.18 and 1.19.

https://issues.apache.org/jira/browse/COMPRESS-498

Now we've got two options.

1. Call this a bug and revert to org.apache.commons.compress in 1.20.

   This will make users of Compress up to 1.17 happy when they upgrade.
   But it alienates people who used 1.1[89], in particular if they have
   already adapted to the new symbolic name.

2. Call this a mistake but stick to org.apache.commons.commons-compress
   for 1.20 and future releases.

   This will make users of Compress up to 1.17 unhappy when they
   upgrade, but makes upgrades seemless for people using 1.18 or 19.

I'm more or less on the fence here as I'm not an OSGi user myself.

Given that COMPRESS-498 came up more than a year after 1.18 has been
released makes me think we either haven't got many OSGi users, they
don't upgrade Compress frequently or they silently suffer and adapt to
whatever we do.

Stefan

-
To unsubscribe, e-mail: dev-unsubscr...@commons.apache.org
For additional commands, e-mail: dev-h...@commons.apache.org

[compress] What to do with Pack200?

[Stefan Bodewig]

Hi

our travis build also uses the combination

- dist: trusty
  jdk: openjdk-ea

which has started to use OpenJDK 14 a while ago and our build has been
failing there ever since. The reason is
https://openjdk.java.net/jeps/367 - the whole Pack200 stuff has been
removed without any replacement.

AFAICT there is no (official) alternative implementation. There has been
some discussion on the Eclipse lists eighteen months ago but it looks as
if it never came to fruition - mostly because of legal reasons. The same
legal reasons would probably apply if we resurrected the pack200 code
from the atticed Apache Harmony codebase.

If there was an alternative that used the same packages and APIs we
could add a new dependency conditional on JDK being 1.14 or
above.

Without that it looks as if we had to remove the pack200 code from
commons-compress and create a separate artifact for that which simply
would not be built on Java 14+.

Any other ideas?

Stefan

-
To unsubscribe, e-mail: dev-unsubscr...@commons.apache.org
For additional commands, e-mail: dev-h...@commons.apache.org

Re: [compress] What to do with Pack200?

[Stefan Bodewig]

On 2020-01-02, Bruno P. Kinoshita wrote:

> I think if we are going to release it soon, the only option to support java 
> 14+ the would be to remove it. Tell users it may be added back later either 
> as-was or as a separate artefact.

If we want to cut a new release soon, we can just tell people to not use
JDK 14+ when building Compress ;-)

Actually I hope to release 1.20 soonish and it will still target Java 7
- as there is not reason to require more than that right now.  Compress
as it is can be built with any released version of Java (14 is still
early access) newer than Java 6, this is good enough for now.

This question - unlike the question about the OSGi bundle name is more
about what happens after the next release. We could even mention our
plans about pack200 with the 1.20 release notes.

Stefan

-
To unsubscribe, e-mail: dev-unsubscr...@commons.apache.org
For additional commands, e-mail: dev-h...@commons.apache.org

Re: [compress] What to do with the changed OSGi Bundle Name?

[Stefan Bodewig]

On 2020-01-02, Oliver Heger wrote:

> The change of the bundle name does not necessarily break all OSGi users.
> AFAIK, the recommended way to use an OSGi bundle is to import the
> packages, not to require a specific bundle - this use case would not be
> affected by the change of the bundle name.

Ah, I see. This is backed by Julian Reschke's comment
https://issues.apache.org/jira/browse/COMPRESS-498?focusedCommentId=17006749&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-17006749
where he says

,
| same code in two different artefacts
`

would be the problem. So I guess we are confusing the OSGi container if
Compress 1.17 and 1.18 are available as artifacts.

Stefan

-
To unsubscribe, e-mail: dev-unsubscr...@commons.apache.org
For additional commands, e-mail: dev-h...@commons.apache.org

Re: [compress] What to do with Pack200?

[Stefan Bodewig]

On 2020-01-04, Emmanuel Bourg wrote:

> Le 02/01/2020 à 11:06, Stefan Bodewig a écrit :

>> Any other ideas?

> The removal of pack200 isn't an issue for the upcoming release yet since
> we still target Java 7. For now this is just an issue with the CI using
> Java 14+. Maybe we can deal with this by using a Maven profile excluding
> the pack200 classes from the compilation when Java 14+ is used.

This won't work as CompressorStreamFactory depends on the package.

> This leaves some time to find an alternative implementation, I don't
> think Commons Compress will require Java 14+ in the next 5 years.

Me neither.

> FYI I plan to fork the JDK pack200 implementation into a standalone
> project. I've reserved the 'pack200' group on GitHub for this. The code
> isn't published yet

https://github.com/pack200/pack200 looks pretty public to me :-)

> but I've already Mavenized the project and converted the tests to
> JUnit. I'm struggling a bit to get the native code to build.

I vaguely recall there've been some legal strings attached that were
raised when Eclipse discussed ther alternatives for p2 but its been a
while and I haven't re-read the threads, yet.

> The fork will inherit the GPL2 with classpath exception license from
> OpenJDK. I'm not sure Apache projects are allowed to depend on such
> libraries, but that would be odd not to be able to use it since it's
> exactly the same code that we used in the JDK.

I'd say it is fair to call pack200 support an optional feature of
Commons Compress https://apache.org/legal/resolved.html#optional

Stefan

-
To unsubscribe, e-mail: dev-unsubscr...@commons.apache.org
For additional commands, e-mail: dev-h...@commons.apache.org

Re: [compress] What to do with Pack200?

[Stefan Bodewig]

On 2020-01-04, Gary Gregory wrote:

> Java has a service leader mechanism for that. No sense in duplicating it.

Compress already supports that, and IIRC you've added that yourself :-)

Stefan

-
To unsubscribe, e-mail: dev-unsubscr...@commons.apache.org
For additional commands, e-mail: dev-h...@commons.apache.org

Re: [compress] What to do with Pack200?

[Stefan Bodewig]

On 2020-01-05, Gary Gregory wrote:

> Is Pack200 100% Java? Even if not, I think it would be neat to bring it in
> from Apache Harmony as a new module.

What Emmanuel says.

> I think we should consider converting compress into a multi-module
> project and avoid hard dependencies. For folks who want everything, we
> can create a BOM POM.

Time to give the compress-2.0 branch another chance?

Our current ServiceLoader approach does not include auto-detection of
formats, something that is supported by Pack200. So if we want to move
pack200 - or any of the other formats requiring an external lib that
would support auto-detection - out of the main code base we need to
extend the CompressorStreamProvider interface anyway (likely create a
new one). compress-2.0 is completely built around the abstraction of a
(service-loadable) format.

> For the test, the simplest might be to use a JUnit Assume to skip running
> that test on Java > 14.

Yes, we already do so for the tests with optional dependencies.

Stefan

-
To unsubscribe, e-mail: dev-unsubscr...@commons.apache.org
For additional commands, e-mail: dev-h...@commons.apache.org

[compress] Towards a 1.20 Release

[Stefan Bodewig]

Hi all

I'm trying to make the builds work on Jenkins and intend to cut a new
COmpress release soon. Right now I don't see any chance of understanding
COMPRESS-494 or COMPRESS-500 and would address 501 and 502 after the
release.

Also I'd like to postpone the pack200 issue and mention that Compress
1.20 will not build on Java 14 and we we'd be discussing how to deal
with it for 1.21.

Anything that must go into the next release? Any other reason why I
should not proceeed?

Stefan

-
To unsubscribe, e-mail: dev-unsubscr...@commons.apache.org
For additional commands, e-mail: dev-h...@commons.apache.org

Re: [commons-compress] branch master updated: one more javadoc warning

[Stefan Bodewig]

On 2020-01-26, Gary Gregory wrote:

> On Sun, Jan 26, 2020, 07:39  wrote:

>>+ * @return the parsed PAX header

> Plural?

Not sure. I believe it is a single PAX header block holding multiple
header variables - while the code calls those header variable headers to
make things more confusing.

Stefan

-
To unsubscribe, e-mail: dev-unsubscr...@commons.apache.org
For additional commands, e-mail: dev-h...@commons.apache.org

[VOTE] Release Compress 1.20 Based on RC1

[Stefan Bodewig]

This time there are not so many bugs but significant improvements that
people are waiting to get their hands on.

Compress 1.20 RC1 is available for review here:
https://dist.apache.org/repos/dist/dev/commons/compress/
(svn revision 37832)

The tag is here:

https://gitbox.apache.org/repos/asf?p=commons-compress.git;a=tag;h=refs/tags/1.20-RC1
on commit

https://gitbox.apache.org/repos/asf?p=commons-compress.git;a=commit;h=db615d141f01d2575b0d3996030db81ed8541e72

Maven artifacts are here:

https://repository.apache.org/content/repositories/orgapachecommons-/org/apache/commons/commons-compress/1.20/

These are the Maven artifacts and their hashes

7c9990e54ba0c7833054e003a386afd5205452c437501d0c49f4c26154116f44765176346229fbf131f22cb2fba43a5ea11c8c93e57af677841ba6d91e2883d0
  commons-compress-1.20.jar
30634f984b24cfca6151541b611a8b07bcaae2536fef1453bc0a96dccc8c083c188c4d26d5feb12dc33c21cd1da6216af3fb7ab11167b1d8008c432656e1e579
  commons-compress-1.20-javadoc.jar
7a9881a5b705f472805471ca46982cbe71ceceff3d45923cd9db7aee239825c96daed3c52ea4b7a54bca88910ff4659b5cb140c9c411e5c1d8e60bb46dd8a333
  commons-compress-1.20-sources.jar
c7cbc2d7ac2f08f0c10f7b88af87f70c33db12b4d3b8124874e60ccffd5cf2735dc64bc2edf94bb806b283202bdc301dc05cb3fb694afc22f5831788abcd1a8f
  commons-compress-1.20-tests.jar
d63e5ea6195ce5b0b7db7b5d881430dbc68f5b9f37131db1aef4d00ea37bb58793b0382e0af201ec8864b33aa9ca071ddc0abb06b788d4efea097cf970b6c88f
  commons-compress-1.20-test-sources.jar
a7ad5f75c1509aa8d238e583a76fcf7f69fb93c40ac6cf6a3cf33dbb8c5dba98bfc0d81c88815b123d1196d958dd94a79c417e685a3440629d63b2068fe90bc3
  commons-compress-1.20.pom

I have tested this with JDK 8 using Maven 3.5.4

Details of changes since 1.19 are in the release notes:
https://dist.apache.org/repos/dist/dev/commons/compress/RELEASE-NOTES.txt

https://stefan.samaflost.de/staging/commons-compress-1.20/changes-report.html

Site:
https://stefan.samaflost.de/staging/commons-compress-1.20/
  (note Javadocs of 1.20 have not been created right now and this is -
as usual when I cut a release - not the site I'm going to publish
after the release. I'll create a new site once the release date is known).

Japicmp Report (compared to 1.19):
https://stefan.samaflost.de/staging/commons-compress-1.20/japicmp.html

RAT Report:
https://stefan.samaflost.de/staging/commons-compress-1.20/rat-report.html

KEYS:
  https://www.apache.org/dist/commons/KEYS

Please review the release candidate and vote.
This vote will close no sooner that 72 hours from now,
i.e. sometime after 04:00 UTC 31-March 2010

  [ ] +1 Release these artifacts
  [ ] +0 OK, but...
  [ ] -0 OK, but really should fix...
  [ ] -1 I oppose this release because...

Thanks!

-
To unsubscribe, e-mail: dev-unsubscr...@commons.apache.org
For additional commands, e-mail: dev-h...@commons.apache.org

Re: [VOTE] Release Compress 1.20 Based on RC1

[Stefan Bodewig]

this is what you get when you copy-paste from the release
instructions :-)

> This vote will close no sooner that 72 hours from now,
> i.e. sometime after 04:00 UTC 31-March 2010

i.e. sometime after 16:00 UTC 05-Feb-2020

Stefan

-
To unsubscribe, e-mail: dev-unsubscr...@commons.apache.org
For additional commands, e-mail: dev-h...@commons.apache.org

Re: [VOTE] Release Compress 1.20 Based on RC1

[Stefan Bodewig]

On 2020-02-03, Rob Tompkins wrote:

> I’m not sure what changed between yesterday and today, but it seems that the 
> org.apache.commons.compress.OsgiTest is failing due to a 501 being returned 
> from 
> http://repo1.maven.org/maven2/org/ops4j/pax/exam/pax-exam-inject/4.13.1/pax-exam-inject-4.13.1.jar
>  
> 

That should be https rather than http. Maven Central requires TLS by
now.

I don't think Compress configures repositories explicitly, so it might
be the version f Maven?

Stefan

-
To unsubscribe, e-mail: dev-unsubscr...@commons.apache.org
For additional commands, e-mail: dev-h...@commons.apache.org

Re: [VOTE] Release Compress 1.20 Based on RC1

[Stefan Bodewig]

On 2020-02-03, Rob Tompkins wrote:

> If you declare your .m2 directory to be somewhere else, do you see the same 
> failure?

I do.

It looks as if there was an option to explicitly set remote
repositories, I'll try that.

Stefan

-
To unsubscribe, e-mail: dev-unsubscr...@commons.apache.org
For additional commands, e-mail: dev-h...@commons.apache.org

Re: [VOTE] Release Compress 1.20 Based on RC1

[Stefan Bodewig]

On 2020-02-03, Stefan Bodewig wrote:

> On 2020-02-03, Rob Tompkins wrote:

>> If you declare your .m2 directory to be somewhere else, do you see the same 
>> failure?

> I do.

> It looks as if there was an option to explicitly set remote
> repositories, I'll try that.

https://gitbox.apache.org/repos/asf?p=commons-compress.git;a=commitdiff;h=151e0fb3fa241887ae3192cc92c572401c6a182b

works for me.

I'll leave it up to you whether you think this needs to go into the 1.20
release or can remain as a fix post-release.

Stefan

-
To unsubscribe, e-mail: dev-unsubscr...@commons.apache.org
For additional commands, e-mail: dev-h...@commons.apache.org

Re: [VOTE] Release Compress 1.20 Based on RC1

[Stefan Bodewig]

On 2020-02-03, Rob Tompkins wrote:

> I'm fairly indifferent. I would minimally state it in the release
> notes that test fails with that.

After having slept over it, I think it is awkward that a build on a
fresh machine won't work. I'll cancel the vote.

> If you want, I'm happy to RC for 1.20-RC2, if you think we want to go
> that route.

Thanks for the offer.

> Just want to be helpful as opposed to slowing things down.

I know that and it is good, don't worry. :-)

And thank you

  Stefan

-
To unsubscribe, e-mail: dev-unsubscr...@commons.apache.org
For additional commands, e-mail: dev-h...@commons.apache.org

[CANCEL] Release Compress 1.20 Based on RC1

[Stefan Bodewig]

Hi all

as Rob pointed out one of Compress' tests based on Pax Exam fails unless
you happen to have built it once before Maven Central started enforcing
TLS. This is fixed in master and I'll tag a RC2 and call for a new vote.
Probably tomorrow.

Thanks

Stefan

-
To unsubscribe, e-mail: dev-unsubscr...@commons.apache.org
For additional commands, e-mail: dev-h...@commons.apache.org

[VOTE] Release Compress 1.20 based on RC2

[Stefan Bodewig]

The only change compared to RC1 is one to a unit test which wouldn't
pass unless you already had PAX Exam in your local mvn repo.

Compress 1.20 RC2 is available for review here:
https://dist.apache.org/repos/dist/dev/commons/compress/
(svn revision 37873)

The tag is here:

https://gitbox.apache.org/repos/asf?p=commons-compress.git;a=tag;h=refs/tags/1.20-RC2
on commit

https://gitbox.apache.org/repos/asf?p=commons-compress.git;a=commit;h=f7503adfbad8b44eb079d564f2784aeaa034647c

Maven artifacts are here:

https://repository.apache.org/content/repositories/orgapachecommons-1492/org/apache/commons/commons-compress/1.20/

These are the Maven artifacts and their hashes

5d3ce7ae4518c0fe5094b512c5c4825cc213ebc388e1e1e7692475406d5876c893140e5bbaaecb1676262a6b73c7250d40b83653cd03600815693474f92f0fa7
  commons-compress-1.20.jar
9d6682b6e101e41a3f351267ef684fdc1a9c312c28923e0a921799f7c70aff73f04541b4d01a4a8af1b699e0769d406697a47748ef717c606fc902475aa7a911
  commons-compress-1.20-javadoc.jar
1208094341901a776e40f81ef48dc5af406a659f1bbc88593156df4c7d46d963d1c14ba67a4e5295694af51c03882b2a60f01aa56973b3d8e0d55f8977eaff75
  commons-compress-1.20-sources.jar
c1df5f8a951ac41f65e62cc48d6c7d19cdb484073d03bd36639b4f383db8f3cb0cf0b587643112f4a3a82ee56b109f8d7213bdd22d648459f54cc8b100de9c7d
  commons-compress-1.20-tests.jar
64d535998785bc884da4698b0445f13656a5f5fad3963c3ca26a7d978e5b1e039080cef97eb4c9de5d1bfb36b75e2130f8b3b934c777918e0a28a34c807d57cc
  commons-compress-1.20-test-sources.jar
a7ad5f75c1509aa8d238e583a76fcf7f69fb93c40ac6cf6a3cf33dbb8c5dba98bfc0d81c88815b123d1196d958dd94a79c417e685a3440629d63b2068fe90bc3
  commons-compress-1.20.pom

I have tested this with JDK 8 using Maven 3.5.4

Details of changes since 1.19 are in the release notes:
https://dist.apache.org/repos/dist/dev/commons/compress/RELEASE-NOTES.txt

https://stefan.samaflost.de/staging/commons-compress-1.20/changes-report.html

Site:
https://stefan.samaflost.de/staging/commons-compress-1.20/
  (note Javadocs of 1.20 have not been created right now and this is -
as usual when I cut a release - not the site I'm going to publish
after the release. I'll create a new site once the release date is
known. This is the same site as the one for RC1)

Japicmp Report (compared to 1.19):
https://stefan.samaflost.de/staging/commons-compress-1.20/japicmp.html

RAT Report:
https://stefan.samaflost.de/staging/commons-compress-1.20/rat-report.html

KEYS:
  https://www.apache.org/dist/commons/KEYS

Please review the release candidate and vote.
This vote will close no sooner that 72 hours from now,
i.e. sometime after 05:30 UTC 08-Feb 2020

  [ ] +1 Release these artifacts
  [ ] +0 OK, but...
  [ ] -0 OK, but really should fix...
  [ ] -1 I oppose this release because...

Thanks!

-
To unsubscribe, e-mail: dev-unsubscr...@commons.apache.org
For additional commands, e-mail: dev-h...@commons.apache.org

Re: [VOTE] Release Compress 1.20 based on RC2

[Stefan Bodewig]

Making my own vote explicit.

We still lack another +1, though.

On 2020-02-05, Stefan Bodewig wrote:

> Please review the release candidate and vote.
> This vote will close no sooner that 72 hours from now,
> i.e. sometime after 05:30 UTC 08-Feb 2020

+1

Stefan

-
To unsubscribe, e-mail: dev-unsubscr...@commons.apache.org
For additional commands, e-mail: dev-h...@commons.apache.org

[RESULT] Release Compress 1.20 based on RC2

[Stefan Bodewig]

With binding +1s by Gary, Rob and myself the vote has passed.

I'll start with publishing the artifacts and will announce the release
once the mirrors have had time to catch up.

Many thanks

 Stefan

-
To unsubscribe, e-mail: dev-unsubscr...@commons.apache.org
For additional commands, e-mail: dev-h...@commons.apache.org

Re: [VOTE] Release Compress 1.20 based on RC2

[Stefan Bodewig]

On 2020-02-08, Gary Gregory wrote:

> But next time:
> - On the site: it would be nice to keep all "What's new in 1.zzz?" sections
> so users can see what's new based on the version _they_ currently have.

Really, this is going to become a pretty long list. I've resurrected all
sections since we started providing them back with 1.6 so we'll see how
the page is going to look like :-)

> - mvn javadoc:javadoc outputs LOTS of errors.

Not a single one for me (building with Java 8).

Stefan

-
To unsubscribe, e-mail: dev-unsubscr...@commons.apache.org
For additional commands, e-mail: dev-h...@commons.apache.org

[ANN] Apache Commons Compress 1.20 Released

[Stefan Bodewig]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

The Apache Commons Team is pleased to announce the release of Apache
Commons Compress 1.20.

Apache Commons Compress software defines an API for working with
compression and archive formats.  These include: bzip2, gzip, pack200,
lzma, xz, Snappy, traditional Unix Compress, DEFLATE, DEFLATE64, LZ4,
Brotli, Zstandard and ar, cpio, jar, tar, zip, dump, 7z, arj.

Compress 1.20 adds random access when reading 7zip archives, support for
reading sparse tar archives and support for split zip archives - among
other improvements and fixes.

During the development of 1.20 we learned that Commons Compress 1.18
changed the symbolic name of the OSGi bundle by accident. We have
decided to stick with the "new" symbolic bundle name.

Commons Compress 1.20 like any version of Commons Compress since 1.3 can
not be built from sources using Java 14 as Java 14 removes support for
the Pack200 format. We will address this issue with the next release.

Source and binary distributions are available for download from the
Apache Commons download site:

https://commons.apache.org/proper/commons-compress/download_compress.cgi

When downloading, please verify signatures using the KEYS file available
at the above location when downloading the release.

Changes in this version include:

Fixed Bugs:
o SevenZFile could throw NullPointerException rather than
  IOException for certain archives. In addition it now handles
  certain empty archives more gracefully.
  Issue: COMPRESS-492.
o Deflate64CompressorInputStream.read would return 0 for some
  inputs in violation of the InputStream.read contract.
  Issue: COMPRESS-491.
o SeekableInMemoryByteChannel's truncate didn't set position
  according to the spec in an edge case.
  Issue: COMPRESS-499.
o BZip2CompressorInputStream now incorporates a similar patch as
  the one that fixed CVE-2019-12900 in libbzip2.

  Commons Compress has not been vulnerable to this CVE as it
  would have rejected a file with too many selectors. With this
  patch Commons Compress will be able to read certain archives
  that would have caused errors in Compress 1.19.
  Thanks to Joseph Allemandou.

Changes:
o Update optional library com.github.luben:zstd-jni from
  1.4.0-1 to 1.4.4-7.
  Issue: COMPRESS-493.
o Update tests from org.apache.felix:org.apache.felix.framework
  6.0.2 to 6.0.3.
o SevenZFile can now recover from a certain corruption that
  seems to happen occasionally when split archives are created.
  Issue: COMPRESS-497.
  Thanks to Stefan Schlott.
o Added random access support to SevenZFile.
  Issue: COMPRESS-342.
  Thanks to Peter Alfred Lee.
o Added support for split ZIP archives.
  Issue: COMPRESS-477.
  Thanks to Peter Alfred Lee.
o Added support for reading sparse entries to the TAR package.
  Issue: COMPRESS-124.
  Thanks to Peter Alfred Lee.
o Update JUnit from 4.12 to 4.13.

Removed:
o Removed the extraction code from the example CLI class inside
  of the SevenZ package. Not only is it superseeded by the
  examples package, its implementation was vulnerable to the
  ZipSlip attack.
  Issue: COMPRESS-495.

For complete information on Commons Compress, including instructions
on how to submit bug reports, patches, or suggestions for improvement,
see the Apache Commons Compress website:

https://commons.apache.org/compress/
-BEGIN PGP SIGNATURE-
Version: GnuPG v1

iEYEARECAAYFAl4/EY4ACgkQohFa4V9ri3KIyQCg3Dhv6iN/mBjjyLi3DPuM7MXr
gEsAn2qPuYbQp9AtHxGaBWoAv9RI3eKe
=CX2W
-END PGP SIGNATURE-

-
To unsubscribe, e-mail: dev-unsubscr...@commons.apache.org
For additional commands, e-mail: dev-h...@commons.apache.org

[compress] javadoc (was Re: [VOTE] Release Compress 1.20 based on RC2)

[Stefan Bodewig]

On 2020-02-08, Gary Gregory wrote:

> On Sat, Feb 8, 2020 at 11:50 AM Stefan Bodewig  wrote:

>> On 2020-02-08, Gary Gregory wrote:

>>> - mvn javadoc:javadoc outputs LOTS of errors.

>> Not a single one for me (building with Java 8).

> Did you run 'mvn javadoc:javadoc'?

Yes

> Please see https://pastebin.com/ZLVdrEhr

Th eoutput is talking about modules, so you must be using something
newer than Java8, which may explain the difference. We may need to
change the configuration to get rid of

,
| javadoc: error - The code being documented uses modules but the packages 
defined in https://docs.oracle.com/javase/9/docs/api/ are in the unnamed module
`

but I have no idea how. The whole bunch of error messages prior to that
seem unrelated to Commons Compress. The rest are HTML5 warnings, I'll
look into them.

Stefan

-
To unsubscribe, e-mail: dev-unsubscr...@commons.apache.org
For additional commands, e-mail: dev-h...@commons.apache.org

Re: [Compress] Zip Files: History, Explanation and Implementation

[Stefan Bodewig]

On 2020-03-07, Peter Lee wrote:

> I'm planning to build a pure Java deflater/inflater on my own. Believe this
> may help a lot.

Compress contains a pure Java Deflate64 deflater, which also is a
"normal" deflater by defintion. You may want to take a look at it.

When I implemented the LZ4 encoder I leaned on Peter Deutsch's
description of the LZ77 part of the deflate algorithm in RFC1951 but I
believe the original LZ4 code contains a faster matching algorithm than
that - zlib itself probably does so as well by now.

Stefan

-
To unsubscribe, e-mail: dev-unsubscr...@commons.apache.org
For additional commands, e-mail: dev-h...@commons.apache.org

Re: [geometry] distribution svn url

[Stefan Bodewig]

On 2020-03-08, Matt Juntunen wrote:

> I'm creating a dist-archive module for commons-geometry using
> commons-rng as a template [1]. However, when I build the project it
> fails with an errors saying that the url
> https://dist.apache.org/repos/dist/dev/commons/geometry does not
> exist, which is indeed the case. Browsing that repo, I can see that
> directories exist for rng, collections, math and other released
> commons projects. How are these directories created?

Most of them are probably older than the commons release plugin and have
been created manually.

You should be able to do

svn mkdir https://dist.apache.org/repos/dist/dev/commons/geometry

yourself.

The release plugin should probably check whether the directory exists
and create it if necessary.

Stefan

-
To unsubscribe, e-mail: dev-unsubscr...@commons.apache.org
For additional commands, e-mail: dev-h...@commons.apache.org

Re: [geometry] distribution svn url

[Stefan Bodewig]

On 2020-03-08, Matt Juntunen wrote:

> I don't currently have permissions for that. Is someone able to create
> "geometry" and "numbers" directories in there?

Done.

But I suspect you won't have permission to upload a distribution there
either.

Stefan

-
To unsubscribe, e-mail: dev-unsubscr...@commons.apache.org
For additional commands, e-mail: dev-h...@commons.apache.org

Re: [VOTE] Release Apache Commons Configuration 2.7 based on RC2

[Stefan Bodewig]

On 2020-03-09, Rob Tompkins wrote:

> We have fixed quite a few bugs and added some significant enhancements since 
> Apache Commons Configuration 2.6 was released, so I would like to release 
> Apache Commons Configuration 2.7.

+1

Stefan

-
To unsubscribe, e-mail: dev-unsubscr...@commons.apache.org
For additional commands, e-mail: dev-h...@commons.apache.org

Re: [Compress]Add some easy-to-use APIs for Zip and other archivers

[Stefan Bodewig]

On 2020-03-09, Peter Lee wrote:

> I'm thinking about adding some easy-to-use APIs for Zip. Currently I got
> some ideas :
> 1. Add extractAll(String targetPath) in ZipFile : extract all the files to
> the specific directory.
> 2. Add getInputStream(String fileName) in ZipFile : get the input stream
> for a file by name.

> And I believe these could also work in other archivers like tar, 7z and
> some other format.

> Do you think if this is a good idea or not?

There is https://issues.apache.org/jira/browse/COMPRESS-118 which at one
point in time was the issue with the most votes IIRC.

Around the time the ZipSlip vulnerability had to be fixed in various
projects we discussed adding such a high-level API to Compress. You can
find some code that I sketched back then inside of the examples package.

At that time the majority of Compress developers felt a high-level API
was out-of-scope for Compress and many people felt effort should rather
be spent at Commons VFS. See the threads

https://lists.apache.org/thread.html/0b86f62127f771a8ac3b6357a1c1bdb6b4d21bf18bc4a30d0b3650c8%40%3Cdev.commons.apache.org%3E

https://lists.apache.org/thread.html/bb205705291c00ac8d36516b287a14d814dc5f1fe4d422f3f5c0db28%40%3Cdev.commons.apache.org%3E

Stefan

-
To unsubscribe, e-mail: dev-unsubscr...@commons.apache.org
For additional commands, e-mail: dev-h...@commons.apache.org

Re: [commons-compress] branch master updated: Update my(Peter Lee) personal information in pom

[Stefan Bodewig]

welcome :-)

Stefan

-
To unsubscribe, e-mail: dev-unsubscr...@commons.apache.org
For additional commands, e-mail: dev-h...@commons.apache.org

Re: [Vote] Format of "git" tags

[Stefan Bodewig]

On 2020-04-01, Gilles Sadowski wrote:

> Alternatives (using the yet-to-be-created tag for the release
> candidate of the first beta version of [Numbers] as an example):

>  [ ] Option 1: NUMBERS_1_0_BETA1_RC1
>  [ ] Option 2: commons-numbers-1.0-beta1-rc1
>  [ ] Option 3: commons-numbers_v1.0-beta1_rc1

+0 to each of them as long as
http://commons.apache.org/releases/prepare.html gets updated with the
outcome.

Stefan

-
To unsubscribe, e-mail: dev-unsubscr...@commons.apache.org
For additional commands, e-mail: dev-h...@commons.apache.org

Re: [Vote] Format of "git" tags

[Stefan Bodewig]

On 2020-04-01, Gary Gregory wrote:

> The docs should also make sure that release tags are in the form rel/...
> which makes them read-only.

So far I've created a new tag under rel/ for the RC tag when the vote
has been accepted. So only "real" releases end up there.

If we want to create all our tags there, things may look a bit
confusing. But I could live with that as well.

Stefan

-
To unsubscribe, e-mail: dev-unsubscr...@commons.apache.org
For additional commands, e-mail: dev-h...@commons.apache.org

Re: what became of beanshell in Apache commons?

[Stefan Bodewig]

On 2020-04-24, Peter Kovacs wrote:

> Now I figured that beanshell was included into apache copmmons, which we also 
> use.

The move has been proposed but actually never actually happened IIRC.

https://www.mail-archive.com/dev@taverna.incubator.apache.org/msg00224.html
is the best reference (more than five years ago, mind you) I could find.

Stefan

-
To unsubscribe, e-mail: dev-unsubscr...@commons.apache.org
For additional commands, e-mail: dev-h...@commons.apache.org

Re: [COMPRESS] Travis build fail with JDK14

[Stefan Bodewig]

On 2020-05-13, Peter Lee wrote:

>  Hi,all

> The travis build of Compress is failing now cause the openjdk14 was added
> to travis.yml recently. The reason is the Pack200 was removed from JDK14
> and there was a discussion about it in January. Emmanuel is working on his
> replacement project(https://github.com/pack200/pack200) but not finished
> yet. Seems we have no good replacement for now.

> I'm thinking we should disable openjdk14 in travis before we have find a
> solution for this. WDYT?

I'm fine with disabling the travis build for now.

Had a quick look through JIRA as I was totally sure there must be an
issue tracking this, but it seems I haven't created any. Anyway with the
last release announcement I promised we'd deal with JDK14 for the next
release - one way or the other. :-)

Stefan

-
To unsubscribe, e-mail: dev-unsubscr...@commons.apache.org
For additional commands, e-mail: dev-h...@commons.apache.org

Re: [COMPRESS] Travis build fail with JDK14

[Stefan Bodewig]

On 2020-05-14, Peter Lee wrote:

> Unfortunately it seems commons-compress can not build on Java14. Maybe we
> should provide a statement about this in README or somewhere else?

Done.

Also I added something to the "known limitations" page and will
re-generate the website.

Stefan

-
To unsubscribe, e-mail: dev-unsubscr...@commons.apache.org
For additional commands, e-mail: dev-h...@commons.apache.org

Re: Jenkins build is back to normal : Commons-Compress-Windows » Apache Commons Compress #731

[Stefan Bodewig]

there doesn't seem to be a JDK 7 for Windows in our Jenkins farm anymore
https://cwiki.apache.org/confluence/display/INFRA/JDK+Installation+Matrix

I've made the Windows build use JDK 8 and we now rely on the Linux build
to catch JDK 7 incompatibilites.

Stefan

-
To unsubscribe, e-mail: dev-unsubscr...@commons.apache.org
For additional commands, e-mail: dev-h...@commons.apache.org

Re: [commons-compress] branch master updated: COMPRESS-529 : throws IOException if non-number exists in pax header

[Stefan Bodewig]

On 2020-05-26,  wrote:

> -public void addPaxHeader(String name,String value) {
> - processPaxHeader(name,value);
> +public void addPaxHeader(String name, String value) throws IOException {
> +processPaxHeader(name, value);

no, we can't do that. Adding a checked exception to a public method
breaks source compatibility.

Stefan

-
To unsubscribe, e-mail: dev-unsubscr...@commons.apache.org
For additional commands, e-mail: dev-h...@commons.apache.org

Re: [commons-compress] branch master updated: COMPRESS-530 : skip non-number when parsing pax header

[Stefan Bodewig]

On 2020-05-26,  wrote:

>+// COMPRESS-530 : skip non-number chars
>+if (ch < '0' || ch > '9') {
>+continue;
>+}

if this ever happens, doesn't that mean the PAX header is malformed? In
that case may it be better to throw an IOException?

Stefan

-
To unsubscribe, e-mail: dev-unsubscr...@commons.apache.org
For additional commands, e-mail: dev-h...@commons.apache.org

Re: [commons-compress] branch master updated: COMPRESS-529 : throws IOException if non-number exists in pax header

[Stefan Bodewig]

On 2020-05-27, Peter Lee wrote:

> Oops, sorry about that.

No big deal. Better we detect that now than at the point when we want to
cut the release.

>  Will undo all the commits.

It may be possible to keep most of your code changes without breaking
the public API. I must admit I haven't lloked at the patch in full
detail, yet.

Stefan

-
To unsubscribe, e-mail: dev-unsubscr...@commons.apache.org
For additional commands, e-mail: dev-h...@commons.apache.org

Re: [commons-compress] branch master updated: COMPRESS-530 : skip non-number when parsing pax header

[Stefan Bodewig]

On 2020-05-27, Peter Lee wrote:

> Did some googles, can't find too much but  this :
> https://www.systutorials.com/docs/linux/man/5-star/
> And it says :
>> Each record starts with a a decimal length field. The length includes the
>> total size of a record including the length field itself and the trailing
>> new line.

For some reason I always end up in either GNU tar's info or the FreeBDS
man page :-). In this case

https://www.freebsd.org/cgi/man.cgi?query=tar&sektion=5

see "Pax Interchange Format"

,
| The extended attributes themselves are stored as a series of text-format
| lines encoded in the portable UTF-8 encoding. Each line consists of a
| decimal number, a space, a key string, an equals sign, a value string,
| and a new line.
`

> Seems we should throw a exception.

+1 - likely an IOException.

Stefan

-
To unsubscribe, e-mail: dev-unsubscr...@commons.apache.org
For additional commands, e-mail: dev-h...@commons.apache.org

Re: [commons-compress] branch master updated: minor typos cleanup

[Stefan Bodewig]

On 2020-06-01,  wrote:

> The following commit(s) were added to refs/heads/master by this push:
>  new 42b6aa4  minor typos cleanup
> - * @throws IllegalArgumentException if the {@link 
> TarArchiveOutputStream#longFileMode} equals
> - *  {@link 
> TarArchiveOutputStream#LONGFILE_ERROR} and the file
> - *  name is too long

why are you removing the javadocs about throwing on long names?

Stefan

-
To unsubscribe, e-mail: dev-unsubscr...@commons.apache.org
For additional commands, e-mail: dev-h...@commons.apache.org

Re: [commons-compress] branch master updated: minor typos cleanup

[Stefan Bodewig]

On 2020-06-02, Peter Lee wrote:

> Oops, I was looking the commit and found two @throws
> IllegalArgumentException, and I was thinking this is a duplicated throws
> caused by copy-paste. And I was so much foolish that I deleted it without
> any invesgating into the code. Really sorry about this.

No problem and thank you for fixing it so quickly.

Stefan

-
To unsubscribe, e-mail: dev-unsubscr...@commons.apache.org
For additional commands, e-mail: dev-h...@commons.apache.org

Re: some questions about commons projects.

[Stefan Bodewig]

On 2020-06-12, Xeno Amess wrote:

> Hi.

>>> 2. How are commons projects related?

>> They are not necessarily related.  Usually it is considered
>> a feature if a component has zero dependency (as it is was
>> easier to avoid "JAR hell").
>> However, there are also drawbacks, e.g. duplicating functionality
>> (and work) needed by several components.

> Something was not quite right about this.  For example, in
> commons-vfs, we just use commons-lang3 as a dependency.  But in
> commons-email, we fork some of utility functions in commons-lang3 as a
> java class in commons-email.  Which is the right way, or a more
> commonly accepted way in commons projects?

Neither is right or wrong in general, it all depends on the context.

VFS has a bunch of dependencies anyway, so adding a dependency on
commons-lang3 is no big deal. Email may have decided to copy a few
classes in order to avoid a depencency.

Another example I'm aware of is Compress which has copied code from
commons-io (basically parts of IOUtils) in order to avoid a
dependency. And it has copied classes developed in Comnpress to Codec
(some of the more exotic hashes/checksums) because they seemed to fit
there - but Compress didn't want to pay for this by adding a dependency.

One thing that may not have become clear from Gilles' great answer: many
decisions are made by the people who work on a concrete code base while
people only active elsewhere get out of the way. There are some common
grounds - rules that are common to all Apache projects mostly - but the
components operate rather autonomously.

Stefan

-
To unsubscribe, e-mail: dev-unsubscr...@commons.apache.org
For additional commands, e-mail: dev-h...@commons.apache.org

Re: some questions about commons projects.

[Stefan Bodewig]

On 2020-06-12, Gilles Sadowski wrote:

> 2020-06-12 15:52 UTC+02:00, Xeno Amess :

>> But if Apache Commons is thought to be a whole project, I do think
>> the relationship between each of its components should be enforced.

The Commons project is the legal entity that binds people with similar
interest in creating reusable components.

This group of people involves some who work on lots of components and
may strive for more standardization and others who are mostly interested
in one component and don't see any benefit in changing the placement of
braces in "their" component just because people who never worked on
"their" component liked a different style better.

Realistically there is far less cross polination between components than
you may expect. Things lice BCEL or Weaver need people who are familiar
with Java byte code. The Math components require a deeper understanding
of certain mathematical concepts than many coders have. Crypto, Compress
and others attract people with certain interests.

> Some regular contributors (or ancient contributors for
> old/mature components) will veto touching the code
> just for the sake of standardization.

That group likely includes me. Well, argue against not veto, actually.

>> For example, we might start from trying to use a same code style
>> formatter.

If you really want to discuss this we should split out a different
thread rather than polluting this one. It would probably lead to an
exchange of arguments and an agreement to disagree.

Stefan

-
To unsubscribe, e-mail: dev-unsubscr...@commons.apache.org
For additional commands, e-mail: dev-h...@commons.apache.org

Re: [Compress] COMPRESS-538 : about Zip64

[Stefan Bodewig]

On 2020-06-28, Peter Lee wrote:

> Currently we will add a Zip64 extra field for the entries with uncompressed
> size unspecified. And we will update the zip64 extra field in
> ZipArchiveOutputStream.rewriteSizesAndCrc a little bit : if we actually
> doesn't need a Zip64 extra, we will not remove it. Instead we keep it in
> Local File Header, and we update the 'Zip Version Needed to Extract' to the
> ones without zip64. Then we removed the extra field(after it's already
> written to the zip archive) to avoid the zip64 written to the Central
> Directory.

> Not sure why we are doing it like that.

We only know the sizes once we've written the entry's content to the
archive. The content is behind the extra field. Removing the extra field
from the archive would require us move all content a few bytes towards
the front of the archive. Potentially a lot of effort which will only
gain us a few bytes.

Stefan

-
To unsubscribe, e-mail: dev-unsubscr...@commons.apache.org
For additional commands, e-mail: dev-h...@commons.apache.org

Re: [all] release validation

[Stefan Bodewig]

On 2020-07-12, Rob Tompkins wrote:

> given the consistency of the signatures from the plugins…do we need to
> check them for releases anymore?

Yes, please. Not everybody uses the plugins and even if everybody did a
misconfiguration could be pulling in the wrong key or a key not
available from the expected download location.

Stefan

-
To unsubscribe, e-mail: dev-unsubscr...@commons.apache.org
For additional commands, e-mail: dev-h...@commons.apache.org

[fileupload] Re: A release train...

[Stefan Bodewig]

On 2020-07-18, Merbin J Anselm wrote:

> Well. Commons Fileupload's last release was in December 2018 and it has
> been released at least once a year before that. My thoughts were on this
> line

Well, the question is whether there have been changes that would warrant
a new release at all. It hasn't seen that mayn commits the past eighteen
months.

I just had a very quick look at the commits since the last release and
one of the first things done was bumping the major version. So if
anybody cut a release from the master branch it would not be compatible
with 1.4 at all. Not sure this would help anybody.

As far as changes go I see your (Merbin's) FILEUPLOAD-274, adding
support for Jakarta API coordinates and some performance
improvements. I'm not sure whether this is all the people looking after
fileupload (I'm not one of them) had planned for a new major version.

Stefan

-
To unsubscribe, e-mail: dev-unsubscr...@commons.apache.org
For additional commands, e-mail: dev-h...@commons.apache.org

Re: [all] Dependabot PRs

[Stefan Bodewig]

On 2020-07-22, Rob Tompkins wrote:

> I’m happy to merge them….will get to them by tomorrow morning ok?

TBH I'd prefer to turn them off and reject the PRs.

Personally I don't see any value for our downstream users if we update
our dependencies without actually needing an update - with the exception
of security updates. I don't like the idea of forcing our users to
update a different dependency just because they update our component, it
should be their choice when to update what.

Of course this is just my opinion and I'm not exactly known as somebody
who embraces the idea of automatic resolution of transitive dependencies
in the first place ;-)

Stefan

-
To unsubscribe, e-mail: dev-unsubscr...@commons.apache.org
For additional commands, e-mail: dev-h...@commons.apache.org

Re: [all]should we really allow denpabot upgrade a dependency that changes major version?

[Stefan Bodewig]

To answer the question of your subject: my opinion is a very strong NO.

Stefan

-
To unsubscribe, e-mail: dev-unsubscr...@commons.apache.org
For additional commands, e-mail: dev-h...@commons.apache.org

Re: [commons-compress] branch master updated: Enable GitHub Dependabot.

[Stefan Bodewig]

I hope anybody sees this message.

Can we please discuss this per component? I personally do like the idea
of dependabot for applications but feel it is completly wrong for
libraries and would prefer to not use it.

Stefan

-
To unsubscribe, e-mail: dev-unsubscr...@commons.apache.org
For additional commands, e-mail: dev-h...@commons.apache.org

Re: [ALL] CI builds

[Stefan Bodewig]

On 2020-07-22, Gary Gregory wrote:

> My main driver is that we already use GitHub for source mirroring and PRs,
> so it feels better to me to have builds in the same place.

My preference would be for using less of github rather than more. But
I'm probably alone with that.

-0 on defaulting to github actions (my preference would be defaulting to
Apache infrastructure CI systems).

> I propose we default to GitHub while allowing each component to do whatever
> it wants. Specifically, I would like to drop Travis CI and use GitHub where
> both are used by a component.

+1 for "allowing each component to do whatever it wants" including
sticking to Travis and/or setting up Jenkins builds.

Stefan

-
To unsubscribe, e-mail: dev-unsubscr...@commons.apache.org
For additional commands, e-mail: dev-h...@commons.apache.org

Re: [VOTE] Create additional mailing lists for automated posts

[Stefan Bodewig]

On 2020-07-23, Gilles Sadowski wrote:

> If I'm not mistaken, the issues@ ML was intended to keep one
> posted of and reactive on a human discussion happening on
> JIRA.  With the advent of JIRA-GitHub integration, the ratio of
> auto-generated messages relayed through that channel has
> exploded, with literally hundreds of redundant messages per
> week (or in a single day, today).

Most of them have been created by humans who open pull requests or
comment on them.

I agree that it is unfortunate we get each message twice for a github PR
that is linked to a JIRA issue and it would be nice if this could be
disabled somehow - but that's probably not possible without volunteering
to improve the tooling around the github/JIRA bridge. This (the
duplicate messages) would not be solved by spillting the MLs - you'd
still see each message twice if you subscribe to both lists.

> Could we have a ML dedicated to bot-generated messages

For things like dependabot I doubt we can really tell PRs opened by a
bot from those opened by a human.

> Specifically, I propose that
> github-iss...@commons.apache.org
> be set up for relaying GitHub generated posts (like comments,
> PRs merging, and so on) and that
> iss...@commons.apache.org
> returns to its original purpose (only).

Personally I don't care as I'd probably subscribe to the new list as
well and route all messages to the same place. If it helps with your
workflow then I'm +0.

For components that have github PRs enabled a PR not linked to a JIRA
issue could be missed by somebody not subscribed to github-issues -
which is not a problem as long as anybody who cares for the component is
subscribed.

Stefan

-
To unsubscribe, e-mail: dev-unsubscr...@commons.apache.org
For additional commands, e-mail: dev-h...@commons.apache.org

Re: [ALL] CI builds

[Stefan Bodewig]

On 2020-07-23, Stefan Bodewig wrote:

> My preference would be for using less of github rather than more. But
> I'm probably alone with that.

Of course I'm not. Sorry Gilles. :-)

Stefan

-
To unsubscribe, e-mail: dev-unsubscr...@commons.apache.org
For additional commands, e-mail: dev-h...@commons.apache.org

Re: [all] Dependabot PRs

[Stefan Bodewig]

On 2020-07-23, Oliver Heger wrote:

> Am 22.07.20 um 18:28 schrieb Stefan Bodewig:
>> On 2020-07-22, Rob Tompkins wrote:

>>> I’m happy to merge them….will get to them by tomorrow morning ok?

>> Personally I don't see any value for our downstream users if we update
>> our dependencies without actually needing an update - with the exception
>> of security updates. I don't like the idea of forcing our users to
>> update a different dependency just because they update our component, it
>> should be their choice when to update what.

> Stefan has a valid point here IMHO.

Thank you ;-)

I intend to raise a separate thread for this later as I'm afraid this
thread has been burnt by a different discussion.

> From out user's POV, our components are in some sense "more
> compatible" if they reference the oldest possible version of a
> dependency rather than the newest one.

+1

Stefan

-
To unsubscribe, e-mail: dev-unsubscr...@commons.apache.org
For additional commands, e-mail: dev-h...@commons.apache.org

[compress] not reading archive stream completely

[Stefan Bodewig]

Hi all,

COMPRESS-202 and COMPRESS-206 only talk about TAR but something similar
aplies at least to ZIP as well: once we detect that an archive doesn't
contain any more entries, we stop reading the input stream, even if it
contains more stuff that is part of the archive.  This causes problems
for use cases where the stream holds interesting data behind the
archive.

I'm a bit torn between two approaches - and it is quite possible I'm
overlooking even more alternatives.

(1) As soon as we detect there are no more entries, we immediately read
the remainder and consume all of the stream that made up the archive.
At least for ZIP and TAR this is possible as getNextEntry "knows" when
it has seen the last entry.

(2) Add additional "readRemainderOfArchive() throws IOException" method
in ArchiveInputStream (or just those of the affected formats) that could
be invoked any time and consume as much of the stream as belongs to the
archive.

Alternative (1) is somewhat breaking backwards compatibility - but only
for some contrieved cases AFAICT.  Alternative (2) would be useful in a
case where the user isn't interested in the rest of an archive's content
after finding an entry but wants to consume it completely.

I realize the two alternative could be implemented at the same time
where the most naive implementation of readRemainderOfArchive simply
reads entries until getNextEntry returns null.

Any thoughts?

Stefan

-
To unsubscribe, e-mail: dev-unsubscr...@commons.apache.org
For additional commands, e-mail: dev-h...@commons.apache.org

Re: svn commit: r1430097 - in /commons/proper/compress/trunk/src/main/java/org/apache/commons/compress/archivers/zip: ZipEightByteInteger.java ZipLong.java ZipShort.java

[Stefan Bodewig]

Hi Julius,

can you please record the changes in src/chanhes/changes.xml as well -
and add yourself as developer to the POM if you feel like it?

Thanks

Stefan

-
To unsubscribe, e-mail: dev-unsubscr...@commons.apache.org
For additional commands, e-mail: dev-h...@commons.apache.org

Re: [GUMP@vmgump]: Project commons-email (in module apache-commons) failed

[Stefan Bodewig]

On 2013-01-08, Gump wrote:

>   at java.lang.Short.parseShort(Short.java:143)
>   at 
> org.powermock.modules.junit4.common.internal.impl.VersionCompatibility.getJUnitVersion(VersionCompatibility.java:40)

...

>   initializationError(org.apache.commons.mail.MultiPartEmailTest): Value out 
> of range. Value:"08012013" Radix:10

Oompf.

Gump sets JUnit's version to the current date while building it (right
now in the format DDMM but this will change soonish).  This doesn't
fit into a short.

Is there any way around this version check or do we need to change the
way Gump builds JUnit to make it work?

Stefan

-
To unsubscribe, e-mail: dev-unsubscr...@commons.apache.org
For additional commands, e-mail: dev-h...@commons.apache.org

Re: [GUMP@vmgump]: Project commons-email (in module apache-commons) failed

[Stefan Bodewig]

On 2013-01-08, Thomas Neidhart wrote:

> I would like to keep powermock, but we could disable the tests in question
> until the version format changes?

We can split the commons-email build into two - one that builds and one
that runs the tests - and disable nagging for the test build until the
JUnit build deals with versions differently.

I hope to find time to look into this (setting JUnit's version in Gump)
tomorrow.

Stefan

-
To unsubscribe, e-mail: dev-unsubscr...@commons.apache.org
For additional commands, e-mail: dev-h...@commons.apache.org

Re: [GUMP@vmgump]: Project commons-email (in module apache-commons) failed

[Stefan Bodewig]

On 2013-01-09, Gump wrote:

> Tests in error:
>   initializationError(org.apache.commons.mail.MultiPartEmailTest): For input 
> string: "12-SNAPSHOT"

getting closer.  When you build JUnit from github's master the version
right now is 4.12-SNAPSHOT which powermock doesn't seem to handle
either.

I hope to have stripped the SNAPSHOT qualifier for the next run.

Stefan

-
To unsubscribe, e-mail: dev-unsubscr...@commons.apache.org
For additional commands, e-mail: dev-h...@commons.apache.org

Re: svn commit: r1430097 - in /commons/proper/compress/trunk/src/main/java/org/apache/commons/compress/archivers/zip: ZipEightByteInteger.java ZipLong.java ZipShort.java

[Stefan Bodewig]

On 2013-01-09, Julius Davies wrote:

> On Mon, Jan 7, 2013 at 9:16 PM, Stefan Bodewig  wrote:

>> can you please record the changes in src/chanhes/changes.xml as well -
>> and add yourself as developer to the POM if you feel like it?

> Thanks for the tip.  Done!

Great, thanks

   Stefan

-
To unsubscribe, e-mail: dev-unsubscr...@commons.apache.org
For additional commands, e-mail: dev-h...@commons.apache.org

Re: [VFS]

[Stefan Bodewig]

On 2013-01-10, Gary Gregory wrote:

> The new HDFS provider only works on Linux and it works for me on Ubuntu
> (via VirtualBox on Win7). How can Gump be fixed?

vmgump is Ubuntu 10.04 which may be too old, we also have FreeBSD and
MacOS nodes.

One option may be to Check for the proper environment inside the test
and skip the "bad" envs via Assume.

Stefan

-
To unsubscribe, e-mail: dev-unsubscr...@commons.apache.org
For additional commands, e-mail: dev-h...@commons.apache.org

Re: [GUMP@vmgump]: Project commons-compress-test (in module apache-commons) failed

[Stefan Bodewig]

On 2013-01-10, Julius Davies wrote:

> This is one my unit tests that I recently added that's failing.

They fail on my machine as well (GMT+1 right now) and create times one
hour different from vmgump which runs in GMT.

The times that are different from the expected times are the times taken
directly from ZipEntry and not those taken from the extra field.

Stefan

-
To unsubscribe, e-mail: dev-unsubscr...@commons.apache.org
For additional commands, e-mail: dev-h...@commons.apache.org

Re: [GUMP@vmgump]: Project commons-compress-test (in module apache-commons) failed

[Stefan Bodewig]

On 2013-01-11, Stefan Bodewig wrote:

> On 2013-01-10, Julius Davies wrote:

>> This is one my unit tests that I recently added that's failing.

> They fail on my machine as well (GMT+1 right now) and create times one
> hour different from vmgump which runs in GMT.

> The times that are different from the expected times are the times taken
> directly from ZipEntry and not those taken from the extra field.

At least for my machine I've fixed the tests.

I think you have created the archive using InfoZIP's zip.  zip writes
(and expects) the time stored in ZIP LFHs (and CDs) to be in GMT while
java.util.zip.ZipEntry expects them to be in local time.

Your local YEAR-01-01-00:00 was translated into (YEAR-1)-12-31-16:00
when you created the archive and java.util.zip takes this as local time.
Think I have this the right way around, but I'm starting to confuse
myself here :-)

Tests pass on my machine now by first adding my local GMT offset and
then adding eight hours.  I haven't tested it for any other time zone
but am confident it will work for vmgump as well (since the GMT offset
is 0 there).  In particular I didn't try a system where DST is active on
new years day - the test likely needs to take this into account as well.

Stefan

-
To unsubscribe, e-mail: dev-unsubscr...@commons.apache.org
For additional commands, e-mail: dev-h...@commons.apache.org

Re: [GUMP@vmgump]: Project commons-compress-test (in module apache-commons) failed

[Stefan Bodewig]

On 2013-01-11, Stefan Bodewig wrote:

> In particular I didn't try a system where DST is active on new years
> day - the test likely needs to take this into account as well.

I've modified the test further so I think it should now work for
regions with DST in January as well.

Stefan

-
To unsubscribe, e-mail: dev-unsubscr...@commons.apache.org
For additional commands, e-mail: dev-h...@commons.apache.org

Re: [compress] not reading archive stream completely

[Stefan Bodewig]

On 2013-01-17, Torsten Curdt wrote:

> If we see `getNextEntry` return null we should position the stream at
> the end of the archive.  I think that's your (1). Sounds simpler and
> more straight forward from an API POV.  IIUC that reading should only
> be a few bytes. A second EOF marker for TAR for example.

For tar it would be one block (usualy 512 bytes), for zip the full
central directory has to be read which could be quite a bit.

I currently plan to implement it inside getNextEntry as it is cleaner.
In the tar case I vaguely recall some implementation only write one EOF
marker so a more careful aproach is needed in order to not read beyond
the end of the archive (likely mark and reset if the stream supports
mark).

Thanks

Stefan

-
To unsubscribe, e-mail: dev-unsubscr...@commons.apache.org
For additional commands, e-mail: dev-h...@commons.apache.org

Re: [compress] not reading archive stream completely

[Stefan Bodewig]

On 2013-01-17, Bear Giles wrote:

> I think a number of applications use a concatenation of a standard archive
> format and custom data.

Absolutely, and I think we should support that use-case.

I'll try to free up some coding time this weekend.

Stefan

-
To unsubscribe, e-mail: dev-unsubscr...@commons.apache.org
For additional commands, e-mail: dev-h...@commons.apache.org

Re: [commons-compress] branch master updated: Enable GitHub Dependabot.

[Stefan Bodewig]

On 2020-07-24, Rob Tompkins wrote:

>> On Jul 23, 2020, at 10:16 PM, Matt Sicker  wrote:

>> Also, how different is a bot proposing a dependency update from a human
>> doing the same? The bot includes far more context about the update in the
>> PR comment, too, which is super useful for determining whether or not the
>> dependency is worth updating. You can even configure it to only notify
>> about security updates if it’s too noisy.

> I don’t understand how substantive forward progress on a project can be 
> considered noisy. It’s just audit.

Oh my, please calm down.

Peter just said he hasn't been reading mails for a few days, is
overwhelmed now and will need time to review what has happened. He
didn't complain, he was apologizing for not responding immediately -
which he shouldn't feel was necessary IMHO.

Stefan

-
To unsubscribe, e-mail: dev-unsubscr...@commons.apache.org
For additional commands, e-mail: dev-h...@commons.apache.org

[all] When to update dependencies?

[Stefan Bodewig]

Hi all

here I'd like to explain why I prefer not to update dependencies just
because we can. Maybe you can convince me that I'm wrong. I've tried to
make this point in different threads but either it has been lost or it
just wasn't worth discussing.

First of all let me get a few things out of the way

* I'm not talking about emails, I can deal with them

* I don't care whether a bot or a human asks for a version update

* I'm only talking about dependencies that are visible to our
  users. Test time dependencies or versions of Maven plugins are
  probably not an issue. Although Compress has mananaged to break its
  OSGi bundle just by upgrading the parent POM in the past.
  https://issues.apache.org/jira/browse/COMPRESS-498

All our components have downstream users. I.e. our dependencies become
somebody else's dependencies as well.

Let's say commons-foo 1.1.0 depends on A 1.12.4 and bumps the dependency
to A 1.12.18 for commons-foo 1.2.0.

When a user of commons-foo upgrades to 1.2.0 and hasn't defined their
dependency on A explicitly they will also upgrade A to 1.12.18. This may
be fine or it may cause problems. The new version of A may have made
incompatible changes that break the user's code or it may just have bugs
that were not present in A 1.12.4 and now raise their head.

Of course the users can explicitly state a dependency on A 1.12.4
themselves. But there is no guarantee commons-foo compiled against A
1.12.18 will still work with A 1.12.4.

About fifteen years ago Ant was bitten by StringBuffer adding a new
method append(StringBuffer) in Java 1.4 (if memory serves me
right). Code that called someStringBuffer.append(anotherStringBuffer)
compiled on Java 1.3 would call append(Object), but compiled on 1.4 it
would call the new version and thus could not run on Java 1.3. This is
the kind of change animal sniffer was invented to detect and the
--release option of javac deals with. There is no such tool helping us
with APIs that are not part of the Java classlib.

Therefore I believe updating a dependency is a risk and we should leave
it to the users to decide which version they want to use.

Unless we've got real reasons to update. Real reasons IMHO are security
issues, bugs in dependencies causing bugs in our code or when we really
want to use new features introduced in a new version.

Outside of these good reasons I wouldn't want to ever update a
dependency.

Stefan

-
To unsubscribe, e-mail: dev-unsubscr...@commons.apache.org
For additional commands, e-mail: dev-h...@commons.apache.org

Re: [all] When to update dependencies?

[Stefan Bodewig]

On 2020-07-24, Torsten Curdt wrote:

> It still needs a person to decide to merge a PR for a new version.
> So this indeed is just about the dependency upgrade policies.

Right.

> But isn't that what the version definition is for?
> I'd argue that 1.12.4 <-> 1.12.6 should be a compatible upgrade AND
> downgrade,
> 1.12.4 -> 1.20.0 not so much.

As Gary pointed out else-thread most of the time we do not know how
strict the team developing our dependency adheres to SemVer.

Even if it was completely API compatible, we'd replace a version that
worked for our users with a different version that may introduce
problems. No matter how small the risk is, what is the benefit of
upgrading if we don't need the new version ourselves?

> But to avoid all this is why I usually try to inline dependencies for
> libraries as much as possible. Basically pretending to not have any.

Agreed, this is a different strategy that makes the whole question moot.

> Also a point I made many times.
> Just wanted to mention it - again :)

;-)

Stefan

-
To unsubscribe, e-mail: dev-unsubscr...@commons.apache.org
For additional commands, e-mail: dev-h...@commons.apache.org

Re: [all] When to update dependencies?

[Stefan Bodewig]

On 2020-07-24, Gary Gregory wrote:

> Now back to our code depending on other dependencies. My view is that there
> are bugs, you just have not hit them. If I find one in a dependency and it
> gets fixed, it's going to mean a new version of that dependency.

This either is "we hit a bug that affects us" - where I already say this
is a reason to upgrade for me - or "the user hits a bug" - in which case
I prefer to let the user update the buggy dependency. I don't want to
force the user to update because they might be affected by bugs.

And of course we both know that newer versions not only fix bugs, they
introduce new ones as well.

> Furthermore, when I look online for Javadoc and examples, if I use a
> current jar version, then my odds are better that I can implement what I
> find online.

https://javadoc.io/ or similar services?

> I view it as simpler and safer to update from a "near" dependency than
> letting a dependency acquire "bit rot" and upgrade later, especially
> if an update means making adjustments. I want to make smaller
> increments of adjustment rather than a larger set. Just like I prefer
> to RERO instead of big bangs for releasing Commons components.

I fully agree with you if nobody else depends on my stuff. I.e. I'm
working on an application rather than a library. When working on a
library I really want the user to be and stay in control.

> Another way to look at this is to look at a large software stack: If every
> library developer never updates dependencies, then your application,
> through transitive dependencies will end up depending (virtually) on many
> versions of the each library, which is much more likely to create jar hell
> and other problems than the other edge case where everyone uses the latest
> version (or a fixed version.)

You are just hitting home on Stefan's almost 20 year old impression that
automatic transitive dependency resolution is a BAD IDEA. ;-)

I fully expect you (all of you) to ignore that last remark.

> From a hand-waving-talking-over-beers-more-FOSS-y philosophical-POV, I'd
> like to think that by eating our own current Apache dog food as well as
> other FOSS dog food, we all help each other make our software better.

We might help the developers of our dependencies but I'm afraid we are
hurting our dependees - and I tend to care more for the latter.

Stefan

-
To unsubscribe, e-mail: dev-unsubscr...@commons.apache.org
For additional commands, e-mail: dev-h...@commons.apache.org

Re: [all] When to update dependencies?

[Stefan Bodewig]

On 2020-07-24, Xeno Amess wrote:

> how about:
> 1. we force versions of dependencies in commons-parent
> 2. we make every commons repo use commons-parent as parent.
> 3. we make sure no other repos forces versions of dependencies; all of the
> versions number shall be inherited from commons-parent
> 4. we upgrade versions in commons-parent every several months.

As you may guess I'd be strongly opposed to this as it would mean we
updated everey few months just because we can, noot because there was a
good reason. On top no component could upgrade more agressively if it
really needed a feature of a newer version.

Apart from that I'd expect some component developers to be rather
unhappy if *anything* was forced on them. :-)

Honestly I doubt we could even find a common ground on which version of
Java to support if we tried to have one version for all components.

Stefan

-
To unsubscribe, e-mail: dev-unsubscr...@commons.apache.org
For additional commands, e-mail: dev-h...@commons.apache.org

Re: [all] When to update dependencies?

[Stefan Bodewig]

On 2020-07-24, Bernd Eckenfels wrote:

> When it comes to dependencies wie have both problems: if we upgrade 
> dependencies to aggressively (or if we don't test with older dependencies) 
> then users have the problem that they might not easily be able to upgrade to 
> a new commons version since the required new dependency version might 
> conflict with other (thirdparty) users of that dependency.

> On the other hand if we not continuously update external dependencies we 
> might miss out on their new features, performance and fixes. In addition we 
> might fall behind and have the  to do painful Big Bang Upgrades. Also when 
> our transitive dependencies are outdated and contain bugs (or compliance 
> violations due to old code) some customers might not be happy.

I hear you. I'm not opposed to updating a dependency if we want to use
new features. What I'd like to avoid is updating without a reason other
than "there is a new version and it seems to work according to our
tests".

> So there is a middle ground to be found, which unfortunately collides with 
> the current limited effort maintenance of some of the components:

> - we should define a minimum baseline version of dependencies and runtimes 
> and on each release we check if we still meet them. When we raise the 
> baseline we should ship a new minor (or even major) version. Also we might 
> want to ship security fixes only as a micro update (I.e. not requiring major 
> updates besides the affected code)

One problem with patch updates for security releases is the release vote
will reveal a security update is in the making and the diff will be
small enough to give away the details before the vote has passed. I may
be paranoid.

> - we should regularly test against latest dependency versions (at least 
> within the same minor branch).

Apache Gump?

What you describe sounds good, but "unfortunately collides with the
current limited effort maintenance of some of the components".

Stefan

-
To unsubscribe, e-mail: dev-unsubscr...@commons.apache.org
For additional commands, e-mail: dev-h...@commons.apache.org

[all] github (was Re: [VOTE] Create additional mailing lists for automated posts)

[Stefan Bodewig]

This is an attempt at answering something raised be Gilles in a
different thread. I'm afraid it is getting longer than I
intended. Something seems to need to get out. Sorry.

On 2020-07-23, Gilles Sadowski wrote:

> I missed the turn where this project's PMC decided that we must
> be present on GH in order to continue what some of us have been
> doing for more than 10 years.

The project decided to set up github mirrors and with the exposure to
github you get the rest of the package.

I don't remember whether we had an explicit vote about enabling github
mirrors, much less whether it has been a component by component
decision.

Putting aside what I think about github as a company one thing that I
have observed with projects migrating to github across the board is it
seems to lower the barrier for new contributors. This results in two
things: (1) you get more contributions and (2) the contributors usually
don't stick around, most contributions are drive-by one-off
contributions.

Gilles, I believe you have seen an uptick in contributions to the
sevaral maths components even if you don't like they way they have
happened.

> There is a trend to make GH central to the development process
> (marginalizing "dev@" and JIRA and colonizing "issues@").

This is troubling me as well. Where "this" is that using github PRs
seems to keep the potential future committers away from the dev list and
they never become part of a component's community.

We do get more contributions anf the quality of the codebases likely
improves because of this, but I feel it makes the community weaker.
Discussions only rarely happen "here" nowadays.

There are a few people like Gary, Rob and a few others who manage to
devote time across almost all components and I adore them for that. And
we need them because most of the components wouldn't stand a chance to
get a release vote through without them. Our component communities have
become too small to sustain themselves and at the same time some of them
see more commits than they used to have in "the good old days" with many
people active on this list.

I haven't got an answer to this.

[Sidenote: In the early 90s I contributed to my first open source
project before CVS had a network protocol. I can not understand why
having to checkout a SCM repository, creating a patch and sending it to
somebody else is a burden that keeps people from contributing. I have
learnt to accept this as a fact. There are lots of facts I cannot
explain. And yes, I know I sound old. I am :-)]

Stefan

-
To unsubscribe, e-mail: dev-unsubscr...@commons.apache.org
For additional commands, e-mail: dev-h...@commons.apache.org

Re: [all] github (was Re: [VOTE] Create additional mailing lists for automated posts)

[Stefan Bodewig]

On 2020-07-24, Xeno Amess wrote:

> I will explain why github come to be center, but not apache gitbox.
> 1.1
> I have right to register an account on github.
> 1.2
> I registered an account at github.
> 1.3
> I commit then create pr.
> 1.4
> pr get reviewed then merged.

I am fully aware of how github works, I use PRs myself.

The perceived ease[1] of doing this comes at a price and I'm mourning
the loss of community building.

Stefan

[1] with gitbox you are certainly able to contribute

git clone
git checkout -b my-work
...
git format-patch
attach patch to JIRA

but you know that.

-
To unsubscribe, e-mail: dev-unsubscr...@commons.apache.org
For additional commands, e-mail: dev-h...@commons.apache.org

Re: [all] github (was Re: [VOTE] Create additional mailing lists for automated posts)

[Stefan Bodewig]

On 2020-07-24, Xeno Amess wrote:

> As for community building, I agree with you that commons seems not a
> close community, but I doubt it be github's fault.  even there be no
> github, sub-repos in commons are not that close to each other.

Commons is an old project and it started with a striving community. Most
people have left, as is natural for any open source project. I believe
we've done better attracting new community members before, but this is
certainly colored by my mood and preferences.

As for your follow-up response about agreeing on when to upgrade
dependencies. Community doesn't mean uniformity to me. We respectfully
discuss and in the end come to a compromise or a common ground where we
can agree to disagree. I still see this happen here and don't think all
of us need to have the same opinion.

Stefan

-
To unsubscribe, e-mail: dev-unsubscr...@commons.apache.org
For additional commands, e-mail: dev-h...@commons.apache.org

Re: [ALL] CI builds

[Stefan Bodewig]

On 2020-07-23, Olivier Lamy wrote:

> In the Maven project we have plenty of maven-* git repo so we have created
> a dedicated Jenkins plugin (which is used by other TLP such netbeans) which
> scan the gitbox server to get repo based on regular expression or name
> content and create the build reusing the same build file.

This sounds very interesting, thank you Olivier. Is this
https://github.com/apache/maven-jenkins-lib ?

I'm not sure there are enough people around here still willing to use
Jenkins to enable something like this for all repos :-)

Stefan

-
To unsubscribe, e-mail: dev-unsubscr...@commons.apache.org
For additional commands, e-mail: dev-h...@commons.apache.org

Re: [all] github (was Re: [VOTE] Create additional mailing lists for automated posts)

[Stefan Bodewig]

On 2020-07-24, Xeno Amess wrote:

>> We respectfully discuss and in the end come to a  compromise or a common
>> ground where we can agree to disagree. I still see this happen here and
>> don't think all of us need to have the same opinion.

> So maybe at the end some of commons repos using as new version of
> dependencies as they can, others using as old version of dependencies as
> they can?

Could be such an outcome, yes, or maybe a stronger recommendation for a
certain approach. We'll see in a different thread :-)

Stefan

-
To unsubscribe, e-mail: dev-unsubscr...@commons.apache.org
For additional commands, e-mail: dev-h...@commons.apache.org

Re: [all] github

[Stefan Bodewig]

On 2020-07-26, Melloware wrote:

> I know there seems to to be a holy war about the use of GitHub going
> on here

This has never been my intention. Far from it. And if you believ I've
tried to start any kind of war you must have misread my original mail
completely. I'm really sorry about thaty.

I appreciate the benefits github brings but realize it also causes
problems that I don't know how to solve. I would have liked to talk
about what we can do to build communities around components again rather
than whether creating PRs is better than not doing that.

Maybe I should have written an essay and not bother the list. All that
hapened is that I've caused another stirrup about topics I didn't even
intend to touch.

I'll rest my case as this is not going to become productive.

Stefan

-
To unsubscribe, e-mail: dev-unsubscr...@commons.apache.org
For additional commands, e-mail: dev-h...@commons.apache.org

Re: [COMPRESS] OSS-Fuzz integration

[Stefan Bodewig]

On 2021-03-05, Fabian Meumertzheim wrote:

> I am one of the maintainers of Jazzer
> (https://github.com/CodeIntelligenceTesting/jazzer), a new open-source
> fuzzer for JVM projects based on libFuzzer.

> I have set up a few Commons projects for local fuzzing with Jazzer,
> which lead to quite a few bug reports in Compress and other projects
> (https://issues.apache.org/jira/browse/COMPRESS-569?jql=reporter%20%3D%20Meumertzheim).
> While the majority of the bugs found are undeclared exceptions, this
> approach also caught an infinite loop on a crafted 0.5KB .tar before
> it could make it into a release (see COMPRESS-569).

Yes, many thanks for that.

> Jazzer is in the process of being integrated into OSS-Fuzz
> (https://github.com/google/oss-fuzz) for continuous fuzzing on
> Google-provided infrastructure (ClusterFuzz).

> If you agree this is a good idea, I could set up Compress for fuzzing
> on OSS-Fuzz.

Also I'd like to point out issues detected by Maksim Zuev last spring
and summer who used a different fuzzing tool.

When reading archives or compressed streams, Compress consumes a lot of
input and we are obviously not validating it in all cases as good as we
should. Much of the code assumes it will only ever encounter valid
archives. I believe fuzzing can help us finding places where we trust
input too much.

commons-codec or commons-imaging likely are in similar places.

OTOH I'm not sure I understand the requirements of OSS-Fuzz. I haven't
read the docs only looked at the image of the process. Seeing a
Sheriffbot tracking deadlines makes the me very uncomfortable. I'm a
volunteer and so are most others around here.

> All I would need from you is a list of emails to which the automated
> bug reports should go. The reports are usually directly actionable as
> they include stack traces and minimized reproducers.

In general I'd think the notifications list of the Commons project would
be a the best fit. Of course the nature of the issues detected could
lead to the fuzzer uncovering security critical bugs that we may not
want to become public before a release fixing it has become available.

Stefan

-
To unsubscribe, e-mail: dev-unsubscr...@commons.apache.org
For additional commands, e-mail: dev-h...@commons.apache.org