Re: Possible LTS regression in commons-compress

[Oliver Gondža]

James, thanks for the updates. The way I understand that is, problem is 
easily fixable, affects proprietary component and is a fairly unlikely 
thing to do to begin with, so we do not consider this to be a blocker 
for LTS.


Unless someone tells me I am wrong, I am targeting the RC for tomorrow.

On 27/11/2019 12.50, James Nord wrote:
I have tracked the cause of the issue to an optimisation we had for the 
old Hudson code to not flush all the time to prevent lots of small 
chunks being flushed (as the old implementation flushed for each block 
(512 bytes))


If I disable that then the code works with 1.15 and above.  I have not 
looked into the code to see if commons compress also has issues with 
flushing, but I doubt any other plugins will be doing any similar 
optimisation.


/James




On Wednesday, November 27, 2019 at 8:23:47 AM UTC, Baptiste Mathus wrote:

Sounds to me we should git bisect commons-compress between 1.14 and
1.15 usage, to nail and possibly confirm Björn's hypothesis?

On Wed, Nov 27, 2019 at 8:36 AM 'Björn Pedersen' via Jenkins
Developers > wrote:

Hi,

I have 2 changes that seem candiates:
1) 1.15 enforces blocklen=512 which is different from the old
default and raises an error if anything else is specified
  
(https://commons.apache.org/proper/commons-compress/javadocs/api-1.19/index.html 
)
2) 1.16 renamed the preserveAbsolutePath to |TarArchiveEntry

(String


 name,
boolean preserveAbsolutePath)|
|(less likely that is the cause).
|

Björn

Am Mittwoch, 27. November 2019 01:03:40 UTC+1 schrieb James Nord:

Hi all,

just a quick note, I /think /I have discovered a regression
in 2.204 upcoming LTS due to commons-compress library bump.

One of our plugins was using TarInputStream

for
2.190 but this is now restricted.
so without bumping core we moved the code over to use the
commons-compress version (and all is happy).

However when bumping the Jenkins version to 2.204 which
picks up the newer compress we have lots of unit test
failures due to what seems like (at very first glance as I
just narrowed it down and its late!) empty tar files (when
they should not be empty)

as a (quick and dirty) test I made the commons-compress
version configurable and it does seem like this is the issue

mvn test -Dtest=TheTest -Djenkins.version=2.204
-DcompressVersion=1.19 -Denforcer.skip *fails*
mvn test -Dtest=TheTest -Djenkins.version=2.204
-DcompressVersion=1.10 -Denforcer.skip passes
mvn test -Dtest=TheTest -Djenkins.version=2.190.3
-DcompressVersion=1.10 -Denforcer.skip passes
mvn test -Dtest=TheTest -Djenkins.version=2.190.3
-DcompressVersion=1.19 -Denforcer.skip *fails
*
doing some more bisection

mvn test -Dtest=TheTest -Djenkins.version=2.190.3
-DcompressVersion=1.15 -Denforcer.skip *fails
*mvn test -Dtest=TheTest -Djenkins.version=2.190.3
-DcompressVersion=1.13 -Denforcer.skip passes
mvn test -Dtest=TheTest -Djenkins.version=2.190.3
-DcompressVersion=1.14 -Denforcer.skip passes


I don't yet know what is causing this (but it does appear to
be something in commons-compress 1.15 and I will investigate
that.  (changelog

)

But this seems very scary to me (and I am surprised there
has not been some reports in the weeklies of archives etc
not working correctly).  Also very weired as 1.15 has been
released for a good amount of time.

Will continue the investigation tomorrow but if this rings a
bell with anyone in any reported Jiras it would be good to
correlate.

/James

-- 
You received this message because you are subscribed to the

Google Groups "Jenkins Developers" group.
To unsubscribe from this group and stop receiving emails from
it, send an email to jenkin...@googlegroups.com .
To view this discussion on the web visit

Re: Possible LTS regression in commons-compress

[James Nord]

I have tracked the cause of the issue to an optimisation we had for the old 
Hudson code to not flush all the time to prevent lots of small chunks being 
flushed (as the old implementation flushed for each block (512 bytes)) 

If I disable that then the code works with 1.15 and above.  I have not 
looked into the code to see if commons compress also has issues with 
flushing, but I doubt any other plugins will be doing any similar 
optimisation.

/James




On Wednesday, November 27, 2019 at 8:23:47 AM UTC, Baptiste Mathus wrote:
>
> Sounds to me we should git bisect commons-compress between 1.14 and 1.15 
> usage, to nail and possibly confirm Björn's hypothesis?
>
> On Wed, Nov 27, 2019 at 8:36 AM 'Björn Pedersen' via Jenkins Developers <
> jenkin...@googlegroups.com > wrote:
>
>> Hi,
>>
>> I have 2 changes that seem candiates:
>> 1) 1.15 enforces blocklen=512 which is different from the old default and 
>> raises an error if anything else is specified
>>  (
>> https://commons.apache.org/proper/commons-compress/javadocs/api-1.19/index.html
>> )
>> 2) 1.16 renamed the preserveAbsolutePath to TarArchiveEntry 
>> 
>> (String 
>> 
>>  name, 
>> boolean preserveAbsolutePath)
>> (less likely that is the cause).
>>
>> Björn
>>
>> Am Mittwoch, 27. November 2019 01:03:40 UTC+1 schrieb James Nord:
>>>
>>> Hi all,
>>>
>>> just a quick note, I *think *I have discovered a regression in 2.204 
>>> upcoming LTS due to commons-compress library bump.
>>>
>>> One of our plugins was using TarInputStream 
>>> for
>>>  
>>> 2.190 but this is now restricted.
>>> so without bumping core we moved the code over to use the 
>>> commons-compress version (and all is happy).
>>>
>>> However when bumping the Jenkins version to 2.204 which picks up the 
>>> newer compress we have lots of unit test failures due to what seems like 
>>> (at very first glance as I just narrowed it down and its late!) empty tar 
>>> files (when they should not be empty)
>>>
>>> as a (quick and dirty) test I made the commons-compress version 
>>> configurable and it does seem like this is the issue
>>>
>>> mvn test -Dtest=TheTest -Djenkins.version=2.204 -DcompressVersion=1.19 
>>> -Denforcer.skip *fails*
>>> mvn test -Dtest=TheTest -Djenkins.version=2.204 -DcompressVersion=1.10 
>>> -Denforcer.skip passes
>>> mvn test -Dtest=TheTest -Djenkins.version=2.190.3 -DcompressVersion=1.10 
>>> -Denforcer.skip passes
>>> mvn test -Dtest=TheTest -Djenkins.version=2.190.3 -DcompressVersion=1.19 
>>> -Denforcer.skip 
>>> *fails*
>>> doing some more bisection
>>>
>>> mvn test -Dtest=TheTest -Djenkins.version=2.190.3 -DcompressVersion=1.15 
>>> -Denforcer.skip 
>>> *fails*mvn test -Dtest=TheTest -Djenkins.version=2.190.3 
>>> -DcompressVersion=1.13 -Denforcer.skip passes
>>> mvn test -Dtest=TheTest -Djenkins.version=2.190.3 -DcompressVersion=1.14 
>>> -Denforcer.skip passes
>>>
>>>
>>> I don't yet know what is causing this (but it does appear to be 
>>> something in commons-compress 1.15 and I will investigate that.  (
>>> changelog 
>>> 
>>> )
>>>
>>> But this seems very scary to me (and I am surprised there has not been 
>>> some reports in the weeklies of archives etc not working correctly).  Also 
>>> very weired as 1.15 has been released for a good amount of time.
>>>
>>> Will continue the investigation tomorrow but if this rings a bell with 
>>> anyone in any reported Jiras it would be good to correlate.
>>>
>>> /James
>>>
>> -- 
>> You received this message because you are subscribed to the Google Groups 
>> "Jenkins Developers" group.
>> To unsubscribe from this group and stop receiving emails from it, send an 
>> email to jenkin...@googlegroups.com .
>> To view this discussion on the web visit 
>> https://groups.google.com/d/msgid/jenkinsci-dev/8b3e8989-4f34-4b1d-a285-036837dc6bd1%40googlegroups.com
>>  
>> 
>> .
>>
>

-- 
You received this message because you are subscribed to the Google Groups 
"Jenkins Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to jenkinsci-dev+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/jenkinsci-dev/6259ccf6-f5de-469e-a7db-18ff7dd52651%40googlegroups.com.

Re: Possible LTS regression in commons-compress

[Baptiste Mathus]

Sounds to me we should git bisect commons-compress between 1.14 and 1.15
usage, to nail and possibly confirm Björn's hypothesis?

On Wed, Nov 27, 2019 at 8:36 AM 'Björn Pedersen' via Jenkins Developers <
jenkinsci-dev@googlegroups.com> wrote:

> Hi,
>
> I have 2 changes that seem candiates:
> 1) 1.15 enforces blocklen=512 which is different from the old default and
> raises an error if anything else is specified
>  (
> https://commons.apache.org/proper/commons-compress/javadocs/api-1.19/index.html
> )
> 2) 1.16 renamed the preserveAbsolutePath to TarArchiveEntry
> 
> (String
> 
>  name,
> boolean preserveAbsolutePath)
> (less likely that is the cause).
>
> Björn
>
> Am Mittwoch, 27. November 2019 01:03:40 UTC+1 schrieb James Nord:
>>
>> Hi all,
>>
>> just a quick note, I *think *I have discovered a regression in 2.204
>> upcoming LTS due to commons-compress library bump.
>>
>> One of our plugins was using TarInputStream
>> for
>> 2.190 but this is now restricted.
>> so without bumping core we moved the code over to use the
>> commons-compress version (and all is happy).
>>
>> However when bumping the Jenkins version to 2.204 which picks up the
>> newer compress we have lots of unit test failures due to what seems like
>> (at very first glance as I just narrowed it down and its late!) empty tar
>> files (when they should not be empty)
>>
>> as a (quick and dirty) test I made the commons-compress version
>> configurable and it does seem like this is the issue
>>
>> mvn test -Dtest=TheTest -Djenkins.version=2.204 -DcompressVersion=1.19
>> -Denforcer.skip *fails*
>> mvn test -Dtest=TheTest -Djenkins.version=2.204 -DcompressVersion=1.10
>> -Denforcer.skip passes
>> mvn test -Dtest=TheTest -Djenkins.version=2.190.3 -DcompressVersion=1.10
>> -Denforcer.skip passes
>> mvn test -Dtest=TheTest -Djenkins.version=2.190.3 -DcompressVersion=1.19
>> -Denforcer.skip
>> *fails*
>> doing some more bisection
>>
>> mvn test -Dtest=TheTest -Djenkins.version=2.190.3 -DcompressVersion=1.15
>> -Denforcer.skip
>> *fails*mvn test -Dtest=TheTest -Djenkins.version=2.190.3
>> -DcompressVersion=1.13 -Denforcer.skip passes
>> mvn test -Dtest=TheTest -Djenkins.version=2.190.3 -DcompressVersion=1.14
>> -Denforcer.skip passes
>>
>>
>> I don't yet know what is causing this (but it does appear to be something
>> in commons-compress 1.15 and I will investigate that.  (changelog
>> 
>> )
>>
>> But this seems very scary to me (and I am surprised there has not been
>> some reports in the weeklies of archives etc not working correctly).  Also
>> very weired as 1.15 has been released for a good amount of time.
>>
>> Will continue the investigation tomorrow but if this rings a bell with
>> anyone in any reported Jiras it would be good to correlate.
>>
>> /James
>>
> --
> You received this message because you are subscribed to the Google Groups
> "Jenkins Developers" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to jenkinsci-dev+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/jenkinsci-dev/8b3e8989-4f34-4b1d-a285-036837dc6bd1%40googlegroups.com
> 
> .
>

-- 
You received this message because you are subscribed to the Google Groups 
"Jenkins Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to jenkinsci-dev+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/jenkinsci-dev/CAPyTVp0pRuigKbyCNqW%3DJPdSTE7zdxMLEMHpPdpe9t2L3Z8k_w%40mail.gmail.com.

Re: Possible LTS regression in commons-compress

['Björn Pedersen' via Jenkins Developers]

Hi,

I have 2 changes that seem candiates:
1) 1.15 enforces blocklen=512 which is different from the old default and 
raises an error if anything else is specified
 
(https://commons.apache.org/proper/commons-compress/javadocs/api-1.19/index.html)
2) 1.16 renamed the preserveAbsolutePath to TarArchiveEntry 

(String 

 name, 
boolean preserveAbsolutePath)
(less likely that is the cause).

Björn

Am Mittwoch, 27. November 2019 01:03:40 UTC+1 schrieb James Nord:
>
> Hi all,
>
> just a quick note, I *think *I have discovered a regression in 2.204 
> upcoming LTS due to commons-compress library bump.
>
> One of our plugins was using TarInputStream 
> for
>  
> 2.190 but this is now restricted.
> so without bumping core we moved the code over to use the commons-compress 
> version (and all is happy).
>
> However when bumping the Jenkins version to 2.204 which picks up the newer 
> compress we have lots of unit test failures due to what seems like (at very 
> first glance as I just narrowed it down and its late!) empty tar files 
> (when they should not be empty)
>
> as a (quick and dirty) test I made the commons-compress version 
> configurable and it does seem like this is the issue
>
> mvn test -Dtest=TheTest -Djenkins.version=2.204 -DcompressVersion=1.19 
> -Denforcer.skip *fails*
> mvn test -Dtest=TheTest -Djenkins.version=2.204 -DcompressVersion=1.10 
> -Denforcer.skip passes
> mvn test -Dtest=TheTest -Djenkins.version=2.190.3 -DcompressVersion=1.10 
> -Denforcer.skip passes
> mvn test -Dtest=TheTest -Djenkins.version=2.190.3 -DcompressVersion=1.19 
> -Denforcer.skip 
> *fails*
> doing some more bisection
>
> mvn test -Dtest=TheTest -Djenkins.version=2.190.3 -DcompressVersion=1.15 
> -Denforcer.skip 
> *fails*mvn test -Dtest=TheTest -Djenkins.version=2.190.3 
> -DcompressVersion=1.13 -Denforcer.skip passes
> mvn test -Dtest=TheTest -Djenkins.version=2.190.3 -DcompressVersion=1.14 
> -Denforcer.skip passes
>
>
> I don't yet know what is causing this (but it does appear to be something 
> in commons-compress 1.15 and I will investigate that.  (changelog 
> 
> )
>
> But this seems very scary to me (and I am surprised there has not been 
> some reports in the weeklies of archives etc not working correctly).  Also 
> very weired as 1.15 has been released for a good amount of time.
>
> Will continue the investigation tomorrow but if this rings a bell with 
> anyone in any reported Jiras it would be good to correlate.
>
> /James
>

-- 
You received this message because you are subscribed to the Google Groups 
"Jenkins Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to jenkinsci-dev+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/jenkinsci-dev/8b3e8989-4f34-4b1d-a285-036837dc6bd1%40googlegroups.com.

Possible LTS regression in commons-compress

[James Nord]

Hi all,

just a quick note, I *think *I have discovered a regression in 2.204 
upcoming LTS due to commons-compress library bump.

One of our plugins was using TarInputStream 
for
 
2.190 but this is now restricted.
so without bumping core we moved the code over to use the commons-compress 
version (and all is happy).

However when bumping the Jenkins version to 2.204 which picks up the newer 
compress we have lots of unit test failures due to what seems like (at very 
first glance as I just narrowed it down and its late!) empty tar files 
(when they should not be empty)

as a (quick and dirty) test I made the commons-compress version 
configurable and it does seem like this is the issue

mvn test -Dtest=TheTest -Djenkins.version=2.204 -DcompressVersion=1.19 
-Denforcer.skip *fails*
mvn test -Dtest=TheTest -Djenkins.version=2.204 -DcompressVersion=1.10 
-Denforcer.skip passes
mvn test -Dtest=TheTest -Djenkins.version=2.190.3 -DcompressVersion=1.10 
-Denforcer.skip passes
mvn test -Dtest=TheTest -Djenkins.version=2.190.3 -DcompressVersion=1.19 
-Denforcer.skip 
*fails*
doing some more bisection

mvn test -Dtest=TheTest -Djenkins.version=2.190.3 -DcompressVersion=1.15 
-Denforcer.skip 
*fails*mvn test -Dtest=TheTest -Djenkins.version=2.190.3 
-DcompressVersion=1.13 -Denforcer.skip passes
mvn test -Dtest=TheTest -Djenkins.version=2.190.3 -DcompressVersion=1.14 
-Denforcer.skip passes


I don't yet know what is causing this (but it does appear to be something 
in commons-compress 1.15 and I will investigate that.  (changelog 

)

But this seems very scary to me (and I am surprised there has not been some 
reports in the weeklies of archives etc not working correctly).  Also very 
weired as 1.15 has been released for a good amount of time.

Will continue the investigation tomorrow but if this rings a bell with 
anyone in any reported Jiras it would be good to correlate.

/James

-- 
You received this message because you are subscribed to the Google Groups 
"Jenkins Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to jenkinsci-dev+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/jenkinsci-dev/14e1d85b-2702-496b-9cd0-e3a42896341a%40googlegroups.com.