Re: Jars in disguise (was: [VOTE] Release Apache Groovy 2.4.4-incubating)

[Cédric Champeau]

So the current status is that the extension module jars used for tests are
now built at compile time. So there's no need for the jars in source
anymore. The only jar left is GroovyJarTest.jar, which doesn't contain any
binary (just Groovy scripts). The fun thing is that the test is broken.
First of all, it's not executed at all, because security tests are disabled
(it should have been a temporary disabling, but it's getting complicated).
Second, even if you execute the test, it doesn't take long, looking at the
contents of the scripts in the jar, that they are not executed. In short,
it is IMHO safe to remove, and the test should be rewritten.

2015-06-16 9:16 GMT+02:00 Bertrand Delacretaz :

> On Mon, Jun 15, 2015 at 11:00 PM, Emmanuel Lécharny 
> wrote:
> > ...The ideal situation would be to create those jars in teh process,
> before
> > using them in a test...
>
> Yes, that's by far the best way - create them from (readable, usual)
> source code.
>
> Second best IMO is to have them in a separate folder with a README
> that explains what they are and lists their sha1 digests so to help
> reassure people that they are legit.
>
> -Bertrand
>

Re: Jars in disguise (was: [VOTE] Release Apache Groovy 2.4.4-incubating)

[Bertrand Delacretaz]

On Mon, Jun 15, 2015 at 11:00 PM, Emmanuel Lécharny  wrote:
> ...The ideal situation would be to create those jars in teh process, before
> using them in a test...

Yes, that's by far the best way - create them from (readable, usual)
source code.

Second best IMO is to have them in a separate folder with a README
that explains what they are and lists their sha1 digests so to help
reassure people that they are legit.

-Bertrand

Re: Jars in disguise (was: [VOTE] Release Apache Groovy 2.4.4-incubating)

[Bertrand Delacretaz]

On Mon, Jun 15, 2015 at 9:01 PM, Pascal Schumacher
 wrote:
> I reverted, but you could have told me last monday (when I asked in this
> thread)...

Sorry about that, I have missed that question. I'm not following all
the details, so in general if you want advice for the mentors it's
good to create a new specific thread with a [MENTOR] subject line
header.

(not an excuse for me missing that though...but that might help in the future)

-Bertrand

Re: Jars in disguise (was: [VOTE] Release Apache Groovy 2.4.4-incubating)

[Greg Trasuk]

I polled general@incubator.a.o  a few years ago 
on  a similar topic.  Discussion at:
http://mail-archives.apache.org/mod_mbox/incubator-general/201312.mbox/%3c24f4a019-b30c-4cfa-a00b-f792a22de...@stratuscom.com%3E
 


Cheers,

Greg Trasuk

> On Jun 15, 2015, at 9:36 PM, Greg Trasuk  wrote:
> 
> 
> 
> 
>> On Jun 15, 2015, at 7:02 PM, David Nalley  wrote:
>> 
>> On Mon, Jun 15, 2015 at 4:17 PM, Cédric Champeau
>>  wrote:
>>> My understanding is that since those jars are for tests, it is OK to keep
>>> them as long as we:
>>> 
>>> - have a Rat rule that explicitly states why those jars are there
>>> - have a JIRA ticket that we can point people at when they ask
>>> 
>> 
>> I don't know that your understanding agrees with the Incubator's
>> stance on binaries in releases. I'd at a minimum start a proactive
>> discussion on general@., and I would expect a lot of resistance to
>> binaries in a source release.
>> 
>> —David
> 
> Imagine if there were an incubator project that was an image-processing 
> library, and part of the test was to perform an averaging function on a test 
> image (e.g. of a 60%-gray image) and see if the result came to 60%.
> 
> The test image would be a “binary” (e.g. a PNG), right?  Would we insist that 
> the test be removed because having it would mean packaging a “binary” in the 
> source distribution?  That would seem not to make much sense.  
> 
> We would certainly want to make sure that the image file in question was 
> licensed appropriately for distribution, but if you have a package whose 
> functionality is processing binary images, then it would make sense that 
> you’d need binary images in the test resources to prove out the package.
> 
> Similarly, if part of Groovy’s job is to read jar files, then it certainly 
> makes sense that you might need jar files in the test resources.
> 
> Obviously, if you were shipping a jar that was part of the final executable 
> environment, that would be unacceptable in a source distribution.  But  a 
> sample artifact used as a test input should be fine, so long as it’s obvious 
> that it’s a test resource, and there’s no issue with giving out copies of it.
> 
> 
> Cheers,
> 
> Greg Trasuk
> 

Re: Jars in disguise (was: [VOTE] Release Apache Groovy 2.4.4-incubating)

[Greg Trasuk]

> On Jun 15, 2015, at 7:02 PM, David Nalley  wrote:
> 
> On Mon, Jun 15, 2015 at 4:17 PM, Cédric Champeau
>  wrote:
>> My understanding is that since those jars are for tests, it is OK to keep
>> them as long as we:
>> 
>> - have a Rat rule that explicitly states why those jars are there
>> - have a JIRA ticket that we can point people at when they ask
>> 
> 
> I don't know that your understanding agrees with the Incubator's
> stance on binaries in releases. I'd at a minimum start a proactive
> discussion on general@., and I would expect a lot of resistance to
> binaries in a source release.
> 
> —David

Imagine if there were an incubator project that was an image-processing 
library, and part of the test was to perform an averaging function on a test 
image (e.g. of a 60%-gray image) and see if the result came to 60%.

The test image would be a “binary” (e.g. a PNG), right?  Would we insist that 
the test be removed because having it would mean packaging a “binary” in the 
source distribution?  That would seem not to make much sense.  

We would certainly want to make sure that the image file in question was 
licensed appropriately for distribution, but if you have a package whose 
functionality is processing binary images, then it would make sense that you’d 
need binary images in the test resources to prove out the package.

Similarly, if part of Groovy’s job is to read jar files, then it certainly 
makes sense that you might need jar files in the test resources.

Obviously, if you were shipping a jar that was part of the final executable 
environment, that would be unacceptable in a source distribution.  But  a 
sample artifact used as a test input should be fine, so long as it’s obvious 
that it’s a test resource, and there’s no issue with giving out copies of it.


Cheers,

Greg Trasuk

Re: Jars in disguise (was: [VOTE] Release Apache Groovy 2.4.4-incubating)

[David Nalley]

On Mon, Jun 15, 2015 at 4:17 PM, Cédric Champeau
 wrote:
> My understanding is that since those jars are for tests, it is OK to keep
> them as long as we:
>
> - have a Rat rule that explicitly states why those jars are there
> - have a JIRA ticket that we can point people at when they ask
>

I don't know that your understanding agrees with the Incubator's
stance on binaries in releases. I'd at a minimum start a proactive
discussion on general@., and I would expect a lot of resistance to
binaries in a source release.

--David

Re: Jars in disguise (was: [VOTE] Release Apache Groovy 2.4.4-incubating)

[Paul King]

On 16/06/2015 6:26 AM, Roman Shaposhnik wrote:

One more follow up question before I answer: is there a way to run
a subset of tests without these present?


We can certainly run subsets of tests but I wouldn't want to
ever release a version of Groovy which couldn't load it's
(or our users) extension modules from jars, so I would be
-1 for turning those tests off.

cheers, Paul.

 

On Mon, Jun 15, 2015 at 1:17 PM, Cédric Champeau
 wrote:

My understanding is that since those jars are for tests, it is OK to keep
them as long as we:

- have a Rat rule that explicitly states why those jars are there
- have a JIRA ticket that we can point people at when they ask


2015-06-15 21:01 GMT+02:00 Pascal Schumacher :


I reverted, but you could have told me last monday (when I asked in this
thread).


Am 15.06.2015 um 09:12 schrieb Bertrand Delacretaz:


Hi,

On Sun, Jun 14, 2015 at 8:43 PM, Pascal Schumacher
 wrote:


...jars under "src/test/resources" are now gone


Encoding those jars in base64 as in [1] does not mean they are gone,
they're just encoded in a different way.

If those jars are useful for testing, it's much preferable IMO to have
them as jar files to make it obvious that the source code does contain
binaries. Maybe isolate them in a single folder and include a readme
file in their which explains why they are useful and lists their sha1
digests so people can detect any changes to them.

That code says


// Apache License does not allow jars/binaries in the source


but avoiding binaries in Apache releases has nothing to do with the
Apache License, it's just that there's usually no way for someone to
verify that a lone binary file has not been tampered with, and we want
our users to be able to trust what we release. Hiding stuff in source
code does not look very trustful to me.

-Bertrand

[1]
https://git1-us-west.apache.org/repos/asf?p=incubator-groovy.git;a=blob;f=src/test/org/codehaus/groovy/runtime/m12n/ExtensionModuleHelperForTests.groovy;h=01fb34e1fcfb6ed84bbc1630d12218e6dfd566d4;hb=1bb5c5a9










---
This email has been checked for viruses by Avast antivirus software.
https://www.avast.com/antivirus

Re: Jars in disguise (was: [VOTE] Release Apache Groovy 2.4.4-incubating)

[Emmanuel Lécharny]

I would add one more thing : mentors are here to help, but we are not
lawyers. Regarding this question, I would ask legal@a.o to know what's
the best way to deal with that. It might take a bit more time, but at
least you would get a clear answer.

And all in all, we have this legal@a.o mailong list to decipher complex
questions, so better use it !

Re: Jars in disguise (was: [VOTE] Release Apache Groovy 2.4.4-incubating)

[Emmanuel Lécharny]

Le 15/06/15 22:17, Cédric Champeau a écrit :
> My understanding is that since those jars are for tests, it is OK to keep
> them as long as we:
>
> - have a Rat rule that explicitly states why those jars are there
> - have a JIRA ticket that we can point people at when they ask

This is also my understanding. Now, the jars provenance has to be
explicited, so that anyone getting Groovy can know where they are coming
from, what is the license they are bound with, etc.

Ypu have to understand where those rules are coming from : we want to be
sure that we are not contaminated by any license like GPL/LGPL, but more
important, we want to be sure none of our users has the irsk of being
contaminated themselves. This is quite clear : a company or foundation
(owning a coponent we are embedding) who would like to sue someone will
*not* sue the ASF, but instead some company using our software :they do
have money, we don't. The impact would be devastating for The ASF though
: we would not be a trustable source for Open Source software .

So we prefer to be extra cautious here.

The ideal situation would be to create those jars in teh process, before
using them in a test.

My perception of the pb.

Re: Jars in disguise (was: [VOTE] Release Apache Groovy 2.4.4-incubating)

[Roman Shaposhnik]

One more follow up question before I answer: is there a way to run
a subset of tests without these present?

Thanks,
Roman.

On Mon, Jun 15, 2015 at 1:17 PM, Cédric Champeau
 wrote:
> My understanding is that since those jars are for tests, it is OK to keep
> them as long as we:
>
> - have a Rat rule that explicitly states why those jars are there
> - have a JIRA ticket that we can point people at when they ask
>
>
> 2015-06-15 21:01 GMT+02:00 Pascal Schumacher :
>>
>> I reverted, but you could have told me last monday (when I asked in this
>> thread).
>>
>>
>> Am 15.06.2015 um 09:12 schrieb Bertrand Delacretaz:
>>>
>>> Hi,
>>>
>>> On Sun, Jun 14, 2015 at 8:43 PM, Pascal Schumacher
>>>  wrote:

 ...jars under "src/test/resources" are now gone
>>>
>>> Encoding those jars in base64 as in [1] does not mean they are gone,
>>> they're just encoded in a different way.
>>>
>>> If those jars are useful for testing, it's much preferable IMO to have
>>> them as jar files to make it obvious that the source code does contain
>>> binaries. Maybe isolate them in a single folder and include a readme
>>> file in their which explains why they are useful and lists their sha1
>>> digests so people can detect any changes to them.
>>>
>>> That code says
>>>
 // Apache License does not allow jars/binaries in the source
>>>
>>> but avoiding binaries in Apache releases has nothing to do with the
>>> Apache License, it's just that there's usually no way for someone to
>>> verify that a lone binary file has not been tampered with, and we want
>>> our users to be able to trust what we release. Hiding stuff in source
>>> code does not look very trustful to me.
>>>
>>> -Bertrand
>>>
>>> [1]
>>> https://git1-us-west.apache.org/repos/asf?p=incubator-groovy.git;a=blob;f=src/test/org/codehaus/groovy/runtime/m12n/ExtensionModuleHelperForTests.groovy;h=01fb34e1fcfb6ed84bbc1630d12218e6dfd566d4;hb=1bb5c5a9
>>
>>
>

Re: Jars in disguise (was: [VOTE] Release Apache Groovy 2.4.4-incubating)

[Cédric Champeau]

My understanding is that since those jars are for tests, it is OK to keep
them as long as we:

- have a Rat rule that explicitly states why those jars are there
- have a JIRA ticket that we can point people at when they ask


2015-06-15 21:01 GMT+02:00 Pascal Schumacher :

> I reverted, but you could have told me last monday (when I asked in this
> thread).
>
>
> Am 15.06.2015 um 09:12 schrieb Bertrand Delacretaz:
>
>> Hi,
>>
>> On Sun, Jun 14, 2015 at 8:43 PM, Pascal Schumacher
>>  wrote:
>>
>>> ...jars under "src/test/resources" are now gone
>>>
>> Encoding those jars in base64 as in [1] does not mean they are gone,
>> they're just encoded in a different way.
>>
>> If those jars are useful for testing, it's much preferable IMO to have
>> them as jar files to make it obvious that the source code does contain
>> binaries. Maybe isolate them in a single folder and include a readme
>> file in their which explains why they are useful and lists their sha1
>> digests so people can detect any changes to them.
>>
>> That code says
>>
>>  // Apache License does not allow jars/binaries in the source
>>>
>> but avoiding binaries in Apache releases has nothing to do with the
>> Apache License, it's just that there's usually no way for someone to
>> verify that a lone binary file has not been tampered with, and we want
>> our users to be able to trust what we release. Hiding stuff in source
>> code does not look very trustful to me.
>>
>> -Bertrand
>>
>> [1]
>> https://git1-us-west.apache.org/repos/asf?p=incubator-groovy.git;a=blob;f=src/test/org/codehaus/groovy/runtime/m12n/ExtensionModuleHelperForTests.groovy;h=01fb34e1fcfb6ed84bbc1630d12218e6dfd566d4;hb=1bb5c5a9
>>
>
>

Re: Jars in disguise (was: [VOTE] Release Apache Groovy 2.4.4-incubating)

[Pascal Schumacher]

I reverted, but you could have told me last monday (when I asked in this 
thread).


Am 15.06.2015 um 09:12 schrieb Bertrand Delacretaz:

Hi,

On Sun, Jun 14, 2015 at 8:43 PM, Pascal Schumacher
 wrote:

...jars under "src/test/resources" are now gone

Encoding those jars in base64 as in [1] does not mean they are gone,
they're just encoded in a different way.

If those jars are useful for testing, it's much preferable IMO to have
them as jar files to make it obvious that the source code does contain
binaries. Maybe isolate them in a single folder and include a readme
file in their which explains why they are useful and lists their sha1
digests so people can detect any changes to them.

That code says


// Apache License does not allow jars/binaries in the source

but avoiding binaries in Apache releases has nothing to do with the
Apache License, it's just that there's usually no way for someone to
verify that a lone binary file has not been tampered with, and we want
our users to be able to trust what we release. Hiding stuff in source
code does not look very trustful to me.

-Bertrand

[1] 
https://git1-us-west.apache.org/repos/asf?p=incubator-groovy.git;a=blob;f=src/test/org/codehaus/groovy/runtime/m12n/ExtensionModuleHelperForTests.groovy;h=01fb34e1fcfb6ed84bbc1630d12218e6dfd566d4;hb=1bb5c5a9

Re: Jars in disguise (was: [VOTE] Release Apache Groovy 2.4.4-incubating)

[Jochen Theodorou]

Am 15.06.2015 19:31, schrieb Roman Shaposhnik:
[...]

It is not even that. It is the issue that anytime you allow a binary artifact
as a requirement you take a huge step away from bootstrapping your
system from source. To me this is a super important requirement, but
then again I come from OS UNIX distribution building background.


the source will compile and work just fine without that jar, so it is 
not about bootstrapping. It is about a test that requires a jar file.


Couldn't we create the file using the build system? I know, it is 
getting more and more complicated since we got to apache and all that, 
still generating the jars would solve the problem, I imagine... well 
license wise... running tests from IDEs won't become easier with that.


bye blackdrag

--
Jochen "blackdrag" Theodorou
blog: http://blackdragsview.blogspot.com/

Re: Jars in disguise (was: [VOTE] Release Apache Groovy 2.4.4-incubating)

[Roman Shaposhnik]

On Mon, Jun 15, 2015 at 12:12 AM, Bertrand Delacretaz
 wrote:
> Hi,
>
> On Sun, Jun 14, 2015 at 8:43 PM, Pascal Schumacher
>  wrote:
>> ...jars under "src/test/resources" are now gone
>
> Encoding those jars in base64 as in [1] does not mean they are gone,
> they're just encoded in a different way.

+1.

> If those jars are useful for testing, it's much preferable IMO to have
> them as jar files to make it obvious that the source code does contain
> binaries. Maybe isolate them in a single folder and include a readme
> file in their which explains why they are useful and lists their sha1
> digests so people can detect any changes to them.
>
> That code says
>
>> // Apache License does not allow jars/binaries in the source
>
> but avoiding binaries in Apache releases has nothing to do with the
> Apache License, it's just that there's usually no way for someone to
> verify that a lone binary file has not been tampered with, and we want
> our users to be able to trust what we release. Hiding stuff in source
> code does not look very trustful to me.

It is not even that. It is the issue that anytime you allow a binary artifact
as a requirement you take a huge step away from bootstrapping your
system from source. To me this is a super important requirement, but
then again I come from OS UNIX distribution building background.

Thanks,
Roman.