Re: Maven coordinates going forward

[Paul Moore]

On 27 March 2017 at 19:49, Keith Suderman  wrote:
> +1 for changing Maven coordinates.
>
> -1 for changing package names.  Sure, new code can use the new package
> names, but changing existing packages is just breaking changes for the sake
> of breaking things with no real benefit.  I hope the Groovy team tries to
> break as little as possible to avoid the "Python3 Effect", whether real or
> imagined.

As a Python dev, who's just started working with Groovy for some work
projects, I hope people don't mind me adding my opinion here.

I think you're right. The Python problems were largely because we
broke people's source code (that was a deliberate choice, 2-3 was
always intended as a backward compatibility breaking change, but in
hindsight I think the view is that we won't ever take that approach
again). So the package name change would be a similar compatibility
break for Groovy and you should consider carefully whether the
benefits justify it. From what I've seen of Groovy code, the package
names are needed relatively infrequently, so it's possible that the
breakage would be limited - but what's the benefit? Python's
experience is that simply "tidying up" is pretty risky as a
justification...

I'm not familiar enough with Java's packaging ecosystem and Maven to
know what the impact of changing the Maven coordinates would be, so
I'll say nothing on that one.

I know very little about the Groovy or Java communities, though. Their
tolerance for change is likely very different from the Python
community's (both for better and worse) so I'd rely more on your sense
of what your users will think than on what happened with Python.

Hope this helps,

Paul

Re: Optimising a Groovy script

[Paul Moore]

On 29 March 2017 at 14:56, Nelson, Erick  wrote:
> I'm not sure using or not using def would cause performance differences.

There definitely *seems* to be a difference.

def rng = new MersenneTwister()

def roll = {
rng.nextInt(6) + rng.nextInt(6) + rng.nextInt(6) + 3
}

Both def: 0.4 to 0.5 sec
No def on rng, def on roll: 0.5 to 0.8 sec - spread seems greater, too
Def on rng, not on roll: 1.2 to 1.6 sec, with a couple of 1.9s.
Def on neither: 1.4 to 1.9 sec.

int roll() and no def on rng looks similar to def on roll and not on
rng. But def on rng isn't allowed when I'm using int roll().

All results from multiple runs on my PC, summarising the average/spread by eye.

So the big difference is removing def from roll, with removing def
from rng having a smaller but detectable effect.

> It just affects variable scope.
>
> http://mrhaki.blogspot.com/2009/11/groovy-goodness-variable-scope-in.html

I still find this confusing. That article says that def or type are
equivalent (in terms of scope). But int roll() doesn't allow def rng,
whereas def roll does.

I assume that the issue here is that

def roll = { ... }

is making roll an attribute whose value is a closure. Whereas

int roll() { ... }

is making roll a method? So that implies that you can't reference a
def from within a method, but you can reference it from within a
closure? OK, as a rule I can accept that might be the case, but I
don't understand why. Furthermore, I can't even refer to rng as
"this.rng" from within roll, which is very confusing, as I thought a
script was compiled as an implicit class, in which case isn't this the
instance of that class, and hence this.attr should be a means of
accessing any attribute of the class?

Looking at the details in the manual ("Program Structure" section 3
"Scripts vs Classes") I think I start to understand:

* def roll = { } is declaring a local variable in the implied run()
method, whose value is a closure
* roll = { } declares a variable in the "script binding" whose value
is a closure
* int roll { } is a method of the script class
* def rng = ... is a local variable of the run method, which the docs
explicitly state is not available to methods
* rng = ... is a variable in the script binding which *is* visible from methods
* @Field rng = ...is a field, which is what I'd need to use for
this.rng to work as I'm expecting

I'll have to look some more into the script binding. I don't really
understand the visibility rules for values in the binding, apart from
the few cases explicitly noted (e.g. "they are available from
methods").

Wow, this is more complicated than it looks at first glance!!!

On the other hand, fields, methods, closures and script bindings are
all very different things, so I can easily imagine them having
different performance characteristics. In don't know how I'd develop
an intuition about which is the right one to use in any specific
situation, though. Try them all and see which works better feels like
the only practical option at the moment :-(

Thanks,
Paul

Re: Optimising a Groovy script

[Paul Moore]

On 29 March 2017 at 15:34, Paul Moore <p.f.mo...@gmail.com> wrote:
> So the big difference is removing def from roll, with removing def
> from rng having a smaller but detectable effect.

I just tried to generalise the script, by making a simulate function
that takes the action as a closure. I won't post all the code here,
but basically:

def simulate = { N, cl ->

def results = ...
N.times {
int n = cl()
results[n]++
}

// Report the results
}

simulate(100) {
... body of roll() here
}

This took 20 seconds to run.

It's certainly possible I've made a stupid mistake here, but I thought
that doing this was essentially a simple refactoring of the original
code, and I'm pretty surprised to see a 40x increase in runtime.
Before I spend ages hunting for my mistake, is there any obvious
reason why this *isn't* just a refactoring, and I should have expected
it to run a lot slower?

(If nothing else, this is a great learning exercise for me :-))

Thanks,
Paul

Re: Optimising a Groovy script

[Paul Moore]

On 28 March 2017 at 22:08, Nelson, Erick  wrote:
> Try this...

Thanks for the suggestion - there were some nice improvements in here.

> def rng = new MersenneTwister()
>
> def roll = {
> rng.nextInt(6) + rng.nextInt(6) + rng.nextInt(6) + 3
> }

You changed my definitions to use "def" here. This seems to be the
thing that makes the most difference in performance. I'm really
struggling to find a good explanation as to the effect of using or not
using "def". I had imagined that using "int roll() {..." would be
better, as it explicitly states the types which would help the
compiler avoid the need for generic code. Obviously I was wrong, but
I'm not at all clear why.

Also, if I use "def rng" but keep "int roll()", I get an error "No
such property: rng". I'm not clear why that is.

Do you know of a good resource that explains the difference between
using def and not doing so? I'm currently working my way through
"Groovy in Action" and while the subject has been discussed, I didn't
really follow it. I've also looked at the online docs and they haven't
helped a lot. It's quite possible that my confusion comes from the
fact that I only really have a casual knowledge of Java, so the
precise way classes reference properties and variables isn't clear to
me - if there's some background reading in Java that would help
clarify, that would be useful too.

> int N = 100
> def results = [:].withDefault{0}

I never knew about withDefault - that's a really nice feature, thanks!

Thanks for your help,
Paul

Re: foo() || return false ?

[Paul Moore]

Why do you need something "more concise"? It seems pretty OK to me...

On 18 July 2017 at 15:00, Guy Matz  wrote:
> Right!  This is what I'm trying to do, but in a more concise way . . .  any
> thoughts anyone?
>
> On Mon, Jul 17, 2017 at 8:43 PM, J. David Beutel  wrote:
>>
>> I guess OP wants to do:
>>
>> if (!doSomething()) {
>> return false
>> }

Re: What is the best replacement for running scripts using groovy-all?

[Paul Moore]

I can understand that logic - there are quirks I've hit with the fat
jar approach, but because I made the fat jar myself, I'm OK with just
assuming it's a weirdness that I can either live with or investigate.
But if it happened with a supplied jar, I'd feel that I should report
the problem (even if I could work around it) in case it hit others,
too. So that's an extra burden on the developers, as you say.

Having an uber jar in the zip distribution would have solved my use
case. Or even just a recipe in the documentation explaining how to
build an uber jar for yourself. If the Gradle script I posted here
earlier is of any help, feel free to use it.

Paul

On Thu, 20 Dec 2018 at 00:44, Paul King  wrote:
>
> We don't want to publish a fat jar to maven because it will cause problems 
> that we will continually be asked about but perhaps having an uber jar within 
> the zip distribution might be something we could look at. I suspect over time 
> though that even that could be problematic.
>
> On Thu, Dec 20, 2018 at 8:46 AM MG  wrote:
>>
>> Hi,
>>
>> out of curiosity (and because having a fat jar again might be
>> conventient at some point in the future in my work environment (also no
>> internet access)):
>>
>> This solution proposed by Keith does not work
>> https://github.com/gradle/gradle-groovy-all
>> ?
>>
>> Cheers,
>> mg
>>
>>
>>
>> Am 19.12.2018 um 23:33 schrieb Paul Moore:
>> > On Wed, 19 Dec 2018 at 21:23, James Kleeh  wrote:
>> >> Paul,
>> >>
>> >> The best solution is to use Maven or Gradle to create an all-in-one (fat) 
>> >> jar that you can ship and run with java -jar
>> >>
>> >> Gradle has a shadow plugin and Maven has a shade plugin to do just that.
>> > Thanks. I'd come to the conclusion that Gradle was likely the solution
>> > I should be looking at, and I've spent the evening trying to set up a
>> > basic Gradle script that does what I want. After a lot of
>> > experimentation, I came up with the following, which seems to do what
>> > I want:
>> >
>> > -- start build.gradle --
>> >
>> > version = "0.1"
>> >
>> > configurations {
>> >  deploy
>> > }
>> >
>> > dependencies {
>> >  deploy 'org.codehaus.groovy:groovy-all:2.5.4'
>> > }
>> >
>> > repositories {
>> >  jcenter()
>> > }
>> >
>> > task copyDeps(type:Copy, group: "Custom", description: "Copies project
>> > dependencies") {
>> >  from configurations.deploy.collect { it.absolutePath }
>> >  into "dest/lib"
>> > }
>> >
>> > task copy(type: Copy, group: "Custom", description: "Copies sources to
>> > the dest directory") {
>> >  from "src"
>> >  include "*.groovy"
>> >  into "dest"
>> > }
>> >
>> > task deploy(type:Zip, group: "Custom", description: "Build a deployment 
>> > zip") {
>> >  dependsOn copyDeps
>> >  dependsOn copy
>> >  from "dest"
>> >  setArchiveName "${project.name}-${project.version}.zip"
>> > }
>> >
>> > -- end build.gradle --
>> >
>> > It doesn't create a fat jar yet, but I can look into setting that up.
>> > The various existing plugins seem to be dependent upon the
>> > infrastructure set up by the java plugin, which I don't really
>> > understand (or need, as far as I can tell) so they may not be of much
>> > help. But I'm not sure what I need to do yet to write my own.
>> > Something simple like
>> >
>> > task customFatJar(type: Jar) {
>> >  dependsOn copyDeps
>> >  baseName = 'all-in-one-jar'
>> >  from "dest/lib"
>> > }
>> >
>> > gives me an "all-in-one-jar.jar" that contains the dependency jars
>> > directly included, rather than being unpacked. So there's more I need
>> > to do here...
>> >
>> > Paul
>> >
>>

Re: What is the best replacement for running scripts using groovy-all?

[Paul Moore]

On Wed, 19 Dec 2018 at 08:56, Paul Moore  wrote:
>
> On Wed, 19 Dec 2018 at 00:03, Keith Suderman  wrote:
> >
> > Option 4) Use the Maven Assembly plugin or the Shade plugin to build your 
> > own groovy-all Jar file.  Or just use 
> > https://github.com/gradle/gradle-groovy-all
>
> Thanks. Are there any "beginner guide" style instructions on how to
> use the Maven Assembly plugin or Shade plugin that you can point me
> to? As I say, I don't use Maven, so the instructions for the plugins
> use a lot of terms and ideas I'm not familiar with. I can (and
> probably will!) use the gradle-groovy-all but I'd like to learn a bit
> more about the Java ecosystem (I'm mostly a Python programmer, but I
> use Groovy as an alternative for environments where JVM-based tools
> are a better fit than Python-based ones). I find that starting Groovy
> *without* a Java/JVM background, there's a lot of assumed knowledge
> it's quite hard to pick up (unless you're willing to learn Java at the
> same time ;-))

I've been digging around with this some more, and I've come to the
conclusion that it's not that important to me in fact to have a single
groovy-all jar for my deployment. But what I *do* need is a simple way
to collect together everything I need to run my script(s) and ship
them to the target machine(s). So my starting point is one or more
.groovy files. I do *not* want to compile these - I want to ship the
source script to the server, so that minor changes can be made in
place using just a text editor. And with them, I want a directory full
of supporting jar files.

Having created and tested the scripts, I need to collect together all
of the jar files I used to run them. Obviously, the first thing I need
is the Groovy jars. Ideally I'd try to strip out unneeded jars (my
code is to be run on a server with no GUI, so I suspect the
groovy-swing jar could be skipped, for example). But that's probably
way more trouble than it's worth, so I'm OK with skipping that step.
Other dependencies, I've tended to collect from various places (for
development, I can use @Grab annotations in the source, but my server
doesn't have Internet access, so that won't work for the deployed
version).

>From what I gather with Java projects, dependencies get managed by a
tool like Maven or Gradle or by the IDE. But it's very hard for me to
understand the documentation for these tools, as they are typically
looking at the problem from the point of view of "compile and build a
binary from the sources" rather than "collect dependencies into one
place, but don't compile anything". One problem I'm struggling with is
that with my background, what I'm trying to do is "obviously" the
right approach, but I get the feeling that it's very different from
the Java/Groovy way of doing things, so I keep missing the point of
people's explanations.

Essentially, what I want is a project structure like this:

MyProject
script1.groovy
script2.groovy
script3.groovy
script4.groovy
dependencies.txt
target
lib

dependencies.txt can be anything but what it contains should be a list
of dependencies - something like

org.codehaus.groovy:groovy-all:pom:2.5.4
javax.mail:mail:jar:1.4.4
org.apache.commons:commons-csv:jar:1.6

Running "some command" should then copy all the jars needed (based on
those dependencies) to target/lib. Ideally, copy *.groovy to target as
well, so I can just zip up the target directory, ship it to the
destination machine, where I can unzip it and run it with whatever JVM
is present there.

Am I missing something fundamental which makes this impossible to
achieve with Java, or is it just that my Google skills have failed me?
Or is it that Java projects simply aren't normally of this form?

Paul

Re: What is the best replacement for running scripts using groovy-all?

[Paul Moore]

On Wed, 19 Dec 2018 at 21:23, James Kleeh  wrote:
>
> Paul,
>
> The best solution is to use Maven or Gradle to create an all-in-one (fat) jar 
> that you can ship and run with java -jar
>
> Gradle has a shadow plugin and Maven has a shade plugin to do just that.

Thanks. I'd come to the conclusion that Gradle was likely the solution
I should be looking at, and I've spent the evening trying to set up a
basic Gradle script that does what I want. After a lot of
experimentation, I came up with the following, which seems to do what
I want:

-- start build.gradle --

version = "0.1"

configurations {
deploy
}

dependencies {
deploy 'org.codehaus.groovy:groovy-all:2.5.4'
}

repositories {
jcenter()
}

task copyDeps(type:Copy, group: "Custom", description: "Copies project
dependencies") {
from configurations.deploy.collect { it.absolutePath }
into "dest/lib"
}

task copy(type: Copy, group: "Custom", description: "Copies sources to
the dest directory") {
from "src"
include "*.groovy"
into "dest"
}

task deploy(type:Zip, group: "Custom", description: "Build a deployment zip") {
dependsOn copyDeps
dependsOn copy
from "dest"
setArchiveName "${project.name}-${project.version}.zip"
}

-- end build.gradle --

It doesn't create a fat jar yet, but I can look into setting that up.
The various existing plugins seem to be dependent upon the
infrastructure set up by the java plugin, which I don't really
understand (or need, as far as I can tell) so they may not be of much
help. But I'm not sure what I need to do yet to write my own.
Something simple like

task customFatJar(type: Jar) {
dependsOn copyDeps
baseName = 'all-in-one-jar'
from "dest/lib"
}

gives me an "all-in-one-jar.jar" that contains the dependency jars
directly included, rather than being unpacked. So there's more I need
to do here...

Paul

Re: What is the best replacement for running scripts using groovy-all?

[Paul Moore]

On Wed, 19 Dec 2018 at 22:33, Paul Moore  wrote:

> Something simple like
>
> task customFatJar(type: Jar) {
> dependsOn copyDeps
> baseName = 'all-in-one-jar'
> from "dest/lib"
> }
>
> gives me an "all-in-one-jar.jar" that contains the dependency jars
> directly included, rather than being unpacked. So there's more I need
> to do here...

Actually, I found the following on the web, which seems to work:

task customFatJar(type: Jar) {
dependsOn copyDeps
baseName = 'all-in-one-jar'
from {
configurations.deploy.collect {
it.isDirectory() ? it : zipTree(it)
}
}
}

I'm not at all sure *why* it works, but it does :-)

Paul

Re: What is the best replacement for running scripts using groovy-all?

[Paul Moore]

On Wed, 19 Dec 2018 at 20:18, Søren Berg Glasius  wrote:
>
> Hi Paul,
>
> This is where The @Grab anotation comes in handy: 
> http://docs.groovy-lang.org/latest/html/documentation/grape.html
>
> It wil automatically download your dependencies and it works in Groovy 
> scripts too.

Thanks - yes, I've seen @Grab, and used it while testing. But the
problem is that it puts the dependency files "somewhere", but not
alongside the script. I need to ship the script and its dependencies
to another machine with no web access, so I need better control over
where the dependencies end up. (I could probably hunt out where the
files downloaded by @Grab went, but it would be a completely manual
task to locate them all and copy them, and mistakes would happen - so
I'd prefer something automated.

Paul

Re: What is the best replacement for running scripts using groovy-all?

[Paul Moore]

On Wed, 19 Dec 2018 at 22:46, MG  wrote:
>
> Hi,
>
> out of curiosity (and because having a fat jar again might be
> conventient at some point in the future in my work environment (also no
> internet access)):
>
> This solution proposed by Keith does not work
> https://github.com/gradle/gradle-groovy-all
> ?

See https://github.com/gradle/gradle-groovy-all/issues/1

Basically, yes it does, but only for groovy.ui.GroovyMain (which is
fine) and they don't support usage for anything other than Gradle's
purposes and they are looking to discontinue it once Gradle no longer
needs it (which isn't quite as fine ;-))

In addition, by not going with that solution, I've learned a lot about
Gradle and how to use it to solve my problem, which is much better, as
not only do I have a solution, I also learned something new :-) By the
way - my Gradle solution also doesn't work for groovy.ui.Console, but
that's fine as I say above (and it's a chance to learn more, working
out why :-))

Paul