Ittay
All run without tests
Hans Dockter wrote:
Hi,
I have focused on improving and understanding Gradle performance the
last days. I'm pretty happy with the results and the outlook.
In trunk we have already implemented caching the compiled build
script. On my machine this compiling the build script for the Gradle
build takes about 0.4 seconds. This compilation happens so far for
every primary task execution. So gradle clean dists consumes 1
second for compilation. With caching this is now in realm of the
milliseconds.
What we also want to implement soon is caching the jar for the build
sources (for build sources see UG 15.1). That would make using build
sources even more attractive. There would be no performance penalty
involved any longer.
The next issue is about evaluating the compiled build script. It was
important for me to learn that Groovy not just has a startup time
but also does further initialization along the way. For example
evaluating the Gradle build script takes 0.8 seconds the first time.
If you do the same call another time it comes down to 0.1 seconds.
This 0.1 seconds is likely to even dramatically improve due to the
expected performance improvements of Groovy in the future. For a
single project build is does not matter any way. But if you have a
multi-project build with 20 projects it sums up. But there are no
Gradle bottlenecks to deal with. Ittay has proposed to cache the
whole project tree. I have come to the conclusion that this is
problematic:
- Technological problems: Right now Groovy does not allow to
serialize closures. This is supposed to change with 1.6 I think. But
our project objects can contain any other objects. They all have to
be serializable to allow caching. At least I can't see any other
means to cache the project tree except serialization.
- Caching the project tree is really freezing things. Even changing
properties would not be reflected any longer except you manually
declare the cache invalid.
- The performance benefits are not as high as originally expected by
myself.
I think improving Groovy's startup and initialization time is the
way to go. The Groovy team is not focusing on this right now and it
is not sure if there are means to achieve this. I think the way to
go for Gradle is what Ittay has proposed. Have a background process
that is fired up the first time Gradle is called. After this you
have a fully initialized Gradle/Groovy. This would solve the startup
time issue. Of course this would be optional. This feature is not on
the 0.3 list of features but should make it into 1.0. We might use
nailgun for this (Also used by JRuby to improve startup time,
although JRuby has a much bigger startup time issue than Groovy).
All this is also good news for the IDE efforts on the way. If
someone change the build script it would take right now 0.5 seconds
to recompile and regenerate the tasks tree for a big build script
like the Gradle build (if the system is initialized, which is easy
to achieve within an IDE). This is kind of OK already I would say.
We can expect it to improve significantly in the future (due to
Groovy's performance improvements).
Even right now I don't think Gradle has a big performance issue. For
example for the Gradle build the compile and unit test times makes 1
second startup time pretty irrelevant. On the other for short tasks
its nice to have a responsive system. And we reduce the carbon foot
print ;)
- Hans
--
Hans Dockter
Gradle Project lead
http://www.gradle.org
---------------------------------------------------------------------
To unsubscribe from this list, please visit:
http://xircles.codehaus.org/manage_email
--
--
Ittay Dror <[EMAIL PROTECTED]>
allprojects{
usePlugin('java')
sourceCompatibility = 1.5
targetCompatibility = 1.5
group = 'org.apache.abdera'
version = '1.0'
dependencies {
addMavenRepo()
addMavenStyleRepo('my-incubating',
'http://people.apache.org/repo/m2-incubating-repository')
testCompile "junit:junit:3.8.2"
}
}
def ACTIVATION =
'org.apache.geronimo.specs:geronimo-activation_1.0.2_spec:1.1'
def AXIOM = ['org.apache.ws.commons.axiom:axiom-api:1.2.5',
'org.apache.ws.commons.axiom:axiom-impl:1.2.5']
def COMMONS = [
codec: 'commons-codec:commons-codec:1.3',
collections: 'commons-collections:commons-collections:3.2',
httpclient: 'commons-httpclient:commons-httpclient:3.1-rc1',
logging: 'commons-logging:commons-logging:1.0.4'
]
def JAXEN = 'jaxen:jaxen:1.1.1'
// def JETTY = group('jetty', 'jetty-util', :under =>
'org.mortbay.jetty', :version => '6.1.7')
def SERVLET_API = 'javax.servlet:servlet-api:2.4'
def STAX = ['stax:stax-api:1.0.1',
'org.codehaus.woodstox:wstx-asl:3.2.1']
project(':core') {
dependencies.compile project(':dependencies:i18n'), ACTIVATION,
COMMONS.codec, COMMONS.logging
}
project(':parser') {
dependencies.compile project(':core'), AXIOM, STAX, JAXEN,
COMMONS.logging
}
project(':protocol') {
dependencies.compile project(':core')
}
project(':client') {
dependencies.compile project(':parser'), project(':protocol'),
COMMONS.httpclient
}
project(':server') {
dependencies.compile project(':parser'), project(':protocol'),
SERVLET_API
}
project(':security') {
dependencies.compile project(':server'), project(':client')
}
project(':spring') {
dependencies.compile project(':server')
}
project(':jcr') {
dependencies.compile project(':client'), project(':spring'),
'javax.jcr:jcr:jar:1.0', ['jackrabbit-api', 'jackrabbit-core',
'jackrabbit-jcr-commons',
'jackrabbit-text-extractors'].collect{'org.apache.jackrabbit:' + it +
':1.3.1'}
}
project(':extensions:main') {
dependencies.compile project(':client')
}
project(':extensions:html') {
dependencies.compile project(':client')
}
project(':extensions:gdata') {
dependencies.compile project(':client'), COMMONS.httpclient
}
project(':extensions:geo') {
dependencies.compile project(':client')
}
project(':extensions:json') {
dependencies.compile project(':extensions:main'),
project(':extensions:html'), project(':server')
}
project(':extensions:media') {
dependencies.compile project(':client')
}
project(':extensions:opensearch') {
dependencies.compile project(':client')
}
project(':extensions:sharing') {
dependencies.compile project(':client')
}
project(':extensions:wsse') {
dependencies.compile project(':client')
}
project(':extensions:oauth') {
dependencies.compile project(':client')
}
project(':extensions:serializer') {
dependencies.compile project(':server')
}
project(':extensions:features') {
dependencies.compile project(':server'), project(':client')
}
project(':examples') {
dependencies.compile project(':extensions').subprojects
}
VERSION_NUMBER = '0.4.0-incubating-SNAPSHOT'
GROUP = 'org.apache.abdera'
ACTIVATION =
'org.apache.geronimo.specs:geronimo-activation_1.0.2_spec:jar:1.1'
AXIOM = group('axiom-api', 'axiom-impl', :under =>
'org.apache.ws.commons.axiom', :version => '1.2.5')
COMMONS = struct(
:codec => 'commons-codec:commons-codec:jar:1.3',
:collections => 'commons-collections:commons-collections:jar:3.2',
:httpclient => 'commons-httpclient:commons-httpclient:jar:3.1-rc1',
:logging => 'commons-logging:commons-logging:jar:1.0.4'
)
JAXEN = 'jaxen:jaxen:jar:1.1.1'
JETTY = group('jetty', 'jetty-util', :under => 'org.mortbay.jetty',
:version => '6.1.7')
SERVLET_API = 'javax.servlet:servlet-api:jar:2.4'
STAX = ['stax:stax-api:jar:1.0.1',
'org.codehaus.woodstox:wstx-asl:jar:3.2.1']
repositories.remote << 'http://repo1.maven.org/maven2' <<
'http://people.apache.org/repo/m2-incubating-repository'
desc 'Atom Specification Implementation'
define 'abdera' do
project.version = VERSION_NUMBER
project.group = GROUP
compile.options.target = '1.5'
desc 'Abdera Source Dependencies'
define 'dependencies' do
desc 'RFC 3987 (Internationlaized Resource Identifier) libraries'
define 'i18n' do
package :jar
end
end
desc 'Atom Specification Implementation Core'
define 'core' do
compile.with project('dependencies:i18n'), ACTIVATION, COMMONS.codec,
COMMONS.logging
package :jar
end
desc 'Atom Specification Implementation Parser'
define 'parser' do
compile.with projects('dependencies:i18n', 'core'), AXIOM, STAX,
JAXEN, COMMONS.logging
package :jar
end
desc 'Atom Publishing Protocol Implementation Core'
define 'protocol' do
compile.with projects('dependencies:i18n', 'core'), ACTIVATION,
COMMONS.codec
package :jar
end
desc 'Atom Publishing Protocol Client Implementation'
define 'client' do
compile.with projects('dependencies:i18n', 'core', 'parser',
'protocol'), COMMONS.httpclient
test.with JETTY, SERVLET_API
package :jar
end
desc 'Atom Publishing Protocol Specification Server-Side Implementation'
define 'server' do
compile.with projects('dependencies:i18n', 'core', 'parser',
'protocol'), SERVLET_API, COMMONS.logging
test.with project('client'), COMMONS.httpclient, JETTY
package :jar
end
desc 'Atom Specification Implementation Security'
define 'security' do
compile.with projects('dependencies:i18n', 'core', 'parser',
'client', 'protocol', 'server'), AXIOM, COMMONS.codec,
'bouncycastle:bcprov-jdk15:jar:124', 'xalan:xalan:jar:2.7.0',
'xerces:xercesImpl:jar:2.8.1', 'xml-security:xmlsec:jar:1.3.0'
test.with COMMONS.httpclient, JETTY, SERVLET_API
package :jar
end
desc 'Abdera Spring Integration'
define 'spring' do
compile.with projects('dependencies:i18n', 'core','parser',
'protocol', 'server'), SERVLET_API,
Buildr::group('spring-beans', 'spring-context', 'spring-core',
'spring-web', :under => 'org.springframework', :version => '2.0.6')
test.with COMMONS.logging, 'org.springframework:spring-mock:jar:2.0.6'
package :jar
end
desc 'JCR AtomPub Server implementation Abdera'
define 'jcr' do
compile.with projects('dependencies:i18n', 'core', 'parser',
'protocol', 'client', 'server', 'spring'),
COMMONS.logging, 'javax.jcr:jcr:jar:1.0',
Buildr::group('jackrabbit-api', 'jackrabbit-core',
'jackrabbit-jcr-commons', 'jackrabbit-text-extractors', :under =>
'org.apache.jackrabbit', :version => '1.3.1')
test.with JETTY, SERVLET_API, COMMONS.httpclient,
COMMONS.collections, COMMONS.codec, AXIOM, JAXEN,
Buildr::group('slf4j-api', 'slf4j-jdk14', :under => 'org.slf4j',
:version => '1.4.3'),
'concurrent:concurrent:jar:1.3.4',
'org.apache.derby:derby:jar:10.2.1.6',
'org.apache.lucene:lucene-core:jar:2.0.0'
package :jar
end
desc 'Abdera Extensions'
define 'extensions' do
desc 'Atom Specification Extensions - Main'
define 'main' do
compile.with projects('dependencies:i18n', 'core', 'parser',
'protocol', 'client'), AXIOM, STAX
package :jar
end
desc 'Atom Specification Extensions - HTML'
define 'html' do
compile.with projects('dependencies:i18n', 'core', 'parser',
'protocol', 'client'), AXIOM, STAX,
'nu.validator.htmlparser:htmlparser:jar:1.0.5'
package :jar
end
desc 'Atom Specification Extensions - GData'
define 'gdata' do
compile.with projects('dependencies:i18n', 'core', 'parser',
'protocol', 'client'), AXIOM, STAX, COMMONS.httpclient
package :jar
end
desc 'Atom Specification Extensions - Geo'
define 'geo' do
compile.with projects('dependencies:i18n', 'core', 'parser',
'protocol', 'client'), AXIOM, STAX
package :jar
end
desc 'Atom Specification Extensions - JSON'
define 'json' do
compile.with projects('dependencies:i18n', 'core', 'parser',
'protocol', 'client', 'main', 'html', 'server'), AXIOM, STAX,
SERVLET_API
package :jar
end
desc 'Atom Specification Extensions - Media'
define 'media' do
compile.with projects('dependencies:i18n', 'core', 'parser',
'protocol', 'client'), AXIOM, STAX
package :jar
end
desc 'Atom Specification Extensions - OpenSearch'
define 'opensearch' do
compile.with projects('dependencies:i18n', 'core', 'parser',
'protocol', 'client'), AXIOM, STAX
package :jar
end
desc 'Atom Specification Extensions - Sharing'
define 'sharing' do
compile.with projects('dependencies:i18n', 'core', 'parser',
'protocol', 'client'), AXIOM, STAX
package :jar
end
desc 'Atom Specification Extensions - WSSE'
define 'wsse' do
compile.with projects('dependencies:i18n', 'core', 'parser',
'protocol', 'client'), COMMONS.httpclient, COMMONS.codec
package :jar
end
desc 'Atom Specification Extensions - OAuth'
define 'oauth' do
compile.with projects('dependencies:i18n', 'core', 'parser',
'protocol', 'client'), AXIOM, STAX, COMMONS.httpclient, COMMONS.codec
test.with COMMONS.logging
package :jar
end
desc 'Atom Specification Extensions - Serializer'
define 'serializer' do
compile.with projects('dependencies:i18n', 'core', 'parser',
'protocol', 'server'), AXIOM, STAX
package :jar
end
desc 'Some features I guess'
define 'features' do
compile.with projects('dependencies:i18n', 'core', 'parser',
'protocol', 'server', 'client'), AXIOM, STAX
package :jar
end
end
desc 'Atom Specification Implementation Examples'
define 'examples' do
compile.with projects('dependencies:i18n', 'core', 'parser',
'protocol', 'client', 'server', 'security'),
project('extensions').projects('main', 'serializer', 'gdata', 'geo',
'sharing', 'features'), JETTY, AXIOM, JAXEN, SERVLET_API,
COMMONS.httpclient
package :jar
end
end
include
'client','core','dependencies','dependencies:i18n','examples','extensions','extensions:features','extensions:gdata','extensions:geo','extensions:html','extensions:json','extensions:main','extensions:media','extensions:oauth','extensions:opensearch','extensions:serializer','extensions:sharing','extensions:wsse','jcr','parser','protocol','security','server','spring'
---------------------------------------------------------------------
To unsubscribe from this list, please visit:
http://xircles.codehaus.org/manage_email
