hdockter wrote:
>
>
> On Jun 23, 2008, at 1:02 PM, Ittay Dror wrote:
>
>> Hi Hans,
>>
>> I've tried to test the performance of gradle on a not trivial project.
>> So far, it doesn't look good. I couldn't make the project to build and
>> the performance was very poor.
>>
>> The project I'm using as reference is abder
>> (http://svn.apache.org/repos/asf/incubator/abdera/java/trunk). It is a
>> simple project mainly with jars that is currently built using maven. A
>> guy by the name of Stephen Duncan posted a buildr buildfile for this
>> project and I created build.gradle and settings.gradle. All attached.
>>
>> Buildr (> time buildr install test=off) compiles the project (without
>> the 'bundle' module) in 6 seconds on my machine
>> Maven (> time mvn -Dmaven.test.skip=true -o compile) takes 36 seconds
>> for the 'compile' phase
>> Gradle (> time gradle -Dskip.test -Dskip.integTest libs) takes 46
>> seconds until it fails trying to compile the parser module (the 3rd
>> module only) because javac doesn't get the path to the i18n.jar
>> although
>> it depends on 'core' which depends on the dependencies:i18n project.
>
> Oops. The generation of the DAG takes a very long time. This needs
> significant improvement!
>
> I have to figure out what the issue with the transitive dependencies is.
>
what about the build failure (transitive project dependencies)?
hdockter wrote:
>
>> furthermore, the i18n jar is created as just 'i18n', i would have
>> expected the full name of the project ('dependencies-i18n'), to avoid
>> name collisions.
>
> We had it first this way. Then I found it to be usually not what I
> want. We might offer a convenient way to switch 'full project path'
> naming on and off.
>
> - Hans
>
>>
>> Ittay
>>
>> All run without tests
>>
>> Hans Dockter wrote:
>>> Hi,
>>>
>>> I have focused on improving and understanding Gradle performance
>>> the last days. I'm pretty happy with the results and the outlook.
>>>
>>> In trunk we have already implemented caching the compiled build
>>> script. On my machine this compiling the build script for the
>>> Gradle build takes about 0.4 seconds. This compilation happens so
>>> far for every primary task execution. So gradle clean dists
>>> consumes 1 second for compilation. With caching this is now in
>>> realm of the milliseconds.
>>>
>>> What we also want to implement soon is caching the jar for the
>>> build sources (for build sources see UG 15.1). That would make
>>> using build sources even more attractive. There would be no
>>> performance penalty involved any longer.
>>>
>>> The next issue is about evaluating the compiled build script. It
>>> was important for me to learn that Groovy not just has a startup
>>> time but also does further initialization along the way. For
>>> example evaluating the Gradle build script takes 0.8 seconds the
>>> first time. If you do the same call another time it comes down to
>>> 0.1 seconds. This 0.1 seconds is likely to even dramatically
>>> improve due to the expected performance improvements of Groovy in
>>> the future. For a single project build is does not matter any way.
>>> But if you have a multi-project build with 20 projects it sums up.
>>> But there are no Gradle bottlenecks to deal with. Ittay has
>>> proposed to cache the whole project tree. I have come to the
>>> conclusion that this is problematic:
>>> - Technological problems: Right now Groovy does not allow to
>>> serialize closures. This is supposed to change with 1.6 I think.
>>> But our project objects can contain any other objects. They all
>>> have to be serializable to allow caching. At least I can't see any
>>> other means to cache the project tree except serialization.
>>> - Caching the project tree is really freezing things. Even
>>> changing properties would not be reflected any longer except you
>>> manually declare the cache invalid.
>>> - The performance benefits are not as high as originally expected
>>> by myself.
>>>
>>> I think improving Groovy's startup and initialization time is the
>>> way to go. The Groovy team is not focusing on this right now and
>>> it is not sure if there are means to achieve this. I think the way
>>> to go for Gradle is what Ittay has proposed. Have a background
>>> process that is fired up the first time Gradle is called. After
>>> this you have a fully initialized Gradle/Groovy. This would solve
>>> the startup time issue. Of course this would be optional. This
>>> feature is not on the 0.3 list of features but should make it into
>>> 1.0. We might use nailgun for this (Also used by JRuby to improve
>>> startup time, although JRuby has a much bigger startup time issue
>>> than Groovy).
>>>
>>> All this is also good news for the IDE efforts on the way. If
>>> someone change the build script it would take right now 0.5
>>> seconds to recompile and regenerate the tasks tree for a big build
>>> script like the Gradle build (if the system is initialized, which
>>> is easy to achieve within an IDE). This is kind of OK already I
>>> would say. We can expect it to improve significantly in the future
>>> (due to Groovy's performance improvements).
>>>
>>> Even right now I don't think Gradle has a big performance issue.
>>> For example for the Gradle build the compile and unit test times
>>> makes 1 second startup time pretty irrelevant. On the other for
>>> short tasks its nice to have a responsive system. And we reduce
>>> the carbon foot print ;)
>>>
>>> - Hans
>>>
>>> --
>>> Hans Dockter
>>> Gradle Project lead
>>> http://www.gradle.org
>>>
>>>
>>>
>>>
>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe from this list, please visit:
>>>
>>> http://xircles.codehaus.org/manage_email
>>>
>>>
>>>
>>
>> --
>> --
>> Ittay Dror <[EMAIL PROTECTED]>
>> allprojects{
>> usePlugin('java')
>> sourceCompatibility = 1.5
>> targetCompatibility = 1.5
>> group = 'org.apache.abdera'
>> version = '1.0'
>> dependencies {
>> addMavenRepo()
>> addMavenStyleRepo('my-incubating', 'http://people.apache.org/
>> repo/m2-incubating-repository')
>> testCompile "junit:junit:3.8.2"
>> }
>> }
>>
>> def ACTIVATION = 'org.apache.geronimo.specs:geronimo-
>> activation_1.0.2_spec:1.1'
>> def AXIOM = ['org.apache.ws.commons.axiom:axiom-api:1.2.5',
>> 'org.apache.ws.commons.axiom:axiom-impl:1.2.5']
>> def COMMONS = [
>> codec: 'commons-codec:commons-codec:1.3',
>> collections: 'commons-collections:commons-collections:3.2',
>> httpclient: 'commons-httpclient:commons-httpclient:3.1-rc1',
>> logging: 'commons-logging:commons-logging:1.0.4'
>> ]
>> def JAXEN = 'jaxen:jaxen:1.1.1'
>> // def JETTY = group('jetty', 'jetty-util', :under =>
>> 'org.mortbay.jetty', :version => '6.1.7')
>> def SERVLET_API = 'javax.servlet:servlet-api:2.4'
>> def STAX = ['stax:stax-api:1.0.1', 'org.codehaus.woodstox:wstx-asl:
>> 3.2.1']
>>
>>
>> project(':core') {
>> dependencies.compile project(':dependencies:i18n'), ACTIVATION,
>> COMMONS.codec, COMMONS.logging
>> }
>>
>> project(':parser') {
>> dependencies.compile project(':core'), AXIOM, STAX, JAXEN,
>> COMMONS.logging
>> }
>>
>> project(':protocol') {
>> dependencies.compile project(':core')
>> }
>>
>> project(':client') {
>> dependencies.compile project(':parser'), project(':protocol'),
>> COMMONS.httpclient
>> }
>>
>> project(':server') {
>> dependencies.compile project(':parser'), project(':protocol'),
>> SERVLET_API
>> }
>>
>> project(':security') {
>> dependencies.compile project(':server'), project(':client')
>> }
>>
>> project(':spring') {
>> dependencies.compile project(':server')
>> }
>>
>> project(':jcr') {
>> dependencies.compile project(':client'), project(':spring'),
>> 'javax.jcr:jcr:jar:1.0', ['jackrabbit-api', 'jackrabbit-core',
>> 'jackrabbit-jcr-commons', 'jackrabbit-text-extractors'].collect
>> {'org.apache.jackrabbit:' + it + ':1.3.1'}
>>
>> }
>>
>> project(':extensions:main') {
>> dependencies.compile project(':client')
>> }
>>
>> project(':extensions:html') {
>> dependencies.compile project(':client')
>> }
>>
>> project(':extensions:gdata') {
>> dependencies.compile project(':client'), COMMONS.httpclient
>> }
>>
>> project(':extensions:geo') {
>> dependencies.compile project(':client')
>> }
>>
>> project(':extensions:json') {
>> dependencies.compile project(':extensions:main'), project
>> (':extensions:html'), project(':server')
>> }
>>
>> project(':extensions:media') {
>> dependencies.compile project(':client')
>> }
>>
>> project(':extensions:opensearch') {
>> dependencies.compile project(':client')
>> }
>>
>> project(':extensions:sharing') {
>> dependencies.compile project(':client')
>> }
>>
>> project(':extensions:wsse') {
>> dependencies.compile project(':client')
>> }
>>
>> project(':extensions:oauth') {
>> dependencies.compile project(':client')
>> }
>>
>> project(':extensions:serializer') {
>> dependencies.compile project(':server')
>> }
>>
>> project(':extensions:features') {
>> dependencies.compile project(':server'), project(':client')
>> }
>>
>> project(':examples') {
>> dependencies.compile project(':extensions').subprojects
>> }
>>
>>
>>
>>
>>
>>
>>
>> VERSION_NUMBER = '0.4.0-incubating-SNAPSHOT'
>> GROUP = 'org.apache.abdera'
>>
>> ACTIVATION = 'org.apache.geronimo.specs:geronimo-
>> activation_1.0.2_spec:jar:1.1'
>> AXIOM = group('axiom-api', 'axiom-impl', :under =>
>> 'org.apache.ws.commons.axiom', :version => '1.2.5')
>> COMMONS = struct(
>> :codec => 'commons-codec:commons-codec:jar:1.3',
>> :collections => 'commons-collections:commons-collections:jar:3.2',
>> :httpclient => 'commons-httpclient:commons-httpclient:jar:3.1-rc1',
>> :logging => 'commons-logging:commons-logging:jar:1.0.4'
>> )
>> JAXEN = 'jaxen:jaxen:jar:1.1.1'
>> JETTY = group('jetty', 'jetty-util', :under =>
>> 'org.mortbay.jetty', :version => '6.1.7')
>> SERVLET_API = 'javax.servlet:servlet-api:jar:2.4'
>> STAX = ['stax:stax-api:jar:1.0.1', 'org.codehaus.woodstox:wstx-
>> asl:jar:3.2.1']
>>
>> repositories.remote << 'http://repo1.maven.org/maven2' << 'http://
>> people.apache.org/repo/m2-incubating-repository'
>>
>> desc 'Atom Specification Implementation'
>> define 'abdera' do
>> project.version = VERSION_NUMBER
>> project.group = GROUP
>> compile.options.target = '1.5'
>>
>> desc 'Abdera Source Dependencies'
>> define 'dependencies' do
>> desc 'RFC 3987 (Internationlaized Resource Identifier) libraries'
>> define 'i18n' do
>> package :jar
>> end
>> end
>>
>> desc 'Atom Specification Implementation Core'
>> define 'core' do
>> compile.with project('dependencies:i18n'), ACTIVATION,
>> COMMONS.codec, COMMONS.logging
>> package :jar
>> end
>>
>> desc 'Atom Specification Implementation Parser'
>> define 'parser' do
>> compile.with projects('dependencies:i18n', 'core'), AXIOM,
>> STAX, JAXEN, COMMONS.logging
>> package :jar
>> end
>>
>> desc 'Atom Publishing Protocol Implementation Core'
>> define 'protocol' do
>> compile.with projects('dependencies:i18n', 'core'), ACTIVATION,
>> COMMONS.codec
>> package :jar
>> end
>>
>> desc 'Atom Publishing Protocol Client Implementation'
>> define 'client' do
>> compile.with projects('dependencies:i18n', 'core', 'parser',
>> 'protocol'), COMMONS.httpclient
>> test.with JETTY, SERVLET_API
>> package :jar
>> end
>>
>> desc 'Atom Publishing Protocol Specification Server-Side
>> Implementation'
>> define 'server' do
>> compile.with projects('dependencies:i18n', 'core', 'parser',
>> 'protocol'), SERVLET_API, COMMONS.logging
>> test.with project('client'), COMMONS.httpclient, JETTY
>> package :jar
>> end
>>
>> desc 'Atom Specification Implementation Security'
>> define 'security' do
>> compile.with projects('dependencies:i18n', 'core', 'parser',
>> 'client', 'protocol', 'server'), AXIOM, COMMONS.codec,
>> 'bouncycastle:bcprov-jdk15:jar:124', 'xalan:xalan:jar:2.7.0',
>> 'xerces:xercesImpl:jar:2.8.1', 'xml-security:xmlsec:jar:1.3.0'
>> test.with COMMONS.httpclient, JETTY, SERVLET_API
>> package :jar
>> end
>>
>> desc 'Abdera Spring Integration'
>> define 'spring' do
>> compile.with projects('dependencies:i18n', 'core','parser',
>> 'protocol', 'server'), SERVLET_API,
>> Buildr::group('spring-beans', 'spring-context', 'spring-
>> core', 'spring-web', :under => 'org.springframework', :version =>
>> '2.0.6')
>> test.with COMMONS.logging, 'org.springframework:spring-mock:jar:
>> 2.0.6'
>> package :jar
>> end
>>
>> desc 'JCR AtomPub Server implementation Abdera'
>> define 'jcr' do
>> compile.with projects('dependencies:i18n', 'core', 'parser',
>> 'protocol', 'client', 'server', 'spring'),
>> COMMONS.logging, 'javax.jcr:jcr:jar:1.0',
>> Buildr::group('jackrabbit-api', 'jackrabbit-core',
>> 'jackrabbit-jcr-commons', 'jackrabbit-text-extractors', :under =>
>> 'org.apache.jackrabbit', :version => '1.3.1')
>> test.with JETTY, SERVLET_API, COMMONS.httpclient,
>> COMMONS.collections, COMMONS.codec, AXIOM, JAXEN,
>> Buildr::group('slf4j-api', 'slf4j-jdk14', :under =>
>> 'org.slf4j', :version => '1.4.3'),
>> 'concurrent:concurrent:jar:1.3.4',
>> 'org.apache.derby:derby:jar:10.2.1.6', 'org.apache.lucene:lucene-
>> core:jar:2.0.0'
>> package :jar
>> end
>>
>> desc 'Abdera Extensions'
>> define 'extensions' do
>> desc 'Atom Specification Extensions - Main'
>> define 'main' do
>> compile.with projects('dependencies:i18n', 'core', 'parser',
>> 'protocol', 'client'), AXIOM, STAX
>> package :jar
>> end
>>
>> desc 'Atom Specification Extensions - HTML'
>> define 'html' do
>> compile.with projects('dependencies:i18n', 'core', 'parser',
>> 'protocol', 'client'), AXIOM, STAX,
>> 'nu.validator.htmlparser:htmlparser:jar:1.0.5'
>> package :jar
>> end
>>
>> desc 'Atom Specification Extensions - GData'
>> define 'gdata' do
>> compile.with projects('dependencies:i18n', 'core', 'parser',
>> 'protocol', 'client'), AXIOM, STAX, COMMONS.httpclient
>> package :jar
>> end
>>
>> desc 'Atom Specification Extensions - Geo'
>> define 'geo' do
>> compile.with projects('dependencies:i18n', 'core', 'parser',
>> 'protocol', 'client'), AXIOM, STAX
>> package :jar
>> end
>>
>> desc 'Atom Specification Extensions - JSON'
>> define 'json' do
>> compile.with projects('dependencies:i18n', 'core', 'parser',
>> 'protocol', 'client', 'main', 'html', 'server'), AXIOM, STAX,
>> SERVLET_API
>> package :jar
>> end
>>
>> desc 'Atom Specification Extensions - Media'
>> define 'media' do
>> compile.with projects('dependencies:i18n', 'core', 'parser',
>> 'protocol', 'client'), AXIOM, STAX
>> package :jar
>> end
>>
>> desc 'Atom Specification Extensions - OpenSearch'
>> define 'opensearch' do
>> compile.with projects('dependencies:i18n', 'core', 'parser',
>> 'protocol', 'client'), AXIOM, STAX
>> package :jar
>> end
>>
>> desc 'Atom Specification Extensions - Sharing'
>> define 'sharing' do
>> compile.with projects('dependencies:i18n', 'core', 'parser',
>> 'protocol', 'client'), AXIOM, STAX
>> package :jar
>> end
>>
>> desc 'Atom Specification Extensions - WSSE'
>> define 'wsse' do
>> compile.with projects('dependencies:i18n', 'core', 'parser',
>> 'protocol', 'client'), COMMONS.httpclient, COMMONS.codec
>> package :jar
>> end
>>
>> desc 'Atom Specification Extensions - OAuth'
>> define 'oauth' do
>> compile.with projects('dependencies:i18n', 'core', 'parser',
>> 'protocol', 'client'), AXIOM, STAX, COMMONS.httpclient, COMMONS.codec
>> test.with COMMONS.logging
>> package :jar
>> end
>>
>> desc 'Atom Specification Extensions - Serializer'
>> define 'serializer' do
>> compile.with projects('dependencies:i18n', 'core', 'parser',
>> 'protocol', 'server'), AXIOM, STAX
>> package :jar
>> end
>>
>> desc 'Some features I guess'
>> define 'features' do
>> compile.with projects('dependencies:i18n', 'core', 'parser',
>> 'protocol', 'server', 'client'), AXIOM, STAX
>> package :jar
>> end
>> end
>>
>> desc 'Atom Specification Implementation Examples'
>> define 'examples' do
>> compile.with projects('dependencies:i18n', 'core', 'parser',
>> 'protocol', 'client', 'server', 'security'),
>> project('extensions').projects('main', 'serializer', 'gdata',
>> 'geo', 'sharing', 'features'), JETTY, AXIOM, JAXEN, SERVLET_API,
>> COMMONS.httpclient
>> package :jar
>> end
>> end
>>
>> include
>> 'client','core','dependencies','dependencies:i18n','examples','extensi
>> ons','extensions:features','extensions:gdata','extensions:geo','extens
>> ions:html','extensions:json','extensions:main','extensions:media','ext
>> ensions:oauth','extensions:opensearch','extensions:serializer','extens
>> ions:sharing','extensions:wsse','jcr','parser','protocol','security','
>> server','spring'
>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe from this list, please visit:
>>
>> http://xircles.codehaus.org/manage_email
>
> --
> Hans Dockter
> Gradle Project lead
> http://www.gradle.org
>
>
>
>
>
> ---------------------------------------------------------------------
> To unsubscribe from this list, please visit:
>
> http://xircles.codehaus.org/manage_email
>
>
>
>
--
View this message in context:
http://www.nabble.com/Performance-Improvements-tp18063743p18074168.html
Sent from the gradle-user mailing list archive at Nabble.com.
---------------------------------------------------------------------
To unsubscribe from this list, please visit:
http://xircles.codehaus.org/manage_email
