Re: Massive increase in compile time with GWT 2.3
Hello, In fact I have exactly the same problem than Dennis. We migrate our application from GWT 2.0 to 2.3 and th ecompilation time has increased about 40%. My problem is not how to optimize the compilation time but why this huge difference between GWT versions. Thank's On 19 mai, 18:30, Hilco Wijbenga hilco.wijbe...@gmail.com wrote: On 19 May 2011 00:37, googelybear googelyb...@gmail.com wrote: This build I am trying to optimize is compiled on our build server by the continuous integration tool (hudson in our case triggered after every commit). It is mainly used to run unit tests and for general testing by the developers to get instant feedback (well, it used to do that when we started). It is not a production build. But I don't like to take too many things out, e.g. take out browsers then you can no longer test it on different browsers and your feedback cycle - the timeuntil you notice something doesn't work after you implemented it - gets longer). For the production build then it is absolutely OK to take longer. In general, I don't think it is a good idea to have one build for (many) different purposes. For unit tests you don't need all browsers so pick one and stick with it. In fact, for unit tests you don't need any browser. :-) Your unit test build can and should be very fast. This should be the most stripped down version you can think of. Mind you, it would be even better if you broke up your app into separate modules so that all the unit testing is done in the small, fast module builds. The second build would be for integration testing. For your automated integration testing you don't need more than one browser either. (Unless, of course, you have a very advanced setup testing multiple browsers.) Run this build once or twice a day at a specifictime(say lunchtimeand dinnertime). (The specifictimeis so that people know about it and can try to make sure their change is (or is not) included.) If the automated integration test build is successful then kick off the full build for all browsers. This need only happen once a day or even once a week. This build is then used for manual testing. It should be auto deployed to some QA/test environment. Most (test/QA) people don't like working with a moving target (for obvious reasons), hence the build once a week suggestion. Then, if QA says this build is good, promote it to production; no need for another build. I.e. assuming you follow the best practice of not including your environment configuration in the WAR. -- You received this message because you are subscribed to the Google Groups Google Web Toolkit group. To post to this group, send email to google-web-toolkit@googlegroups.com. To unsubscribe from this group, send email to google-web-toolkit+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-web-toolkit?hl=en.
Re: Massive increase in compile time with GWT 2.3
Because GWT 2.3.0 include support to IE9, so you have another permutation. Juan 2011/6/2 Twentyseven ebarthel...@gmail.com Hello, In fact I have exactly the same problem than Dennis. We migrate our application from GWT 2.0 to 2.3 and th ecompilation time has increased about 40%. My problem is not how to optimize the compilation time but why this huge difference between GWT versions. Thank's On 19 mai, 18:30, Hilco Wijbenga hilco.wijbe...@gmail.com wrote: On 19 May 2011 00:37, googelybear googelyb...@gmail.com wrote: This build I am trying to optimize is compiled on our build server by the continuous integration tool (hudson in our case triggered after every commit). It is mainly used to run unit tests and for general testing by the developers to get instant feedback (well, it used to do that when we started). It is not a production build. But I don't like to take too many things out, e.g. take out browsers then you can no longer test it on different browsers and your feedback cycle - the timeuntil you notice something doesn't work after you implemented it - gets longer). For the production build then it is absolutely OK to take longer. In general, I don't think it is a good idea to have one build for (many) different purposes. For unit tests you don't need all browsers so pick one and stick with it. In fact, for unit tests you don't need any browser. :-) Your unit test build can and should be very fast. This should be the most stripped down version you can think of. Mind you, it would be even better if you broke up your app into separate modules so that all the unit testing is done in the small, fast module builds. The second build would be for integration testing. For your automated integration testing you don't need more than one browser either. (Unless, of course, you have a very advanced setup testing multiple browsers.) Run this build once or twice a day at a specifictime(say lunchtimeand dinnertime). (The specifictimeis so that people know about it and can try to make sure their change is (or is not) included.) If the automated integration test build is successful then kick off the full build for all browsers. This need only happen once a day or even once a week. This build is then used for manual testing. It should be auto deployed to some QA/test environment. Most (test/QA) people don't like working with a moving target (for obvious reasons), hence the build once a week suggestion. Then, if QA says this build is good, promote it to production; no need for another build. I.e. assuming you follow the best practice of not including your environment configuration in the WAR. -- You received this message because you are subscribed to the Google Groups Google Web Toolkit group. To post to this group, send email to google-web-toolkit@googlegroups.com. To unsubscribe from this group, send email to google-web-toolkit+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-web-toolkit?hl=en. -- You received this message because you are subscribed to the Google Groups Google Web Toolkit group. To post to this group, send email to google-web-toolkit@googlegroups.com. To unsubscribe from this group, send email to google-web-toolkit+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-web-toolkit?hl=en.
Re: Massive increase in compile time with GWT 2.3
First of all: Thank you very much for all the feedback! This build I am trying to optimize is compiled on our build server by the continuous integration tool (hudson in our case triggered after every commit). It is mainly used to run unit tests and for general testing by the developers to get instant feedback (well, it used to do that when we started). It is not a production build. But I don't like to take too many things out, e.g. take out browsers then you can no longer test it on different browsers and your feedback cycle - the time until you notice something doesn't work after you implemented it - gets longer). For the production build then it is absolutely OK to take longer. I tried using the draftCompile switch and it already reduces the build time to 53minutes - that's a huge 25% decrease compared to the previous 1h12min. I will try to play with the optimize level now. Regarding the SSD: This is not really an option for me as the build is running on a server used by other projects/users so that would be a major operation. Please let me know if you have any other suggestions. Dennis On May 18, 8:47 pm, Sanjiv Jivan sanjiv.ji...@gmail.com wrote: Have you tried compilation using SSD? I'm my experience from last year, SSD's were great for reads but terrible for writes and compilation of medium to large projects actually took a fair bit longer on SSD's. It's possible the newer SSD's have gotten better but I would recommend doing some more research before making this expensive purchase with expectations of significant compile time speed improvements. On Wed, May 18, 2011 at 2:22 PM, Jeff Larsen larse...@gmail.com wrote: What is the purpose of the build? Is it to deploy the actual code to a production/test server or is it to enable some sort of selenium/webdriver test framework. If it is the latter, you could add -draftCompile which will not be highly obfuscated code, but it should be a quicker compile, especially with 48 perms. You could also link up multiple computers to do distrubted permutation computations. Ray Cromwell had a talk about this at last year's IO. Depending on your budget, a SSD drive could potentially help you out a lot here too. -- You received this message because you are subscribed to the Google Groups Google Web Toolkit group. To post to this group, send email to google-web-toolkit@googlegroups.com. To unsubscribe from this group, send email to google-web-toolkit+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-web-toolkit?hl=en. -- You received this message because you are subscribed to the Google Groups Google Web Toolkit group. To post to this group, send email to google-web-toolkit@googlegroups.com. To unsubscribe from this group, send email to google-web-toolkit+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-web-toolkit?hl=en.
Re: Massive increase in compile time with GWT 2.3
You could try the distributed build. Basically, you will have N machines and each will only compile a set of permutations, and after all you link everything together and you have your compiled app ready for use http://code.google.com/p/google-web-toolkit/wiki/DistributedBuilds On Thu, May 19, 2011 at 4:37 AM, googelybear googelyb...@gmail.com wrote: First of all: Thank you very much for all the feedback! This build I am trying to optimize is compiled on our build server by the continuous integration tool (hudson in our case triggered after every commit). It is mainly used to run unit tests and for general testing by the developers to get instant feedback (well, it used to do that when we started). It is not a production build. But I don't like to take too many things out, e.g. take out browsers then you can no longer test it on different browsers and your feedback cycle - the time until you notice something doesn't work after you implemented it - gets longer). For the production build then it is absolutely OK to take longer. I tried using the draftCompile switch and it already reduces the build time to 53minutes - that's a huge 25% decrease compared to the previous 1h12min. I will try to play with the optimize level now. Regarding the SSD: This is not really an option for me as the build is running on a server used by other projects/users so that would be a major operation. Please let me know if you have any other suggestions. Dennis On May 18, 8:47 pm, Sanjiv Jivan sanjiv.ji...@gmail.com wrote: Have you tried compilation using SSD? I'm my experience from last year, SSD's were great for reads but terrible for writes and compilation of medium to large projects actually took a fair bit longer on SSD's. It's possible the newer SSD's have gotten better but I would recommend doing some more research before making this expensive purchase with expectations of significant compile time speed improvements. On Wed, May 18, 2011 at 2:22 PM, Jeff Larsen larse...@gmail.com wrote: What is the purpose of the build? Is it to deploy the actual code to a production/test server or is it to enable some sort of selenium/webdriver test framework. If it is the latter, you could add -draftCompile which will not be highly obfuscated code, but it should be a quicker compile, especially with 48 perms. You could also link up multiple computers to do distrubted permutation computations. Ray Cromwell had a talk about this at last year's IO. Depending on your budget, a SSD drive could potentially help you out a lot here too. -- You received this message because you are subscribed to the Google Groups Google Web Toolkit group. To post to this group, send email to google-web-toolkit@googlegroups.com. To unsubscribe from this group, send email to google-web-toolkit+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-web-toolkit?hl=en. -- You received this message because you are subscribed to the Google Groups Google Web Toolkit group. To post to this group, send email to google-web-toolkit@googlegroups.com. To unsubscribe from this group, send email to google-web-toolkit+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-web-toolkit?hl=en. -- Magno Machado Paulo http://blog.magnomachado.com.br http://code.google.com/p/emballo/ -- You received this message because you are subscribed to the Google Groups Google Web Toolkit group. To post to this group, send email to google-web-toolkit@googlegroups.com. To unsubscribe from this group, send email to google-web-toolkit+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-web-toolkit?hl=en.
Re: Massive increase in compile time with GWT 2.3
On 19 May 2011 00:37, googelybear googelyb...@gmail.com wrote: This build I am trying to optimize is compiled on our build server by the continuous integration tool (hudson in our case triggered after every commit). It is mainly used to run unit tests and for general testing by the developers to get instant feedback (well, it used to do that when we started). It is not a production build. But I don't like to take too many things out, e.g. take out browsers then you can no longer test it on different browsers and your feedback cycle - the time until you notice something doesn't work after you implemented it - gets longer). For the production build then it is absolutely OK to take longer. In general, I don't think it is a good idea to have one build for (many) different purposes. For unit tests you don't need all browsers so pick one and stick with it. In fact, for unit tests you don't need any browser. :-) Your unit test build can and should be very fast. This should be the most stripped down version you can think of. Mind you, it would be even better if you broke up your app into separate modules so that all the unit testing is done in the small, fast module builds. The second build would be for integration testing. For your automated integration testing you don't need more than one browser either. (Unless, of course, you have a very advanced setup testing multiple browsers.) Run this build once or twice a day at a specific time (say lunch time and dinner time). (The specific time is so that people know about it and can try to make sure their change is (or is not) included.) If the automated integration test build is successful then kick off the full build for all browsers. This need only happen once a day or even once a week. This build is then used for manual testing. It should be auto deployed to some QA/test environment. Most (test/QA) people don't like working with a moving target (for obvious reasons), hence the build once a week suggestion. Then, if QA says this build is good, promote it to production; no need for another build. I.e. assuming you follow the best practice of not including your environment configuration in the WAR. -- You received this message because you are subscribed to the Google Groups Google Web Toolkit group. To post to this group, send email to google-web-toolkit@googlegroups.com. To unsubscribe from this group, send email to google-web-toolkit+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-web-toolkit?hl=en.
Απ: Massive increase in compile time with GWT 2.3
45 minutes ?!?! what kind of app is that? -g. -- You received this message because you are subscribed to the Google Groups Google Web Toolkit group. To post to this group, send email to google-web-toolkit@googlegroups.com. To unsubscribe from this group, send email to google-web-toolkit+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-web-toolkit?hl=en.
Re: Απ: Massive increase in compile time with GWT 2.3
technically the project consists of 4 separate apps, so 4 modules that are compiled individually and it supports 4 locales. Is it unusual to have such a large compile time? On May 18, 1:14 pm, George Moschovitis george.moschovi...@gmail.com wrote: 45 minutes ?!?! what kind of app is that? -g. -- You received this message because you are subscribed to the Google Groups Google Web Toolkit group. To post to this group, send email to google-web-toolkit@googlegroups.com. To unsubscribe from this group, send email to google-web-toolkit+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-web-toolkit?hl=en.
Re: Απ: Massive increase in compile time with GWT 2.3
I've noticed a pretty significant jump as well. My app used to be around 65 seconds, and it's up to around 135 seconds now, sometimes spiking up to 3-4 minutes. I had just chalked it up to installing the full WindowBuilder and GAE plugins that I had skipped in the past, but maybe it is the SDK. One thing I noticed was the upgrade clobbered my -draftCompile and -localWorkers settings in Eclipse, you might want to check those on your compile settings if you're on a multi-core processor. -- You received this message because you are subscribed to the Google Groups Google Web Toolkit group. To post to this group, send email to google-web-toolkit@googlegroups.com. To unsubscribe from this group, send email to google-web-toolkit+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-web-toolkit?hl=en.
Re: Απ: Massive increase in compile time with GWT 2.3
On Wednesday, May 18, 2011 1:39:00 PM UTC+2, googelybear wrote: technically the project consists of 4 separate apps, so 4 modules that are compiled individually and it supports 4 locales. Is it unusual to have such a large compile time? We have something like 65000 LOCs (ncloc metric from Sonar) spread across 2000 classes, with many generators involved (UiBinder, RequestFactory, Editor framework), and it compiles in 212,486s (output from the compiler; that's approx. 3½ minutes). FYI, a compileReport from a month and a half back (before I added the Apache Wave editor to our app, which brings a whole lot of new files) says: Source files: 3122 Initial Type Oracle Types: 4409 Final Type Oracle Types: 6187 GeneratedTypes: 1778 AST Referenced Types: 6187 Unreferenced Types: 3409 So yes, I'd say 45 minutes is unusual. You might want to tweak the number of workers, or the memory you allocate the java process (I'm using Maven, so it automatically uses as many workers as there are cores/processors on the machine –but it's a VM, so I don't even know how many of them we have/it thinks we have–, and I otherwise simply bumped the Xmx to 512m –yes, only 512m–). Also, try using the server JVM (when using an Oracle/Sun JDK, pass the -server argument). I didn't made any measure, but Ray Cromwell found it made a difference (a few years back). FYI, I'm using a custom GWT build out of trunk @ r9848 (w/ half a dozen patches applied), which is long after 2.2, but before 2.3 (and before the com.google.web.bindery fwiw), so I can't tell if I already made the jump in compile time or not (I don't remember noticing any jump in our metrics when I upgraded from our custom-built 2.2) -- You received this message because you are subscribed to the Google Groups Google Web Toolkit group. To post to this group, send email to google-web-toolkit@googlegroups.com. To unsubscribe from this group, send email to google-web-toolkit+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-web-toolkit?hl=en.
Re: Massive increase in compile time with GWT 2.3
On Tue, May 17, 2011 at 11:55 PM, googelybear googelyb...@gmail.com wrote: Hi, I have recently updated from GWT 2.2.0 to 2.3 and noticed a significant increas in compile time: Compile time with 2.2.0 was 45minutes and now with 2.3 it increased to 1h 12m - that's almost a 40% increase. I didn't change any settings at all. Are others experiencing this as well? Do you have any suggestions how to get the compile time down again? GWT 2.3 introduces a new permutation for IE9. This can explain your compile time. If you want to support IE9 natively you have to accept that I guess... Best, Raphael -- You received this message because you are subscribed to the Google Groups Google Web Toolkit group. To post to this group, send email to google-web-toolkit@googlegroups.com. To unsubscribe from this group, send email to google-web-toolkit+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-web-toolkit?hl=en.
Re: Massive increase in compile time with GWT 2.3
In the project I'm working on we have about 300k LOC and ~7000 classes. The GWT Compiler is invoked with the following arguments on the build machine (this machine has 8 cores and 64gb of ram): - Dgwt.localWorkers=8 -Xmx2048M -Xss1024M and there are 48 permutations to compute. Do you guys have any idea what else we could tweak here? On May 18, 4:02 pm, Raphael André Bauer raphael.andre.ba...@gmail.com wrote: On Tue, May 17, 2011 at 11:55 PM, googelybear googelyb...@gmail.com wrote: Hi, I have recently updated from GWT 2.2.0 to 2.3 and noticed a significant increas in compile time: Compile time with 2.2.0 was 45minutes and now with 2.3 it increased to 1h 12m - that's almost a 40% increase. I didn't change any settings at all. Are others experiencing this as well? Do you have any suggestions how to get the compile time down again? GWT 2.3 introduces a new permutation for IE9. This can explain your compile time. If you want to support IE9 natively you have to accept that I guess... Best, Raphael -- You received this message because you are subscribed to the Google Groups Google Web Toolkit group. To post to this group, send email to google-web-toolkit@googlegroups.com. To unsubscribe from this group, send email to google-web-toolkit+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-web-toolkit?hl=en.
Re: Massive increase in compile time with GWT 2.3
On Wednesday, May 18, 2011 5:37:36 PM UTC+2, googelybear wrote: In the project I'm working on we have about 300k LOC and ~7000 classes. The GWT Compiler is invoked with the following arguments on the build machine (this machine has 8 cores and 64gb of ram): - Dgwt.localWorkers=8 -Xmx2048M -Xss1024M and there are 48 permutations to compute. Do you guys have any idea what else we could tweak here? Huh, OK, with such metrics, maybe 45 minutes is normal then... (and I forgot that we actually only compile 2 permutations!) Maybe try tweaking the -optimize level. I've read when it was being added that -optimize 9 (the default) is not always worth it, and something like -optimize 5 could give similar results (output size) for a, obviously, shorter compile time. -- You received this message because you are subscribed to the Google Groups Google Web Toolkit group. To post to this group, send email to google-web-toolkit@googlegroups.com. To unsubscribe from this group, send email to google-web-toolkit+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-web-toolkit?hl=en.
Re: Massive increase in compile time with GWT 2.3
What is the purpose of the build? Is it to deploy the actual code to a production/test server or is it to enable some sort of selenium/webdriver test framework. If it is the latter, you could add -draftCompile which will not be highly obfuscated code, but it should be a quicker compile, especially with 48 perms. You could also link up multiple computers to do distrubted permutation computations. Ray Cromwell had a talk about this at last year's IO. Depending on your budget, a SSD drive could potentially help you out a lot here too. -- You received this message because you are subscribed to the Google Groups Google Web Toolkit group. To post to this group, send email to google-web-toolkit@googlegroups.com. To unsubscribe from this group, send email to google-web-toolkit+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-web-toolkit?hl=en.
Re: Massive increase in compile time with GWT 2.3
Have you tried compilation using SSD? I'm my experience from last year, SSD's were great for reads but terrible for writes and compilation of medium to large projects actually took a fair bit longer on SSD's. It's possible the newer SSD's have gotten better but I would recommend doing some more research before making this expensive purchase with expectations of significant compile time speed improvements. On Wed, May 18, 2011 at 2:22 PM, Jeff Larsen larse...@gmail.com wrote: What is the purpose of the build? Is it to deploy the actual code to a production/test server or is it to enable some sort of selenium/webdriver test framework. If it is the latter, you could add -draftCompile which will not be highly obfuscated code, but it should be a quicker compile, especially with 48 perms. You could also link up multiple computers to do distrubted permutation computations. Ray Cromwell had a talk about this at last year's IO. Depending on your budget, a SSD drive could potentially help you out a lot here too. -- You received this message because you are subscribed to the Google Groups Google Web Toolkit group. To post to this group, send email to google-web-toolkit@googlegroups.com. To unsubscribe from this group, send email to google-web-toolkit+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-web-toolkit?hl=en. -- You received this message because you are subscribed to the Google Groups Google Web Toolkit group. To post to this group, send email to google-web-toolkit@googlegroups.com. To unsubscribe from this group, send email to google-web-toolkit+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-web-toolkit?hl=en.
Massive increase in compile time with GWT 2.3
Hi, I have recently updated from GWT 2.2.0 to 2.3 and noticed a significant increas in compile time: Compile time with 2.2.0 was 45minutes and now with 2.3 it increased to 1h 12m - that's almost a 40% increase. I didn't change any settings at all. Are others experiencing this as well? Do you have any suggestions how to get the compile time down again? thanks, Dennis -- You received this message because you are subscribed to the Google Groups Google Web Toolkit group. To post to this group, send email to google-web-toolkit@googlegroups.com. To unsubscribe from this group, send email to google-web-toolkit+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-web-toolkit?hl=en.
