Re: svn commit: r1327411 - in /ofbiz/trunk: bin/ tools/functions.sh tools/git-rebase-runner.sh
On Apr 18, 2012, at 6:55 PM, Jacopo Cappellato wrote: On Apr 18, 2012, at 6:43 PM, Adam Heath wrote: It currently takes my desktop 1 hour to run a full test suite. clean, load-demo(30m), run-tests(30m). It used to take 12-15m. wow, it takes me 2-3 minutes in my laptop. Jacopo To be more precise: (less than) 3 minutes for load-demo 4 minutes and 3 seconds for run-tests Jacopo
Re: svn commit: r1327411 - in /ofbiz/trunk: bin/ tools/functions.sh tools/git-rebase-runner.sh
OK seems after all that I need a brand new CPU, hence a new box... I must say with the SSDs when running run-tests my CPU is always at 100% (I put the run-tests process priority level to idle to be able to continue other tasks) Do you have a SSD in your laptop Jacopo? Jacques From: Jacopo Cappellato jacopo.cappell...@hotwaxmedia.com On Apr 18, 2012, at 6:55 PM, Jacopo Cappellato wrote: On Apr 18, 2012, at 6:43 PM, Adam Heath wrote: It currently takes my desktop 1 hour to run a full test suite. clean, load-demo(30m), run-tests(30m). It used to take 12-15m. wow, it takes me 2-3 minutes in my laptop. Jacopo To be more precise: (less than) 3 minutes for load-demo 4 minutes and 3 seconds for run-tests Jacopo
Re: svn commit: r1327411 - in /ofbiz/trunk: bin/ tools/functions.sh tools/git-rebase-runner.sh
Jacques, here are some details: Hardware Overview: Model Name: MacBook Pro Model Identifier: MacBookPro8,2 Processor Name: Intel Core i7 Processor Speed: 2.2 GHz Number of Processors: 1 Total Number of Cores:4 L2 Cache (per Core): 256 KB L3 Cache: 6 MB Memory: 8 GB HD details are: Intel 6 Series Chipset: Vendor: Intel Product: 6 Series Chipset Link Speed: 6 Gigabit Negotiated Link Speed:3 Gigabit Description: AHCI Version 1.30 Supported Hitachi HTS725050A9A362: Capacity: 500.11 GB (500,107,862,016 bytes) Model:Hitachi HTS725050A9A362 Queue Depth: 32 Rotational Rate: 7200 Jacopo On Apr 19, 2012, at 9:58 AM, Jacques Le Roux wrote: OK seems after all that I need a brand new CPU, hence a new box... I must say with the SSDs when running run-tests my CPU is always at 100% (I put the run-tests process priority level to idle to be able to continue other tasks) Do you have a SSD in your laptop Jacopo? Jacques From: Jacopo Cappellato jacopo.cappell...@hotwaxmedia.com On Apr 18, 2012, at 6:55 PM, Jacopo Cappellato wrote: On Apr 18, 2012, at 6:43 PM, Adam Heath wrote: It currently takes my desktop 1 hour to run a full test suite. clean, load-demo(30m), run-tests(30m). It used to take 12-15m. wow, it takes me 2-3 minutes in my laptop. Jacopo To be more precise: (less than) 3 minutes for load-demo 4 minutes and 3 seconds for run-tests Jacopo
Re: svn commit: r1327411 - in /ofbiz/trunk: bin/ tools/functions.sh tools/git-rebase-runner.sh
On 04/18/2012 11:33 PM, Jacopo Cappellato wrote: On Apr 18, 2012, at 7:37 PM, Adam Heath wrote: On 04/18/2012 12:28 PM, Jacopo Cappellato wrote: On Apr 18, 2012, at 7:20 PM, Adam Heath wrote: ps: I am going to switch if in ant *back* to javascript, *away* from ant-contrib. The latter is significantly slower when using a system installed ant. No please, don't do that. I've pointed out a problem, and I have a solution. Don't just respond saying don't do that, without a reason. That isn't enough to stop me. I thought that all the recent threads about code refactoring, moving out custom code etc... were enough to motivate my response; but it seems that you ignored all of them and now pretend a response from me, for such a trivial and small topic. The reason is that we want to keep code as clean as possible, as slim as possible (ok, I know it looks silly if you look at the code now... but we are determined to work on this): using an external tool for this instead of custom code is a step in this direction. The stats below show that the problem is trivial: it is only relevant when you use your own ant that is not what we recommend; and it is something that should be fixed with the ant community: in fact we want to increase the cooperation with other communities (especially if they are in the ASF world) rather than blobbing everything into OFBiz. I didn't ignore them all, I've been away and haven't had time to keep uptodate. I haven't committed anything for this yet, once you said you had issues with the switch. I have 130 commits in my local ofbiz tree, keeping some out is *not* a problem (for instance, that salted-based password change I've kept around for about 2 years). This explanation is a *good* one. And yes, I do realize that the problem is with ant, even system level shouldn't be running as slow as it is.
Re: svn commit: r1327411 - in /ofbiz/trunk: bin/ tools/functions.sh tools/git-rebase-runner.sh
On 04/19/2012 03:15 AM, Jacopo Cappellato wrote: Jacques, here are some details: Hardware Overview: Model Name: MacBook Pro Model Identifier: MacBookPro8,2 Processor Name: Intel Core i7 Processor Speed:2.2 GHz Number of Processors: 1 Total Number of Cores: 4 So 4 threads. That doesn't help that much, unless the test suite is multithreaded now. L2 Cache (per Core):256 KB L3 Cache: 6 MB Memory: 8 GB HD details are: Intel 6 Series Chipset: Vendor: Intel Product:6 Series Chipset Link Speed: 6 Gigabit Negotiated Link Speed: 3 Gigabit Description:AHCI Version 1.30 Supported Hitachi HTS725050A9A362: Capacity: 500.11 GB (500,107,862,016 bytes) Model: Hitachi HTS725050A9A362 Queue Depth:32 Rotational Rate:7200 Jacopo On Apr 19, 2012, at 9:58 AM, Jacques Le Roux wrote: OK seems after all that I need a brand new CPU, hence a new box... I must say with the SSDs when running run-tests my CPU is always at 100% (I put the run-tests process priority level to idle to be able to continue other tasks) Do you have a SSD in your laptop Jacopo? Jacques From: Jacopo Cappellato jacopo.cappell...@hotwaxmedia.com On Apr 18, 2012, at 6:55 PM, Jacopo Cappellato wrote: On Apr 18, 2012, at 6:43 PM, Adam Heath wrote: It currently takes my desktop 1 hour to run a full test suite. clean, load-demo(30m), run-tests(30m). It used to take 12-15m. wow, it takes me 2-3 minutes in my laptop. Jacopo To be more precise: (less than) 3 minutes for load-demo 4 minutes and 3 seconds for run-tests Jacopo
Re: svn commit: r1327411 - in /ofbiz/trunk: bin/ tools/functions.sh tools/git-rebase-runner.sh
On 04/18/2012 02:57 AM, jaco...@apache.org wrote: Author: jacopoc Date: Wed Apr 18 07:57:46 2012 New Revision: 1327411 URL: http://svn.apache.org/viewvc?rev=1327411view=rev Log: Removed bin folder and moved its content to the tools folder: this was discussed extensively in the dev list recently. Added: ofbiz/trunk/tools/functions.sh - copied unchanged from r1327403, ofbiz/trunk/bin/functions.sh ofbiz/trunk/tools/git-rebase-runner.sh - copied unchanged from r1327403, ofbiz/trunk/bin/git-rebase-runner.sh Removed: ofbiz/trunk/bin/ Sorry, missed that. I've been away from ofbiz too long.
Re: svn commit: r1327411 - in /ofbiz/trunk: bin/ tools/functions.sh tools/git-rebase-runner.sh
Le 18/04/2012 17:21, Adam Heath a écrit : On 04/18/2012 02:57 AM, jaco...@apache.org wrote: Author: jacopoc Date: Wed Apr 18 07:57:46 2012 New Revision: 1327411 URL: http://svn.apache.org/viewvc?rev=1327411view=rev Log: Removed bin folder and moved its content to the tools folder: this was discussed extensively in the dev list recently. Added: ofbiz/trunk/tools/functions.sh - copied unchanged from r1327403, ofbiz/trunk/bin/functions.sh ofbiz/trunk/tools/git-rebase-runner.sh - copied unchanged from r1327403, ofbiz/trunk/bin/git-rebase-runner.sh Removed: ofbiz/trunk/bin/ Sorry, missed that. I've been away from ofbiz too long. So welcome back, then ! -- Erwan de FERRIERES www.nereide.biz
Re: svn commit: r1327411 - in /ofbiz/trunk: bin/ tools/functions.sh tools/git-rebase-runner.sh
On 04/18/2012 10:55 AM, Erwan de FERRIERES wrote: Le 18/04/2012 17:21, Adam Heath a écrit : On 04/18/2012 02:57 AM, jaco...@apache.org wrote: Author: jacopoc Date: Wed Apr 18 07:57:46 2012 New Revision: 1327411 URL: http://svn.apache.org/viewvc?rev=1327411view=rev Log: Removed bin folder and moved its content to the tools folder: this was discussed extensively in the dev list recently. Added: ofbiz/trunk/tools/functions.sh - copied unchanged from r1327403, ofbiz/trunk/bin/functions.sh ofbiz/trunk/tools/git-rebase-runner.sh - copied unchanged from r1327403, ofbiz/trunk/bin/git-rebase-runner.sh Removed: ofbiz/trunk/bin/ Sorry, missed that. I've been away from ofbiz too long. So welcome back, then ! The main reason is the ofbiz community is not paying attention to how slow the software has gotten. It currently takes my desktop 1 hour to run a full test suite. clean, load-demo(30m), run-tests(30m). It used to take 12-15m. This has caused me to not do much, as it takes to long to do anything. The issue that helps the most is getting a larger l2-cache on the cpu. The load-demo time is greatly affected by derby being *way* to large, code-size wise. I've started doing work now, by using a virtualized instance that has more l2-cache, but the problem still remains.
Re: svn commit: r1327411 - in /ofbiz/trunk: bin/ tools/functions.sh tools/git-rebase-runner.sh
On Apr 18, 2012, at 6:43 PM, Adam Heath wrote: It currently takes my desktop 1 hour to run a full test suite. clean, load-demo(30m), run-tests(30m). It used to take 12-15m. wow, it takes me 2-3 minutes in my laptop. Jacopo
Re: svn commit: r1327411 - in /ofbiz/trunk: bin/ tools/functions.sh tools/git-rebase-runner.sh
On 4/18/2012 5:43 PM, Adam Heath wrote: On 04/18/2012 10:55 AM, Erwan de FERRIERES wrote: Le 18/04/2012 17:21, Adam Heath a écrit : On 04/18/2012 02:57 AM, jaco...@apache.org wrote: Author: jacopoc Date: Wed Apr 18 07:57:46 2012 New Revision: 1327411 URL: http://svn.apache.org/viewvc?rev=1327411view=rev Log: Removed bin folder and moved its content to the tools folder: this was discussed extensively in the dev list recently. Added: ofbiz/trunk/tools/functions.sh - copied unchanged from r1327403, ofbiz/trunk/bin/functions.sh ofbiz/trunk/tools/git-rebase-runner.sh - copied unchanged from r1327403, ofbiz/trunk/bin/git-rebase-runner.sh Removed: ofbiz/trunk/bin/ Sorry, missed that. I've been away from ofbiz too long. So welcome back, then ! The main reason is the ofbiz community is not paying attention to how slow the software has gotten. It currently takes my desktop 1 hour to run a full test suite. clean, load-demo(30m), run-tests(30m). It used to take 12-15m. This has caused me to not do much, as it takes to long to do anything. Hard disk speed is what helps. With my high-speed SATA RAID array I can build a fresh checkout and load demo data in less than 5 minutes. Ant clean-data plus ant load-demo takes 4 minutes. The tests take 25 minutes - but that is mainly due to the many timing loops involved. A faster computer will not solve that. -Adrian
Re: svn commit: r1327411 - in /ofbiz/trunk: bin/ tools/functions.sh tools/git-rebase-runner.sh
Just buy a pair of SSDs, I can recommend Crucial M4 128Gb. My old box (Intel Core Duo E8300 2.9 Mz from 2008) with XP and 4Go RAM is still happy, and it seems for long now... :o) The disk subsystem has always been the bootleneck, not only on Windows... The complete test takes me 32 minutes. I think with a faster/newer CPU could be twice less. I don't care, changing the machines does not worth it (all that to move again, pfew I get too old, thanks SSDs...) Jacques From: Adrian Crum adrian.c...@sandglass-software.com On 4/18/2012 5:43 PM, Adam Heath wrote: On 04/18/2012 10:55 AM, Erwan de FERRIERES wrote: Le 18/04/2012 17:21, Adam Heath a écrit : On 04/18/2012 02:57 AM, jaco...@apache.org wrote: Author: jacopoc Date: Wed Apr 18 07:57:46 2012 New Revision: 1327411 URL: http://svn.apache.org/viewvc?rev=1327411view=rev Log: Removed bin folder and moved its content to the tools folder: this was discussed extensively in the dev list recently. Added: ofbiz/trunk/tools/functions.sh - copied unchanged from r1327403, ofbiz/trunk/bin/functions.sh ofbiz/trunk/tools/git-rebase-runner.sh - copied unchanged from r1327403, ofbiz/trunk/bin/git-rebase-runner.sh Removed: ofbiz/trunk/bin/ Sorry, missed that. I've been away from ofbiz too long. So welcome back, then ! The main reason is the ofbiz community is not paying attention to how slow the software has gotten. It currently takes my desktop 1 hour to run a full test suite. clean, load-demo(30m), run-tests(30m). It used to take 12-15m. This has caused me to not do much, as it takes to long to do anything. Hard disk speed is what helps. With my high-speed SATA RAID array I can build a fresh checkout and load demo data in less than 5 minutes. Ant clean-data plus ant load-demo takes 4 minutes. The tests take 25 minutes - but that is mainly due to the many timing loops involved. A faster computer will not solve that. -Adrian
Re: svn commit: r1327411 - in /ofbiz/trunk: bin/ tools/functions.sh tools/git-rebase-runner.sh
On 04/18/2012 11:56 AM, Adrian Crum wrote: On 4/18/2012 5:43 PM, Adam Heath wrote: On 04/18/2012 10:55 AM, Erwan de FERRIERES wrote: Le 18/04/2012 17:21, Adam Heath a écrit : On 04/18/2012 02:57 AM, jaco...@apache.org wrote: Author: jacopoc Date: Wed Apr 18 07:57:46 2012 New Revision: 1327411 URL: http://svn.apache.org/viewvc?rev=1327411view=rev Log: Removed bin folder and moved its content to the tools folder: this was discussed extensively in the dev list recently. Added: ofbiz/trunk/tools/functions.sh - copied unchanged from r1327403, ofbiz/trunk/bin/functions.sh ofbiz/trunk/tools/git-rebase-runner.sh - copied unchanged from r1327403, ofbiz/trunk/bin/git-rebase-runner.sh Removed: ofbiz/trunk/bin/ Sorry, missed that. I've been away from ofbiz too long. So welcome back, then ! The main reason is the ofbiz community is not paying attention to how slow the software has gotten. It currently takes my desktop 1 hour to run a full test suite. clean, load-demo(30m), run-tests(30m). It used to take 12-15m. This has caused me to not do much, as it takes to long to do anything. Hard disk speed is what helps. With my high-speed SATA RAID array I can build a fresh checkout and load demo data in less than 5 minutes. Ant clean-data plus ant load-demo takes 4 minutes. The tests take 25 minutes - but that is mainly due to the many timing loops involved. A faster computer will not solve that. Believe me, this is an *ofbiz* problem, not a hardware problem. It definately used to take 15m to do a full run(ant clean-all, ant install-demo, ant run-tests). 15m was sufficient for me to start a loop against a whole series of git commits, go away to lunch, or drive home, and log back in to see 10 test runs all happily finished. Things have changed in ofbiz, so that now it takes a *huge* amount of time. Some of that is related to a larger amount of code overflowing the cpu's L2 cache. Some of that is related to catalina startup, when it tries to seed the sessions randomness, and the local machine runs out of entropy. I suggest that every so often software gets run on older computers; this allows for poorly written software to be tested in a more real-world situation, and then moree problems can be detected and fixed. ps: I am going to switch if in ant *back* to javascript, *away* from ant-contrib. The latter is significantly slower when using a system installed ant.
Re: svn commit: r1327411 - in /ofbiz/trunk: bin/ tools/functions.sh tools/git-rebase-runner.sh
On 04/18/2012 12:20 PM, Adam Heath wrote: Things have changed in ofbiz, so that now it takes a *huge* amount of time. Some of that is related to a larger amount of code overflowing the cpu's L2 cache. Some of that is related to catalina startup, when it tries to seed the sessions randomness, and the local machine runs out of entropy. The method of placing every single theme as a *separate* catalina webapp context is also rather stupid. Each context drains more random entropy. It'd be better to have a single theme webapp, and then each theme is a sub-folder of that. Since themes are separate components, this is not immediately simple; perhaps a servlet filter that redirects paths to the appropriate theme folder.
Re: svn commit: r1327411 - in /ofbiz/trunk: bin/ tools/functions.sh tools/git-rebase-runner.sh
On Apr 18, 2012, at 7:20 PM, Adam Heath wrote: ps: I am going to switch if in ant *back* to javascript, *away* from ant-contrib. The latter is significantly slower when using a system installed ant. No please, don't do that. Jacopo
Re: svn commit: r1327411 - in /ofbiz/trunk: bin/ tools/functions.sh tools/git-rebase-runner.sh
From: Adam Heath doo...@brainfood.com On 04/18/2012 11:56 AM, Adrian Crum wrote: On 4/18/2012 5:43 PM, Adam Heath wrote: On 04/18/2012 10:55 AM, Erwan de FERRIERES wrote: Le 18/04/2012 17:21, Adam Heath a écrit : On 04/18/2012 02:57 AM, jaco...@apache.org wrote: Author: jacopoc Date: Wed Apr 18 07:57:46 2012 New Revision: 1327411 URL: http://svn.apache.org/viewvc?rev=1327411view=rev Log: Removed bin folder and moved its content to the tools folder: this was discussed extensively in the dev list recently. Added: ofbiz/trunk/tools/functions.sh - copied unchanged from r1327403, ofbiz/trunk/bin/functions.sh ofbiz/trunk/tools/git-rebase-runner.sh - copied unchanged from r1327403, ofbiz/trunk/bin/git-rebase-runner.sh Removed: ofbiz/trunk/bin/ Sorry, missed that. I've been away from ofbiz too long. So welcome back, then ! The main reason is the ofbiz community is not paying attention to how slow the software has gotten. It currently takes my desktop 1 hour to run a full test suite. clean, load-demo(30m), run-tests(30m). It used to take 12-15m. This has caused me to not do much, as it takes to long to do anything. Hard disk speed is what helps. With my high-speed SATA RAID array I can build a fresh checkout and load demo data in less than 5 minutes. Ant clean-data plus ant load-demo takes 4 minutes. The tests take 25 minutes - but that is mainly due to the many timing loops involved. A faster computer will not solve that. Believe me, this is an *ofbiz* problem, not a hardware problem. It definately used to take 15m to do a full run(ant clean-all, ant install-demo, ant run-tests). 15m was sufficient for me to start a loop against a whole series of git commits, go away to lunch, or drive home, and log back in to see 10 test runs all happily finished. Things have changed in ofbiz, so that now it takes a *huge* amount of time. Some of that is related to a larger amount of code overflowing the cpu's L2 cache. Some of that is related to catalina startup, when it tries to seed the sessions randomness, and the local machine runs out of entropy. I suggest that every so often software gets run on older computers; this allows for poorly written software to be tested in a more real-world situation, and then moree problems can be detected and fixed. ps: I am going to switch if in ant *back* to javascript, *away* from ant-contrib. The latter is significantly slower when using a system installed ant. Maybe simply because there are more things in OFBiz. Then the current slim-down iteration should help... Jacques
Re: svn commit: r1327411 - in /ofbiz/trunk: bin/ tools/functions.sh tools/git-rebase-runner.sh
On 04/18/2012 12:28 PM, Jacopo Cappellato wrote: On Apr 18, 2012, at 7:20 PM, Adam Heath wrote: ps: I am going to switch if in ant *back* to javascript, *away* from ant-contrib. The latter is significantly slower when using a system installed ant. No please, don't do that. I've pointed out a problem, and I have a solution. Don't just respond saying don't do that, without a reason. That isn't enough to stop me. ant-contrib If-ant.js (system-ant) Apache Ant version 1.8.0 compiled on March 11 2010 time ant clean 18.131s 8.291s time ant build 44.072s 34.749s time ant build 20.228s 10.559s time ant clean 18.222s 8.490s (local-ant) Apache Ant(TM) version 1.8.3 compiled on February 26 2012 time ./ant clean3.575s 3.218s time ./ant build28.985s 28.909s time ./ant build5.486s 5.232s time ./ant clean3.876s 3.378s The first ant clean is run when the system is already cleaned. The second ant build is also run when everything is already built. It's a way to test what happens when nothing is done.
Re: svn commit: r1327411 - in /ofbiz/trunk: bin/ tools/functions.sh tools/git-rebase-runner.sh
From: Adam Heath doo...@brainfood.com On 04/18/2012 12:28 PM, Jacopo Cappellato wrote: On Apr 18, 2012, at 7:20 PM, Adam Heath wrote: ps: I am going to switch if in ant *back* to javascript, *away* from ant-contrib. The latter is significantly slower when using a system installed ant. No please, don't do that. I've pointed out a problem, and I have a solution. Don't just respond saying don't do that, without a reason. That isn't enough to stop me. ant-contrib If-ant.js (system-ant) Apache Ant version 1.8.0 compiled on March 11 2010 time ant clean 18.131s 8.291s time ant build 44.072s 34.749s time ant build 20.228s 10.559s time ant clean 18.222s 8.490s (local-ant) Apache Ant(TM) version 1.8.3 compiled on February 26 2012 time ./ant clean3.575s 3.218s time ./ant build28.985s 28.909s time ./ant build5.486s 5.232s time ./ant clean3.876s 3.378s The first ant clean is run when the system is already cleaned. The second ant build is also run when everything is already built. It's a way to test what happens when nothing is done. first ant clean ie with (system-ant)? second ant build ie with (local-ant)? I don't see much diff with (local-ant). why not using the embedded ant? Jacques
Re: svn commit: r1327411 - in /ofbiz/trunk: bin/ tools/functions.sh tools/git-rebase-runner.sh
On 04/18/2012 12:46 PM, Jacques Le Roux wrote: From: Adam Heath doo...@brainfood.com On 04/18/2012 12:28 PM, Jacopo Cappellato wrote: On Apr 18, 2012, at 7:20 PM, Adam Heath wrote: ps: I am going to switch if in ant *back* to javascript, *away* from ant-contrib. The latter is significantly slower when using a system installed ant. No please, don't do that. You didn't give a reason as to why I shouldn't switch back. If it's because of things not working(at all), then list those problems. I've already discovered some cases of embedded jar paths in the build.xml files that are out-of-date(I can't do cd framework/base; ../../ant tests-cobertura, but using the system-ant does work(once I fix the broken paths). I've pointed out a problem, and I have a solution. Don't just respond saying don't do that, without a reason. That isn't enough to stop me. ant-contrib If-ant.js (system-ant) Apache Ant version 1.8.0 compiled on March 11 2010 time ant clean 18.131s 8.291s time ant build 44.072s 34.749s time ant build 20.228s 10.559s time ant clean 18.222s 8.490s (local-ant) Apache Ant(TM) version 1.8.3 compiled on February 26 2012 time ./ant clean3.575s 3.218s time ./ant build28.985s 28.909s time ./ant build5.486s 5.232s time ./ant clean3.876s 3.378s The first ant clean is run when the system is already cleaned. The second ant build is also run when everything is already built. It's a way to test what happens when nothing is done. first ant clean ie with (system-ant)? The first 4 runs are with the system ant. The second 4 are with the local ant. notice the './' on the second set. That's the clue. second ant build ie with (local-ant)? I don't see much diff with (local-ant). why not using the embedded ant? They are all slightly slower. And that's not noise, it was consistent in my runs.
Re: svn commit: r1327411 - in /ofbiz/trunk: bin/ tools/functions.sh tools/git-rebase-runner.sh
From: Adam Heath doo...@brainfood.com On 04/18/2012 12:46 PM, Jacques Le Roux wrote: From: Adam Heath doo...@brainfood.com On 04/18/2012 12:28 PM, Jacopo Cappellato wrote: On Apr 18, 2012, at 7:20 PM, Adam Heath wrote: ps: I am going to switch if in ant *back* to javascript, *away* from ant-contrib. The latter is significantly slower when using a system installed ant. No please, don't do that. You didn't give a reason as to why I shouldn't switch back. If it's because of things not working(at all), then list those problems. I've already discovered some cases of embedded jar paths in the build.xml files that are out-of-date(I can't do cd framework/base; ../../ant tests-cobertura, but using the system-ant does work(once I fix the broken paths). I've pointed out a problem, and I have a solution. Don't just respond saying don't do that, without a reason. That isn't enough to stop me. ant-contrib If-ant.js (system-ant) Apache Ant version 1.8.0 compiled on March 11 2010 time ant clean 18.131s 8.291s time ant build 44.072s 34.749s time ant build 20.228s 10.559s time ant clean 18.222s 8.490s (local-ant) Apache Ant(TM) version 1.8.3 compiled on February 26 2012 time ./ant clean3.575s 3.218s time ./ant build28.985s 28.909s time ./ant build5.486s 5.232s time ./ant clean3.876s 3.378s The first ant clean is run when the system is already cleaned. The second ant build is also run when everything is already built. It's a way to test what happens when nothing is done. first ant clean ie with (system-ant)? The first 4 runs are with the system ant. The second 4 are with the local ant. notice the './' on the second set. That's the clue. second ant build ie with (local-ant)? I don't see much diff with (local-ant). why not using the embedded ant? They are all slightly slower. And that's not noise, it was consistent in my runs. I'd not worry about so low diff. respectively: +11 % +0.2 % +4 % +14 % not significant enough to take action IMO (don't fix when it works) I don't remember why I prefered ant-contrib,but I can digg deeper... Jacques
Re: svn commit: r1327411 - in /ofbiz/trunk: bin/ tools/functions.sh tools/git-rebase-runner.sh
Le 18/04/2012 18:43, Adam Heath a écrit : On 04/18/2012 10:55 AM, Erwan de FERRIERES wrote: Le 18/04/2012 17:21, Adam Heath a écrit : On 04/18/2012 02:57 AM, jaco...@apache.org wrote: Author: jacopoc Date: Wed Apr 18 07:57:46 2012 New Revision: 1327411 URL: http://svn.apache.org/viewvc?rev=1327411view=rev Log: Removed bin folder and moved its content to the tools folder: this was discussed extensively in the dev list recently. Added: ofbiz/trunk/tools/functions.sh - copied unchanged from r1327403, ofbiz/trunk/bin/functions.sh ofbiz/trunk/tools/git-rebase-runner.sh - copied unchanged from r1327403, ofbiz/trunk/bin/git-rebase-runner.sh Removed: ofbiz/trunk/bin/ Sorry, missed that. I've been away from ofbiz too long. So welcome back, then ! The main reason is the ofbiz community is not paying attention to how slow the software has gotten. It currently takes my desktop 1 hour to run a full test suite. clean, load-demo(30m), run-tests(30m). It used to take 12-15m. This has caused me to not do much, as it takes to long to do anything. The issue that helps the most is getting a larger l2-cache on the cpu. The load-demo time is greatly affected by derby being *way* to large, code-size wise. I've started doing work now, by using a virtualized instance that has more l2-cache, but the problem still remains. Hi, I had better stats with the new ant 1.8.3. But the times you have are really impressive ! can't explain why... -- Erwan de FERRIERES www.nereide.biz
Re: svn commit: r1327411 - in /ofbiz/trunk: bin/ tools/functions.sh tools/git-rebase-runner.sh
On 19/04/2012, at 5:54 AM, Adam Heath wrote: On 04/18/2012 12:46 PM, Jacques Le Roux wrote: From: Adam Heath doo...@brainfood.com On 04/18/2012 12:28 PM, Jacopo Cappellato wrote: On Apr 18, 2012, at 7:20 PM, Adam Heath wrote: ps: I am going to switch if in ant *back* to javascript, *away* from ant-contrib. The latter is significantly slower when using a system installed ant. No please, don't do that. You didn't give a reason as to why I shouldn't switch back. If it's because of things not working(at all), then list those problems. I've already discovered some cases of embedded jar paths in the build.xml files that are out-of-date(I can't do cd framework/base; ../../ant tests-cobertura, but using the system-ant does work(once I fix the broken paths). Have you considered working with the ant project at all to fix the issue? The problem we have and I think we want to address is simply a case of way too much code in this project trying to do way too many things and being maintained by nowhere near enough people with the expertise to do so. We're attempting to take a good hard look at where we can pass the responsibility for maintaining that functionality off to more appropriate parties. Unless there is a very good reason for not contributing improvements back to ant then I can't see why it wouldn't more sense to do so. I've pointed out a problem, and I have a solution. Don't just respond saying don't do that, without a reason. That isn't enough to stop me. ant-contrib If-ant.js (system-ant) Apache Ant version 1.8.0 compiled on March 11 2010 time ant clean 18.131s 8.291s time ant build 44.072s 34.749s time ant build 20.228s 10.559s time ant clean 18.222s 8.490s (local-ant) Apache Ant(TM) version 1.8.3 compiled on February 26 2012 time ./ant clean3.575s 3.218s time ./ant build28.985s 28.909s time ./ant build5.486s 5.232s time ./ant clean3.876s 3.378s The first ant clean is run when the system is already cleaned. The second ant build is also run when everything is already built. It's a way to test what happens when nothing is done. first ant clean ie with (system-ant)? The first 4 runs are with the system ant. The second 4 are with the local ant. notice the './' on the second set. That's the clue. second ant build ie with (local-ant)? I don't see much diff with (local-ant). why not using the embedded ant? They are all slightly slower. And that's not noise, it was consistent in my runs.
Re: svn commit: r1327411 - in /ofbiz/trunk: bin/ tools/functions.sh tools/git-rebase-runner.sh
We resolved per theme web app problem by moving the them header/footer to CMS. In multi-tenant e-commerce applications it is possible to have 100's of themes and it is not possible to create web application for each theme. What is needed is create different header/footer content for each theme and create the theme data accordingly. Raj On Wednesday 18 April 2012 10:54 PM, Adam Heath wrote: On 04/18/2012 12:20 PM, Adam Heath wrote: Things have changed in ofbiz, so that now it takes a *huge* amount of time. Some of that is related to a larger amount of code overflowing the cpu's L2 cache. Some of that is related to catalina startup, when it tries to seed the sessions randomness, and the local machine runs out of entropy. The method of placing every single theme as a *separate* catalina webapp context is also rather stupid. Each context drains more random entropy. It'd be better to have a single theme webapp, and then each theme is a sub-folder of that. Since themes are separate components, this is not immediately simple; perhaps a servlet filter that redirects paths to the appropriate theme folder.
Re: svn commit: r1327411 - in /ofbiz/trunk: bin/ tools/functions.sh tools/git-rebase-runner.sh
On Apr 18, 2012, at 7:37 PM, Adam Heath wrote: On 04/18/2012 12:28 PM, Jacopo Cappellato wrote: On Apr 18, 2012, at 7:20 PM, Adam Heath wrote: ps: I am going to switch if in ant *back* to javascript, *away* from ant-contrib. The latter is significantly slower when using a system installed ant. No please, don't do that. I've pointed out a problem, and I have a solution. Don't just respond saying don't do that, without a reason. That isn't enough to stop me. I thought that all the recent threads about code refactoring, moving out custom code etc... were enough to motivate my response; but it seems that you ignored all of them and now pretend a response from me, for such a trivial and small topic. The reason is that we want to keep code as clean as possible, as slim as possible (ok, I know it looks silly if you look at the code now... but we are determined to work on this): using an external tool for this instead of custom code is a step in this direction. The stats below show that the problem is trivial: it is only relevant when you use your own ant that is not what we recommend; and it is something that should be fixed with the ant community: in fact we want to increase the cooperation with other communities (especially if they are in the ASF world) rather than blobbing everything into OFBiz. Jacopo ant-contrib If-ant.js (system-ant) Apache Ant version 1.8.0 compiled on March 11 2010 time ant clean 18.131s 8.291s time ant build 44.072s 34.749s time ant build 20.228s 10.559s time ant clean 18.222s 8.490s (local-ant) Apache Ant(TM) version 1.8.3 compiled on February 26 2012 time ./ant clean3.575s 3.218s time ./ant build28.985s 28.909s time ./ant build5.486s 5.232s time ./ant clean3.876s 3.378s The first ant clean is run when the system is already cleaned. The second ant build is also run when everything is already built. It's a way to test what happens when nothing is done.
