[chromium-dev] Re: A suggestion to Drastically improve build times,
On Wed, Jul 8, 2009 at 10:50 AM, Evan Martine...@chromium.org wrote: On Wed, Jul 8, 2009 at 10:46 AM, nakroyoav.zilberb...@gmail.com wrote: the reason i think one from chrome is the best is because these changes could affect MAC and UNIX as well Regarding incremental linking: I don't believe it exists on Linux, but gold is quite fast. My links take around ~5s, though I have a very fast machine. M-A asked about these numbers and revealed I was lying. Debug builds take 8 seconds to link, while release is between 1 and 2 seconds. --~--~-~--~~~---~--~~ Chromium Developers mailing list: chromium-dev@googlegroups.com View archives, change email options, or unsubscribe: http://groups.google.com/group/chromium-dev -~--~~~~--~~--~--~---
[chromium-dev] Re: A suggestion to Drastically improve build times,
I wonder if it would be possible to have a conditional flag, maybe in common.gypi, that you would flip to turn on incremental linking, with still non-incremental linking being the default. On Wed, Jul 8, 2009 at 05:00, nakroyoav.zilberb...@gmail.com wrote: First off, i am not complaining, just suggesting as the build (especially link gets slower and slower) I contribed code before to chrome, and plan to do so in the future, however even changing one line of Code in say browser.vcproj leads to 10+ mins of linking (yes TEN) what i did was wrote scripts which allow me to re-enable incremental linking and i was happy with it since you lately moved to libs for chrome_dll and the like, the process became harder but i still manage to enable incremental linking (there is an option in VS2008 to use objs instead of static libraries which re-enables incremental linking) just to give you a taste of why i say this without incremental linking i get 10+ mins of build time for changing just one line of code with inc linking i am down to ~15 seconds for a simple change,, which is SUCH a great improvement i recall you removed inc linking since some projects (i know of ui_tests) needs more than 2GB for linking but you could just disable it for this one project i hope you will re-consider introducing inc linking on VS thanx (btw, there are issues with re-introducing inc linking as lib and obj rules are different, but they are kinda small) --~--~-~--~~~---~--~~ Chromium Developers mailing list: chromium-dev@googlegroups.com View archives, change email options, or unsubscribe: http://groups.google.com/group/chromium-dev -~--~~~~--~~--~--~---
[chromium-dev] Re: A suggestion to Drastically improve build times,
Pawel, since i have it working on my machine (or i really could not code with chrome) i can tell you it is much more than a flag just as an example the name of chrome_dll.vcproj produces chrome.dll, but the incremental files would clash with chrome.vcproj who does the exe another example : you seem to have several 'TCMalloc_SystemAlloc' functions, which the linker can resolve if they are in a lib but would clash if linked as objects but none of this is too hard to solve, and i think is worth it i have a T5200 cpu (translated slow) and a very slow hard disk, but unless all of you have 8 cores and SSD i think we would all benefit from it especially that lately it seems more non google employees are trying to help with chrome --~--~-~--~~~---~--~~ Chromium Developers mailing list: chromium-dev@googlegroups.com View archives, change email options, or unsubscribe: http://groups.google.com/group/chromium-dev -~--~~~~--~~--~--~---
[chromium-dev] Re: A suggestion to Drastically improve build times,
As a general observation on large development teams, infrastructure can get less love, because everyone comes to the team to work on the exciting main product, not the more mundane build infrastructure. If you have ideas on how to fix this stuff, I think it would be perfectly reasonable to put up patches that will improve things. Not as exciting as changing something that millions of people will see, perhaps, but if your request is ever to happen, SOMEONE has to do the gruntwork to make it happen... -scott On Wed, Jul 8, 2009 at 8:14 AM, nakroyoav.zilberb...@gmail.com wrote: Pawel, since i have it working on my machine (or i really could not code with chrome) i can tell you it is much more than a flag just as an example the name of chrome_dll.vcproj produces chrome.dll, but the incremental files would clash with chrome.vcproj who does the exe another example : you seem to have several 'TCMalloc_SystemAlloc' functions, which the linker can resolve if they are in a lib but would clash if linked as objects but none of this is too hard to solve, and i think is worth it i have a T5200 cpu (translated slow) and a very slow hard disk, but unless all of you have 8 cores and SSD i think we would all benefit from it especially that lately it seems more non google employees are trying to help with chrome --~--~-~--~~~---~--~~ Chromium Developers mailing list: chromium-dev@googlegroups.com View archives, change email options, or unsubscribe: http://groups.google.com/group/chromium-dev -~--~~~~--~~--~--~---
[chromium-dev] Re: A suggestion to Drastically improve build times,
On Wed, Jul 8, 2009 at 11:14 AM, nakro yoav.zilberb...@gmail.com wrote: Pawel, since i have it working on my machine (or i really could not code with chrome) i can tell you it is much more than a flag just as an example the name of chrome_dll.vcproj produces chrome.dll, but the incremental files would clash with chrome.vcproj who does the exe This was fixed in the past by putting chrome.dll in the intermediary directory and hardlinking it afterward in the ouput directory. I don't know what happens with the GYP conversion. Before the conversion, only the single projects that failed to link on 32 bit OS had incremental linking disabled. I don't know why it was disabled for every projects afterward. Maybe Steven can shed some light on that. another example : you seem to have several 'TCMalloc_SystemAlloc' functions, which the linker can resolve if they are in a lib but would clash if linked as objects but none of this is too hard to solve, and i think is worth it About directly linking object files, it's just nobody taking the time to fix the remaining issues and enabling the flag. Patches are welcome. M-A --~--~-~--~~~---~--~~ Chromium Developers mailing list: chromium-dev@googlegroups.com View archives, change email options, or unsubscribe: http://groups.google.com/group/chromium-dev -~--~~~~--~~--~--~---
[chromium-dev] Re: A suggestion to Drastically improve build times,
On Wed, Jul 8, 2009 at 11:03 AM, Paweł Hajdan Jr. phajdan...@chromium.orgwrote: I wonder if it would be possible to have a conditional flag, maybe in common.gypi, that you would flip to turn on incremental linking, with still non-incremental linking being the default. Something like this probably makes sense to be a generator flag. I just beefed up the support for these in GYP. A cl for that should be pretty simple. The general build layout changes would be within the GYP files and probably want a flag to enable. TVL On Wed, Jul 8, 2009 at 05:00, nakroyoav.zilberb...@gmail.com wrote: First off, i am not complaining, just suggesting as the build (especially link gets slower and slower) I contribed code before to chrome, and plan to do so in the future, however even changing one line of Code in say browser.vcproj leads to 10+ mins of linking (yes TEN) what i did was wrote scripts which allow me to re-enable incremental linking and i was happy with it since you lately moved to libs for chrome_dll and the like, the process became harder but i still manage to enable incremental linking (there is an option in VS2008 to use objs instead of static libraries which re-enables incremental linking) just to give you a taste of why i say this without incremental linking i get 10+ mins of build time for changing just one line of code with inc linking i am down to ~15 seconds for a simple change,, which is SUCH a great improvement i recall you removed inc linking since some projects (i know of ui_tests) needs more than 2GB for linking but you could just disable it for this one project i hope you will re-consider introducing inc linking on VS thanx (btw, there are issues with re-introducing inc linking as lib and obj rules are different, but they are kinda small) --~--~-~--~~~---~--~~ Chromium Developers mailing list: chromium-dev@googlegroups.com View archives, change email options, or unsubscribe: http://groups.google.com/group/chromium-dev -~--~~~~--~~--~--~---
[chromium-dev] Re: A suggestion to Drastically improve build times,
So... do you think it will make things quicker on Visual Studio 2005 also?(Will the incremental linking work there, too, or do these changes require some 2008 features?) ☆PhistucK On Wed, Jul 8, 2009 at 18:43, nakro yoav.zilberb...@gmail.com wrote: Ok, i am happy it catches your attention sadly, i cannot submit patches for it as i wrote an external tool who does it automatically for me by patching the sln and vcproj/vsprops (i can pass the source code to anyone if wanted, its a C++ file) but what i would be glad to do if you want is clobber my build and submit here a report on how to enable this incremental build which files need changes (i had to modify source code for it, which is why it is not a simple flag thing) +the nasty flags of VS2008 like UseLibraryDependencyInputs and things like it again we are talking ~40x and beyond speedup on my machine, and i guess on anyones --~--~-~--~~~---~--~~ Chromium Developers mailing list: chromium-dev@googlegroups.com View archives, change email options, or unsubscribe: http://groups.google.com/group/chromium-dev -~--~~~~--~~--~--~---
[chromium-dev] Re: A suggestion to Drastically improve build times,
Yeah, I guess so, but they are not going to invest too much time on the build system, as they have stated in this thread. ☆PhistucK On Wed, Jul 8, 2009 at 19:05, nakro yoav.zilberb...@gmail.com wrote: Phistuck, this is Exactly why this issue is best handled by chrome ppl and not outsides like myslef if VS2005 does not have the tricks of 2008 with regards to using the objects who made the library then all that is needed is to not produce libs as much and turn browser.vcproj and renderer.vcproj to simple folders under chrome_dll then again, i only use 3 programs from this solution (ui_test, unit_test, and chrome) so maybe putting it as folder under chrome_dll would actually hurt other things but all in all, VS2005 works pretty good with incremental linking from past experience but i really trust (and not as fake flattery) that the chrome ppl have a much better overview on how to properly solve it --~--~-~--~~~---~--~~ Chromium Developers mailing list: chromium-dev@googlegroups.com View archives, change email options, or unsubscribe: http://groups.google.com/group/chromium-dev -~--~~~~--~~--~--~---
[chromium-dev] Re: A suggestion to Drastically improve build times,
On Wed, Jul 8, 2009 at 8:43 AM, nakro yoav.zilberb...@gmail.com wrote: Ok, i am happy it catches your attention sadly, i cannot submit patches for it as i wrote an external tool who does it automatically for me by patching the sln and vcproj/vsprops (i can pass the source code to anyone if wanted, its a C++ file) That's fine. It might make sense to port it to another language (like Python) but that's hard to say without being able to see the code. but what i would be glad to do if you want is clobber my build and submit here a report on how to enable this incremental build which files need changes (i had to modify source code for it, which is why it is not a simple flag thing) +the nasty flags of VS2008 like UseLibraryDependencyInputs and things like it again we are talking ~40x and beyond speedup on my machine, and i guess on anyones There might be ways to speed up 2005, but it seems like a win if we can (cleanly) speed up even 2008. Why not post the code somewhere? Or (even better) take a stab at some patches? This could help a lot of people...even those with the fast machines. :-) J --~--~-~--~~~---~--~~ Chromium Developers mailing list: chromium-dev@googlegroups.com View archives, change email options, or unsubscribe: http://groups.google.com/group/chromium-dev -~--~~~~--~~--~--~---
[chromium-dev] Re: A suggestion to Drastically improve build times,
Jeremy, First off, i will be happy to share the code or exe or whatever you will wish the thing is this, i am very interested in chrome (code wise) but i never took a stab with all your gyp and scripts... and while i know enough python to read your scripts (i learned it for it) i have no access to all the machines you do and all the platforms . to keep it short i would guess it should take one of you who knows your gyp/gclient(?) system 2 hours (at MOST ... nothing close to it) to work this out, and i would be more than happy to have this person chat with me live so i can share what i learned marc suggested i could try this on a try server, but i thought only commiters have access to these the reason i think one from chrome is the best is because these changes could affect MAC and UNIX as well normally i solve my own problems and submit patches (i recall you from the text only zoom, right?) but this one requires an inside job i think my mail if you have a person who wants to chat about this is yoav.zilberb...@gmail.com --~--~-~--~~~---~--~~ Chromium Developers mailing list: chromium-dev@googlegroups.com View archives, change email options, or unsubscribe: http://groups.google.com/group/chromium-dev -~--~~~~--~~--~--~---
[chromium-dev] Re: A suggestion to Drastically improve build times,
On Wed, Jul 8, 2009 at 10:46 AM, nakroyoav.zilberb...@gmail.com wrote: the reason i think one from chrome is the best is because these changes could affect MAC and UNIX as well Regarding incremental linking: I don't believe it exists on Linux, but gold is quite fast. My links take around ~5s, though I have a very fast machine. --~--~-~--~~~---~--~~ Chromium Developers mailing list: chromium-dev@googlegroups.com View archives, change email options, or unsubscribe: http://groups.google.com/group/chromium-dev -~--~~~~--~~--~--~---
