[webkit-dev] Compiling WebKit up to 25% faster in Windows?
Hi WebKittens, As you might be aware, we at Opera now have Chromium based products which means that we compile WebKit quite a lot. A big issue for us and our automatic systems has been the long time needed to compile WebKit (inside Chromium), especially in Windows. The big compilation time difference between Linux and Windows didn't seem to make sense so we did some analysis of what was going on and we think that we have found a cause. In WebKit include directives are without path, and instead the compiler is given a very long list of directories to search through. That process takes a lot of time in Windows. It must take some time in OSX and in Linux too but probably less. As an experiment we took the (chromium) project webcore_dom, that normally compiles in 56 seconds in Windows on a generic computer and fixed it. Removing the many include paths in the build system and instead specifying the path in the include directives changed that to 42 seconds, a 25% reduction. There is no reason to think the same reduction doesn't apply to all projects and then there are many many minutes to save for developers and build servers here. Caveat: I don't know if the resulting binary is correct. Is this something that has been talked about in the past, and would you be interested in replacing the long list of directories to search for every include with paths (relative some good base) directly in the include directives? /Daniel ___ webkit-dev mailing list webkit-dev@lists.webkit.org https://lists.webkit.org/mailman/listinfo/webkit-dev
Re: [webkit-dev] Compiling WebKit up to 25% faster in Windows?
On Tue, Mar 26, 2013 at 11:21 AM, Daniel Bratell brat...@opera.com wrote: Is this something that has been talked about in the past, and would you be interested in replacing the long list of directories to search for every include with paths (relative some good base) directly in the include directives? Using explicit paths to include files has been talked about in the past; e.g. https://lists.webkit.org/pipermail/webkit-dev/2011-November/018632.html The most convincing counter argument I can remember is that it'll make refactoring harder because you'll have to update all #include's when you move headers. It would be great if we can figure out if this also improves the build time on Mac/Linux. - R. Niwa ___ webkit-dev mailing list webkit-dev@lists.webkit.org https://lists.webkit.org/mailman/listinfo/webkit-dev
Re: [webkit-dev] Compiling WebKit up to 25% faster in Windows?
On Tue, Mar 26, 2013 at 2:37 PM, Ryosuke Niwa rn...@webkit.org wrote: On Tue, Mar 26, 2013 at 11:21 AM, Daniel Bratell brat...@opera.comwrote: Is this something that has been talked about in the past, and would you be interested in replacing the long list of directories to search for every include with paths (relative some good base) directly in the include directives? Using explicit paths to include files has been talked about in the past; e.g. https://lists.webkit.org/pipermail/webkit-dev/2011-November/018632.html The most convincing counter argument I can remember is that it'll make refactoring harder because you'll have to update all #include's when you move headers. A few tailored scripts make that an easier job. We're all engineers after all. It would be great if we can figure out if this also improves the build time on Mac/Linux. Agreed. Somewhat apples and oranges of course, but the noticeable slow down when compiling WebKit versus the rest of Chromium, file to file, is enormous. Chromium does a full path include directive from the base of the src by convention. I'd love to see Windows builds sped up; definitely the biggest pain point in continuous integration. - R. Niwa ___ webkit-dev mailing list webkit-dev@lists.webkit.org https://lists.webkit.org/mailman/listinfo/webkit-dev ___ webkit-dev mailing list webkit-dev@lists.webkit.org https://lists.webkit.org/mailman/listinfo/webkit-dev
Re: [webkit-dev] Compiling WebKit up to 25% faster in Windows?
On Tue, Mar 26, 2013 at 11:42 AM, Jarred Nicholls jar...@webkit.org wrote: On Tue, Mar 26, 2013 at 2:37 PM, Ryosuke Niwa rn...@webkit.org wrote: On Tue, Mar 26, 2013 at 11:21 AM, Daniel Bratell brat...@opera.comwrote: Is this something that has been talked about in the past, and would you be interested in replacing the long list of directories to search for every include with paths (relative some good base) directly in the include directives? Using explicit paths to include files has been talked about in the past; e.g. https://lists.webkit.org/pipermail/webkit-dev/2011-November/018632.html The most convincing counter argument I can remember is that it'll make refactoring harder because you'll have to update all #include's when you move headers. A few tailored scripts make that an easier job. We're all engineers after all. We have been talking about this for years, and we even have bugs: https://bugs.webkit.org/show_bug.cgi?id=61772 Given that, I'm somewhat skeptical if we ever get around to it in foreseeable future. - R. Niwa ___ webkit-dev mailing list webkit-dev@lists.webkit.org https://lists.webkit.org/mailman/listinfo/webkit-dev
Re: [webkit-dev] Compiling WebKit up to 25% faster in Windows?
In WebKit include directives are without path, and instead the compiler is given a very long list of directories to search through. That process takes a lot of time in Windows. It must take some time in OSX and in Linux too but probably less. Can we make a first-order improvement just by making sure that paths are searched in order of likelihood? Geoff ___ webkit-dev mailing list webkit-dev@lists.webkit.org https://lists.webkit.org/mailman/listinfo/webkit-dev
Re: [webkit-dev] Compiling WebKit up to 25% faster in Windows?
On Tue, Mar 26, 2013 at 11:47 AM, Ryosuke Niwa rn...@webkit.org wrote: On Tue, Mar 26, 2013 at 11:42 AM, Jarred Nicholls jar...@webkit.orgwrote: On Tue, Mar 26, 2013 at 2:37 PM, Ryosuke Niwa rn...@webkit.org wrote: On Tue, Mar 26, 2013 at 11:21 AM, Daniel Bratell brat...@opera.comwrote: Is this something that has been talked about in the past, and would you be interested in replacing the long list of directories to search for every include with paths (relative some good base) directly in the include directives? Using explicit paths to include files has been talked about in the past; e.g. https://lists.webkit.org/pipermail/webkit-dev/2011-November/018632.html The most convincing counter argument I can remember is that it'll make refactoring harder because you'll have to update all #include's when you move headers. A few tailored scripts make that an easier job. We're all engineers after all. We have been talking about this for years, and we even have bugs: https://bugs.webkit.org/show_bug.cgi?id=61772 Given that, I'm somewhat skeptical if we ever get around to it in foreseeable future. Scripts that update include paths from Source/WebCore/foo.h to Source/Platform/foo.h are substantially easier than something to modify XCode projects robustly ;) If we have consensus that we should just switch to paths relative to Source (or maybe a couple different options), that would be (IMO) a big win. It sounds like Daniel co. have already done the big bang conversion. On Tue, Mar 26, 2013 at 11:58 AM, Geoffrey Garen gga...@apple.com wrote: In WebKit include directives are without path, and instead the compiler is given a very long list of directories to search through. That process takes a lot of time in Windows. It must take some time in OSX and in Linux too but probably less. Can we make a first-order improvement just by making sure that paths are searched in order of likelihood? That actually sounds harder and more fragile to me than switch to full paths across the board. -- Dirk ___ webkit-dev mailing list webkit-dev@lists.webkit.org https://lists.webkit.org/mailman/listinfo/webkit-dev
Re: [webkit-dev] Compiling WebKit up to 25% faster in Windows?
On Tue, Mar 26, 2013 at 1:20 PM, Dirk Pranke dpra...@chromium.org wrote If we have consensus that we should just switch to paths relative to Source (or maybe a couple different options), that would be (IMO) a big win. It sounds like Daniel co. have already done the big bang conversion. I think using full path would be a serious hit regarding hackability. I would rather spend some time tweaking my compiler to cache each directory content than waste time finding where is every single header I need to include. Benjamin ___ webkit-dev mailing list webkit-dev@lists.webkit.org https://lists.webkit.org/mailman/listinfo/webkit-dev
Re: [webkit-dev] Compiling WebKit up to 25% faster in Windows?
On Tue, Mar 26, 2013 at 11:37 AM, Ryosuke Niwa rn...@webkit.org wrote: On Tue, Mar 26, 2013 at 11:21 AM, Daniel Bratell brat...@opera.comwrote: Is this something that has been talked about in the past, and would you be interested in replacing the long list of directories to search for every include with paths (relative some good base) directly in the include directives? Using explicit paths to include files has been talked about in the past; e.g. https://lists.webkit.org/pipermail/webkit-dev/2011-November/018632.html The most convincing counter argument I can remember is that it'll make refactoring harder because you'll have to update all #include's when you move headers. It would be great if we can figure out if this also improves the build time on Mac/Linux. Another idea. What you copied all headers into one directory (in a pre compilation step) and used that as the single include directory? - R. Niwa ___ webkit-dev mailing list webkit-dev@lists.webkit.org https://lists.webkit.org/mailman/listinfo/webkit-dev
Re: [webkit-dev] Compiling WebKit up to 25% faster in Windows?
On Tue, Mar 26, 2013 at 1:29 PM, Benjamin Poulain benja...@webkit.orgwrote: On Tue, Mar 26, 2013 at 1:20 PM, Dirk Pranke dpra...@chromium.org wrote If we have consensus that we should just switch to paths relative to Source (or maybe a couple different options), that would be (IMO) a big win. It sounds like Daniel co. have already done the big bang conversion. I think using full path would be a serious hit regarding hackability. I would rather spend some time tweaking my compiler to cache each directory content than waste time finding where is every single header I need to include. Interesting. I have the exact opposite experience, having to paw around to figure out where Font.h actually lives rather than just seeing WebCore/platform/graphics/Font.h. At any rate, to be clear, I would be in favor of that change but I'm not expecting it to happen :). -- Dirk ___ webkit-dev mailing list webkit-dev@lists.webkit.org https://lists.webkit.org/mailman/listinfo/webkit-dev
Re: [webkit-dev] Compiling WebKit up to 25% faster in Windows?
On Tue, Mar 26, 2013 at 5:40 PM, Dirk Pranke dpra...@chromium.org wrote: On Tue, Mar 26, 2013 at 1:29 PM, Benjamin Poulain benja...@webkit.org wrote: On Tue, Mar 26, 2013 at 1:20 PM, Dirk Pranke dpra...@chromium.org wrote If we have consensus that we should just switch to paths relative to Source (or maybe a couple different options), that would be (IMO) a big win. It sounds like Daniel co. have already done the big bang conversion. I think using full path would be a serious hit regarding hackability. I would rather spend some time tweaking my compiler to cache each directory content than waste time finding where is every single header I need to include. Interesting. I have the exact opposite experience, having to paw around to figure out where Font.h actually lives rather than just seeing WebCore/platform/graphics/Font.h. But a modern IDE can easily solve that for you. You just click on the header and it opens it for you. Xcode, QtCreator handles it very well. That said if you don't use an IDE then I understand your point :). At any rate, to be clear, I would be in favor of that change but I'm not expecting it to happen :). -- Dirk ___ webkit-dev mailing list webkit-dev@lists.webkit.org https://lists.webkit.org/mailman/listinfo/webkit-dev -- Software Engineer @ Intel Open Source Technology Center ___ webkit-dev mailing list webkit-dev@lists.webkit.org https://lists.webkit.org/mailman/listinfo/webkit-dev
Re: [webkit-dev] Compiling WebKit up to 25% faster in Windows?
On Tue, Mar 26, 2013 at 1:42 PM, Alexis Menard ale...@webkit.org wrote: On Tue, Mar 26, 2013 at 5:40 PM, Dirk Pranke dpra...@chromium.org wrote: On Tue, Mar 26, 2013 at 1:29 PM, Benjamin Poulain benja...@webkit.org wrote: On Tue, Mar 26, 2013 at 1:20 PM, Dirk Pranke dpra...@chromium.org wrote If we have consensus that we should just switch to paths relative to Source (or maybe a couple different options), that would be (IMO) a big win. It sounds like Daniel co. have already done the big bang conversion. I think using full path would be a serious hit regarding hackability. I would rather spend some time tweaking my compiler to cache each directory content than waste time finding where is every single header I need to include. Interesting. I have the exact opposite experience, having to paw around to figure out where Font.h actually lives rather than just seeing WebCore/platform/graphics/Font.h. But a modern IDE can easily solve that for you. You just click on the header and it opens it for you. Xcode, QtCreator handles it very well. That said if you don't use an IDE then I understand your point :). I think to me the fact that the paths are spelled out is both one less action I have to do to get my question answered (I don't have to click), and a helpful reminder of the general layout of the code (and which directories I might need to care about in relation to a particular file). -- Dirk ___ webkit-dev mailing list webkit-dev@lists.webkit.org https://lists.webkit.org/mailman/listinfo/webkit-dev
Re: [webkit-dev] Compiling WebKit up to 25% faster in Windows?
Also keep in mind that currently different build systems hack the include path up to have the same #include point to different headers depending on the build configuration, so the path expansion for a given #include will not be the same for all ports. It's basically a very non-obvious way to do #if PLATFORM() guards at include sites without looking like it. For instance there are 7 different versions of AuthenticationChallenge.h but only one #include statement in ResourceLoader.cpp. Consider: $ find Source/WebCore -name *.h -printf %f\n | wc -l 3383 $ find Source/WebCore -name *.h -printf %f\n | sort | uniq | wc -l 3288 - James On Tue, Mar 26, 2013 at 1:40 PM, Dirk Pranke dpra...@chromium.org wrote: On Tue, Mar 26, 2013 at 1:29 PM, Benjamin Poulain benja...@webkit.orgwrote: On Tue, Mar 26, 2013 at 1:20 PM, Dirk Pranke dpra...@chromium.org wrote If we have consensus that we should just switch to paths relative to Source (or maybe a couple different options), that would be (IMO) a big win. It sounds like Daniel co. have already done the big bang conversion. I think using full path would be a serious hit regarding hackability. I would rather spend some time tweaking my compiler to cache each directory content than waste time finding where is every single header I need to include. Interesting. I have the exact opposite experience, having to paw around to figure out where Font.h actually lives rather than just seeing WebCore/platform/graphics/Font.h. At any rate, to be clear, I would be in favor of that change but I'm not expecting it to happen :). -- Dirk ___ webkit-dev mailing list webkit-dev@lists.webkit.org https://lists.webkit.org/mailman/listinfo/webkit-dev ___ webkit-dev mailing list webkit-dev@lists.webkit.org https://lists.webkit.org/mailman/listinfo/webkit-dev
Re: [webkit-dev] Compiling WebKit up to 25% faster in Windows?
On Tuesday, March 26, 2013 01:40:56 PM Dirk Pranke wrote: On Tue, Mar 26, 2013 at 1:29 PM, Benjamin Poulain benja...@webkit.orgwrote: On Tue, Mar 26, 2013 at 1:20 PM, Dirk Pranke dpra...@chromium.org wrote If we have consensus that we should just switch to paths relative to Source (or maybe a couple different options), that would be (IMO) a big win. It sounds like Daniel co. have already done the big bang conversion. I think using full path would be a serious hit regarding hackability. I would rather spend some time tweaking my compiler to cache each directory content than waste time finding where is every single header I need to include. Interesting. I have the exact opposite experience, having to paw around to figure out where Font.h actually lives rather than just seeing WebCore/platform/graphics/Font.h. At any rate, to be clear, I would be in favor of that change but I'm not expecting it to happen :). I would be in favor of that change but I'm not expecting it to happen :) This summarizes my feel on any attempt to do a relative big change on WebKit project. -- Dirk signature.asc Description: This is a digitally signed message part. ___ webkit-dev mailing list webkit-dev@lists.webkit.org https://lists.webkit.org/mailman/listinfo/webkit-dev
[webkit-dev] Fwd: Compiling WebKit up to 25% faster in Windows?
Resent from the right email address. On Tue, Mar 26, 2013 at 1:40 PM, Dirk Pranke dpra...@chromium.org wrote: On Tue, Mar 26, 2013 at 1:29 PM, Benjamin Poulain benja...@webkit.orgwrote: On Tue, Mar 26, 2013 at 1:20 PM, Dirk Pranke dpra...@chromium.org wrote If we have consensus that we should just switch to paths relative to Source (or maybe a couple different options), that would be (IMO) a big win. It sounds like Daniel co. have already done the big bang conversion. I think using full path would be a serious hit regarding hackability. I would rather spend some time tweaking my compiler to cache each directory content than waste time finding where is every single header I need to include. Interesting. I have the exact opposite experience, having to paw around to figure out where Font.h actually lives rather than just seeing WebCore/platform/graphics/Font.h. On TextMate or Xcode, all I need need to is to type in Font.h and they'll automatically find the right file for me. I'll be annoyed if people started using same filenames in multiple directories (this has already happened :( ) because then various tools will start listing multiple files from all those directories. - R. Niwa ___ webkit-dev mailing list webkit-dev@lists.webkit.org https://lists.webkit.org/mailman/listinfo/webkit-dev
Re: [webkit-dev] Compiling WebKit up to 25% faster in Windows?
On Mar 26, 2013, at 1:40 PM, Dirk Pranke dpra...@chromium.org wrote: On Tue, Mar 26, 2013 at 1:29 PM, Benjamin Poulain benja...@webkit.org wrote: On Tue, Mar 26, 2013 at 1:20 PM, Dirk Pranke dpra...@chromium.org wrote If we have consensus that we should just switch to paths relative to Source (or maybe a couple different options), that would be (IMO) a big win. It sounds like Daniel co. have already done the big bang conversion. I think using full path would be a serious hit regarding hackability. I would rather spend some time tweaking my compiler to cache each directory content than waste time finding where is every single header I need to include. Interesting. I have the exact opposite experience, having to paw around to figure out where Font.h actually lives rather than just seeing WebCore/platform/graphics/Font.h. At any rate, to be clear, I would be in favor of that change but I'm not expecting it to happen :). I'm with Dirk on this. Full path would help hackability for me. I don't use an IDE, so I'll be typing more. But I spend more time reading code than typing code. Also we have a lot of stupid in header file naming right now. For example the DFG calls the JSC::DFG::Node header DFGNode.h, and puts it in JavaScriptCore/dfg/DFGNode.h. So we duplicate the namespacing of JSC::DFG::Node in both the filename and the directory name. Ridiculous! If we had a discipline to always include using paths relative to Source, then we could just rename it to JavaScriptCore/dfg/Node.h. That would make me happy. -F -- Dirk ___ webkit-dev mailing list webkit-dev@lists.webkit.org https://lists.webkit.org/mailman/listinfo/webkit-dev ___ webkit-dev mailing list webkit-dev@lists.webkit.org https://lists.webkit.org/mailman/listinfo/webkit-dev
Re: [webkit-dev] Compiling WebKit up to 25% faster in Windows?
On Tuesday, March 26, 2013 01:47:26 PM James Robinson wrote: Also keep in mind that currently different build systems hack the include path up to have the same #include point to different headers depending on the build configuration, so the path expansion for a given #include will not be the same for all ports. It's basically a very non-obvious way to do #if PLATFORM() guards at include sites without looking like it. For instance there are 7 different versions of AuthenticationChallenge.h but only one #include statement in ResourceLoader.cpp. I think this is another issue, but for sure a blocking issue for the one being discussed. Consider: $ find Source/WebCore -name *.h -printf %f\n | wc -l 3383 $ find Source/WebCore -name *.h -printf %f\n | sort | uniq | wc -l 3288 - James On Tue, Mar 26, 2013 at 1:40 PM, Dirk Pranke dpra...@chromium.org wrote: On Tue, Mar 26, 2013 at 1:29 PM, Benjamin Poulain benja...@webkit.orgwrote: On Tue, Mar 26, 2013 at 1:20 PM, Dirk Pranke dpra...@chromium.org wrote If we have consensus that we should just switch to paths relative to Source (or maybe a couple different options), that would be (IMO) a big win. It sounds like Daniel co. have already done the big bang conversion. I think using full path would be a serious hit regarding hackability. I would rather spend some time tweaking my compiler to cache each directory content than waste time finding where is every single header I need to include. Interesting. I have the exact opposite experience, having to paw around to figure out where Font.h actually lives rather than just seeing WebCore/platform/graphics/Font.h. At any rate, to be clear, I would be in favor of that change but I'm not expecting it to happen :). -- Dirk ___ webkit-dev mailing list webkit-dev@lists.webkit.org https://lists.webkit.org/mailman/listinfo/webkit-dev signature.asc Description: This is a digitally signed message part. ___ webkit-dev mailing list webkit-dev@lists.webkit.org https://lists.webkit.org/mailman/listinfo/webkit-dev
Re: [webkit-dev] Compiling WebKit up to 25% faster in Windows?
+1. I also like that this will make layering violations clearer. On Tue, Mar 26, 2013 at 1:53 PM, Filip Pizlo fpi...@apple.com wrote: On Mar 26, 2013, at 1:40 PM, Dirk Pranke dpra...@chromium.org wrote: On Tue, Mar 26, 2013 at 1:29 PM, Benjamin Poulain benja...@webkit.org wrote: On Tue, Mar 26, 2013 at 1:20 PM, Dirk Pranke dpra...@chromium.org wrote If we have consensus that we should just switch to paths relative to Source (or maybe a couple different options), that would be (IMO) a big win. It sounds like Daniel co. have already done the big bang conversion. I think using full path would be a serious hit regarding hackability. I would rather spend some time tweaking my compiler to cache each directory content than waste time finding where is every single header I need to include. Interesting. I have the exact opposite experience, having to paw around to figure out where Font.h actually lives rather than just seeing WebCore/platform/graphics/Font.h. At any rate, to be clear, I would be in favor of that change but I'm not expecting it to happen :). I'm with Dirk on this. Full path would help hackability for me. I don't use an IDE, so I'll be typing more. But I spend more time reading code than typing code. Also we have a lot of stupid in header file naming right now. For example the DFG calls the JSC::DFG::Node header DFGNode.h, and puts it in JavaScriptCore/dfg/DFGNode.h. So we duplicate the namespacing of JSC::DFG::Node in both the filename and the directory name. Ridiculous! If we had a discipline to always include using paths relative to Source, then we could just rename it to JavaScriptCore/dfg/Node.h. That would make me happy. -F -- Dirk ___ webkit-dev mailing list webkit-dev@lists.webkit.org https://lists.webkit.org/mailman/listinfo/webkit-dev ___ webkit-dev mailing list webkit-dev@lists.webkit.org https://lists.webkit.org/mailman/listinfo/webkit-dev ___ webkit-dev mailing list webkit-dev@lists.webkit.org https://lists.webkit.org/mailman/listinfo/webkit-dev
Re: [webkit-dev] Compiling WebKit up to 25% faster in Windows?
On Tue, Mar 26, 2013 at 1:53 PM, Filip Pizlo fpi...@apple.com wrote: On Mar 26, 2013, at 1:40 PM, Dirk Pranke dpra...@chromium.org wrote: On Tue, Mar 26, 2013 at 1:29 PM, Benjamin Poulain benja...@webkit.org wrote: On Tue, Mar 26, 2013 at 1:20 PM, Dirk Pranke dpra...@chromium.org wrote If we have consensus that we should just switch to paths relative to Source (or maybe a couple different options), that would be (IMO) a big win. It sounds like Daniel co. have already done the big bang conversion. I think using full path would be a serious hit regarding hackability. I would rather spend some time tweaking my compiler to cache each directory content than waste time finding where is every single header I need to include. Interesting. I have the exact opposite experience, having to paw around to figure out where Font.h actually lives rather than just seeing WebCore/platform/graphics/Font.h. At any rate, to be clear, I would be in favor of that change but I'm not expecting it to happen :). I'm with Dirk on this. Full path would help hackability for me. I don't use an IDE, so I'll be typing more. But I spend more time reading code than typing code. Also we have a lot of stupid in header file naming right now. For example the DFG calls the JSC::DFG::Node header DFGNode.h, and puts it in JavaScriptCore/dfg/DFGNode.h. So we duplicate the namespacing of JSC::DFG::Node in both the filename and the directory name. Ridiculous! If we had a discipline to always include using paths relative to Source, then we could just rename it to JavaScriptCore/dfg/Node.h. That would make me happy. That'll make me sad because then Xcode will then find both WebCore/dom/Node.h and JavaScriptCode/dfg/Node.h when I type Node.h. Unfortunately, we can't name Node.h in WebCore/dom DOMNode.h because DOMNode.h already exists for Objective C bindings. - R. Niwa ___ webkit-dev mailing list webkit-dev@lists.webkit.org https://lists.webkit.org/mailman/listinfo/webkit-dev
Re: [webkit-dev] Compiling WebKit up to 25% faster in Windows?
2013/3/26 Ryosuke Niwa rn...@webkit.org: On Tue, Mar 26, 2013 at 1:53 PM, Filip Pizlo fpi...@apple.com wrote: On Mar 26, 2013, at 1:40 PM, Dirk Pranke dpra...@chromium.org wrote: On Tue, Mar 26, 2013 at 1:29 PM, Benjamin Poulain benja...@webkit.org wrote: On Tue, Mar 26, 2013 at 1:20 PM, Dirk Pranke dpra...@chromium.org wrote If we have consensus that we should just switch to paths relative to Source (or maybe a couple different options), that would be (IMO) a big win. It sounds like Daniel co. have already done the big bang conversion. I think using full path would be a serious hit regarding hackability. I would rather spend some time tweaking my compiler to cache each directory content than waste time finding where is every single header I need to include. Interesting. I have the exact opposite experience, having to paw around to figure out where Font.h actually lives rather than just seeing WebCore/platform/graphics/Font.h. At any rate, to be clear, I would be in favor of that change but I'm not expecting it to happen :). I'm with Dirk on this. Full path would help hackability for me. I don't use an IDE, so I'll be typing more. But I spend more time reading code than typing code. Also we have a lot of stupid in header file naming right now. For example the DFG calls the JSC::DFG::Node header DFGNode.h, and puts it in JavaScriptCore/dfg/DFGNode.h. So we duplicate the namespacing of JSC::DFG::Node in both the filename and the directory name. Ridiculous! If we had a discipline to always include using paths relative to Source, then we could just rename it to JavaScriptCore/dfg/Node.h. That would make me happy. That'll make me sad because then Xcode will then find both WebCore/dom/Node.h and JavaScriptCode/dfg/Node.h when I type Node.h. Unfortunately, we can't name Node.h in WebCore/dom DOMNode.h because DOMNode.h already exists for Objective C bindings. IMHO, we should be favoring code readability instead of a tool's feature. +1 for full paths. -jesus - R. Niwa ___ webkit-dev mailing list webkit-dev@lists.webkit.org https://lists.webkit.org/mailman/listinfo/webkit-dev ___ webkit-dev mailing list webkit-dev@lists.webkit.org https://lists.webkit.org/mailman/listinfo/webkit-dev
Re: [webkit-dev] Compiling WebKit up to 25% faster in Windows?
On 2013-03-26, at 11:37, Ryosuke Niwa rn...@webkit.org wrote: On Tue, Mar 26, 2013 at 11:21 AM, Daniel Bratell brat...@opera.com wrote: Is this something that has been talked about in the past, and would you be interested in replacing the long list of directories to search for every include with paths (relative some good base) directly in the include directives? Using explicit paths to include files has been talked about in the past; e.g. https://lists.webkit.org/pipermail/webkit-dev/2011-November/018632.html The most convincing counter argument I can remember is that it'll make refactoring harder because you'll have to update all #include's when you move headers. It would be great if we can figure out if this also improves the build time on Mac/Linux. I’d be very interested on what effect, if any, this has on Mac too. My initial impression is that any improvement should be minimal since a header map is used to optimize for precisely this situation. If that mechanism isn’t helping then it would be useful to know. - Mark ___ webkit-dev mailing list webkit-dev@lists.webkit.org https://lists.webkit.org/mailman/listinfo/webkit-dev
Re: [webkit-dev] Compiling WebKit up to 25% faster in Windows?
On Tue, Mar 26, 2013 at 5:17 PM, Jesus Sanchez-Palencia je...@webkit.orgwrote: 2013/3/26 Ryosuke Niwa rn...@webkit.org: On Tue, Mar 26, 2013 at 1:53 PM, Filip Pizlo fpi...@apple.com wrote: On Mar 26, 2013, at 1:40 PM, Dirk Pranke dpra...@chromium.org wrote: On Tue, Mar 26, 2013 at 1:29 PM, Benjamin Poulain benja...@webkit.org wrote: On Tue, Mar 26, 2013 at 1:20 PM, Dirk Pranke dpra...@chromium.org wrote If we have consensus that we should just switch to paths relative to Source (or maybe a couple different options), that would be (IMO) a big win. It sounds like Daniel co. have already done the big bang conversion. I think using full path would be a serious hit regarding hackability. I would rather spend some time tweaking my compiler to cache each directory content than waste time finding where is every single header I need to include. Interesting. I have the exact opposite experience, having to paw around to figure out where Font.h actually lives rather than just seeing WebCore/platform/graphics/Font.h. At any rate, to be clear, I would be in favor of that change but I'm not expecting it to happen :). I'm with Dirk on this. Full path would help hackability for me. I don't use an IDE, so I'll be typing more. But I spend more time reading code than typing code. Also we have a lot of stupid in header file naming right now. For example the DFG calls the JSC::DFG::Node header DFGNode.h, and puts it in JavaScriptCore/dfg/DFGNode.h. So we duplicate the namespacing of JSC::DFG::Node in both the filename and the directory name. Ridiculous! If we had a discipline to always include using paths relative to Source, then we could just rename it to JavaScriptCore/dfg/Node.h. That would make me happy. That'll make me sad because then Xcode will then find both WebCore/dom/Node.h and JavaScriptCode/dfg/Node.h when I type Node.h. Unfortunately, we can't name Node.h in WebCore/dom DOMNode.h because DOMNode.h already exists for Objective C bindings. IMHO, we should be favoring code readability instead of a tool's feature. +1 for full paths. -jesus My sentiment exactly. And I agree with Dirk and Filip, seeing full paths improves (my) hackability and clearly shows layer dependencies (and violations). I'd say that it would also help people understand where certain headers live (educational) and to know which header specifically is being included at compile time rather than it being a guessing game. This is just my opinion, but its rendered through experience working in both chromium code and webkit code. Having a few files named the same hasn't at all effected my ability to find the right file (Sublime Text 2), though I understand that argument. In fact I'm forced to learn the difference and distinguish the locations between similarly named files in the code base; I've found it very helpful. Jarred - R. Niwa ___ webkit-dev mailing list webkit-dev@lists.webkit.org https://lists.webkit.org/mailman/listinfo/webkit-dev ___ webkit-dev mailing list webkit-dev@lists.webkit.org https://lists.webkit.org/mailman/listinfo/webkit-dev ___ webkit-dev mailing list webkit-dev@lists.webkit.org https://lists.webkit.org/mailman/listinfo/webkit-dev
Re: [webkit-dev] Compiling WebKit up to 25% faster in Windows?
On Mar 26, 2013, at 2:34 PM, Jarred Nicholls jar...@webkit.org wrote: On Tue, Mar 26, 2013 at 5:17 PM, Jesus Sanchez-Palencia je...@webkit.org wrote: 2013/3/26 Ryosuke Niwa rn...@webkit.org: On Tue, Mar 26, 2013 at 1:53 PM, Filip Pizlo fpi...@apple.com wrote: On Mar 26, 2013, at 1:40 PM, Dirk Pranke dpra...@chromium.org wrote: On Tue, Mar 26, 2013 at 1:29 PM, Benjamin Poulain benja...@webkit.org wrote: On Tue, Mar 26, 2013 at 1:20 PM, Dirk Pranke dpra...@chromium.org wrote If we have consensus that we should just switch to paths relative to Source (or maybe a couple different options), that would be (IMO) a big win. It sounds like Daniel co. have already done the big bang conversion. I think using full path would be a serious hit regarding hackability. I would rather spend some time tweaking my compiler to cache each directory content than waste time finding where is every single header I need to include. Interesting. I have the exact opposite experience, having to paw around to figure out where Font.h actually lives rather than just seeing WebCore/platform/graphics/Font.h. At any rate, to be clear, I would be in favor of that change but I'm not expecting it to happen :). I'm with Dirk on this. Full path would help hackability for me. I don't use an IDE, so I'll be typing more. But I spend more time reading code than typing code. Also we have a lot of stupid in header file naming right now. For example the DFG calls the JSC::DFG::Node header DFGNode.h, and puts it in JavaScriptCore/dfg/DFGNode.h. So we duplicate the namespacing of JSC::DFG::Node in both the filename and the directory name. Ridiculous! If we had a discipline to always include using paths relative to Source, then we could just rename it to JavaScriptCore/dfg/Node.h. That would make me happy. That'll make me sad because then Xcode will then find both WebCore/dom/Node.h and JavaScriptCode/dfg/Node.h when I type Node.h. Unfortunately, we can't name Node.h in WebCore/dom DOMNode.h because DOMNode.h already exists for Objective C bindings. IMHO, we should be favoring code readability instead of a tool's feature. +1 for full paths. -jesus My sentiment exactly. And I agree with Dirk and Filip, seeing full paths improves (my) hackability and clearly shows layer dependencies (and violations). I'd say that it would also help people understand where certain headers live (educational) and to know which header specifically is being included at compile time rather than it being a guessing game. This is just my opinion, but its rendered through experience working in both chromium code and webkit code. Having a few files named the same hasn't at all effected my ability to find the right file (Sublime Text 2), though I understand that argument. In fact I'm forced to learn the difference and distinguish the locations between similarly named files in the code base; I've found it very helpful. I realize that this is partly a matter of taste, and I do not feel too strongly. But I prefer being able to include headers by the short name without the include path. To me, listing the path feels like visual noise and doesn't aid my understanding. But I realize others may feel differently. I also dislike having files with the same name, whether or not you include them by full path. If you do that, then you need to disambiguate to even talk about a file. Regards, Maciej ___ webkit-dev mailing list webkit-dev@lists.webkit.org https://lists.webkit.org/mailman/listinfo/webkit-dev
Re: [webkit-dev] Compiling WebKit up to 25% faster in Windows?
Den 2013-03-26 21:29:32 skrev Benjamin Poulain benja...@webkit.org: On Tue, Mar 26, 2013 at 1:20 PM, Dirk Pranke dpra...@chromium.org wrote If we have consensus that we should just switch to paths relative to Source (or maybe a couple different options), that would be (IMO) a big win. It sounds like Daniel co. have already done the big bang conversion. I think using full path would be a serious hit regarding hackability. I would rather spend some time tweaking my compiler to cache each directory content than waste time finding where is every single header I need to include. I guess you mean that it will be more job moving files around, but that is a rather rare operation, while reading an include directive and wondering what it's part of is rather common (both for compilers, tools and humans). I like the paths as a tool to indicate module dependencies. You can more easily see that a file depends on foo and bar (but not on fie) if you see: #include foo/object.h #include foo/thing.h #include bar/stuff.h That will tell you useful things, and avoid making layering violations by accident. But I realize it's a question of style and as such there is not a right and a wrong, unless there are other factors. And here we have the seemingly heavy compilation time cost for it which I think is a strong argument against delegating the task of finding the header files to the compiler. /Daniel ___ webkit-dev mailing list webkit-dev@lists.webkit.org https://lists.webkit.org/mailman/listinfo/webkit-dev
Re: [webkit-dev] Compiling WebKit up to 25% faster in Windows?
Den 2013-03-26 21:20:10 skrev Dirk Pranke dpra...@chromium.org: If we have consensus that we should just switch to paths relative to Source (or maybe a couple different options), that would be (IMO) a big win. It sounds like Daniel co. have already done the big bang conversion. Not the whole conversion, no. Just enough to confirm the suspicion that it affects compile time in Windows. As other people in the thread have mentioned there are some tricky areas so before investing too much time in this, it seemed to be best to check with people with longer experience of the code if there was something we've missed. Though the conversion is scripted so it can possibly be applied to large portions of the source code and then the tricky areas can be manually fixed. /Daniel ___ webkit-dev mailing list webkit-dev@lists.webkit.org https://lists.webkit.org/mailman/listinfo/webkit-dev
Re: [webkit-dev] Compiling WebKit up to 25% faster in Windows?
On Tue, Mar 26, 2013 at 4:35 PM, Daniel Bratell brat...@opera.com wrote: Den 2013-03-26 21:29:32 skrev Benjamin Poulain benja...@webkit.org: On Tue, Mar 26, 2013 at 1:20 PM, Dirk Pranke dpra...@chromium.org wrote If we have consensus that we should just switch to paths relative to Source (or maybe a couple different options), that would be (IMO) a big win. It sounds like Daniel co. have already done the big bang conversion. I think using full path would be a serious hit regarding hackability. I would rather spend some time tweaking my compiler to cache each directory content than waste time finding where is every single header I need to include. I guess you mean that it will be more job moving files around, but that is a rather rare operation, while reading an include directive and wondering what it's part of is rather common (both for compilers, tools and humans). I like the paths as a tool to indicate module dependencies. You can more easily see that a file depends on foo and bar (but not on fie) if you see: #include foo/object.h #include foo/thing.h #include bar/stuff.h That will tell you useful things, and avoid making layering violations by accident. Sure, but we don't really have multiple modules/components inside WebCore. I can definitely see being able to spot this across different projects in Source (e.g. between WTF, JSC, WebCore, WebKit) but we already do this so I'm not sure what kind of layering violations this is going to help us detect. Do you have a specific use case in mind where this would have helped us? I can see that this would be helpful in separating WebCore and Platform (currently misplaced in WebCore/platform/) but we don't have many self-contained components like Platform in WebCore. It's also worth noting that we can't easily use relative file paths to specify headers across WTF, JSC, WebCore, WebKit, etc.. due to build system constriants. - R. Niwa ___ webkit-dev mailing list webkit-dev@lists.webkit.org https://lists.webkit.org/mailman/listinfo/webkit-dev
Re: [webkit-dev] Compiling WebKit up to 25% faster in Windows?
On Tue, Mar 26, 2013 at 1:31 PM, Ryosuke Niwa rn...@webkit.org wrote: Another idea. What you copied all headers into one directory (in a pre compilation step) and used that as the single include directory? I don't dare chime in about full paths versus short paths, but I'd like to point out that many ports already have tooling in place to emulate Mac style framework includes for WebKit2. For instance, #include WebCore/Page.h works via a script that scans source files and creates (hard, I believe) links in a single directory. This script is surely easily adaptable to use on a project-wide basis. So, if we decide that a shorter search path is a win, it doesn't necessarily mean full paths. --Martin ___ webkit-dev mailing list webkit-dev@lists.webkit.org https://lists.webkit.org/mailman/listinfo/webkit-dev
Re: [webkit-dev] Compiling WebKit up to 25% faster in Windows?
On Tue, Mar 26, 2013 at 4:35 PM, Daniel Bratell brat...@opera.com wrote: Den 2013-03-26 21:29:32 skrev Benjamin Poulain benja...@webkit.org: On Tue, Mar 26, 2013 at 1:20 PM, Dirk Pranke dpra...@chromium.org wrote If we have consensus that we should just switch to paths relative to Source (or maybe a couple different options), that would be (IMO) a big win. It sounds like Daniel co. have already done the big bang conversion. I think using full path would be a serious hit regarding hackability. I would rather spend some time tweaking my compiler to cache each directory content than waste time finding where is every single header I need to include. I guess you mean that it will be more job moving files around, but that is a rather rare operation, while reading an include directive and wondering what it's part of is rather common (both for compilers, tools and humans). My personal issue with full path is I don't think of the code base in term of files but in terms of classes. I don't care where the files are, it is a detail. The problem of moving file is also obvious. There is already a huge cost associated with moving and renaming files (all the build systems). It is to the point that people will prefer leaving broken name rather than renaming headers. By having full path, you would increase that cost further, making refactoring even harder. I like the paths as a tool to indicate module dependencies. You can more easily see that a file depends on foo and bar (but not on fie) if you see: #include foo/object.h #include foo/thing.h #include bar/stuff.h That will tell you useful things, and avoid making layering violations by accident. I don't understand this argument. We already have WTF, WebCore, WebKit and we use them as prefix when including headers. The last problem is platform. But it should be fixed by moving it outside WebCore, not by changing everything else. I think is already silly to have wtf/text as a weird exception. But I realize it's a question of style and as such there is not a right and a wrong, unless there are other factors. And here we have the seemingly heavy compilation time cost for it which I think is a strong argument against delegating the task of finding the header files to the compiler. Hackabilty is a project goal. Compile time is not. If the change means people are afraid to move/rename files, it is a bad idea. If such a change comes with the appropriate tooling for moving/renaming files, I am not opposed to it. Benjamin ___ webkit-dev mailing list webkit-dev@lists.webkit.org https://lists.webkit.org/mailman/listinfo/webkit-dev
Re: [webkit-dev] Compiling WebKit up to 25% faster in Windows?
On Tue, Mar 26, 2013 at 4:51 PM, Daniel Bratell brat...@opera.com wrote: Den 2013-03-26 21:20:10 skrev Dirk Pranke dpra...@chromium.org: If we have consensus that we should just switch to paths relative to Source (or maybe a couple different options), that would be (IMO) a big win. It sounds like Daniel co. have already done the big bang conversion. Not the whole conversion, no. Just enough to confirm the suspicion that it affects compile time in Windows. As other people in the thread have mentioned there are some tricky areas so before investing too much time in this, it seemed to be best to check with people with longer experience of the code if there was something we've missed. Though the conversion is scripted so it can possibly be applied to large portions of the source code and then the tricky areas can be manually fixed. Can you share your scripts, so that we can measure how this changes things on other platforms? Thanks! Nico /Daniel __**_ webkit-dev mailing list webkit-dev@lists.webkit.org https://lists.webkit.org/**mailman/listinfo/webkit-devhttps://lists.webkit.org/mailman/listinfo/webkit-dev ___ webkit-dev mailing list webkit-dev@lists.webkit.org https://lists.webkit.org/mailman/listinfo/webkit-dev
Re: [webkit-dev] Fwd: Compiling WebKit up to 25% faster in Windows?
Hi, On Mar 26, 2013, at 1:50 PM, Ryosuke Niwa rn...@webkit.org wrote: Interesting. I have the exact opposite experience, having to paw around to figure out where Font.h actually lives rather than just seeing WebCore/platform/graphics/Font.h. On TextMate or Xcode, all I need need to is to type in Font.h and they'll automatically find the right file for me. I'll be annoyed if people started using same filenames in multiple directories (this has already happened :( ) because then various tools will start listing multiple files from all those directories. This can be a problem for implementation files, too, at least under windows. Can I share the joy of tracking down linker errors due to multiple compilation units with the name Foo.obj :) That said, I really do prefer the segmented include paths for the same reasons Dirk shared. Thanks, -Brent ___ webkit-dev mailing list webkit-dev@lists.webkit.org https://lists.webkit.org/mailman/listinfo/webkit-dev
