[webkit-dev] Building WebKit's GTK+ back-end using distcc timing
Hi, We've got a distcc setup/environment where we're used to build WebKit Gtk+, a rather an old code base (WebKit.org corresponding commit 9161f3277b15aae8d15a929d5362e28cad7e01c0) compared to the current WebKit's trunk and the thing is that we could achieve a complete build in about 22 min or so, however when we recently moved to a more recent code base, close to the tip of the trunk, we found that with the same infrastructure in place and parameters (# of distcc servers, compilers, optimizing levels, etc.) , the build takes about 1h:15 min which is kind of way too much. Could some one enlighten us about what have changed in the build system of the Gtk+ port of WebKit that would have an impact on the total build time? Could it be that configure.ac has changed so that the final generated GNUMakefile is less optimal and spends more time doing something else, when going through the CXX files to compile them? Thanks, -Ilyes ___ webkit-dev mailing list webkit-dev@lists.webkit.org http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev
Re: [webkit-dev] Building WebKit's GTK+ back-end using distcc timing
On Thu, Mar 10, 2011 at 11:55:15AM +0100, Ilyes Gouta wrote: Hi, [...] Could some one enlighten us about what have changed in the build system of the Gtk+ port of WebKit that would have an impact on the total build time? Could it be that configure.ac has changed so that the final generated GNUMakefile is less optimal and spends more time doing something else, when going through the CXX files to compile them? I think after that commit (if I checked it correctly it was 1 year ago) a lot things have changed and I do not think there is just one patch to blame, probably just the additions of new files to the compilation after that commit could be increasing the time. I've also noticed compilation is slower since 1 year ago, but I do not think it has been just one patch the reason. Anyway the main change lately in the compilation system has been this: http://trac.webkit.org/changeset/75362 But not sure if that could cause such a big impact in your distcc compilation, you can try to check it. I hope this helps. ___ webkit-dev mailing list webkit-dev@lists.webkit.org http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev
Re: [webkit-dev] Building WebKit's GTK+ back-end using distcc timing
Hi Alejandro, I don't think that the slowdown is due to the addition of new C++/code. Even building with most of the WebKit options turned off, actually takes that much of time. By just observing how the files are dispatched to the distcc servers, how these are loaded by the client machine (which initiate the build), almost could tell that this latter is doing more work (slower, more preprocessor work?) for dispatching source code, than a pre-9161f3277b15aae8d15a929d5362e28cad7e01c0 commit code base. In our setup, the client machine which initiates the distributed build, doesn't contribute to it, i.e doesn't compile code, but just serves the source code. Anyway the main change lately in the compilation system has been this: http://trac.webkit.org/changeset/75362 Thanks for the reference! I'll check it out. -Ilyes On Thu, Mar 10, 2011 at 12:46 PM, Alejandro Garcia Castro a...@igalia.com wrote: On Thu, Mar 10, 2011 at 11:55:15AM +0100, Ilyes Gouta wrote: Hi, [...] Could some one enlighten us about what have changed in the build system of the Gtk+ port of WebKit that would have an impact on the total build time? Could it be that configure.ac has changed so that the final generated GNUMakefile is less optimal and spends more time doing something else, when going through the CXX files to compile them? I think after that commit (if I checked it correctly it was 1 year ago) a lot things have changed and I do not think there is just one patch to blame, probably just the additions of new files to the compilation after that commit could be increasing the time. I've also noticed compilation is slower since 1 year ago, but I do not think it has been just one patch the reason. Anyway the main change lately in the compilation system has been this: http://trac.webkit.org/changeset/75362 But not sure if that could cause such a big impact in your distcc compilation, you can try to check it. I hope this helps. ___ webkit-dev mailing list webkit-dev@lists.webkit.org http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev ___ webkit-dev mailing list webkit-dev@lists.webkit.org http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev
Re: [webkit-dev] Building WebKit's GTK+ back-end using distcc timing
Hi Ilyes, On Thu, 10 Mar 2011 13:51:48 +0100 Ilyes Gouta ilyes.go...@gmail.com wrote: I don't think that the slowdown is due to the addition of new C++/code. Even building with most of the WebKit options turned off, actually takes that much of time. By just observing how the files are dispatched to the distcc servers, how these are loaded by the client machine (which initiate the build), almost could tell that this latter is doing more work (slower, more preprocessor work?) for dispatching source code, than a pre-9161f3277b15aae8d15a929d5362e28cad7e01c0 commit code base. In our setup, the client machine which initiates the distributed build, doesn't contribute to it, i.e doesn't compile code, but just serves the source code. As far as I know, distcc master process pre-processes the code before sending it to the slaves. Having more (or bigger) files in the tree could be the cause of your master machine having to do more work before offloading the compilation to the slaves. Last, but not least, more code means more work when linking libraries and programs, and this step is also done by the distcc build master. There could me some other additional reasons why it is taking so much time now, but I would point to pre-processing and linking as possible causes for the distcc master now doing more work. Hope this helps, or provides some hint to help you troubleshoot your builds. Cheers, -- Adrian Perez ape...@igalia.com Igalia - Free Software Engineering signature.asc Description: PGP signature ___ webkit-dev mailing list webkit-dev@lists.webkit.org http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev
Re: [webkit-dev] Building WebKit's GTK+ back-end using distcc timing
As far as I know, distcc master process pre-processes the code before sending it to the slaves. Having more (or bigger) files in the tree could be the cause of your master machine having to do more work before offloading the compilation to the slaves. Last, but not least, more code means more work when linking libraries and programs, and this step is also done by the distcc build master. There could me some other additional reasons why it is taking so much time now, but I would point to pre-processing and linking as possible causes for the distcc master now doing more work. Especially when you do not have enough memory and the system start swapping. Have you checked that? ___ webkit-dev mailing list webkit-dev@lists.webkit.org http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev
Re: [webkit-dev] Webkit handling of plug-ins out-of-process on the mac
One of the specifics is: is there a separate process for each object/document controlled by a given plug-in, or are all objects controlled by that plug-in in the same extra process, or is it a mix? On Mar 9, 2011, at 10:08 PM, Sam Weinig wrote: WebKit has had the ability to run plugins out of process on the mac for quite some time. We recently reimplemented a bunch of that functionality to share infrastructure with WebKit's general out-of-process strategy, and that code can be found http://trac.webkit.org/browser/trunk/Source/WebKit2/PluginProcess. What specifics are you interested in? -Sam On Wed, Mar 9, 2011 at 1:50 PM, Rudi Sherry rshe...@adobe.commailto:rshe...@adobe.com wrote: Sorry if this is the wrong forum. Is WebKit adding the capability to load plug-ins out-of-process on the mac? If so, would this forum be the right one for questions about how this is done, and technical specifics about it? If not, where? Thanks, Rudi ___ webkit-dev mailing list webkit-dev@lists.webkit.orgmailto:webkit-dev@lists.webkit.org http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev ___ webkit-dev mailing list webkit-dev@lists.webkit.org http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev
[webkit-dev] SnowLeopard debug buildbot not there
tL;dr Why isn't there a SnowLeopard debug buildbot? Related: Why does the commit queue (appear) to only run release builds through tests? Details: Yesterday, I did a build of WebKit on SnowLeopard and hit ~12 crashes (mostly in inspector tests). Then I realized that we don't have such a bot, so perhaps that's why these crashes aren't getting noticed by people. It also looks like the commit queue only runs release builds. I have a concern that not having this allows the code to become less stable than it should be. thanks, dave ___ webkit-dev mailing list webkit-dev@lists.webkit.org http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev
Re: [webkit-dev] SnowLeopard debug buildbot not there
On 2011-03-10, at 12:08, David Levin wrote: tL;dr Why isn't there a SnowLeopard debug buildbot? Related: Why does the commit queue (appear) to only run release builds through tests? Details: Yesterday, I did a build of WebKit on SnowLeopard and hit ~12 crashes (mostly in inspector tests). Then I realized that we don't have such a bot, so perhaps that's why these crashes aren't getting noticed by people. It also looks like the commit queue only runs release builds. I have a concern that not having this allows the code to become less stable than it should be. The SnowLeopard Leaks bot builds and tests the debug configuration. - Mark ___ webkit-dev mailing list webkit-dev@lists.webkit.org http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev
Re: [webkit-dev] SnowLeopard debug buildbot not there
On Thu, Mar 10, 2011 at 12:11 PM, Mark Rowe mr...@apple.com wrote: On 2011-03-10, at 12:08, David Levin wrote: tL;dr Why isn't there a SnowLeopard debug buildbot? Related: Why does the commit queue (appear) to only run release builds through tests? Details: Yesterday, I did a build of WebKit on SnowLeopard and hit ~12 crashes (mostly in inspector tests). Then I realized that we don't have such a bot, so perhaps that's why these crashes aren't getting noticed by people. It also looks like the commit queue only runs release builds. I have a concern that not having this allows the code to become less stable than it should be. The SnowLeopard Leaks bot builds and tests the debug configuration. The problem with the leaks bot is that it's always red. At the moment it has 10792 leaks. Would it be possible to report the leaks in a separate column than the test failures/crashes? That way I could look at the failures/crashes column and see at-a-glance whether I'd broken the Debug build. Adam ___ webkit-dev mailing list webkit-dev@lists.webkit.org http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev
Re: [webkit-dev] SnowLeopard debug buildbot not there
On 2011-03-10, at 12:27, Adam Barth wrote: On Thu, Mar 10, 2011 at 12:11 PM, Mark Rowe mr...@apple.com wrote: On 2011-03-10, at 12:08, David Levin wrote: tL;dr Why isn't there a SnowLeopard debug buildbot? Related: Why does the commit queue (appear) to only run release builds through tests? Details: Yesterday, I did a build of WebKit on SnowLeopard and hit ~12 crashes (mostly in inspector tests). Then I realized that we don't have such a bot, so perhaps that's why these crashes aren't getting noticed by people. It also looks like the commit queue only runs release builds. I have a concern that not having this allows the code to become less stable than it should be. The SnowLeopard Leaks bot builds and tests the debug configuration. The problem with the leaks bot is that it's always red. At the moment it has 10792 leaks. Fixing the leaks would help address this, and is obviously beneficial in itself. It certainly didn't have anywhere near this number of leaks last week. Would it be possible to report the leaks in a separate column than the test failures/crashes? That way I could look at the failures/crashes column and see at-a-glance whether I'd broken the Debug build. Like most things, it's possible but not particularly easy. - Mark ___ webkit-dev mailing list webkit-dev@lists.webkit.org http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev
Re: [webkit-dev] SnowLeopard debug buildbot not there
On Thu, Mar 10, 2011 at 12:33 PM, Mark Rowe mr...@apple.com wrote: On 2011-03-10, at 12:27, Adam Barth wrote: On Thu, Mar 10, 2011 at 12:11 PM, Mark Rowe mr...@apple.com wrote: On 2011-03-10, at 12:08, David Levin wrote: tL;dr Why isn't there a SnowLeopard debug buildbot? Related: Why does the commit queue (appear) to only run release builds through tests? Details: Yesterday, I did a build of WebKit on SnowLeopard and hit ~12 crashes (mostly in inspector tests). Then I realized that we don't have such a bot, so perhaps that's why these crashes aren't getting noticed by people. It also looks like the commit queue only runs release builds. I have a concern that not having this allows the code to become less stable than it should be. The SnowLeopard Leaks bot builds and tests the debug configuration. The problem with the leaks bot is that it's always red. At the moment it has 10792 leaks. Fixing the leaks would help address this, and is obviously beneficial in itself. It certainly didn't have anywhere near this number of leaks last week. What's the best way to tack down when that regression occurred? I scrolled to the end of the waterfall for that builder http://build.webkit.org/waterfall?last_time=1299183958show=SnowLeopard%20Intel%20Leaks and the earliest build I could find had 10344 leaks. Would it be possible to report the leaks in a separate column than the test failures/crashes? That way I could look at the failures/crashes column and see at-a-glance whether I'd broken the Debug build. Like most things, it's possible but not particularly easy. Oh, I see that the bot turns golden when there are only leaks but no test failures. Can it turn green instead? (Or maybe have the column turn green but the leaks box turn gold?) I'm showing my buildbox ignorance. Adam ___ webkit-dev mailing list webkit-dev@lists.webkit.org http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev
Re: [webkit-dev] SnowLeopard debug buildbot not there
On 2011-03-10, at 12:40, Adam Barth wrote: On Thu, Mar 10, 2011 at 12:33 PM, Mark Rowe mr...@apple.com wrote: On 2011-03-10, at 12:27, Adam Barth wrote: On Thu, Mar 10, 2011 at 12:11 PM, Mark Rowe mr...@apple.com wrote: On 2011-03-10, at 12:08, David Levin wrote: tL;dr Why isn't there a SnowLeopard debug buildbot? Related: Why does the commit queue (appear) to only run release builds through tests? Details: Yesterday, I did a build of WebKit on SnowLeopard and hit ~12 crashes (mostly in inspector tests). Then I realized that we don't have such a bot, so perhaps that's why these crashes aren't getting noticed by people. It also looks like the commit queue only runs release builds. I have a concern that not having this allows the code to become less stable than it should be. The SnowLeopard Leaks bot builds and tests the debug configuration. The problem with the leaks bot is that it's always red. At the moment it has 10792 leaks. Fixing the leaks would help address this, and is obviously beneficial in itself. It certainly didn't have anywhere near this number of leaks last week. What's the best way to tack down when that regression occurred? I scrolled to the end of the waterfall for that builder http://build.webkit.org/waterfall?last_time=1299183958show=SnowLeopard%20Intel%20Leaks and the earliest build I could find had 10344 leaks. Hrm. It used to be possible to track things down by looking at the historical results at http://build.webkit.org/results/SnowLeopard%20Intel%20Leaks/, but we don't appear to keep the results for very long any more. It would be possible to build and bisect locally to track down when they were introduced. However, given the sheer volume of leaks I suspect it would be relatively easy for someone familiar with whatever code is leaking most prominently to track down the leak. Would it be possible to report the leaks in a separate column than the test failures/crashes? That way I could look at the failures/crashes column and see at-a-glance whether I'd broken the Debug build. Like most things, it's possible but not particularly easy. Oh, I see that the bot turns golden when there are only leaks but no test failures. Can it turn green instead? (Or maybe have the column turn green but the leaks box turn gold?) I'm showing my buildbox ignorance. I don't recall whether this is possible. - Mark ___ webkit-dev mailing list webkit-dev@lists.webkit.org http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev
Re: [webkit-dev] SnowLeopard debug buildbot not there
On Mar 10, 2011, at 3:40 PM, Adam Barth wrote: On Thu, Mar 10, 2011 at 12:33 PM, Mark Rowe mr...@apple.com wrote: On 2011-03-10, at 12:27, Adam Barth wrote: On Thu, Mar 10, 2011 at 12:11 PM, Mark Rowe mr...@apple.com wrote: On 2011-03-10, at 12:08, David Levin wrote: tL;dr Why isn't there a SnowLeopard debug buildbot? Related: Why does the commit queue (appear) to only run release builds through tests? Details: Yesterday, I did a build of WebKit on SnowLeopard and hit ~12 crashes (mostly in inspector tests). Then I realized that we don't have such a bot, so perhaps that's why these crashes aren't getting noticed by people. It also looks like the commit queue only runs release builds. I have a concern that not having this allows the code to become less stable than it should be. The SnowLeopard Leaks bot builds and tests the debug configuration. The problem with the leaks bot is that it's always red. At the moment it has 10792 leaks. Fixing the leaks would help address this, and is obviously beneficial in itself. It certainly didn't have anywhere near this number of leaks last week. What's the best way to tack down when that regression occurred? I scrolled to the end of the waterfall for that builder http://build.webkit.org/waterfall?last_time=1299183958show=SnowLeopard%20Intel%20Leaks and the earliest build I could find had 10344 leaks. You can look at a page like: http://build.webkit.org/builders/SnowLeopard%20Intel%20Leaks?numbuilds=500 And then you can start manually decreasing the build number on pages like: http://build.webkit.org/builders/SnowLeopard%20Intel%20Leaks/builds/15273 -Adam ___ webkit-dev mailing list webkit-dev@lists.webkit.org http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev
Re: [webkit-dev] SnowLeopard debug buildbot not there
On Thu, Mar 10, 2011 at 12:48 PM, Adam Roben aro...@apple.com wrote: On Mar 10, 2011, at 3:40 PM, Adam Barth wrote: On Thu, Mar 10, 2011 at 12:33 PM, Mark Rowe mr...@apple.com wrote: On 2011-03-10, at 12:27, Adam Barth wrote: On Thu, Mar 10, 2011 at 12:11 PM, Mark Rowe mr...@apple.com wrote: On 2011-03-10, at 12:08, David Levin wrote: tL;dr Why isn't there a SnowLeopard debug buildbot? Related: Why does the commit queue (appear) to only run release builds through tests? Details: Yesterday, I did a build of WebKit on SnowLeopard and hit ~12 crashes (mostly in inspector tests). Then I realized that we don't have such a bot, so perhaps that's why these crashes aren't getting noticed by people. It also looks like the commit queue only runs release builds. I have a concern that not having this allows the code to become less stable than it should be. The SnowLeopard Leaks bot builds and tests the debug configuration. The problem with the leaks bot is that it's always red. At the moment it has 10792 leaks. Fixing the leaks would help address this, and is obviously beneficial in itself. It certainly didn't have anywhere near this number of leaks last week. What's the best way to tack down when that regression occurred? I scrolled to the end of the waterfall for that builder http://build.webkit.org/waterfall?last_time=1299183958show=SnowLeopard%20Intel%20Leaks and the earliest build I could find had 10344 leaks. You can look at a page like: http://build.webkit.org/builders/SnowLeopard%20Intel%20Leaks?numbuilds=500 And then you can start manually decreasing the build number on pages like: http://build.webkit.org/builders/SnowLeopard%20Intel%20Leaks/builds/15273 I feel like I should write a webkit-patch command to do this. Looks like http://build.webkit.org/builders/SnowLeopard%20Intel%20Leaks/builds/15164 was (relatively) clean. Starting at http://build.webkit.org/builders/SnowLeopard%20Intel%20Leaks/builds/15165, there's a fog of tons of crashes. When we finally emerge from the fog at http://build.webkit.org/builders/SnowLeopard%20Intel%20Leaks/builds/15220, we have 14437 leaks. So, that's a regression range of 79781--79968. :-( Adam ___ webkit-dev mailing list webkit-dev@lists.webkit.org http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev
Re: [webkit-dev] SnowLeopard debug buildbot not there
On Mar 10, 2011, at 3:51 PM, Adam Roben wrote: I'm also planning to file a bug about CoreText-related leaks beneath GlyphPage::fill. http://webkit.org/b/56136 CoreText-related leaks seen beneath GlyphPage::fill -Adam ___ webkit-dev mailing list webkit-dev@lists.webkit.org http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev
Re: [webkit-dev] SnowLeopard debug buildbot not there
All these inspector's crashes were introduced at r80478 They were not detected immediately because: 1) the noncore Intel Leopard Debug Test box are constantly red; 2) most of the time it has a problem with svn; 3) the box was offline? for a week from mar 1 to mar 8. (~800 patches) Patch was rolled out today. Regards, Tim. On Thu, Mar 10, 2011 at 11:08 PM, David Levin le...@google.com wrote: tL;dr Why isn't there a SnowLeopard debug buildbot? Related: Why does the commit queue (appear) to only run release builds through tests? Details: Yesterday, I did a build of WebKit on SnowLeopard and hit ~12 crashes (mostly in inspector tests). Then I realized that we don't have such a bot, so perhaps that's why these crashes aren't getting noticed by people. It also looks like the commit queue only runs release builds. I have a concern that not having this allows the code to become less stable than it should be. thanks, dave ___ webkit-dev mailing list webkit-dev@lists.webkit.org http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev ___ webkit-dev mailing list webkit-dev@lists.webkit.org http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev
[webkit-dev] How to implement an API taking int or float array as function arguments?
Hi all, I'm looking for a good example of how to implement an API in IDL and source code in WebCore, which takes integer or float array as function arguments from JavaScript. I appreciate your help. Regards, Won ___ webkit-dev mailing list webkit-dev@lists.webkit.org http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev
