Re: [webkit-dev] Starting implementation on W3C Filter Effects
On Nov 3, 2011, at 7:00 PM, Charles Pritchard wrote: In my experience, implementing filters leads to writing them multiple times for various targets. I suggest starting with the lowest common denominator before targeting platforms like webgl. I understand that Google is working on an in-software webgl implementation (angle is just a conversion lib); at some point LLVM may have sufficient semantics-- it's certainly been attempted (there's a polyhedron article somewhere on the site). You're saying you believe Google is developing a version of WebGL that runs completely in the CPU? I haven't heard of such a thing and I would be surprised if it were true. Running a GLSL shader in software is possible, in fact OSX has a software renderer that does just that. And while it can get a few fps with a simple shader, it's not practical for serious realtime 3D graphics. The initial WebKit implementation of CSS filters will use the filter code already in the SVG implementation. This does use vector optimizations on some platforms for some shaders. So it will be fully CPU based. From there several options exist for hardware acceleration, some platform specific and others more generic, based on WebGL or some other GPU based acceleration. In https://bugs.webkit.org/show_bug.cgi?id=68479 I plan on adding some filter infrastructure at the GraphicsLayer level to make it simpler to implement layer-based hardware accelerated filters. - ~Chris cmar...@apple.com ___ webkit-dev mailing list webkit-dev@lists.webkit.org http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev
Re: [webkit-dev] DumpRenderTree for the EFL port :)
Leandro Pereira lean...@profusion.mobi writes: At last, the EFL port of WebKit got a DumpRenderTree implementation! We're still working on ironing out a lot of bugs found by some of the LayoutTests, but the DRT (and ImageDiff) code has been submitted to Bugzilla. It would be awesome if anyone could help reviewing these patches. ... And we finally finished upstreaming our work a few weeks ago, with baselines being committed some time later. We'd like to thank everyone, especially those who helped review our changes (eseidel, rniwa, tkent, tonikitoo, kenneth, mrobinson and so many others). Cheers! -- Raphael Kubo da Costa ProFUSION embedded systems http://profusion.mobi ___ webkit-dev mailing list webkit-dev@lists.webkit.org http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev
Re: [webkit-dev] Moving WTF out of JavaScriptCore (revisited)
Hi Darin, On Nov 2, 2011, at 4:42 PM, Darin Adler wrote: On Nov 2, 2011, at 4:09 PM, Mark Rowe wrote: There are a few related goals here that I'm aware of: a) Separate WTF out of JavaScriptCore since it doesn't logically belong there, but was simply a convenient home for it. b) Separate WebCore/platform out of WebCore to help avoid layering violations. c) Rework the Mac build process so that we can eliminate forwarding headers and remove the duplication of .xcconfig files. The process for addressing a) and b) will be similar: 1) Move the relevant code from its current location to the new location. 2) Create a new Xcode project that builds the desired output in the appropriate fashion. Update other build systems as is applicable. 3) Apple starts including the new project in our build process. Step (2) here involves coming up with a good solution for export control in both the WTF and platform cases. Today we use an explicit .exp file for JavaScriptCore and WebCore on Mac and I believe a .def file in the Apple Windows WebKit port. So there might be a necessary first step of moving to a different export approach. And I know someone has been working on that. If you guys want to go the route of finishing up the export macros work, let me know and I'll see what I can do. I probably can't devote a lot of time to it for the next week or two, but I'd like to see this fixed regardless of if it's used to move forward the WTF split, obviously, so I can put a higher priority on it if I know there are reviewers waiting to land things. :) FWIW, I did distinguish between JS and WTF symbol export macros in the work I've been doing, so the macros approach will support the split as-is. Thanks, Kevin -- Darin ___ webkit-dev mailing list webkit-dev@lists.webkit.org http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev ___ webkit-dev mailing list webkit-dev@lists.webkit.org http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev
Re: [webkit-dev] Moving WTF out of JavaScriptCore (revisited)
On Nov 4, 2011, at 8:48 AM, Kevin Ollivier wrote: Step (2) here involves coming up with a good solution for export control in both the WTF and platform cases. Today we use an explicit .exp file for JavaScriptCore and WebCore on Mac and I believe a .def file in the Apple Windows WebKit port. So there might be a necessary first step of moving to a different export approach. And I know someone has been working on that. If you guys want to go the route of finishing up the export macros work, let me know and I'll see what I can do. I probably can't devote a lot of time to it for the next week or two, but I'd like to see this fixed regardless of if it's used to move forward the WTF split, obviously, so I can put a higher priority on it if I know there are reviewers waiting to land things. :) FWIW, I did distinguish between JS and WTF symbol export macros in the work I've been doing, so the macros approach will support the split as-is. If WTF is to be a static library, there's no need to change anything in the .def file on Windows. .def files apply only to DLLs. FWIW, WTF is already a static library in Apple's Windows port, just self-contained inside the JavaScriptCore project. See: http://trac.webkit.org/browser/trunk/Source/JavaScriptCore/JavaScriptCore.vcproj/WTF/WTF.vcproj -steve ___ webkit-dev mailing list webkit-dev@lists.webkit.org http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev
Re: [webkit-dev] It's time to remove the Haiku port
On Thu, Nov 3, 2011 at 7:32 PM, Ryan Leavengood leaveng...@gmail.comwrote: The primary problem with our port right now is some of the developers made the choice (which I objected to) to make our own Subversion repo containing the WebKit code to make it easier (in their eyes) to develop the port. Where can we go from here? What does the WebKit project need to see from the Haiku maintainers to have us be a supported port again? It seems like either you should be a private fork/port, or a public upstreamed one, but not both. If you want to live in the upstream tree, I think most of your development work should land quickly in public. If you want to work in a separate repository without landing changes back upstream quickly*, I think it's better to stay private until the point at which that changes. *At most I don't think your private repo should be more than a couple weeks off the public repo. Based on our experience with the Chromium port I think it would be far easier on your group to be public and do development directly in the public tree. PK ___ webkit-dev mailing list webkit-dev@lists.webkit.org http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev
Re: [webkit-dev] It's time to remove the Haiku port
On Fri, Nov 4, 2011 at 12:33 PM, Peter Kasting pkast...@google.com wrote: It seems like either you should be a private fork/port, or a public upstreamed one, but not both. If you want to live in the upstream tree, I think most of your development work should land quickly in public. If you want to work in a separate repository without landing changes back upstream quickly*, I think it's better to stay private until the point at which that changes. Yes, this was definitely the hard lesson we learned, and as I'll describe below we are going to move to the latter set up for now. *At most I don't think your private repo should be more than a couple weeks off the public repo. Most definitely, I will strive to do that from now on :) Based on our experience with the Chromium port I think it would be far easier on your group to be public and do development directly in the public tree. That is definitely our end goal. But it seems at this point the Haiku port is too much of a burden to the WebKit community. I talked to Adam Barth in IRC last night and we came to an agreement that it is best *for now* if Haiku maintains a Git repo cloned off the WebKit repo with a branch containing our port. When our port is further along with layout tests working, most of the platform files implemented, a Buildbot, and maintainers with the time and skill to keep our port up-to-date, we can explore putting our code back in the public WebKit repo. This puts the burden on maintaining our port where it belongs: with our developers. It won't always be fun doing it this way, but I think using Git will help (as compared to Subversion which is where our current copy of WebKit is, and now almost 42,000 changesets behind.) Actually regarding the 42,000 changesets: these have all come in the last year and a half (), and are almost as many changesets as ever came before in the current WebKit repo. This is exponential growth! How is a small port possibly suppose to stay up-to-date under those conditions? Of course this just means that WebKit is a vibrant project and you cannot be faulted for that, but I think with this onslaught of changes it may be time to reconsider past methods of keeping ports up-to-date. Adam tells me Google has two full-time engineers keeping the Chromium port up-to-date. I think there is room for improvement in this area. Actually I have various thoughts about WebKit platform code which I'm sure many of you would agree with, but I can express those in another email. The short version is WebCore should have no platform code or PLATFORM macros and all platform code should plug in using clean abstract classes and delegates. Maybe this is an area where I or other Haiku developers can give back to the WebKit community. I just hope that when that time comes (and maybe sooner rather than later) the WebKit community will let us back in and quickly apply our patches. -- Regards, Ryan ___ webkit-dev mailing list webkit-dev@lists.webkit.org http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev
Re: [webkit-dev] Moving WTF out of JavaScriptCore (revisited)
On 2011-11-04, at 10:57, Kevin Ollivier wrote: Hi Steve, On Nov 4, 2011, at 9:12 AM, Steve Falkenburg wrote: On Nov 4, 2011, at 8:48 AM, Kevin Ollivier wrote: Step (2) here involves coming up with a good solution for export control in both the WTF and platform cases. Today we use an explicit .exp file for JavaScriptCore and WebCore on Mac and I believe a .def file in the Apple Windows WebKit port. So there might be a necessary first step of moving to a different export approach. And I know someone has been working on that. If you guys want to go the route of finishing up the export macros work, let me know and I'll see what I can do. I probably can't devote a lot of time to it for the next week or two, but I'd like to see this fixed regardless of if it's used to move forward the WTF split, obviously, so I can put a higher priority on it if I know there are reviewers waiting to land things. :) FWIW, I did distinguish between JS and WTF symbol export macros in the work I've been doing, so the macros approach will support the split as-is. If WTF is to be a static library, there's no need to change anything in the .def file on Windows. .def files apply only to DLLs. Yes, I know, just saying I'd be willing to help if they wanted to go the DLL route. Making it static would make life easier for me also by allowing us to remove the need for WTF symbol exports entirely, of course, so either way is a plus for me. WTF being a static library doesn't change anything with respect to exports. It will still only be linked by a single dynamic library, and that library will be expected to export the necessary symbols for other clients. Doing otherwise would cause problems due to multiple instances of WTF's data being created (e.g., separate FastMalloc heaps for each library that linked directly to WTF). - Mark ___ webkit-dev mailing list webkit-dev@lists.webkit.org http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev
Re: [webkit-dev] Starting implementation on W3C Filter Effects
On 11/4/11 7:23 AM, Chris Marrin wrote: On Nov 3, 2011, at 7:00 PM, Charles Pritchard wrote: In my experience, implementing filters leads to writing them multiple times for various targets. I suggest starting with the lowest common denominator before targeting platforms like webgl. I understand that Google is working on an in-software webgl implementation (angle is just a conversion lib); at some point LLVM may have sufficient semantics-- it's certainly been attempted (there's a polyhedron article somewhere on the site). You're saying you believe Google is developing a version of WebGL that runs completely in the CPU? I haven't heard of such a thing and I would be surprised if it were true. Running a GLSL shader in software is possible, in fact OSX has a software renderer that does just that. And while it can get a few fps with a simple shader, it's not practical for serious realtime 3D graphics. http://code.google.com/p/chromium/issues/detail?id=91445 a software fallback is in the works Similarly, here's WebGL implemented in Canvas 2d and ECMAScript: http://code.google.com/p/cwebgl/ It's certainly the case that CPU rendering will not be practical for serious realtime 3D graphics. There's absolutely a divide between computers that have sufficient GPUs and ones that do not. The initial WebKit implementation of CSS filters will use the filter code already in the SVG implementation. This does use vector optimizations on some platforms for some shaders. So it will be fully CPU based. From there several options exist for hardware acceleration, some platform specific and others more generic, based on WebGL or some other GPU based acceleration. I'm a bit behind on the bleeding edge: Is there work / a foundation for running these rendering process across multiple cores? In https://bugs.webkit.org/show_bug.cgi?id=68479 I plan on adding some filter infrastructure at the GraphicsLayer level to make it simpler to implement layer-based hardware accelerated filters. Much appreciated. -Charles ___ webkit-dev mailing list webkit-dev@lists.webkit.org http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev
Re: [webkit-dev] Moving WTF out of JavaScriptCore (revisited)
Hi Mark, On Nov 4, 2011, at 10:59 AM, Mark Rowe wrote: On 2011-11-04, at 10:57, Kevin Ollivier wrote: Hi Steve, On Nov 4, 2011, at 9:12 AM, Steve Falkenburg wrote: On Nov 4, 2011, at 8:48 AM, Kevin Ollivier wrote: Step (2) here involves coming up with a good solution for export control in both the WTF and platform cases. Today we use an explicit .exp file for JavaScriptCore and WebCore on Mac and I believe a .def file in the Apple Windows WebKit port. So there might be a necessary first step of moving to a different export approach. And I know someone has been working on that. If you guys want to go the route of finishing up the export macros work, let me know and I'll see what I can do. I probably can't devote a lot of time to it for the next week or two, but I'd like to see this fixed regardless of if it's used to move forward the WTF split, obviously, so I can put a higher priority on it if I know there are reviewers waiting to land things. :) FWIW, I did distinguish between JS and WTF symbol export macros in the work I've been doing, so the macros approach will support the split as-is. If WTF is to be a static library, there's no need to change anything in the .def file on Windows. .def files apply only to DLLs. Yes, I know, just saying I'd be willing to help if they wanted to go the DLL route. Making it static would make life easier for me also by allowing us to remove the need for WTF symbol exports entirely, of course, so either way is a plus for me. WTF being a static library doesn't change anything with respect to exports. It will still only be linked by a single dynamic library, and that library will be expected to export the necessary symbols for other clients. Doing otherwise would cause problems due to multiple instances of WTF's data being created (e.g., separate FastMalloc heaps for each library that linked directly to WTF). So I'm assuming that dynamic library would be JSCore, then? Is the idea basically just to have a clean separation between WTF and JSCore build projects? Thanks, Kevin - Mark ___ webkit-dev mailing list webkit-dev@lists.webkit.org http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev
Re: [webkit-dev] Moving WTF out of JavaScriptCore (revisited)
On Fri, Nov 4, 2011 at 11:12 AM, Kevin Ollivier kev...@theolliviers.com wrote: So I'm assuming that dynamic library would be JSCore, then? Is the idea basically just to have a clean separation between WTF and JSCore build projects? Yes. Conceptually, WTF is a separate layer from JavaScriptCore. Adam ___ webkit-dev mailing list webkit-dev@lists.webkit.org http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev
Re: [webkit-dev] Moving WTF out of JavaScriptCore (revisited)
Mark, I've created a stub WTF library at http://trac.webkit.org/browser/trunk/Source/WTF Would you be willing to create an appropriate xcodeproj file that builds Stub.h and Stub.cpp (and integrates with whatever magic is needed internally at Apple)? I'm also happy to attempt webkit.org side of that change if you'd prefer, but I suspect you know better what the contrains are. Thanks, Adam On Wed, Nov 2, 2011 at 4:47 PM, Adam Barth aba...@webkit.org wrote: On Wed, Nov 2, 2011 at 4:35 PM, Mark Rowe mr...@apple.com wrote: On 2011-11-02, at 16:32, Adam Barth wrote: On Wed, Nov 2, 2011 at 4:09 PM, Mark Rowe mr...@apple.com wrote: On 2011-11-02, at 13:23, Adam Barth wrote: As discussed previously, I think it would benefit the project to move WTF out of JavaScriptCore: https://lists.webkit.org/pipermail/webkit-dev/2010-December/015427.html https://lists.webkit.org/pipermail/webkit-dev/2011-February/015940.html Previously, we've been unable to do this because of Apple's internal build process. In thinking about this problem again recently, I wonder if the following would work: 1) Move JavaScriptCore.xcodeproj from Source/JavaScriptCore/JavaScriptCore.xcodeproj to Source/JavaScriptCore.xcodeproj. 2) Change how Apple submits JavaScriptCore to the internal build process to submit Source as the code for JavaScriptCore instead of Source/JavaScriptCore. 3) Move Source/JavaScriptCore/WTF to Source/WTF. Mark, do you have a sense for whether this plan is feasible? If not, is there another approach that would work better? (If my understanding is correct, we could also apply this approach to the other xcodeproj files, which would let us get rid of ForwardingHeaders and move Source/WebCore/platform to Source/Platform.) There are a few related goals here that I'm aware of: a) Separate WTF out of JavaScriptCore since it doesn't logically belong there, but was simply a convenient home for it. b) Separate WebCore/platform out of WebCore to help avoid layering violations. c) Rework the Mac build process so that we can eliminate forwarding headers and remove the duplication of .xcconfig files. Yes. These are the goals. The process for addressing a) and b) will be similar: 1) Move the relevant code from its current location to the new location. 2) Create a new Xcode project that builds the desired output in the appropriate fashion. Update other build systems as is applicable. 3) Apple starts including the new project in our build process. I don't see any benefit or need to move existing Xcode projects as part of this process. Can you expand on why you included this in your proposal? Based on our previous discussions, I was unsure how difficult (3) was on your end. If we can make (a) and (b) happen in the near term without moving xcodeproj files around, that would make me a happy camper. The code moving and the new Xcode project is relatively easy. I'm not sure what will be involved in updating all of the other build systems though. I'm happy to coordinate that part of the effort. I'm not entirely clear on what we'll need to do to tackle c). My current feeling is that it will mainly involve reshuffling of Apple's build processes rather than any significant changes to WebKit. Maybe we should aim to do (a) first, then (b), and then work on (c) once we've figured out what needs to happen on Apple's end? My recollection of the situation with WebCore/platform is that there are a number of existing layering violations in some ports. Given the approach outlined above I suspect they may need to be addressed before we can start making progress on b). Yes. Fixing these issues is going to be a fair amount of work. Perhaps it would make sense to create a stub Platform project for now, which will let us move code from WebCore/platform into Platform as we clean up the dependencies. Adam ___ webkit-dev mailing list webkit-dev@lists.webkit.org http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev
Re: [webkit-dev] Moving WTF out of JavaScriptCore (revisited)
On 2011-11-04, at 11:56, Adam Barth wrote: Mark, I've created a stub WTF library at http://trac.webkit.org/browser/trunk/Source/WTF Would you be willing to create an appropriate xcodeproj file that builds Stub.h and Stub.cpp (and integrates with whatever magic is needed internally at Apple)? I'm also happy to attempt webkit.org side of that change if you'd prefer, but I suspect you know better what the contrains are. Yup, it's on my list. - Mark On Wed, Nov 2, 2011 at 4:47 PM, Adam Barth aba...@webkit.org wrote: On Wed, Nov 2, 2011 at 4:35 PM, Mark Rowe mr...@apple.com wrote: On 2011-11-02, at 16:32, Adam Barth wrote: On Wed, Nov 2, 2011 at 4:09 PM, Mark Rowe mr...@apple.com wrote: On 2011-11-02, at 13:23, Adam Barth wrote: As discussed previously, I think it would benefit the project to move WTF out of JavaScriptCore: https://lists.webkit.org/pipermail/webkit-dev/2010-December/015427.html https://lists.webkit.org/pipermail/webkit-dev/2011-February/015940.html Previously, we've been unable to do this because of Apple's internal build process. In thinking about this problem again recently, I wonder if the following would work: 1) Move JavaScriptCore.xcodeproj from Source/JavaScriptCore/JavaScriptCore.xcodeproj to Source/JavaScriptCore.xcodeproj. 2) Change how Apple submits JavaScriptCore to the internal build process to submit Source as the code for JavaScriptCore instead of Source/JavaScriptCore. 3) Move Source/JavaScriptCore/WTF to Source/WTF. Mark, do you have a sense for whether this plan is feasible? If not, is there another approach that would work better? (If my understanding is correct, we could also apply this approach to the other xcodeproj files, which would let us get rid of ForwardingHeaders and move Source/WebCore/platform to Source/Platform.) There are a few related goals here that I'm aware of: a) Separate WTF out of JavaScriptCore since it doesn't logically belong there, but was simply a convenient home for it. b) Separate WebCore/platform out of WebCore to help avoid layering violations. c) Rework the Mac build process so that we can eliminate forwarding headers and remove the duplication of .xcconfig files. Yes. These are the goals. The process for addressing a) and b) will be similar: 1) Move the relevant code from its current location to the new location. 2) Create a new Xcode project that builds the desired output in the appropriate fashion. Update other build systems as is applicable. 3) Apple starts including the new project in our build process. I don't see any benefit or need to move existing Xcode projects as part of this process. Can you expand on why you included this in your proposal? Based on our previous discussions, I was unsure how difficult (3) was on your end. If we can make (a) and (b) happen in the near term without moving xcodeproj files around, that would make me a happy camper. The code moving and the new Xcode project is relatively easy. I'm not sure what will be involved in updating all of the other build systems though. I'm happy to coordinate that part of the effort. I'm not entirely clear on what we'll need to do to tackle c). My current feeling is that it will mainly involve reshuffling of Apple's build processes rather than any significant changes to WebKit. Maybe we should aim to do (a) first, then (b), and then work on (c) once we've figured out what needs to happen on Apple's end? My recollection of the situation with WebCore/platform is that there are a number of existing layering violations in some ports. Given the approach outlined above I suspect they may need to be addressed before we can start making progress on b). Yes. Fixing these issues is going to be a fair amount of work. Perhaps it would make sense to create a stub Platform project for now, which will let us move code from WebCore/platform into Platform as we clean up the dependencies. Adam ___ webkit-dev mailing list webkit-dev@lists.webkit.org http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev
[webkit-dev] Supporting w3c ref tests and changing our convention
Hi, There was a discussion about supporting W3C ref tests at TPAC yesterday. However, there appears to be some disagreement in how we support them. *Overview* CSS WG ref tests contain link elements that specify reference files. e.g. !DOCTYPE html PUBLIC -//W3C//DTD XHTML 1.0 Strict//EN http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd;html http://december.com/html/4/element/html.html xmlns=http://www.w3.org/1999/xhtml; head http://december.com/html/4/element/head.html title http://december.com/html/4/element/title.htmlCSS Reftest Reference/title http://december.com/html/4/element/title.html link http://december.com/html/4/element/link.html rel=author title=NAME_OF_AUTHOR href=mailto:EMAIL OR http://CONTACT_PAGE/ link http://december.com/html/4/element/link.html rel=match href=green-box-ref.xht / style http://december.com/html/4/element/style.html type=text/css![CDATA[ CSS FOR REFERENCE ]]/style http://december.com/html/4/element/style.html /head http://december.com/html/4/element/head.html body http://december.com/html/4/element/body.html CONTENT OF REFERENCE /body http://december.com/html/4/element/body.html/html http://december.com/html/4/element/html.html The highlighted line specifies that the rendered result of this page should be compared with that of green-box-ref.xht. This is very different from our current method to use filenames such as -match.html, -mismatch.html, etc.. to identify reference files. In addition, W3C test suites have a build step, which generates Mozilla-like test manifest files. This file will contain entries that identifies tests and corresponding reference files. e.g. == floats-wrap-bfc-outside-001.xht floats-wrap-bfc-outside-001-ref.xht == floats-wrap-top-below-bfc-001l.xht floats-wrap-top-below-001l-ref.xht != floats-wrap-top-below-bfc-001l.xht floats-wrap-top-below-001l-notref.xht == floats-wrap-top-below-bfc-001r.xht floats-wrap-top-below-001r-ref.xht != floats-wrap-top-below-bfc-001r.xht floats-wrap-top-below-001r-notref.xht All browser vendors who attended the meeting preferred this format. Microsoft representatives didn't attend the meeting. *Link element approach* Pros: - Can reuse same ref. file for multiple tests - Can have multiple ref. files for single test - Information is self-contained in the test file - We may get away with test suite build step (It turns out that we can't convert W3C ref tests to use WebKit conventions due to the first two points.) Cons: - Requires us modifying each port's DRT to support this format - Adding link elements itself may affect tests (all W3C tests are required to have link elements at the moment) - Hard to understand relationship between files. e.g. if we want to figure out which tests use ref.html, we must look at all test files - Other browser vendors (Firefox and Opera) prefer manifest file approach *Manifest file approach* Pros: - Easy to parse, and do not require us modifiying each port's DRT to support - Easy to tell relationships between files at glance - Do not affect tests since manifest files are external to tests - Preferred approach of Firefox and Opera Cons: - Have to maintain information in two places, tests and manifests (W3C test suites auto-generates manifest files in their build step) - Require build step (new W3C tests only have link elements but not manifests) I strongly prefer WebKit use the manifest file approach and deprecate our current file-name convention because it won't burden each port, and it's the preferred method of other browser vendors. Any opinions? * * Best, Ryosuke Niwa Software Engineer Google Inc. ___ webkit-dev mailing list webkit-dev@lists.webkit.org http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev
Re: [webkit-dev] Moving WTF out of JavaScriptCore (revisited)
On Fri, Nov 4, 2011 at 12:04 PM, Mark Rowe mr...@apple.com wrote: On 2011-11-04, at 11:56, Adam Barth wrote: I've created a stub WTF library at http://trac.webkit.org/browser/trunk/Source/WTF Would you be willing to create an appropriate xcodeproj file that builds Stub.h and Stub.cpp (and integrates with whatever magic is needed internally at Apple)? I'm also happy to attempt webkit.org side of that change if you'd prefer, but I suspect you know better what the contrains are. Yup, it's on my list. Thanks! Adam On Wed, Nov 2, 2011 at 4:47 PM, Adam Barth aba...@webkit.org wrote: On Wed, Nov 2, 2011 at 4:35 PM, Mark Rowe mr...@apple.com wrote: On 2011-11-02, at 16:32, Adam Barth wrote: On Wed, Nov 2, 2011 at 4:09 PM, Mark Rowe mr...@apple.com wrote: On 2011-11-02, at 13:23, Adam Barth wrote: As discussed previously, I think it would benefit the project to move WTF out of JavaScriptCore: https://lists.webkit.org/pipermail/webkit-dev/2010-December/015427.html https://lists.webkit.org/pipermail/webkit-dev/2011-February/015940.html Previously, we've been unable to do this because of Apple's internal build process. In thinking about this problem again recently, I wonder if the following would work: 1) Move JavaScriptCore.xcodeproj from Source/JavaScriptCore/JavaScriptCore.xcodeproj to Source/JavaScriptCore.xcodeproj. 2) Change how Apple submits JavaScriptCore to the internal build process to submit Source as the code for JavaScriptCore instead of Source/JavaScriptCore. 3) Move Source/JavaScriptCore/WTF to Source/WTF. Mark, do you have a sense for whether this plan is feasible? If not, is there another approach that would work better? (If my understanding is correct, we could also apply this approach to the other xcodeproj files, which would let us get rid of ForwardingHeaders and move Source/WebCore/platform to Source/Platform.) There are a few related goals here that I'm aware of: a) Separate WTF out of JavaScriptCore since it doesn't logically belong there, but was simply a convenient home for it. b) Separate WebCore/platform out of WebCore to help avoid layering violations. c) Rework the Mac build process so that we can eliminate forwarding headers and remove the duplication of .xcconfig files. Yes. These are the goals. The process for addressing a) and b) will be similar: 1) Move the relevant code from its current location to the new location. 2) Create a new Xcode project that builds the desired output in the appropriate fashion. Update other build systems as is applicable. 3) Apple starts including the new project in our build process. I don't see any benefit or need to move existing Xcode projects as part of this process. Can you expand on why you included this in your proposal? Based on our previous discussions, I was unsure how difficult (3) was on your end. If we can make (a) and (b) happen in the near term without moving xcodeproj files around, that would make me a happy camper. The code moving and the new Xcode project is relatively easy. I'm not sure what will be involved in updating all of the other build systems though. I'm happy to coordinate that part of the effort. I'm not entirely clear on what we'll need to do to tackle c). My current feeling is that it will mainly involve reshuffling of Apple's build processes rather than any significant changes to WebKit. Maybe we should aim to do (a) first, then (b), and then work on (c) once we've figured out what needs to happen on Apple's end? My recollection of the situation with WebCore/platform is that there are a number of existing layering violations in some ports. Given the approach outlined above I suspect they may need to be addressed before we can start making progress on b). Yes. Fixing these issues is going to be a fair amount of work. Perhaps it would make sense to create a stub Platform project for now, which will let us move code from WebCore/platform into Platform as we clean up the dependencies. Adam ___ webkit-dev mailing list webkit-dev@lists.webkit.org http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev
Re: [webkit-dev] Supporting w3c ref tests and changing our convention
Could you clarify why we have to need to modify DRT if we have Link Element approach? I guess one of the reasons is the performance. It might be better that we use DRT to parse and extract reference links from HTML since parsing HTML using Python might take unacceptable time and might be inaccurate. Is there any other reasons we should modify DRT to support Link Element approach? On Fri, Nov 4, 2011 at 12:34 PM, Ryosuke Niwa rn...@webkit.org wrote: Hi, There was a discussion about supporting W3C ref tests at TPAC yesterday. However, there appears to be some disagreement in how we support them. *Overview* CSS WG ref tests contain link elements that specify reference files. e.g. !DOCTYPE html PUBLIC -//W3C//DTD XHTML 1.0 Strict//EN http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd;html http://december.com/html/4/element/html.html xmlns=http://www.w3.org/1999/xhtml; head http://december.com/html/4/element/head.html title http://december.com/html/4/element/title.htmlCSS Reftest Reference/title http://december.com/html/4/element/title.html link http://december.com/html/4/element/link.html rel=author title=NAME_OF_AUTHOR href=mailto:EMAIL OR http://CONTACT_PAGE/ link http://december.com/html/4/element/link.html rel=match href=green-box-ref.xht / style http://december.com/html/4/element/style.html type=text/css![CDATA[ CSS FOR REFERENCE ]]/style http://december.com/html/4/element/style.html /head http://december.com/html/4/element/head.html body http://december.com/html/4/element/body.html CONTENT OF REFERENCE /body http://december.com/html/4/element/body.html/html http://december.com/html/4/element/html.html The highlighted line specifies that the rendered result of this page should be compared with that of green-box-ref.xht. This is very different from our current method to use filenames such as -match.html, -mismatch.html, etc.. to identify reference files. In addition, W3C test suites have a build step, which generates Mozilla-like test manifest files. This file will contain entries that identifies tests and corresponding reference files. e.g. == floats-wrap-bfc-outside-001.xht floats-wrap-bfc-outside-001-ref.xht == floats-wrap-top-below-bfc-001l.xht floats-wrap-top-below-001l-ref.xht != floats-wrap-top-below-bfc-001l.xht floats-wrap-top-below-001l-notref.xht == floats-wrap-top-below-bfc-001r.xht floats-wrap-top-below-001r-ref.xht != floats-wrap-top-below-bfc-001r.xht floats-wrap-top-below-001r-notref.xht All browser vendors who attended the meeting preferred this format. Microsoft representatives didn't attend the meeting. *Link element approach* Pros: - Can reuse same ref. file for multiple tests - Can have multiple ref. files for single test - Information is self-contained in the test file - We may get away with test suite build step (It turns out that we can't convert W3C ref tests to use WebKit conventions due to the first two points.) Cons: - Requires us modifying each port's DRT to support this format - Adding link elements itself may affect tests (all W3C tests are required to have link elements at the moment) - Hard to understand relationship between files. e.g. if we want to figure out which tests use ref.html, we must look at all test files - Other browser vendors (Firefox and Opera) prefer manifest file approach *Manifest file approach* Pros: - Easy to parse, and do not require us modifiying each port's DRT to support - Easy to tell relationships between files at glance - Do not affect tests since manifest files are external to tests - Preferred approach of Firefox and Opera Cons: - Have to maintain information in two places, tests and manifests (W3C test suites auto-generates manifest files in their build step) - Require build step (new W3C tests only have link elements but not manifests) I strongly prefer WebKit use the manifest file approach and deprecate our current file-name convention because it won't burden each port, and it's the preferred method of other browser vendors. Any opinions? * * Best, Ryosuke Niwa Software Engineer Google Inc. ___ webkit-dev mailing list webkit-dev@lists.webkit.org http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev -- Hayato ___ webkit-dev mailing list webkit-dev@lists.webkit.org http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev
Re: [webkit-dev] Supporting w3c ref tests and changing our convention
On Fri, Nov 4, 2011 at 1:32 PM, Hayato Ito hay...@chromium.org wrote: Could you clarify why we have to need to modify DRT if we have Link Element approach? I guess one of the reasons is the performance. It might be better that we use DRT to parse and extract reference links from HTML since parsing HTML using Python might take unacceptable time and might be inaccurate. Is there any other reasons we should modify DRT to support Link Element approach? That's the primary reason. I can see that there are more than 100,000 .xht files just in css2.1 test suite, and the number is growing. If we include css3, and other w3c test suites, we'll end up having tens of thousands of tests, if not millions. In addition, link element approach doesn't scale well in that there's no mandate as to how reference files are named, so we'll end up needlessly parsing all reference files as well depending on the order they appear in the directory. - Ryosuke ___ webkit-dev mailing list webkit-dev@lists.webkit.org http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev
Re: [webkit-dev] Supporting w3c ref tests and changing our convention
On Fri, Nov 4, 2011 at 1:32 PM, Hayato Ito hay...@chromium.org wrote: Could you clarify why we have to need to modify DRT if we have Link Element approach? I guess one of the reasons is the performance. It might be better that we use DRT to parse and extract reference links from HTML since parsing HTML using Python might take unacceptable time and might be inaccurate. Is there any other reasons we should modify DRT to support Link Element approach? I agree with Ito-san that I don't think you have to modify each DRT. I see no reason to think that we couldn't parse the files reliably in NRWT. It's unclear how much of a perf impact there would be but that's easy enough to determine - I would expect it to be minimal compared to the time of actually rendering a page. That said, supporting a manifest file is clearly fairly easy to do in NRWT, and presumably easy to add as a build step (e.g., make DRT depend on it) and may have the added bonus of allowing us to run various mozilla reference test suites that wouldn't be using the links, so I'm fine with that. I think we should voice a concern w/ the W3C that their tests must follow consistent naming styles (for maintainability); we shouldn't view the links and the manifest step as a carte blanche to name tests and results whatever they want. Separately, if we are throwing around numbers in the range of 100K for tests to run, we should consider when we actually want to run them - i.e., what will the cycle time be if we run them on every change, etc.? But that can be dealt with when we get there. -- Dirk ___ webkit-dev mailing list webkit-dev@lists.webkit.org http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev
Re: [webkit-dev] Supporting w3c ref tests and changing our convention
If we import a bunch of w3c reftests in the future, it sounds reasonable and unavoidable to use manifest file, assuming the manifest file is auto-generated by w3c's build process. But I'd like to leave an option to developers to write reftests more casually without worrying about maintaing the manifest file. At the same time, I don't want to increase the number of ways to write/run tests anymore. So that would be great that we have the best of both worlds somehow. On Fri, Nov 4, 2011 at 1:39 PM, Ryosuke Niwa rn...@webkit.org wrote: On Fri, Nov 4, 2011 at 1:32 PM, Hayato Ito hay...@chromium.org wrote: Could you clarify why we have to need to modify DRT if we have Link Element approach? I guess one of the reasons is the performance. It might be better that we use DRT to parse and extract reference links from HTML since parsing HTML using Python might take unacceptable time and might be inaccurate. Is there any other reasons we should modify DRT to support Link Element approach? That's the primary reason. I can see that there are more than 100,000 .xht files just in css2.1 test suite, and the number is growing. If we include css3, and other w3c test suites, we'll end up having tens of thousands of tests, if not millions. In addition, link element approach doesn't scale well in that there's no mandate as to how reference files are named, so we'll end up needlessly parsing all reference files as well depending on the order they appear in the directory. - Ryosuke -- Hayato ___ webkit-dev mailing list webkit-dev@lists.webkit.org http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev
Re: [webkit-dev] Supporting w3c ref tests and changing our convention
On Fri, Nov 4, 2011 at 1:47 PM, Dirk Pranke dpra...@chromium.org wrote: It's unclear how much of a perf impact there would be but that's easy enough to determine - I would expect it to be minimal compared to the time of actually rendering a page. Since I expect w3c to end up having hundreds of thousands of tests, I see any performance implication to be a serious threat. That said, supporting a manifest file is clearly fairly easy to do in NRWT, and presumably easy to add as a build step (e.g., make DRT depend on it) and may have the added bonus of allowing us to run various mozilla reference test suites that wouldn't be using the links, so I'm fine with that. In fact, we already have a patch to support it on : https://bugs.webkit.org/show_bug.cgi?id=66837 and https://bugs.webkit.org/show_bug.cgi?id=71567 I think we should voice a concern w/ the W3C that their tests must follow consistent naming styles (for maintainability); we shouldn't view the links and the manifest step as a carte blanche to name tests and results whatever they want. Yes, I have. More comments on http://www.w3.org/html/wg/wiki/Testing or http://wiki.csswg.org/test will be helpful. Separately, if we are throwing around numbers in the range of 100K for tests to run, we should consider when we actually want to run them - i.e., what will the cycle time be if we run them on every change, etc.? But that can be dealt with when we get there. We need separate bots in the long term for sure. - Ryosuke ___ webkit-dev mailing list webkit-dev@lists.webkit.org http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev
Re: [webkit-dev] Supporting w3c ref tests and changing our convention
On Fri, Nov 4, 2011 at 4:47 PM, Dirk Pranke dpra...@chromium.org wrote: Separately, if we are throwing around numbers in the range of 100K for tests to run, we should consider when we actually want to run them - i.e., what will the cycle time be if we run them on every change, etc.? But that can be dealt with when we get there. I would also like to throw out the idea of pulling the layout tests out into their own repo, maybe even per platform. Currently the huge number of layout tests in WebKit make many git operations unbearably slow, such as git status, which basically does a stat() on every file in a repo when the plain git status is used. Even on Linux where stat() is mega-fast it takes many seconds to show git status even on a clean checkout (at least on my laptop, which admittedly isn't the fastest.) It is worse on other platforms with a slower stat(). But I expect there might be a lot of pushback on such an idea, especially just to make one tool run faster. -- Regards, Ryan ___ webkit-dev mailing list webkit-dev@lists.webkit.org http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev
Re: [webkit-dev] It's time to remove the Haiku port
On Fri, Nov 4, 2011 at 10:56 AM, Ryan Leavengood leaveng...@gmail.com wrote: Actually regarding the 42,000 changesets: these have all come in the last year and a half (), and are almost as many changesets as ever came before in the current WebKit repo. This is exponential growth! How is a small port possibly suppose to stay up-to-date under those conditions? Of course this just means that WebKit is a vibrant project and you cannot be faulted for that, but I think with this onslaught of changes it may be time to reconsider past methods of keeping ports up-to-date. Adam tells me Google has two full-time engineers keeping the Chromium port up-to-date. I think there is room for improvement in this area. As far as keeping ports up-to-date goes, it seems like (in my limited experience) there's roughly four kinds of changes you have to work about: changes to tests that require new baselines, changes to the build system (adding, renaming, deleting files), changes to platform-specific core functionality (usually o/s-specific hooks), and changes to platform functionality (e.g., webaudio). Hopefully we will start writing more and more tests as reftests, to address the first category. Moving to one of the existing cross-port build mechanisms like GYP or CMake would help. Changes to core platform code in the third category are hopefully pretty rare, and I think you're just out of luck for the fourth category, but hopefully they're done in a modular manner and at least easy to enable/disable for a while. I think the more feedback we can get on what sorts of things you do have to do to keep up to date, the better things will get. -- Dirk ___ webkit-dev mailing list webkit-dev@lists.webkit.org http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev
[webkit-dev] Putting layout tests on a separate repository (was Supporting w3c ref tests and changing our convention)
On Fri, Nov 4, 2011 at 2:39 PM, Ryan Leavengood leaveng...@gmail.comwrote: I would also like to throw out the idea of pulling the layout tests out into their own repo, maybe even per platform. How do we keep webkit/layout repositories in sync? Currently the huge number of layout tests in WebKit make many git operations unbearably slow, such as git status, which basically does a stat() on every file in a repo when the plain git status is used. Even on Linux where stat() is mega-fast it takes many seconds to show git status even on a clean checkout (at least on my laptop, which admittedly isn't the fastest.) It is worse on other platforms with a slower stat(). Even if we put in a separate repo, you'll likely need to checkout all of them anyway because if your patch ever require baselines in some port that you don't work on, you'll still need to rebaseline them. But I expect there might be a lot of pushback on such an idea, especially just to make one tool run faster. Indeed, I'm against this idea. - Ryosuke ___ webkit-dev mailing list webkit-dev@lists.webkit.org http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev
Re: [webkit-dev] It's time to remove the Haiku port
On Fri, Nov 4, 2011 at 5:40 PM, Dirk Pranke dpra...@chromium.org wrote: Hopefully we will start writing more and more tests as reftests, to address the first category. Moving to one of the existing cross-port build mechanisms like GYP or CMake would help. Yeah I will attempt to make use of GYP myself for the Haiku port, except I'll need to add a new backend to generate Jamfiles, since Jam is the preferred Haiku build system. Changes to core platform code in the third category are hopefully pretty rare, and I think you're just out of luck for the fourth category, but hopefully they're done in a modular manner and at least easy to enable/disable for a while. I think the more feedback we can get on what sorts of things you do have to do to keep up to date, the better things will get. For additions to platform, I think a harmless default implementation should suffice. Having to add an empty method to a port just to satisfy the compiler seems like a lot of trouble. I can't find a good example at the moment, so maybe it really isn't an issue. Looking over some of the changes made to the Haiku port by non-Haiku people just shows a lot of renames and changing of method arguments and return values in the platform directory. This tells me that platform really isn't any sort of API (and yes I know it isn't meant to be) but maybe it should be? I personally would like to see WebCore evolve into being as self-contained as possible, with clear APIs for a platform to attach to. Maybe the new Platform directory will be the start of this, and if so, I applaud the effort. Right now the platform coupling is pretty bad. I hope one day #if PLATFORM(X) doesn't exist in WebCore, at least not where such code adds a class member or methods to core platform classes. -- Regards, Ryan ___ webkit-dev mailing list webkit-dev@lists.webkit.org http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev
Re: [webkit-dev] Putting layout tests on a separate repository (was Supporting w3c ref tests and changing our convention)
On Fri, Nov 4, 2011 at 5:58 PM, Ryosuke Niwa rn...@webkit.org wrote: Indeed, I'm against this idea. You make good points. There may be solutions to your objections, but they might not be much better than having a slow git status right now. Do other people who use WebKit with Git see this problem? If so, what do you do to stay productive? If not then I'll just be sure to only develop WebKit on a faster machine (which I do have and look forward to testing soon.) Sorry for hijacking the other thread, thanks for changing the subject. -- Regards, Ryan ___ webkit-dev mailing list webkit-dev@lists.webkit.org http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev
Re: [webkit-dev] It's time to remove the Haiku port
On Fri, Nov 4, 2011 at 3:04 PM, Ryan Leavengood leaveng...@gmail.comwrote: I personally would like to see WebCore evolve into being as self-contained as possible, with clear APIs for a platform to attach to. I suspect the pain hasn't been big enough so far for any person/organization to decide to address this. You do sound enthusiastic about it though, and I suspect there would be people willing to review these changes as long as they didn't slow down WebKit or make it harder to maintain. dave ___ webkit-dev mailing list webkit-dev@lists.webkit.org http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev
Re: [webkit-dev] Putting layout tests on a separate repository (was Supporting w3c ref tests and changing our convention)
On Fri, Nov 4, 2011 at 3:09 PM, Ryan Leavengood leaveng...@gmail.comwrote: You make good points. There may be solutions to your objections, but they might not be much better than having a slow git status right now. Do other people who use WebKit with Git see this problem? If so, what do you do to stay productive? If not then I'll just be sure to only develop WebKit on a faster machine (which I do have and look forward to testing soon.) I use even slower svn and never had an issue with it except when I revert/merge commits. - Ryosuke ___ webkit-dev mailing list webkit-dev@lists.webkit.org http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev
[webkit-dev] Removing Support for Python 2.5
Now that Apple has removed the Leopard build bot (and presumably stopped supporting WebKit on Leopard), all webkit platforms I know of have Python 2.6 or higher. My plan is to remove all of our 2.5 supporting code in the next week, requiring Python 2.6 or later for WebKit. Let me know if this will be an issue for you. Thanks! -eric ___ webkit-dev mailing list webkit-dev@lists.webkit.org http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev
Re: [webkit-dev] Removing Support for Python 2.5
The chromium port still has a bot that runs tests (but doesn't build) on 10.5. Nico On Fri, Nov 4, 2011 at 3:20 PM, Eric Seidel e...@webkit.org wrote: Now that Apple has removed the Leopard build bot (and presumably stopped supporting WebKit on Leopard), all webkit platforms I know of have Python 2.6 or higher. My plan is to remove all of our 2.5 supporting code in the next week, requiring Python 2.6 or later for WebKit. Let me know if this will be an issue for you. Thanks! -eric ___ webkit-dev mailing list webkit-dev@lists.webkit.org http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev ___ webkit-dev mailing list webkit-dev@lists.webkit.org http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev
Re: [webkit-dev] It's time to remove the Haiku port
On Fri, Nov 4, 2011 at 6:12 PM, David Levin le...@chromium.org wrote: I suspect the pain hasn't been big enough so far for any person/organization to decide to address this. Well it may not really be worth the trouble, I just always found the #ifdefs in WebCore/platform to be a little smelly. But they work. You do sound enthusiastic about it though, and I suspect there would be people willing to review these changes as long as they didn't slow down WebKit or make it harder to maintain. That is good to know. There may be a time when I do this work, but I have bigger fish to fry at the moment with updating the Haiku port in our own repo, and my open source time is fairly limited currently (in case it isn't obvious since it took a month and a half to notice our port was removed from WebKit.) -- Regards, Ryan ___ webkit-dev mailing list webkit-dev@lists.webkit.org http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev
Re: [webkit-dev] Putting layout tests on a separate repository (was Supporting w3c ref tests and changing our convention)
On 11/4/11 3:09 PM, Ryan Leavengood leaveng...@gmail.com wrote: On Fri, Nov 4, 2011 at 5:58 PM, Ryosuke Niwa rn...@webkit.org wrote: Indeed, I'm against this idea. You make good points. There may be solutions to your objections, but they might not be much better than having a slow git status right now. Do other people who use WebKit with Git see this problem? If so, what do you do to stay productive? If not then I'll just be sure to only develop WebKit on a faster machine (which I do have and look forward to testing soon.) Note that supporting reftests has the possibility of slowing LayoutTests folder growth for WebKit-specific tests. If you have the choice of writing a reftest with a single html reference as the expected result, you should not get a proliferation of dumprendertree or png baselines in the platform folder. Once we figure out how to support imported reftests, we should be encouraging people to use reftests internally (even for tests we have no intention of pushing to the W3C) instead of dumprendertree or pixel tests (where possible - I assume reftests won't always work for visual tests). Alan ___ webkit-dev mailing list webkit-dev@lists.webkit.org http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev
Re: [webkit-dev] Removing Support for Python 2.5
I believe the chromium port always uses 2.6 though, no? On Fri, Nov 4, 2011 at 3:23 PM, Nico Weber tha...@chromium.org wrote: The chromium port still has a bot that runs tests (but doesn't build) on 10.5. Nico On Fri, Nov 4, 2011 at 3:20 PM, Eric Seidel e...@webkit.org wrote: Now that Apple has removed the Leopard build bot (and presumably stopped supporting WebKit on Leopard), all webkit platforms I know of have Python 2.6 or higher. My plan is to remove all of our 2.5 supporting code in the next week, requiring Python 2.6 or later for WebKit. Let me know if this will be an issue for you. Thanks! -eric ___ webkit-dev mailing list webkit-dev@lists.webkit.org http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev ___ webkit-dev mailing list webkit-dev@lists.webkit.org http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev ___ webkit-dev mailing list webkit-dev@lists.webkit.org http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev
Re: [webkit-dev] It's time to remove the Haiku port
On Fri, Nov 4, 2011 at 3:24 PM, Ryan Leavengood leaveng...@gmail.comwrote: There may be a time when I do this work, but I have bigger fish to fry at the moment... Exactly :) ___ webkit-dev mailing list webkit-dev@lists.webkit.org http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev
Re: [webkit-dev] Removing Support for Python 2.5
Yes, Chromium versions its Python independently from the OS. Adam On Fri, Nov 4, 2011 at 3:25 PM, Ojan Vafai o...@chromium.org wrote: I believe the chromium port always uses 2.6 though, no? On Fri, Nov 4, 2011 at 3:23 PM, Nico Weber tha...@chromium.org wrote: The chromium port still has a bot that runs tests (but doesn't build) on 10.5. Nico On Fri, Nov 4, 2011 at 3:20 PM, Eric Seidel e...@webkit.org wrote: Now that Apple has removed the Leopard build bot (and presumably stopped supporting WebKit on Leopard), all webkit platforms I know of have Python 2.6 or higher. My plan is to remove all of our 2.5 supporting code in the next week, requiring Python 2.6 or later for WebKit. Let me know if this will be an issue for you. Thanks! -eric ___ webkit-dev mailing list webkit-dev@lists.webkit.org http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev ___ webkit-dev mailing list webkit-dev@lists.webkit.org http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev ___ webkit-dev mailing list webkit-dev@lists.webkit.org http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev ___ webkit-dev mailing list webkit-dev@lists.webkit.org http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev
Re: [webkit-dev] It's time to remove the Haiku port
On Fri, Nov 4, 2011 at 6:37 PM, David Levin le...@chromium.org wrote: Exactly :) What are you saying, the rest of the WebKit community doesn't want to spend weeks perfecting the architecture to make my life easier? :) When the need comes, it'll be done. Which may be never, but that's life. -- Regards, Ryan ___ webkit-dev mailing list webkit-dev@lists.webkit.org http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev
Re: [webkit-dev] Removing Support for Python 2.5
Are you sure? This output has references to System/Library/Frameworks/Python.framework/Versions/2.5. I also thought that's why NRWT was slow on the Leopard bots: python 2.5 doesn't have the multiprocess module. http://build.chromium.org/p/chromium.webkit/builders/Webkit%20Mac10.5%20%28dbg%29%281%29/builds/4090/steps/webkit_tests/logs/stdio It may just be a bug that these bots aren't using python2.6. On Fri, Nov 4, 2011 at 3:40 PM, Adam Barth aba...@webkit.org wrote: Yes, Chromium versions its Python independently from the OS. Adam On Fri, Nov 4, 2011 at 3:25 PM, Ojan Vafai o...@chromium.org wrote: I believe the chromium port always uses 2.6 though, no? On Fri, Nov 4, 2011 at 3:23 PM, Nico Weber tha...@chromium.org wrote: The chromium port still has a bot that runs tests (but doesn't build) on 10.5. Nico On Fri, Nov 4, 2011 at 3:20 PM, Eric Seidel e...@webkit.org wrote: Now that Apple has removed the Leopard build bot (and presumably stopped supporting WebKit on Leopard), all webkit platforms I know of have Python 2.6 or higher. My plan is to remove all of our 2.5 supporting code in the next week, requiring Python 2.6 or later for WebKit. Let me know if this will be an issue for you. Thanks! -eric ___ webkit-dev mailing list webkit-dev@lists.webkit.org http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev ___ webkit-dev mailing list webkit-dev@lists.webkit.org http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev ___ webkit-dev mailing list webkit-dev@lists.webkit.org http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev ___ webkit-dev mailing list webkit-dev@lists.webkit.org http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev ___ webkit-dev mailing list webkit-dev@lists.webkit.org http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev
Re: [webkit-dev] Supporting w3c ref tests and changing our convention
I don't see any need for manifest files. Needing to maintain information about a test somewhere other than in the test itself or in the expected result file is a significant maintenance burden. We should avoid it if we can. On Fri, Nov 4, 2011 at 2:00 PM, Ryosuke Niwa rn...@webkit.org wrote: On Fri, Nov 4, 2011 at 1:47 PM, Dirk Pranke dpra...@chromium.org wrote: It's unclear how much of a perf impact there would be but that's easy enough to determine - I would expect it to be minimal compared to the time of actually rendering a page. Since I expect w3c to end up having hundreds of thousands of tests, I see any performance implication to be a serious threat. There is no inherent performance problem that I can see. We just need to structure things in a way that avoids performance problems. This only requires that we ensure that all reference files are either themselves tests or have ref in the name and/or path. Even if the W3C and/or Mozilla test suites don't end up enforcing this requirement we can check in a script to Tools/Scripts that restructures the tests appropriately before we commit them to our tree. The performance benefits of the manifest files could be addressed by not walking the tree before running the tests. We only do that now because that was the expedient way to write the code. If walking the tree turns out to be a performance problem down the road, we can run the tests as we walk the tree. If that's still too slow, we can generate a transient manifest file that goes in your output directory. Separately, if we are throwing around numbers in the range of 100K for tests to run, we should consider when we actually want to run them - i.e., what will the cycle time be if we run them on every change, etc.? But that can be dealt with when we get there. We need separate bots in the long term for sure. I don't see what's special about reftests that we'd run them on a separate bot. We might decide to shard test running across different bots in some way, but sharding by test type seems unhelpful. ___ webkit-dev mailing list webkit-dev@lists.webkit.org http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev
Re: [webkit-dev] Removing Support for Python 2.5
I misremembered. Looking at depot_tools, it seems Chromium only does this on Windows. Looks like we might need to upgrade the Chromium bots to use 2.6. Python 2.5 is super old at this point. Adam On Fri, Nov 4, 2011 at 4:01 PM, Tony Chang t...@chromium.org wrote: Are you sure? This output has references to System/Library/Frameworks/Python.framework/Versions/2.5. I also thought that's why NRWT was slow on the Leopard bots: python 2.5 doesn't have the multiprocess module. http://build.chromium.org/p/chromium.webkit/builders/Webkit%20Mac10.5%20%28dbg%29%281%29/builds/4090/steps/webkit_tests/logs/stdio It may just be a bug that these bots aren't using python2.6. On Fri, Nov 4, 2011 at 3:40 PM, Adam Barth aba...@webkit.org wrote: Yes, Chromium versions its Python independently from the OS. Adam On Fri, Nov 4, 2011 at 3:25 PM, Ojan Vafai o...@chromium.org wrote: I believe the chromium port always uses 2.6 though, no? On Fri, Nov 4, 2011 at 3:23 PM, Nico Weber tha...@chromium.org wrote: The chromium port still has a bot that runs tests (but doesn't build) on 10.5. Nico On Fri, Nov 4, 2011 at 3:20 PM, Eric Seidel e...@webkit.org wrote: Now that Apple has removed the Leopard build bot (and presumably stopped supporting WebKit on Leopard), all webkit platforms I know of have Python 2.6 or higher. My plan is to remove all of our 2.5 supporting code in the next week, requiring Python 2.6 or later for WebKit. Let me know if this will be an issue for you. Thanks! -eric ___ webkit-dev mailing list webkit-dev@lists.webkit.org http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev ___ webkit-dev mailing list webkit-dev@lists.webkit.org http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev ___ webkit-dev mailing list webkit-dev@lists.webkit.org http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev ___ webkit-dev mailing list webkit-dev@lists.webkit.org http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev ___ webkit-dev mailing list webkit-dev@lists.webkit.org http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev
Re: [webkit-dev] Removing Support for Python 2.5
For those wishing to follow along at home (or just to spectate at the epic-hack that was python 2.5 support), the bug is https://bugs.webkit.org/show_bug.cgi?id=71593. On Fri, Nov 4, 2011 at 4:03 PM, Adam Barth aba...@webkit.org wrote: I misremembered. Looking at depot_tools, it seems Chromium only does this on Windows. Looks like we might need to upgrade the Chromium bots to use 2.6. Python 2.5 is super old at this point. Adam On Fri, Nov 4, 2011 at 4:01 PM, Tony Chang t...@chromium.org wrote: Are you sure? This output has references to System/Library/Frameworks/Python.framework/Versions/2.5. I also thought that's why NRWT was slow on the Leopard bots: python 2.5 doesn't have the multiprocess module. http://build.chromium.org/p/chromium.webkit/builders/Webkit%20Mac10.5%20%28dbg%29%281%29/builds/4090/steps/webkit_tests/logs/stdio It may just be a bug that these bots aren't using python2.6. On Fri, Nov 4, 2011 at 3:40 PM, Adam Barth aba...@webkit.org wrote: Yes, Chromium versions its Python independently from the OS. Adam On Fri, Nov 4, 2011 at 3:25 PM, Ojan Vafai o...@chromium.org wrote: I believe the chromium port always uses 2.6 though, no? On Fri, Nov 4, 2011 at 3:23 PM, Nico Weber tha...@chromium.org wrote: The chromium port still has a bot that runs tests (but doesn't build) on 10.5. Nico On Fri, Nov 4, 2011 at 3:20 PM, Eric Seidel e...@webkit.org wrote: Now that Apple has removed the Leopard build bot (and presumably stopped supporting WebKit on Leopard), all webkit platforms I know of have Python 2.6 or higher. My plan is to remove all of our 2.5 supporting code in the next week, requiring Python 2.6 or later for WebKit. Let me know if this will be an issue for you. Thanks! -eric ___ webkit-dev mailing list webkit-dev@lists.webkit.org http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev ___ webkit-dev mailing list webkit-dev@lists.webkit.org http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev ___ webkit-dev mailing list webkit-dev@lists.webkit.org http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev ___ webkit-dev mailing list webkit-dev@lists.webkit.org http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev ___ webkit-dev mailing list webkit-dev@lists.webkit.org http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev ___ webkit-dev mailing list webkit-dev@lists.webkit.org http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev
Re: [webkit-dev] Removing Support for Python 2.5
Tony: I would recommend upgrading to at least 2.7 on those machines. http://www.python.org/download/releases/2.7.2/ I would love to switch us to require 2.7 but such would currently too much of a burden on SnowLeopard-based developers. -eric On Fri, Nov 4, 2011 at 4:03 PM, Adam Barth aba...@webkit.org wrote: I misremembered. Looking at depot_tools, it seems Chromium only does this on Windows. Looks like we might need to upgrade the Chromium bots to use 2.6. Python 2.5 is super old at this point. Adam On Fri, Nov 4, 2011 at 4:01 PM, Tony Chang t...@chromium.org wrote: Are you sure? This output has references to System/Library/Frameworks/Python.framework/Versions/2.5. I also thought that's why NRWT was slow on the Leopard bots: python 2.5 doesn't have the multiprocess module. http://build.chromium.org/p/chromium.webkit/builders/Webkit%20Mac10.5%20%28dbg%29%281%29/builds/4090/steps/webkit_tests/logs/stdio It may just be a bug that these bots aren't using python2.6. On Fri, Nov 4, 2011 at 3:40 PM, Adam Barth aba...@webkit.org wrote: Yes, Chromium versions its Python independently from the OS. Adam On Fri, Nov 4, 2011 at 3:25 PM, Ojan Vafai o...@chromium.org wrote: I believe the chromium port always uses 2.6 though, no? On Fri, Nov 4, 2011 at 3:23 PM, Nico Weber tha...@chromium.org wrote: The chromium port still has a bot that runs tests (but doesn't build) on 10.5. Nico On Fri, Nov 4, 2011 at 3:20 PM, Eric Seidel e...@webkit.org wrote: Now that Apple has removed the Leopard build bot (and presumably stopped supporting WebKit on Leopard), all webkit platforms I know of have Python 2.6 or higher. My plan is to remove all of our 2.5 supporting code in the next week, requiring Python 2.6 or later for WebKit. Let me know if this will be an issue for you. Thanks! -eric ___ webkit-dev mailing list webkit-dev@lists.webkit.org http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev ___ webkit-dev mailing list webkit-dev@lists.webkit.org http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev ___ webkit-dev mailing list webkit-dev@lists.webkit.org http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev ___ webkit-dev mailing list webkit-dev@lists.webkit.org http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev ___ webkit-dev mailing list webkit-dev@lists.webkit.org http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev ___ webkit-dev mailing list webkit-dev@lists.webkit.org http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev
Re: [webkit-dev] Putting layout tests on a separate repository (was Supporting w3c ref tests and changing our convention)
On Fri, Nov 4, 2011 at 3:21 PM, Alan Stearns stea...@adobe.com wrote: Once we figure out how to support imported reftests, we should be encouraging people to use reftests internally (even for tests we have no intention of pushing to the W3C) instead of dumprendertree or pixel tests (where possible - I assume reftests won't always work for visual tests). Actually, we should be encouraging people to use reftests now, since every port but two supports them, and we should be moving the last two over ASAP. -- Dirk ___ webkit-dev mailing list webkit-dev@lists.webkit.org http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev
Re: [webkit-dev] Supporting w3c ref tests and changing our convention
On Fri, Nov 4, 2011 at 4:01 PM, Ojan Vafai o...@chromium.org wrote: I don't see any need for manifest files. W3C's build step auto-generates them. On Fri, Nov 4, 2011 at 2:00 PM, Ryosuke Niwa rn...@webkit.org wrote: On Fri, Nov 4, 2011 at 1:47 PM, Dirk Pranke dpra...@chromium.org wrote: It's unclear how much of a perf impact there would be but that's easy enough to determine - I would expect it to be minimal compared to the time of actually rendering a page. Since I expect w3c to end up having hundreds of thousands of tests, I see any performance implication to be a serious threat. There is no inherent performance problem that I can see. We just need to structure things in a way that avoids performance problems. But we don't need to structure them. They're organized in W3C's repo, and we're just going to import them in some directory; e.g. LayoutTests/w3c/. If we're adding new tests that we intend to contribute back to W3C, then those tests should live outside of this directory. The preferred approach, however, is to add tests to W3C repo first, then import them back to WebKit. This only requires that we ensure that all reference files are either themselves tests or have ref in the name and/or path. Even if the W3C and/or Mozilla test suites don't end up enforcing this requirement we can check in a script to Tools/Scripts that restructures the tests appropriately before we commit them to our tree. That, to me, is a problem. We try not to modify external tests or restructure them when we import them. Renaming files will also make it hard to figure out the correspondence between tests in WebKit's repo and W3C's repo. It's a significant cognitive stress if one is to contribute tests back to W3C's test suite. The performance benefits of the manifest files could be addressed by not walking the tree before running the tests. We only do that now because that was the expedient way to write the code. If walking the tree turns out to be a performance problem down the road, we can run the tests as we walk the tree. If that's still too slow, we can generate a transient manifest file that goes in your output directory. Generating a transient manifest file may make sense for WebKit tests but not for W3C tests although I don't see why generating a transient manifest file should improve the performance. Also, I don't see why we should be generating manifest files on demand when W3C test suite's build step automatically does that for us. We can build check tests into the repo once and we'll have manifest files. Since we shouldn't be modifying those tests anyway, I don't see why we need to generate manifests on demand. I don't see what's special about reftests that we'd run them on a separate bot. We might decide to shard test running across different bots in some way, but sharding by test type seems unhelpful. I'm not talking about ref-tests. I'm talking about tests imported from the W3C test suite. - Ryosuke ___ webkit-dev mailing list webkit-dev@lists.webkit.org http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev
Re: [webkit-dev] Supporting w3c ref tests and changing our convention
On Fri, Nov 4, 2011 at 6:31 PM, Ojan Vafai o...@chromium.org wrote: On Fri, Nov 4, 2011 at 5:52 PM, Ryosuke Niwa rn...@webkit.org wrote: W3C's build step auto-generates them. I thought you were arguing that we should use manifest files for all reftests because having multiple ways to do things is confusing. I am, but I'm particularly concerned about W3C tests. It'll be nice if we had exactly one way of running / writing ref tests. I think we can easily automate the process of generating manifest files. The work flow will be as follows: 1. Write new ref tests using link element 2. Run some tool (maybe we can teach run-webkit-tests to do it automatically) 3. Upload patch with auto-regenerated manifest file But we don't need to structure them. They're organized in W3C's repo, and we're just going to import them in some directory; e.g. LayoutTests/w3c/. I'm reluctantly OK with doing this exclusively for test suites we import that come with a manifest file, since the maintenance cost is lower. Although, I'd prefer if we limited the number of ways you could mark a test as a reftest. Yeah, I won't be particularly happy about having two ways to write ref tests. If we're adding new tests that we intend to contribute back to W3C, then those tests should live outside of this directory. The preferred approach, however, is to add tests to W3C repo first, then import them back to WebKit. I don't think this is realistic. For example, for the new flexbox tests, we're writing them against WebKit's code and they're changing as we go. In addition, all the properties are webkit prefixed. We're not writing test suites in isolation. We're writing them as we implement the feature or fix the appropriate bugs. Right, but then you'll need to unprefix those tests and reformat them in order to upstream the tests anyway. I don't see what's special about reftests that we'd run them on a separate bot. We might decide to shard test running across different bots in some way, but sharding by test type seems unhelpful. I'm not talking about ref-tests. I'm talking about tests imported from the W3C test suite. Even then, I don't see any benefit to not running them as part of our main test suite. It all depends on the number of tests. CSS 2.1 test suite appear to contain 10,000+ tests (there are 10,000+ .xht files). If we're adding CSS3, and other test suites from W3C, it can easily be in the order of 100,000s given that W3C is trying to increase the number of tests significantly in the coming years. Given we spend 10+ minutes running 26,000+ layout tests on our bots, adding 100,000+ tests will result in quadrupling our bot cycle time (to 40-50+ minutes; or to 3-4 hours for any bots that run pixel tests). It's definitely nice if we could run them all on the main waterfall with our regression tests, but slowed cycle time may significantly hinder our ability to catch new test failures on time. - Ryosuke ___ webkit-dev mailing list webkit-dev@lists.webkit.org http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev
Re: [webkit-dev] Supporting w3c ref tests and changing our convention
On Fri, Nov 4, 2011 at 7:07 PM, Ojan Vafai o...@chromium.org wrote: On Fri, Nov 4, 2011 at 7:03 PM, Ryosuke Niwa rn...@webkit.org wrote: I am, but I'm particularly concerned about W3C tests. It'll be nice if we had exactly one way of running / writing ref tests. I think we can easily automate the process of generating manifest files. The work flow will be as follows: 1. Write new ref tests using link element 2. Run some tool (maybe we can teach run-webkit-tests to do it automatically) 3. Upload patch with auto-regenerated manifest file It's mainly step 2 that I have a problem with, although I also don't like that the test is not self-contained. Generating manifest file when we add a test is much more efficient than generating it every time we run tests because we tend to do the latter much more often than the former. If we're adding new tests that we intend to contribute back to W3C, then those tests should live outside of this directory. The preferred approach, however, is to add tests to W3C repo first, then import them back to WebKit. I don't think this is realistic. For example, for the new flexbox tests, we're writing them against WebKit's code and they're changing as we go. In addition, all the properties are webkit prefixed. We're not writing test suites in isolation. We're writing them as we implement the feature or fix the appropriate bugs. Right, but then you'll need to unprefix those tests and reformat them in order to upstream the tests anyway. Yes, but the point is that it doesn't work to write the tests in the W3C repo first because what goes in the W3C repo doesn't work for us until we unprefix our implementation. Yeah, so those tests should be kept separate from W3C test suites. - Ryosuke ___ webkit-dev mailing list webkit-dev@lists.webkit.org http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev
Re: [webkit-dev] Supporting w3c ref tests and changing our convention
On Fri, Nov 4, 2011 at 7:16 PM, Ryosuke Niwa rn...@webkit.org wrote: On Fri, Nov 4, 2011 at 7:07 PM, Ojan Vafai o...@chromium.org wrote: On Fri, Nov 4, 2011 at 7:03 PM, Ryosuke Niwa rn...@webkit.org wrote: I am, but I'm particularly concerned about W3C tests. It'll be nice if we had exactly one way of running / writing ref tests. I think we can easily automate the process of generating manifest files. The work flow will be as follows: 1. Write new ref tests using link element 2. Run some tool (maybe we can teach run-webkit-tests to do it automatically) 3. Upload patch with auto-regenerated manifest file It's mainly step 2 that I have a problem with, although I also don't like that the test is not self-contained. Generating manifest file when we add a test is much more efficient than generating it every time we run tests because we tend to do the latter much more often than the former. I think we're at an empass here. I don't see that further technical arguments will sway either of us. I do, however, expect that the vast majority of webkit developers would prefer to avoid a manifest file given the way the project has been structured up to now. ___ webkit-dev mailing list webkit-dev@lists.webkit.org http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev
