Re: [webkit-dev] Is the wxWidgets port maintained?
On Tuesday, February 12, 2013 03:07:33 AM Dirk Pranke wrote: On Mon, Feb 11, 2013 at 7:44 PM, Benjamin Poulain benja...@webkit.org wrote: I am sorry, I should have given more context. There is visibly a growing discontent in the community about the cost imposed from small ports. Just two weeks ago, there were 2 threads discussing the cost of peripheral ports. I am convinced a part of this is technical. The project has not changed its policies while the number of ports was growing. While duplicated code and interfaces was okay when there were only 3 ports, it has become a pain when we have 7+ ports to updates ... In his email WebKit Wishes, Eric said It can’t be the job of the core maintainers to care about all the peripheral ports which contribute very little core code. I think it was an entirely reasonable question to ask if the wx port was being maintained, but I'm surprised by how this thread has evolved. There is a lot of discussion going on about the cost of so many ports, but not much about the benefits. Indeed. For example the small ports have successfully attracted people in the past to join the project. Those people built skills and gained visibility in in trunk, certainly helping them to get hired later by other bigger companies to continue to work on WebKit. Speaking personally, even before I joined Google, I was drawn to WebKit partially because it was used on such a wide variety of projects and in so many different ways. I was fortunate to be able to get a job that allows me to contribute to it, and I have found the work that I've done to help maintain the peripheral ports, while not pain free, quite rewarding (although I would be quite happy if it was less costly, of course). My point is that I think that lots of ports are part of what makes WebKit the goodness it is. Maybe I'm alone here, or at best part of a minority, but I wanted us to not lose sight of this idea. It's indeed an expressed goal of the project to be portable and to provide infrastructure. Also quoting from the project goals: In addition, we strive to create a courteous, welcoming environment that feels approachable to newcomers. Simon ___ webkit-dev mailing list webkit-dev@lists.webkit.org https://lists.webkit.org/mailman/listinfo/webkit-dev
Re: [webkit-dev] Is the wxWidgets port maintained?
On Mon, Feb 11, 2013 at 7:44 PM, Benjamin Poulain benja...@webkit.org wrote: I am sorry, I should have given more context. There is visibly a growing discontent in the community about the cost imposed from small ports. Just two weeks ago, there were 2 threads discussing the cost of peripheral ports. I am convinced a part of this is technical. The project has not changed its policies while the number of ports was growing. While duplicated code and interfaces was okay when there were only 3 ports, it has become a pain when we have 7+ ports to updates ... In his email WebKit Wishes, Eric said It can’t be the job of the core maintainers to care about all the peripheral ports which contribute very little core code. I think it was an entirely reasonable question to ask if the wx port was being maintained, but I'm surprised by how this thread has evolved. There is a lot of discussion going on about the cost of so many ports, but not much about the benefits. Speaking personally, even before I joined Google, I was drawn to WebKit partially because it was used on such a wide variety of projects and in so many different ways. I was fortunate to be able to get a job that allows me to contribute to it, and I have found the work that I've done to help maintain the peripheral ports, while not pain free, quite rewarding (although I would be quite happy if it was less costly, of course). My point is that I think that lots of ports are part of what makes WebKit the goodness it is. Maybe I'm alone here, or at best part of a minority, but I wanted us to not lose sight of this idea. -- Dirk ___ webkit-dev mailing list webkit-dev@lists.webkit.org https://lists.webkit.org/mailman/listinfo/webkit-dev
Re: [webkit-dev] Is the wxWidgets port maintained?
My point is that I think that lots of ports are part of what makes WebKit the goodness it is. Maybe I'm alone here, or at best part of a minority, but I wanted us to not lose sight of this idea. I totally agree. But I also agree with those, who thinks the different ports should not overburden the project. I would not mind to set up conditions for being part of the trunk. That could be useful for both adding and removing ports. There was also an unanswered question: is there anyone here who had difficulties because of the wxWidgets port? Or the only reason is that it has only one active maintainer, who has not committed anything for 6 months? Regards, Zoltan ___ webkit-dev mailing list webkit-dev@lists.webkit.org https://lists.webkit.org/mailman/listinfo/webkit-dev
Re: [webkit-dev] Is the wxWidgets port maintained?
On Feb 11, 2013, at 11:44 PM, Eric Seidel wrote: This entire email thread makes me sad. I'm sad because we did this to ourselves (our porting systems/policies are lacking), we did this very abruptly (the last port we kicked out of webkit.org took a year?), and we don't have any clear policy for this sort of thing (adding/removing ports in webkit.org). I think this, and many recent threads on the subject, are symptoms of the same problem: We are at a point in WebKit (and have been for a long time) where we are focused less on the Web Platform (via WebKit) being everywhere, and more on that Web Platform being awesome. Maybe it's time we clarified this in our goals doc? Our (my!) previous platform-abstraction mistakes have also left us in an awkward position where it is difficult for us to support 7+ ports with separate build systems, and separate sets of constraints (however minor) on WebCore. I think it is the right decision to de-couple all ports from the core code, but doing that is going to take us a very long time. I think that moving closer to one build system will help, but won't help those trying to advance WebCore from having to deal with 8 different port APIs, 5 different networking libraries, 2 different XML parser, 4 different GraphicsContext implementations, and countless other configuration options! As a port maintainer, I totally agree. I've been talking about how I switched to waf for the benefits it offered the wx port, but it is also true that part of my reason for going that route was to see if I could find a way to get out of the business of needing others to update my port's build system and code. It's a very gracious policy, but I don't like the feeling that just by being in trunk, I'm putting a fair amount of the burden on others to update my build system, etc. for me. The build system choice was my decision, and accordingly, it should also be my responsibility to maintain. I guess the point of this all is that I'm sorry. I'm sorry to Kevin, and all the other ports that I've helped integrate with WebKit, that I and others didn't sit down years ago and design better, more maintainable porting systems for WebKit. As someone whose worked on cross-platform software for over a decade now, I can say that WebKit has, by and large, done a very good job of managing the complexity inherent in supporting multiple ports. You do it far better than wx does in some respects, to be honest. It is not perfect, but no one can get everything right from day one, and overall you did get a lot of important things right. The problems the project faces right now are largely a result of its successes, and those are the sorts of problems that aren't so bad to have. :) I think if we as a community are actively interested in maintaining as many ports as we have (and welcoming new ones) we need to come up with better ways to do so. And clearer policies for what it means to be a port in WebKit. In the specific case of Wx, I am reluctantly agreed that code with only one(?) maintainer is pretty close to dead and thus per WebKit's long-standing dead/commented-out code policies should be removed. :( Kevin has been with us a long time, and asking him to leave in this manner is saddening. I actually do agree with most of the points raised, and agree wx being removed is probably the most sensible approach. I think, as you said, it was more the manner in which things were done that was disheartening. Of course saying Wx is dead code, begs the question as to what is needed for a port to be considered live in WebKit.org? With our current porting architecture, I would argue that at least 3 full time people are probably needed, and that this should be considered before accepting new ports. Yes, I think the rules can probably change these days. Back in the days when there was only SVN, not being on trunk was a very painful experience because branch merging on such a large tree was so problematic, so it really was necessary for the threshold for inclusion to be lower. Today though, with git, it is a lot more feasible for ports to be a layer 'above' the core code. If I were starting the project today, that is probably how I would do it. I'm not in any rush to see Wx removed, and I agree it makes sense to let Kevin write the patches as much as he's willing. I think certainly having to wake up one day to see that one of your Open Source projects is kicking you out, is pretty jaring. :( I hope we can do this better next time, and maybe we should have a separate discussion about what it means to be a supported port in WebKit.org and what new/existing ports need to do to meet that bar. As for the future of WebKit on Wx: I don't know enough about WebKit2 to know if it has a more easily portable/maintainable architecture. For the port I work on (chromium), others haven't really ported in WebKit.org, but rather port at a higher
Re: [webkit-dev] Is the wxWidgets port maintained?
Hi, On Mon, Feb 11, 2013 at 11:44 PM, Eric Seidel e...@webkit.org wrote: This entire email thread makes me sad. I'm sad because we did this to ourselves (our porting systems/policies are lacking), we did this very abruptly (the last port we kicked out of webkit.org took a year?), and we don't have any clear policy for this sort of thing (adding/removing ports in webkit.org). I agree that this is sad. Wouldn't it be better to first agree on policies for adding/removing ports from WebKit BEFORE giving Kevin his walking papers? Why don't we first decide that aspect of things, before giving small teams the bum's rush? I think that moving closer to one build system will help, but won't help those trying to advance WebCore from having to deal with 8 different port APIs, 5 different networking libraries, 2 different XML parser, 4 different GraphicsContext implementations, and countless other configuration options! Not to mention two JavaScript engines! Of course saying Wx is dead code, begs the question as to what is needed for a port to be considered live in WebKit.org? With our current porting architecture, I would argue that at least 3 full time people are probably needed, and that this should be considered before accepting new ports. This is tantamount to saying that the only ports allowed in WebKit are corporate-sponsored efforts. Is that what we want this project to be? -Brent ___ webkit-dev mailing list webkit-dev@lists.webkit.org https://lists.webkit.org/mailman/listinfo/webkit-dev
Re: [webkit-dev] Is the wxWidgets port maintained?
On Tue, Feb 12, 2013 at 9:29 AM, Kevin Ollivier kev...@theolliviers.comwrote: On Feb 11, 2013, at 11:44 PM, Eric Seidel wrote: I think if we as a community are actively interested in maintaining as many ports as we have (and welcoming new ones) we need to come up with better ways to do so. And clearer policies for what it means to be a port in WebKit. In the specific case of Wx, I am reluctantly agreed that code with only one(?) maintainer is pretty close to dead and thus per WebKit's long-standing dead/commented-out code policies should be removed. :( Kevin has been with us a long time, and asking him to leave in this manner is saddening. I actually do agree with most of the points raised, and agree wx being removed is probably the most sensible approach. I think, as you said, it was more the manner in which things were done that was disheartening. Sorry, I didn't mean to rush or harass you in any way. Of course saying Wx is dead code, begs the question as to what is needed for a port to be considered live in WebKit.org? With our current porting architecture, I would argue that at least 3 full time people are probably needed, and that this should be considered before accepting new ports. Yes, I think the rules can probably change these days. Back in the days when there was only SVN, not being on trunk was a very painful experience because branch merging on such a large tree was so problematic, so it really was necessary for the threshold for inclusion to be lower. Today though, with git, it is a lot more feasible for ports to be a layer 'above' the core code. If I were starting the project today, that is probably how I would do it. Right. The way I see it, as arrogantly as it sounds, we're not really kicking Wx port out. It's just that it won't be on svn.webkit.org anymore. There is no reason why github repository should be a second-class citizen to the community. We can make a wiki page or other form of documentation on how to maintain a port on github, and let all perpetual ports live there instead. Port maintainers can merge sync up to trunk on their own pace then. I agree with you that we should include a port on svn.webkit.org when and only when that it gets popular enough to have EWS and build.webkit.org bots since setting up a buildbot via github will be extremely tricky. (Also see https://lists.webkit.org/pipermail/webkit-dev/2011-September/018075.html) In other words, I'm suggesting to create a two-tier system where popular ports with lots of contributors live on svn.webkit.org with build slaves and EWS bots, and those ports that do not have luxury of keeping up with trunk changes will live on github. - R. Niwa ___ webkit-dev mailing list webkit-dev@lists.webkit.org https://lists.webkit.org/mailman/listinfo/webkit-dev
Re: [webkit-dev] Is the wxWidgets port maintained?
Hi Benjamin, On Feb 10, 2013, at 7:23 PM, Benjamin Poulain wrote: Hi again, While looking into DRT changes, I notice there is zero activity on the wxWidgets implementation. Looking at WebCore/WebKit, excluding general changes, the last commit for wxWidgets is 6 months old: http://trac.webkit.org/changeset/126457 So...is the port still active? Maybe it should live outside webkit.org like the Haiku port? The project is not dead per-se, but it's true that it's not very active these days. Recently, I haven't had a lot of time to devote to the port. The other big factor has been that I've been the only active contributor to the port pretty much since its inception, which has made patch submission pretty difficult due to the challenge of finding a reviewer with knowledge of the wx code. As a result, the wx activity on trunk will look pretty sparse since I'm making most of my changes on the wx port's git repo on gitorious.org rather than submitting patches for trunk. When it comes to trunk, I basically just periodically submit build fixes for any build breakages in order to merge and update my git mirror. I actually have recently been working on a patch to get trunk building again, though. The path of least resistance for me would simply be to ask people to ignore the wx port in their changes, but leave it in the tree. git is much better about these things, but I still remember what a pain it was to maintain the entire port in an svn branch back in the day, and I'd rather not re-try that experiment unless I really have to. If people feel strongly that I should approach things another way, though, I'm certainly open to ideas. Thanks, Kevin Benjamin ___ webkit-dev mailing list webkit-dev@lists.webkit.org https://lists.webkit.org/mailman/listinfo/webkit-dev ___ webkit-dev mailing list webkit-dev@lists.webkit.org https://lists.webkit.org/mailman/listinfo/webkit-dev
Re: [webkit-dev] Is the wxWidgets port maintained?
On Mon, Feb 11, 2013 at 11:27 AM, Kevin Ollivier kev...@theolliviers.comwrote: I actually have recently been working on a patch to get trunk building again, though. The path of least resistance for me would simply be to ask people to ignore the wx port in their changes, but leave it in the tree. git is much better about these things, but I still remember what a pain it was to maintain the entire port in an svn branch back in the day, and I'd rather not re-try that experiment unless I really have to. If people feel strongly that I should approach things another way, though, I'm certainly open to ideas. Asking people to ignore wx is basically the same as having the code outside the tree. Having the code outside the tree would make this change effective instead of having one more rule to the project. Looking at the history of wx's WebKit and platform code, the vast majority of changes are from Apple/Google. I think it is unfair to have many people maintain your code while you develop the project outside of the tree. Given the data, I am in favor of removing the wx port from webkit.org and have it be a separate tree on Gitorious or GitHub. What are the main pain points of developing outside webkit.org for you? Is there anything that can be done in the structure to simplify such changes? Benjamin ___ webkit-dev mailing list webkit-dev@lists.webkit.org https://lists.webkit.org/mailman/listinfo/webkit-dev
Re: [webkit-dev] Is the wxWidgets port maintained?
On Mon, Feb 11, 2013 at 11:27 AM, Kevin Ollivier kev...@theolliviers.com wrote: The project is not dead per-se, but it's true that it's not very active these days. Recently, I haven't had a lot of time to devote to the port. The other big factor has been that I've been the only active contributor to the port pretty much since its inception, which has made patch submission pretty difficult due to the challenge of finding a reviewer with knowledge of the wx code. As a result, the wx activity on trunk will look pretty sparse since I'm making most of my changes on the wx port's git repo on gitorious.org rather than submitting patches for trunk. When it comes to trunk, I basically just periodically submit build fixes for any build breakages in order to merge and update my git mirror. Regardless of whether the Wx port stays in the tree, I think you should consider changing your build system. I believe the only port that uses waf is Wx, so using cmake or gyp will make your work a lot easier. --Martin ___ webkit-dev mailing list webkit-dev@lists.webkit.org https://lists.webkit.org/mailman/listinfo/webkit-dev
Re: [webkit-dev] Is the wxWidgets port maintained?
On Feb 11, 2013, at 12:27 PM, Benjamin Poulain wrote: On Mon, Feb 11, 2013 at 11:27 AM, Kevin Ollivier kev...@theolliviers.com wrote: I actually have recently been working on a patch to get trunk building again, though. The path of least resistance for me would simply be to ask people to ignore the wx port in their changes, but leave it in the tree. git is much better about these things, but I still remember what a pain it was to maintain the entire port in an svn branch back in the day, and I'd rather not re-try that experiment unless I really have to. If people feel strongly that I should approach things another way, though, I'm certainly open to ideas. Asking people to ignore wx is basically the same as having the code outside the tree. Having the code outside the tree would make this change effective instead of having one more rule to the project. Looking at the history of wx's WebKit and platform code, the vast majority of changes are from Apple/Google. I think it is unfair to have many people maintain your code while you develop the project outside of the tree. Given the data, I am in favor of removing the wx port from webkit.org and have it be a separate tree on Gitorious or GitHub. What are the main pain points of developing outside webkit.org for you? Is there anything that can be done in the structure to simplify such changes? My big concern is not losing history while moving the wx files out of the tree and into a git branch, and also at the same time not losing the ability to merge the rest of trunk safely. Though I haven't investigated the process yet, so I don't know how easy or hard it may be. Thanks, Kevin Benjamin ___ webkit-dev mailing list webkit-dev@lists.webkit.org https://lists.webkit.org/mailman/listinfo/webkit-dev
Re: [webkit-dev] Is the wxWidgets port maintained?
Hi Martin, On Feb 11, 2013, at 12:47 PM, Martin Robinson wrote: On Mon, Feb 11, 2013 at 11:27 AM, Kevin Ollivier kev...@theolliviers.com wrote: The project is not dead per-se, but it's true that it's not very active these days. Recently, I haven't had a lot of time to devote to the port. The other big factor has been that I've been the only active contributor to the port pretty much since its inception, which has made patch submission pretty difficult due to the challenge of finding a reviewer with knowledge of the wx code. As a result, the wx activity on trunk will look pretty sparse since I'm making most of my changes on the wx port's git repo on gitorious.org rather than submitting patches for trunk. When it comes to trunk, I basically just periodically submit build fixes for any build breakages in order to merge and update my git mirror. Regardless of whether the Wx port stays in the tree, I think you should consider changing your build system. I believe the only port that uses waf is Wx, so using cmake or gyp will make your work a lot easier. Actually, it will, if anything, increase the workload. Because I use waf, I am able to use Python to auto-generate the list of sources to build. In other words, I tell it to build all sources in a defined list of base source directories, along with all sources in baseDirectory/portName subdirs (where in this case I set portName to wx) and FileNameportName.cpp (e.g. FileNameWX.cpp if the port is wx), along with some additional similar rules for USE defines, like CF. Then if there are exceptions to these rules, I just filter them out of the results or explicitly add files when I need to, say if I need to compile a single Mac port source file. Since the WebKit tree is well-structured, this approach works quite well, with me having to define exceptions fairly rarely. The main issue I run into seems to be derived sources files that are no longer built / used but are still being generated. The performance hit of this is about 1 added second to my build, though on slower machines it might be a couple seconds. For me, it's negligible given the benefit I get from it. As an example, I'm still at the linking phase right now on Mac (dealing with the removal of the libWebKitLibrarySnowLeopard.a and a couple other things), but my patch so far to catch up with 6 months of changes in trunk is right now at 19KB, and about 30 lines or so of that is actually the build system changes. The number of exceptions I needed to add to the build over that time is 4, and I added about as many build directories to the list as well. I suspect if we were to accumulate the changes to cmake and gyp files over those 6 months, it will probably be well above 30-40 lines. Sure, I wouldn't have written all those lines myself, but sharing work with other developers is still more work for any individual developer than having the computer do the work for you. :) Everyone tells me how great cmake and gyp is, but I'm not sure any of them have taken any time out to actually investigate whether or not waf is as good as or better than their preferred tool. (It's not as good as auto-generating project files, but it's also in Python so integrating it with gyp would probably be a simple task.) I've often wanted to take some time out to get other ports building with it, as it probably would take a day or two at most, but I lack both the time and expertise in other ports to spend much time on that. Much of the code in there already is port-agnostic, with most wx-specific bits in a port_name == wx check, but of course there's the issue of the various port-specific build configurations and such for each port. Regards, Kevin --Martin ___ webkit-dev mailing list webkit-dev@lists.webkit.org https://lists.webkit.org/mailman/listinfo/webkit-dev
Re: [webkit-dev] Is the wxWidgets port maintained?
On Mon, Feb 11, 2013 at 1:34 PM, Kevin Ollivier kev...@theolliviers.comwrote: On Feb 11, 2013, at 12:27 PM, Benjamin Poulain wrote: On Mon, Feb 11, 2013 at 11:27 AM, Kevin Ollivier kev...@theolliviers.comwrote: I actually have recently been working on a patch to get trunk building again, though. The path of least resistance for me would simply be to ask people to ignore the wx port in their changes, but leave it in the tree. git is much better about these things, but I still remember what a pain it was to maintain the entire port in an svn branch back in the day, and I'd rather not re-try that experiment unless I really have to. If people feel strongly that I should approach things another way, though, I'm certainly open to ideas. Asking people to ignore wx is basically the same as having the code outside the tree. Having the code outside the tree would make this change effective instead of having one more rule to the project. Looking at the history of wx's WebKit and platform code, the vast majority of changes are from Apple/Google. I think it is unfair to have many people maintain your code while you develop the project outside of the tree. Given the data, I am in favor of removing the wx port from webkit.org and have it be a separate tree on Gitorious or GitHub. What are the main pain points of developing outside webkit.org for you? Is there anything that can be done in the structure to simplify such changes? My big concern is not losing history while moving the wx files out of the tree and into a git branch, and also at the same time not losing the ability to merge the rest of trunk safely. Though I haven't investigated the process yet, so I don't know how easy or hard it may be. I kindly ask you to let us remove it from trunk in that case. Given that we have 400+ contributors, having a port that's only maintained by a single person on trunk imposes a significant maintenance burden on others even if you said that people can break Wx port because not everyone reads this thread and wouldn't necessarily know your intent. - R. Niwa ___ webkit-dev mailing list webkit-dev@lists.webkit.org https://lists.webkit.org/mailman/listinfo/webkit-dev
Re: [webkit-dev] Is the wxWidgets port maintained?
I'll add that now that https://github.com/WebKit/webkit uses the same hashing as git.webkit.org, it'll be trivial for folks wanting to have their own ports to fork on github and maintain svn history. Maybe we should even encourage that for new ports: initially forking WebKit on github to create their port and then wait until it gets enough tractions to land it on trunk. Meanwhile, are there anything we can do as the WebKit community to help maintaing a one-man port like Wx in the future? e.g. rebumping build systems, etc... - R. Niwa On Mon, Feb 11, 2013 at 1:51 PM, Ryosuke Niwa rn...@webkit.org wrote: On Mon, Feb 11, 2013 at 1:34 PM, Kevin Ollivier kev...@theolliviers.comwrote: On Feb 11, 2013, at 12:27 PM, Benjamin Poulain wrote: On Mon, Feb 11, 2013 at 11:27 AM, Kevin Ollivier kev...@theolliviers.com wrote: I actually have recently been working on a patch to get trunk building again, though. The path of least resistance for me would simply be to ask people to ignore the wx port in their changes, but leave it in the tree. git is much better about these things, but I still remember what a pain it was to maintain the entire port in an svn branch back in the day, and I'd rather not re-try that experiment unless I really have to. If people feel strongly that I should approach things another way, though, I'm certainly open to ideas. Asking people to ignore wx is basically the same as having the code outside the tree. Having the code outside the tree would make this change effective instead of having one more rule to the project. Looking at the history of wx's WebKit and platform code, the vast majority of changes are from Apple/Google. I think it is unfair to have many people maintain your code while you develop the project outside of the tree. Given the data, I am in favor of removing the wx port from webkit.organd have it be a separate tree on Gitorious or GitHub. What are the main pain points of developing outside webkit.org for you? Is there anything that can be done in the structure to simplify such changes? My big concern is not losing history while moving the wx files out of the tree and into a git branch, and also at the same time not losing the ability to merge the rest of trunk safely. Though I haven't investigated the process yet, so I don't know how easy or hard it may be. I kindly ask you to let us remove it from trunk in that case. Given that we have 400+ contributors, having a port that's only maintained by a single person on trunk imposes a significant maintenance burden on others even if you said that people can break Wx port because not everyone reads this thread and wouldn't necessarily know your intent. - R. Niwa ___ webkit-dev mailing list webkit-dev@lists.webkit.org https://lists.webkit.org/mailman/listinfo/webkit-dev
Re: [webkit-dev] Is the wxWidgets port maintained?
On Mon, Feb 11, 2013 at 1:41 PM, Kevin Ollivier kev...@theolliviers.com wrote: Actually, it will, if anything, increase the workload. Because I use waf, I am able to use Python to auto-generate the list of sources to build. In other words, I tell it to build all sources in a defined list of base source directories, along with all sources in baseDirectory/portName subdirs (where in this case I set portName to wx) and FileNameportName.cpp (e.g. FileNameWX.cpp if the port is wx), along with some additional similar rules for USE defines, like CF. Then if there are exceptions to these rules, I just filter them out of the results or explicitly add files when I need to, say if I need to compile a single Mac port source file. Since the WebKit tree is well-structured, this approach works quite well, with me having to define exceptions fairly rarely. The main issue I run into seems to be derived sources files that are no longer built / used but are still being generated. The performance hit of this is about 1 added second to my build, though on slower machine s it might be a couple seconds. For me, it's negligible given the benefit I get from it. If I understand correctly, gyp is also capable of this kind of wildcard inclusion and exclusion. This will be the tool that allows the gyp build to be shared among many ports with the same source lists. The situation we have now is that we have many build systems, so we probably need to think about discarding some awesome ones [1] for ones that are popular with our peers. If the Wx port were to move out of the tree, obviously this isn't a huge deal -- just a suggestion. 1. All things being equal, waf looks to be the best replacement for autotools for WebKitGTK+. I've heard it's faster than both scons and cmake and it supports 'make install' and 'make dist.' Sadly, not all things are equal at svn.webkit.org, so gyp or cmake are the best hope at the moment. --Martin ___ webkit-dev mailing list webkit-dev@lists.webkit.org https://lists.webkit.org/mailman/listinfo/webkit-dev
Re: [webkit-dev] Is the wxWidgets port maintained?
Hi Everyone, On Mon, Feb 11, 2013 at 1:51 PM, Ryosuke Niwa rn...@webkit.org wrote: Asking people to ignore wx is basically the same as having the code outside the tree. Having the code outside the tree would make this change effective instead of having one more rule to the project. My big concern is not losing history while moving the wx files out of the tree and into a git branch, and also at the same time not losing the ability to merge the rest of trunk safely. Though I haven't investigated the process yet, so I don't know how easy or hard it may be. I kindly ask you to let us remove it from trunk in that case. Given that we have 400+ contributors, having a port that's only maintained by a single person on trunk imposes a significant maintenance burden on others even if you said that people can break Wx port because not everyone reads this thread and wouldn't necessarily know your intent. I would be sad if the WebKit project started to exclude ports simply because they were (largely) maintained by a single person. I do agree that it's not fair to other ports if the primary developer is not working to keep the port building, but life happens and sometimes a lone developer cannot stay on top of the rapidly-changing source base. Having these ports in the main tree is a good thing, in my opinion, because it helps avoid fragmentation, and often the refactoring required to enable these ports provides architectural benefits to the project as a whole. -Brent ___ webkit-dev mailing list webkit-dev@lists.webkit.org https://lists.webkit.org/mailman/listinfo/webkit-dev
Re: [webkit-dev] Is the wxWidgets port maintained?
On Feb 11, 2013, at 1:34 PM, Kevin Ollivier kev...@theolliviers.com wrote: On Feb 11, 2013, at 12:27 PM, Benjamin Poulain wrote: On Mon, Feb 11, 2013 at 11:27 AM, Kevin Ollivier kev...@theolliviers.com wrote: I actually have recently been working on a patch to get trunk building again, though. The path of least resistance for me would simply be to ask people to ignore the wx port in their changes, but leave it in the tree. git is much better about these things, but I still remember what a pain it was to maintain the entire port in an svn branch back in the day, and I'd rather not re-try that experiment unless I really have to. If people feel strongly that I should approach things another way, though, I'm certainly open to ideas. Asking people to ignore wx is basically the same as having the code outside the tree. Having the code outside the tree would make this change effective instead of having one more rule to the project. Looking at the history of wx's WebKit and platform code, the vast majority of changes are from Apple/Google. I think it is unfair to have many people maintain your code while you develop the project outside of the tree. Given the data, I am in favor of removing the wx port from webkit.org and have it be a separate tree on Gitorious or GitHub. What are the main pain points of developing outside webkit.org for you? Is there anything that can be done in the structure to simplify such changes? My big concern is not losing history while moving the wx files out of the tree and into a git branch, and also at the same time not losing the ability to merge the rest of trunk safely. Though I haven't investigated the process yet, so I don't know how easy or hard it may be. If you create a github repository by forking from the canonical webkit github repository, you'll have history for all the wx files and a good way to merge from trunk at times that you find convenient. I think that's the motivation behind the suggestion. - Maciej ___ webkit-dev mailing list webkit-dev@lists.webkit.org https://lists.webkit.org/mailman/listinfo/webkit-dev
Re: [webkit-dev] Is the wxWidgets port maintained?
On Mon, Feb 11, 2013 at 2:13 PM, Brent Fulgham bfulg...@gmail.com wrote: On Mon, Feb 11, 2013 at 1:51 PM, Ryosuke Niwa rn...@webkit.org wrote: Asking people to ignore wx is basically the same as having the code outside the tree. Having the code outside the tree would make this change effective instead of having one more rule to the project. My big concern is not losing history while moving the wx files out of the tree and into a git branch, and also at the same time not losing the ability to merge the rest of trunk safely. Though I haven't investigated the process yet, so I don't know how easy or hard it may be. I kindly ask you to let us remove it from trunk in that case. Given that we have 400+ contributors, having a port that's only maintained by a single person on trunk imposes a significant maintenance burden on others even if you said that people can break Wx port because not everyone reads this thread and wouldn't necessarily know your intent. I would be sad if the WebKit project started to exclude ports simply because they were (largely) maintained by a single person. I do agree that it's not fair to other ports if the primary developer is not working to keep the port building, but life happens and sometimes a lone developer cannot stay on top of the rapidly-changing source base. Having these ports in the main tree is a good thing, in my opinion, because it helps avoid fragmentation, and often the refactoring required to enable these ports provides architectural benefits to the project as a whole. I'm not opposed to keeping those refactoring in trunk. As I said on https://lists.webkit.org/pipermail/webkit-dev/2013-January/023563.html, the problem is the ratio of contribution each port makes to the core code vs. the cost it imposes on the project. Having hundreds, if not thousands, of engineers sync Wx code in WebKit trunk whenever they run svn up or git fetch is a significant cost on its own, and we need a good justification for it. - R. Niwa ___ webkit-dev mailing list webkit-dev@lists.webkit.org https://lists.webkit.org/mailman/listinfo/webkit-dev
Re: [webkit-dev] Is the wxWidgets port maintained?
On Feb 11, 2013, at 2:13 PM, Brent Fulgham wrote: Hi Everyone, On Mon, Feb 11, 2013 at 1:51 PM, Ryosuke Niwa rn...@webkit.org wrote: Asking people to ignore wx is basically the same as having the code outside the tree. Having the code outside the tree would make this change effective instead of having one more rule to the project. My big concern is not losing history while moving the wx files out of the tree and into a git branch, and also at the same time not losing the ability to merge the rest of trunk safely. Though I haven't investigated the process yet, so I don't know how easy or hard it may be. I kindly ask you to let us remove it from trunk in that case. Given that we have 400+ contributors, having a port that's only maintained by a single person on trunk imposes a significant maintenance burden on others even if you said that people can break Wx port because not everyone reads this thread and wouldn't necessarily know your intent. I would be sad if the WebKit project started to exclude ports simply because they were (largely) maintained by a single person. I do agree that it's not fair to other ports if the primary developer is not working to keep the port building, but life happens and sometimes a lone developer cannot stay on top of the rapidly-changing source base. Having these ports in the main tree is a good thing, in my opinion, because it helps avoid fragmentation, and often the refactoring required to enable these ports provides architectural benefits to the project as a whole. FWIW, I think it'd be pretty simple to add a generic rule that if you want your port to benefit from shared maintenance, you must set up a build bot on the waterfall and/or an EWS bot. In fact, I personally think this ought to be a rule regardless of what happens to the wx port, as IMHO it's pretty much a pre-requisite for allowing people contributors on other ports to make changes to your code safely. Thanks, Kevin -Brent ___ webkit-dev mailing list webkit-dev@lists.webkit.org https://lists.webkit.org/mailman/listinfo/webkit-dev
Re: [webkit-dev] Is the wxWidgets port maintained?
On Mon, Feb 11, 2013 at 1:55 PM, Ryosuke Niwa rn...@webkit.org wrote: Meanwhile, are there anything we can do as the WebKit community to help maintaing a one-man port like Wx in the future? e.g. rebumping build systems, etc... I think the recent discussions about standardizing on something like gyp seems like a really good first step. That would greatly reduce the burden caused by various build systems. Many common cases that break builds now might not ever be a problem using something like gyp. -Brent ___ webkit-dev mailing list webkit-dev@lists.webkit.org https://lists.webkit.org/mailman/listinfo/webkit-dev
Re: [webkit-dev] Is the wxWidgets port maintained?
On Mon, Feb 11, 2013 at 1:41 PM, Kevin Ollivier kev...@theolliviers.com wrote: Hi Martin, On Feb 11, 2013, at 12:47 PM, Martin Robinson wrote: On Mon, Feb 11, 2013 at 11:27 AM, Kevin Ollivier kev...@theolliviers.com wrote: The project is not dead per-se, but it's true that it's not very active these days. Recently, I haven't had a lot of time to devote to the port. The other big factor has been that I've been the only active contributor to the port pretty much since its inception, which has made patch submission pretty difficult due to the challenge of finding a reviewer with knowledge of the wx code. As a result, the wx activity on trunk will look pretty sparse since I'm making most of my changes on the wx port's git repo on gitorious.org rather than submitting patches for trunk. When it comes to trunk, I basically just periodically submit build fixes for any build breakages in order to merge and update my git mirror. Regardless of whether the Wx port stays in the tree, I think you should consider changing your build system. I believe the only port that uses waf is Wx, so using cmake or gyp will make your work a lot easier. Actually, it will, if anything, increase the workload. Because I use waf, I am able to use Python to auto-generate the list of sources to build. In other words, I tell it to build all sources in a defined list of base source directories, along with all sources in baseDirectory/portName subdirs (where in this case I set portName to wx) and FileNameportName.cpp (e.g. FileNameWX.cpp if the port is wx), along with some additional similar rules for USE defines, like CF. Then if there are exceptions to these rules, I just filter them out of the results or explicitly add files when I need to, say if I need to compile a single Mac port source file. Since the WebKit tree is well-structured, this approach works quite well, with me having to define exceptions fairly rarely. The main issue I run into seems to be derived sources files that are no longer built / used but are still being generated. The performance hit of this is about 1 added second to my build, though on slower machine s it might be a couple seconds. For me, it's negligible given the benefit I get from it. As an example, I'm still at the linking phase right now on Mac (dealing with the removal of the libWebKitLibrarySnowLeopard.a and a couple other things), but my patch so far to catch up with 6 months of changes in trunk is right now at 19KB, and about 30 lines or so of that is actually the build system changes. The number of exceptions I needed to add to the build over that time is 4, and I added about as many build directories to the list as well. I suspect if we were to accumulate the changes to cmake and gyp files over those 6 months, it will probably be well above 30-40 lines. Sure, I wouldn't have written all those lines myself, but sharing work with other developers is still more work for any individual developer than having the computer do the work for you. :) Everyone tells me how great cmake and gyp is, but I'm not sure any of them have taken any time out to actually investigate whether or not waf is as good as or better than their preferred tool. (It's not as good as auto-generating project files, but it's also in Python so integrating it with gyp would probably be a simple task.) I've often wanted to take some time out to get other ports building with it, as it probably would take a day or two at most, but I lack both the time and expertise in other ports to spend much time on that. Much of the code in there already is port-agnostic, with most wx-specific bits in a port_name == wx check, but of course there's the issue of the various port-specific build configurations and such for each port. I have to admit, I thought waf was more like scons and less like cmake and gyp, i.e., I thought it was a portable build system rather than a meta-build system (and had never looked at it in detail accordingly). I am taking another look at it now :) -- Dirk ___ webkit-dev mailing list webkit-dev@lists.webkit.org https://lists.webkit.org/mailman/listinfo/webkit-dev
Re: [webkit-dev] Is the wxWidgets port maintained?
On Feb 11, 2013, at 2:01 PM, Martin Robinson wrote: On Mon, Feb 11, 2013 at 1:41 PM, Kevin Ollivier kev...@theolliviers.com wrote: Actually, it will, if anything, increase the workload. Because I use waf, I am able to use Python to auto-generate the list of sources to build. In other words, I tell it to build all sources in a defined list of base source directories, along with all sources in baseDirectory/portName subdirs (where in this case I set portName to wx) and FileNameportName.cpp (e.g. FileNameWX.cpp if the port is wx), along with some additional similar rules for USE defines, like CF. Then if there are exceptions to these rules, I just filter them out of the results or explicitly add files when I need to, say if I need to compile a single Mac port source file. Since the WebKit tree is well-structured, this approach works quite well, with me having to define exceptions fairly rarely. The main issue I run into seems to be derived sources files that are no longer built / used but are still being generated. The performance hit of this is about 1 added second to my build, though on slower machin es it might be a couple seconds. For me, it's negligible given the benefit I get from it. If I understand correctly, gyp is also capable of this kind of wildcard inclusion and exclusion. This will be the tool that allows the gyp build to be shared among many ports with the same source lists. Yes, but that is not what I see when I check, say, Source/WebCore/WebCore.gypi. Which, BTW, has had around 500 revisions in it over the past 6 months in comparison to the ~10 lines of changes to source files in my patch-in-progress. So while gyp itself might have that feature in it, for whatever reason, the WebKit projects do not utilize that feature right now, so in practice, a switch to gyp means a switch away from rule-based compilation. Plus, my overarching reason for switching away from a tool like gyp or bakefile was to get away from using domain-specific languages for build tools. In short, waf doesn't have a syntax, it has an API. With waf and Python, I don't have to worry about fitting my design goals into some project file format built perhaps by someone with a different set of needs than mine. If I need it, I just code it. On-demand automated dependency download is one example of this. Another thing on my TODO list, actually, is to integrate in a little tool I have started coding up called 'gattai', http://sf.net/projects/gattai, which aims to be able to pull and, if necessary, build the various deps a project needs in source or binary form. And yet another thing I wanted to play with actually was having it dynamically generate AllInOne.cpp files for various directories and profile how much of a speed change there is in the builds by doing so. I suspect that we could squeeze out some considerable gains that way. Anyway, those are just some examples. Just wanted to say that for me, the rule-based compilation is certainly one of the most used features of waf, but it's far from the only thing I chose it for. The situation we have now is that we have many build systems, so we probably need to think about discarding some awesome ones [1] for ones that are popular with our peers. If the Wx port were to move out of the tree, obviously this isn't a huge deal -- just a suggestion. 1. All things being equal, waf looks to be the best replacement for autotools for WebKitGTK+. I've heard it's faster than both scons and cmake and it supports 'make install' and 'make dist.' Sadly, not all things are equal at svn.webkit.org, so gyp or cmake are the best hope at the moment. Yes, it's light years faster than Scons, and I don't know about cmake but honestly, I have a hard time thinking my build could get much faster than waf makes it, aside from speeding up the compiler itself. Efficient use of multiple cores, and smart dependency tracking both help considerably. Anyway, I do understand your point about adoption, and it's the primary reason I don't really say much on the issue. I think waf has a lot of things going for it, some of which I feel are pretty well-suited to a project like WebKit, but if people already have their minds set on other tools, that's pretty much the end of the conversation. Thanks, Kevin --Martin ___ webkit-dev mailing list webkit-dev@lists.webkit.org https://lists.webkit.org/mailman/listinfo/webkit-dev
Re: [webkit-dev] Is the wxWidgets port maintained?
On Mon, Feb 11, 2013 at 3:26 PM, Kevin Ollivier kev...@theolliviers.com wrote: On Feb 11, 2013, at 2:01 PM, Martin Robinson wrote: On Mon, Feb 11, 2013 at 1:41 PM, Kevin Ollivier kev...@theolliviers.com wrote: Actually, it will, if anything, increase the workload. Because I use waf, I am able to use Python to auto-generate the list of sources to build. In other words, I tell it to build all sources in a defined list of base source directories, along with all sources in baseDirectory/portName subdirs (where in this case I set portName to wx) and FileNameportName.cpp (e.g. FileNameWX.cpp if the port is wx), along with some additional similar rules for USE defines, like CF. Then if there are exceptions to these rules, I just filter them out of the results or explicitly add files when I need to, say if I need to compile a single Mac port source file. Since the WebKit tree is well-structured, this approach works quite well, with me having to define exceptions fairly rarely. The main issue I run into seems to be derived sources files that are no longer built / used but are still being generated. The performance hit of this is about 1 added second to my build, though on slower machi n es it might be a couple seconds. For me, it's negligible given the benefit I get from it. If I understand correctly, gyp is also capable of this kind of wildcard inclusion and exclusion. This will be the tool that allows the gyp build to be shared among many ports with the same source lists. Yes, but that is not what I see when I check, say, Source/WebCore/WebCore.gypi. Which, BTW, has had around 500 revisions in it over the past 6 months in comparison to the ~10 lines of changes to source files in my patch-in-progress. So while gyp itself might have that feature in it, for whatever reason, the WebKit projects do not utilize that feature right now, so in practice, a switch to gyp means a switch away from rule-based compilation. There's two different things going on here: ease of maintenance, and reproducibility. Using wildcards in the (meta-)build file itself can make things easier but make it harder to get reproducible builds since you might accidentally include files in the tree you didn't want. When you're on a minority port, having wildcards certainly makes your life easier because you don't have to track every change we make explicitly, and it's probably easier for you to ensure you're doing things correctly in your own checkouts, so you don't have to worry about reproducibility as much. On the other hand, if you were to convert to using the same gypi files we were using, then you wouldn't have to track every change we make, either; you'd get them for free. -- Dirk ___ webkit-dev mailing list webkit-dev@lists.webkit.org https://lists.webkit.org/mailman/listinfo/webkit-dev
Re: [webkit-dev] Is the wxWidgets port maintained?
On Mon, Feb 11, 2013 at 3:26 PM, Kevin Ollivier kev...@theolliviers.comwrote: On Feb 11, 2013, at 2:01 PM, Martin Robinson wrote: On Mon, Feb 11, 2013 at 1:41 PM, Kevin Ollivier kev...@theolliviers.com wrote: Actually, it will, if anything, increase the workload. Because I use waf, I am able to use Python to auto-generate the list of sources to build. In other words, I tell it to build all sources in a defined list of base source directories, along with all sources in baseDirectory/portName subdirs (where in this case I set portName to wx) and FileNameportName.cpp (e.g. FileNameWX.cpp if the port is wx), along with some additional similar rules for USE defines, like CF. Then if there are exceptions to these rules, I just filter them out of the results or explicitly add files when I need to, say if I need to compile a single Mac port source file. Since the WebKit tree is well-structured, this approach works quite well, with me having to define exceptions fairly rarely. The main issue I run into seems to be derived sources files that are no longer built / used but are still being generated. The performance hit of this is about 1 added second to my build, though on slower machin es it might be a couple seconds. For me, it's negligible given the benefit I get from it. If I understand correctly, gyp is also capable of this kind of wildcard inclusion and exclusion. This will be the tool that allows the gyp build to be shared among many ports with the same source lists. Yes, but that is not what I see when I check, say, Source/WebCore/WebCore.gypi. Which, BTW, has had around 500 revisions in it over the past 6 months in comparison to the ~10 lines of changes to source files in my patch-in-progress. So while gyp itself might have that feature in it, for whatever reason, the WebKit projects do not utilize that feature right now, so in practice, a switch to gyp means a switch away from rule-based compilation. We do! WebCore.gypi just lists all files and http://trac.webkit.org/browser/trunk/Source/WebCore/WebCore.gyp/WebCore.gypdefines what gets built on each platform. - R. Niwa ___ webkit-dev mailing list webkit-dev@lists.webkit.org https://lists.webkit.org/mailman/listinfo/webkit-dev
Re: [webkit-dev] Is the wxWidgets port maintained?
On Feb 11, 2013, at 3:33 PM, Dirk Pranke wrote: On Mon, Feb 11, 2013 at 3:26 PM, Kevin Ollivier kev...@theolliviers.com wrote: On Feb 11, 2013, at 2:01 PM, Martin Robinson wrote: On Mon, Feb 11, 2013 at 1:41 PM, Kevin Ollivier kev...@theolliviers.com wrote: Actually, it will, if anything, increase the workload. Because I use waf, I am able to use Python to auto-generate the list of sources to build. In other words, I tell it to build all sources in a defined list of base source directories, along with all sources in baseDirectory/portName subdirs (where in this case I set portName to wx) and FileNameportName.cpp (e.g. FileNameWX.cpp if the port is wx), along with some additional similar rules for USE defines, like CF. Then if there are exceptions to these rules, I just filter them out of the results or explicitly add files when I need to, say if I need to compile a single Mac port source file. Since the WebKit tree is well-structured, this approach works quite well, with me having to define exceptions fairly rarely. The main issue I run into seems to be derived sources files that are no longer built / used but are still being generated. The performance hit of this is about 1 added second to my build, though on slower mach in es it might be a couple seconds. For me, it's negligible given the benefit I get from it. If I understand correctly, gyp is also capable of this kind of wildcard inclusion and exclusion. This will be the tool that allows the gyp build to be shared among many ports with the same source lists. Yes, but that is not what I see when I check, say, Source/WebCore/WebCore.gypi. Which, BTW, has had around 500 revisions in it over the past 6 months in comparison to the ~10 lines of changes to source files in my patch-in-progress. So while gyp itself might have that feature in it, for whatever reason, the WebKit projects do not utilize that feature right now, so in practice, a switch to gyp means a switch away from rule-based compilation. There's two different things going on here: ease of maintenance, and reproducibility. Using wildcards in the (meta-)build file itself can make things easier but make it harder to get reproducible builds since you might accidentally include files in the tree you didn't want. When you're on a minority port, having wildcards certainly makes your life easier because you don't have to track every change we make explicitly, and it's probably easier for you to ensure you're doing things correctly in your own checkouts, so you don't have to worry about reproducibility as much. If the filesystem always follows clearly defined rules, then I'd argue that reproducibility would not be an issue, though. WebKit currently doesn't always, but the vast majority of times it does. The cases where it doesn't are few and far between, and I don't think fixes would be all that difficult. You can always create specific locations, like perhaps a Shared directory, for any special cases that should explicitly be added by more than one, but a subset of, ports, and can't be properly encapsulated into a feature or use define. (For example, the WhateverNone.cpp are one of the special exceptions I have to account for, and they could go in there.) I think the concern is really that this approach seems 'implicit' rather than 'explicit'. With clear filesystem rules, though, it's not actually implicit. A file going into one of the directories listed as a primary build dir means it is to be built by all ports. So long as the project defines said rules, the act of putting it there is explicit, and expresses intent. This also, BTW, addresses the problem of old stuff being left in the tree. To remove it from the build, you have to remove it, so you can just leave it sitting in the tree for someone to stumble across later. (I'm pretty sure a number of the exceptions I define in waf are actually examples of this.) On the other hand, if you were to convert to using the same gypi files we were using, then you wouldn't have to track every change we make, either; you'd get them for free. Yes, but this mechanism relies on WebKit's policies about fixing others' build systems to be effective. waf, on the other hand, removes that task almost entirely. Plus, see my other comments about all the other benefits of using a meta-build system. (And BTW, on that topic, in newer versions of waf there is even a build system kit that provides a starting point for building your own build system off of waf. :) Thanks, Kevin -- Dirk ___ webkit-dev mailing list webkit-dev@lists.webkit.org https://lists.webkit.org/mailman/listinfo/webkit-dev
Re: [webkit-dev] Is the wxWidgets port maintained?
On Feb 11, 2013, at 3:37 PM, Ryosuke Niwa wrote: On Mon, Feb 11, 2013 at 3:26 PM, Kevin Ollivier kev...@theolliviers.com wrote: On Feb 11, 2013, at 2:01 PM, Martin Robinson wrote: On Mon, Feb 11, 2013 at 1:41 PM, Kevin Ollivier kev...@theolliviers.com wrote: Actually, it will, if anything, increase the workload. Because I use waf, I am able to use Python to auto-generate the list of sources to build. In other words, I tell it to build all sources in a defined list of base source directories, along with all sources in baseDirectory/portName subdirs (where in this case I set portName to wx) and FileNameportName.cpp (e.g. FileNameWX.cpp if the port is wx), along with some additional similar rules for USE defines, like CF. Then if there are exceptions to these rules, I just filter them out of the results or explicitly add files when I need to, say if I need to compile a single Mac port source file. Since the WebKit tree is well-structured, this approach works quite well, with me having to define exceptions fairly rarely. The main issue I run into seems to be derived sources files that are no longer built / used but are still being generated. The performance hit of this is about 1 added second to my build, though on slower machin es it might be a couple seconds. For me, it's negligible given the benefit I get from it. If I understand correctly, gyp is also capable of this kind of wildcard inclusion and exclusion. This will be the tool that allows the gyp build to be shared among many ports with the same source lists. Yes, but that is not what I see when I check, say, Source/WebCore/WebCore.gypi. Which, BTW, has had around 500 revisions in it over the past 6 months in comparison to the ~10 lines of changes to source files in my patch-in-progress. So while gyp itself might have that feature in it, for whatever reason, the WebKit projects do not utilize that feature right now, so in practice, a switch to gyp means a switch away from rule-based compilation. We do! WebCore.gypi just lists all files and http://trac.webkit.org/browser/trunk/Source/WebCore/WebCore.gyp/WebCore.gyp defines what gets built on each platform. Sorry if I wasn't clear, but by rule-based compilation, I mean not having a WebCore.gypi to maintain at all. waf needs no such file because it checks the filesystem with a set of rules to determine what sources to build. Thanks, Kevin - R. Niwa ___ webkit-dev mailing list webkit-dev@lists.webkit.org https://lists.webkit.org/mailman/listinfo/webkit-dev
Re: [webkit-dev] Is the wxWidgets port maintained?
On Mon, Feb 11, 2013 at 4:30 PM, Kevin Ollivier kev...@theolliviers.com wrote: On Feb 11, 2013, at 3:33 PM, Dirk Pranke wrote: On Mon, Feb 11, 2013 at 3:26 PM, Kevin Ollivier kev...@theolliviers.com wrote: On Feb 11, 2013, at 2:01 PM, Martin Robinson wrote: On Mon, Feb 11, 2013 at 1:41 PM, Kevin Ollivier kev...@theolliviers.com wrote: Actually, it will, if anything, increase the workload. Because I use waf, I am able to use Python to auto-generate the list of sources to build. In other words, I tell it to build all sources in a defined list of base source directories, along with all sources in baseDirectory/portName subdirs (where in this case I set portName to wx) and FileNameportName.cpp (e.g. FileNameWX.cpp if the port is wx), along with some additional similar rules for USE defines, like CF. Then if there are exceptions to these rules, I just filter them out of the results or explicitly add files when I need to, say if I need to compile a single Mac port source file. Since the WebKit tree is well-structured, this approach works quite well, with me having to define exceptions fairly rarely. The main issue I run into seems to be derived sources files that are no longer built / used but are still being generated. The performance hit of this is about 1 added second to my build, though on slower mac hin es it might be a couple seconds. For me, it's negligible given the benefit I get from it. If I understand correctly, gyp is also capable of this kind of wildcard inclusion and exclusion. This will be the tool that allows the gyp build to be shared among many ports with the same source lists. Yes, but that is not what I see when I check, say, Source/WebCore/WebCore.gypi. Which, BTW, has had around 500 revisions in it over the past 6 months in comparison to the ~10 lines of changes to source files in my patch-in-progress. So while gyp itself might have that feature in it, for whatever reason, the WebKit projects do not utilize that feature right now, so in practice, a switch to gyp means a switch away from rule-based compilation. There's two different things going on here: ease of maintenance, and reproducibility. Using wildcards in the (meta-)build file itself can make things easier but make it harder to get reproducible builds since you might accidentally include files in the tree you didn't want. When you're on a minority port, having wildcards certainly makes your life easier because you don't have to track every change we make explicitly, and it's probably easier for you to ensure you're doing things correctly in your own checkouts, so you don't have to worry about reproducibility as much. If the filesystem always follows clearly defined rules, then I'd argue that reproducibility would not be an issue, though. WebKit currently doesn't always, but the vast majority of times it does. The cases where it doesn't are few and far between, and I don't think fixes would be all that difficult. You can always create specific locations, like perhaps a Shared directory, for any special cases that should explicitly be added by more than one, but a subset of, ports, and can't be properly encapsulated into a feature or use define. (For example, the WhateverNone.cpp are one of the special exceptions I have to account for, and they could go in there.) I think the concern is really that this approach seems 'implicit' rather than 'explicit'. With clear filesystem rules, though, it's not actually implicit. A file going into one of the directories listed as a primary build dir means it is to be built by all ports. So long as the project defines said rules, the act of putting it there is explicit, and expresses intent. This also, BTW, addresses the problem of old stuff being left in the tree. To remove it from the build, you have to remove it, so you can just leave it sitting in the tree for someone to stumble across later. (I'm pretty sure a number of the exceptions I define in waf are actually examples of this.) I'm not sure we're talking about the same sort of concerns. The case I'm talking about when you have a makefile that says something like SRCS=*.cpp. Then, I have to remember to save a local copy of a file as foo.cpp.orig rather than foo_orig.cpp, and if I want to rename a file I have to actually rename it rather than copy it (or at least remember to delete it). I think you're talking more about conventions over which directories contain files for each port? On the other hand, if you were to convert to using the same gypi files we were using, then you wouldn't have to track every change we make, either; you'd get them for free. Yes, but this mechanism relies on WebKit's policies about fixing others' build systems to be effective. waf, on the other hand, removes that task almost entirely. Plus, see my other comments about all the other benefits of using a meta-build system. (And BTW, on that
Re: [webkit-dev] Is the wxWidgets port maintained?
On Mon, Feb 11, 2013 at 4:53 PM, Kevin Ollivier kev...@theolliviers.comwrote: I know you're speaking more in general here, but just to be clear, over the past 6 months, the data hit that WebKit engineers have taken on account of the wx port is close to 0 bytes. The only time svn up or git fetch has pulled anything related to wx is when other developers attempted to update the wx port based on changes made to common code. As things stand, I'd not only be okay with, but actually prefer, that no other port maintainers do anything to try and fix the wx build. This way the port is not causing anyone else a burden except occasionally when I do land some fixes. As I said in my reply to Brent, I think a no build bot / no shared maintenance project rule is not only reasonable, as a practical matter it is required. I'd argue that others contributors having to know to ignore Wx port's code is the most significant cost. How is a new contributor supposed to know to ignore Wx port code? - R. Niwa ___ webkit-dev mailing list webkit-dev@lists.webkit.org https://lists.webkit.org/mailman/listinfo/webkit-dev
Re: [webkit-dev] Is the wxWidgets port maintained?
On Feb 11, 2013, at 5:03 PM, Dirk Pranke wrote: On Mon, Feb 11, 2013 at 4:30 PM, Kevin Ollivier kev...@theolliviers.com wrote: On Feb 11, 2013, at 3:33 PM, Dirk Pranke wrote: On Mon, Feb 11, 2013 at 3:26 PM, Kevin Ollivier kev...@theolliviers.com wrote: On Feb 11, 2013, at 2:01 PM, Martin Robinson wrote: On Mon, Feb 11, 2013 at 1:41 PM, Kevin Ollivier kev...@theolliviers.com wrote: Actually, it will, if anything, increase the workload. Because I use waf, I am able to use Python to auto-generate the list of sources to build. In other words, I tell it to build all sources in a defined list of base source directories, along with all sources in baseDirectory/portName subdirs (where in this case I set portName to wx) and FileNameportName.cpp (e.g. FileNameWX.cpp if the port is wx), along with some additional similar rules for USE defines, like CF. Then if there are exceptions to these rules, I just filter them out of the results or explicitly add files when I need to, say if I need to compile a single Mac port source file. Since the WebKit tree is well-structured, this approach works quite well, with me having to define exceptions fairly rarely. The main issue I run into seems to be derived sources files that are no longer built / used but are still being generated. The performance hit of this is about 1 added second to my build, though on slower ma chin es it might be a couple seconds. For me, it's negligible given the benefit I get from it. If I understand correctly, gyp is also capable of this kind of wildcard inclusion and exclusion. This will be the tool that allows the gyp build to be shared among many ports with the same source lists. Yes, but that is not what I see when I check, say, Source/WebCore/WebCore.gypi. Which, BTW, has had around 500 revisions in it over the past 6 months in comparison to the ~10 lines of changes to source files in my patch-in-progress. So while gyp itself might have that feature in it, for whatever reason, the WebKit projects do not utilize that feature right now, so in practice, a switch to gyp means a switch away from rule-based compilation. There's two different things going on here: ease of maintenance, and reproducibility. Using wildcards in the (meta-)build file itself can make things easier but make it harder to get reproducible builds since you might accidentally include files in the tree you didn't want. When you're on a minority port, having wildcards certainly makes your life easier because you don't have to track every change we make explicitly, and it's probably easier for you to ensure you're doing things correctly in your own checkouts, so you don't have to worry about reproducibility as much. If the filesystem always follows clearly defined rules, then I'd argue that reproducibility would not be an issue, though. WebKit currently doesn't always, but the vast majority of times it does. The cases where it doesn't are few and far between, and I don't think fixes would be all that difficult. You can always create specific locations, like perhaps a Shared directory, for any special cases that should explicitly be added by more than one, but a subset of, ports, and can't be properly encapsulated into a feature or use define. (For example, the WhateverNone.cpp are one of the special exceptions I have to account for, and they could go in there.) I think the concern is really that this approach seems 'implicit' rather than 'explicit'. With clear filesystem rules, though, it's not actually implicit. A file going into one of the directories listed as a primary build dir means it is to be built by all ports. So long as the project defines said rules, the act of putting it there is explicit, and expresses intent. This also, BTW, addresses the problem of old stuff being left in the tree. To remove it from the build, you have to remove it, so you can just leave it sitting in the tree for someone to stumble across later. (I'm pretty sure a number of the exceptions I define in waf are actually examples of this.) I'm not sure we're talking about the same sort of concerns. The case I'm talking about when you have a makefile that says something like SRCS=*.cpp. Then, I have to remember to save a local copy of a file as foo.cpp.orig rather than foo_orig.cpp, and if I want to rename a file I have to actually rename it rather than copy it (or at least remember to delete it). I think you're talking more about conventions over which directories contain files for each port? Yes, that was what I was talking about. I tend to avoid making copies of files like what you describe so as not to clutter my local tree. If I'm going to make substantial or more experimental changes, I just move it onto a git branch so I can just wipe everything if it doesn't pan out. On the other hand, if you were to convert to using the same gypi files we were
Re: [webkit-dev] Is the wxWidgets port maintained?
On Feb 11, 2013, at 5:20 PM, Benjamin Poulain wrote: On Mon, Feb 11, 2013 at 5:11 PM, Ryosuke Niwa rn...@webkit.org wrote: On Mon, Feb 11, 2013 at 4:53 PM, Kevin Ollivier kev...@theolliviers.com wrote: I know you're speaking more in general here, but just to be clear, over the past 6 months, the data hit that WebKit engineers have taken on account of the wx port is close to 0 bytes. The only time svn up or git fetch has pulled anything related to wx is when other developers attempted to update the wx port based on changes made to common code. As things stand, I'd not only be okay with, but actually prefer, that no other port maintainers do anything to try and fix the wx build. This way the port is not causing anyone else a burden except occasionally when I do land some fixes. As I said in my reply to Brent, I think a no build bot / no shared maintenance project rule is not only reasonable, as a practical matter it is required. I'd argue that others contributors having to know to ignore Wx port's code is the most significant cost. How is a new contributor supposed to know to ignore Wx port code? It looks like everyone agree wx would be better maintained outside webkit.org. Kevin just need to find a technical solution to the problem of maintaining the history. I will make a patch removing wxWidgets from the tree. Kevin, can you communicate when you are ready for the transition? I'd like to do that this week if that is okay with you. To be honest, I think I know more about the wx port, and would probably be better able to cleanly remove it from the tree, than you would. As I've said from the start, I'll remove it if that's what people want, but I do think the severity of this problem is being exaggerated, and going from posing the question of project status to giving a one week eviction notice in less than 24 hours seems a bit rash to say the least. I wish I could get paid good money to spend my time doing things like creating patches to remove inactive files from source trees. :) As it stands, though, this wasn't exactly on my TODO list for this week as of, say, this morning, and I do have plenty on it right now. I'm already even regretting how much time I've put into this discussion. To think, it was wx and GTK that started the whole WebKit porting experiment in the first place, thanks to the gracious help of the WebKit community, and particularly Eric, and now I feel like the project can't push me out the door fast enough. ;-) Please be patient, I will take care of it, but I doubt I will manage it this week. Regards, Kevin Benjamin ___ webkit-dev mailing list webkit-dev@lists.webkit.org https://lists.webkit.org/mailman/listinfo/webkit-dev
Re: [webkit-dev] Is the wxWidgets port maintained?
On Mon, Feb 11, 2013 at 6:17 PM, Kevin Ollivier kev...@theolliviers.comwrote: To be honest, I think I know more about the wx port, and would probably be better able to cleanly remove it from the tree, than you would. As I've said from the start, I'll remove it if that's what people want, but I do think the severity of this problem is being exaggerated, and going from posing the question of project status to giving a one week eviction notice in less than 24 hours seems a bit rash to say the least. I wish I could get paid good money to spend my time doing things like creating patches to remove inactive files from source trees. :) As it stands, though, this wasn't exactly on my TODO list for this week as of, say, this morning, and I do have plenty on it right now. I'm already even regretting how much time I've put into this discussion. I am sorry, I should have given more context. There is visibly a growing discontent in the community about the cost imposed from small ports. Just two weeks ago, there were 2 threads discussing the cost of peripheral ports. I am convinced a part of this is technical. The project has not changed its policies while the number of ports was growing. While duplicated code and interfaces was okay when there were only 3 ports, it has become a pain when we have 7+ ports to updates. This weekend, I was looking at technical ways to reduce the problems, and the wx port looks like one of the issue. Everyone with whom I have discussed that today seems to agree. You said yourself As things stand, I'd not only be okay with, but actually prefer, that no other port maintainers do anything to try and fix the wx build.. The best way to achieve that and reduce the maintenance for everyone is to have the wx port developed outside the tree. Which is why I propose to do the change. To think, it was wx and GTK that started the whole WebKit porting experiment in the first place, thanks to the gracious help of the WebKit community, and particularly Eric, and now I feel like the project can't push me out the door fast enough. ;-) Please be patient, I will take care of it, but I doubt I will manage it this week. In his email WebKit Wishes, Eric said It can’t be the job of the core maintainers to care about all the peripheral ports which contribute very little core code. I also think it is just not fair having hundred of peoples taking care of updating the port. There is no rush to move the port out of the tree. I suggested this week in order to have a definite timeline but there is no urgency. If there is interest in the wxWidgets community to make the port actively developed, that would be a great solution too. You do not have to do the change by yourself. I'd be happy to start if that helps you, and you can refine the changes. Cheers, Benjamin ___ webkit-dev mailing list webkit-dev@lists.webkit.org https://lists.webkit.org/mailman/listinfo/webkit-dev
Re: [webkit-dev] Is the wxWidgets port maintained?
This entire email thread makes me sad. I'm sad because we did this to ourselves (our porting systems/policies are lacking), we did this very abruptly (the last port we kicked out of webkit.org took a year?), and we don't have any clear policy for this sort of thing (adding/removing ports in webkit.org). I think this, and many recent threads on the subject, are symptoms of the same problem: We are at a point in WebKit (and have been for a long time) where we are focused less on the Web Platform (via WebKit) being everywhere, and more on that Web Platform being awesome. Maybe it's time we clarified this in our goals doc? Our (my!) previous platform-abstraction mistakes have also left us in an awkward position where it is difficult for us to support 7+ ports with separate build systems, and separate sets of constraints (however minor) on WebCore. I think it is the right decision to de-couple all ports from the core code, but doing that is going to take us a very long time. I think that moving closer to one build system will help, but won't help those trying to advance WebCore from having to deal with 8 different port APIs, 5 different networking libraries, 2 different XML parser, 4 different GraphicsContext implementations, and countless other configuration options! I guess the point of this all is that I'm sorry. I'm sorry to Kevin, and all the other ports that I've helped integrate with WebKit, that I and others didn't sit down years ago and design better, more maintainable porting systems for WebKit. I think if we as a community are actively interested in maintaining as many ports as we have (and welcoming new ones) we need to come up with better ways to do so. And clearer policies for what it means to be a port in WebKit. In the specific case of Wx, I am reluctantly agreed that code with only one(?) maintainer is pretty close to dead and thus per WebKit's long-standing dead/commented-out code policies should be removed. :( Kevin has been with us a long time, and asking him to leave in this manner is saddening. Of course saying Wx is dead code, begs the question as to what is needed for a port to be considered live in WebKit.org? With our current porting architecture, I would argue that at least 3 full time people are probably needed, and that this should be considered before accepting new ports. I'm not in any rush to see Wx removed, and I agree it makes sense to let Kevin write the patches as much as he's willing. I think certainly having to wake up one day to see that one of your Open Source projects is kicking you out, is pretty jaring. :( I hope we can do this better next time, and maybe we should have a separate discussion about what it means to be a supported port in WebKit.org and what new/existing ports need to do to meet that bar. As for the future of WebKit on Wx: I don't know enough about WebKit2 to know if it has a more easily portable/maintainable architecture. For the port I work on (chromium), others haven't really ported in WebKit.org, but rather port at a higher level -- Chromium's Content layer or https://code.google.com/p/chromiumembedded/. Maybe one of those options are a better solution for a port like Wx with limited resources? Again, my apologies to you Kevin for my part in all this. On Mon, Feb 11, 2013 at 7:44 PM, Benjamin Poulain benja...@webkit.org wrote: On Mon, Feb 11, 2013 at 6:17 PM, Kevin Ollivier kev...@theolliviers.com wrote: To be honest, I think I know more about the wx port, and would probably be better able to cleanly remove it from the tree, than you would. As I've said from the start, I'll remove it if that's what people want, but I do think the severity of this problem is being exaggerated, and going from posing the question of project status to giving a one week eviction notice in less than 24 hours seems a bit rash to say the least. I wish I could get paid good money to spend my time doing things like creating patches to remove inactive files from source trees. :) As it stands, though, this wasn't exactly on my TODO list for this week as of, say, this morning, and I do have plenty on it right now. I'm already even regretting how much time I've put into this discussion. I am sorry, I should have given more context. There is visibly a growing discontent in the community about the cost imposed from small ports. Just two weeks ago, there were 2 threads discussing the cost of peripheral ports. I am convinced a part of this is technical. The project has not changed its policies while the number of ports was growing. While duplicated code and interfaces was okay when there were only 3 ports, it has become a pain when we have 7+ ports to updates. This weekend, I was looking at technical ways to reduce the problems, and the wx port looks like one of the issue. Everyone with whom I have discussed that today seems to agree. You said yourself As things stand, I'd not only be okay with, but actually prefer, that no
