Re: [webkit-dev] Common build system (was Re: WebKit Wishes)
The parser (and the grammar) works the way it does because it’s just
Python—the whole thing can be slurped in quickly by the interpreter and
made available to GYP as data. It’s faster than a parser written in Python
for a similar language would be, and it’d likely take a binary Python
module to approach or exceed this speed. Developers don’t like waiting for
tools.
With that in mind, I’d be open to experimenting with alternative (probably
optional) formats.
On Tue, Feb 5, 2013 at 7:48 AM, Simon Hausmann simon.hausm...@digia.comwrote:
On Monday, February 04, 2013 10:57:46 AM Maciej Stachowiak wrote:
On Feb 4, 2013, at 10:46 AM, Mark Mentovai m...@chromium.org wrote:
GYP was written in Python to address point (b). Python was already
part of
the baseline requirements on all platforms, so we already had Python
available everywhere we needed it. There are no dependencies on
external
binaries, and no compiled code needs to be checked in anywhere or
maintained as part of a base image.
As for point (a), you can easily have a top-level Makefile not
generated
by GYP that says “run GYP to produce the build files for whatever
environment you like and then pass control to that build system to do
the
rest of the build. Developers who like it can use ninja for their own
builds, and your bots can use Xcode or make if that’s a requirement (or
if ninja doesn’t meet your requirements given point (b)).
Checking in the generated Xcode projects is another alternative. The
Makefile might be better for the reasons you suggest, though.
I'm reasonably confident at this point that Gyp can meet our hard
requirements. Our remaining issues are finding time to do it and
comprehensibility/readability of the syntax.
On the topic of the Gyp syntax:
I find it pretty readable, but I wonder if you guys are open to the idea of
relaxing the parser a bit to also allow for the omission of quotes and
commas
for the dictionary keys. I feel a syntax like
{
includes: [
...
]
variables: [
]
}
is slightly less error prone to type and you could probably still support
the
stricter language, too.
Simon
___
webkit-dev mailing list
webkit-dev@lists.webkit.org
https://lists.webkit.org/mailman/listinfo/webkit-dev
Re: [webkit-dev] Common build system (was Re: WebKit Wishes)
You’re not supposed to use arbitrary Python, it’s highly discouraged. We have a linter that keeps you from doing things you’re not supposed to do (like this), but it slows things down, so it’s not part of the “standard” GYP run that developers normally use. It can run as a pre-commit script or test on the bot or something else. Used as we’re using it, GYP basically is simple JSON, except the rules about commas are looser and # introduces comments. On Tue, Feb 5, 2013 at 2:47 PM, Ryosuke Niwa rn...@webkit.org wrote: On Tue, Feb 5, 2013 at 6:09 AM, Mark Mentovai m...@chromium.org wrote: The parser (and the grammar) works the way it does because it’s just Python This works great for people who like Python syntax but not for someone like myself who dislikes Python syntax. I also find it particularly annoying that people can use whatever Python constructs they want to use in GYP. It dramatically reduces language portability because you need to support quite a few Python constructs and quirks in order to correctly parse GYP. I personally would have much preferred for it be a simple JSON file. - R. Niwa ___ webkit-dev mailing list webkit-dev@lists.webkit.org https://lists.webkit.org/mailman/listinfo/webkit-dev
Re: [webkit-dev] Common build system (was Re: WebKit Wishes)
The run time comparisons were based on actual comparison with available
Python modules at the time. JSON was substantially slower than having the
Python interpreter slurp it up on its own. In fact, GYP started out reading
JSON input and we switched it to Python once we learned this. It was four
years ago, and I don’t have the numbers handy anymore.
In the end, I don’t think that the actual syntax (as opposed to the
structured data that it contains) is all that important. GYP input files
are fairly simple structured data consisting of very few types: dicts,
lists, strings, and infrequent bools and ints. Any extant or nascent format
that can handle these is a candidate. If there’s some kind of convergence
on a beautiful one that’s easy to maintain and quick to parse, there’s no
reason GYP can’t be taught to read it, even alongside the existing
JSON-esque format that’s fed to the Python interpreter. If this is the only
thing standing between a project and its adopting GYP, then it’s a really
easy one to get over.
On Tue, Feb 5, 2013 at 3:59 PM, Maciej Stachowiak m...@apple.com wrote:
On Feb 5, 2013, at 6:09 AM, Mark Mentovai m...@chromium.org wrote:
The parser (and the grammar) works the way it does because it’s just
Python—the whole thing can be slurped in quickly by the interpreter and
made available to GYP as data. It’s faster than a parser written in Python
for a similar language would be, and it’d likely take a binary Python
module to approach or exceed this speed. Developers don’t like waiting for
tools.
Is that a result based on testing, or just a projectiom? My guess would be
that most incremental builds do not add or remove any files, and therefore
do not need to regenerate the project files at all, so the speed of parsing
GYP files is unlikely to have a major impact, unless it's a truly massive
difference.
I have to say that the syntax weight of gyp files (quotes, commas, braces,
brackets) makes them feel harder to read and edit than other build file
formats designed for human consumption (obviously Xcode is worse).
Compare this bit of gyp for generating a derived source file:
{
'action_name': 'XMLNames',
'inputs': [
'../dom/make_names.pl',
'../xml/xmlattrs.in',
],
'outputs': [
'(SHARED_INTERMEDIATE_DIR)/webkit/XMLNames.cpp',
'(SHARED_INTERMEDIATE_DIR)/webkit/XMLNames.h',
],
'action': [
'python',
'scripts/action_makenames.py',
'@(_outputs)',
'--',
'@(_inputs)',
'--',
'--extraDefines', '(feature_defines)'
],
'msvs_cygwin_shell': 1,
},
vs the equivalent bit of DerivedSources.make:
XMLNames.cpp : dom/make_names.pl xml/xmlattrs.in
perl -I $(WebCore)/bindings/scripts $ --attrs $(WebCore)/xml/
xmlattrs.in
Or GNUMakefile.am:
DerivedSources/WebCore/XMLNames.cpp DerivedSources/WebCore/XMLNames.h:
$(WebCore)/dom/make_names.pl $(WebCore)/xml/xmlattrs.in
$(AM_V_GEN)$(PERL) -I$(WebCore)/bindings/scripts $ --attrs
$(WebCore)/xml/xmlattrs.in --outputDir $(GENSOURCES_WEBCORE)
Or DerivedSources.pri:
xmlnames.output = XMLNames.cpp
xmlnames.input = XML_NAMES
xmlnames.script = $$PWD/dom/make_names.pl
xmlnames.commands = perl -I$$PWD/bindings/scripts $$xmlnames.script
--attrs $$PWD/xml/xmlattrs.in --preprocessor \$${QMAKE_MOC} -E\
--outputDir ${QMAKE_FUNC_FILE_OUT_PATH}
GENERATORS += xmlnames
Gyp kind of seems like an outlier in heaviness of the syntax compared to
the other build systems.The irony is that Python itself doesn't use a lot
of punctuation in its syntax, so actual Python code looks much cleaner. I
like to read Python code, but I can't say I enjoy reading the Gyp syntax.
This makes me wonder if a Python DSL might be more legible than simply
using a dictionary.
Alternately, maybe it turns out that parsing a custom language is not that
huge a hit in practice.
Regards,
Maciej
___
webkit-dev mailing list
webkit-dev@lists.webkit.org
https://lists.webkit.org/mailman/listinfo/webkit-dev
Re: [webkit-dev] Common build system (was Re: WebKit Wishes)
GYP was written in Python to address point (b). Python was already part of the baseline requirements on all platforms, so we already had Python available everywhere we needed it. There are no dependencies on external binaries, and no compiled code needs to be checked in anywhere or maintained as part of a base image. As for point (a), you can easily have a top-level Makefile not generated by GYP that says “run GYP to produce the build files for whatever environment you like and then pass control to that build system to do the rest of the build. Developers who like it can use ninja for their own builds, and your bots can use Xcode or make if that’s a requirement (or if ninja doesn’t meet your requirements given point (b)). I’d be happy to discuss this further with anyone who’s interested in moving in this direction. On Mon, Feb 4, 2013 at 12:20 AM, Maciej Stachowiak m...@apple.com wrote: On Feb 3, 2013, at 8:18 PM, Mike Lawther mikelawt...@chromium.org wrote: Hi Maciej! On 31 January 2013 11:15, Maciej Stachowiak m...@apple.com wrote: It's far simplest for us if: (a) There is an Xcode project (or a Makefile) that builds the Mac port checked in to source control. (b) The generated project invokes only tools that are part of the default Mac OS X install. For b), does 'default' include an Xcode install? From my memory of setting up a Mac dev box Xcode was needed to compile. The limitation in (a) is all about how the build fleet is managed, and yes, the machines in question have the Mac OS X user install as well as developer tools installed, and it is ok to rely on that. To clarify: the things that are not kosher are submitting a binary tool as part of the module, downloading things as part of the build, or relying on something to be installed out of band before building. It is ok for the build to rely on scripts, so long as the relevant interpreter is part of the default install. Any such script would have to be in each directory under Source/ since these build separately without access to the source of the others. I should mention that there's a lot of interest right now at Apple in the possibility of switching to Gyp. We think it's likely closer to what we need than CMake, and we suspect there is probably somewhat better probability of modifying it if necessary. If we find someone at Apple who is able to work on such a project in the short term (not completely impossible), would any Google folks be interested in collaborating? Ideally we could use assistance from someone who (a) understands how the Chromium WebKit Gyp-based build works; and (b) is willing and able to change Gyp if necessary. Regards, Maciej mike On 31 January 2013 11:15, Maciej Stachowiak m...@apple.com wrote: On Jan 30, 2013, at 3:24 PM, Dirk Pranke dpra...@chromium.org wrote: On Wed, Jan 30, 2013 at 1:50 PM, Filip Pizlo fpi...@apple.com wrote: Thanks for sharing this. On Jan 30, 2013, at 1:28 PM, Eric Seidel e...@webkit.org wrote: I wish we only had one build system (it were easy to add/remove/move files). I believe changes like http://trac.webkit.org/changeset/74849 are an unhealthy sign for the project. Adam is not the only person who has chosen to empty files instead of removing them. The pain of updating 8 build system is so great, we jump through hoops to avoid it. Which means it took us months to move JavaScriptCore/wtf to WTF, and will take us years to kill WebCore/platform. +1 This is a hard problem. It is a problem worth solving. Do you have more thoughts on this, particularly since you know quite well how both Xcode and gyp work? I suspect this is one of those things that it would be hard to achieve consensus on since there are so many stakeholders. But it may be fruitful to have a what if discussion about what this might look like. I think we can solve this problem if we agree that we want to. I think we haven't in the past mostly because we couldn't reach a consensus that it was worth solving enough to really try. I would love to see this fixed and would be glad to work on it. I think we should at least pursue this far enough to fully understand what our options are and what the costs and tradeoffs might be; does anyone disagree, and is anyone else willing to help pitch in? I think there are several possible ways we could solve this. One would be to switch to a common meta-build system. My understanding is that Apple's internal production build processes impose certain constraints here that I don't fully understand, but I know we've discussed the possibility of checking in generated project files as a workaround. Maybe there are other options as well to those constraints? I would love to discuss this further w/ someone from Apple ... It's far simplest for us if: (a) There is an Xcode project (or a Makefile) that builds the Mac port checked in to source control. (b)
Re: [webkit-dev] Removing BUILDING_ON / TARGETING macros in favor of system availability macros
We just discovered (via a rollout on bug 91103) that there’s a bug
with __MAC_OS_X_VERSION_MAX_ALLOWED in the version of the 10.5 SDK as
present in Xcode 3.2.6. This may be the last version of the 10.5 SDK ever
released (I haven’t checked all of the early Xcode 4 releases). Chromium
uses this SDK for its builds, although will likely switch to the 10.6 SDK
in the near future.
In short, __MAC_OS_X_VERSION_MAX_ALLOWED, which is supposed to track the
SDK version, is set to 1060 instead of the expected 1050 in this version of
the 10.5 SDK.
--
// clang t.c -isysroot /Xcode3.2/SDKs/MacOSX10.5.sdk
-mmacosx-version-min=10.4 -o t
#include Availability.h
#include AvailabilityMacros.h
#include stdio.h
int main(int argc, char* argv[]) {
#define PRINT_MACRO(m) printf(%s = %d\n, # m, m)
PRINT_MACRO(MAC_OS_X_VERSION_MIN_REQUIRED);
PRINT_MACRO(MAC_OS_X_VERSION_MAX_ALLOWED);
PRINT_MACRO(__MAC_OS_X_VERSION_MIN_REQUIRED);
PRINT_MACRO(__MAC_OS_X_VERSION_MAX_ALLOWED);
#undef PRINT_MACRO
return 0;
}
--
produces
MAC_OS_X_VERSION_MIN_REQUIRED = 1040
MAC_OS_X_VERSION_MAX_ALLOWED = 1050
__MAC_OS_X_VERSION_MIN_REQUIRED = 1040
__MAC_OS_X_VERSION_MAX_ALLOWED = 1060
Oops.
I traced this problem to AvailabilityInternal.h, which in this SDK, says
(slightly abbreviated):
#elif defined(__ENVIRONMENT_MAC_OS_X_VERSION_MIN_REQUIRED__)
#ifndef __MAC_OS_X_VERSION_MAX_ALLOWED
#define __MAC_OS_X_VERSION_MAX_ALLOWED __MAC_10_6
#endif
#endif
At least for the purposes of testing whether the 10.5 SDK is in use, we’ll
need to use AvailabilityMacros.h’s MAC_OS_X_VERSION_MAX_ALLOWED instead
of Availability.h’s __MAC_OS_X_VERSION_MAX_ALLOWED. I guess it’s a
question of style whether the rest of WebKit should follow.
___
webkit-dev mailing list
webkit-dev@lists.webkit.org
http://lists.webkit.org/mailman/listinfo/webkit-dev
Re: [webkit-dev] Removing BUILDING_ON / TARGETING macros in favor of system availability macros
Mark Rowe wrote: It'd be complicated to do this more widely since the AvailabilityMacros.h version of the macros are defined even when building for iOS. We'd need to review all of the uses to ensure that they were handled correctly. Given that I think it'd be better to just use this as a localized workaround until we drop support for building for Leopard. The one place where we found it to be significant so far is in a Chromium-specific file that’s not relevant to iOS, and we’re just going to use MAC_OS_X_VERSION_MAX_ALLOWED in that one instance for the time being. If it happens anywhere else with this SDK, it’ll likely also be Chromium-specific, and we’ll just deal with the ugliness of having to use a different macro for this purpose until we get rid of the 10.5 SDK altogether. Shouldn’t be long. ___ webkit-dev mailing list webkit-dev@lists.webkit.org http://lists.webkit.org/mailman/listinfo/webkit-dev
Re: [webkit-dev] Removing BUILDING_ON / TARGETING macros in favor of system availability macros
Tony brought me in to comment on what impact this might have on the Chromium Mac build. It shouldn’t have any impact. Any use of the compiler-defined macros is fine. In Chrome code, we usually use MAC_OS_X_VERSION_MAX_ALLOWED and MAC_OS_X_VERSION_MIN_REQUIRED from AvailabilityMacros.h, along with symbolic macros like MAC_OS_X_VERSION_10_7 instead of 1070. This has an annoying disadvantage that older SDKs (like the 10.6 SDK) don’t define MAC_OS_X_VERSION_10_7, so we’re stuck testing things like defined(MAC_OS_X_VERSION_10_7) MAC_OS_X_VERSION_MAX_ALLOWED = MAC_OS_X_VERSION_10_7. The same applies when using Availability.h, where the symbolic names are of the form __MAC_10_7, but you seem to have chosen not to use those, thus condensing the macro logic. We used AvailabilityMacros.h’s MAC_OS_X_VERSION_MAX_ALLOWED and MAC_OS_X_VERSION_MIN_REQUIRED instead of Availability.h’s __MAC_OS_X_VERSION_MAX_ALLOWED and __MAC_OS_X_VERSION_MIN_REQUIRED because the latter header’s comments discuss how it’s appropriate for OS headers, while the former documents its use for user (non-OS) code. TN2064 also recommends AvailabilityMacros.h. I never liked the WebKit-specific BUILDING_ON_*/TARGETING_* macros and am happy to see them go. My one real gripe with the macros from both Availability.h and AvailabilityMacros.h is that it’s not terribly obvious which one refers to the SDK (it’s MAX_ALLOWED) and which refers to the deployment target (it’s MIN_REQUIRED). Until you’re familiar with the macros, I think it’s unclear that “allowed” refers to APIs allowed to be called because they’re present in the SDK, and unclear that “required” refers to the runtime requirement. This probably leads to more misuse and abuse than if they had SDK and DT in their names. I guess BUILDING_ON_* and TARGETING_* were a little better in that regard, but they were misused and abused too. SDKs and DTs are probably just too confusing to begin with. On Tue, Jul 10, 2012 at 7:24 PM, Mark Rowe mr...@apple.com wrote: I would like to propose removing the BUILDING_ON and TARGETING family of macros that are used to build code conditionally for different versions of OS X. I propose this in order to address several problems: - The checks are verbose, and getting worse. For instance, in order to write code targeting OS X 10.8 and newer I have to enumerate all other supported OS versions: #if !defined(TARGETING_LEOPARD) !defined(TARGETING_SNOW_LEOPARD) !defined(TARGETING_LION) … #endif This problem has become worse over time as the number of supported OS X versions in the WebKit code base has increased. - The nature of the version checks are often not obvious at first glance. In order to understand the checks you have to first remember the marketing names of the various OS X releases. You must then reason about the conditional logic of the check itself, which will often contains multiple negated clauses. - Almost all current uses are incorrect in the context of SDKs. The vast majority of the checks in WebKit use the BUILDING_ON macros where the TARGETING macros would be more appropriate. This hasn't cause many problems to date since builds of WebKit on OS X for the most part do not use SDKs. This means that they build with __MAC_OS_X_VERSION_MIN_REQUIRED == __MAC_OS_X_VERSION_MAX_ALLOWED, resulting in the BUILDING_ON and TARGETING macros being equivalent. However, when building with a deployment target of an older OS release than the SDK you're building against, the BUILDING_ON and TARGETING macros will have different behavior. The result is that WebKit fails to build against an SDK when targeting an older OS release. My proposed solution to these problems is to remove the BUILDING_ON and TARGETING macros. The vast majority of the BUILDING_ON uses and all of the TARGETING uses would be replaced with tests against __MAC_OS_X_VERSION_MIN_REQUIRED. The small number of uses of BUILDING_ON that are being used correctly would be replaced with tests against __MAC_OS_X_VERSION_MAX_ALLOWED. The example above of code targeting OS X 10.8 and newer would become: #if __MAC_OS_X_VERSION_MIN_REQUIRED = 1080 … #endif Code that wishes to target only OS X 10.6 and older would become: #if __MAC_OS_X_VERSION_MIN_REQUIRED = 1060 … #endif This is much more concise and understandable than the current approach. I'm open to feedback on this proposal, but I'd like to move forward with this change in the next day or two if no one objects. - Mark ___ webkit-dev mailing list webkit-dev@lists.webkit.org http://lists.webkit.org/mailman/listinfo/webkit-dev ___ webkit-dev mailing list webkit-dev@lists.webkit.org http://lists.webkit.org/mailman/listinfo/webkit-dev
Re: [webkit-dev] Removing BUILDING_ON / TARGETING macros in favor of system availability macros
Mark Rowe wrote: TN2064 appears to have been last modified in 2003, so it predates the existence of Availability.h. Well, that’s about as long as I’ve been writing SDK- and deployment-target-aware code… Availability.h was introduced with the iOS SDK in order to support availability macros that are compatible with both OS X and iOS, where the older macros from AvailabilityMacros.h were specific to OS X. Cool. Other than double-underscore paranoia, there’s no real difference or complaint from me. I guess the one drawback shared by Availability.h, AvailabilityMacros.h, and the outgoing BUILDING_ON_*/TARGETING_* macros is that if you forget to #include the right header, the preprocessor silently does something that you’re probably not expecting. Using a function-like macro like USE(), OS(), and PLATFORM() would help. ___ webkit-dev mailing list webkit-dev@lists.webkit.org http://lists.webkit.org/mailman/listinfo/webkit-dev
Re: [webkit-dev] libWebKitSystemInterfaceLeopard.a with private extern symbols?
Darin Adler wrote: I don’t remember if anyone answered you. You're the first, but thanks for the response. A simple way to transform the library into one with private extern would be to use ld and -r, -exported_symbol or -exported_symbols_list, and -keep_private_externs to make all the symbols private. This would eliminate any need to get a new version of the library from someone at Apple, and should be quite simple to do. The only thing I don’t know is how to say “no exported symbols”, but it could be as simple as -exported_symbols_list /dev/null -- you should give it a try. This would work, but the steps to prepare it get a little bit heinous. I'd need to lipo -thin it first to break it up by architecture, and then ar -x it to break it up into .o files. Then it's ld as you describe, ar -rc to get it back into a .a, and lipo -create. Sure, it's doable, but with the way our build is structured now, we don't really do anything with libWebCoreSystemInterfaceLeopard.a outside of WebCore.gyp, which is now in your repository, so all of these machinations would really belong there... For that matter, I considered just writing something to rewrite the archive and set N_PEXT on everything in the symbol tables. Same effect, more work, fewer temporary files. I believe the problem occurs because the Chromium build uses a different method for controlling what’s exported than what we use in the standard Mac OS X WebKit build, which uses -exported_symbols_list. Yup, that's right. The first thing I tried, which involved minimal boat-rocking, was to strip -R our output to trim the symbols we didn't need. That would have been fine, except in order to do it properly, I'd have to enumerate libWebCoreSystemInterfaceLeopard.a's symbols first. With archive member selection and dead code stripping, the set of symbols that we'd see in the linked product isn't fully deterministic, though, and Apple's strip -R gives an error when it's can't find a symbol that it's told to remove. Oh well. Would these libraries with private extern symbols be unsuitable for Apple's WebKit build? The easiest approach, if it would work for you, would just be to build those libraries with -fvisibility=hidden (GCC_SYMBOLS_PRIVATE_EXTERN=YES), check them in, and call it a day. I'd be surprised if you ever needed these symbols exported now or in the future, and grep WK $(find . -name \*exp) doesn't show anything. Mark ___ webkit-dev mailing list webkit-dev@lists.webkit.org http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev
Re: [webkit-dev] libWebKitSystemInterfaceLeopard.a with private extern symbols?
Darin Adler wrote: Probably not. But even though this is easy it could be a long time until someone gets the free time to make and test this simple change. I was trying to offer you the power to do this on your own schedule, but I see that you’ve already tried it. Given your story, it does seem that using nm combined with strip -R you could do this pretty simply, but perhaps that’s just another dead end. Yup, given that we can't easily forecast which symbols will wind up in the linked product without actually linking it, this isn't easy, unless we want to start hacking in more magic with a custom strip command. All things considered, I thought your initial recommendation was closest to ideal, short of getting rebuilds of those libraries. I've put a patch to implement this for the GYP-based Mac build (currently only used by Chromium) at https://bugs.webkit.org/show_bug.cgi?id=30590 . Thanks, Mark ___ webkit-dev mailing list webkit-dev@lists.webkit.org http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev
[webkit-dev] libWebKitSystemInterfaceLeopard.a with private extern symbols?
'kittens, Would it be possible to get a copy of libWebKitSystemInterfaceLeopard.a (from WebKit/WebKitLibraries) with all of its symbols marked private extern? Or, if nobody actually needs these symbols to be exported so globally, can the version that's currently checked in be replaced with one that's got all of the symbols marked private extern? I'm doing symbol table maintenance on Mac Chromium, and these symbols are some of the few that remain that we actually don't need to expose to anyone. I could set something up as a workaround to make these private extern, but if at all possible, I'd rather do it the right way by getting a build of the library with the symbols already marked private extern. Thanks, Mark ___ webkit-dev mailing list webkit-dev@lists.webkit.org http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev
Re: [webkit-dev] SVN on Windows and the merits of svn:eol-style
Peter Kasting wrote: Note that you don't always want native (at least I think you don't). For example, in Chromium we've ended up marking .cc/.h files, as well as .sh files, as LF. .sln/.vcproj are CRLF (although I'm less sure what the issue there is... it seems like native had some subtle issue or other). We (Chromium) don't really have .sln or .vcproj files checked in any longer, but when we did, I think they were set to native. I suppose this made it easier to edit them on platforms with editors that don't expect CRLFs. Still, I would have chosen CRLF to make it easier to carry diffs, whole files, and even entire tarballs between platforms. Obviously, LF would be out because MSVS is going to write CRLFs in .sln and .vcproj files regardless of what you tell Subversion. Mark ___ webkit-dev mailing list webkit-dev@lists.webkit.org http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev
Re: [webkit-dev] Haiku + GYP (WAS: Build File Maintenance)
This sounds like a good experiment to me too. I don't know much about jam (I've generally avoided it in Xcode) but I'd be happy to provide GYP-side support. Mark Jeremy Orlow wrote: I'm adding a bunch of the GYP experts to this thread and re-naming it for sanity's sake. :-) On Wed, Jul 15, 2009 at 11:20 PM, Maciej Stachowiak m...@apple.com wrote: If you're willing to give it a shot, then that sounds like a fine idea. - Maciej On Jul 15, 2009, at 10:51 PM, Ryan Leavengood wrote: On Mon, Jul 13, 2009 at 4:56 PM, Maciej Stachowiakm...@apple.com wrote: One belated comment on this topic. It would be neat if some port agreed to be the guinea pig to see if gyp could plausibly work for more than Google's ports. The Wx port probably has the lowest resources of any complete port in the tree, so they might not be the best choice of experimental subject, particularly if for them the process required writing a new gyp back end and if they are not yet entirely comfortable going the gyp route. I would need to discuss it with my student, but what about the brand new Haiku port being the gyp guinea pig? For those who don't know, I am mentoring a student in the Google Summer of Code for the Haiku operating system (http://www.haiku-os.org) and we are working on a native Haiku web browser with WebKit as the rendering engine. I don't know if our port is any better of a choice than the Wx port, since the resources are also small (just two of us for now) and we aren't even in the WebKit tree yet, but I think we still might be a good choice because: 1) We obviously don't yet have a production browser using our port so breakage isn't an issue. Plus only my student (Maxime Simon) and I are working on it. 2) I have decent experience with build systems and think I could handle working with gyp and writing a new back end. 3) Haiku generally uses Jam for building and we would like our port to do the same. Rather than adding Yet Another Build System to WebKit, we could use gyp and write a Jam backend for it. This can therefore serve as a test of gyp for another platform as well as for another backend. I would rather not have to maintain a Jamfile for WebKit if I can avoid it, and I certainly don't want to burden the other WebKit developers with having to maintain it for what is now (and may forever be) a tiny port. Though we certainly hope Haiku's popularity increases in the future (it hasn't even had a first release anyhow, so there is plenty of room to grow.) Anyhow, I'd be interested in hearing what other people think. ___ webkit-dev mailing list webkit-dev@lists.webkit.org http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev
Re: [webkit-dev] Build File Maintenance (was Re: Please welcome GYP to the our dysfunctional build family)
Kevin Ollivier wrote: I'd be interested to know what limitations you guys ran into when trying to use SCons for this sort of thing. The SCons input language is too free-form to be easily shoehorned into things like Xcode and Visual Studio projects. SCons is flexible enough that mapping its input language, and all of the logic that its input might contain, to just about anything else would have been an incredibly difficult task. For anything more involved than compile these files and produce a library or executable, with SCons, we would have needed to produce project files that weren't strictly native. The project files would have had to have contained instructions to call SCons to perform the actual build. We sought a greater level of native tool integration. Mark ___ webkit-dev mailing list webkit-dev@lists.webkit.org http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev
Re: [webkit-dev] Build File Maintenance (was Re: Please welcome GYP to the our dysfunctional build family)
Brent Fulgham wrote: The thing that worries me about this latest push to add GYP is that there was much Google enthusiasm for the SCons stuff six months or so ago, and SCons stuff started landing in the tree, and now that's all been tossed away (apparently for performance reasons). Do we feel that GYP is finally the right tool? It's fair to say that it's always been the Chromium team's desire to move at least some chunk of our WebKit build system upstream into the WebKit repository. Conceptually, the instructions for building WebKit belong with WebKit. What you're seeing with Dimitri's recent checkin of .gypi files is a step in this direction. Given this, you're quite right to pick up on the build system du jour nature of the discussion. A year ago, the Chromium team was expecting SCons files to become the master description of our build, but for various reasons including performance, that didn't really work out. We were gung-ho about SCons, but our enthusiasm may have been premature: at least for Chromium, it turned out to not be the right solution right now. The difference between the SCons-driven and GYP-driven builds of Chromium is that with the latter, we really were able to unify our disparate build systems. With GYP, we've got a single build system instead of three (or four), and we were able to do that while meeting the other requirements peculiar to our project. GYP may just be another build system du jour, but cette jour, it's actually carrying Chromium on all of its platforms. It's also important to keep in mind that as long as Chromium is using GYP for its build, we (the Chromium team) are committed to maintaining .gyp/.gypi files checked into the WebKit repository at least as well as they need to be maintained to support our use. Speaking as someone heavily involved with GYP's development, I'd be thrilled if it worked for other projects too; speaking as a Chromium team member, it'd be helpful if people working on other WebKit ports maintained the .gyp/.gypi files as they add, move, and remove files in the WebKit tree, even if no port other than Chromium is actually using GYP. To be honest, though, I don't think that WebKit (or any other project) should choose a build system hastily, and it's entirely conceivable that what works best for Chromium might not work best for WebKit. Mark ___ webkit-dev mailing list webkit-dev@lists.webkit.org http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev
Re: [webkit-dev] Please welcome GYP to the our dysfunctional build family
Generate Your Projects, or GYP, was designed around Chromium's needs. Maintaining distinct project files for different build environments and different platforms put a great burden on our team, and was never intended to be a long-term solution. We really wanted to have a single canonical description of the build from which we could generate the native project files that developers were accustomed to. One proposed solution was to treat our SCons build as canonical and have it generate native project files. This integration never really came to fruition. We had SCons generating Visual Studio projects for simpler projects, but mapping the flexible SCons input language to the more restrictive descriptions used in IDE project files proved difficult. We also considered CMake, and had it demonstrably working for some of our smaller projects as well. Unfortunately, transitioning to CMake would have required moving everything over at once, without allowing for some existing projects to be maintained by hand during a transition period. CMake-generated files contain absolute paths, so a .tar or .zip of the source tree could not be primed with CMake output, complicating the bootstrapping process for new contributors. A less significant factor was that CMake introduced an additional binary build prerequisite, which would have had to have been installed everywhere. Python is already a prerequisite for Chromium, so a Python tool was easier to deploy. The net result of all of this was GYP, which was designed specifically to meet Chromium's needs. We were able to get GYP to generate project files on all three of our main platforms fairly rapidly, and we designed for (and eventually executed on some platforms) a staged transition from the existing build. GYP helped Chromium solve more problems in less time than any of the other options that we experimented with, but it's important to remember that Chromium's needs are not necessarily the same as those of other projects. There are several valid ways to solve the too many build systems problem, and GYP just happens to be one that has worked well for us. Mark (Full disclosure: I'm the principal designer of GYP, and would be happy to discuss it in further detail) Brent Fulgham wrote: So Cmake seems equivalent to GYP. GYP probably has unique features that make it more desireable than Cmake for one reason or another (perhaps it's easier to add new platform support or something), but at the moment those benefits are not obvious to me. I'm afraid that as far as I can tell, GYP seems like a case of NIH syndrome. ___ webkit-dev mailing list webkit-dev@lists.webkit.org http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev
