One belated comment on this topic. It would be neat if some port
agreed to be the guinea pig to see if gyp could plausibly work for
more than Google's ports. The Wx port probably has the lowest
resources of any complete port in the tree, so they might not be the
best choice of experimental subject, particularly if for them the
process required writing a new gyp back end and if they are not yet
entirely comfortable going the gyp route.
Regards,
Maciej
On Jul 10, 2009, at 3:03 PM, Jeremy Orlow wrote:
On Fri, Jul 10, 2009 at 1:14 PM, Kevin Ollivier <kev...@theolliviers.com
> wrote:
Hi David,
On Jul 10, 2009, at 11:06 AM, David Kilzer wrote:
So, does anyone think this would be a bad idea, or have any
alternate suggestions on how to improve things?
What about adding support for waf to gyp?
If we're talking about Chromium, waf is much faster for building
WebKit than scons is, so yeah, I would say adding the ability to use
it instead of scons is a no-brainer.
As for wx, while I'm open to the idea of using gyp to help others
out or simplify matters for the core devs, for us I'm pretty sure it
will mostly just be one more added layer to our build system. I've
been using Bakefile for years now and while I thought the idea was
great at first, I've found it mostly a pain to work with. Some of
that is due to Bakefile itself, of course (like the latest bug...),
but a lot of it is just the general approach and issues inherent
with adding one more tool to the toolchain. Even though there's some
sort of meta-tool generating the build projects, it only reduces,
not removes, the need to understand and deal with the native project
formats it outputs. Also, since the meta-tool can have bugs or
missing features too, there's an additional point of failure added,
and also added work in terms of maintaining the build system. Plus,
any time you want to add a feature, you should look into how to
implement it in the various formats that gyp supports to avoid
hacks, a process which can sometimes require pretty creative
solutions for less capable build tools. And there's also the fact
that, if wx uses gyp, people are going to start asking for support
for their favorite project format, which is just one more thing to
deal with.
In short, I think I've just come to be soured on the approach in
general. I think I've come to agree with the point that the waf site
makes about domain-specific languages not being a good idea for
build systems, and when you use a tool like Bakefile or gyp, you not
only need to become familiar with that tool's project file language,
but also to some level with the formats of the project files that it
outputs in order to debug or add new capabilities. With waf, all I
ever really need to know is Python, which I already know, and the
waf APIs, which I'm learning but for the most part have been pretty
simple and straightforward to work with. The bottom line for me is
that anything that has me spending less time on maintaining the
build system and more on the actual product is a good thing in my
book. ;-)
Your argument makes sense if WebKit is only built for one platform/
build-system. Unfortunately it's not. So the question is whether
it's easier to maintain lots of different build files or whether
it's easier to maintain one file that's perhaps a bit more abstract
+ the tool that interprets it.
I agree that working directly in the project file for your platform
is easier IF you're only developing for one platform. But once you
start maintaining more than one project file, I think GYP is a big
win.
While we hope that others will update our GYPI file when they add/
remove files, our build depends on it...so we'll definitely be
keeping it in sync. So I think the question then becomes whether
it's easier for you to maintain your new build format, or whether
it's easier to make it a target for GYP. I honestly don't know what
the answer is, but I think it's worth taking a closer look at GYP.
J
Thanks,
Kevin
The only format I'm not sure if we could automate > reliably would
be the XCode format (at least, on non-Mac
machines), because IIUC we'd need some sort of parser for it,
but Apple is the only port maintaining those directly IIUC,
as now Chromium will be using GYP to update their XCode
projects.
If you hand-edit Xcode project files enough times you start to
understand them, but you also may go insane in the process. I don't
know that you have to write a full parser for it, but there is some
non-trivial, minimal structure you have to understand to update the
file properly.
So even if we couldn't solve the XCode issue, that
would drop it to updating two locations tops.
I count 6 build systems in use currently (SCons support was added
and removed within the last year):
- Apple's Xcode
- Apple's vcproj (also used by at least one other Windows port)
- wx Bakefile (which will be replaced by waf soon)
- Qt Qmake
- GTK GNUMakefile
- Google's gyp (added recently)
Are any of the other ports going to switch to generating their build
files using gyp?
Dave
----- Original Message ----
From: Kevin Ollivier <kev...@theolliviers.com>
To: Dimitri Glazkov <dglaz...@chromium.org>
Cc: Mark Mentovai <m...@chromium.org>; WebKit Development <webkit-dev@lists.webkit.org
>
Sent: Friday, July 10, 2009 8:52:57 AM
Subject: [webkit-dev] Build File Maintenance (was Re: Please welcome
GYP to the our dysfunctional build family)
Hi Dimitri and all,
Congrats on getting this into WebKit! Actually, I'm in the middle of
a build
system switch as well - to waf, a re-write of scons that removed
many of the
performance issues related to searching and calculating
dependencies, and which
has added some nice features as well (such as .app bundle building).
I haven't
completed the switch so I can't do preliminary benchmarks, but I'm
pretty sure
it's actually as fast or faster than make on *nix/Mac. (And BTW, it
will
probably make Apple devs happy to hear that I'm no longer using the
horrid
build-wxwebkit bash script to manage the build, but instead have
integrated
everything into build-webkit finally!)
The main reason I bring this up, though, is because I think this
sort of thing
shows that we're unlikely to centralize our build systems any time
soon, and I
feel a bit sorry for the core devs who, as they accept new ports,
are probably
finding it more and more tedious, if not difficult, to make sure all
the
projects get updated by a change to the common parts of the build.
They've been
very helpful in terms of trying to keep the ports in shape when they
make
changes, and I feel like I'd like to do what I can to make it easier
and faster
to keep the other ports in sync.
At one point, I started on a script (located in
WebKitTools/Scripts/update-sources-list.py) whose idea was to take
the list of
common sources from one file, and make changes to MSVC, Qt, GTK,
etc. build
systems, so that WebKit devs need only add the file once, run the
script, and
commit the results. I got it as far as theoretically generating the
Qt and GTK
file lists, but I don't think anyone on those ports tried
integrating it into
their build system, and I sort of moved on to other things.
Unfortunately, right now I'm really swamped (my build system rewrite
was
prompted by WebKit exceeding internal, hardcoded, Bakefile limits,
not by
choice), but if a common location for the source files list could be
decided
upon, I really think a script would be a simple matter to write up
(even in Perl
:P ), and I think it would probably save developers time and reduce
build
breakages as well, which I think would add up to a lot of saved time
for a lot
of people over the long term. The only format I'm not sure if we
could automate
reliably would be the XCode format (at least, on non-Mac machines),
because IIUC
we'd need some sort of parser for it, but Apple is the only port
maintaining
those directly IIUC, as now Chromium will be using GYP to update
their XCode
projects. So even if we couldn't solve the XCode issue, that would
drop it to
updating two locations tops.
So, does anyone think this would be a bad idea, or have any alternate
suggestions on how to improve things? If not, is anyone willing to
take the ball
and run with it? I'd be willing to invest some more time in it, but
my ability
to commit to it over the next couple weeks at least would be very
limited.
Thanks,
Kevin
On Jul 9, 2009, at 9:23 PM, Dimitri Glazkov wrote:
Dear WebKiteurs,
In our persisting quest to be more like a common WebKit port, we have
added Chromium build files to the tree this afternoon. These files are
WebCore/WebCore.gypi and JavaScriptCore/JavaScriptCore.gypi and they
are the GYP include files. As you may know, we use GYP
(http://code.google.com/p/gyp) for generating MSVC, XCode, Scons, and
even Make projects for Chromium.
We are rather fond of GYP. Perhaps it is because it allows us to
maintain one set of project files for all three Chromium platforms;
or maybe because it lets us to do things like WebCore.gypi, where we
can just mindlessly add all project files to the list and then use
various neat GYP filtering facilities to narrow them down to sets that
are relevant for specific builds;
or maybe because it easifies creating cross-platform and
cross-build-system targets, actions, and rules;
or maybe because we just love saying "Gyp!"
I don't truthfully know.
What I do know is that starting now, we'd love for you to remember
WebCore.gypi and JavaScriptCore.gypi when you are adding or removing
files from WebCore or JavaScriptCore. Thanks to the power of GYP, you
don't have worry whether this file will be used by Chromium: the rule
is that if there's a project file change, it applies to the *.gypi
files. The format is very simple and intuitive, a simple Python/JSONey
list+dict.
Thank you for your attention, men and women of WebKit.
:DG<
_______________________________________________
webkit-dev mailing list
webkit-dev@lists.webkit.org
http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev
_______________________________________________
webkit-dev mailing list
webkit-dev@lists.webkit.org
http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev
_______________________________________________
webkit-dev mailing list
webkit-dev@lists.webkit.org
http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev
_______________________________________________
webkit-dev mailing list
webkit-dev@lists.webkit.org
http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev
_______________________________________________
webkit-dev mailing list
webkit-dev@lists.webkit.org
http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev
