Re: [webkit-dev] exporting symbols for building .so/.dll's
(From the right address again...)
On Sun, Mar 11, 2012 at 9:34 AM, Alp Toker a...@nuanti.com wrote:
On 09/03/2012 03:52, Ami Fischman wrote:
Hi webkittens,
The over-all question: how should webkit libraries declare which symbols
they export?
The trigger for the question: as described in bug 80062, the chromium
shared-library-based build links test code into the (non-test) libwebkit.so
target, which is terrible.
The details:
I took a crack at fixing the above bug (see patch in the bug) by pulling the
test files out of the non-test build target, and sprinkling various
{WTF,WEBKIT}_EXPORT{,DATA} macros around declarations that need them (as
discovered by build-time and run-time failures). This style of export
declaration is what the webkit codebase does in various places
(WTF, Platform, WebCore, and WebKit, AFAICT), and incidentally also what
the chromium project uses for its sub-components.
But I'm told other ports use different mechanisms such as .exp.in files for
apple/mac (and maybe others for other ports? IDK).
Is there consensus on the list for what the Right Thing To Do(tm) is?
ISTM my options are:
1) Add EXPORT declarations as in the patch on the bug.
2) Drop the patch from the bug and replace it with chromium-specific
.exp.in-style files, one per layer from which I need to export (WebCore,
WTF, WebKit). And build the build-time machinery to use them. I don't
really know what's involved here and would appreciate any pointers/hints
people had if this is the way to go.
3) Something else (preferably unifying other ports' approaches).
Help me webkit-dev, you're my only hope (for consensus).
I think the export macros would only ever have made sense if they were put
there explicitly to guide refactoring of the classes into a library /
interface structure. And this isn't the case.
At present I don't see an active effort towards that, or much interest in
defining the public interfaces in each 'module' more strictly. They're
intentionally fluid.
Having said that, the macros are /vaguely/ useful to see what could be made
private or hidden in future shuffling of the code in wtf, for example, but
that's about it.
The very fact that the export macros have to be updated with a tool every
time a library higher in the link chain uses or doesn't use a public entry
point, and that the set of imported functions or variables varies between
ports indicates that this is not going to have wide adoption.
This is same for port specific export lists.
To keep the tree green, the community needs to maintain the list
regardless it is externalized from or embedded in the source.
And maintaining the set of export lists has larger pain than
maintaining a macro-based annotation.
Thus keeping it isolated won't help us much.
, and that the set of imported functions or variables varies between
ports indicates that this is not going to have wide adoption.
As mrobinson@ mentioned, there is a set of the least common denominator.
My guess is that the number of symbols which is different between
ports will be less than a half of the total.
If we accept such overhead, it could be a preferable option.
If we follow this to the logical conclusion, no unification of granular
export lists is realistic with the current WebKit porting layer. If the
strategy were adapted to define exported functionality at class granularity,
it might just be feasible, but again that is a contract that is begging to
be broken, and besides, most toolchains lack export-by-class so it's a moot
point.
At least recent gcc+ld and link.exe support such a class level annotation.
For GNU chain, the lack of support seems like an old story:
http://gcc.gnu.org/bugzilla/show_bug.cgi?id=9283
My feeling is that the macro annotation is never for declaring a public API.
It's just a pretty convenient replacement of the port specific export
lists, with small overhead from possible unwanted exports.
It's different from WebKit API which each port is providing.
There is no commitment for compatibility, as the _PRIVATE suffix indicates.
To clarify: I think that there should be WEBCORE_EXPORT_PRIVATE but no
WEBCORE_EXORT.
This is unlike JSC, which has both JS_EXPORT and JS_EXPORT_PRIVATE.
(I prefer some shorter name btw... but it's a different topic.)
So the ultimate course of action is then to revert the macros and leave
everyone to do what they think best in terms of export lists, then tying
together those solutions where there's overlap.
So, as I wrote above, this course seems based on a wrong assumption:
The annotation as a public API.
No, it won't. It could be fluid. it would be just an internal tool
which only WebKit port implementations can rely on.
Does this make sense?
--
morrita
___
webkit-dev mailing list
webkit-dev@lists.webkit.org
http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev
Re: [webkit-dev] exporting symbols for building .so/.dll's
I'm inclined to give this thread one more business day and then call it
Tuesday morning pacific time.
On current showing I think the approach in the patch on the bug (incl.
morrita@'s preference for WEBCORE_EXPORT_PRIVATE) is the way to go, at
least for now.
If you have strong concerns please speak up before then.
Cheers,
-a
On Sun, Mar 11, 2012 at 3:02 AM, Hajime Morrita morr...@chromium.orgwrote:
(From the right address again...)
On Sun, Mar 11, 2012 at 9:34 AM, Alp Toker a...@nuanti.com wrote:
On 09/03/2012 03:52, Ami Fischman wrote:
Hi webkittens,
The over-all question: how should webkit libraries declare which symbols
they export?
The trigger for the question: as described in bug 80062, the chromium
shared-library-based build links test code into the (non-test)
libwebkit.so
target, which is terrible.
The details:
I took a crack at fixing the above bug (see patch in the bug) by pulling
the
test files out of the non-test build target, and sprinkling various
{WTF,WEBKIT}_EXPORT{,DATA} macros around declarations that need them (as
discovered by build-time and run-time failures). This style of export
declaration is what the webkit codebase does in various places
(WTF, Platform, WebCore, and WebKit, AFAICT), and incidentally also what
the chromium project uses for its sub-components.
But I'm told other ports use different mechanisms such as .exp.infiles for
apple/mac (and maybe others for other ports? IDK).
Is there consensus on the list for what the Right Thing To Do(tm) is?
ISTM my options are:
1) Add EXPORT declarations as in the patch on the bug.
2) Drop the patch from the bug and replace it with chromium-specific
.exp.in-style files, one per layer from which I need to export (WebCore,
WTF, WebKit). And build the build-time machinery to use them. I don't
really know what's involved here and would appreciate any pointers/hints
people had if this is the way to go.
3) Something else (preferably unifying other ports' approaches).
Help me webkit-dev, you're my only hope (for consensus).
I think the export macros would only ever have made sense if they were
put
there explicitly to guide refactoring of the classes into a library /
interface structure. And this isn't the case.
At present I don't see an active effort towards that, or much interest in
defining the public interfaces in each 'module' more strictly. They're
intentionally fluid.
Having said that, the macros are /vaguely/ useful to see what could be
made
private or hidden in future shuffling of the code in wtf, for example,
but
that's about it.
The very fact that the export macros have to be updated with a tool every
time a library higher in the link chain uses or doesn't use a public
entry
point, and that the set of imported functions or variables varies between
ports indicates that this is not going to have wide adoption.
This is same for port specific export lists.
To keep the tree green, the community needs to maintain the list
regardless it is externalized from or embedded in the source.
And maintaining the set of export lists has larger pain than
maintaining a macro-based annotation.
Thus keeping it isolated won't help us much.
, and that the set of imported functions or variables varies between
ports indicates that this is not going to have wide adoption.
As mrobinson@ mentioned, there is a set of the least common denominator.
My guess is that the number of symbols which is different between
ports will be less than a half of the total.
If we accept such overhead, it could be a preferable option.
If we follow this to the logical conclusion, no unification of granular
export lists is realistic with the current WebKit porting layer. If the
strategy were adapted to define exported functionality at class
granularity,
it might just be feasible, but again that is a contract that is begging
to
be broken, and besides, most toolchains lack export-by-class so it's a
moot
point.
At least recent gcc+ld and link.exe support such a class level annotation.
For GNU chain, the lack of support seems like an old story:
http://gcc.gnu.org/bugzilla/show_bug.cgi?id=9283
My feeling is that the macro annotation is never for declaring a public
API.
It's just a pretty convenient replacement of the port specific export
lists, with small overhead from possible unwanted exports.
It's different from WebKit API which each port is providing.
There is no commitment for compatibility, as the _PRIVATE suffix
indicates.
To clarify: I think that there should be WEBCORE_EXPORT_PRIVATE but no
WEBCORE_EXORT.
This is unlike JSC, which has both JS_EXPORT and JS_EXPORT_PRIVATE.
(I prefer some shorter name btw... but it's a different topic.)
So the ultimate course of action is then to revert the macros and leave
everyone to do what they think best in terms of export lists, then tying
together those solutions where
Re: [webkit-dev] exporting symbols for building .so/.dll's
On Sun, Mar 11, 2012 at 10:49 AM, Ami Fischman fisch...@chromium.org wrote:
I'm inclined to give this thread one more business day and then call it
Tuesday morning pacific time.
On current showing I think the approach in the patch on the bug (incl.
morrita@'s preference for WEBCORE_EXPORT_PRIVATE) is the way to go, at least
for now.
If you have strong concerns please speak up before then.
I'm not sure I understand your proposal fully. Specifically, how
would these macros work for, say, the Chromium, Apple-Mac, and
Apple-Win ports, which export overlapping, but not identical, sets of
symbols?
Adam
On Sun, Mar 11, 2012 at 3:02 AM, Hajime Morrita morr...@chromium.org
wrote:
(From the right address again...)
On Sun, Mar 11, 2012 at 9:34 AM, Alp Toker a...@nuanti.com wrote:
On 09/03/2012 03:52, Ami Fischman wrote:
Hi webkittens,
The over-all question: how should webkit libraries declare which symbols
they export?
The trigger for the question: as described in bug 80062, the chromium
shared-library-based build links test code into the (non-test)
libwebkit.so
target, which is terrible.
The details:
I took a crack at fixing the above bug (see patch in the bug) by pulling
the
test files out of the non-test build target, and sprinkling various
{WTF,WEBKIT}_EXPORT{,DATA} macros around declarations that need them (as
discovered by build-time and run-time failures). This style of export
declaration is what the webkit codebase does in various places
(WTF, Platform, WebCore, and WebKit, AFAICT), and incidentally also what
the chromium project uses for its sub-components.
But I'm told other ports use different mechanisms such as .exp.in
files for
apple/mac (and maybe others for other ports? IDK).
Is there consensus on the list for what the Right Thing To Do(tm) is?
ISTM my options are:
1) Add EXPORT declarations as in the patch on the bug.
2) Drop the patch from the bug and replace it with chromium-specific
.exp.in-style files, one per layer from which I need to export (WebCore,
WTF, WebKit). And build the build-time machinery to use them. I don't
really know what's involved here and would appreciate any pointers/hints
people had if this is the way to go.
3) Something else (preferably unifying other ports' approaches).
Help me webkit-dev, you're my only hope (for consensus).
I think the export macros would only ever have made sense if they were
put
there explicitly to guide refactoring of the classes into a library /
interface structure. And this isn't the case.
At present I don't see an active effort towards that, or much interest
in
defining the public interfaces in each 'module' more strictly. They're
intentionally fluid.
Having said that, the macros are /vaguely/ useful to see what could be
made
private or hidden in future shuffling of the code in wtf, for example,
but
that's about it.
The very fact that the export macros have to be updated with a tool
every
time a library higher in the link chain uses or doesn't use a public
entry
point, and that the set of imported functions or variables varies
between
ports indicates that this is not going to have wide adoption.
This is same for port specific export lists.
To keep the tree green, the community needs to maintain the list
regardless it is externalized from or embedded in the source.
And maintaining the set of export lists has larger pain than
maintaining a macro-based annotation.
Thus keeping it isolated won't help us much.
, and that the set of imported functions or variables varies between
ports indicates that this is not going to have wide adoption.
As mrobinson@ mentioned, there is a set of the least common denominator.
My guess is that the number of symbols which is different between
ports will be less than a half of the total.
If we accept such overhead, it could be a preferable option.
If we follow this to the logical conclusion, no unification of granular
export lists is realistic with the current WebKit porting layer. If the
strategy were adapted to define exported functionality at class
granularity,
it might just be feasible, but again that is a contract that is begging
to
be broken, and besides, most toolchains lack export-by-class so it's a
moot
point.
At least recent gcc+ld and link.exe support such a class level annotation.
For GNU chain, the lack of support seems like an old story:
http://gcc.gnu.org/bugzilla/show_bug.cgi?id=9283
My feeling is that the macro annotation is never for declaring a public
API.
It's just a pretty convenient replacement of the port specific export
lists, with small overhead from possible unwanted exports.
It's different from WebKit API which each port is providing.
There is no commitment for compatibility, as the _PRIVATE suffix
indicates.
To clarify: I think that there should be WEBCORE_EXPORT_PRIVATE but no
WEBCORE_EXORT.
This is
Re: [webkit-dev] exporting symbols for building .so/.dll's
On Sun, Mar 11, 2012 at 11:13 AM, Adam Barth aba...@webkit.org wrote:
On Sun, Mar 11, 2012 at 10:49 AM, Ami Fischman fisch...@chromium.org wrote:
I'm inclined to give this thread one more business day and then call it
Tuesday morning pacific time.
On current showing I think the approach in the patch on the bug (incl.
morrita@'s preference for WEBCORE_EXPORT_PRIVATE) is the way to go, at least
for now.
If you have strong concerns please speak up before then.
I'm not sure I understand your proposal fully. Specifically, how
would these macros work for, say, the Chromium, Apple-Mac, and
Apple-Win ports, which export overlapping, but not identical, sets of
symbols?
Relatedly, do you plan to replace
http://trac.webkit.org/browser/trunk/Source/WebCore/WebCore.exp.in
with EXPORT macros, or will we have both macros and an export list for
an extended period of time?
Adam
On Sun, Mar 11, 2012 at 3:02 AM, Hajime Morrita morr...@chromium.org
wrote:
(From the right address again...)
On Sun, Mar 11, 2012 at 9:34 AM, Alp Toker a...@nuanti.com wrote:
On 09/03/2012 03:52, Ami Fischman wrote:
Hi webkittens,
The over-all question: how should webkit libraries declare which symbols
they export?
The trigger for the question: as described in bug 80062, the chromium
shared-library-based build links test code into the (non-test)
libwebkit.so
target, which is terrible.
The details:
I took a crack at fixing the above bug (see patch in the bug) by pulling
the
test files out of the non-test build target, and sprinkling various
{WTF,WEBKIT}_EXPORT{,DATA} macros around declarations that need them (as
discovered by build-time and run-time failures). This style of export
declaration is what the webkit codebase does in various places
(WTF, Platform, WebCore, and WebKit, AFAICT), and incidentally also what
the chromium project uses for its sub-components.
But I'm told other ports use different mechanisms such as .exp.in
files for
apple/mac (and maybe others for other ports? IDK).
Is there consensus on the list for what the Right Thing To Do(tm) is?
ISTM my options are:
1) Add EXPORT declarations as in the patch on the bug.
2) Drop the patch from the bug and replace it with chromium-specific
.exp.in-style files, one per layer from which I need to export (WebCore,
WTF, WebKit). And build the build-time machinery to use them. I don't
really know what's involved here and would appreciate any pointers/hints
people had if this is the way to go.
3) Something else (preferably unifying other ports' approaches).
Help me webkit-dev, you're my only hope (for consensus).
I think the export macros would only ever have made sense if they were
put
there explicitly to guide refactoring of the classes into a library /
interface structure. And this isn't the case.
At present I don't see an active effort towards that, or much interest
in
defining the public interfaces in each 'module' more strictly. They're
intentionally fluid.
Having said that, the macros are /vaguely/ useful to see what could be
made
private or hidden in future shuffling of the code in wtf, for example,
but
that's about it.
The very fact that the export macros have to be updated with a tool
every
time a library higher in the link chain uses or doesn't use a public
entry
point, and that the set of imported functions or variables varies
between
ports indicates that this is not going to have wide adoption.
This is same for port specific export lists.
To keep the tree green, the community needs to maintain the list
regardless it is externalized from or embedded in the source.
And maintaining the set of export lists has larger pain than
maintaining a macro-based annotation.
Thus keeping it isolated won't help us much.
, and that the set of imported functions or variables varies between
ports indicates that this is not going to have wide adoption.
As mrobinson@ mentioned, there is a set of the least common denominator.
My guess is that the number of symbols which is different between
ports will be less than a half of the total.
If we accept such overhead, it could be a preferable option.
If we follow this to the logical conclusion, no unification of granular
export lists is realistic with the current WebKit porting layer. If the
strategy were adapted to define exported functionality at class
granularity,
it might just be feasible, but again that is a contract that is begging
to
be broken, and besides, most toolchains lack export-by-class so it's a
moot
point.
At least recent gcc+ld and link.exe support such a class level annotation.
For GNU chain, the lack of support seems like an old story:
http://gcc.gnu.org/bugzilla/show_bug.cgi?id=9283
My feeling is that the macro annotation is never for declaring a public
API.
It's just a pretty convenient replacement of the port specific export
lists, with small
Re: [webkit-dev] exporting symbols for building .so/.dll's
On 11/03/2012 18:13, Adam Barth wrote: On Sun, Mar 11, 2012 at 10:49 AM, Ami Fischmanfisch...@chromium.org wrote: I'm inclined to give this thread one more business day and then call it Tuesday morning pacific time. On current showing I think the approach in the patch on the bug (incl. morrita@'s preference for WEBCORE_EXPORT_PRIVATE) is the way to go, at least for now. If you have strong concerns please speak up before then. I'm not sure I understand your proposal fully. Specifically, how would these macros work for, say, the Chromium, Apple-Mac, and Apple-Win ports, which export overlapping, but not identical, sets of symbols? Adam I have some questions to add to Adam's. Doesn't the list of exports change massively based on the level of inlining and compiler optimisation? Which configuration is the target to aim for, and which compiler vendor? This should be documented. Doesn't the list also change between releases of the same compiler? If so, are the export lists still really a better option than, say, generating the export lists by analysis of nm/objdump at compile time, or maintaining export lists manually? If the export macros are enforced, will it become the responsibility of patch authors or committers to choose the correct places to add them? If so, such a responsibility should also be documented. -- http://www.nuanti.com the browser experts ___ webkit-dev mailing list webkit-dev@lists.webkit.org http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev
Re: [webkit-dev] exporting symbols for building .so/.dll's
Responding to several emails below. Adam wrote: I'm not sure I understand your proposal fully. Specifically, how would these macros work for, say, the Chromium, Apple-Mac, and Apple-Win ports, which export overlapping, but not identical, sets of symbols? I do not have a proposal to unify the export story across ports. My proposal (such as it is) is limited to removing test code from the non-test webkit target (see the patch on the bug). AFAICT from this thread there is no consensus on unifying options that doesn't have as its first step achieve consensus on the complete public API of each library layer, and I didn't get the sense that this is a realistic goal, at least for me at my current level of engagement. Adam also wrote: Relatedly, do you plan to replace http://trac.webkit.org/browser/trunk/Source/WebCore/WebCore.exp.in with EXPORT macros, or will we have both macros and an export list for an extended period of time? The latter. To put things in perspective, WebCore.exp.in is over 2000 lines long; my patch macro-annotates ~40 classes in WebCore (which is a far cry from, say, doubling the pain of WebCore.exp.in). I'm also emboldened in this respect by the coexistence of WEBKIT_EXPORT and Source/WebKit/mac/WebKit.exphttp://code.google.com/p/chromium/source/search?q=file%3AWebKit%2FSource%2FWebKit%2Fmac%2FWebKit.exporigq=file%3AWebKit%2FSource%2FWebKit%2Fmac%2FWebKit.expbtnG=Search+Trunk . Alp wrote: Doesn't the list of exports change massively based on the level of inlining and compiler optimisation? Which configuration is the target to aim for, and which compiler vendor? Doesn't the list also change between releases of the same compiler? AFAICT the answers to your questions are all no, for the limited scope of my change (some indication of this is that I only tested my patch on chromium/linux, but it's now green on all EWS bots after only needing to add #include's to get the definition of the export macro, not due to having to add extra annotations for some ports). As with the existing JSC_EXPORT and WEBKIT_EXPORT macros, the new WEBCORE_EXPORT_PRIVATE (nee (unused) WEBKIT_EXPORTDATA) will be an annotation added by whoever lands a patch that requires access to something from outside the library that today doesn't require it. Ash proposed replacing both the explicit post-mangled export lists and in-line macros with per-port export-lists that would be used at build time to emit modified headers. I'd have to see a PoC of this to believe that it wouldn't combine the worst aspects of all the options on the table ;) The proposal assumes a lot of things I don't believe are *strictly* true, and I believe the gap between almost right and strictly right is where enough pain will come from to sink this option. Cheers, -a ___ webkit-dev mailing list webkit-dev@lists.webkit.org http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev
Re: [webkit-dev] exporting symbols for building .so/.dll's
I found some confusion here. There are two kinds of symbols which need to be exported from WebCore. - A. Symbols from WebCore to WebKit(API). This is basically what WebCore.exp.in covers. - B. Symbols from WebCore to Application. In principle, there should be no such symbols. But in reality there is some. Symbols for Internals objects follow this pattern. Note that the number of B is significantly smaller than one of A. It looks Alp's main concern is about B. But I implicitly assumed A. Currently A matters only for Mac WK1 port (Webcore.exp.in). Some other ports including GTK, WebKit2 needs B. (symbols.fiter, WebKit_Cairo.def, etc.) Mac WK1 also needs B. But it's in the same WebCore.exp.in list. For A, it should be marked as WEBCORE_EXPORT_PRIVATE if we use inline annotations. For B, it should use WEBCORE_EXPORT. For A, I think it's infeasible to replace it with inline annotations because the .exp list and inline annotations are mutually exclusive. So we need to make it happen at once. But it's not a simple job. I think we need some incremental transition plan. It might be someting like what Alp mentioned, that is, to use nm or whatever to automate exp list generation, etc. For B, it's relatively easy to turn the macro on because its number is small. The symbols.filter file shows about 100 symbols, which would be manually convertible. What's tricky is that Chromium component build wants yet another kind of symbols: The symbols is for unit testing. Let me call it C otpion here. C is different from either A or B. It touches deep WebCore internals which people don't need to export unless they write unit tests, that is current WebKit situation. So, - Some ports need A+B (ex. Mac WK1 Port) - Some ports need B (ex. GTK, Windows) - Some ports need B+C (Chromium) And my feeling is that Chromium is OK to have A if it's available. Having that said, here is my proposal: - Let's introduce WEBCORE_EXPORT for B. Chromium will use this, GTK and Windows also could use. (For GTK, I expect this to have same definition as WEBKIT_API.) - Let's NOT introduce WEBCORE_EXPORT_PRIVATE for now because it would be useless unless we don't get rid of WebCore.exp.in, which won't easy. - Let's introduce WEBCORE_EXPORT_TEST for C. Chromium will enable and use this for now. - If WEBCORE_EXPORT and WEBCORE_EXPORT_TEST conflicts, WEBCORE_EXPORT win. - If WEBCORE_EXPORT_PRIVATE and WEBCORE_EXPORT_TEST conflicts, WEBCORE_EXPORT_TEST win. - If WEBCORE_EXPORT and WEBCORE_EXPORT_PRIVATE conflicts (in the future), WEBCORE_EXPORT win. Then we can start from adding C for Chromium, which doesn't have any export list before. Chromium will also need B. Having B would be good news also for GTK and Windows port because these no longer need to touch their symbols file for each time the Internals object needs one. I'm happy to help this to happen. (... and I'd like to ask Ami to help ;-)) What do you think? -- morrita On Mon, Mar 12, 2012 at 9:16 AM, Ami Fischman fisch...@chromium.org wrote: Responding to several emails below. Adam wrote: I'm not sure I understand your proposal fully. Specifically, how would these macros work for, say, the Chromium, Apple-Mac, and Apple-Win ports, which export overlapping, but not identical, sets of symbols? I do not have a proposal to unify the export story across ports. My proposal (such as it is) is limited to removing test code from the non-test webkit target (see the patch on the bug). AFAICT from this thread there is no consensus on unifying options that doesn't have as its first step achieve consensus on the complete public API of each library layer, and I didn't get the sense that this is a realistic goal, at least for me at my current level of engagement. Adam also wrote: Relatedly, do you plan to replace http://trac.webkit.org/browser/trunk/Source/WebCore/WebCore.exp.in with EXPORT macros, or will we have both macros and an export list for an extended period of time? The latter. To put things in perspective, WebCore.exp.in is over 2000 lines long; my patch macro-annotates ~40 classes in WebCore (which is a far cry from, say, doubling the pain of WebCore.exp.in). I'm also emboldened in this respect by the coexistence of WEBKIT_EXPORT and Source/WebKit/mac/WebKit.exp. Alp wrote: Doesn't the list of exports change massively based on the level of inlining and compiler optimisation? Which configuration is the target to aim for, and which compiler vendor? Doesn't the list also change between releases of the same compiler? AFAICT the answers to your questions are all no, for the limited scope of my change (some indication of this is that I only tested my patch on chromium/linux, but it's now green on all EWS bots after only needing to add #include's to get the definition of the export macro, not due to having to add extra annotations for some ports). As with the existing JSC_EXPORT and WEBKIT_EXPORT macros, the new
Re: [webkit-dev] exporting symbols for building .so/.dll's
On Sun, Mar 11, 2012 at 5:16 PM, Ami Fischman fisch...@chromium.org wrote: Adam wrote: I'm not sure I understand your proposal fully. Specifically, how would these macros work for, say, the Chromium, Apple-Mac, and Apple-Win ports, which export overlapping, but not identical, sets of symbols? I do not have a proposal to unify the export story across ports. My proposal (such as it is) is limited to removing test code from the non-test webkit target (see the patch on the bug). It's important to consider how these approaches work for ports besides Chromium. WebCore is used by many folks besides just Chromium. AFAICT from this thread there is no consensus on unifying options that doesn't have as its first step achieve consensus on the complete public API of each library layer, and I didn't get the sense that this is a realistic goal, at least for me at my current level of engagement. WebCore has no public API, so that part is easy to reach consensus about. :) On Sun, Mar 11, 2012 at 5:54 PM, Hajime Morrita morr...@chromium.org wrote: Having that said, here is my proposal: - Let's introduce WEBCORE_EXPORT for B. Chromium will use this, GTK and Windows also could use. (For GTK, I expect this to have same definition as WEBKIT_API.) - Let's NOT introduce WEBCORE_EXPORT_PRIVATE for now because it would be useless unless we don't get rid of WebCore.exp.in, which won't easy. - Let's introduce WEBCORE_EXPORT_TEST for C. Chromium will enable and use this for now. [...] What do you think? This approach sounds promising because the symbols needed for Internals and for unit testing are likely to be quite similar across ports (as opposed to, for example, the WebCore symbols needed by the various different WebKit layers). That means that the symbols annotated with WEBCORE_EXPORT and WEBCORE_EXPORT_TEST are likely to be exported by many ports. Adam ___ webkit-dev mailing list webkit-dev@lists.webkit.org http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev
Re: [webkit-dev] exporting symbols for building .so/.dll's
Hajime I chatted and it became clear that we've been miscommunicating. I'm taking that conversation off webkit-dev and putting this thread on hold pending that resolving. Sorry for the noise. Cheers, -a On Sun, Mar 11, 2012 at 5:54 PM, Hajime Morrita morr...@chromium.orgwrote: I found some confusion here. There are two kinds of symbols which need to be exported from WebCore. - A. Symbols from WebCore to WebKit(API). This is basically what WebCore.exp.in covers. - B. Symbols from WebCore to Application. In principle, there should be no such symbols. But in reality there is some. Symbols for Internals objects follow this pattern. Note that the number of B is significantly smaller than one of A. It looks Alp's main concern is about B. But I implicitly assumed A. Currently A matters only for Mac WK1 port (Webcore.exp.in). Some other ports including GTK, WebKit2 needs B. (symbols.fiter, WebKit_Cairo.def, etc.) Mac WK1 also needs B. But it's in the same WebCore.exp.in list. For A, it should be marked as WEBCORE_EXPORT_PRIVATE if we use inline annotations. For B, it should use WEBCORE_EXPORT. For A, I think it's infeasible to replace it with inline annotations because the .exp list and inline annotations are mutually exclusive. So we need to make it happen at once. But it's not a simple job. I think we need some incremental transition plan. It might be someting like what Alp mentioned, that is, to use nm or whatever to automate exp list generation, etc. For B, it's relatively easy to turn the macro on because its number is small. The symbols.filter file shows about 100 symbols, which would be manually convertible. What's tricky is that Chromium component build wants yet another kind of symbols: The symbols is for unit testing. Let me call it C otpion here. C is different from either A or B. It touches deep WebCore internals which people don't need to export unless they write unit tests, that is current WebKit situation. So, - Some ports need A+B (ex. Mac WK1 Port) - Some ports need B (ex. GTK, Windows) - Some ports need B+C (Chromium) And my feeling is that Chromium is OK to have A if it's available. Having that said, here is my proposal: - Let's introduce WEBCORE_EXPORT for B. Chromium will use this, GTK and Windows also could use. (For GTK, I expect this to have same definition as WEBKIT_API.) - Let's NOT introduce WEBCORE_EXPORT_PRIVATE for now because it would be useless unless we don't get rid of WebCore.exp.in, which won't easy. - Let's introduce WEBCORE_EXPORT_TEST for C. Chromium will enable and use this for now. - If WEBCORE_EXPORT and WEBCORE_EXPORT_TEST conflicts, WEBCORE_EXPORT win. - If WEBCORE_EXPORT_PRIVATE and WEBCORE_EXPORT_TEST conflicts, WEBCORE_EXPORT_TEST win. - If WEBCORE_EXPORT and WEBCORE_EXPORT_PRIVATE conflicts (in the future), WEBCORE_EXPORT win. Then we can start from adding C for Chromium, which doesn't have any export list before. Chromium will also need B. Having B would be good news also for GTK and Windows port because these no longer need to touch their symbols file for each time the Internals object needs one. I'm happy to help this to happen. (... and I'd like to ask Ami to help ;-)) What do you think? -- morrita On Mon, Mar 12, 2012 at 9:16 AM, Ami Fischman fisch...@chromium.org wrote: Responding to several emails below. Adam wrote: I'm not sure I understand your proposal fully. Specifically, how would these macros work for, say, the Chromium, Apple-Mac, and Apple-Win ports, which export overlapping, but not identical, sets of symbols? I do not have a proposal to unify the export story across ports. My proposal (such as it is) is limited to removing test code from the non-test webkit target (see the patch on the bug). AFAICT from this thread there is no consensus on unifying options that doesn't have as its first step achieve consensus on the complete public API of each library layer, and I didn't get the sense that this is a realistic goal, at least for me at my current level of engagement. Adam also wrote: Relatedly, do you plan to replace http://trac.webkit.org/browser/trunk/Source/WebCore/WebCore.exp.in with EXPORT macros, or will we have both macros and an export list for an extended period of time? The latter. To put things in perspective, WebCore.exp.in is over 2000 lines long; my patch macro-annotates ~40 classes in WebCore (which is a far cry from, say, doubling the pain of WebCore.exp.in). I'm also emboldened in this respect by the coexistence of WEBKIT_EXPORT and Source/WebKit/mac/WebKit.exp. Alp wrote: Doesn't the list of exports change massively based on the level of inlining and compiler optimisation? Which configuration is the target to aim for, and which compiler vendor? Doesn't the list also change between releases of the same compiler? AFAICT the answers to your
Re: [webkit-dev] exporting symbols for building .so/.dll's
On 09/03/2012 03:52, Ami Fischman wrote:
Hi webkittens,
The over-all question: how should webkit libraries declare which
symbols they export?
The trigger for the question: as described in bug 80062
https://bugs.webkit.org/show_bug.cgi?id=80062, the chromium
shared-library-based build links test code into the (non-test)
libwebkit.so target, which is terrible.
The details:
I took a crack at fixing the above bug (see patch in the bug) by
pulling the test files out of the non-test build target, and
sprinkling various {WTF,WEBKIT}_EXPORT{,DATA} macros around
declarations that need them (as discovered by build-time and run-time
failures). This style of export declaration is what the webkit
codebase does in various places (WTF
http://code.google.com/codesearch#OAMlx_jo-ck/src/third_party/WebKit/Source/JavaScriptCore/wtf/ExportMacros.hexact_package=chromiumq=WTF_EXPORTtype=csl=39,
Platform
http://code.google.com/codesearch#OAMlx_jo-ck/src/third_party/WebKit/Source/Platform/chromium/public/WebCommon.hexact_package=chromiumq=WEBKIT_EXPORTl=68,
WebCore
http://code.google.com/codesearch#OAMlx_jo-ck/src/third_party/WebKit/Source/WebCore/platform/PlatformExportMacros.hexact_package=chromiumq=define%5C%20WEBKIT_EXPORTct=rccd=2sq=l=39,
and WebKit
http://code.google.com/codesearch#OAMlx_jo-ck/src/third_party/WebKit/Source/WebKit/chromium/public/platform/WebCommon.hexact_package=chromiumq=file:%28%5E%7C/%29platform/WebCommon%5C.h$,
AFAICT), and incidentally also what the chromium project uses for its
sub-components.
But I'm told other ports use different mechanisms such as .exp.in
files
http://code.google.com/p/chromium/source/search?q=file%3ASource%2FWebCore%2FWebCore.exp.inorigq=file%3ASource%2FWebCore%2FWebCore.exp.inbtnG=Search+Trunk for
apple/mac (and maybe others for other ports? IDK).
Is there consensus on the list for what the Right Thing To Do(tm) is?
ISTM my options are:
1) Add EXPORT declarations as in the patch on the bug.
2) Drop the patch from the bug and replace it with chromium-specific
.exp.in-style files, one per layer from which I need to export
(WebCore, WTF, WebKit). And build the build-time machinery to use
them. I don't really know what's involved here and would appreciate
any pointers/hints people had if this is the way to go.
3) Something else (preferably unifying other ports' approaches).
Help me webkit-dev, you're my only hope (for consensus).
I think the export macros would only ever have made sense if they were
put there explicitly to guide refactoring of the classes into a library
/ interface structure. And this isn't the case.
At present I don't see an active effort towards that, or much interest
in defining the public interfaces in each 'module' more strictly.
They're intentionally fluid.
Having said that, the macros are /vaguely/ useful to see what could be
made private or hidden in future shuffling of the code in wtf, for
example, but that's about it.
The very fact that the export macros have to be updated with a tool
every time a library higher in the link chain uses or doesn't use a
public entry point, and that the set of imported functions or variables
varies between ports indicates that this is not going to have wide adoption.
If we follow this to the logical conclusion, no unification of granular
export lists is realistic with the current WebKit porting layer. If the
strategy were adapted to define exported functionality at class
granularity, it might just be feasible, but again that is a contract
that is begging to be broken, and besides, most toolchains lack
export-by-class so it's a moot point.
So the ultimate course of action is then to revert the macros and leave
everyone to do what they think best in terms of export lists, then tying
together those solutions where there's overlap.
The exception is, of course, clearly defined public API (of which there
is not much), such as this case where we added JS_EXPORT to the
JavaScriptCore API for the benefit of multiple ports and also consumers:
http://trac.webkit.org/changeset/28097
(In a port I worked on in the past we developed a vendor tool to detect
inter-dependencies at compile time so there were no lists to update, but
again, this would not be portable.)
Spoken with my devil's advocate hat on, would be great if you can prove
me wrong.
--
http://www.nuanti.com
the browser experts
___
webkit-dev mailing list
webkit-dev@lists.webkit.org
http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev
Re: [webkit-dev] exporting symbols for building .so/.dll's
On Sat, Mar 10, 2012 at 4:34 PM, Alp Toker a...@nuanti.com wrote: If we follow this to the logical conclusion, no unification of granular export lists is realistic with the current WebKit porting layer. If the strategy were adapted to define exported functionality at class granularity, it might just be feasible, but again that is a contract that is begging to be broken, and besides, most toolchains lack export-by-class so it's a moot point. While it seems that ports require a different set of exported macros, there are certain subsets that all ports *do* need. One example of this are symbols necessary for WebCoreInternals. I think this is a great effort, because it improves test coverage for ports that don't necessarily have the time to keep DumpRenderTree up to date. I watch a lot of the WebCoreInternals bugs and I see that maintaining export lists for different platforms, all which have their own unique way of mangling symbols, is a huge burden. So the ultimate course of action is then to revert the macros and leave everyone to do what they think best in terms of export lists, then tying together those solutions where there's overlap. I'm not sure I necessarily agree that the only answer to this problem is to continue to force all ports to maintain their own export lists. If the macros don't work out (and I'm not convinced that they cannot), then we could simply teach the tools how to mangle symbols and make a unified list of exports. Perhaps that's what you meant by tying together those solutions though. --Martin ___ webkit-dev mailing list webkit-dev@lists.webkit.org http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev
Re: [webkit-dev] exporting symbols for building .so/.dll's
Thanks for your reply, Morrita! For ports on approach C, it doesn't matter which WebCore/JSC API is used from WebKit API layer because they are built in the same library. If you mean that nothing in WebCore/JSC needs to be annotated as exported then I don't think that's right, because libwebkit.so has various other libwebcore_*.a's linked into it. E.g. if I revert the change I made to JavaScriptCore/wtf/chromium/ChromiumThreading.h to *not* have ChromiumThreading be declared WTF_EXPORT, then the link of webkit_unit_tests fails because libwtf.a (JavaScriptCore/wtf/chromium/MainThreadChromium.cpp) contains a call to WTF::ChromiumThreading::callOnMainThread which is a hidden symbol in libwebkit.so (WebKit/chromium/src/ChromiumThreading.cpp). Maybe I misunderstood your point? For JSC API, which A and B folks care about, there are two ways to export it. - For Windows port, there is JavaScriptCore.def. - For Mac (and WX?) port, there are macro-based annotations like JS_EXPORT, WTF_EXPORT_PRIVATE. I'm in the way to eliminate JavaScriptCore.def by replacing it with these annotations, but the migration isn't finished yet. (http://wkb.ug/76257) Yay. For WebCore API, we only need to care about WebCore.exp.in because Mac port is the only one which follows A pattern. (I remember Windows port used to use this pattern. But it looks no longer the case. Maybe they changed the way when they switched to WebKit2. But my memory might be just wrong...) I'm curious whether you have plans (or know of others' plans) to replace WebCore.exp.in in favor of macros, like you did in http://webk.it/72854 for JavaScriptCore.exp. My personal preference is to introduce WebCore version of export macros like WK_EXPORT_PRIVATE. In this way, we could use it to get rid of WebCore.exp.in in the future. Isn't that what WEBKIT_EXPORTDATAhttp://cs/#chrome/src/third_party/WebKit/Source/WebCore/platform/PlatformExportMacros.hq=file:WebCore/platform/PlatformExportMacros.h%20WEBKIT_EXPORTDATAl=39 already is? (contrary to the macro's name, it lives in WebCore; perhaps it is trickier than I realize and somehow only applies to WebKit code even though it lives in WebCore? Or maybe it just needs to be renamed?) Yet another approach could be just giving up to have the unit test as a separate binary and build it into the library for non-prod configuration. The last one seems exactly what you are trying to avoid though... Right; linking test code into the main library is what HEAD does now. Anyway, I'd like to hear thoughts from other folks. Me too. As someone pointed out to me, others might be too busy with the git thread to reply to this one :) Cheers, -a ___ webkit-dev mailing list webkit-dev@lists.webkit.org http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev
Re: [webkit-dev] exporting symbols for building .so/.dll's
On Thu, Mar 8, 2012 at 7:52 PM, Ami Fischman fisch...@chromium.org wrote:
Hi webkittens,
The over-all question: how should webkit libraries declare which symbols
they export?
The trigger for the question: as described in bug 80062, the chromium
shared-library-based build links test code into the (non-test) libwebkit.so
target, which is terrible.
The details:
I took a crack at fixing the above bug (see patch in the bug) by pulling the
test files out of the non-test build target, and sprinkling various
{WTF,WEBKIT}_EXPORT{,DATA} macros around declarations that need them (as
discovered by build-time and run-time failures). This style of export
declaration is what the webkit codebase does in various places
(WTF, Platform, WebCore, and WebKit, AFAICT), and incidentally also what
the chromium project uses for its sub-components.
But I'm told other ports use different mechanisms such as .exp.in files for
apple/mac (and maybe others for other ports? IDK).
Is there consensus on the list for what the Right Thing To Do(tm) is?
ISTM my options are:
1) Add EXPORT declarations as in the patch on the bug.
2) Drop the patch from the bug and replace it with chromium-specific
.exp.in-style files, one per layer from which I need to export (WebCore,
WTF, WebKit). And build the build-time machinery to use them. I don't
really know what's involved here and would appreciate any pointers/hints
people had if this is the way to go.
3) Something else (preferably unifying other ports' approaches).
I haven't really done much with the export macros in WebKit, but I
have had plenty of experience w/ this stuff in Chromium ...
As far as I know, if you take the EXPORT macro approach you should be
able to build everywhere; i.e., there's no platform I'm aware of that
requires .def / .exp files. If there was, I would hope that we could
generate those from the annotated source.
In addition, the EXPORT macros have the benefit (at least, I think
it's a benefit) of being in the source file, so it's one less place to
look and one less thing to keep in sync. I believe overall maintenance
is lowest this way.
So, my preference would be to use EXPORT macros.
I believe the fact that different ports carve up the components
differently can be addressed through the proper definition of the
macros (we do this in Chromium today, as you probably know).
The whole this symbol only needs to be exported for testing purposes
problem is somewhat orthogonal and I don't know of a good solution to
this; either you export more than you want, or you have to link test
code into the library, or your test build is different from your
release build :(.
-- Dirk
___
webkit-dev mailing list
webkit-dev@lists.webkit.org
http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev
Re: [webkit-dev] exporting symbols for building .so/.dll's
On Thu, Mar 8, 2012 at 10:32 PM, Hajime Morrita morr...@chromium.orgwrote: My personal preference is to introduce WebCore version of export macros like WK_EXPORT_PRIVATE. In this way, we could use it to get rid of WebCore.exp.in in the future. That sounds like a good idea. I hate having to figure out mangled symbol names. But let's not use abbreviations like WK. Also, it's symbol exports from WebCore so I'd prefer calling it WEBCORE_EXPORT. Another possibility is to ride existing WebCore.exp.in somehow. But mangled symbol names will be different on different platforms so I don't think that's a good idea. - Ryosuke ___ webkit-dev mailing list webkit-dev@lists.webkit.org http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev
Re: [webkit-dev] exporting symbols for building .so/.dll's
(From the right address...)
Hi Ami,
I don't think there is a consensus about that. Here is my
understanding of the current status of the export symbol management:
At first, each port has different way to split libraries. For example,
- A) Mac port (WebKit1) splits WebKit into three frameworks: WebCore,
JavaScriptCore and WebKit.
(The last one is for Objective-C API which wraps WebCore.)
- B) Some others split it into JavaScriptCore and WebCore+WebKit.
- C) Chromium chooses yet another variation: making a single library
for whole WebKit.
This difference is one of the reasons why the way to export symbols is
different among ports.
For ports on approach C, it doesn't matter which WebCore/JSC API is
used from WebKit API layer because they are built in the same library.
For ports on approach B, only the symbols exported from JSC matter.
The dependency from WebKit API to WebCore doesn't.
For ports on approach A, symbols exported both from JSC and WebCore
matter because these two have their own libraries. APIs of these
libraries should be exported to be used by WebKit API library.
Also, there are port specific WebKit APIs (like WebFrame and WebString
of Chromium.) which have their own ways to export symbols. But in
general, people other than the port maintainers don't need to care
about that because there are few chances to change such APIs.
---
I'd try to clarify some more detail:
For JSC API, which A and B folks care about, there are two ways to
export it.
- For Windows port, there is JavaScriptCore.def.
- For Mac (and WX?) port, there are macro-based annotations like
JS_EXPORT, WTF_EXPORT_PRIVATE.
I'm in the way to eliminate JavaScriptCore.def by replacing it with
these annotations, but the migration isn't finished yet.
(http://wkb.ug/76257)
For WebCore API, we only need to care about WebCore.exp.in because Mac
port is the only one which follows A pattern. (I remember Windows
port used to use this pattern. But it looks no longer the case. Maybe
they changed the way when they switched to WebKit2. But my memory
might be just wrong...)
---
Having that long said, back to the original question:
Is there consensus on the list for what the Right Thing To Do(tm) is?
ISTM my options are:
1) Add EXPORT declarations as in the patch on the bug.
2) Drop the patch from the bug and replace it with chromium-specific
.exp.in-style files, one per layer from which I need to export (WebCore,
WTF, WebKit). And build the build-time machinery to use them. I don't
really know what's involved here and would appreciate any pointers/hints
people had if this is the way to go.
3) Something else (preferably unifying other ports' approaches).
- If the API is under JavaScriptCore/ (or WTF/), I'd recommend to use
JS_EXPORT and WTF_EXPORT_PRIVATE because we are under transition to
this path.
- If the API is under WebKit/chromium/, we can just use WEBKIT_EXPORT as before.
- If the API is under WebCore/, there is no consensus except we
already have WebCore.exp.in.
My personal preference is to introduce WebCore version of export
macros like WK_EXPORT_PRIVATE. In this way, we could use it to get rid
of WebCore.exp.in in the future.
Another possibility is to ride existing WebCore.exp.in somehow. Yet
another approach could be just giving up to have the unit test as a
separate binary and build it into the library for non-prod
configuration. The last one seems exactly what you are trying to avoid
though... One clear thing is that maintaining yet another exp file
will be bad news for almost everyone ;-)
Anyway, I'd like to hear thoughts from other folks.
Bests,
--
morrita
___
webkit-dev mailing list
On Fri, Mar 9, 2012 at 12:52 PM, Ami Fischman fisch...@chromium.org wrote:
Hi webkittens,
The over-all question: how should webkit libraries declare which symbols
they export?
The trigger for the question: as described in bug 80062, the chromium
shared-library-based build links test code into the (non-test) libwebkit.so
target, which is terrible.
The details:
I took a crack at fixing the above bug (see patch in the bug) by pulling the
test files out of the non-test build target, and sprinkling various
{WTF,WEBKIT}_EXPORT{,DATA} macros around declarations that need them (as
discovered by build-time and run-time failures). This style of export
declaration is what the webkit codebase does in various places
(WTF, Platform, WebCore, and WebKit, AFAICT), and incidentally also what
the chromium project uses for its sub-components.
But I'm told other ports use different mechanisms such as .exp.in files for
apple/mac (and maybe others for other ports? IDK).
Is there consensus on the list for what the Right Thing To Do(tm) is?
ISTM my options are:
1) Add EXPORT declarations as in the patch on the bug.
2) Drop the patch from the bug and replace it with chromium-specific
.exp.in-style files, one per layer from which I need to export (WebCore,
WTF, WebKit). And
