On 3/21/2013 4:33 PM, Thompson, John wrote:
Matthew,
Thank you so much! This was very enlightening.
I didn't know that a .exe file could have exports.
This opens up new possibilities, even though it's not a full generic
solution.
I'm assuming symbols using declspec(export)'s would cause these to be
exported in the .exe as well.
Yes, they do.
In the .def file, you had to use the mangled names, right?
Yes.
In my experiment of a couple of years ago, I created a new common
header for each llvm/clang library which had the purpose of
#define'ing a (module name)_LINKAGE preprocessor symbol to be used in
the declarations of symbols to exported/imported. Thus I could switch
it between declspec(export) and declspec(import) based on a flag input
from the build. I did this with an eye toward having either a "big
clang DLL" or individual clang library DLLs, using build flags to
switch between them.
We could now do the same thing, except that clang.exe itself could now
be the "big clang DLL". Assuming a DLL build of clang is a future or
never to be done thing, we could start out adding the *_LINKAGE tags
to just those items likely to be used in a plugin, without worrying
about breaking a DLL build. This would probably be easier than
dealing with mangled names and the .def file. It might complicate the
plug-in writer's lives though, since if they need a symbol that wasn't
exported, they would need to patch clang for it.
I know using import/export tags is a sore spot for many. But at least
with this scheme, it seems it would be less of a hassle because the
main build won't general break if someone forgets to add one.
This all seems reasonable to me.
I wonder what the impact of an embedded export table would have on the
load time for clang.exe?
I looked for some kind of "export all" option in VC++ and the linker,
but didn't see one, and it probably would be a bad idea anyway,
because of the number of exports, and would hit the (supposed) import
limit anyway in the import library, unless Microsoft has fixed or will
fix it.
FYI, our clang.def has a little less than 400 symbols in it. And keep in
mind that we only exported what Polly was using.
Comments anyone from the llvm/clang community? I can't work on it
right now, but I probably could later.
-John
*From:*Matthew Curtis [mailto:[email protected]]
*Sent:* Thursday, March 21, 2013 12:19 PM
*To:* Thompson, John
*Cc:* Gao, Yunzhong; '[email protected]'
*Subject:* Re: [PATCH] Implement a sane plugin API for clang
On 3/20/2013 8:17 PM, Thompson, John wrote:
Hi Matthew,
Could you point me to more information on how you got the plug-in
mechanism to work with clang on Windows?
I understand the need to add access functions for static data, but
I don't understand how you build the plugin DLL in general.
How does a clang.exe built from statically-linked libraries have
the necessary exports to satisfy the dynamic loading of the plugin
DLL?
We made the following change to clang's tools/driver/CMakeLists.txt
diff --git a/tools/driver/CMakeLists.txt b/tools/driver/CMakeLists.txt
index 2545610..a3fe292 100644
--- a/tools/driver/CMakeLists.txt
+++ b/tools/driver/CMakeLists.txt
@@ -10,10 +10,17 @@ set( LLVM_LINK_COMPONENTS
selectiondag
)
+if(MSVC)
+ set(def_file "clang.def")
+else(MSVC)
+ set(def_file "")
+endif(MSVC)
+
add_clang_executable(clang
driver.cpp
cc1_main.cpp
cc1as_main.cpp
+ ${def_file}
)
target_link_libraries(clang
@@ -40,6 +47,10 @@ target_link_libraries(clang
set_target_properties(clang PROPERTIES VERSION
${CLANG_EXECUTABLE_VERSION})
+if(MSVC)
+ set_target_properties(clang PROPERTIES ENABLE_EXPORTS true)
+endif(MSVC)
+
add_dependencies(clang clang-headers)
if(UNIX)
Note that on Windows we only support CMake builds.
Documentation for ENABLE_EXPORTS is here:
http://www.cmake.org/cmake/help/v2.8.10/cmake.html#prop_tgt:ENABLE_EXPORTS
On the plug-in side we just added 'clang' to the list of Polly's
target link libraries.
How do you get the list of symbols to import in the plugin DLL build?
Do you manually collect it from a bare link of the plug-in .obj's with
no library references?
I wish I could say I did something intelligent here, but unfortunately
this was a manual process. I would attempt to link, note any
unresolved externals, add them to the def file, and then relink. I
don't consider this to be a maintainable solution in general. However
at the time we were under pressure to release and discovered our
Windows issues very late. So I did not spend much time thinking about
a better solution to managing the exports.
Or did you succeed in building a DLL-based version of clang? (Not one
big clang DLL like I was experimenting with, but the normal libraries
as DLLs.) If so, how did you do the exports and imports in general?
No. We're just building a stand-alone clang executable.
Doing so without declspec(import/export) seemed problematic because of
the 16K limit on the number of exports or some-such. I know exporting
everything in one big DLL would exceed that, but I'm not sure about
separate DLLs.
Hmm. We did not run into any size limitations. Perhaps because we were
exporting just what Polly needed.
How do you register the plugin's action? In my hack, I had to have
the plugin DLL provide a named function derived from the plug-in name
in order for clang to find it.
We didn't have to do anything special here. Static initializers in the
Polly dll run on load which register the Polly passes (see
lib/RegisterPasses.cpp in the Polly source). Is there a difference in
the way that back-end plug-ins work versus clang AST plug-ins?
Hope this helps.
Cheers,
Matthew
Thanks.
-John
*From:*Gao, Yunzhong
*Sent:* Wednesday, March 20, 2013 2:42 PM
*To:* Thompson, John
*Cc:* 'Matthew Curtis'; '[email protected]
<mailto:[email protected]>'
*Subject:* RE: [PATCH] Implement a sane plugin API for clang
Hi John,
The other day I ran across a post by Matthew of the Polly team and it
seems that the Polly
team might have made some progress regarding plug-in support on Windows.
http://lists.cs.uiuc.edu/pipermail/llvm-commits/Week-of-Mon-20130311/168147.html
Although I do not fully understand their implementation, it sounds
like the Polly team was
not trying to add __declspec(dllimport/dllexport) all over the
clang/llvm source base. They
used a .def file to export the necessary symbols from the clang
executable, and then they
just tried to use the imported symbols without __declspec(dllimport).
According to the
MSDN blog entry below, dllimport is not necessary for function calls
and is only necessary
for data access. I am cc'ing Matthew in case I misunderstood Polly's
implementation.
http://blogs.msdn.com/b/russellk/archive/2005/03/20/399465.aspx?wa=wsignin1.0
I think both the Cygwin and the mingw environments on Windows provide
a dlltool
executable that can automate the generation of .def files from either
the object files or a
static library. I am not aware of equivalent tools in the Visual
Studio package; you may have
to write some scripts to extract and package the symbols if you are
using Visual Studio.
HTH,
- Gao
*From:*[email protected]
<mailto:[email protected]>
[mailto:[email protected]] *On Behalf Of *Thompson, John
*Sent:* Friday, March 15, 2013 6:04 PM
*To:* [email protected] <mailto:[email protected]>
*Subject:* RE: [PATCH] Implement a sane plugin API for clang
Hi Sean,
I missed the discussion, but I saw your message and wanted to find out
where things stood on the plug-in API with respect to Windows.
A year or two ago I wrote a plug-in for Clang for generating LUA
bindings, and developing on Windows, found that the plug-in mechanism
didn't work on Windows, because it relies on how *nix systems do
DLLs. Unfortunately, in Windows land, a plug-in DLL would have to be
linked with Clang libraries which are DLLs, and apparently building
the Clang libraries as DLLs is problematic, due to the build system
and the messy business of importing/exporting symbols.
I managed to skirt the issue by still building the plug-in against the
static libraries, and hacking clang to look up call an entry point
function with a name based on the plug-in name so it could hook up the
objects in the plug-in. This, of course, was kind of dangerous,
because the plugin module and the clang executable would have totally
separate static data areas, not to mention the duplicated code segments.
Then I did some work on making a big Clang DLL, to skirt some of the
DLL import/export issues, just exporting the symbol references needed
between the driver and the library, using some header conventions for
handling the exports. I intended to pursue this further, and try to
resolve the clang DLL build issues, but haven't been able to get back
to it.
Has anyone else worked on resolving the plugin issues for Windows or
the Clang-libraries-as-DLLs build?
-John
--
Qualcomm Innovation Center, Inc. is a member of Code Aurora Forum, hosted by
The Linux Foundation
--
Qualcomm Innovation Center, Inc. is a member of Code Aurora Forum, hosted by
The Linux Foundation
_______________________________________________
cfe-commits mailing list
[email protected]
http://lists.cs.uiuc.edu/mailman/listinfo/cfe-commits
