Hi Dave, all,
This is fantastic news - especially the bit about GHCi. I have put a couple
of general comments inline, but it looks like you have found most of the
issues, at least on Linux.
I think that at least some of the issues would go away if, as you say,
*"we could have an entirely separate cabal project (perhaps "wxc") for this
shared library, and then have wxcore depend on it?"
*
In this case, we would be able to ensure that libwxc.so.x.y.z (or wxc.dll
or libwxc.dylib or whatever) was in a 'normal' place before wxcore gets
built.
On 9 December 2011 14:13, Dave Tapley <duked...@gmail.com> wrote:
> On 8 December 2011 22:34, Dave Tapley <duked...@gmail.com> wrote:
> > Hello everyone,
> >
> > I wrote a story, and I invite you all to comment and help me make it
> better.
> > As an experiment I've decided to put it on hpaste instead of having
> > the mail thread get out of hand (I'm also cross posting it to the
> > cabal list and have mentioned it in #haskell, and I want to keep all
> > correspondence in one place):
> > http://hpaste.org/55027
>
> Ha, well that was good timing for hpaste to go down!
> Here's it is:
>
> I've been trying to resurrect the idea of building a shared library
> for the C++ component of the wxhaskell library.
>
> Jeremy (to my knowledge) first put this forward, here:
>
> http://wewantarock.wordpress.com/2010/11/01/working-around-the-static-libstdc-restriction/
>
> In addition to the advantages he lists there, I have the following reason:
> When building wx-core (in its current incarnation) cabal will *always*
> rebuild all the C++ code, which takes a majority of the build time.
> This becomes very frustrating when you change one line of Haskell and
> have to wait seven minutes to rebuild.
> I complained about it here:
> http://sourceforge.net/mailarchive/message.php?msg_id=28099997
>
> So, I decided to try and stop this complete C++ rebuild, I partially
> succeed, but I eventually got stuck for reasons I complained here:
> http://www.haskell.org/pipermail/cabal-devel/2011-October/007816.html
>
> Side note: I did eventually discover why (or, where) the c-sources
> (cSources) list is used to compile and link, and it is here:
>
> http://hackage.haskell.org/packages/archive/Cabal/latest/doc/html/src/Distribution-Simple-GHC.html#buildLib
> You can see:
> "| filename <- cSources libBi]" under "-- build any C sources", and
> you can see:
> "cSharedObjs = map (`replaceExtension` ("dyn_" ++ objExtension))
> (cSources libBi)"
> under "-- link:".
> Is this assumption correct?
>
This is correct - it's the standard way Cabal builds C/C++ code. To be
honest, this is really a consequence of the fact that Cabal is more of a
tool for simple distribution than a Make replacement, and the developers
probably didn't expect Cabal would be used to build large bodies of C++
code.
My approach is far from perfect, too - in fact it is incorrect because I
don't do dependency tracking on header files, so if you edit a header, it
(incorrectly) fails to rebuild. This is because I wasn't keen on rewriting
'make' in the wxHaskell build system - it seems to me like something Cabal
could usefully support. I did play with some very hacky code which
automatically rebuilt all of the C++ code if any header is newer than any
C++ source. This is aesthetically dreadful, but it somewhat captures the
reality, since in practice, pretty much all of the C++ files depend on all
of the headers (and it's easy to write).
On reflection, we should probably do this as what exists at present could
lead to unexpectedly incorrect code being generated if someone is in full
flow of development. I'll see if I can dig up some code - I think I have it
somewhere.
At this point I thought about either:
> 1. Getting the cabal source and starting to write code to give
> BuildInfo a "cObjs" in addition to "cSources".
> 2. Picking up Jeremy's shared library code, so wxhaskell would have
> its own code to build the library, in which I could do sensible
> re-compilation.
>
> Given that there were other advantages to 2, I went with that.
>
> Jeremy had written one blog post on building a such a shared library, here:
>
> http://wewantarock.wordpress.com/2010/11/03/building-a-shared-library-in-cabal/
> In response to an email on the wxhaskell-devel list he also kindly put
> up this gist, with the code he'd been working on:
> https://gist.github.com/1301115
>
> I dutifully took this gist, and have now attempted to integrate in to
> my wxhaskell-dev branch, which you can find here:
> http://darcsden.com/DukeDave/wxhaskell-dev
> (note: I haven't pushed any of the shared library code to darcsden
> yet, for reasons I'm about to explain)
>
> This is where things got interesting, after a few hours of hacking I
> have built a shared library, but in this very back-handed way:
>
> Firstly, in Jeremy's code the to-be-compiled shared library is added
> to the wxcore BuildInfo through a custom hook.
> We can see this because:
> > let all_dlls = parseDLLs ["x-dll-name", "x-dll-extra-libraries"]
> custom_bi
> (the modified wxcore.cabal contains the line: "x-dll-name: wxc")
>
> However on a clean build of wxcore we get the following error:
>
> setup: Missing dependency on a foreign library:
> * Missing C library: wxc
> This problem can usually be solved by installing the system package that
> provides this library (you may need the "-dev" version). If the library is
> already installed but in a non-standard location then you can use the flags
> --extra-include-dirs= and --extra-lib-dirs= to specify where it is.
>
My code worked for Windows - but this is a better cross-platform solution.
> The error is dumped *before* cabal calls myBuildHook, and since this
> actually builds the library the error makes sense; cabal is looking
> for the shared library before we've built it.
>
> To get around this I modified the offending line, thus:
> > let all_dlls = parseDLLs ["x-dll-extra-libraries"] custom_bi
>
> With that modification cabal would now get to myBuildHook, but then
> another curious error arose:
> We see that the actual linking is called here:
> > runProgram verbose gcc (opts' ++ objs' ++ lib_dirs' ++ libs')
>
> But on hitting that line the following error is spat out:
> /usr/bin/ld: cannot open output file dist/build/libwxc.so.0.13.1: No
> such file or directory
>
Windows doesn't behave this way - this part worked for me as well.
> I checked all my permissions and couldn't see anything wrong, I could
> touch the file.
> Conveniently I noticed that, if the verbosity is set high enough,
> runProgram will call printRawCommandAndArgs:
>
> http://hackage.haskell.org/packages/archive/Cabal/latest/doc/html/src/Distribution-Simple-Utils.html#printRawCommandAndArgs
>
> The output (i.e. the linker invocation) looks like this:
> /usr/bin/gcc -fno-stack-protector -shared -Wl,-soname,libwxc.so.0 -o
> dist/build/libwxc.so.0.13.1 dist/build/src/cpp/apppath.o
> dist/build/src/cpp/dragimage.o dist/build/src/cpp/eljaccelerator.o
> [snip-rest-of-.o-files] -L/usr/local/lib -lstdc++ -lwx_baseu-2.9
> [snip-rest-of-wx-libs]
>
> Now if I cd into the wxcore and paste the command *verbatim* then gcc
> works and generates libwxc.so.0.13.1 as expected.
> You can see in Jeremy's code the linkShareLib function contains:
> > cwd <- getCurrentDirectory
>
> I used this to confirm that the we were in ./wxcore and we are, even
> making the path for -o absolute didn't sovle the issue.
> I ended up replacing runProgram line with this, less satisfactory line:
> > system $ (unwords ([show . locationPath . programLocation $ gcc] ++
> opts' ++ objs' ++ lib_dirs' ++ libs'))
>
> Which works, but still doesn't explain why runProgram doesn't.
> Any suggestions?
>
I've tried this on Linux. You're absolutely right and I'm stumped as to
why. I think we should pull this specific part of the thread out and post
to the Haskell list. Maybe someone who knows Cabal and/or GHC would have an
idea why this is the case.
> So with that complete I was able to link a shared library to:
> wxcore/dist/build/libwxc.so.0.13.1
>
> Of course, at this point we've generated the shared library, but
> wxcore still isn't aware of it, so the build completes successfully,
> but any attempt to compile against it results (understandably) in
> hundreds of linker errors.
> To resolve this I had to add "x-dll-name" (read in from wxcore.cabal
> as "wxc") back to parseDLLs:
> > let all_dlls = parseDLLs ["x-dll-name", "x-dll-extra-libraries"]
> custom_bi
>
> Well, not quite, attempt to build wxcore again and we're back to this
> error:
> * Missing C library: wxc
>
> Of course, the .so isn't in a 'normal' place, so add its location as
> an extra lib directory:
> > { extraLibDirs = extraLibDirs libbi ++ extraLibDirs wx ++
> ["/full/path/to/wxcore/dist/build"]
> (note: I tried using just "dist/build", but cabal said: "library-dirs:
> dist/build is a relative path")
>
I think we need a more robust solution in the longer run, as
<wxhaskell>/wcore/dirt/build is really a temporary location.
> This still isn't quite enough, it took one more kludge:
> /full/path/to/wxcore/dist/build/$ ln -s
> /full/path/to/wxcore/dist/build/libwxc.so.0.13.1 libwxc.so
>
Earlier you mentioned that, at least on Linux, the build command is:
/usr/bin/gcc -fno-stack-protector -shared -Wl,-soname,libwxc.so.0 -o
dist/build/libwxc.so.0.13.1 dist/build/src/cpp/apppath.o
dist/build/src/cpp/dragimage.o dist/build/src/cpp/eljaccelerator.o
[snip-rest-of-.o-files] -L/usr/local/lib -lstdc++ -lwx_baseu-2.9
[snip-rest-of-wx-libs]
I know GCC generated this and not you, but it looks like the root of the
issue. I am slightly confused though. ld.so is supposed to *know* that
libwxc.so.0.13.1 is a valid instance of libwxc.so.0
In normal Linux-land, you run /sbin/ldconfig after installing a new
library, and it creates these symlinks automatically You could do this by
executing something like (not tested)
/sbin/ldconfig -n <path-to-directory>
I'd like to think there's a more elegant solution than this and I'm
> open to suggestions.
> I should note that at this point I became aware that we could have an
> entirely separate cabal project (perhaps "wxc") for this shared
> library, and then have wxcore depend on it?
>
> Running one more time I *thought* I was finally there, until I noticed
> this line, hidden in cabal's output:
> /full/path/to/wxcore/dist/build/libwxc.so: file not recognized: File
> truncated
>
> As far as I can tell, this occurs because we now load(?)
> "libwxc.so.0.13.1" in myConfHook, but then in myBuildHook we want to
> use it as the destination for the linker. Worse still it seems that
> 'truncated' actually means 'deleted'. To get around this I had to one
> again remove "x-dll-name" from the parseDLLs call (so I could get to
> myBuildHook and create "libwxc.so.0.13.1" again), build wxcore, then
> add "x-dll-name" back again, but this time also comment out the call
> to linkSharedLib (and so stop cabal attempting to open
> "libwxc.so.0.13.1") before building wxcore one last time.
>
> Finally everything seemed to work.
>
> Now, I should note that I have been using cabal-dev, and so I now have
> a package-conf I need to reference when building with my new shared
> library wxhaskell. To test it I built the wxhaskell HelloWorld sample
> thus:
> /full/path/to/samples/wxcore$ ghc --make -package-conf
> ../../cabal-dev/packages-7.0.3.conf HelloWorld.hs
>
> It worked, but then when I try to run HelloWorld I get:
> ./HelloWorld: error while loading shared libraries: libwxc.so.0:
> cannot open shared object file: No such file or directory
>
> Okay, again, libwxc.so.0 isn't in a 'normal' place, so I kludged it
> with another symlink:
> /full/path/to/samples/wxcore$ ln -s
> /full/path/to/wxcore/dist/build/libwxc.so libwxc.so.0
>
> And now, I can report that HelloWorld runs.
> [imagine a screen shot of the HelloWorld sample application running here]
> Rejoice.
>
> But wait, there's more:
> One of the promises of the shared library approach is that we'd all be
> able to use wxhaskell with ghci again, and to my utter disbelief, it's
> true:
> /full/path/to/samples/wxcore$ ghci -package-conf
> ../../cabal-dev/packages-7.0.3.conf HelloWorld.hs
> Ok, modules loaded: Main.
> Prelude Main> main
>
> [imagine a screen shot of the HelloWorld sample application running here]
> Double rejoice.
>
>
>
> >
> > Dave,
>
>
> ------------------------------------------------------------------------------
> Cloud Services Checklist: Pricing and Packaging Optimization
> This white paper is intended to serve as a reference, checklist and point
> of
> discussion for anyone considering optimizing the pricing and packaging
> model
> of a cloud services business. Read Now!
> http://www.accelacomm.com/jaw/sfnl/114/51491232/
> _______________________________________________
> wxhaskell-devel mailing list
> wxhaskell-devel@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/wxhaskell-devel
>
------------------------------------------------------------------------------
Learn Windows Azure Live! Tuesday, Dec 13, 2011
Microsoft is holding a special Learn Windows Azure training event for
developers. It will provide a great way to learn Windows Azure and what it
provides. You can attend the event by watching it streamed LIVE online.
Learn more at http://p.sf.net/sfu/ms-windowsazure
_______________________________________________
wxhaskell-devel mailing list
wxhaskell-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/wxhaskell-devel
