Re: [PD-dev] symlinks render pdstring unbuildable on MinGW/Windows
moin Hans, Short version: should be fixed now. package-local links to the common code are now no longer tracked by SVN, but kludged into place by an ./svn-prepare.sh (symlink ;-) in each package directory, alternately by the DIR.autogen_stamp target in extended/Makefile. More comments: see below. On 2009-03-05 23:44:58, Hans-Christoph Steiner h...@eds.org appears to have written: On Mar 5, 2009, at 5:22 PM, Bryan Jurish wrote: On 2009-03-05 21:33:36, Hans-Christoph Steiner h...@eds.org appears to have written: On Mar 4, 2009, at 4:23 PM, Bryan Jurish wrote: On 2009-03-04 05:37:40, Hans-Christoph Steiner h...@eds.org appears to have written: I was just trying to build string2any and friends on Windows for a student, but the symlinks used in moocow are throwing a huge wrench in the process. They show up at .lnk files, and are not links at all. That's because cygwin translates symlinks into Windows shortcuts, aka .lnk. [snip] the rsync -L flag (--copy-links) works for me here, even with a preceding -a (--archive)... does that not work on cygwin? The only times I've ever had problems with -L were symlink cycles (./dir - .), which I certainly am not inserting into the repository. Sorry, I had no intention to insult or demean, I guess I was just terse. No worries; the whole mess really arises from my not having the time to dig around in the automake internals (bad programmer, no biscuit): I'm sure there's a way to get the common code into its own automake+autoconf package, but I haven't figured it out yet. Ideally, I'd like to have an family of automake _PDEXTERNALS targets (analagous to _PROGRAMS, _LIBRARIES, _DATA, etc.), but that's not happening yet; hence the intermediate solution ../common, which goes pretty far towards eliminating the headaches necessary to roll up a new external package or add new functionality to an existing package. The bad news is that its not that simple. I added --no-l --copy-links to cygwin rsync and it still doesn't work. Curioser and curioser. I just tried an rsync from linux (with symlinks) to cygwin with -a --no-l --copy-links here, and I got copies of the symlinks rather than windoof shortcuts. Can you point me at the (script containining the) full rsync call so I can test that here? There is nothing you have to do here, I just thought it'd be nice to have those externals included in Windows. Here are the logs, it seems it doesn't find the compiler properly: http://bxmc.poly.edu/pdlab/moocow_log.txt http://bxmc.poly.edu/pdlab/config.log 403 forbidden not really important though, since the AC_CONFIG_LINKS() symlinks aren't the problem. What I don't understand is why this code needs such a complicated build system? As far as I can tell, it is mostly pretty standard C code. If you're asking that now, it's probably a good thing I didn't delegate external building to libtool ;-) It's (mostly) not the *code* which needs automake, it's *me* who wants it. Having 'distcheck' and 'dist' targets generated auto-magically is really outrageously spiffy. And with the shared code, building a new external in an existing package is as easy as setting a couple of make variables: pdexterns_PROGRAMS += myexternal myexternal_SOURCES = myexternal.c mycommon.c mycommon.h ... and automake takes care of the rest (make, make install, make uninstall, ...). Creating a new external package using this system is pretty easy too: see the SVN directory externals/moocow/hello for a complete working tutorial, or its README here: http://pure-data.svn.sourceforge.net/viewvc/pure-data/trunk/externals/moocow/hello/README.txt?view=markup The ./configure options are there for debugging, support for multiple installations of Pd on the same machine, as well as support for pd-extended (e.g. --enable-object-externals). autotools support for use environment variables (CFLAGS, CPPFLAGS, etc.) is really handy too, e.g. for messing with optimization debugging flags. I wish every external build system would support these. That said, in some cases the code actually *does* require some platform-dependent initialization in ./configure. [locale] for instance checks for definitions of all of the LC_* variables it supports, [flite] needs to know where to find the libflite include files, as well as how to link to libflite: this is exactly the kind of thing autoconf was made to handle, and which it does quite elegantly. I find that in the long run, simple Makefiles are the least work overall. To each his own, I suppose, or maybe I'm missing something. If I were maintaining only 1 or 2 external packages (or a single global build system ;-), I think I might tend to agree. As it is, I think the autotools beat copy+paste Makefiles (which I still use a lot, e.g. for my LaTeX documents, which rarely need to be synchronized or updated once the paper has been written) hands down. Are automake+autoconf+autoheader overkill for my externals? Sure they are: I think this is
Re: [PD-dev] symlinks render pdstring unbuildable on MinGW/Windows
On Mar 4, 2009, at 4:23 PM, Bryan Jurish wrote: moin Hans, On 2009-03-04 05:37:40, Hans-Christoph Steiner h...@eds.org appears to have written: I was just trying to build string2any and friends on Windows for a student, but the symlinks used in moocow are throwing a huge wrench in the process. They show up at .lnk files, and are not links at all. That's because cygwin translates symlinks into Windows shortcuts, aka .lnk. So symlinks will never work on Windows. uh, *which* symlinks exactly are you referring to? trunk/externals/moocow/pdstring/m4 trunk/externals/moocow/pdstring/pdexternal.am do have an error message or a make log or a config.log for me? I am working on getting the Windows build server up, not quite there yet, almost. Then I can give you shell access. SVN can handle symlinks with the svn:special property, and according to the Subversion FAQ (http://subversion.tigris.org/ faq.html#symlinks): the Subversion repository has no internal concept of a symlink. It stores a versioned symlink as an ordinary file with an 'svn:special' property attached. The svn client (on unix) sees the property and translates the file into a symlink in the working copy. Win32 has no symlinks, so a win32 client won't do any such translation: the object appears as a normal file. The most recent SVN book says much the same about svn:special symlinks. ... so I wonder how you managed to get .lnk files out of svn at all, if indeed that is what's happening... ... or am I creating symlinks somewhere with AC_CONFIG_LINK? I don't think I am anymore (but I'll check), and even that should default to something safe on the configuring system (copy for win32, according to the autoconf docs), so I don't really know what is going wrong where. [grep -r AC_CONFIG_LINKS pdstring/*] pdstring/configure.ac:AC_CONFIG_LINKS([src/any2string-help.pd:src/ any2bytes-help.pd]) pdstring/configure.ac:AC_CONFIG_LINKS([src/string2any-help.pd:src/ bytes2any-help.pd]) ... oops: ok, I guess I am calling AC_CONFIG_LINKS() too (therefore the behavior sounds like a bug in cygwin, but that's beside the point I guess) -- this would be easy to replace e.g. with svn:special symlinks (preferred) or an automake install-local target or whatever, but I still need to know if any of the svn:special symlinks are exploding on you as well (e.g. pdstring/m4, pdstring/pdexternal.am, pdstring/src/mooPdUtils.h)... If the svn:special symlinks are going sour, maybe you should try a different svn client? If AC_CONFIG_LINKS() is exploding, then maybe you need to run ./configure in a different shell environment (MSYS?) ? The whole thing is run in MSYS/MinGW except for the rsync, which is run in Cygwin. Running rsync in MSYS did not work for me, I could not get it going, no matter how hard I tried. Most of it ran, but there were some technical details which I forget. Basically, dealing with Windows is a massive pain in the butt. The other thing is that the new code is actually copied to the build server using rsync rather than svn. rsync has the very handy -- delete feature, meaning rsyncing leaves you with a perfect recreation of the original with the additions and subtractions. That would not be easy to do with svn and make clean. Unfortunately, rsync is very aware of symlinks. I tried to make it copy the symlinks target instead, but that didn't work. So basically, moocow is the only build system included in Pd-extended that relies on symlinks, so I am guessing the easiest path forward is to not use symlinks or not support Windows. Instead of using symlinks, the build system should just use the paths to the shared files. I don't know automake, but that is possible with other build tools, so it seems likely to work there too. I understand your concerns, and I do wish my externals to remain compatible with pd-extended builds even on (shudder) windoof. And yes, of course referencing paths outside the project root directory is possible with automake. My issues are not with the possibility (heck, it's *possible* to write a C compiler entirely in DOS .BAT, but who would want to?), but rather portability and ease of maintainability (i.e. I'm lazy). Unfortunately, I don't see any good way to keep my code centralized *and* still maintain independent self-contained external packages which play (mostly) nicely together with the pd-extended build system *without* using either svn:special symlinks or svn:externals (repository-internally, but still svn:externals, which I seem to recall has been deprecated for use with pd-extended auto-builds), unless I create a second copy of each external package in the repository for pd-extended builds, which contains copies of all shared code (and all other code), and which has to be explicitly published, rather than built from the straight SVN sources I use. I suppose the SVN catch-phrase
Re: [PD-dev] symlinks render pdstring unbuildable on MinGW/Windows
moin Hans, On 2009-03-05 21:33:36, Hans-Christoph Steiner h...@eds.org appears to have written: On Mar 4, 2009, at 4:23 PM, Bryan Jurish wrote: On 2009-03-04 05:37:40, Hans-Christoph Steiner h...@eds.org appears to have written: I was just trying to build string2any and friends on Windows for a student, but the symlinks used in moocow are throwing a huge wrench in the process. They show up at .lnk files, and are not links at all. That's because cygwin translates symlinks into Windows shortcuts, aka .lnk. So symlinks will never work on Windows. uh, *which* symlinks exactly are you referring to? trunk/externals/moocow/pdstring/m4 trunk/externals/moocow/pdstring/pdexternal.am argh. these are svn:special links. Just so I understand, what form does the huge wrench take? Does the whole auto-build fail, or just the moocow/ packages? The whole thing is run in MSYS/MinGW except for the rsync, which is run in Cygwin. Running rsync in MSYS did not work for me, I could not get it going, no matter how hard I tried. Most of it ran, but there were some technical details which I forget. Basically, dealing with Windows is a massive pain in the butt. agreed. The other thing is that the new code is actually copied to the build server using rsync rather than svn. rsync has the very handy -- delete feature, meaning rsyncing leaves you with a perfect recreation of the original with the additions and subtractions. That would not be easy to do with svn and make clean. Unfortunately, rsync is very aware of symlinks. I tried to make it copy the symlinks target instead, but that didn't work. the rsync -L flag (--copy-links) works for me here, even with a preceding -a (--archive)... does that not work on cygwin? The only times I've ever had problems with -L were symlink cycles (./dir - .), which I certainly am not inserting into the repository. So basically, moocow is the only build system included in Pd-extended that relies on symlinks, ok. so I am guessing the easiest path forward is to not use symlinks or not support Windows. I'm not convinced. Instead of using symlinks, the build system should just use the paths to the shared files. I don't know automake, but that is possible with other build tools, so it seems likely to work there too. I understand your concerns, and I do wish my externals to remain compatible with pd-extended builds even on (shudder) windoof. And yes, of course referencing paths outside the project root directory is possible with automake. My issues are not with the possibility (heck, it's *possible* to write a C compiler entirely in DOS .BAT, but who would want to?), but rather portability and ease of maintainability (i.e. I'm lazy). Unfortunately, I don't see any good way to keep my code centralized *and* still maintain independent self-contained external packages which play (mostly) nicely together with the pd-extended build system *without* using either svn:special symlinks or svn:externals [snip] With makefiles you can use include instead of symlinks, so I am sure that automake has a similar functionality. Again: of course it does. Actually, I find the suggestion that I might not have though of this mildly insulting. I don't think it was meant to be; guess I'm just a bit peevish today. Sorry. You could make pdexternal.am just a file that includes the common file, and nothing else. Yes I could, but it wouldn't solve anything. I realize you probably don't really care about the details of my build subsystem, but please try to understand at least this much: as soon as I include ../common/pdexternal.am (regardless of whether that happens in ./Makefile.am or ./pdexternal.am), I no longer have a self-contained and independent package. Same goes for the -I m4 (rsp. -I ../common/m4) option to aclocal, as declared in Makefile.am. Try this, and automake's 'distcheck' target explodes, and the 'dist' target produces a package which is incomplete (doesn't contain anything from ../common and cannot be used for further autotools-sensitive development without that code). So it's not as simple as include, given my desire to continue to maintain self-contained and independent packages (in addition to supporting the pd-extended build system). Possibilities I see, all of which are unsatisfying in one way or another: I could svn-copy the common stuff into each and every quasi-independent external package that uses it, but that results in precisely the kind of maintainance snafu I wrote pdexternal.am co to avoid, since changes in ../common/x then wouldn't get propagated to ./x without another explicit SVN copy operation to each package dir . ... unsatisfying. Essentially the same applies to multiple copies of the common code within SVN, which is even uglier (for SVN itself). Re make: I can't even write (standard package-local ./Makefile) make targets for the 'common' stuff, because much of it
Re: [PD-dev] symlinks render pdstring unbuildable on MinGW/Windows
On Mar 5, 2009, at 5:22 PM, Bryan Jurish wrote: moin Hans, On 2009-03-05 21:33:36, Hans-Christoph Steiner h...@eds.org appears to have written: On Mar 4, 2009, at 4:23 PM, Bryan Jurish wrote: On 2009-03-04 05:37:40, Hans-Christoph Steiner h...@eds.org appears to have written: I was just trying to build string2any and friends on Windows for a student, but the symlinks used in moocow are throwing a huge wrench in the process. They show up at .lnk files, and are not links at all. That's because cygwin translates symlinks into Windows shortcuts, aka .lnk. So symlinks will never work on Windows. uh, *which* symlinks exactly are you referring to? trunk/externals/moocow/pdstring/m4 trunk/externals/moocow/pdstring/pdexternal.am argh. these are svn:special links. Just so I understand, what form does the huge wrench take? Does the whole auto-build fail, or just the moocow/ packages? The whole thing is run in MSYS/MinGW except for the rsync, which is run in Cygwin. Running rsync in MSYS did not work for me, I could not get it going, no matter how hard I tried. Most of it ran, but there were some technical details which I forget. Basically, dealing with Windows is a massive pain in the butt. agreed. The other thing is that the new code is actually copied to the build server using rsync rather than svn. rsync has the very handy -- delete feature, meaning rsyncing leaves you with a perfect recreation of the original with the additions and subtractions. That would not be easy to do with svn and make clean. Unfortunately, rsync is very aware of symlinks. I tried to make it copy the symlinks target instead, but that didn't work. the rsync -L flag (--copy-links) works for me here, even with a preceding -a (--archive)... does that not work on cygwin? The only times I've ever had problems with -L were symlink cycles (./dir - .), which I certainly am not inserting into the repository. So basically, moocow is the only build system included in Pd-extended that relies on symlinks, ok. so I am guessing the easiest path forward is to not use symlinks or not support Windows. I'm not convinced. Instead of using symlinks, the build system should just use the paths to the shared files. I don't know automake, but that is possible with other build tools, so it seems likely to work there too. I understand your concerns, and I do wish my externals to remain compatible with pd-extended builds even on (shudder) windoof. And yes, of course referencing paths outside the project root directory is possible with automake. My issues are not with the possibility (heck, it's *possible* to write a C compiler entirely in DOS .BAT, but who would want to?), but rather portability and ease of maintainability (i.e. I'm lazy). Unfortunately, I don't see any good way to keep my code centralized *and* still maintain independent self-contained external packages which play (mostly) nicely together with the pd-extended build system *without* using either svn:special symlinks or svn:externals [snip] With makefiles you can use include instead of symlinks, so I am sure that automake has a similar functionality. Again: of course it does. Actually, I find the suggestion that I might not have though of this mildly insulting. I don't think it was meant to be; guess I'm just a bit peevish today. Sorry. You could make pdexternal.am just a file that includes the common file, and nothing else. Yes I could, but it wouldn't solve anything. I realize you probably don't really care about the details of my build subsystem, but please try to understand at least this much: as soon as I include ../common/pdexternal.am (regardless of whether that happens in ./Makefile.am or ./pdexternal.am), I no longer have a self-contained and independent package. Same goes for the -I m4 (rsp. -I ../common/m4) option to aclocal, as declared in Makefile.am. Try this, and automake's 'distcheck' target explodes, and the 'dist' target produces a package which is incomplete (doesn't contain anything from ../common and cannot be used for further autotools-sensitive development without that code). So it's not as simple as include, given my desire to continue to maintain self-contained and independent packages (in addition to supporting the pd-extended build system). Possibilities I see, all of which are unsatisfying in one way or another: I could svn-copy the common stuff into each and every quasi-independent external package that uses it, but that results in precisely the kind of maintainance snafu I wrote pdexternal.am co to avoid, since changes in ../common/x then wouldn't get propagated to ./x without another explicit SVN copy operation to each package dir . ... unsatisfying. Essentially the same applies to multiple copies of the common code within SVN, which is even uglier (for SVN itself). Re make: I
Re: [PD-dev] symlinks render pdstring unbuildable on MinGW/Windows
moin Hans, On 2009-03-04 05:37:40, Hans-Christoph Steiner h...@eds.org appears to have written: I was just trying to build string2any and friends on Windows for a student, but the symlinks used in moocow are throwing a huge wrench in the process. They show up at .lnk files, and are not links at all. That's because cygwin translates symlinks into Windows shortcuts, aka .lnk. So symlinks will never work on Windows. uh, *which* symlinks exactly are you referring to? do have an error message or a make log or a config.log for me? SVN can handle symlinks with the svn:special property, and according to the Subversion FAQ (http://subversion.tigris.org/faq.html#symlinks): the Subversion repository has no internal concept of a symlink. It stores a versioned symlink as an ordinary file with an 'svn:special' property attached. The svn client (on unix) sees the property and translates the file into a symlink in the working copy. Win32 has no symlinks, so a win32 client won't do any such translation: the object appears as a normal file. The most recent SVN book says much the same about svn:special symlinks. ... so I wonder how you managed to get .lnk files out of svn at all, if indeed that is what's happening... ... or am I creating symlinks somewhere with AC_CONFIG_LINK? I don't think I am anymore (but I'll check), and even that should default to something safe on the configuring system (copy for win32, according to the autoconf docs), so I don't really know what is going wrong where. [grep -r AC_CONFIG_LINKS pdstring/*] pdstring/configure.ac:AC_CONFIG_LINKS([src/any2string-help.pd:src/any2bytes-help.pd]) pdstring/configure.ac:AC_CONFIG_LINKS([src/string2any-help.pd:src/bytes2any-help.pd]) ... oops: ok, I guess I am calling AC_CONFIG_LINKS() too (therefore the behavior sounds like a bug in cygwin, but that's beside the point I guess) -- this would be easy to replace e.g. with svn:special symlinks (preferred) or an automake install-local target or whatever, but I still need to know if any of the svn:special symlinks are exploding on you as well (e.g. pdstring/m4, pdstring/pdexternal.am, pdstring/src/mooPdUtils.h)... If the svn:special symlinks are going sour, maybe you should try a different svn client? If AC_CONFIG_LINKS() is exploding, then maybe you need to run ./configure in a different shell environment (MSYS?) ? Instead of using symlinks, the build system should just use the paths to the shared files. I don't know automake, but that is possible with other build tools, so it seems likely to work there too. I understand your concerns, and I do wish my externals to remain compatible with pd-extended builds even on (shudder) windoof. And yes, of course referencing paths outside the project root directory is possible with automake. My issues are not with the possibility (heck, it's *possible* to write a C compiler entirely in DOS .BAT, but who would want to?), but rather portability and ease of maintainability (i.e. I'm lazy). Unfortunately, I don't see any good way to keep my code centralized *and* still maintain independent self-contained external packages which play (mostly) nicely together with the pd-extended build system *without* using either svn:special symlinks or svn:externals (repository-internally, but still svn:externals, which I seem to recall has been deprecated for use with pd-extended auto-builds), unless I create a second copy of each external package in the repository for pd-extended builds, which contains copies of all shared code (and all other code), and which has to be explicitly published, rather than built from the straight SVN sources I use. I suppose the SVN catch-phrase copies are cheap could seem to suggest just such a solution, but it strikes me as kludgy... then again, checking autotools-generated files (configure, Makefile.in) into SVN strikes me as kludgy too, and I'm already doing that, so maybe I should just bite the bullet... Thoughts, comments, criticism, flames welcome ;-) marmosets, Bryan ps - it appears now that I've spent this whole evening I had planned to use for working on utf-8 for pd-devel typing emails and wiki pages... maybe tomorrow ;-) -- Bryan Jurish There is *always* one more bug. jur...@ling.uni-potsdam.de -Lubarsky's Law of Cybernetic Entomology ___ Pd-dev mailing list Pd-dev@iem.at http://lists.puredata.info/listinfo/pd-dev
[PD-dev] symlinks render pdstring unbuildable on MinGW/Windows
Hey, I was just trying to build string2any and friends on Windows for a student, but the symlinks used in moocow are throwing a huge wrench in the process. They show up at .lnk files, and are not links at all. That's because cygwin translates symlinks into Windows shortcuts, aka .lnk. So symlinks will never work on Windows. Instead of using symlinks, the build system should just use the paths to the shared files. I don't know automake, but that is possible with other build tools, so it seems likely to work there too. .hc [W]e have invented the technology to eliminate scarcity, but we are deliberately throwing it away to benefit those who profit from scarcity.-John Gilmore ___ Pd-dev mailing list Pd-dev@iem.at http://lists.puredata.info/listinfo/pd-dev
