Re: Reconsideration of MinGW work
On Tue 23 Mar 2010 07:59, Ken Raeburn raeb...@raeburn.org writes: The build farms are a good improvement; we should try to get as much variety there as we can. And maybe some sort of alert (bot messages to #guile ?) when a build fails... Subscribe to guile-comm...@gnu.org :) Andy -- http://wingolog.org/
Re: Reconsideration of MinGW work
Hi Peter ( Neil co), On Mon 22 Mar 2010 09:10, Peter Brett pe...@peter-b.co.uk writes: Neil Jerram n...@ossau.uklinux.net writes: I've been making gradual progress on MinGW cross building, but I've reached a point where I'm no longer sure that this is worthwhile. This email explains why, and invites comments from anyone interested in this - especially from anyone who is really trying to use Guile on Windows. We get people coming to the gEDA user mailing list on a regular basis saying, Where can I find a version of gEDA for Windows? and the Windows builds we've put out have been generally well-received. Since Guile is one of our core dependencies, lack of Windows support in Guile would mean that we wouldn't be able to provide a Windows build at all (we already had massive problems at the start of the Guile 1.8.x series with GMP portability, or lack thereof, and this meant that it took almost three years after 1.8 became the supported stable release for us to be able to stop supporting 1.6). As Neil mentioned, hopefully we can get there via the gnulib path; it does seem to be the right thing to do GNU-wise in this case. Guile's code would stay the same, as much as possible, and Gnulib would have mingw support in it, so that if you compile Guile on mingw, the Gnulib compat layer gets compiled in. Sounds acceptable, no? Granted it might take a little while to get it right, but hopefully not three years. Andy -- http://wingolog.org/
Re: Reconsideration of MinGW work
On 22 March 2010 14:00, Andy Wingo wi...@pobox.com wrote: Hi Peter ( Neil co), On Mon 22 Mar 2010 09:10, Peter Brett pe...@peter-b.co.uk writes: We get people coming to the gEDA user mailing list on a regular basis saying, Where can I find a version of gEDA for Windows? and the Windows builds we've put out have been generally well-received. Since Guile is one of our core dependencies, lack of Windows support in Guile would mean that we wouldn't be able to provide a Windows build at all FWIW, somehow gnucash (which uses guile) runs on windows, not sure how. My pet peeve with mingw is the lack of ready-to-go regex. This is completely unrelated to guile; I have another project that made the mistake of assuming that regex just worked on windows, and I've been bitched at ever since. Getting regex into mingw and having it just work would be excellent. --linas
Re: Reconsideration of MinGW work
Ken Raeburn raeb...@raeburn.org writes: One nagging concern I've got about my Guile-Emacs project is the seemingly narrow focus of active Guile developers as far as platforms are concerned. I'm one of, what, two or three people testing the development versions on Mac OS X now and then, and most of the rest of the work is on x86 or x86-64 GNU/Linux systems, it seems? But Emacs works on a lot more systems (including MinGW, for people who don't want all of Cygwin), and saying hey, we can change Emacs to be Guile-based on x86 GNU/Linux systems; too bad about all the other platforms wouldn't go over terribly well. I test on NetBSD, and in theory care about not only i386 and amd64 but also sparc64. But I have not had a lot of spare time lately to hack on guile. I am running autobuilds on list.ir.bbn.com (NetBSD amd64): http://autobuild.josefsson.org/guile/ and it looked like some non-portable assumptions have crept in: http://autobuild.josefsson.org/guile/log-201003220603936147000.txt For a random Scheme implementation, it's okay to pick the set of platforms you want to support, and drop whatever's inconvenient. But if you want to be the official extension language for the GNU project, used by (theoretically) lots of GNU packages, you've got to support all the platforms the developers of those platforms want to support, if you possibly can. I think that includes both Cygwin and MinGW, and probably not just supporting whatever subset can be mapped into POSIX functions via Gnulib. We can probably punt on VMS, though The target set definitely ought to include cygwin, but the GNU project has a bias for Free and/or POSIX operating systems so I am willing to forgo getting upset about lack of mingw support. But surely we should be happy if someone provides it. pgpVaC3NEdKKu.pgp Description: PGP signature
Re: Reconsideration of MinGW work
Ken Raeburn raeb...@raeburn.org writes: Yes... you then also need to decide if Guile is exposing GNU/POSIX functionality, whatever the native OS functionality is, or some abstraction... Ideally, yes, I think. In other words, I think it's preferable if Guile provides the same function to applications on all platforms. (Emacs shows how fantastically useful this is.) Having just bought a Lemote Yeelong notebook at LibrePlanet [...] Aside: I was wondering about buying one of those too, but haven't yet because of performance concerns. Can it compile Guile successfully, and if so how long does it take? You *are* reporting these bugs, aren't you? :-) Err... not yet. So thanks for the reminder. But I just checked, and their (i.e. Wine's) bugzilla requires creating an account, so I'm not sure I'll bother. (Is that really irresponsible of me?) I think cross-compilation and cross-testing is a good thing to be able to do. Obviously cross-testing requires some additional setup -- as a trivial example, the test suite code isn't going to know the name or IP address of your Windows or Yeelong machine. I know it's hard to set up, though, and especially hard to maintain if few people actually use it. Perhaps having build farms available with multiple platform types can help there. I agree that this is useful, but it is, as you say, difficult to set up. Personally, to the extent that I continue working on this, I think I'm happy to limit myself to what can be achieved with emulation (i.e. with Wine) on a single Linux box. At least so far as regular builds are concerned; of course I'd also expect occasionally to copy the built DLLs over to Windows and try them there. One nagging concern I've got about my Guile-Emacs project is the seemingly narrow focus of active Guile developers as far as platforms are concerned. I'm one of, what, two or three people testing the development versions on Mac OS X now and then, and most of the rest of the work is on x86 or x86-64 GNU/Linux systems, it seems? Yes, but this isn't intentional, just a matter of limited resources. On the other hand, in the Windows case, it seems there is independent effort being put in to the problem downstream. I wonder why that's downstream - perhaps because we're not good enough at accepting patches when they're offered? On the MacOS side I haven't been closely involved recently, but I think - for all platforms other than the really mainstream ones that you've mentioned above - we need people who are authoritative about what the solution for a given problem is. If I see a patch for MacOS that seems to be done at the right place in the build, and obviously doesn't inappropriately impact other platforms, and the submitter is authoritative about it being the right solution for MacOS, it's easy to say OK, I believe you, and there's no risk, so let's commit it. But in the conversations that I remember there hasn't been that authoritative tone; it's more like I've needed to work out half of the solution myself, which I'm not in a position to do. So then things get left sitting, etc. And similarly for other less mainstream platforms. For a random Scheme implementation, it's okay to pick the set of platforms you want to support, and drop whatever's inconvenient. Which is absolutely not my intention... But if you want to be the official extension language for the GNU project, used by (theoretically) lots of GNU packages, you've got to support all the platforms the developers of those platforms want to support, if you possibly can. I agree. I think that includes both Cygwin and MinGW, This is a key point. My previous email hinted that Cygwin support might be enough, and several people have commented that it isn't. So I'm happy now that that hint was wrong. For me, then, the questions are whether we need any more Cygwin/MinGW support for 1.8.x than what already exists, and how, structurally speaking, we do Cygwin/MinGW support for 1.9/2.0 and future releases. Having said that, I don't feel I understand technically why a MinGW version of Guile, including all necessary emulation code, would run faster or be less bloated than a Cygwin Guile. Can someone explain that? and probably not just supporting whatever subset can be mapped into POSIX functions via Gnulib. I don't think I understand your point here. Gnulib isn't a fixed point. If we can add emulation code into Guile, can't we just as easily add it into Gnulib? Regards, Neil
Re: Reconsideration of MinGW work
Peter Brett pe...@peter-b.co.uk writes: Neil Jerram n...@ossau.uklinux.net writes: I've been making gradual progress on MinGW cross building, but I've reached a point where I'm no longer sure that this is worthwhile. This email explains why, and invites comments from anyone interested in this - especially from anyone who is really trying to use Guile on Windows. We get people coming to the gEDA user mailing list on a regular basis saying, Where can I find a version of gEDA for Windows? and the Windows builds we've put out have been generally well-received. Since Guile is one of our core dependencies, lack of Windows support in Guile would mean that we wouldn't be able to provide a Windows build at all (we already had massive problems at the start of the Guile 1.8.x series with GMP portability, or lack thereof, and this meant that it took almost three years after 1.8 became the supported stable release for us to be able to stop supporting 1.6). Hi Peter, I'm sorry, I didn't mean my email to suggest dropping Windows support. It was more about what more (if anything) is needed for 1.8 releases, and how to handle Windows support in future, and how those points relate to a line of work that I've been spending time on recently. Where are you getting your MinGW guile from at the moment? Is it that MinGW SF page that I mentioned, or somewhere else? Cygwin isn't an option, unfortunately; we think it's totally reasonable for users to want to use the native windowing system, not to mention the fact that Cygwin is *dog slow*. OK, understood. I'd like to understand more about what makes Cygwin slow, though, in order to see why a MinGW Guile wouldn't end up being equally slow. But in the interim I'm happy to accept that MinGW is needed. Regards, Neil
Re: Reconsideration of MinGW work
Linas Vepstas linasveps...@gmail.com writes: My pet peeve with mingw is the lack of ready-to-go regex. This is completely unrelated to guile; I have another project that made the mistake of assuming that regex just worked on windows, and I've been bitched at ever since. Gnulib has a regex library. Getting regex into mingw and having it just work would be excellent. But that won't happen, because that's not part of MinGW's mission. All MinGW is trying to do is provide a free software toolset for building native Windows programs - which at runtime will link to the standard set of DLLs that M$ provides. Those DLLs don't provide regex, so the MinGW header files don't provide regex either. Neil
Reconsideration of MinGW work
I've been making gradual progress on MinGW cross building, but I've reached a point where I'm no longer sure that this is worthwhile. This email explains why, and invites comments from anyone interested in this - especially from anyone who is really trying to use Guile on Windows. First, I've found that completing a successful build (i.e. autogen.sh, configure and make) is not at all the end of the story; it's only the first part of what is really needed - because at runtime some key pieces of function can still be missing, or can behave differently on MinGW/Windows than on Linux/POSIX. Two examples of this are operations on file descriptors - where on MinGW/Windows different functions must be called for sockets than for real files - and regular expressions, which are not supported by the MinGW/Windows C library. Therefore the definition of success also needs to include a successful make check, or some equivalent of that. Using MSYS and MinGW on Windows, I assume that make check can be run just as easily as make. When cross-building with i586-mingw32msvc-* tools on Linux, make check can be run (in principle) if you also have Wine installed, and that's the setup that I've been using recently. Second, though, it turns out that using i586-mingw32msvc-* and Wine on Linux unfortunately does not give the same results as MSYS and MinGW on Windows. For example I've found that system(NULL) throws a SIGSEGV under Wine, but for Carlo Bramix, working on Windows, that wasn't a problem; and instead, for Carlo, there were other problems that I don't see with a Linux cross build. This is hardly surprising. Although Wine aims to emulate the Windows runtime DLLs precisely, of course it is only software and so can have bugs. But it is an extra hassle in practice. Third, I reviewed my own need for Guile on Windows - and in fact I've been perfectly happily using the Cygwin version for some time now. So actually I don't currently need a MinGW version - and maybe that would be true for other Guile on Windows users too. Looking forwards, supporting a Cygwin build of Guile will always be much easier than supporting a MinGW build, because the whole point of Cygwin is to provide an emulation on Windows of the POSIX API, and that's what most of the Guile code assumes. Fourth, I've realized that I significantly misunderstood MinGW's objective. Its true objective is to provide free software tools for building native Windows programs to run with the Win32 runtime DLLs, and the MinGW website is actually very clear about this. That means that it is 100% targeting the API provided by the Win32 DLLs. But somehow or other I had got the idea that it was also a bit Cygwin-ish, in trying to provide pieces of the Linux/POSIX API that aren't provided by those DLLs. That last idea is completely wrong, which means that trying to build a POSIX-assuming project for MinGW is always going to be hard. Fifth, and now thinking more of the future with 1.9/2.0, I wondered if it would be better to address MinGW porting problems within Gnulib, instead of putting lots of #ifdef __MINGW32__ into Guile's own code. And in fact yes, based on a small and completely unscientific sample of Google results, it seems the Gnulib guys do regard MinGW compatibility as part of their remit. So putting #ifdef __MINGW32__ stuff into Guile is probably unhelpful anyway, because it would be better to contribute to Gnulib instead. And it's possible that Gnulib might already have what we need. And finally, I noticed that there already claims to be a MinGW port of Guile 1.8, here: http://sourceforge.net/projects/mingw/files/ http://sourceforge.net/projects/mingw/files/MSYS%20guile/guile-1.8.7-1/guile-1.8.7-1-msys.RELEASE_NOTES/download Therefore I'm inclined to conclude the following. - Overall, it isn't as important as I had been thinking to get a complete MinGW build of Guile. I personally plan to concentrate more now on 1.8 - 1.9/2.0 compatibility, and on the manual. - As far as future development is concerned, including the current master branch, MinGW portability fixes should be directed at Gnulib if possible, instead of done directly in the Guile code. - As far as 1.8.x is concerned, I'm not sure if there's any need out there that isn't already met either by Cygwin or by the MinGW port linked above. Anyone still reading? Congratulations if so - and please let me know what you think! Regards, Neil
Re: Reconsideration of MinGW work
On Sun, Mar 21, 2010 at 3:51 PM, Neil Jerram n...@ossau.uklinux.net wrote: This email explains why, and invites comments from anyone interested in this - especially from anyone who is really trying to use Guile on Windows. I've decided only to run UNIX origin software on Cygwin as there is always some gotcha that makes any other approach not worth the effort.
Re: Reconsideration of MinGW work
On Mar 21, 2010, at 16:51, Neil Jerram wrote: First, I've found that completing a successful build (i.e. autogen.sh, configure and make) is not at all the end of the story; it's only the first part of what is really needed - because at runtime some key pieces of function can still be missing, or can behave differently on MinGW/Windows than on Linux/POSIX. Two examples of this are operations on file descriptors - where on MinGW/Windows different functions must be called for sockets than for real files - and regular expressions, which are not supported by the MinGW/Windows C library. Yes... you then also need to decide if Guile is exposing GNU/POSIX functionality, whatever the native OS functionality is, or some abstraction... e.g., looking at file descriptors, do you try to make sockets look like files, expose the differing underlying functionality to the Scheme coder, or make them distinct object types with some APIs that can handle both (and may do so with identical or different code, depending on the OS)? I can imagine some ways in which distinguishing them would be beneficial. You could code getpeername to reject arguments that aren't socket objects (instead of calling into the kernel to see if it works), and similarly for fstat and sockets. On the other hand, the program will be started up with file descriptors on objects of types you don't know initially (on UNIX), so you still have to handle that. Therefore the definition of success also needs to include a successful make check, or some equivalent of that. Using MSYS and MinGW on Windows, I assume that make check can be run just as easily as make. When cross-building with i586-mingw32msvc-* tools on Linux, make check can be run (in principle) if you also have Wine installed, and that's the setup that I've been using recently. As an aside, one nice thing about the DejaGnu setup we had at Cygnus was that you could easily(?) define how to run tests in cross-build setups, and it could include copy these files over to the target machine described in this config file and then use this command to invoke the program over there. This was mostly used for embedded systems testing, with target boards connected via serial ports. Once you've got the framework in place, defining new target systems and ways of communicating with them can be relatively easy. Having just bought a Lemote Yeelong notebook at LibrePlanet this weekend, it has already occurred to me that cross-compilation from a faster machine (like my 8-core home Mac) targeting the Yeelong would be desirable; running tests too would be better still, when a machine is available. Second, though, it turns out that using i586-mingw32msvc-* and Wine on Linux unfortunately does not give the same results as MSYS and MinGW on Windows. For example I've found that system(NULL) throws a SIGSEGV under Wine, but for Carlo Bramix, working on Windows, that wasn't a problem; and instead, for Carlo, there were other problems that I don't see with a Linux cross build. You *are* reporting these bugs, aren't you? :-) Anyone still reading? Congratulations if so - and please let me know what you think! I think cross-compilation and cross-testing is a good thing to be able to do. Obviously cross-testing requires some additional setup -- as a trivial example, the test suite code isn't going to know the name or IP address of your Windows or Yeelong machine. I know it's hard to set up, though, and especially hard to maintain if few people actually use it. Perhaps having build farms available with multiple platform types can help there. One nagging concern I've got about my Guile-Emacs project is the seemingly narrow focus of active Guile developers as far as platforms are concerned. I'm one of, what, two or three people testing the development versions on Mac OS X now and then, and most of the rest of the work is on x86 or x86-64 GNU/Linux systems, it seems? But Emacs works on a lot more systems (including MinGW, for people who don't want all of Cygwin), and saying hey, we can change Emacs to be Guile-based on x86 GNU/Linux systems; too bad about all the other platforms wouldn't go over terribly well. For a random Scheme implementation, it's okay to pick the set of platforms you want to support, and drop whatever's inconvenient. But if you want to be the official extension language for the GNU project, used by (theoretically) lots of GNU packages, you've got to support all the platforms the developers of those platforms want to support, if you possibly can. I think that includes both Cygwin and MinGW, and probably not just supporting whatever subset can be mapped into POSIX functions via Gnulib. We can probably punt on VMS, though Ken
