Bill, Shelling out in Julia on Windows currently calls into the bundled Git-bash, for the purposes of the package manager. That will hopefully change in 0.4 by replacing the bundled Git-bash with libgit2 bindings, but PR 7339 is currently a bit stalled for lack of anyone who understands the LibGit2.jl / Pkg code digging into it.
We've generally tried to avoid the "force users to build packages from source" approach for Windows, since getting MSYS2 and the correct version of MinGW installed and working can be a bit challenging. It's been more effective to upload binaries containing the appropriate DLL's, and use BinDeps to download and extract them upon Pkg.add. The question is how you want to build and where you want to upload those binaries. If your package is possible to cross-compile from Linux to Windows (using Linux versions of the MinGW compilers), then we can write a short spec file and submit it to the OpenSUSE build service, which has a large selection of several hundred cross-compiled libraries for 32 and 64 bit Windows: https://build.opensuse.org/project/show/windows:mingw:win64 . I had recent success doing this for Lapack, HDF5, Metis, and the stack of COIN-OR libraries used by various JuliaOpt packages. There's a Julia package called WinRPM that works like a package-manager provider with BinDeps, and handles all the dependencies properly. Worrying about cross-compiling is a little confusing at first and more effort to set up initially, but using WinRPM and the build service makes future maintenance and interaction with dependencies much easier in the long term. You only need small tweaks to the spec file to update future versions or rebuild with a different set of options, for example. If you can point me to the source of the package and provide representative configure invocations, I can take a look at working on a spec file with you. -Tony On Friday, September 12, 2014 3:52:33 PM UTC-7, Bill Hart wrote: > > One hint I have is that I keep getting the error message "couldnt make > stderr distinct from stdout". > > Even if I build from a naked ,sys2 sh invoked without any decoration from > the Windows command prompt, everything builds fine. > > Although I can't detect any differences in the versions of gcc, make, ld, > etc, I can detect one difference between the shell from Julia and the shell > from msys2. When I do run(`ls /`) from Julia I am clearly in Git Bash. When > I do it from msys2 I am clearly in msys2. > > So it looks to me like some of the files distributed with Julia's GitBash > are causing issues with building from Julia on Windows. Maybe there are > some dll conflicts or something. > > Either way, I don't know how to fix the problem. Any advice would be > welcome. > > Bill. > > On Friday, 12 September 2014 23:06:38 UTC+2, Bill Hart wrote: >> >> Hi Isaiah, >> >> Thanks for the offer of help on Windows. I think I am at the stage where >> I need to ask for it. >> >> Here is the repo so far: >> >> https://github.com/wbhart/Nemo >> >> The problem I have now is as follows. >> >> I have msys2 installed on my computer, with Ming64. Whilst each of the >> packages MPIR, MPFR and flint builds just fine from within the msys shell, >> the tools when called individually from Julia always seem to fail, exactly >> as the old Mingw 1.0 tools did. I've checked carefully that the versions of >> make, gcc, bash, etc that answer at the command prompt are all from msys2 >> and I've removed all old mingw and msys installations from my machine >> totally. I've tried building dozens of times, but it usually crashes early >> on in the MPIR install. >> >> I have set MSYSTEM=MINGW64, which I notice the msys2-mingw64 shell does. >> But I cannot figure out what else it does to stop the crashes. >> >> Do you have any insight into this? >> >> I wasn't sure what you meant about contacting you on github, as they've >> removed the PM service. So I figured I'd just post here, also giving other >> knowledgeable individuals a chance to comment. >> >> Bill. >> >> On Wednesday, 10 September 2014 16:26:10 UTC+2, Isaiah wrote: >>> >>> >>> This was what I thought of trying first. But I couldn't figure out how >>>> it worked out what GitHub repository to associate this with, or whether it >>>> would try to create one, possibly scrubbing my existing nemo repository on >>>> GitHub. Obviously I don't want to lose my commit history. >>> >>> >>> For Pkg manager purposes, the association will be created later (when >>> you register the package). >>> >>> It also isn't clear where Julia creates the empty git repository. In the >>>> current directory? Or in some subdirectory of the Julia source tree? >>> >>> >>> Under `$HOME/.julia/v0.3` (or v0.4 if you are on git master) >>> >>> For the most part I can just run configure, make, make install for now >>>> and set some library paths (if I can figure out what kind of system I am >>>> on). >>> >>> >>> There are some macros to help with this: @osx, @linux, @unix (both), and >>> @windows. There is also a variable called OS_NAME with a platform-specific >>> value (:Windows, :Linux, etc.) See: >>> >>> http://docs.julialang.org/en/release-0.3/manual/calling-c-and-fortran-code/#handling-platform-variations >>> >>> >>> >>> On Wed, Sep 10, 2014 at 10:18 AM, Bill Hart <[email protected]> >>> wrote: >>> >>>> >>>> >>>> On Wednesday, 10 September 2014 15:57:56 UTC+2, Isaiah wrote: >>>>> >>>>> Is there documentation somewhere explaining how to do the latter? Or >>>>>> can someone help me with doing the latter? >>>>> >>>>> >>>>> You could run `Pkg.generate("Nemo")` and then copy and commit (some >>>>> of) the resulting files in your own Nemo git tree; there aren't very many. >>>>> >>>> >>>> This was what I thought of trying first. But I couldn't figure out how >>>> it worked out what GitHub repository to associate this with, or whether it >>>> would try to create one, possibly scrubbing my existing nemo repository on >>>> GitHub. Obviously I don't want to lose my commit history. >>>> >>>> It also isn't clear where Julia creates the empty git repository. In >>>> the current directory? Or in some subdirectory of the Julia source tree? >>>> >>>> >>>>> I can't find any documentation explaining where to put the commands in >>>>>> a Pkg to actually git clone flint, build it, install it and set up paths >>>>>> for Nemo. Given the complexities of installing flint for the user, I'd >>>>>> like >>>>>> to have the Julia package manager do this automatically if at all >>>>>> possible. >>>>>> And I see it does seem to be possible. I just can't figure out how. >>>>>> >>>>> >>>>> The Pkg manager will look for a file called `MYPKG/deps/build.jl` and >>>>> run that if it exists. That's just a Julia file, so you can do whatever >>>>> you >>>>> want there (shell out, etc.). >>>>> >>>> >>>> Perfect. For the most part I can just run configure, make, make install >>>> for now and set some library paths (if I can figure out what kind of >>>> system >>>> I am on). >>>> >>>> Finding the Julia installation on the system in order to link against >>>> the gmp/mpfr might be slightly more difficult. >>>> >>>> >>>>> One option is to use the BinDeps package which provides primitives for >>>>> interacting with various package managers and build systems: >>>>> >>>>> https://github.com/JuliaLang/BinDeps.jl >>>>> >>>>> A very advanced and fully-developed usage example can be found in the >>>>> Cairo package, which has Autotools, Apt, Yum, and several other targets: >>>>> >>>>> https://github.com/JuliaLang/Cairo.jl/blob/master/deps/build.jl >>>>> >>>>> There are a number of other examples to draw from. Hopefully the above >>>>> links will give you a sense of where to start. I can help out on Linux >>>>> and >>>>> Windows (@ihnorton on github). >>>>> >>>> >>>> Thanks. >>>> >>>> Bill. >>>> >>>> >>>>> >>>>> >>>>> On Wed, Sep 10, 2014 at 9:31 AM, Bill Hart <[email protected]> >>>>> wrote: >>>>> >>>>>> Hi, >>>>>> >>>>>> I have been writing a new Julia package, which I have called Nemo >>>>>> (it's essentially a limited computer algebra system). >>>>>> >>>>>> I have two specific problems: >>>>>> >>>>>> 1) The git and GitHub repository for Nemo already exists, but I >>>>>> haven't created a Julia Pkg yet. >>>>>> >>>>>> https://github.com/wbhart/nemo >>>>>> >>>>>> The documentation on creating a Julia Pkg seems to assume you are >>>>>> going to start with the Pkg then commit code to the git repository that >>>>>> it >>>>>> creates, not create a git/github project and then add the necessary >>>>>> stuff >>>>>> to turn it into a Julia package. >>>>>> >>>>>> Is there documentation somewhere explaining how to do the latter? Or >>>>>> can someone help me with doing the latter? >>>>>> >>>>>> (I have a couple of small build issues to fix in order for flint to >>>>>> work on Windows 64 before it will work there. But I will be working on >>>>>> those right away. I have managed to get it to work with Julia there, >>>>>> just >>>>>> not hacked the fixes into the flint build system yet. Other than this >>>>>> minor >>>>>> thing, I am quite ready to publish Nemo as a package right away (well, >>>>>> apart from a horrible 3x slowdown and excessive memory usage caused by >>>>>> gc, >>>>>> but I think I've given up on solving that problem for now).) >>>>>> >>>>>> 2) Nemo relies on mpir (or GMP), mpfr and flint, which are large >>>>>> external C/assembly libraries which need to get built or be available to >>>>>> run Nemo. I understand Julia has its own GMP and MPFR which I can >>>>>> probably >>>>>> link to if they are recent enough. >>>>>> >>>>>> Flint needs to be built when the package is installed. It takes a >>>>>> long time to build, e.g. 40 minutes or so on Windows, maybe a third of >>>>>> that >>>>>> on Linux. >>>>>> >>>>>> I can't find any documentation explaining where to put the commands >>>>>> in a Pkg to actually git clone flint, build it, install it and set up >>>>>> paths >>>>>> for Nemo. Given the complexities of installing flint for the user, I'd >>>>>> like >>>>>> to have the Julia package manager do this automatically if at all >>>>>> possible. >>>>>> And I see it does seem to be possible. I just can't figure out how. >>>>>> >>>>>> Flint is here: >>>>>> >>>>>> https://github.com/wbhart/flint2 >>>>>> >>>>>> Can anyone help, or point me in the right direction? >>>>>> >>>>>> Bill. >>>>>> >>>>> >>>>> >>>
