Hi Tony, I'm worried about the legality of downloading binaries for users, since the packages involved, GMP/MPIR, MPFR and FLINT are (L)GPL.
I see now that Julia adds things to the system PATH, which is why it is picking up things in the GitBash bundled in Julia. If I can figure out how to get PATH back to its original state temporarily, I think I can get the build to succeed (supposing the user has MSYS2 and Mingw64 correctly installed -- not easy, admittedly). I agree binary distribution is the normal method on Windows. But I've already been forced to go through one FSF "license compliance review" because a GNU project reported us for being in breach of the license. I'm not doing it again. If there is a way to do things in strict accordance with the LGPL and GPL I can go down that route. In that case, you can see the invocations here: https://github.com/wbhart/Nemo/blob/master/deps/build.jl Is that sufficient? Bill. On 13 September 2014 02:24, Tony Kelman <[email protected]> wrote: > Bill, > > Shelling out in Julia on Windows currently calls into the bundled > Git-bash, for the purposes of the package manager. That will hopefully > change in 0.4 by replacing the bundled Git-bash with libgit2 bindings, but > PR 7339 is currently a bit stalled for lack of anyone who understands the > LibGit2.jl / Pkg code digging into it. > > We've generally tried to avoid the "force users to build packages from > source" approach for Windows, since getting MSYS2 and the correct version > of MinGW installed and working can be a bit challenging. It's been more > effective to upload binaries containing the appropriate DLL's, and use > BinDeps to download and extract them upon Pkg.add. > > The question is how you want to build and where you want to upload those > binaries. If your package is possible to cross-compile from Linux to > Windows (using Linux versions of the MinGW compilers), then we can write a > short spec file and submit it to the OpenSUSE build service, which has a > large selection of several hundred cross-compiled libraries for 32 and 64 > bit Windows: https://build.opensuse.org/project/show/windows:mingw:win64 > . I had recent success doing this for Lapack, HDF5, Metis, and the stack of > COIN-OR libraries used by various JuliaOpt packages. There's a Julia > package called WinRPM that works like a package-manager provider with > BinDeps, and handles all the dependencies properly. > > Worrying about cross-compiling is a little confusing at first and more > effort to set up initially, but using WinRPM and the build service makes > future maintenance and interaction with dependencies much easier in the > long term. You only need small tweaks to the spec file to update future > versions or rebuild with a different set of options, for example. > > If you can point me to the source of the package and provide > representative configure invocations, I can take a look at working on a > spec file with you. > > -Tony > > > On Friday, September 12, 2014 3:52:33 PM UTC-7, Bill Hart wrote: >> >> One hint I have is that I keep getting the error message "couldnt make >> stderr distinct from stdout". >> >> Even if I build from a naked ,sys2 sh invoked without any decoration from >> the Windows command prompt, everything builds fine. >> >> Although I can't detect any differences in the versions of gcc, make, ld, >> etc, I can detect one difference between the shell from Julia and the shell >> from msys2. When I do run(`ls /`) from Julia I am clearly in Git Bash. When >> I do it from msys2 I am clearly in msys2. >> >> So it looks to me like some of the files distributed with Julia's GitBash >> are causing issues with building from Julia on Windows. Maybe there are >> some dll conflicts or something. >> >> Either way, I don't know how to fix the problem. Any advice would be >> welcome. >> >> Bill. >> >> On Friday, 12 September 2014 23:06:38 UTC+2, Bill Hart wrote: >>> >>> Hi Isaiah, >>> >>> Thanks for the offer of help on Windows. I think I am at the stage where >>> I need to ask for it. >>> >>> Here is the repo so far: >>> >>> https://github.com/wbhart/Nemo >>> >>> The problem I have now is as follows. >>> >>> I have msys2 installed on my computer, with Ming64. Whilst each of the >>> packages MPIR, MPFR and flint builds just fine from within the msys shell, >>> the tools when called individually from Julia always seem to fail, exactly >>> as the old Mingw 1.0 tools did. I've checked carefully that the versions of >>> make, gcc, bash, etc that answer at the command prompt are all from msys2 >>> and I've removed all old mingw and msys installations from my machine >>> totally. I've tried building dozens of times, but it usually crashes early >>> on in the MPIR install. >>> >>> I have set MSYSTEM=MINGW64, which I notice the msys2-mingw64 shell does. >>> But I cannot figure out what else it does to stop the crashes. >>> >>> Do you have any insight into this? >>> >>> I wasn't sure what you meant about contacting you on github, as they've >>> removed the PM service. So I figured I'd just post here, also giving other >>> knowledgeable individuals a chance to comment. >>> >>> Bill. >>> >>> On Wednesday, 10 September 2014 16:26:10 UTC+2, Isaiah wrote: >>>> >>>> >>>> This was what I thought of trying first. But I couldn't figure out how >>>>> it worked out what GitHub repository to associate this with, or whether it >>>>> would try to create one, possibly scrubbing my existing nemo repository on >>>>> GitHub. Obviously I don't want to lose my commit history. >>>> >>>> >>>> For Pkg manager purposes, the association will be created later (when >>>> you register the package). >>>> >>>> It also isn't clear where Julia creates the empty git repository. In >>>>> the current directory? Or in some subdirectory of the Julia source tree? >>>> >>>> >>>> Under `$HOME/.julia/v0.3` (or v0.4 if you are on git master) >>>> >>>> For the most part I can just run configure, make, make install for now >>>>> and set some library paths (if I can figure out what kind of system I am >>>>> on). >>>> >>>> >>>> There are some macros to help with this: @osx, @linux, @unix (both), >>>> and @windows. There is also a variable called OS_NAME with a >>>> platform-specific value (:Windows, :Linux, etc.) See: >>>> http://docs.julialang.org/en/release-0.3/manual/calling-c- >>>> and-fortran-code/#handling-platform-variations >>>> >>>> >>>> >>>> On Wed, Sep 10, 2014 at 10:18 AM, Bill Hart <[email protected]> >>>> wrote: >>>> >>>>> >>>>> >>>>> On Wednesday, 10 September 2014 15:57:56 UTC+2, Isaiah wrote: >>>>>> >>>>>> Is there documentation somewhere explaining how to do the latter? Or >>>>>>> can someone help me with doing the latter? >>>>>> >>>>>> >>>>>> You could run `Pkg.generate("Nemo")` and then copy and commit (some >>>>>> of) the resulting files in your own Nemo git tree; there aren't very >>>>>> many. >>>>>> >>>>> >>>>> This was what I thought of trying first. But I couldn't figure out how >>>>> it worked out what GitHub repository to associate this with, or whether it >>>>> would try to create one, possibly scrubbing my existing nemo repository on >>>>> GitHub. Obviously I don't want to lose my commit history. >>>>> >>>>> It also isn't clear where Julia creates the empty git repository. In >>>>> the current directory? Or in some subdirectory of the Julia source tree? >>>>> >>>>> >>>>>> I can't find any documentation explaining where to put the commands >>>>>>> in a Pkg to actually git clone flint, build it, install it and set up >>>>>>> paths >>>>>>> for Nemo. Given the complexities of installing flint for the user, I'd >>>>>>> like >>>>>>> to have the Julia package manager do this automatically if at all >>>>>>> possible. >>>>>>> And I see it does seem to be possible. I just can't figure out how. >>>>>>> >>>>>> >>>>>> The Pkg manager will look for a file called `MYPKG/deps/build.jl` and >>>>>> run that if it exists. That's just a Julia file, so you can do whatever >>>>>> you >>>>>> want there (shell out, etc.). >>>>>> >>>>> >>>>> Perfect. For the most part I can just run configure, make, make >>>>> install for now and set some library paths (if I can figure out what kind >>>>> of system I am on). >>>>> >>>>> Finding the Julia installation on the system in order to link against >>>>> the gmp/mpfr might be slightly more difficult. >>>>> >>>>> >>>>>> One option is to use the BinDeps package which provides primitives >>>>>> for interacting with various package managers and build systems: >>>>>> >>>>>> https://github.com/JuliaLang/BinDeps.jl >>>>>> >>>>>> A very advanced and fully-developed usage example can be found in the >>>>>> Cairo package, which has Autotools, Apt, Yum, and several other targets: >>>>>> >>>>>> https://github.com/JuliaLang/Cairo.jl/blob/master/deps/build.jl >>>>>> >>>>>> There are a number of other examples to draw from. Hopefully the >>>>>> above links will give you a sense of where to start. I can help out on >>>>>> Linux and Windows (@ihnorton on github). >>>>>> >>>>> >>>>> Thanks. >>>>> >>>>> Bill. >>>>> >>>>> >>>>>> >>>>>> >>>>>> On Wed, Sep 10, 2014 at 9:31 AM, Bill Hart <[email protected]> >>>>>> wrote: >>>>>> >>>>>>> Hi, >>>>>>> >>>>>>> I have been writing a new Julia package, which I have called Nemo >>>>>>> (it's essentially a limited computer algebra system). >>>>>>> >>>>>>> I have two specific problems: >>>>>>> >>>>>>> 1) The git and GitHub repository for Nemo already exists, but I >>>>>>> haven't created a Julia Pkg yet. >>>>>>> >>>>>>> https://github.com/wbhart/nemo >>>>>>> >>>>>>> The documentation on creating a Julia Pkg seems to assume you are >>>>>>> going to start with the Pkg then commit code to the git repository that >>>>>>> it >>>>>>> creates, not create a git/github project and then add the necessary >>>>>>> stuff >>>>>>> to turn it into a Julia package. >>>>>>> >>>>>>> Is there documentation somewhere explaining how to do the latter? Or >>>>>>> can someone help me with doing the latter? >>>>>>> >>>>>>> (I have a couple of small build issues to fix in order for flint to >>>>>>> work on Windows 64 before it will work there. But I will be working on >>>>>>> those right away. I have managed to get it to work with Julia there, >>>>>>> just >>>>>>> not hacked the fixes into the flint build system yet. Other than this >>>>>>> minor >>>>>>> thing, I am quite ready to publish Nemo as a package right away (well, >>>>>>> apart from a horrible 3x slowdown and excessive memory usage caused by >>>>>>> gc, >>>>>>> but I think I've given up on solving that problem for now).) >>>>>>> >>>>>>> 2) Nemo relies on mpir (or GMP), mpfr and flint, which are large >>>>>>> external C/assembly libraries which need to get built or be available to >>>>>>> run Nemo. I understand Julia has its own GMP and MPFR which I can >>>>>>> probably >>>>>>> link to if they are recent enough. >>>>>>> >>>>>>> Flint needs to be built when the package is installed. It takes a >>>>>>> long time to build, e.g. 40 minutes or so on Windows, maybe a third of >>>>>>> that >>>>>>> on Linux. >>>>>>> >>>>>>> I can't find any documentation explaining where to put the commands >>>>>>> in a Pkg to actually git clone flint, build it, install it and set up >>>>>>> paths >>>>>>> for Nemo. Given the complexities of installing flint for the user, I'd >>>>>>> like >>>>>>> to have the Julia package manager do this automatically if at all >>>>>>> possible. >>>>>>> And I see it does seem to be possible. I just can't figure out how. >>>>>>> >>>>>>> Flint is here: >>>>>>> >>>>>>> https://github.com/wbhart/flint2 >>>>>>> >>>>>>> Can anyone help, or point me in the right direction? >>>>>>> >>>>>>> Bill. >>>>>>> >>>>>> >>>>>> >>>>
