I should mention that there are some complications. Flint needs to be built in situ because it needs a large .txt file at runtime which contains data it uses (Conway polynomials). The only way to tell flint where it will be is through the build system, and the only way to know where it will be is to actually build flint.
Moreover, the system linker cannot find the dependencies MPIR and MPFR of flint unless the -rpath is set by flint during build. And again, the only way -rpath can be set is if flint knows where it is being built. So I think source build is probably the only feasible option, actually. Having said that, I cannot figure out how to revert the PATH to the system provided one in Julia. It just seems to clobber it. Bill. On 13 September 2014 02:47, Bill Hart <[email protected]> wrote: > Hi Tony, > > I'm worried about the legality of downloading binaries for users, since > the packages involved, GMP/MPIR, MPFR and FLINT are (L)GPL. > > I see now that Julia adds things to the system PATH, which is why it is > picking up things in the GitBash bundled in Julia. If I can figure out how > to get PATH back to its original state temporarily, I think I can get the > build to succeed (supposing the user has MSYS2 and Mingw64 correctly > installed -- not easy, admittedly). > > I agree binary distribution is the normal method on Windows. But I've > already been forced to go through one FSF "license compliance review" > because a GNU project reported us for being in breach of the license. I'm > not doing it again. > > If there is a way to do things in strict accordance with the LGPL and GPL > I can go down that route. In that case, you can see the invocations here: > > https://github.com/wbhart/Nemo/blob/master/deps/build.jl > > Is that sufficient? > > Bill. > > On 13 September 2014 02:24, Tony Kelman <[email protected]> wrote: > >> Bill, >> >> Shelling out in Julia on Windows currently calls into the bundled >> Git-bash, for the purposes of the package manager. That will hopefully >> change in 0.4 by replacing the bundled Git-bash with libgit2 bindings, but >> PR 7339 is currently a bit stalled for lack of anyone who understands the >> LibGit2.jl / Pkg code digging into it. >> >> We've generally tried to avoid the "force users to build packages from >> source" approach for Windows, since getting MSYS2 and the correct version >> of MinGW installed and working can be a bit challenging. It's been more >> effective to upload binaries containing the appropriate DLL's, and use >> BinDeps to download and extract them upon Pkg.add. >> >> The question is how you want to build and where you want to upload those >> binaries. If your package is possible to cross-compile from Linux to >> Windows (using Linux versions of the MinGW compilers), then we can write a >> short spec file and submit it to the OpenSUSE build service, which has a >> large selection of several hundred cross-compiled libraries for 32 and 64 >> bit Windows: https://build.opensuse.org/project/show/windows:mingw:win64 >> . I had recent success doing this for Lapack, HDF5, Metis, and the stack of >> COIN-OR libraries used by various JuliaOpt packages. There's a Julia >> package called WinRPM that works like a package-manager provider with >> BinDeps, and handles all the dependencies properly. >> >> Worrying about cross-compiling is a little confusing at first and more >> effort to set up initially, but using WinRPM and the build service makes >> future maintenance and interaction with dependencies much easier in the >> long term. You only need small tweaks to the spec file to update future >> versions or rebuild with a different set of options, for example. >> >> If you can point me to the source of the package and provide >> representative configure invocations, I can take a look at working on a >> spec file with you. >> >> -Tony >> >> >> On Friday, September 12, 2014 3:52:33 PM UTC-7, Bill Hart wrote: >>> >>> One hint I have is that I keep getting the error message "couldnt make >>> stderr distinct from stdout". >>> >>> Even if I build from a naked ,sys2 sh invoked without any decoration >>> from the Windows command prompt, everything builds fine. >>> >>> Although I can't detect any differences in the versions of gcc, make, >>> ld, etc, I can detect one difference between the shell from Julia and the >>> shell from msys2. When I do run(`ls /`) from Julia I am clearly in Git >>> Bash. When I do it from msys2 I am clearly in msys2. >>> >>> So it looks to me like some of the files distributed with Julia's >>> GitBash are causing issues with building from Julia on Windows. Maybe there >>> are some dll conflicts or something. >>> >>> Either way, I don't know how to fix the problem. Any advice would be >>> welcome. >>> >>> Bill. >>> >>> On Friday, 12 September 2014 23:06:38 UTC+2, Bill Hart wrote: >>>> >>>> Hi Isaiah, >>>> >>>> Thanks for the offer of help on Windows. I think I am at the stage >>>> where I need to ask for it. >>>> >>>> Here is the repo so far: >>>> >>>> https://github.com/wbhart/Nemo >>>> >>>> The problem I have now is as follows. >>>> >>>> I have msys2 installed on my computer, with Ming64. Whilst each of the >>>> packages MPIR, MPFR and flint builds just fine from within the msys shell, >>>> the tools when called individually from Julia always seem to fail, exactly >>>> as the old Mingw 1.0 tools did. I've checked carefully that the versions of >>>> make, gcc, bash, etc that answer at the command prompt are all from msys2 >>>> and I've removed all old mingw and msys installations from my machine >>>> totally. I've tried building dozens of times, but it usually crashes early >>>> on in the MPIR install. >>>> >>>> I have set MSYSTEM=MINGW64, which I notice the msys2-mingw64 shell >>>> does. But I cannot figure out what else it does to stop the crashes. >>>> >>>> Do you have any insight into this? >>>> >>>> I wasn't sure what you meant about contacting you on github, as they've >>>> removed the PM service. So I figured I'd just post here, also giving other >>>> knowledgeable individuals a chance to comment. >>>> >>>> Bill. >>>> >>>> On Wednesday, 10 September 2014 16:26:10 UTC+2, Isaiah wrote: >>>>> >>>>> >>>>> This was what I thought of trying first. But I couldn't figure out how >>>>>> it worked out what GitHub repository to associate this with, or whether >>>>>> it >>>>>> would try to create one, possibly scrubbing my existing nemo repository >>>>>> on >>>>>> GitHub. Obviously I don't want to lose my commit history. >>>>> >>>>> >>>>> For Pkg manager purposes, the association will be created later (when >>>>> you register the package). >>>>> >>>>> It also isn't clear where Julia creates the empty git repository. In >>>>>> the current directory? Or in some subdirectory of the Julia source tree? >>>>> >>>>> >>>>> Under `$HOME/.julia/v0.3` (or v0.4 if you are on git master) >>>>> >>>>> For the most part I can just run configure, make, make install for now >>>>>> and set some library paths (if I can figure out what kind of system I am >>>>>> on). >>>>> >>>>> >>>>> There are some macros to help with this: @osx, @linux, @unix (both), >>>>> and @windows. There is also a variable called OS_NAME with a >>>>> platform-specific value (:Windows, :Linux, etc.) See: >>>>> http://docs.julialang.org/en/release-0.3/manual/calling-c- >>>>> and-fortran-code/#handling-platform-variations >>>>> >>>>> >>>>> >>>>> On Wed, Sep 10, 2014 at 10:18 AM, Bill Hart <[email protected]> >>>>> wrote: >>>>> >>>>>> >>>>>> >>>>>> On Wednesday, 10 September 2014 15:57:56 UTC+2, Isaiah wrote: >>>>>>> >>>>>>> Is there documentation somewhere explaining how to do the latter? Or >>>>>>>> can someone help me with doing the latter? >>>>>>> >>>>>>> >>>>>>> You could run `Pkg.generate("Nemo")` and then copy and commit (some >>>>>>> of) the resulting files in your own Nemo git tree; there aren't very >>>>>>> many. >>>>>>> >>>>>> >>>>>> This was what I thought of trying first. But I couldn't figure out >>>>>> how it worked out what GitHub repository to associate this with, or >>>>>> whether >>>>>> it would try to create one, possibly scrubbing my existing nemo >>>>>> repository >>>>>> on GitHub. Obviously I don't want to lose my commit history. >>>>>> >>>>>> It also isn't clear where Julia creates the empty git repository. In >>>>>> the current directory? Or in some subdirectory of the Julia source tree? >>>>>> >>>>>> >>>>>>> I can't find any documentation explaining where to put the commands >>>>>>>> in a Pkg to actually git clone flint, build it, install it and set up >>>>>>>> paths >>>>>>>> for Nemo. Given the complexities of installing flint for the user, I'd >>>>>>>> like >>>>>>>> to have the Julia package manager do this automatically if at all >>>>>>>> possible. >>>>>>>> And I see it does seem to be possible. I just can't figure out how. >>>>>>>> >>>>>>> >>>>>>> The Pkg manager will look for a file called `MYPKG/deps/build.jl` >>>>>>> and run that if it exists. That's just a Julia file, so you can do >>>>>>> whatever >>>>>>> you want there (shell out, etc.). >>>>>>> >>>>>> >>>>>> Perfect. For the most part I can just run configure, make, make >>>>>> install for now and set some library paths (if I can figure out what kind >>>>>> of system I am on). >>>>>> >>>>>> Finding the Julia installation on the system in order to link against >>>>>> the gmp/mpfr might be slightly more difficult. >>>>>> >>>>>> >>>>>>> One option is to use the BinDeps package which provides primitives >>>>>>> for interacting with various package managers and build systems: >>>>>>> >>>>>>> https://github.com/JuliaLang/BinDeps.jl >>>>>>> >>>>>>> A very advanced and fully-developed usage example can be found in >>>>>>> the Cairo package, which has Autotools, Apt, Yum, and several other >>>>>>> targets: >>>>>>> >>>>>>> https://github.com/JuliaLang/Cairo.jl/blob/master/deps/build.jl >>>>>>> >>>>>>> There are a number of other examples to draw from. Hopefully the >>>>>>> above links will give you a sense of where to start. I can help out on >>>>>>> Linux and Windows (@ihnorton on github). >>>>>>> >>>>>> >>>>>> Thanks. >>>>>> >>>>>> Bill. >>>>>> >>>>>> >>>>>>> >>>>>>> >>>>>>> On Wed, Sep 10, 2014 at 9:31 AM, Bill Hart <[email protected] >>>>>>> > wrote: >>>>>>> >>>>>>>> Hi, >>>>>>>> >>>>>>>> I have been writing a new Julia package, which I have called Nemo >>>>>>>> (it's essentially a limited computer algebra system). >>>>>>>> >>>>>>>> I have two specific problems: >>>>>>>> >>>>>>>> 1) The git and GitHub repository for Nemo already exists, but I >>>>>>>> haven't created a Julia Pkg yet. >>>>>>>> >>>>>>>> https://github.com/wbhart/nemo >>>>>>>> >>>>>>>> The documentation on creating a Julia Pkg seems to assume you are >>>>>>>> going to start with the Pkg then commit code to the git repository >>>>>>>> that it >>>>>>>> creates, not create a git/github project and then add the necessary >>>>>>>> stuff >>>>>>>> to turn it into a Julia package. >>>>>>>> >>>>>>>> Is there documentation somewhere explaining how to do the latter? >>>>>>>> Or can someone help me with doing the latter? >>>>>>>> >>>>>>>> (I have a couple of small build issues to fix in order for flint to >>>>>>>> work on Windows 64 before it will work there. But I will be working on >>>>>>>> those right away. I have managed to get it to work with Julia there, >>>>>>>> just >>>>>>>> not hacked the fixes into the flint build system yet. Other than this >>>>>>>> minor >>>>>>>> thing, I am quite ready to publish Nemo as a package right away (well, >>>>>>>> apart from a horrible 3x slowdown and excessive memory usage caused by >>>>>>>> gc, >>>>>>>> but I think I've given up on solving that problem for now).) >>>>>>>> >>>>>>>> 2) Nemo relies on mpir (or GMP), mpfr and flint, which are large >>>>>>>> external C/assembly libraries which need to get built or be available >>>>>>>> to >>>>>>>> run Nemo. I understand Julia has its own GMP and MPFR which I can >>>>>>>> probably >>>>>>>> link to if they are recent enough. >>>>>>>> >>>>>>>> Flint needs to be built when the package is installed. It takes a >>>>>>>> long time to build, e.g. 40 minutes or so on Windows, maybe a third of >>>>>>>> that >>>>>>>> on Linux. >>>>>>>> >>>>>>>> I can't find any documentation explaining where to put the commands >>>>>>>> in a Pkg to actually git clone flint, build it, install it and set up >>>>>>>> paths >>>>>>>> for Nemo. Given the complexities of installing flint for the user, I'd >>>>>>>> like >>>>>>>> to have the Julia package manager do this automatically if at all >>>>>>>> possible. >>>>>>>> And I see it does seem to be possible. I just can't figure out how. >>>>>>>> >>>>>>>> Flint is here: >>>>>>>> >>>>>>>> https://github.com/wbhart/flint2 >>>>>>>> >>>>>>>> Can anyone help, or point me in the right direction? >>>>>>>> >>>>>>>> Bill. >>>>>>>> >>>>>>> >>>>>>> >>>>> >
