Re: Question about lang/ghc module (trying to port git-annex)
> > A few notes / questions: > > > > * I fixed a couple of typos in > > > > https://github.com/falsifian/ports/commit/98111e702634842865a3d27675b6848e3fb2b7ca > > > > * I added example usage for cabal-bundler. I hope I got it right: > > > > https://github.com/falsifian/ports/commit/f21cb097f215831b1d08443f9074073895a13731 > > Neat. I'll pick these up. > > > * If I find more time to work on this, should I polish my git-annex > > port, or is there something more useful I could be doing with your > > Cabal infrastructure? (I installed darcs and it seems to work.) > > Great! I don't think there's much else to do besides waiting for > cabal-install 3.4.0.0 to be pushed to Hackage. Depending on how this > aligns with 6.9 release we may or may not ship it then. > > You could try to build your own port of a Haskell binary you use that we > don't have yet. Your experience will likely tell us something about the > level of maturity of this infra. The only unported Haskell binaries I use are personal projects that are not on Hackage. I could try locally porting one of those as an experiment. > > * I tried to enable tests, partly because Darcs has a comprehensive > > test suite. I ran into some trouble. My WIP is here: > > > > https://github.com/falsifian/ports/commit/99cb0e33e2263cf15979a19bf068d345630c > > One problem is that cabal-bundler doesn't seem to be outputting test > > dependencies. > > I keep track of this issue in https://github.com/blackgnezdo/ports/issues/13. > See if you agree with my assessment there. Thanks, I commented there. > > * It's a pity that your approach doesn't allow Cabal-based ports to > > share dependencies. > > I chose to trade-off compute to gain simlpicity and flexibility: > https://github.com/blackgnezdo/ports/issues/3#issuecomment-650823377 > > Until this choice turns to be expensive enough to cause port builders to > complain I'm inclined to keep this approach. Okay, that's reasonable. My only concern: later on if people do complain and it's decided it's a real problem, is there a path forward that builds on what you've already got? Or would you have to start from scratch to address it? (I could complain already, but for me at least your solution is a definite improvement over the status quo. The build times aren't the end of the world.) > > Some thoughts on mitigating or solving this, in decreasing order of > > plausability: > > * Is it possible to hand-pick some particularly slow ones, make > > packages for them, and then tell Cabal to use those packages? E.g. > > aeson takes a while to build. > > Sadly this will move towards tree-wide configuration management which > was a very non-fun thing to do as you experienced first-hand trying to > deal with a ton of ports when you tried to add git-annex. Maybe there could be a rule of thumb, like adding a reusable port is only worth it if it reduces the total time to build all ports by at least X hours? I don't know if there are any libraries that meet that definition for an appropriate value of X. > > I guess it would be ugly to fill the ports tree with hundreds(?) of > > directories each with an auto-generated package. I'm not sure if > > there's a way to smoosh it all into one big file. E.g. I see > > there's something called "multi-packages", but I'm guessing that's > > not designed to handle hundreds of subpackages. > > I've seen how these things play out in much bigger code bases with many > people over time. We don't have enough people working on ports@ to > manage this. That makes sense. I imagine your approach will keep things easier for maintainers: e.g. I'm not going to accidentally break darcs by updating the dependencies of git-annex. > > * Is it possible to cheat, by having all the Cabal-based ports share > > their Cabal directory? So if I build darcs then git-annex, Cabal > > has access to all the build products from darcs when it builds > > git-annex. I guess one problem is this might make the build output > > depend on the order in which you build things, if Cabal > > opportunistically uses whatever versions are there. > > The root of the problem is GHC compilation speed. My preferred and > universally useful optimizations would be: > > * Use/build something like ccache for ghc > * Speed up ghc itself Those would be cool! > > * Are you planning to commit these changes all at once? Or is there > > some way to have an intermediate state where both old-style and > > new-style packages work? > > I don't see how it would be possible to have the old and new coexist. > ghc-8.10 is incompatible with the haskell ports in the tree. My plan is > to land this as a bundle. I wrote it up in an email to ports@ before. I didn't know about the incompatibility. Maybe there could be two separate ghc versions for a while, but I guess that's more trouble than it's worth. > Thanks > Greg -- James
Re: Question about lang/ghc module (trying to port git-annex)
Greg Steuck writes: > You could test pandoc as it is done: > https://github.com/blackgnezdo/ports/commit/609197ce3f84a2e4a147fcabcc5eb71d3cf10ca2 > > The full list is https://github.com/blackgnezdo/ports/commits/ghc-8.10 Splendid! Thank you for this.
Re: Question about lang/ghc module (trying to port git-annex)
On Fri, Feb 19, 2021 at 5:19 AM Ashton Fagg wrote > Hi Greg, > > I've been considering trying to port pandoc, but was put off by the > sheer amount of dependencies that would need to be ported as well. > > I'll give your work a try and see if it works well for that also, if > that's of interest to you? > You could test pandoc as it is done: https://github.com/blackgnezdo/ports/commit/609197ce3f84a2e4a147fcabcc5eb71d3cf10ca2 The full list is https://github.com/blackgnezdo/ports/commits/ghc-8.10 Thanks Greg -- nest.cx is Gmail hosted, use PGP: https://pgp.key-server.io/0x0B1542BD8DF5A1B0 Fingerprint: 5E2B 2D0E 1E03 2046 BEC3 4D50 0B15 42BD 8DF5 A1B0
Re: Question about lang/ghc module (trying to port git-annex)
Greg Steuck writes: > This is exactly the problem I want to solve with cabal.port.mk. You > can try to look at > https://marc.info/?l=openbsd-ports&m=160858285410366&w=2 > A quick search in the archives will show the justification and the > history of the effort. > > The current state of the work is I'm waiting on cabal 3.4 official > release gated by ghc-9.0. I'm also looking for people needing this work > and your attempt to add git-annex gives me more motivation to finish. Hi Greg, I've been considering trying to port pandoc, but was put off by the sheer amount of dependencies that would need to be ported as well. I'll give your work a try and see if it works well for that also, if that's of interest to you? Cheers, Ash
Re: Question about lang/ghc module (trying to port git-annex)
Hi James, James Cook writes: > Incidentally, I asked the maintainer how man pages are supposed to be > installed, and they said it's best to use the Makefile if you want a > complete installation. Probably not worth changing our port, but > something to keep in mind in case this turns out to be a pattern. > https://git-annex.branchable.com/bugs/__91__Patch__93___fix___34__mdwn2man__58___cannot_execute_-_...__34__/ Thanks. We can leave this as is unless it proves more trouble than adjusting the recipe going forward. > A few notes / questions: > > * I fixed a couple of typos in > > https://github.com/falsifian/ports/commit/98111e702634842865a3d27675b6848e3fb2b7ca > > * I added example usage for cabal-bundler. I hope I got it right: > > https://github.com/falsifian/ports/commit/f21cb097f215831b1d08443f9074073895a13731 Neat. I'll pick these up. > * If I find more time to work on this, should I polish my git-annex > port, or is there something more useful I could be doing with your > Cabal infrastructure? (I installed darcs and it seems to work.) Great! I don't think there's much else to do besides waiting for cabal-install 3.4.0.0 to be pushed to Hackage. Depending on how this aligns with 6.9 release we may or may not ship it then. You could try to build your own port of a Haskell binary you use that we don't have yet. Your experience will likely tell us something about the level of maturity of this infra. > * I tried to enable tests, partly because Darcs has a comprehensive > test suite. I ran into some trouble. My WIP is here: > > https://github.com/falsifian/ports/commit/99cb0e33e2263cf15979a19bf068d345630c > One problem is that cabal-bundler doesn't seem to be outputting test > dependencies. I keep track of this issue in https://github.com/blackgnezdo/ports/issues/13. See if you agree with my assessment there. > * It's a pity that your approach doesn't allow Cabal-based ports to > share dependencies. I chose to trade-off compute to gain simlpicity and flexibility: https://github.com/blackgnezdo/ports/issues/3#issuecomment-650823377 Until this choice turns to be expensive enough to cause port builders to complain I'm inclined to keep this approach. > Some thoughts on mitigating or solving this, in decreasing order of > plausability: > * Is it possible to hand-pick some particularly slow ones, make > packages for them, and then tell Cabal to use those packages? E.g. > aeson takes a while to build. Sadly this will move towards tree-wide configuration management which was a very non-fun thing to do as you experienced first-hand trying to deal with a ton of ports when you tried to add git-annex. > * nixpkgs (the only package system I'm really familiar with) deals > with Cabal by automatically generating package descriptions for > everything on hackage. I wonder if something along those lines > would be do-able. cabal-bundler --openbsd is a play in this area. I just limited it to the absolute minimum I could get away with. > I guess it would be ugly to fill the ports tree with hundreds(?) of > directories each with an auto-generated package. I'm not sure if > there's a way to smoosh it all into one big file. E.g. I see > there's something called "multi-packages", but I'm guessing that's > not designed to handle hundreds of subpackages. I've seen how these things play out in much bigger code bases with many people over time. We don't have enough people working on ports@ to manage this. > * Is it possible to cheat, by having all the Cabal-based ports share > their Cabal directory? So if I build darcs then git-annex, Cabal > has access to all the build products from darcs when it builds > git-annex. I guess one problem is this might make the build output > depend on the order in which you build things, if Cabal > opportunistically uses whatever versions are there. The root of the problem is GHC compilation speed. My preferred and universally useful optimizations would be: * Use/build something like ccache for ghc * Speed up ghc itself > * Are you planning to commit these changes all at once? Or is there > some way to have an intermediate state where both old-style and > new-style packages work? I don't see how it would be possible to have the old and new coexist. ghc-8.10 is incompatible with the haskell ports in the tree. My plan is to land this as a bundle. I wrote it up in an email to ports@ before. Thanks Greg
Re: Question about lang/ghc module (trying to port git-annex)
Hi Greg, > >> I'll dig into this a bit. I simply didn't need to worry about this up > >> until now as all the other ports don't do much custom Setup.hs work. > > > > I don't know if there will be any cases other than man pages and > > git-annex's command aliases. Not sure how much effort it's worth. > > The default action of doing nothing is valid :) Incidentally, I asked the maintainer how man pages are supposed to be installed, and they said it's best to use the Makefile if you want a complete installation. Probably not worth changing our port, but something to keep in mind in case this turns out to be a pattern. https://git-annex.branchable.com/bugs/__91__Patch__93___fix___34__mdwn2man__58___cannot_execute_-_...__34__/ A few notes / questions: * I fixed a couple of typos in https://github.com/falsifian/ports/commit/98111e702634842865a3d27675b6848e3fb2b7ca * I added example usage for cabal-bundler. I hope I got it right: https://github.com/falsifian/ports/commit/f21cb097f215831b1d08443f9074073895a13731 * If I find more time to work on this, should I polish my git-annex port, or is there something more useful I could be doing with your Cabal infrastructure? (I installed darcs and it seems to work.) * I tried to enable tests, partly because Darcs has a comprehensive test suite. I ran into some trouble. My WIP is here: https://github.com/falsifian/ports/commit/99cb0e33e2263cf15979a19bf068d345630c One problem is that cabal-bundler doesn't seem to be outputting test dependencies. (I tried manually adding some, not included in my commit.) Another is that, even if I add test dependencies manually, I end up with strange results like this: falsifian moth darcs $ make test ===> Regression tests for darcs-2.16.2 cd /usr/local/ports/pobj/darcs-2.16.2/darcs-2.16.2 && /usr/bin/env -i PORTSDIR="/usr/ports" LIBTOOL="/usr/bin/libtool" PATH='/usr/local/ports/pobj/darcs-2.16.2/bin:/usr/bin:/bin:/usr/sbin:/sbin:/usr/local/bin:/usr/X11R6/bin' PREFIX='/usr/local' LOCALBASE='/usr/local' X11BASE='/usr/X11R6' CFLAGS='-O2 -pipe' TRUEPREFIX='/usr/local' DESTDIR='' HOME='/darcs-2.16.2_writes_to_HOME' PICFLAG="-fpic" BINGRP=bin BINOWN=root BINMODE=755 NONBINMODE=644 DIRMODE=755 INSTALL_COPY=-c INSTALL_STRIP=-s MANGRP=bin MANOWN=root MANMODE=644 BSD_INSTALL_PROGRAM="/usr/local/ports/pobj/darcs-2.16.2/bin/install -c -s -m 755" BSD_INSTALL_SCRIPT="/usr/local/ports/pobj/darcs-2.16.2/bin/install -c -m 755" BSD_INSTALL_DATA="/usr/local/ports/pobj/darcs-2.16.2/bin/install -c -m 644" BSD_INSTALL_MAN="/usr/local/ports/pobj/darcs-2.16.2/bin/install -c -m 644" BSD_INSTALL_PROGRAM_DIR="/usr/local/ports/pobj/darcs-2.16.2/bin/install -d -m 755" BSD_INSTALL_SCRIPT_DIR="/usr/local/ports/pobj/darcs-2.16.2/bin/install -d -m 755" BSD_INSTALL_DATA_DIR="/usr/local/ports/pobj/darcs-2.16.2/bin/install -d -m 755" BSD_INSTALL_MAN_DIR="/usr/local/ports/pobj/darcs-2.16.2/bin/install -d -m 755" HOME=/usr/local/ports/pobj/darcs-2.16.2 /usr/local/bin/cabal v2-test --offline --disable-benchmarks -w /usr/local/bin/ghc -j1 --flags="curl -library" Warning: No remote package servers have been specified. Usually you would have one specified in the config file. Resolving dependencies... cabal: Could not resolve dependencies: [__0] trying: random-1.2.0 (user goal) [__1] trying: splitmix-0.1.0.3 (user goal) [__2] trying: splitmix:!bench [__3] rejecting: splitmix:*test (cyclic dependencies; conflict set: random, splitmix) [__1] fail (backjumping, conflict set: random, splitmix) After searching the rest of the dependency tree exhaustively, these were the goals I've had most trouble fulfilling: random, splitmix splitmix's test target depends on random and random depends on splitmix. Normally this doesn't cause any problem; I don't know why it's a problem here. * It's a pity that your approach doesn't allow Cabal-based ports to share dependencies. Some thoughts on mitigating or solving this, in decreasing order of plausability: * Is it possible to hand-pick some particularly slow ones, make packages for them, and then tell Cabal to use those packages? E.g. aeson takes a while to build. * nixpkgs (the only package system I'm really familiar with) deals with Cabal by automatically generating package descriptions for everything on hackage. I wonder if something along those lines would be do-able. I guess it would be ugly to fill the ports tree with hundreds(?) of directories each with an auto-generated package. I'm not sure if there's a way to smoosh it all into one big file. E.g. I see there's something called "multi-packages", but I'm guessing that's not designed to handle hundreds of subpackages. * Is it possible to cheat, by having all the Cabal-based ports share their Cabal directory? So if I build darcs then git-annex, Cabal has access to all the build products from da
Re: Question about lang/ghc module (trying to port git-annex)
James Cook writes:
> Thanks. I split up the long command and removed quotes:
>
> https://github.com/falsifian/ports/commit/e24278e3c1d7fca1b3c4961a0e81a5bb2ca5bcf0
I pushed my update:
https://github.com/blackgnezdo/ports/commit/83db945f59633c848a9ae28627cd8e378e316a09
> Two notes:
>
> * Currently "make post-build" fails if you run it twice. Adding -p in
> "mkdir ${MAN1_STAGING_DIR}" would fix that. I don't know if it's
> better to fail or not if "make post-build" gets run more than once.
It seems unusual for `make rebuild` to fail. Your newly added action is
idempotent.
> * I see you put "@" before the ln -s lines. I wasn't sure whether to
> add them to the other commands. The output is not that interesting.
I added them to your commands as I agree the output is not interesting.
> I confirmed the port works by installing on a different machine.
Thank you for testing this!
> One hitch: I needed to manually uninstall the current cabal-install
> port because it conflicts with the new cabal port. Is there a way to
> tell ports that cabal is a successor to cabal-install?
Yes, we'll need to update the quirks package which provides transitions
for this kind of cases.
> (I considered putting the man-building commands in post-install, but I
> wasn't sure where to put the staging directory if I did that. I wanted
> the staging directory because I assumed I should use the ${INSTALL_MAN}
> macro if possible to put the files in their final place.)
I think what we have now is fine.
>> I'll dig into this a bit. I simply didn't need to worry about this up
>> until now as all the other ports don't do much custom Setup.hs work.
>
> I don't know if there will be any cases other than man pages and
> git-annex's command aliases. Not sure how much effort it's worth.
The default action of doing nothing is valid :)
Thanks
Greg
Re: Question about lang/ghc module (trying to port git-annex)
On Fri, Feb 05, 2021 at 09:57:48PM -0800, Greg Steuck wrote:
> James Cook writes:
>
> >> Would you like to try your hand in extending post-install target with
> >> some man formatting magic like we have in other ports?
> >
> > Done (commit a1c5aec8) in my "git-annex" branch:
> >
> > https://github.com/falsifian/ports/commits/git-annex
>
> Very nice!
>
> > It's not pretty. I replicated some logic from Build/Man.hs, which is
> > called from Setup.hs.
>
> Some quick observations. Makefile allows line-continuation with \ and
> this is how people write long command lines. The for loop would be more
> readable if it were split like that. I also don't see a lot of places in
> ports make files with quotes around variables. I suspect we just expect
> people to not go nuts with spaces in directory names when building
> ports.
Thanks. I split up the long command and removed quotes:
https://github.com/falsifian/ports/commit/e24278e3c1d7fca1b3c4961a0e81a5bb2ca5bcf0
Two notes:
* Currently "make post-build" fails if you run it twice. Adding -p in
"mkdir ${MAN1_STAGING_DIR}" would fix that. I don't know if it's
better to fail or not if "make post-build" gets run more than once.
* I see you put "@" before the ln -s lines. I wasn't sure whether to
add them to the other commands. The output is not that interesting.
> > Normally cabal v2-install would do that work for
> > us (putting the result in .cabal/store/ghc-XXX/git-annex-XXX) but I
> > don't know if it's worth getting that working.
> >
> > Caveat: I took a shortcut when testing this: instead of re-running
> > "make build" I just made the post-build target after adding this in.
> > I'll try building from scratch later if you don't beat me to it.
>
> I believe `make rebuild` should reuse the cabal built pieces. post-build
> seems like a reasonable place for this kind of work. Alternatively
> sticking those man-formatting commands into post-install also seems
> acceptable to me.
I confirmed the port works by installing on a different machine.
One hitch: I needed to manually uninstall the current cabal-install
port because it conflicts with the new cabal port. Is there a way to
tell ports that cabal is a successor to cabal-install?
(I considered putting the man-building commands in post-install, but I
wasn't sure where to put the staging directory if I did that. I wanted
the staging directory because I assumed I should use the ${INSTALL_MAN}
macro if possible to put the files in their final place.)
> > On that branch I also removed the runtime dep on devel/git-lfs (it's
> > just one of a large number of optional backends).
>
> Good call.
>
> >> From what I gathered the v2-install target is largely unusable for
> >> installing packages outside of .cabal tree. At least neither I nor
> >> FreeBSD maintainer found a way to leverage that. Hence the manual
> >> install flow.
> >
> > I probably don't know all the subtleties, but when I run
> > "cabal v2-install", I get a nice set of files under
> > ~/.cabal/ghc-XXX/git-annex-XXX, including the man pages and all three
> > needed binaries. I would guess that just copying those files to the
> > destination would do the trick for most cabal executable packages, but
> > I haven't actually tried it. Maybe it's better to be cautious.
>
> I'll dig into this a bit. I simply didn't need to worry about this up
> until now as all the other ports don't do much custom Setup.hs work.
I don't know if there will be any cases other than man pages and
git-annex's command aliases. Not sure how much effort it's worth.
> Thanks
> Greg
--
James
Re: Question about lang/ghc module (trying to port git-annex)
James Cook writes: >> Would you like to try your hand in extending post-install target with >> some man formatting magic like we have in other ports? > > Done (commit a1c5aec8) in my "git-annex" branch: > > https://github.com/falsifian/ports/commits/git-annex Very nice! > It's not pretty. I replicated some logic from Build/Man.hs, which is > called from Setup.hs. Some quick observations. Makefile allows line-continuation with \ and this is how people write long command lines. The for loop would be more readable if it were split like that. I also don't see a lot of places in ports make files with quotes around variables. I suspect we just expect people to not go nuts with spaces in directory names when building ports. > Normally cabal v2-install would do that work for > us (putting the result in .cabal/store/ghc-XXX/git-annex-XXX) but I > don't know if it's worth getting that working. > > Caveat: I took a shortcut when testing this: instead of re-running > "make build" I just made the post-build target after adding this in. > I'll try building from scratch later if you don't beat me to it. I believe `make rebuild` should reuse the cabal built pieces. post-build seems like a reasonable place for this kind of work. Alternatively sticking those man-formatting commands into post-install also seems acceptable to me. > On that branch I also removed the runtime dep on devel/git-lfs (it's > just one of a large number of optional backends). Good call. >> From what I gathered the v2-install target is largely unusable for >> installing packages outside of .cabal tree. At least neither I nor >> FreeBSD maintainer found a way to leverage that. Hence the manual >> install flow. > > I probably don't know all the subtleties, but when I run > "cabal v2-install", I get a nice set of files under > ~/.cabal/ghc-XXX/git-annex-XXX, including the man pages and all three > needed binaries. I would guess that just copying those files to the > destination would do the trick for most cabal executable packages, but > I haven't actually tried it. Maybe it's better to be cautious. I'll dig into this a bit. I simply didn't need to worry about this up until now as all the other ports don't do much custom Setup.hs work. Thanks Greg
Re: Question about lang/ghc module (trying to port git-annex)
> > From what I gathered the v2-install target is largely unusable for > > installing packages outside of .cabal tree. At least neither I nor > > FreeBSD maintainer found a way to leverage that. Hence the manual > > install flow. > > I probably don't know all the subtleties, but when I run > "cabal v2-install", I get a nice set of files under > ~/.cabal/ghc-XXX/git-annex-XXX, including the man pages and all three > needed binaries. I would guess that just copying those files to the > destination would do the trick for most cabal executable packages, but > I haven't actually tried it. Maybe it's better to be cautious. I mean ~/.cabal/store/ghc-XXX/git-annex-XXX. Also forgot to mention the following git-annex patch (already reported upstream) is needed if anyone does want to experiment with using cabal v2-install to get git-annex man pages. diff --git a/Build/Mans.hs b/Build/Mans.hs index 9fb29d4a3..672dcd71c 100644 --- a/Build/Mans.hs +++ b/Build/Mans.hs @@ -38,7 +38,8 @@ buildMans = do if (Just srcm > destm) then do r <- system $ unwords - [ "./Build/mdwn2man" + [ "perl" + , "Build/mdwn2man" , progName src , "1" , src
Re: Question about lang/ghc module (trying to port git-annex)
On Thu, Feb 04, 2021 at 11:13:43AM -0800, Greg Steuck wrote: > James Cook writes: > > > I don't think that's an rsync problem. Unfortunately the > > git-annex-shell binary is missing, and that's pretty critical (needed > > to access the repository from other machines; sort of like git > > push/pull). > > This part is easy, patch attached. Thanks, I can now access my git-annex repositories remotely without an awful kludge. > > Incidentally, the man pages also seem to be missing. > > Would you like to try your hand in extending post-install target with > some man formatting magic like we have in other ports? Done (commit a1c5aec8) in my "git-annex" branch: https://github.com/falsifian/ports/commits/git-annex It's not pretty. I replicated some logic from Build/Man.hs, which is called from Setup.hs. Normally cabal v2-install would do that work for us (putting the result in .cabal/store/ghc-XXX/git-annex-XXX) but I don't know if it's worth getting that working. Caveat: I took a shortcut when testing this: instead of re-running "make build" I just made the post-build target after adding this in. I'll try building from scratch later if you don't beat me to it. On that branch I also removed the runtime dep on devel/git-lfs (it's just one of a large number of optional backends). > > Alternatively, I think the cabal (v2-)install commands installs what's > > needed (at least, it gets git-annex-shell, which is the really > > important thing). Is there some way to take advantage of that in your > > cabal infrastrsucture? > > From what I gathered the v2-install target is largely unusable for > installing packages outside of .cabal tree. At least neither I nor > FreeBSD maintainer found a way to leverage that. Hence the manual > install flow. I probably don't know all the subtleties, but when I run "cabal v2-install", I get a nice set of files under ~/.cabal/ghc-XXX/git-annex-XXX, including the man pages and all three needed binaries. I would guess that just copying those files to the destination would do the trick for most cabal executable packages, but I haven't actually tried it. Maybe it's better to be cautious. -- James
Re: Question about lang/ghc module (trying to port git-annex)
James Cook writes:
> I don't think that's an rsync problem. Unfortunately the
> git-annex-shell binary is missing, and that's pretty critical (needed
> to access the repository from other machines; sort of like git
> push/pull).
This part is easy, patch attached.
> Incidentally, the man pages also seem to be missing.
Would you like to try your hand in extending post-install target with
some man formatting magic like we have in other ports?
> Alternatively, I think the cabal (v2-)install commands installs what's
> needed (at least, it gets git-annex-shell, which is the really
> important thing). Is there some way to take advantage of that in your
> cabal infrastrsucture?
>From what I gathered the v2-install target is largely unusable for
installing packages outside of .cabal tree. At least neither I nor
FreeBSD maintainer found a way to leverage that. Hence the manual
install flow.
>From f190225986ed8e355ecd0622812539f82e35ac36 Mon Sep 17 00:00:00 2001
From: Greg Steuck
Subject: [PATCH] fixup: git-annex post-install
All 975 tests passed (977.23s)
---
devel/git-annex/Makefile | 5 +
devel/git-annex/pkg/PLIST | 2 ++
2 files changed, 7 insertions(+)
diff --git devel/git-annex/Makefile devel/git-annex/Makefile
index 63996c8f9c3..134fe5a030f 100644
--- devel/git-annex/Makefile
+++ devel/git-annex/Makefile
@@ -27,6 +27,11 @@ MAKE_ENV = LC_ALL=en_US.UTF-8
MODCABAL_STEM =git-annex
MODCABAL_VERSION = 8.20210127
+# Manual reimplementation of postCopy hook of Setup.hs
+post-install:
+ @ln -s git-annex ${PREFIX}/bin/git-annex-shell
+ @ln -s git-annex ${PREFIX}/bin/git-remote-tor-annex
+
# Produced by:
# $ cabal-bundler --openbsd git-annex-8.20210127
# Then manually remove:git-annex 8.20210127 0
diff --git devel/git-annex/pkg/PLIST devel/git-annex/pkg/PLIST
index 2a5b445b27d..d52ede016f8 100644
--- devel/git-annex/pkg/PLIST
+++ devel/git-annex/pkg/PLIST
@@ -1,2 +1,4 @@
@comment $OpenBSD: PLIST,v$
@bin bin/${MODCABAL_STEM}
+bin/${MODCABAL_STEM}-shell
+bin/git-remote-tor-annex
--
2.30.0
Re: Question about lang/ghc module (trying to port git-annex)
> > Assuming it works and that "git annex test" doesn't show any problems, > > is there anything else I can do to help with git-annex? > > The idea to link the test code into the main binary is quite > unorthodox. I can imagine some arguments for it, but I'll have to > overcome a lot of prior beliefs to accept this as a best practice. This is the only place I've seen it. Maybe the author did it this way so that users can reassure themselves that their storage backends are working (using the more specialized "git annex testremote" command). > $ touch /home/greg/.config/git-annex/program > $ git-annex test > ... > get (ssh remote): FAIL (1.20s) > ./Test/Framework.hs:57: > get of file failed (transcript follows) > get foo (from origin...) /bin/sh: git-annex-shell: not foundrsync: > connection unexpectedly closed (0 bytes received so far) [Receiver]rsync > error: error in rsync protocol data stream (code 12) at io.c(228) > [Receiver=3.2.3]rsync exited 12 rsync failed -- run git annex again to > resume file transfer transfer failed Unable to access these remotes: origin > Try making some of these remotes available: > a45e9dcd-c958-4026-96e4-a0d9a6fc322b -- test repo [origin]failedgit-annex: > get: 1 failed > ... > 6 out of 975 tests failed (1002.59s) > (Failures above could be due to a bug in git-annex, or an incompatibility >with utilities, such as git, installed on this system.) > > The failing tests are due to unconfigured rsync stuff. I don't think that's an rsync problem. Unfortunately the git-annex-shell binary is missing, and that's pretty critical (needed to access the repository from other machines; sort of like git push/pull). Incidentally, the man pages also seem to be missing. (I think git-annex tries to connect a local rsync process to a remote git-annex-shell process, which would explain why we also see an rsync error message.) I guess this could be solved with custom logic in devel/git-annex/Makefile. I can try to write some custom install commands, maybe tomorrow or Friday. (If you point me to the appropriate variable for overriding/augmenting the install script, that would help; I'm still pretty new to ports.) Alternatively, I think the cabal (v2-)install commands installs what's needed (at least, it gets git-annex-shell, which is the really important thing). Is there some way to take advantage of that in your cabal infrastrsucture? -- James
Re: Question about lang/ghc module (trying to port git-annex)
James Cook writes: > On Sun, Jan 31, 2021 at 11:58:40PM -0800, Greg Steuck wrote: > Thanks. I'm currently building it and will try it with my repos. > > In devel/cabal/Makefile would it make sense to fix the ghc version, > i.e: > > BUILD_DEPENDS +=lang/ghc=8.10.3 Good call. I was updating everything in lock-step so I never noticed. > I quickly discovered cabal won't build with 8.6.4, which is what I have > right now, because the port expects ghc-prim-0.6.1 which doesn't match. > With the above change, "make build" kicks off a build for ghc-8.10.3 > (which I'm now waiting for). It's not fast. I build with MAKE_JOBS=N to help a bit. >> I didn't bother to pare down the dependency list to exclude the test >> stuff, so MODCABAL_MANIFEST is twice the length it should be. > > Hm, I thought git-annex doesn't have test stuff. cabal v2-test doesn't > work; the way to test it is to build the binary and run git annex > test. Right, that was a mistake on my part. I later realized the list can't be shorter. I was just taken aback with the sheer mass of dependencies. > Assuming it works and that "git annex test" doesn't show any problems, > is there anything else I can do to help with git-annex? The idea to link the test code into the main binary is quite unorthodox. I can imagine some arguments for it, but I'll have to overcome a lot of prior beliefs to accept this as a best practice. $ touch /home/greg/.config/git-annex/program $ git-annex test ... get (ssh remote): FAIL (1.20s) ./Test/Framework.hs:57: get of file failed (transcript follows) get foo (from origin...) /bin/sh: git-annex-shell: not foundrsync: connection unexpectedly closed (0 bytes received so far) [Receiver]rsync error: error in rsync protocol data stream (code 12) at io.c(228) [Receiver=3.2.3]rsync exited 12 rsync failed -- run git annex again to resume file transfer transfer failed Unable to access these remotes: origin Try making some of these remotes available: a45e9dcd-c958-4026-96e4-a0d9a6fc322b -- test repo [origin]failedgit-annex: get: 1 failed ... 6 out of 975 tests failed (1002.59s) (Failures above could be due to a bug in git-annex, or an incompatibility with utilities, such as git, installed on this system.) The failing tests are due to unconfigured rsync stuff. > I use darcs a fair bit; would it help if I built it on your branch and > tested it out? Please do. You are the first person I know who still uses darcs on openbsd :) Thanks Greg
Re: Question about lang/ghc module (trying to port git-annex)
On Sun, Jan 31, 2021 at 11:58:40PM -0800, Greg Steuck wrote: > Greg Steuck writes: > > >> git-annex has at least 50 dependencies I couldn't find in ports, so I > >> want to make sure I understand this before I start porting them one at > >> a time (or just give up)... > > > > This is exactly the problem I want to solve with cabal.port.mk. You > > can try to look at > > https://marc.info/?l=openbsd-ports&m=160858285410366&w=2 > > A quick search in the archives will show the justification and the > > history of the effort. > > > > The current state of the work is I'm waiting on cabal 3.4 official > > release gated by ghc-9.0. I'm also looking for people needing this work > > and your attempt to add git-annex gives me more motivation to finish. > > FWIW, a version of git-annex port that can display its help message: > https://github.com/openbsd/ports/commit/0d970c085441b82f0080afee13e2034e47f475c0 Thanks. I'm currently building it and will try it with my repos. In devel/cabal/Makefile would it make sense to fix the ghc version, i.e: BUILD_DEPENDS += lang/ghc=8.10.3 I quickly discovered cabal won't build with 8.6.4, which is what I have right now, because the port expects ghc-prim-0.6.1 which doesn't match. With the above change, "make build" kicks off a build for ghc-8.10.3 (which I'm now waiting for). > I didn't bother to pare down the dependency list to exclude the test > stuff, so MODCABAL_MANIFEST is twice the length it should be. Hm, I thought git-annex doesn't have test stuff. cabal v2-test doesn't work; the way to test it is to build the binary and run git annex test. Assuming it works and that "git annex test" doesn't show any problems, is there anything else I can do to help with git-annex? I use darcs a fair bit; would it help if I built it on your branch and tested it out? > Thanks > Greg -- James
Re: Question about lang/ghc module (trying to port git-annex)
Greg Steuck writes: >> git-annex has at least 50 dependencies I couldn't find in ports, so I >> want to make sure I understand this before I start porting them one at >> a time (or just give up)... > > This is exactly the problem I want to solve with cabal.port.mk. You > can try to look at > https://marc.info/?l=openbsd-ports&m=160858285410366&w=2 > A quick search in the archives will show the justification and the > history of the effort. > > The current state of the work is I'm waiting on cabal 3.4 official > release gated by ghc-9.0. I'm also looking for people needing this work > and your attempt to add git-annex gives me more motivation to finish. FWIW, a version of git-annex port that can display its help message: https://github.com/openbsd/ports/commit/0d970c085441b82f0080afee13e2034e47f475c0 I didn't bother to pare down the dependency list to exclude the test stuff, so MODCABAL_MANIFEST is twice the length it should be. Thanks Greg
Re: Question about lang/ghc module (trying to port git-annex)
Hi James, James Cook writes: > I'm trying to write a port for git-annex, which is built using > Haskell's Cabal. I'm new to OpenBSD porting and don't completely > understand the lang/ghc module. > > Basic question: do all of the Haskell dependencies (from git-annex.cabal) need > to already have their own ports (like devel/hs-async, devel/hs-network, etc)? > (Or will the missing ones be magically built like cabal-install normally > does?) > If they do need to all be ported, how isn't there e.g. a devel/hs-containers > port --- doesn't basically every Haskell project depend on the containers > package? Yes, that's the way it currently is in the ports tree. > git-annex has at least 50 dependencies I couldn't find in ports, so I > want to make sure I understand this before I start porting them one at > a time (or just give up)... This is exactly the problem I want to solve with cabal.port.mk. You can try to look at https://marc.info/?l=openbsd-ports&m=160858285410366&w=2 A quick search in the archives will show the justification and the history of the effort. The current state of the work is I'm waiting on cabal 3.4 official release gated by ghc-9.0. I'm also looking for people needing this work and your attempt to add git-annex gives me more motivation to finish. BTW, trying to build with cabal also needs devel/libmagic port, or else: cabal: Missing dependency on a foreign library: * Missing (or bad) C library: magic Thanks Greg
Re: Question about lang/ghc module (trying to port git-annex)
> Here is what I have so far. P.S. to anyone trying to build git-annex, you'll need the following patch, which I didn't yet get around to adding to my WIP port: https://git-annex.branchable.com/bugs/__91__PATCH__93___OpenBSD__58___fix_Utility.DirWatcher.Kqueue/?updated#comment-0b3828545e6cf6ec417c0f82645888cb -- James
Question about lang/ghc module (trying to port git-annex)
Hi ports@, I'm trying to write a port for git-annex, which is built using Haskell's Cabal. I'm new to OpenBSD porting and don't completely understand the lang/ghc module. Basic question: do all of the Haskell dependencies (from git-annex.cabal) need to already have their own ports (like devel/hs-async, devel/hs-network, etc)? (Or will the missing ones be magically built like cabal-install normally does?) If they do need to all be ported, how isn't there e.g. a devel/hs-containers port --- doesn't basically every Haskell project depend on the containers package? git-annex has at least 50 dependencies I couldn't find in ports, so I want to make sure I understand this before I start porting them one at a time (or just give up)... Also, I'm using devel/darcs/Makefile as my model but don't understand the following comment. Should I also be listing git-annex's dependencies in BUILD_DEPENDS instead of LIB_DEPENDS? # Yes, build dependencies, because GHC libs are still static and we # don't want to pull in all of ghc. BUILD_DEPENDS = archivers/hs-zip-archive>=0.2.3,<0.4 \ archivers/hs-zlib>=0.6.1.2,<0.7.0.0 \ ... Another question: what is MODGHC_PACKAGE_KEY? ghc.port.mk describes it as the "package key" of the library, but I don't know what that means. Here is what I have so far. I only included the dependencies that are already in ports. Currently make build fails with a compiler error "Could not find module ‘System.FilePath.ByteString’", indicating that it is indeed a problem that there's no filepath-bytestring port. diff --git a/devel/git-annex/Makefile b/devel/git-annex/Makefile new file mode 100644 index 000..57ae7fe1079 --- /dev/null +++ b/devel/git-annex/Makefile @@ -0,0 +1,54 @@ +# $OpenBSD$ +# +# TODO: +# Use /usr/ports/infrastructure/bin/portcheck + +COMMENT = manage files with git without checking in file contents + +DISTNAME = git-annex-8.20210127 +CATEGORIES = devel +HOMEPAGE = https://git-annex.branchable.com/ + +MODULES = lang/ghc +MODGHC_BUILD = cabal hackage nort + +# Many licenses listed in COPYRIGHT. All permit redistribution. +PERMIT_PACKAGE = Yes + +# TODO +# "make port-lib-depends-check" can help +#WANTLIB = ??? + +# TODO +# Dependencies +#BUILD_DEPENDS = ??? +RUN_DEPENDS = net/rsync devel/git-lfs +LIB_DEPENDS = devel/hs-network-uri>=2.6 \ + devel/hs-exceptions>=0.6 \ + devel/hs-data-default \ + devel/hs-random \ + devel/hs-dlist \ + devel/hs-unix-compat \ + devel/hs-async \ + devel/hs-hslogger \ + devel/hs-utf8-string \ + devel/hs-sandi \ + devel/hs-monad-control \ + devel/hs-edit-distance \ + devel/hs-resourcet \ + devel/hs-conduit \ + devel/hs-old-locale \ + devel/hs-unliftio-core \ + devel/hs-vector \ + devel/hs-unordered-containers \ + devel/hs-regex-tdfa \ + devel/hs-byteable \ + security/hs-crypto-api \ + devel/hs-split \ + textproc/hs-attoparsec \ + devel/hs-QuickCheck +#version doesn't match +#devel/hs-network>=3.0.0.0 +TEST_DEPENDS = net/rsync + +.include diff --git a/devel/git-annex/distinfo b/devel/git-annex/distinfo new file mode 100644 index 000..9d9fa1c5788 --- /dev/null +++ b/devel/git-annex/distinfo @@ -0,0 +1,2 @@ +SHA256 (ghc/git-annex-8.20210127.tar.gz) = Y29DlCDyipKoJQufi0IlZ+Q5MV8/LSPLC+o7Cg5XVcM= +SIZE (ghc/git-annex-8.20210127.tar.gz) = 1361993
