Re: lang/ocaml and IBT WAS: amd64: IBT bulk build breakage
Hi Anil, > | NOTRACK -> i0 b "notrack" (* TODO does masm support this? *) Seems it doesn't :-) gmake[4]: Entering directory '/home/ports/ocaml/ocaml/stdlib' OCAMLOPT camlinternalFormatBasics.cmx /tmp/camlasm69ee74.s:83:2: error: invalid instruction mnemonic 'notrack' notrack ^~~ /tmp/camlasm69ee74.s:512:2: error: invalid instruction mnemonic 'notrack' notrack ^~~ /tmp/camlasm69ee74.s:942:2: error: invalid instruction mnemonic 'notrack' notrack ^~~ File "/home/ports/ocaml/ocaml/stdlib/camlinternalFormatBasics.ml", line 1: Error: Assembler error, input left in file /tmp/camlasm69ee74.s cheers, Volker On Thu Sep 7, 2023 at 9:37 AM CEST, Anil Madhavapeddy wrote: > Would anyone with an IBT-enabled x86_64 be able to run a test for me on an > OCaml tree to see if the patch works? (my hardware is still two weeks away > from delivery) > > $ git clone https://github.com/avsm/ocaml -b btcfi > $ cd ocaml && gmake -j world.opt && gmake tests > > It just adds notrace to the jump points. I think arm64 needs destination > labels instead; will do that later. > > Anil
Re: lang/ocaml and IBT WAS: amd64: IBT bulk build breakage
Hi, yes, I can do that in the evening today. cheers, Volker On 9/7/23 09:37, Anil Madhavapeddy wrote: Would anyone with an IBT-enabled x86_64 be able to run a test for me on an OCaml tree to see if the patch works? (my hardware is still two weeks away from delivery) $ git clone https://github.com/avsm/ocaml -b btcfi $ cd ocaml && gmake -j world.opt && gmake tests It just adds notrace to the jump points. I think arm64 needs destination labels instead; will do that later. Anil
Re: lang/ocaml and IBT WAS: amd64: IBT bulk build breakage
On Sun Sep 3, 2023 at 3:40 PM CEST, Christian Weisgerber wrote: > Volker Schlecht: > > > Is that an ok? :-) > > Yes, for your ocaml IBT fix and the opam IBT fix from chrisz@. Committed the ocaml fix and checked the build of sysutils/opam in tree. With the fixed ocaml, it seems to build and run without issues now, so I'd leave it to chrisz@ to commit the opam update.
Re: lang/ocaml and IBT WAS: amd64: IBT bulk build breakage
Volker Schlecht: > Is that an ok? :-) Yes, for your ocaml IBT fix and the opam IBT fix from chrisz@. -- Christian "naddy" Weisgerber na...@mips.inka.de
Re: lang/ocaml and IBT WAS: amd64: IBT bulk build breakage
Is that an ok? :-) On Sun Sep 3, 2023 at 1:47 PM CEST, Christian Weisgerber wrote: > Anil Madhavapeddy: > > > Thanks for all the pointers! Every machine for 100 miles around me seems > > to be an AMD these days, so I've ordered myself a Raptor Lake NUC > > to be my new OpenBSD desktop. As soon as that arrives I'll take a look > > at the native code compiler output (unless someone else beats me to it). > > > > In the meanwhile I think we should go for updating the OCaml ports > > to 5.1.0 so that a patch against trunk will be manageable... > > Well, it's been almost three weeks. There is only a month or so > left until the release. I suggest that Volker commits his fixes > now. Alternatively, if this stands in the way to the road to > perfection, I can mark OCaml as BROKEN, because that's what it is > now. > > None of this prevents an update to 5.1.0, but we can't just keep > waiting.
Re: lang/ocaml and IBT WAS: amd64: IBT bulk build breakage
Anil Madhavapeddy: > Thanks for all the pointers! Every machine for 100 miles around me seems > to be an AMD these days, so I've ordered myself a Raptor Lake NUC > to be my new OpenBSD desktop. As soon as that arrives I'll take a look > at the native code compiler output (unless someone else beats me to it). > > In the meanwhile I think we should go for updating the OCaml ports > to 5.1.0 so that a patch against trunk will be manageable... Well, it's been almost three weeks. There is only a month or so left until the release. I suggest that Volker commits his fixes now. Alternatively, if this stands in the way to the road to perfection, I can mark OCaml as BROKEN, because that's what it is now. None of this prevents an update to 5.1.0, but we can't just keep waiting. -- Christian "naddy" Weisgerber na...@mips.inka.de
Re: lang/ocaml and IBT WAS: amd64: IBT bulk build breakage
On 11 Aug 2023, at 12:51, Stuart Henderson wrote: > > yes, for amd64 it's intel Tiger Lake (11th gen) and newer, models of the > format iX-11XXX, iX-12XXX, iX-13XXX. no AMD CPUs. On ark.intel.com, look > for "Control-Flow Enforcement Technology" (the intel term which covers > both shadow stacks and indirect branch tracking). > > for arm64 currently only on Apple M2. Thanks for all the pointers! Every machine for 100 miles around me seems to be an AMD these days, so I've ordered myself a Raptor Lake NUC to be my new OpenBSD desktop. As soon as that arrives I'll take a look at the native code compiler output (unless someone else beats me to it). In the meanwhile I think we should go for updating the OCaml ports to 5.1.0 so that a patch against trunk will be manageable... The only roadbump is from Daniel Dickman, who wrote: > Fine with me so long as compcert continues to work on i386. I think the > only impact is potentially running slower? That’s fine with me. > > That being said I think there were some challenges with newer coq on > i386. So maybe the end is close for compcert on that platform? Indeed, I think it's time to move to a 64-bit license after so many years. I strongly suspect you're the only user of OpenBSD/i386 compcert these days, so it's worth just asking for an upgrade of the license to shift to 64-bit. OCaml 5 will definitely be bytecode-only for 32-bit architectures for the foreseeable future. This in turn pushes up memory requirements (due to the extra boxing of values), which makes it more likely that Compcert/Coq will hit the 4GB process limit. Anil
Re: lang/ocaml and IBT WAS: amd64: IBT bulk build breakage
On 8 Aug 2023, at 20:49, Volker Schlecht wrote: > > Cc: Maintainer > > On 7/31/23 14:28, Christian Weisgerber wrote: >> I managed to run a full bulk build on my T14 G3 with IBT. Kudos >> to Lenovo for engineering a laptop that can build for days on end >> without melting down. >> The following ports fail to build: >> devel/ocaml-menhir # OCaml >> sysutils/opam # OCaml >> x11/lablgtk3# OCaml >> I included an uncommitted USE_NOBTICF fix for sysutils/findlib, but >> there seems to remain another problem with some OCaml ports. > > With the attached patch, I get a lang/ocaml that builds ocaml-menhir and > lablgtk3 successfully on my IBT enabled amd64 machine. I’m just catching up; is a recent amd64 snapshot enough to get an IBT-enabled system? I’m testing out OCaml 5.1~rc1, and the testsuite fully passes. Might be easier to just upgrade to that instead. It would mean turning i386 and arm32 into bytecode-only architectures, but that’s got to happen at some stage. Anil
Re: lang/ocaml and IBT WAS: amd64: IBT bulk build breakage
Volker Schlecht: > With the attached patch, I get a lang/ocaml that builds ocaml-menhir and > lablgtk3 successfully on my IBT enabled amd64 machine. ... and all ports that depend on ocaml, in fact. -- Christian "naddy" Weisgerber na...@mips.inka.de
Re: lang/ocaml and IBT WAS: amd64: IBT bulk build breakage
On 2023/08/10 20:28, Volker Schlecht wrote: > Hi Anil, hey anil :) > On 8/10/23 19:39, Anil Madhavapeddy wrote: > > On 8 Aug 2023, at 20:49, Volker Schlecht wrote: > > > > > > Cc: Maintainer > > > > > > On 7/31/23 14:28, Christian Weisgerber wrote: > > > > I managed to run a full bulk build on my T14 G3 with IBT. Kudos > > > > to Lenovo for engineering a laptop that can build for days on end > > > > without melting down. > > > > The following ports fail to build: > > > > devel/ocaml-menhir # OCaml > > > > sysutils/opam # OCaml > > > > x11/lablgtk3# OCaml > > > > I included an uncommitted USE_NOBTICF fix for sysutils/findlib, but > > > > there seems to remain another problem with some OCaml ports. > > > > > > With the attached patch, I get a lang/ocaml that builds ocaml-menhir and > > > lablgtk3 successfully on my IBT enabled amd64 machine. > > > > > > I’m just catching up; is a recent amd64 snapshot enough to get an > > IBT-enabled system? > > yes, -current has it enabled now. But in order to catch those problems > you'll need a CPU that supports it, too ... on amd64 that would be Tiger > Lake(?) and later. I think. yes, for amd64 it's intel Tiger Lake (11th gen) and newer, models of the format iX-11XXX, iX-12XXX, iX-13XXX. no AMD CPUs. On ark.intel.com, look for "Control-Flow Enforcement Technology" (the intel term which covers both shadow stacks and indirect branch tracking). for arm64 currently only on Apple M2. FWIW the most common laptop that developers are using to test this on is probably the Intel version of Thinkpad T14 gen 3 (which works fairly well for most things, though I've been unable to get suspend to work, and sometimes it decides it won't power on without a hard reset)
Re: lang/ocaml and IBT WAS: amd64: IBT bulk build breakage
Anil Madhavapeddy: > I’m just catching up; is a recent amd64 snapshot enough to get an > IBT-enabled system? You also need a CPU that supports IBT, i.e., where dmesg shows the "IBT" flag in the cpuX line. As far as I know, that means an Intel processor of the Alder Lake generation (12th gen Core) or later. AMD does not have it. -- Christian "naddy" Weisgerber na...@mips.inka.de
Re: lang/ocaml and IBT WAS: amd64: IBT bulk build breakage
> yes, -current has it enabled now. But in order to catch those problems > you'll need a CPU that supports it, too ... on amd64 that would be > Tiger Lake(?) and later. I think. it is any "laptop / desktop" cpu from gen11 onwards, which is most things in the last 3 years
Re: lang/ocaml and IBT WAS: amd64: IBT bulk build breakage
Hi Anil, On 8/10/23 19:39, Anil Madhavapeddy wrote: On 8 Aug 2023, at 20:49, Volker Schlecht wrote: Cc: Maintainer On 7/31/23 14:28, Christian Weisgerber wrote: I managed to run a full bulk build on my T14 G3 with IBT. Kudos to Lenovo for engineering a laptop that can build for days on end without melting down. The following ports fail to build: devel/ocaml-menhir # OCaml sysutils/opam # OCaml x11/lablgtk3# OCaml I included an uncommitted USE_NOBTICF fix for sysutils/findlib, but there seems to remain another problem with some OCaml ports. With the attached patch, I get a lang/ocaml that builds ocaml-menhir and lablgtk3 successfully on my IBT enabled amd64 machine. I’m just catching up; is a recent amd64 snapshot enough to get an IBT-enabled system? yes, -current has it enabled now. But in order to catch those problems you'll need a CPU that supports it, too ... on amd64 that would be Tiger Lake(?) and later. I think. I’m testing out OCaml 5.1~rc1, and the testsuite fully passes. Might be easier to just upgrade to that instead. afaict, 4.14 should have the changes that I backported in my diff, too. Having said that, I've been wanting to play with OCaml 5.x for a while now, so if I can support with that effort, let me know. It would mean turning i386 and arm32 into bytecode-only architectures, but that’s got to happen at some stage. I remember the last time you mentioned that, there were implications for compcert on i386, so I'm copying daniel@ into the discussion. cheers, Volker
Re: lang/ocaml and IBT WAS: amd64: IBT bulk build breakage
For sysutils/opam it should already be sufficient to use MODULES+=lang/ocaml please then also do MODOCAML_RUNDEP=No otherwise I'm ok with your changes although I don't have any IBT-enabled machine to test on. Christopher
lang/ocaml and IBT WAS: amd64: IBT bulk build breakage
Cc: Maintainer
On 7/31/23 14:28, Christian Weisgerber wrote:
I managed to run a full bulk build on my T14 G3 with IBT. Kudos
to Lenovo for engineering a laptop that can build for days on end
without melting down.
The following ports fail to build:
devel/ocaml-menhir # OCaml
sysutils/opam # OCaml
x11/lablgtk3# OCaml
I included an uncommitted USE_NOBTICF fix for sysutils/findlib, but
there seems to remain another problem with some OCaml ports.
With the attached patch, I get a lang/ocaml that builds ocaml-menhir and
lablgtk3 successfully on my IBT enabled amd64 machine.
For sysutils/opam it should already be sufficient to use
MODULES+=lang/ocaml
for it to pick up NOBTCFI. However I tested with a slightly tweaked diff
for opam 2.1.5 by chrisz@, so I'm attaching that for reference, too.
NB: I also applied naddy@'s findlib patch in my tests.Index: Makefile
===
RCS file: /cvs/ports/sysutils/opam/Makefile,v
retrieving revision 1.25
diff -u -p -r1.25 Makefile
--- Makefile 16 Jan 2023 19:03:18 - 1.25
+++ Makefile 8 Aug 2023 19:43:09 -
@@ -2,7 +2,7 @@ COMMENT = OCaml source-based package ma
CATEGORIES = sysutils devel
-V = 2.0.10
+V = 2.1.5
PKGNAME = opam-${V}
DISTNAME = opam-full-${V}
@@ -15,14 +15,11 @@ MAINTAINER = Christopher Zimmermann
Index: distinfo
===
RCS file: /cvs/ports/sysutils/opam/distinfo,v
retrieving revision 1.9
diff -u -p -r1.9 distinfo
--- distinfo 16 Jan 2023 19:03:18 - 1.9
+++ distinfo 8 Aug 2023 19:43:09 -
@@ -1,2 +1,2 @@
-SHA256 (opam-full-2.0.10.tar.gz) = O1dAuOHBvGXc+KohxOjNgc1qv+G/UuosxDZ8P4nVvkA=
-SIZE (opam-full-2.0.10.tar.gz) = 8173617
+SHA256 (opam-full-2.1.5.tar.gz) = CfjZ5BCy9XI8K/7b95cOOzBfUBeJX82RdZ8F51PdzqU=
+SIZE (opam-full-2.1.5.tar.gz) = 10801367
Index: patches/patch-Makefile_config_in
===
RCS file: patches/patch-Makefile_config_in
diff -N patches/patch-Makefile_config_in
--- /dev/null 1 Jan 1970 00:00:00 -
+++ patches/patch-Makefile_config_in 8 Aug 2023 19:43:09 -
@@ -0,0 +1,13 @@
+don't use system wide installed ocaml packages
+
+Index: Makefile.config.in
+--- Makefile.config.in.orig
Makefile.config.in
+@@ -17,7 +17,6 @@ OCAMLFIND = @OCAMLFIND@
+ OCAML = @OCAML@
+ OCAMLC = @OCAMLC@
+ OCAMLOPT = @OCAMLOPT@
+-DUNE = @DUNE@
+ DUNE_SECONDARY = @DUNE_SECONDARY@
+ LN_S = @LN_S@
+
Index: patches/patch-configure_ac
===
RCS file: patches/patch-configure_ac
diff -N patches/patch-configure_ac
--- /dev/null 1 Jan 1970 00:00:00 -
+++ patches/patch-configure_ac 8 Aug 2023 19:43:09 -
@@ -0,0 +1,14 @@
+don't use system wide installed ocaml packages
+
+Index: configure.ac
+--- configure.ac.orig
configure.ac
+@@ -255,8 +255,6 @@ AS_IF([test "x${enable_certificate_check}" = "xno"], [
+
+ AC_CHECK_PROGS(FETCH,[curl wget],no)
+
+-AC_CHECK_TOOL(DUNE,dune)
+-AC_CHECK_TOOL(CPPO,cppo)
+ AC_CHECK_TOOL(PATCH,patch)
+ AC_CHECK_TOOL(BUNZIP2,bunzip2)
+
Index: pkg/PLIST
===
RCS file: /cvs/ports/sysutils/opam/pkg/PLIST,v
retrieving revision 1.3
diff -u -p -r1.3 PLIST
--- pkg/PLIST 11 Mar 2022 19:57:44 - 1.3
+++ pkg/PLIST 8 Aug 2023 19:43:09 -
@@ -24,6 +24,8 @@
@man man/man1/opam-installer.1
@man man/man1/opam-lint.1
@man man/man1/opam-list.1
+@man man/man1/opam-lock.1
+@man man/man1/opam-option.1
@man man/man1/opam-pin.1
@man man/man1/opam-reinstall.1
@man man/man1/opam-remote.1
@@ -46,6 +48,7 @@ share/doc/opam/depexts-plugins
share/doc/opam/depopts-and-features
share/doc/opam/pages/
share/doc/opam/pages/About.md
+share/doc/opam/pages/Distribution.md
share/doc/opam/pages/External_solvers.md
share/doc/opam/pages/FAQ.md
share/doc/opam/pages/Install.md
Index: Makefile
===
RCS file: /cvs/ports/lang/ocaml/Makefile,v
retrieving revision 1.97
diff -u -p -r1.97 Makefile
--- Makefile 24 Jul 2023 12:16:05 - 1.97
+++ Makefile 8 Aug 2023 19:47:59 -
@@ -3,7 +3,7 @@ COMMENT = ML language with complete c
# XXX Don't even think of updating ocaml alone.
# Do check that the ports that depend on it still work, or repair them.
VERSION= 4.12.1
-REVISION= 3
+REVISION= 4
# if the ocaml compiler gains support for BTI, as well as
# removing USE_NOBTCFI here (or changing to an arch-dependent
@@ -50,12 +50,15 @@ LDFLAGS += -L${LOCALBASE}/lib
LDFLAGS+= -Wl,-z,notext
.endif
-USE_GMAKE= Yes
+# We need to pass nobtcfi in LDFLAGS so that ocamlopt
+# will pick it up when building OCaml native binaries
+.if ${MACHINE_ARCH} == "amd64" || ${MACHINE_ARCH} == "arm64"
+LDFLAGS+= -Wl,-z,nobtcfi
+.endif
-WANTLIB = c iberty m pthread z
+USE_GMAKE= Yes
-# for libbfd (used by ocamlobjinfo on .cmxs
