Re: update lang/sbcl to 2.2.1
I'd really, really like to get an updated sbcl in 7.1, so I'll commit
the attached diff in a couple of days if I don't hear complaints.
I've updated to sbcl 2.2.3 (released some days ago) which includes the
diff for sb-capstone. Both flavors builds fine on amd64 and x11/stumpwm
builds too. I've tested with some stuff from quicklisp and works too.
I'm currently doing some builds on i386. I've got one failure but that
was because I had another (wip) version of sbcl installed during the
build; I'm now rebuilding both flavors there a couple of times to be
sure and try x11/stumpwm there too.
Thanks to everyone who tested the previous iterations of the diff! :)
Omar Polo wrote:
> friendly weekly ping
>
> (while here also dropped the last rcs id in patches)
>
> Omar Polo wrote:
> > Joshua Elsasser wrote:
> > > I will try to find some time to take a look at it this week. For what
> > > it's worth, 32-bit powerpc is usually a bit broken on non-openbsd
> > > platforms as well. I have a couple g4 mac minis that I can test on,
> > > assuming they haven't died.
> >
> > thanks for looking into this :)
> >
> > in the meantime sbcl 2.2.2 was released, please find attached an updated
> > diff. The capstone fix didn't make in time for 2.2.2 but should be
> > available in a next version.
> >
> > > Switching to ECL as the cross-build host is something what we probably
> > > want to do, as it would also allow sbcl to be build on arm and
> > > arm64. I had a version of the port which did this but as you saw, it
> > > was rather slower than with clisp.
> >
> > (I don't have any arm or arm64 hw to help with that unfortunately)
> >
> > I think we could keep clisp on powerpc and amd64 and use ecl for other
> > arches eventually, or do you prefer to always use the same CL
> > implementation on all architecture to build sbcl?
> >
> > (I've also removed the RCS id from the makefile and plist as per recent
> > developments since I was here)
> >
> > Cheers,
> >
> > Omar Polo
Index: Makefile
===
RCS file: /home/cvs/ports/lang/sbcl/Makefile,v
retrieving revision 1.48
diff -u -p -r1.48 Makefile
--- Makefile11 Mar 2022 19:29:14 - 1.48
+++ Makefile29 Mar 2022 06:52:54 -
@@ -1,33 +1,15 @@
-BROKEN-i386 = build fails in "Compiling file
[...]/src/compiler/generic/genesis.lisp"
-# ;; Compiling file
/pobj/sbcl-2.0.1/sbcl-2.0.1/src/compiler/generic/genesis.lisp ...
-# ;; Wrote file
/pobj/sbcl-2.0.1/sbcl-2.0.1/obj/from-host/src/compiler/generic/genesis.fas-tmp
-# 0 errors, 0 warnings
-# ;; Loading file obj/from-host/src/compiler/generic/genesis.fas ...
-# ;; Loaded file obj/from-host/src/compiler/generic/genesis.fas
-# *** - OPEN: File
-# #P"/pobj/sbcl-2.0.1/sbcl-2.0.1/obj/from-xc/tls-init.lisp-expr" does not
-# exist
-# The following restarts are available:
-# SKIP :R1 skip (GENESIS OBJECT-FILE-NAMES # ...)
-# RETRY :R2 retry (GENESIS OBJECT-FILE-NAMES # ...)
-# STOP :R3 stop loading file
/pobj/sbcl-2.0.1/sbcl-2.0.1/make-genesis-2.lisp
-# ABORT-BUILD:R4 Abort building SBCL.
-# ABORT :R5 Abort main loop
-# //testing for consistency of first and second GENESIS passes
-# diff: output/genesis-2: No such file or directory
-# error: header files do not match between first and second GENESIS
-
# not yet ported to other arches
ONLY_FOR_ARCHS = amd64 i386 powerpc
USE_WXNEEDED = Yes
COMMENT= compiler and runtime system for ANSI Common Lisp
-V =2.1.11
+V =2.2.3
DISTNAME= sbcl-${V}-source
PKGNAME= sbcl-${V}
WRKDIST= ${WRKDIR}/sbcl-${V}
EXTRACT_SUFX= .tar.bz2
+PATCHORIG =.orig.port
CATEGORIES=lang
HOMEPAGE= http://www.sbcl.org/
@@ -56,10 +38,17 @@ WANTLIB+= pthread
MAKE_PARAMS += --with-sb-core-compression \
--with-sb-xref-for-internals
+# contrib/sb-capstone/test.lisp uses it at build-time if present
+BUILD_DEPENDS =devel/capstone/main
+
.if ${FLAVOR:Mnative_bootstrap}
BUILD_DEPENDS+=lang/sbcl
BOOTSTRAP_CMD= ${LOCALBASE}/bin/sbcl \
--disable-debugger --no-sysinit --no-userinit
+.elif ${MACHINE_ARCH:Mi386}
+# ecl is slower but lang/clisp fails to build sbcl on i386
+BUILD_DEPENDS += lang/ecl
+BOOTSTRAP_CMD =${LOCALBASE}/bin/ecl -q --norc
.else
BUILD_DEPENDS += lang/clisp
BOOTSTRAP_CMD =${LOCALBASE}/bin/clisp -q -norc
Index: distinfo
===
RCS file: /home/cvs/ports/lang/sbcl/distinfo,v
retrieving revision 1.21
diff -u -p -r1.21 distinfo
--- distinfo31 Dec 2021 09:53:11 - 1.21
+++ distinfo29 Mar 2022 06:54:22 -
@@ -1,2 +1,2 @@
-SHA256
Re: update lang/sbcl to 2.2.1
friendly weekly ping
(while here also dropped the last rcs id in patches)
Omar Polo wrote:
> Joshua Elsasser wrote:
> > I will try to find some time to take a look at it this week. For what
> > it's worth, 32-bit powerpc is usually a bit broken on non-openbsd
> > platforms as well. I have a couple g4 mac minis that I can test on,
> > assuming they haven't died.
>
> thanks for looking into this :)
>
> in the meantime sbcl 2.2.2 was released, please find attached an updated
> diff. The capstone fix didn't make in time for 2.2.2 but should be
> available in a next version.
>
> > Switching to ECL as the cross-build host is something what we probably
> > want to do, as it would also allow sbcl to be build on arm and
> > arm64. I had a version of the port which did this but as you saw, it
> > was rather slower than with clisp.
>
> (I don't have any arm or arm64 hw to help with that unfortunately)
>
> I think we could keep clisp on powerpc and amd64 and use ecl for other
> arches eventually, or do you prefer to always use the same CL
> implementation on all architecture to build sbcl?
>
> (I've also removed the RCS id from the makefile and plist as per recent
> developments since I was here)
>
> Cheers,
>
> Omar Polo
Index: Makefile
===
RCS file: /home/cvs/ports/lang/sbcl/Makefile,v
retrieving revision 1.47
diff -u -p -r1.47 Makefile
--- Makefile31 Dec 2021 09:53:11 - 1.47
+++ Makefile3 Mar 2022 14:09:51 -
@@ -1,35 +1,15 @@
-# $OpenBSD: Makefile,v 1.47 2021/12/31 09:53:11 solene Exp $
-
-BROKEN-i386 = build fails in "Compiling file
[...]/src/compiler/generic/genesis.lisp"
-# ;; Compiling file
/pobj/sbcl-2.0.1/sbcl-2.0.1/src/compiler/generic/genesis.lisp ...
-# ;; Wrote file
/pobj/sbcl-2.0.1/sbcl-2.0.1/obj/from-host/src/compiler/generic/genesis.fas-tmp
-# 0 errors, 0 warnings
-# ;; Loading file obj/from-host/src/compiler/generic/genesis.fas ...
-# ;; Loaded file obj/from-host/src/compiler/generic/genesis.fas
-# *** - OPEN: File
-# #P"/pobj/sbcl-2.0.1/sbcl-2.0.1/obj/from-xc/tls-init.lisp-expr" does not
-# exist
-# The following restarts are available:
-# SKIP :R1 skip (GENESIS OBJECT-FILE-NAMES # ...)
-# RETRY :R2 retry (GENESIS OBJECT-FILE-NAMES # ...)
-# STOP :R3 stop loading file
/pobj/sbcl-2.0.1/sbcl-2.0.1/make-genesis-2.lisp
-# ABORT-BUILD:R4 Abort building SBCL.
-# ABORT :R5 Abort main loop
-# //testing for consistency of first and second GENESIS passes
-# diff: output/genesis-2: No such file or directory
-# error: header files do not match between first and second GENESIS
-
# not yet ported to other arches
ONLY_FOR_ARCHS = amd64 i386 powerpc
USE_WXNEEDED = Yes
COMMENT= compiler and runtime system for ANSI Common Lisp
-V =2.1.11
+V =2.2.2
DISTNAME= sbcl-${V}-source
PKGNAME= sbcl-${V}
WRKDIST= ${WRKDIR}/sbcl-${V}
EXTRACT_SUFX= .tar.bz2
+PATCHORIG =.orig.port
CATEGORIES=lang
HOMEPAGE= http://www.sbcl.org/
@@ -58,10 +38,17 @@ WANTLIB+= pthread
MAKE_PARAMS += --with-sb-core-compression \
--with-sb-xref-for-internals
+# contrib/sb-capstone/test.lisp uses it at build-time if present
+BUILD_DEPENDS =devel/capstone/main
+
.if ${FLAVOR:Mnative_bootstrap}
BUILD_DEPENDS+=lang/sbcl
BOOTSTRAP_CMD= ${LOCALBASE}/bin/sbcl \
--disable-debugger --no-sysinit --no-userinit
+.elif ${MACHINE_ARCH:Mi386}
+# ecl is slower but lang/clisp fails to build sbcl on i386
+BUILD_DEPENDS += lang/ecl
+BOOTSTRAP_CMD =${LOCALBASE}/bin/ecl -q --norc
.else
BUILD_DEPENDS += lang/clisp
BOOTSTRAP_CMD =${LOCALBASE}/bin/clisp -q -norc
Index: distinfo
===
RCS file: /home/cvs/ports/lang/sbcl/distinfo,v
retrieving revision 1.21
diff -u -p -r1.21 distinfo
--- distinfo31 Dec 2021 09:53:11 - 1.21
+++ distinfo3 Mar 2022 11:32:07 -
@@ -1,2 +1,2 @@
-SHA256 (sbcl-2.1.11-source.tar.bz2) =
v8FIHef9vfru8qsPDo6E79NDQz3qjSHPvqiwFGy9/v0=
-SIZE (sbcl-2.1.11-source.tar.bz2) = 6687529
+SHA256 (sbcl-2.2.2-source.tar.bz2) =
h5DbvpdxHc4Uu4IxJc5bGFsAc88vPL83vdGtOA55UPY=
+SIZE (sbcl-2.2.2-source.tar.bz2) = 6716272
Index: patches/patch-contrib_sb-capstone_capstone_lisp
===
RCS file: patches/patch-contrib_sb-capstone_capstone_lisp
diff -N patches/patch-contrib_sb-capstone_capstone_lisp
--- /dev/null 1 Jan 1970 00:00:00 -
+++ patches/patch-contrib_sb-capstone_capstone_lisp 10 Mar 2022 16:50:39
-
@@ -0,0 +1,27 @@
+sync a couple of size mismatches with capstone.h; fixes the build on
+32bit arches
Re: update lang/sbcl to 2.2.1
Joshua Elsasser wrote:
> I will try to find some time to take a look at it this week. For what
> it's worth, 32-bit powerpc is usually a bit broken on non-openbsd
> platforms as well. I have a couple g4 mac minis that I can test on,
> assuming they haven't died.
thanks for looking into this :)
in the meantime sbcl 2.2.2 was released, please find attached an updated
diff. The capstone fix didn't make in time for 2.2.2 but should be
available in a next version.
> Switching to ECL as the cross-build host is something what we probably
> want to do, as it would also allow sbcl to be build on arm and
> arm64. I had a version of the port which did this but as you saw, it
> was rather slower than with clisp.
(I don't have any arm or arm64 hw to help with that unfortunately)
I think we could keep clisp on powerpc and amd64 and use ecl for other
arches eventually, or do you prefer to always use the same CL
implementation on all architecture to build sbcl?
(I've also removed the RCS id from the makefile and plist as per recent
developments since I was here)
Cheers,
Omar Polo
Index: Makefile
===
RCS file: /home/cvs/ports/lang/sbcl/Makefile,v
retrieving revision 1.47
diff -u -p -r1.47 Makefile
--- Makefile31 Dec 2021 09:53:11 - 1.47
+++ Makefile3 Mar 2022 14:09:51 -
@@ -1,35 +1,15 @@
-# $OpenBSD: Makefile,v 1.47 2021/12/31 09:53:11 solene Exp $
-
-BROKEN-i386 = build fails in "Compiling file
[...]/src/compiler/generic/genesis.lisp"
-# ;; Compiling file
/pobj/sbcl-2.0.1/sbcl-2.0.1/src/compiler/generic/genesis.lisp ...
-# ;; Wrote file
/pobj/sbcl-2.0.1/sbcl-2.0.1/obj/from-host/src/compiler/generic/genesis.fas-tmp
-# 0 errors, 0 warnings
-# ;; Loading file obj/from-host/src/compiler/generic/genesis.fas ...
-# ;; Loaded file obj/from-host/src/compiler/generic/genesis.fas
-# *** - OPEN: File
-# #P"/pobj/sbcl-2.0.1/sbcl-2.0.1/obj/from-xc/tls-init.lisp-expr" does not
-# exist
-# The following restarts are available:
-# SKIP :R1 skip (GENESIS OBJECT-FILE-NAMES # ...)
-# RETRY :R2 retry (GENESIS OBJECT-FILE-NAMES # ...)
-# STOP :R3 stop loading file
/pobj/sbcl-2.0.1/sbcl-2.0.1/make-genesis-2.lisp
-# ABORT-BUILD:R4 Abort building SBCL.
-# ABORT :R5 Abort main loop
-# //testing for consistency of first and second GENESIS passes
-# diff: output/genesis-2: No such file or directory
-# error: header files do not match between first and second GENESIS
-
# not yet ported to other arches
ONLY_FOR_ARCHS = amd64 i386 powerpc
USE_WXNEEDED = Yes
COMMENT= compiler and runtime system for ANSI Common Lisp
-V =2.1.11
+V =2.2.2
DISTNAME= sbcl-${V}-source
PKGNAME= sbcl-${V}
WRKDIST= ${WRKDIR}/sbcl-${V}
EXTRACT_SUFX= .tar.bz2
+PATCHORIG =.orig.port
CATEGORIES=lang
HOMEPAGE= http://www.sbcl.org/
@@ -58,10 +38,17 @@ WANTLIB+= pthread
MAKE_PARAMS += --with-sb-core-compression \
--with-sb-xref-for-internals
+# contrib/sb-capstone/test.lisp uses it at build-time if present
+BUILD_DEPENDS =devel/capstone/main
+
.if ${FLAVOR:Mnative_bootstrap}
BUILD_DEPENDS+=lang/sbcl
BOOTSTRAP_CMD= ${LOCALBASE}/bin/sbcl \
--disable-debugger --no-sysinit --no-userinit
+.elif ${MACHINE_ARCH:Mi386}
+# ecl is slower but lang/clisp fails to build sbcl on i386
+BUILD_DEPENDS += lang/ecl
+BOOTSTRAP_CMD =${LOCALBASE}/bin/ecl -q --norc
.else
BUILD_DEPENDS += lang/clisp
BOOTSTRAP_CMD =${LOCALBASE}/bin/clisp -q -norc
Index: distinfo
===
RCS file: /home/cvs/ports/lang/sbcl/distinfo,v
retrieving revision 1.21
diff -u -p -r1.21 distinfo
--- distinfo31 Dec 2021 09:53:11 - 1.21
+++ distinfo3 Mar 2022 11:32:07 -
@@ -1,2 +1,2 @@
-SHA256 (sbcl-2.1.11-source.tar.bz2) =
v8FIHef9vfru8qsPDo6E79NDQz3qjSHPvqiwFGy9/v0=
-SIZE (sbcl-2.1.11-source.tar.bz2) = 6687529
+SHA256 (sbcl-2.2.2-source.tar.bz2) =
h5DbvpdxHc4Uu4IxJc5bGFsAc88vPL83vdGtOA55UPY=
+SIZE (sbcl-2.2.2-source.tar.bz2) = 6716272
Index: patches/patch-contrib_sb-capstone_capstone_lisp
===
RCS file: patches/patch-contrib_sb-capstone_capstone_lisp
diff -N patches/patch-contrib_sb-capstone_capstone_lisp
--- /dev/null 1 Jan 1970 00:00:00 -
+++ patches/patch-contrib_sb-capstone_capstone_lisp 24 Feb 2022 16:05:26
-
@@ -0,0 +1,29 @@
+$OpenBSD$
+
+sync a couple of size mismatches with capstone.h; fixes the build on
+32bit arches (i386, powerpc.)
+
+Index: contrib/sb-capstone/capstone.lisp
+--- contrib/sb-capstone/capstone.lisp.orig
Re: update lang/sbcl to 2.2.1
George Koehler wrote: > On Thu, 24 Feb 2022 11:12:50 +0100 > Omar Polo wrote: > > > Attaching an updated diff with the correct types (for cs-version too as > > you spotted in a previous mail.) > > ok gkoehler@ just a quick follow up, the capstone.lisp patch was successfully upstreamed, thanks for your the help reviewing and testing the diff!
Re: update lang/sbcl to 2.2.1
On Thu, 24 Feb 2022 11:12:50 +0100
Omar Polo wrote:
> Attaching an updated diff with the correct types (for cs-version too as
> you spotted in a previous mail.)
ok gkoehler@
"make test" on powerpc with sbcl-2.2.1 is much like 2.1.11.
2.1.11 -> https://marc.info/?l=openbsd-ports=163954004504504=2
2.2.1 "FLAVOR=threads make test" got stuck in "Running :C-CALL-SAVE-P"
so I didn't finish it. (2.1.11 also got stuck somewhere.)
2.2.1 without threads got these results,
Status:
Expected failure: array.pure.lisp / CHECK-BOUND-SIGNED-BOUND-NOTES
Expected failure: compiler-2.pure.lisp / (MAP-ALLOCATED-OBJECTS NO-CONSING)
Expected failure: compiler-2.pure.lisp / DEDUPLICATED-FDEFNS
Expected failure: compiler.pure.lisp / (ODDP BIGNUM NO-CONSING)
Expected failure: compiler.pure.lisp / (LOGTEST BIGNUM NO-CONSING)
Failure:compiler.pure.lisp / REDUCING-CONSTANTS
Failure:compiler.pure.lisp / REDUCING-CONSTANTS.2
Expected failure: dynamic-extent.pure.lisp / DX-COMPILER-NOTES
Expected failure: float.pure.lisp / (ADDITION-OVERFLOW BUG-372)
Expected failure: float.pure.lisp / (ADDITION-OVERFLOW BUG-372 TAKE-2)
Expected failure: hash.pure.lisp / SXHASH-ON-DISPLACED-STRING
Skipped (broken): interface.pure.lisp /
RESTART-BOGUS-ARG-TO-VALUES-LIST-ERROR
Expected failure: random.pure.lisp / FLOAT-NO-CONSING
Expected failure: selfbuild-output.pure.lisp /
MAKE-LIST-%MAKE-LIST-NOT-CALLED
Failure:unicode-misc.pure.lisp / (CL-CASE-INVERTIBILITY)
Expected failure: block-compile.impure.lisp /
BLOCK-DEFCONSTANT-HAIRY-EQNESS-TEST
Expected failure: compiler.impure.lisp / BUG-308921
Expected failure: compiler.impure.lisp / FTYPE-RETURN-TYPE-CONFLICT
Unexpected success: debug.impure.lisp / (TRACE ENCAPSULATE NIL)
Unexpected success: debug.impure.lisp / (TRACE ENCAPSULATE NIL RECURSIVE)
Expected failure: fopcompiler.impure.lisp /
FOPCOMPILER-DEPRECATED-VAR-WARNING
Expected failure: full-eval.impure.lisp / INLINE-FUN-CAPTURES-DECL
Failure:hash-table.impure.lisp / HASH-TABLE-ITERATOR-NO-NOTES
Expected failure: packages.impure.lisp / USE-PACKAGE-CONFLICT-SET
Expected failure: packages.impure.lisp / IMPORT-SINGLE-CONFLICT
Skipped (broken): timer.impure.lisp / (TIMER PARALLEL-UNSCHEDULE)
Expected failure: walk.impure.lisp / (WALK MULTIPLE-VALUE-BIND)
Expected failure: walk.impure.lisp / (WALK MULTIPLE-VALUE-BIND SPECIAL)
(121 tests skipped for this combination of platform and features)
test failed, expected 104 return code, got 1
--George
> Index: Makefile
> ===
> RCS file: /cvs/ports/lang/sbcl/Makefile,v
> retrieving revision 1.47
> diff -u -p -r1.47 Makefile
> --- Makefile 31 Dec 2021 09:53:11 - 1.47
> +++ Makefile 24 Feb 2022 10:22:51 -
> @@ -1,35 +1,17 @@
> # $OpenBSD: Makefile,v 1.47 2021/12/31 09:53:11 solene Exp $
>
> -BROKEN-i386 =build fails in "Compiling file
> [...]/src/compiler/generic/genesis.lisp"
> -# ;; Compiling file
> /pobj/sbcl-2.0.1/sbcl-2.0.1/src/compiler/generic/genesis.lisp ...
> -# ;; Wrote file
> /pobj/sbcl-2.0.1/sbcl-2.0.1/obj/from-host/src/compiler/generic/genesis.fas-tmp
> -# 0 errors, 0 warnings
> -# ;; Loading file obj/from-host/src/compiler/generic/genesis.fas ...
> -# ;; Loaded file obj/from-host/src/compiler/generic/genesis.fas
> -# *** - OPEN: File
> -# #P"/pobj/sbcl-2.0.1/sbcl-2.0.1/obj/from-xc/tls-init.lisp-expr" does
> not
> -# exist
> -# The following restarts are available:
> -# SKIP :R1 skip (GENESIS OBJECT-FILE-NAMES # ...)
> -# RETRY :R2 retry (GENESIS OBJECT-FILE-NAMES # ...)
> -# STOP :R3 stop loading file
> /pobj/sbcl-2.0.1/sbcl-2.0.1/make-genesis-2.lisp
> -# ABORT-BUILD:R4 Abort building SBCL.
> -# ABORT :R5 Abort main loop
> -# //testing for consistency of first and second GENESIS passes
> -# diff: output/genesis-2: No such file or directory
> -# error: header files do not match between first and second GENESIS
> -
> # not yet ported to other arches
> ONLY_FOR_ARCHS = amd64 i386 powerpc
> USE_WXNEEDED = Yes
>
> COMMENT= compiler and runtime system for ANSI Common Lisp
>
> -V = 2.1.11
> +V = 2.2.1
> DISTNAME=sbcl-${V}-source
> PKGNAME= sbcl-${V}
> WRKDIST= ${WRKDIR}/sbcl-${V}
> EXTRACT_SUFX=.tar.bz2
> +PATCHORIG = .orig.port
>
> CATEGORIES= lang
> HOMEPAGE=http://www.sbcl.org/
> @@ -58,10 +40,17 @@ WANTLIB+= pthread
> MAKE_PARAMS += --with-sb-core-compression \
> --with-sb-xref-for-internals
>
> +# contrib/sb-capstone/test.lisp uses it at build-time if present
> +BUILD_DEPENDS = devel/capstone/main
> +
> .if ${FLAVOR:Mnative_bootstrap}
> BUILD_DEPENDS+=
Re: update lang/sbcl to 2.2.1
George Koehler writes:
> On Wed, 23 Feb 2022 09:12:34 +0100
> Omar Polo wrote:
>
>> I've yet to find a list of types and their counterpart in C in the sbcl
>> documentation, I'm only finding stuff for sb-groveler which uses
>> sb-alien. Anyway, I *think* that `integer' and `unsigned' should mirror
>> their C counterpart (int and unsigned int), so 32 bit on amd64, i386 and
>> powerpc.
>
> src/compiler/x86-64/parms.lisp:
> (in-package "SB-VM") ...
> (defconstant n-word-bits 64)
>
> src/code/alieneval.lisp:
> (define-alien-type-translator unsigned ( (bits sb-vm:n-word-bits))
> (make-alien-integer-type :bits bits :signed nil))
>
> src/code/target-c-call.lisp
> (define-alien-type unsigned-int (unsigned 32))
>
> Looks like unsigned defaults to (unsigned 64) on amd64. Capstone
> handles are size_t in C and unsigned in Lisp; size_t has 64 bits on
> amd64, so handles are correct only if unsigned is (unsigned 64):
right, I should have looked the sources, it wasn't hidden after all,
apologies.
> [...]
>
>> That was my first thought as well, but the third test fails if `mode' is
>> `int':
>>
>> Test SB-CAPSTONE-TESTS::PPC-BIG-ENDIAN failed
>> Form: (SB-CAPSTONE-TESTS::CAPSTONE-CHECK '(124 8 2 166) '(:PPC64 :BIG-ENDIAN)
>> '("MFLR R0"))
>> Expected value: T
>> Actual value: #> 2147483656>.
>> The value
>> 2147483656
>> is not of type
>> (SIGNED-BYTE 32)
>>
>> my reading is that capstone.lisp:get-cs-mode tries to set the most
>> significant bit which is fine in C but sbcl begs to differ. (2147483656
>> is actually -2147483640 in base 2 complement if I'm doing the math
>> right, or 0x8008)
>
> Found it: has CS_MODE_BIG_ENDIAN = 1 << 31 which
> overflows to negative; clang says CS_MODE_BIG_ENDIAN < 0.
> capstone.lisp has (defconstant cs-mode-big-endian (ash 1 31)) but Lisp
> integers don't overflow. So you are correct: mode can't be signed.
>
> I believe that it should be (mode unsigned-int), not (mode unsigned).
right
Attaching an updated diff with the correct types (for cs-version too as
you spotted in a previous mail.)
With it I can successfully build sbcl and x11/stumpwm on i386. Since
lang/ecl is significantly slower than lang/clisp I'm using ecl only on
i386.
I forgot to mention that running sbcl as normal user yields an error:
chiaki$ uname -m
i386
chiaki$ man sbcl
chiaki$ /usr/local/bin/sbcl --noinform
RUNTIME WARNING: data size resource limit may be too low,
try decreasing the dynamic space size with --dynamic-space-size
or raising the datasize or datasize-max limits in /etc/login.conf
mmap: Cannot allocate memory
fatal error encountered in SBCL pid 45001:
Can't allocate 0x2000 bytes for space 1
chiaki$ ulimit -m
2029792
but works with --dynamic-space-size 400 and when building stumpwm using
the ports infrastructure and PORTS_PRIVSEP.
Index: Makefile
===
RCS file: /cvs/ports/lang/sbcl/Makefile,v
retrieving revision 1.47
diff -u -p -r1.47 Makefile
--- Makefile31 Dec 2021 09:53:11 - 1.47
+++ Makefile24 Feb 2022 10:22:51 -
@@ -1,35 +1,17 @@
# $OpenBSD: Makefile,v 1.47 2021/12/31 09:53:11 solene Exp $
-BROKEN-i386 = build fails in "Compiling file
[...]/src/compiler/generic/genesis.lisp"
-# ;; Compiling file
/pobj/sbcl-2.0.1/sbcl-2.0.1/src/compiler/generic/genesis.lisp ...
-# ;; Wrote file
/pobj/sbcl-2.0.1/sbcl-2.0.1/obj/from-host/src/compiler/generic/genesis.fas-tmp
-# 0 errors, 0 warnings
-# ;; Loading file obj/from-host/src/compiler/generic/genesis.fas ...
-# ;; Loaded file obj/from-host/src/compiler/generic/genesis.fas
-# *** - OPEN: File
-# #P"/pobj/sbcl-2.0.1/sbcl-2.0.1/obj/from-xc/tls-init.lisp-expr" does not
-# exist
-# The following restarts are available:
-# SKIP :R1 skip (GENESIS OBJECT-FILE-NAMES # ...)
-# RETRY :R2 retry (GENESIS OBJECT-FILE-NAMES # ...)
-# STOP :R3 stop loading file
/pobj/sbcl-2.0.1/sbcl-2.0.1/make-genesis-2.lisp
-# ABORT-BUILD:R4 Abort building SBCL.
-# ABORT :R5 Abort main loop
-# //testing for consistency of first and second GENESIS passes
-# diff: output/genesis-2: No such file or directory
-# error: header files do not match between first and second GENESIS
-
# not yet ported to other arches
ONLY_FOR_ARCHS = amd64 i386 powerpc
USE_WXNEEDED = Yes
COMMENT= compiler and runtime system for ANSI Common Lisp
-V =2.1.11
+V =2.2.1
DISTNAME= sbcl-${V}-source
PKGNAME= sbcl-${V}
WRKDIST= ${WRKDIR}/sbcl-${V}
EXTRACT_SUFX= .tar.bz2
+PATCHORIG =.orig.port
CATEGORIES=lang
HOMEPAGE= http://www.sbcl.org/
@@ -58,10 +40,17 @@ WANTLIB+= pthread
MAKE_PARAMS += --with-sb-core-compression \
--with-sb-xref-for-internals
+#
Re: update lang/sbcl to 2.2.1
On Wed, 23 Feb 2022 09:12:34 +0100
Omar Polo wrote:
> I've yet to find a list of types and their counterpart in C in the sbcl
> documentation, I'm only finding stuff for sb-groveler which uses
> sb-alien. Anyway, I *think* that `integer' and `unsigned' should mirror
> their C counterpart (int and unsigned int), so 32 bit on amd64, i386 and
> powerpc.
src/compiler/x86-64/parms.lisp:
(in-package "SB-VM") ...
(defconstant n-word-bits 64)
src/code/alieneval.lisp:
(define-alien-type-translator unsigned ( (bits sb-vm:n-word-bits))
(make-alien-integer-type :bits bits :signed nil))
src/code/target-c-call.lisp
(define-alien-type unsigned-int (unsigned 32))
Looks like unsigned defaults to (unsigned 64) on amd64. Capstone
handles are size_t in C and unsigned in Lisp; size_t has 64 bits on
amd64, so handles are correct only if unsigned is (unsigned 64):
capstone.h:
typedef size_t csh;
...
CAPSTONE_EXPORT
cs_err CAPSTONE_API cs_open(cs_arch arch, cs_mode mode, csh *handle);
capstone.lisp:
(define-alien-routine cs-open int (arch int) (mode unsigned) (handle unsigned
:out))
> That was my first thought as well, but the third test fails if `mode' is
> `int':
>
> Test SB-CAPSTONE-TESTS::PPC-BIG-ENDIAN failed
> Form: (SB-CAPSTONE-TESTS::CAPSTONE-CHECK '(124 8 2 166) '(:PPC64 :BIG-ENDIAN)
> '("MFLR R0"))
> Expected value: T
> Actual value: #.
> The value
> 2147483656
> is not of type
> (SIGNED-BYTE 32)
>
> my reading is that capstone.lisp:get-cs-mode tries to set the most
> significant bit which is fine in C but sbcl begs to differ. (2147483656
> is actually -2147483640 in base 2 complement if I'm doing the math
> right, or 0x8008)
Found it: has CS_MODE_BIG_ENDIAN = 1 << 31 which
overflows to negative; clang says CS_MODE_BIG_ENDIAN < 0.
capstone.lisp has (defconstant cs-mode-big-endian (ash 1 31)) but Lisp
integers don't overflow. So you are correct: mode can't be signed.
I believe that it should be (mode unsigned-int), not (mode unsigned).
--George
> Index: Makefile
> ===
> RCS file: /home/cvs/ports/lang/sbcl/Makefile,v
> retrieving revision 1.47
> diff -u -p -r1.47 Makefile
> --- Makefile 31 Dec 2021 09:53:11 - 1.47
> +++ Makefile 23 Feb 2022 09:03:58 -
> @@ -1,35 +1,17 @@
> # $OpenBSD: Makefile,v 1.47 2021/12/31 09:53:11 solene Exp $
>
> -BROKEN-i386 =build fails in "Compiling file
> [...]/src/compiler/generic/genesis.lisp"
> -# ;; Compiling file
> /pobj/sbcl-2.0.1/sbcl-2.0.1/src/compiler/generic/genesis.lisp ...
> -# ;; Wrote file
> /pobj/sbcl-2.0.1/sbcl-2.0.1/obj/from-host/src/compiler/generic/genesis.fas-tmp
> -# 0 errors, 0 warnings
> -# ;; Loading file obj/from-host/src/compiler/generic/genesis.fas ...
> -# ;; Loaded file obj/from-host/src/compiler/generic/genesis.fas
> -# *** - OPEN: File
> -# #P"/pobj/sbcl-2.0.1/sbcl-2.0.1/obj/from-xc/tls-init.lisp-expr" does
> not
> -# exist
> -# The following restarts are available:
> -# SKIP :R1 skip (GENESIS OBJECT-FILE-NAMES # ...)
> -# RETRY :R2 retry (GENESIS OBJECT-FILE-NAMES # ...)
> -# STOP :R3 stop loading file
> /pobj/sbcl-2.0.1/sbcl-2.0.1/make-genesis-2.lisp
> -# ABORT-BUILD:R4 Abort building SBCL.
> -# ABORT :R5 Abort main loop
> -# //testing for consistency of first and second GENESIS passes
> -# diff: output/genesis-2: No such file or directory
> -# error: header files do not match between first and second GENESIS
> -
> # not yet ported to other arches
> ONLY_FOR_ARCHS = amd64 i386 powerpc
> USE_WXNEEDED = Yes
>
> COMMENT= compiler and runtime system for ANSI Common Lisp
>
> -V = 2.1.11
> +V = 2.2.1
> DISTNAME=sbcl-${V}-source
> PKGNAME= sbcl-${V}
> WRKDIST= ${WRKDIR}/sbcl-${V}
> EXTRACT_SUFX=.tar.bz2
> +PATCHORIG = .orig.port
>
> CATEGORIES= lang
> HOMEPAGE=http://www.sbcl.org/
> @@ -58,10 +40,17 @@ WANTLIB+= pthread
> MAKE_PARAMS += --with-sb-core-compression \
> --with-sb-xref-for-internals
>
> +# contrib/sb-capstone/test.lisp uses it at build-time if present
> +BUILD_DEPENDS = devel/capstone/main
> +
> .if ${FLAVOR:Mnative_bootstrap}
> BUILD_DEPENDS+= lang/sbcl
> BOOTSTRAP_CMD= ${LOCALBASE}/bin/sbcl \
> --disable-debugger --no-sysinit --no-userinit
> +.elif ${MACHINE_ARCH:Mi386}
> +# ecl is slower but lang/clips fails to build sbcl on i386
> +BUILD_DEPENDS += lang/ecl
> +BOOTSTRAP_CMD = ${LOCALBASE}/bin/ecl -q --norc
> .else
> BUILD_DEPENDS += lang/clisp
> BOOTSTRAP_CMD = ${LOCALBASE}/bin/clisp -q -norc
> Index: distinfo
> ===
> RCS file:
Re: update lang/sbcl to 2.2.1
On Tue, Feb 22, 2022 at 11:58:49PM -0500, George Koehler wrote:
> On Wed, 23 Feb 2022 00:08:34 +0100
> Omar Polo wrote:
>
> > I think I've finally solved it. The problem is that contrib/sb-capstone
> > hardcodes some values from capstone.h so lisp can call the C functions.
> > This "transcription" however is not accurate and worked until now by
> > pure chance on 64 bit arches due to how arguments are passed and struct
> > padding, pure luck!
>
> I tried to patch capstone.lisp but didn't fix my build. Your patch
> looks better, but I haven't built it. I have comments below.
>
> The build deletes every *.orig and breaks "make update-patches", so
> I started using "PATCHORIG= .orig.port". Joshua Elsasser, as the
> maintainer, do you have an opinion about PATCHORIG?
I will try to find some time to take a look at it this week. For what
it's worth, 32-bit powerpc is usually a bit broken on non-openbsd
platforms as well. I have a couple g4 mac minis that I can test on,
assuming they haven't died.
Switching to ECL as the cross-build host is something what we probably
want to do, as it would also allow sbcl to be build on arm and
arm64. I had a version of the port which did this but as you saw, it
was rather slower than with clisp.
> Go down for patch comments.
>
> > Index: Makefile
> > ===
> > RCS file: /cvs/ports/lang/sbcl/Makefile,v
> > retrieving revision 1.47
> > diff -u -p -r1.47 Makefile
> > --- Makefile31 Dec 2021 09:53:11 - 1.47
> > +++ Makefile22 Feb 2022 23:07:25 -
> > @@ -1,31 +1,12 @@
> > # $OpenBSD: Makefile,v 1.47 2021/12/31 09:53:11 solene Exp $
> >
> > -BROKEN-i386 = build fails in "Compiling file
> > [...]/src/compiler/generic/genesis.lisp"
> > -# ;; Compiling file
> > /pobj/sbcl-2.0.1/sbcl-2.0.1/src/compiler/generic/genesis.lisp ...
> > -# ;; Wrote file
> > /pobj/sbcl-2.0.1/sbcl-2.0.1/obj/from-host/src/compiler/generic/genesis.fas-tmp
> > -# 0 errors, 0 warnings
> > -# ;; Loading file obj/from-host/src/compiler/generic/genesis.fas ...
> > -# ;; Loaded file obj/from-host/src/compiler/generic/genesis.fas
> > -# *** - OPEN: File
> > -# #P"/pobj/sbcl-2.0.1/sbcl-2.0.1/obj/from-xc/tls-init.lisp-expr"
> > does not
> > -# exist
> > -# The following restarts are available:
> > -# SKIP :R1 skip (GENESIS OBJECT-FILE-NAMES # ...)
> > -# RETRY :R2 retry (GENESIS OBJECT-FILE-NAMES # ...)
> > -# STOP :R3 stop loading file
> > /pobj/sbcl-2.0.1/sbcl-2.0.1/make-genesis-2.lisp
> > -# ABORT-BUILD:R4 Abort building SBCL.
> > -# ABORT :R5 Abort main loop
> > -# //testing for consistency of first and second GENESIS passes
> > -# diff: output/genesis-2: No such file or directory
> > -# error: header files do not match between first and second GENESIS
> > -
> > # not yet ported to other arches
> > ONLY_FOR_ARCHS = amd64 i386 powerpc
> > USE_WXNEEDED = Yes
> >
> > COMMENT= compiler and runtime system for ANSI Common Lisp
> >
> > -V =2.1.11
> > +V =2.2.1
> > DISTNAME= sbcl-${V}-source
> > PKGNAME= sbcl-${V}
> > WRKDIST= ${WRKDIR}/sbcl-${V}
> > @@ -58,10 +39,17 @@ WANTLIB+= pthread
> > MAKE_PARAMS += --with-sb-core-compression \
> > --with-sb-xref-for-internals
> >
> > +# contrib/sb-capstone/test.lisp opens it at build-time if present
> > +BUILD_DEPENDS =devel/capstone/main
> > +
> > .if ${FLAVOR:Mnative_bootstrap}
> > BUILD_DEPENDS+=lang/sbcl
> > BOOTSTRAP_CMD= ${LOCALBASE}/bin/sbcl \
> > --disable-debugger --no-sysinit --no-userinit
> > +.elif ${MACHINE_ARCH:Mi386}
> > +# ecl is way slower but clips fails to build sbcl on i386
> > +BUILD_DEPENDS += lang/ecl
> > +BOOTSTRAP_CMD =${LOCALBASE}/bin/ecl -q --norc
> > .else
> > BUILD_DEPENDS += lang/clisp
> > BOOTSTRAP_CMD =${LOCALBASE}/bin/clisp -q -norc
> > Index: distinfo
> > ===
> > RCS file: /cvs/ports/lang/sbcl/distinfo,v
> > retrieving revision 1.21
> > diff -u -p -r1.21 distinfo
> > --- distinfo31 Dec 2021 09:53:11 - 1.21
> > +++ distinfo22 Feb 2022 23:07:25 -
> > @@ -1,2 +1,2 @@
> > -SHA256 (sbcl-2.1.11-source.tar.bz2) =
> > v8FIHef9vfru8qsPDo6E79NDQz3qjSHPvqiwFGy9/v0=
> > -SIZE (sbcl-2.1.11-source.tar.bz2) = 6687529
> > +SHA256 (sbcl-2.2.1-source.tar.bz2) =
> > Xdbm4/CLfG7fJioOhEqfi15WLMoIFVA0wfLAFPyQh9o=
> > +SIZE (sbcl-2.2.1-source.tar.bz2) = 6701705
> > Index: patches/patch-contrib_sb-capstone_capstone_lisp
> > ===
> > RCS file: patches/patch-contrib_sb-capstone_capstone_lisp
> > diff -N patches/patch-contrib_sb-capstone_capstone_lisp
> > --- /dev/null
Re: update lang/sbcl to 2.2.1
George Koehler writes:
> On Wed, 23 Feb 2022 00:08:34 +0100
> Omar Polo wrote:
>
>> I think I've finally solved it. The problem is that contrib/sb-capstone
>> hardcodes some values from capstone.h so lisp can call the C functions.
>> This "transcription" however is not accurate and worked until now by
>> pure chance on 64 bit arches due to how arguments are passed and struct
>> padding, pure luck!
>
> I tried to patch capstone.lisp but didn't fix my build. Your patch
> looks better, but I haven't built it. I have comments below.
>
> The build deletes every *.orig and breaks "make update-patches", so
> I started using "PATCHORIG= .orig.port". Joshua Elsasser, as the
> maintainer, do you have an opinion about PATCHORIG?
>
> Go down for patch comments.
>
>> [...]
>> Index: patches/patch-contrib_sb-capstone_capstone_lisp
>> ===
>> RCS file: patches/patch-contrib_sb-capstone_capstone_lisp
>> diff -N patches/patch-contrib_sb-capstone_capstone_lisp
>> --- /dev/null1 Jan 1970 00:00:00 -
>> +++ patches/patch-contrib_sb-capstone_capstone_lisp 22 Feb 2022 23:07:25
>> -
>> @@ -0,0 +1,26 @@
>> +$OpenBSD$
>> +
>> +sync a couple of size mismatches with capstone.h; fixes the build on
>> +32bit arches (i386, powerpc.)
>> +
>> +Index: contrib/sb-capstone/capstone.lisp
>> +--- contrib/sb-capstone/capstone.lisp.orig
>> contrib/sb-capstone/capstone.lisp
>> +@@ -285,7 +285,7 @@
>> + (define-alien-type cs-insn
>> + (struct nil
>> + (insn-id int)
>> +-(insn-addr unsigned)
>> ++(insn-addr (integer 64))
>> + (insn-size short)
>> + (insn-bytes (array char 16))
>> + (insn-mnemonic (array char 32))
>
> I guess that it should be (insn-addr (unsigned 64)), to match the
> capstone.h's struct cs_insn's uint64_t address.
agreed, sorry for the typo, i meant unsigned
> I have not called C from Lisp before now, so sb-alien confused me.
> I now believe that "integer" and "unsigned" have pointer size, but
> "int" and "unsigned-int" have 32 bits. So "unsigned" had 64 bits
> on amd64, or 32 bits on i386|powerpc.
I've yet to find a list of types and their counterpart in C in the sbcl
documentation, I'm only finding stuff for sb-groveler which uses
sb-alien. Anyway, I *think* that `integer' and `unsigned' should mirror
their C counterpart (int and unsigned int), so 32 bit on amd64, i386 and
powerpc.
http://www.lichteblau.com/sbcl/doc/manual/sbcl/Foreign-Type-Specifiers.html
My (unconfirmed) guess is that on amd64 the compiler aligns the short at
64 bit boundaries and so there are 32 bits of padding between ins-addr
and ins-size, which makes the difference between `unsigned' and (integer
64) not noticeable for small values -- at least on LE. test.lisp uses
small chunks that output only a few bytes so the truncation due to the
wrong size went unseen. Testing on a 64bit BE arch would have
discovered this issue too probably.
>> +@@ -310,7 +310,7 @@
>> +
>> + ;; The handle returned by cs-open will be represented as being of type
>> unsigned
>> +
>> +-(define-alien-routine cs-open int (arch int) (mode (integer 64)) (handle
>> unsigned :out))
>> ++(define-alien-routine cs-open int (arch int) (mode unsigned) (handle
>> unsigned :out))
>> +
>> + (define-alien-routine cs-version unsigned (major int :out) (minor int
>> :out))
>> +
>
> cs-open should have "(mode int)". has "cs_mode mode" and
> powerpc clang says that "(cs_mode)-1 > 0" is false, implying that
> cs_mode is a signed type.
(i forgot to explain this in the previous mail, sorry)
That was my first thought as well, but the third test fails if `mode' is
`int':
Test SB-CAPSTONE-TESTS::PPC-BIG-ENDIAN failed
Form: (SB-CAPSTONE-TESTS::CAPSTONE-CHECK '(124 8 2 166) '(:PPC64 :BIG-ENDIAN)
'("MFLR R0"))
Expected value: T
Actual value: #.
The value
2147483656
is not of type
(SIGNED-BYTE 32)
my reading is that capstone.lisp:get-cs-mode tries to set the most
significant bit which is fine in C but sbcl begs to differ. (2147483656
is actually -2147483640 in base 2 complement if I'm doing the math
right, or 0x8008)
This probably explain why upstream went for (integer 64), sbcl doesn't
warn and the value is still correctly passed to and read by the library
on amd64 (don't know the details of the calling convention but I guess
32 and 64 bits integers are passed on registers.)
> I also think that cs-version should return "unsigned-int" (32 bits),
> not "unsigned" (64 bits on amd64), though I suspect that amd64
> tolerates the wrong size.
It's true that the wrong size is tolerated but I think that `unsigned'
mirrors C' `unsigned int', so it's already correct?
> --George
Updated diff below with the type fixed for insn-addr and PATCHORIG for
our sanity :)
(note that I haven't finished to build with the following patch, I just
started it, but I'm quite confident it'll work)
Re: update lang/sbcl to 2.2.1
On Wed, 23 Feb 2022 00:08:34 +0100
Omar Polo wrote:
> I think I've finally solved it. The problem is that contrib/sb-capstone
> hardcodes some values from capstone.h so lisp can call the C functions.
> This "transcription" however is not accurate and worked until now by
> pure chance on 64 bit arches due to how arguments are passed and struct
> padding, pure luck!
I tried to patch capstone.lisp but didn't fix my build. Your patch
looks better, but I haven't built it. I have comments below.
The build deletes every *.orig and breaks "make update-patches", so
I started using "PATCHORIG= .orig.port". Joshua Elsasser, as the
maintainer, do you have an opinion about PATCHORIG?
Go down for patch comments.
> Index: Makefile
> ===
> RCS file: /cvs/ports/lang/sbcl/Makefile,v
> retrieving revision 1.47
> diff -u -p -r1.47 Makefile
> --- Makefile 31 Dec 2021 09:53:11 - 1.47
> +++ Makefile 22 Feb 2022 23:07:25 -
> @@ -1,31 +1,12 @@
> # $OpenBSD: Makefile,v 1.47 2021/12/31 09:53:11 solene Exp $
>
> -BROKEN-i386 =build fails in "Compiling file
> [...]/src/compiler/generic/genesis.lisp"
> -# ;; Compiling file
> /pobj/sbcl-2.0.1/sbcl-2.0.1/src/compiler/generic/genesis.lisp ...
> -# ;; Wrote file
> /pobj/sbcl-2.0.1/sbcl-2.0.1/obj/from-host/src/compiler/generic/genesis.fas-tmp
> -# 0 errors, 0 warnings
> -# ;; Loading file obj/from-host/src/compiler/generic/genesis.fas ...
> -# ;; Loaded file obj/from-host/src/compiler/generic/genesis.fas
> -# *** - OPEN: File
> -# #P"/pobj/sbcl-2.0.1/sbcl-2.0.1/obj/from-xc/tls-init.lisp-expr" does
> not
> -# exist
> -# The following restarts are available:
> -# SKIP :R1 skip (GENESIS OBJECT-FILE-NAMES # ...)
> -# RETRY :R2 retry (GENESIS OBJECT-FILE-NAMES # ...)
> -# STOP :R3 stop loading file
> /pobj/sbcl-2.0.1/sbcl-2.0.1/make-genesis-2.lisp
> -# ABORT-BUILD:R4 Abort building SBCL.
> -# ABORT :R5 Abort main loop
> -# //testing for consistency of first and second GENESIS passes
> -# diff: output/genesis-2: No such file or directory
> -# error: header files do not match between first and second GENESIS
> -
> # not yet ported to other arches
> ONLY_FOR_ARCHS = amd64 i386 powerpc
> USE_WXNEEDED = Yes
>
> COMMENT= compiler and runtime system for ANSI Common Lisp
>
> -V = 2.1.11
> +V = 2.2.1
> DISTNAME=sbcl-${V}-source
> PKGNAME= sbcl-${V}
> WRKDIST= ${WRKDIR}/sbcl-${V}
> @@ -58,10 +39,17 @@ WANTLIB+= pthread
> MAKE_PARAMS += --with-sb-core-compression \
> --with-sb-xref-for-internals
>
> +# contrib/sb-capstone/test.lisp opens it at build-time if present
> +BUILD_DEPENDS = devel/capstone/main
> +
> .if ${FLAVOR:Mnative_bootstrap}
> BUILD_DEPENDS+= lang/sbcl
> BOOTSTRAP_CMD= ${LOCALBASE}/bin/sbcl \
> --disable-debugger --no-sysinit --no-userinit
> +.elif ${MACHINE_ARCH:Mi386}
> +# ecl is way slower but clips fails to build sbcl on i386
> +BUILD_DEPENDS += lang/ecl
> +BOOTSTRAP_CMD = ${LOCALBASE}/bin/ecl -q --norc
> .else
> BUILD_DEPENDS += lang/clisp
> BOOTSTRAP_CMD = ${LOCALBASE}/bin/clisp -q -norc
> Index: distinfo
> ===
> RCS file: /cvs/ports/lang/sbcl/distinfo,v
> retrieving revision 1.21
> diff -u -p -r1.21 distinfo
> --- distinfo 31 Dec 2021 09:53:11 - 1.21
> +++ distinfo 22 Feb 2022 23:07:25 -
> @@ -1,2 +1,2 @@
> -SHA256 (sbcl-2.1.11-source.tar.bz2) =
> v8FIHef9vfru8qsPDo6E79NDQz3qjSHPvqiwFGy9/v0=
> -SIZE (sbcl-2.1.11-source.tar.bz2) = 6687529
> +SHA256 (sbcl-2.2.1-source.tar.bz2) =
> Xdbm4/CLfG7fJioOhEqfi15WLMoIFVA0wfLAFPyQh9o=
> +SIZE (sbcl-2.2.1-source.tar.bz2) = 6701705
> Index: patches/patch-contrib_sb-capstone_capstone_lisp
> ===
> RCS file: patches/patch-contrib_sb-capstone_capstone_lisp
> diff -N patches/patch-contrib_sb-capstone_capstone_lisp
> --- /dev/null 1 Jan 1970 00:00:00 -
> +++ patches/patch-contrib_sb-capstone_capstone_lisp 22 Feb 2022 23:07:25
> -
> @@ -0,0 +1,26 @@
> +$OpenBSD$
> +
> +sync a couple of size mismatches with capstone.h; fixes the build on
> +32bit arches (i386, powerpc.)
> +
> +Index: contrib/sb-capstone/capstone.lisp
> +--- contrib/sb-capstone/capstone.lisp.orig
> contrib/sb-capstone/capstone.lisp
> +@@ -285,7 +285,7 @@
> + (define-alien-type cs-insn
> + (struct nil
> + (insn-id int)
> +-(insn-addr unsigned)
> ++(insn-addr (integer 64))
> + (insn-size short)
> + (insn-bytes (array char 16))
> + (insn-mnemonic (array char 32))
I guess that it should be (insn-addr (unsigned
Re: update lang/sbcl to 2.2.1
>> George Koehler writes:
>>
>>> I have a problem: if devel/capstone/main is installed, then sbcl fails
>>> to build on powerpc. This happens with sbcl-2.1.11 or sbcl-2.2.1.
>>>
>>> sbcl doesn't depend on capstone, and doesn't need capstone to build
>>> contrib/sb-capstone, but does try to test sb-capstone during the
>>> build. If libcapstone.so is missing, then sbcl skips the test. If
>>> libcapstone.so is found, then the test crashes, so the build fails.
>>> An installed capstone broke sbcl-2.1.11 in the ongoing powerpc bulk;
>>> I told the bulk to retry sbcl; it succeeded after removing capstone.
I think I've finally solved it. The problem is that contrib/sb-capstone
hardcodes some values from capstone.h so lisp can call the C functions.
This "transcription" however is not accurate and worked until now by
pure chance on 64 bit arches due to how arguments are passed and struct
padding, pure luck!
I've also discovered that sbcl compiles on i386 if built with ecl. To
be honest the sbcl build infrastructure scares me, so I haven't digged
into understanding why clisp won't do it when it works on amd64 and
powerpc. The downside is that ecl is quite slow, here it takes roughly
an hour and half to compile the port with it.
So, the following patch updates sbcl, adds a bdep on capstone^1, fixes
the issue reported and re-enables it for i386 with the ecl workaround.
The patch should definitely be upstreamed and I'll look into how to do
it tomorrow.
OK/Comments/Tests? :)
1: the build is not affected by the presence of capstone and the module
is built anyway; if it's present it just runs some extra tests that
otherwise are skipped.
Cheers,
Omar Polo
Index: Makefile
===
RCS file: /cvs/ports/lang/sbcl/Makefile,v
retrieving revision 1.47
diff -u -p -r1.47 Makefile
--- Makefile31 Dec 2021 09:53:11 - 1.47
+++ Makefile22 Feb 2022 23:07:25 -
@@ -1,31 +1,12 @@
# $OpenBSD: Makefile,v 1.47 2021/12/31 09:53:11 solene Exp $
-BROKEN-i386 = build fails in "Compiling file
[...]/src/compiler/generic/genesis.lisp"
-# ;; Compiling file
/pobj/sbcl-2.0.1/sbcl-2.0.1/src/compiler/generic/genesis.lisp ...
-# ;; Wrote file
/pobj/sbcl-2.0.1/sbcl-2.0.1/obj/from-host/src/compiler/generic/genesis.fas-tmp
-# 0 errors, 0 warnings
-# ;; Loading file obj/from-host/src/compiler/generic/genesis.fas ...
-# ;; Loaded file obj/from-host/src/compiler/generic/genesis.fas
-# *** - OPEN: File
-# #P"/pobj/sbcl-2.0.1/sbcl-2.0.1/obj/from-xc/tls-init.lisp-expr" does not
-# exist
-# The following restarts are available:
-# SKIP :R1 skip (GENESIS OBJECT-FILE-NAMES # ...)
-# RETRY :R2 retry (GENESIS OBJECT-FILE-NAMES # ...)
-# STOP :R3 stop loading file
/pobj/sbcl-2.0.1/sbcl-2.0.1/make-genesis-2.lisp
-# ABORT-BUILD:R4 Abort building SBCL.
-# ABORT :R5 Abort main loop
-# //testing for consistency of first and second GENESIS passes
-# diff: output/genesis-2: No such file or directory
-# error: header files do not match between first and second GENESIS
-
# not yet ported to other arches
ONLY_FOR_ARCHS = amd64 i386 powerpc
USE_WXNEEDED = Yes
COMMENT= compiler and runtime system for ANSI Common Lisp
-V =2.1.11
+V =2.2.1
DISTNAME= sbcl-${V}-source
PKGNAME= sbcl-${V}
WRKDIST= ${WRKDIR}/sbcl-${V}
@@ -58,10 +39,17 @@ WANTLIB+= pthread
MAKE_PARAMS += --with-sb-core-compression \
--with-sb-xref-for-internals
+# contrib/sb-capstone/test.lisp opens it at build-time if present
+BUILD_DEPENDS =devel/capstone/main
+
.if ${FLAVOR:Mnative_bootstrap}
BUILD_DEPENDS+=lang/sbcl
BOOTSTRAP_CMD= ${LOCALBASE}/bin/sbcl \
--disable-debugger --no-sysinit --no-userinit
+.elif ${MACHINE_ARCH:Mi386}
+# ecl is way slower but clips fails to build sbcl on i386
+BUILD_DEPENDS += lang/ecl
+BOOTSTRAP_CMD =${LOCALBASE}/bin/ecl -q --norc
.else
BUILD_DEPENDS += lang/clisp
BOOTSTRAP_CMD =${LOCALBASE}/bin/clisp -q -norc
Index: distinfo
===
RCS file: /cvs/ports/lang/sbcl/distinfo,v
retrieving revision 1.21
diff -u -p -r1.21 distinfo
--- distinfo31 Dec 2021 09:53:11 - 1.21
+++ distinfo22 Feb 2022 23:07:25 -
@@ -1,2 +1,2 @@
-SHA256 (sbcl-2.1.11-source.tar.bz2) =
v8FIHef9vfru8qsPDo6E79NDQz3qjSHPvqiwFGy9/v0=
-SIZE (sbcl-2.1.11-source.tar.bz2) = 6687529
+SHA256 (sbcl-2.2.1-source.tar.bz2) =
Xdbm4/CLfG7fJioOhEqfi15WLMoIFVA0wfLAFPyQh9o=
+SIZE (sbcl-2.2.1-source.tar.bz2) = 6701705
Index: patches/patch-contrib_sb-capstone_capstone_lisp
===
RCS file:
Re: update lang/sbcl to 2.2.1
Omar Polo writes:
> George Koehler writes:
>
>>> [..]
>>
>> I have a problem: if devel/capstone/main is installed, then sbcl fails
>> to build on powerpc. This happens with sbcl-2.1.11 or sbcl-2.2.1.
>>
>> sbcl doesn't depend on capstone, and doesn't need capstone to build
>> contrib/sb-capstone, but does try to test sb-capstone during the
>> build. If libcapstone.so is missing, then sbcl skips the test. If
>> libcapstone.so is found, then the test crashes, so the build fails.
>> An installed capstone broke sbcl-2.1.11 in the ongoing powerpc bulk;
>> I told the bulk to retry sbcl; it succeeded after removing capstone.
>>
>> [...]
>>
>> Doing 3 pending tests of 3 tests total.
>> CORRUPTION WARNING in SBCL pid 41036:
>> Memory fault at 0x0 (pc=0xb96bb61c)
>> The integrity of this image is possibly compromised.
>> Exiting.
>>0: [I*]0x9361bda0 pc=0xb96bb61c {0x50c37288+68a84394} {code_serialno=9970}
>>1: [*] 0x9361bd80 pc=0x50c374b8 {0x50c37288+0230} SB-CAPSTONE::CS-OPEN
>
> could this be an issue in capstone itself on powerpc?
> SB-CAPSTONE::CS-OPEN is "just"
>
> (define-alien-routine cs-open int (arch int) (mode (integer 64)) (handle
> unsigned :out))
>
> but I don't know how sbcl calls into c. To be fair the `integer 64`
> looks wrong, cs_open takes an enum as second argument which should be
> just an `int' and the fault at 0x0 could be that 32 bits of mode are
> read as the next argument which should be a pointer; I'm probably
> talking crap tho. I'll do a build on i386 out of curiosity thought :)
for once, I think I was right! :D
I've sort of unbroken the build on i386 (using ecl instead of clisp
makes it, even thought it's slowww) and can reproduce the error.
Changing the cs-open declaration to specify `mode' as int makes the test
fail (excerpt attached), but not crash, which is a huge improvement :D
I'm doing yet another build this time without capstone so I can obtain a
working executable to debug the test but it'll take a while, my hardware
is old and slow, and ecl slows it down even more, I just wanted to give
a quick follow up.
Hopefully I can understand why it fails so we can properly add capstone
as build dependency and re-enable the build on i386 :)
Cheers
Test SB-CAPSTONE-TESTS::X86-64 failed
Form: (SB-CAPSTONE-TESTS::CAPSTONE-CHECK '(143 69 8 72 139 240)
'(:X86-64 :LITTLE-ENDIAN)
'("MOV RSI, RAX"
"POP QWORD PTR [RBP + 8]"))
Expected value: T
Actual value: NIL.
Test SB-CAPSTONE-TESTS::PPC-LITTLE-ENDIAN failed
Form: (SB-CAPSTONE-TESTS::CAPSTONE-CHECK '(3 16 64 60 0 113 66 56)
'(:PPC64 :LITTLE-ENDIAN)
'("LIS R2, 0X1003"
"ADDI R2, R2, 0X7100"))
Expected value: T
Actual value: NIL.
Test SB-CAPSTONE-TESTS::PPC-BIG-ENDIAN failed
Form: (SB-CAPSTONE-TESTS::CAPSTONE-CHECK '(124 8 2 166) '(:PPC64 :BIG-ENDIAN)
'("MFLR R0"))
Expected value: T
Actual value: #.
The value
2147483656
is not of type
(SIGNED-BYTE 32)
when binding SB-ALIEN::VALUE
Re: update lang/sbcl to 2.2.1
George Koehler writes:
> On Sat, 19 Feb 2022 12:29:18 +0100
> Omar Polo wrote:
>
>> Hello,
>>
>> please find attached a trivial diff to update lang/sbcl to the latest
>> version, 2.2.1 release the last January 30.
>>
>> The changelog is here: http://www.sbcl.org/news.html#2.2.1
>>
>> `make test' seems just like the current version, I'm typing this mail
>> from stumpwm built with sbcl-threads, and I've also tested a couple of
>> other programs that ends up using various libraries from quicklisp with
>> success.
>>
>> OK/comments?
>
> I have a problem: if devel/capstone/main is installed, then sbcl fails
> to build on powerpc. This happens with sbcl-2.1.11 or sbcl-2.2.1.
>
> sbcl doesn't depend on capstone, and doesn't need capstone to build
> contrib/sb-capstone, but does try to test sb-capstone during the
> build. If libcapstone.so is missing, then sbcl skips the test. If
> libcapstone.so is found, then the test crashes, so the build fails.
> An installed capstone broke sbcl-2.1.11 in the ongoing powerpc bulk;
> I told the bulk to retry sbcl; it succeeded after removing capstone.
>
> contrib might also try to open libmpfr.so (devel/mpfr) and
> libgmp.so (devel/gmp), but both of these are somewhere in sbcl's
> build dependencies.
>
> The fix might be to disable the test, but I have not yet tried this.
> The build failure in sbcl-2.2.1 is,
I can't reproduce the issue on amd64, it builds fine without capstone
and the tests are passing if it's installed.
> Doing 3 pending tests of 3 tests total.
> CORRUPTION WARNING in SBCL pid 41036:
> Memory fault at 0x0 (pc=0xb96bb61c)
> The integrity of this image is possibly compromised.
> Exiting.
>0: [I*]0x9361bda0 pc=0xb96bb61c {0x50c37288+68a84394} {code_serialno=9970}
>1: [*] 0x9361bd80 pc=0x50c374b8 {0x50c37288+0230} SB-CAPSTONE::CS-OPEN
could this be an issue in capstone itself on powerpc?
SB-CAPSTONE::CS-OPEN is "just"
(define-alien-routine cs-open int (arch int) (mode (integer 64)) (handle
unsigned :out))
but I don't know how sbcl calls into c. To be fair the `integer 64`
looks wrong, cs_open takes an enum as second argument which should be
just an `int' and the fault at 0x0 could be that 32 bits of mode are
read as the next argument which should be a pointer; I'm probably
talking crap tho. I'll do a build on i386 out of curiosity thought :)
Re: update lang/sbcl to 2.2.1
On Sat, 19 Feb 2022 12:29:18 +0100
Omar Polo wrote:
> Hello,
>
> please find attached a trivial diff to update lang/sbcl to the latest
> version, 2.2.1 release the last January 30.
>
> The changelog is here: http://www.sbcl.org/news.html#2.2.1
>
> `make test' seems just like the current version, I'm typing this mail
> from stumpwm built with sbcl-threads, and I've also tested a couple of
> other programs that ends up using various libraries from quicklisp with
> success.
>
> OK/comments?
I have a problem: if devel/capstone/main is installed, then sbcl fails
to build on powerpc. This happens with sbcl-2.1.11 or sbcl-2.2.1.
sbcl doesn't depend on capstone, and doesn't need capstone to build
contrib/sb-capstone, but does try to test sb-capstone during the
build. If libcapstone.so is missing, then sbcl skips the test. If
libcapstone.so is found, then the test crashes, so the build fails.
An installed capstone broke sbcl-2.1.11 in the ongoing powerpc bulk;
I told the bulk to retry sbcl; it succeeded after removing capstone.
contrib might also try to open libmpfr.so (devel/mpfr) and
libgmp.so (devel/gmp), but both of these are somewhere in sbcl's
build dependencies.
The fix might be to disable the test, but I have not yet tried this.
The build failure in sbcl-2.2.1 is,
Doing 3 pending tests of 3 tests total.
CORRUPTION WARNING in SBCL pid 41036:
Memory fault at 0x0 (pc=0xb96bb61c)
The integrity of this image is possibly compromised.
Exiting.
0: [I*]0x9361bda0 pc=0xb96bb61c {0x50c37288+68a84394} {code_serialno=9970}
1: [*] 0x9361bd80 pc=0x50c374b8 {0x50c37288+0230} SB-CAPSTONE::CS-OPEN
...
gmake[1]: *** [../asdf-module.mk:41: test] Error 1
gmake[1]: Leaving directory
'/usr/ports/pobj/sbcl-2.2.1/sbcl-2.2.1/contrib/sb-capstone'
sh ./build-contrib sb-rotate-byte
gmake[1]: Entering directory
'/usr/ports/pobj/sbcl-2.2.1/sbcl-2.2.1/contrib/sb-rotate-byte'
...
* gmake[1]: Leaving directory '/usr/ports/pobj/sbcl-2.2.1/sbcl-2.2.1/contrib/sb-
graph'
gmake: Leaving directory '/usr/ports/pobj/sbcl-2.2.1/sbcl-2.2.1/contrib'
WARNING! Some of the contrib modules did not build successfully or pass
their self-tests. Failed contribs:"
sb-capstone
*** Error 1 in . (Makefile:95 'do-build')
--George
> Cheers
>
>
> Index: Makefile
> ===
> RCS file: /home/cvs/ports/lang/sbcl/Makefile,v
> retrieving revision 1.47
> diff -u -p -r1.47 Makefile
> --- Makefile 31 Dec 2021 09:53:11 - 1.47
> +++ Makefile 18 Feb 2022 16:59:18 -
> @@ -25,7 +25,7 @@ USE_WXNEEDED = Yes
>
> COMMENT= compiler and runtime system for ANSI Common Lisp
>
> -V = 2.1.11
> +V = 2.2.1
> DISTNAME=sbcl-${V}-source
> PKGNAME= sbcl-${V}
> WRKDIST= ${WRKDIR}/sbcl-${V}
> Index: distinfo
> ===
> RCS file: /home/cvs/ports/lang/sbcl/distinfo,v
> retrieving revision 1.21
> diff -u -p -r1.21 distinfo
> --- distinfo 31 Dec 2021 09:53:11 - 1.21
> +++ distinfo 18 Feb 2022 16:59:50 -
> @@ -1,2 +1,2 @@
> -SHA256 (sbcl-2.1.11-source.tar.bz2) =
> v8FIHef9vfru8qsPDo6E79NDQz3qjSHPvqiwFGy9/v0=
> -SIZE (sbcl-2.1.11-source.tar.bz2) = 6687529
> +SHA256 (sbcl-2.2.1-source.tar.bz2) =
> Xdbm4/CLfG7fJioOhEqfi15WLMoIFVA0wfLAFPyQh9o=
> +SIZE (sbcl-2.2.1-source.tar.bz2) = 6701705
>
--
George Koehler
