Re: [Sbcl-bugs] hide-packages.test.sh failure with 2.2.6

[Sean Whitton]

Hello,

Patch uploaded to Debian unstable, thank you!

-- 
Sean Whitton

Re: [Sbcl-bugs] hide-packages.test.sh failure with 2.2.6

[Sean Whitton]

Hello,

On Fri 22 Jul 2022 at 01:11pm -04, Douglas Katzman wrote:

> Can you tell me all your build options and whether you are applying any
> patches?

I'm new to Debian's SBCL packaging, so it's possible I'm missing
something in our Makefile, but this is what it looks like we are doing:

./make.sh \
 --xc-host="/usr/bin/sbcl --disable-debugger --no-sysinit 
--no-userinit" \
 --prefix=/$(CURDIR)/stage1 --fancy --with-sb-linkable-runtime
./install.sh
chmod 100 stage1
./clean.sh

./make.sh \
--xc-host="/$(CURDIR)/stage1/bin/sbcl --disable-debugger --no-sysinit 
--no-userinit" \
--prefix=/usr --fancy --with-sb-linkable-runtime
HOME="$(CURDIR)" src/runtime/sbcl --core output/sbcl.core --no-sysinit 
--no-userinit \
 --eval '(sb-ext:set-sbcl-source-location 
"/usr/share/sbcl-source/")' \
 --eval '(sb-ext:save-lisp-and-die "output/sbcl.core")'

chmod 700 stage1

We are applying a number of patches, but none added recently:


> Also, try the following test, both in a freshly started image and inserted
> within the hide-packages test.
> I get 5 lines output when I run it. I suspect that you get at least 6 lines of
> output.

>From the test,

// Running hide-packages.test.sh in COMPILE evaluator mode
"KEYWORD" is a symbol in #
"KEYWORD" is a symbol in #
"COMMON-LISP" is a symbol in #

After installing the built package,

root@develacc:~>sbcl --no-sysinit --no-userinit
[...]
"COMMON-LISP" is a symbol in #
"SB-THREAD" is a symbol in #
"SB-EVAL" is a symbol in #
"SB-VM" is a symbol in #
"SB-UNICODE" is a symbol in #
"KEYWORD" is a symbol in #
"KEYWORD" is a symbol in #

-- 
Sean Whitton

Re: [Sbcl-bugs] hide-packages.test.sh failure with 2.2.6

[Douglas Katzman]

The memory image you sent told me that there is a symbol
sb-kernel::sb-vm which
is why hiding of the string "sb-vm" failed.
And the fact that SEARCH-ROOTS didn't expose the symbol tells me there may
be a bug in search-roots, which is unfortunate but unrelated to the problem.

Can you tell me all your build options and whether you are applying any
patches?
Also, try the following test, both in a freshly started image and inserted
within the hide-packages test.
I get 5 lines output when I run it. I suspect that you get at least 6 lines
of output.

* *(lpnwasinsop)*

"COMMON-LISP" is a symbol in #

"SB-THREAD" is a symbol in #

"SB-UNICODE" is a symbol in #

"KEYWORD" is a symbol in #

"KEYWORD" is a symbol in #


(defun lpnwasinsop ()
"list-package-names-which-are-symbols-in-some-other-package"

  (let* ((pkgs (list-all-packages))

 (names (mapcar 'package-name pkgs)))

(dolist (name names)

  (dolist (pkg pkgs)

(flet ((find-in-symbols (tbl)

 (find name (sb-impl::package-hashtable-cells tbl)

   :test (lambda (x y) (and (symbolp y) (string= x
y))

  (when (or (find-in-symbols (sb-impl::package-internal-symbols
pkg))

(find-in-symbols (sb-impl::package-external-symbols
pkg)))

(format t "~S is a symbol in ~S~%" name pkg)))

>
>

Re: [Sbcl-bugs] hide-packages.test.sh failure with 2.2.6

[Sean Whitton]

Hello,

On Wed 20 Jul 2022 at 02:37pm -04, Douglas Katzman wrote:

> Ok, just to be clear, it printed nothing but still failed the assertion that
> *weak-ptrs* is null? That's not supposed to be possible.

That's what happened with just the first patch, yes.

> So another attempt. Could you apply this diff (which produces no
> output for me) but if it did, it would say: save: 1 threads Writing
> dynamic space (with 1 "threads" because of course it's spelled wrong,
> who would have just 1 thread?)  and then compress as aggressively as
> you can the resulting mem.bin file, maybe with bzip2, and place it
> somewhere accessible. Worst case, email it.

With both patches applied, I got the following output.  I'd prefer not
to post the core publicly; will send off-list.

// Running hide-packages.test.sh in COMPILE evaluator mode
; Liveness tracking: 1/1 live watched objects
0 pins:
Pass 1: Counting heap objects...
9 objs, 24 ptrs, 31 immediates
45487 objs, 74999 ptrs, 115893 immediates
17519 objs, 174190 ptrs, 52954 immediates
688575 objs, 1346884 ptrs, 1123637 immediates
Pass 2: Inverting heap. Initial size=1048576 objects
Scratchpad: 12771328 bytes
Inverted heap: ct=610004, cap=1048607, LF=58.172794 ET=0.004486+0.135073 
sys+usr=0.139559
Target=0x10004be19f (simple-base-str)
Failure tracing from 0x10004be19f. Current targets:
0x10004be19f(g6,simple-base-str)
save: 1 threads
Writing dynamic space
Unhandled SIMPLE-ERROR in thread #:
  The assertion (NULL *WEAK-PTRS*) failed with *WEAK-PTRS* =
  (#).

Backtrace for: #
0: (HIDDEN-SB-DEBUG::DEBUGGER-DISABLED-HOOK #" 
{1001C621B3}> # :QUIT T)
1: (HIDDEN-SB-DEBUG::RUN-HOOK *INVOKE-DEBUGGER-HOOK* #" 
{1001C621B3}>)
2: (INVOKE-DEBUGGER #" 
{1001C621B3}>)
3: (ERROR #" 
{1001C621B3}>)
4: (HIDDEN-SB-KERNEL:WITH-SIMPLE-CONDITION-RESTARTS ERROR NIL #" 
{1001C621B3}>)
5: (HIDDEN-SB-KERNEL:ASSERT-ERROR (NULL *WEAK-PTRS*) 1 *WEAK-PTRS* (#))
6: ((LAMBDA ()))
7: (HIDDEN-SB-INT:SIMPLE-EVAL-IN-LEXENV (ASSERT (NULL *WEAK-PTRS*)) 
#)
8: (EVAL (ASSERT (NULL *WEAK-PTRS*)))
9: (INTERACTIVE-EVAL (ASSERT (NULL *WEAK-PTRS*)) :EVAL NIL)
10: (HIDDEN-SB-IMPL::REPL-FUN T)
11: ((LAMBDA NIL :IN HIDDEN-SB-IMPL::TOPLEVEL-REPL))
12: (HIDDEN-SB-IMPL::%WITH-REBOUND-IO-SYNTAX #)
13: (HIDDEN-SB-IMPL::TOPLEVEL-REPL T)
14: (HIDDEN-SB-IMPL::TOPLEVEL-INIT)
15: ((FLET HIDDEN-SB-UNIX::BODY :IN HIDDEN-SB-IMPL::START-LISP))
16: ((FLET "WITHOUT-INTERRUPTS-BODY-3" :IN HIDDEN-SB-IMPL::START-LISP))
17: (HIDDEN-SB-IMPL::%START-LISP)

unhandled condition in --disable-debugger mode, quitting

-- 
Sean Whitton

Re: [Sbcl-bugs] hide-packages.test.sh failure with 2.2.6

[Douglas Katzman]

how about open a launchpad bug and attach the mem.bin.bz2 file
It should be under 8Mb if it's anything like this
-rw-r--r-- 1 dougk primarygroup 7850196 Jul 20 14:39 mem.bin.bz2

On Wed, Jul 20, 2022 at 2:37 PM Douglas Katzman  wrote:

> Ok, just to be clear, it printed nothing but still failed the assertion
> that *weak-ptrs* is null? That's not supposed to be possible.
> So another attempt. Could you apply this diff (which produces no output
> for me) but if it did, it would say:
>
> *save: 1 threadsWriting dynamic space*
> (with 1 "threads" because of course it's spelled wrong, who would have
> just 1 thread?)
> and then compress as aggressively as you can the resulting mem.bin file,
> maybe with bzip2, and place it somewhere accessible. Worst case, email it.
>
> diff --git a/tests/hide-packages.test.sh b/tests/hide-packages.test.sh
> index 0abf81d19..b250e3032 100644
> --- a/tests/hide-packages.test.sh
> +++ b/tests/hide-packages.test.sh
> @@ -23,6 +23,8 @@ run_sbcl <
>  (defvar *weak-ptrs* nil)
>  (progn
> +  (sb-impl::finalizer-thread-stop)
> +  (setf (extern-alien "heap_trace_verbose" (signed 32)) 1)
>;; Remove features that are also names of packages.
>;; This form needs to obfuscate the spellings of victim symbols, or else
>;; they end up in the *SOURCE-INFO* for this file, which is pinned while
> @@ -50,6 +52,9 @@ run_sbcl <  (gc :gen 7)
>  (setq *weak-ptrs* (remove-if-not #'weak-pointer-value *weak-ptrs*))
>  (when *weak-ptrs* (search-roots *weak-ptrs* :print :verbose :criterion
> :static))
> +(when *weak-ptrs*
> +  (alien-funcall (extern-alien "save_gc_crashdump" (function void
> c-string system-area-pointer))
> + "mem.bin" (hidden-sb-kernel:current-sp)))
>  (assert (null *weak-ptrs*))
>  (format t "Package hiding test 1: PASS~%")
>  EOF
>
> On Wed, Jul 20, 2022 at 1:35 PM Sean Whitton 
> wrote:
>
>> Hello,
>>
>> On Tue 19 Jul 2022 at 09:04pm -04, Douglas Katzman wrote:
>>
>> > You're the only one to report an error in this test so I really have
>> nothing to
>> > go on.
>> >
>> > Could you patch in
>> >
>> https://sourceforge.net/p/sbcl/sbcl/ci/58451118520ef7a7c164638274f939f1ffcff31e/
>> >
>> > and run the test to see what else it might say now?
>>
>> Thanks for looking.  Unfortunately, it doesn't seem to print anything.
>>
>> I added a dummy '(princ "hello")' just before your SEARCH-ROOTS line,
>> and that output is visible, so doesn't seem like it would be an issue
>> with Debian's package building setup.
>>
>> --
>> Sean Whitton
>>
>

Re: [Sbcl-bugs] hide-packages.test.sh failure with 2.2.6

[Sean Whitton]

Hello,

On Tue 19 Jul 2022 at 09:04pm -04, Douglas Katzman wrote:

> You're the only one to report an error in this test so I really have nothing 
> to
> go on.
>
> Could you patch in
> https://sourceforge.net/p/sbcl/sbcl/ci/58451118520ef7a7c164638274f939f1ffcff31e/
>
> and run the test to see what else it might say now?

Thanks for looking.  Unfortunately, it doesn't seem to print anything.

I added a dummy '(princ "hello")' just before your SEARCH-ROOTS line,
and that output is visible, so doesn't seem like it would be an issue
with Debian's package building setup.

-- 
Sean Whitton