Re: [PATCH] Avoid 'frame-local-ref' errors when printing backtrace.

[Andrew Whatson]

Ludovic Courtès  wrote:
>
> Applied, thanks!

Thank you!

> PS: Please use ‘git format-patch’ so the patch can be directly consumed
> by ‘git am’.

My mistake, sorry about that.  Noted for next time :)

Cheers,
Andrew

Re: [PATCH] Add 'bytevector-slice'.

[Maxime Devos]

This looks useful to me, especially once the optimiser recognises 
'bytevector-slice'.  (In Scheme-GNUnet, I defined a wrapper around 
bytevectors called 'bytevectors slices' to implement such a thing.)


The only thing missing for me, is a procedure 
'bytevector-slice/read-only' and 'bytevector-slice/write-only', then I 
could throw the module implementing the wrapping in Scheme-GNUnet and 
the overhead incurred by wrapping away.


On 11-01-2023 16:00, Ludovic Courtès wrote:

+@deffn {Scheme Procedure} bytevector-slice @var{bv} @var{offset} [@var{size}]
+@deffnx {C Function} scm_bytevector_slice (@var{bv}, @var{offset}, @var{size})
+Return the slice of @var{bv} starting at @var{offset} and counting
+@var{size} bytes.  When @var{size} is omitted, the slice covers all
+of @var{bv} starting from @var{offset}.  The returned slice shares
+storage with @var{bv}: changes to the slice are visible in @var{bv}
+and vice-versa.
+
+When @var{bv} is actually a SRFI-4 uniform vector, its element
+type is preserved unless @var{offset} and @var{size} are not aligned
+on its element type size.
+@end deffn
I wrote stuff here about 'What if offset or size unaligned?  This is 
undocumented.', and only later noticed this paragraph documenting 
exactly that.



+
+Here is an example showing how to use it:
+
+@lisp
+(use-modules (rnrs bytevectors)
+ (rnrs bytevectors gnu))


I thought that R6RS reserved (rnrs ...) to the RnRS process, so I would 
think that naming it (rnrs bytevectors gnu) would be 
standards-incompliant, though I cannot find in the specification, so 
maybe it isn't actually reserved.


(SRFI looks a bit looser to me, so I find the (srfi ... gnu) acceptable, 
but (rnrs ... gnu) looks weird to me, I would propose (ice-9 
bytevector-extensions) or such.).



+
diff --git a/doc/ref/guile.texi b/doc/ref/guile.texi
index 6a81a0893..8414c3e2d 100644
--- a/doc/ref/guile.texi
+++ b/doc/ref/guile.texi
@@ -13,7 +13,7 @@
  @copying
  This manual documents Guile version @value{VERSION}.
  
-Copyright (C) 1996-1997, 2000-2005, 2009-2021 Free Software Foundation,

+Copyright (C) 1996-1997, 2000-2005, 2009-2023 Free Software Foundation,



Where does this year 2022 come from?  Does a previous version of this 
patch predate the new year?




diff --git a/libguile/bytevectors.c b/libguile/bytevectors.c
index bbc23f449..6b920c88a 100644
--- a/libguile/bytevectors.c
+++ b/libguile/bytevectors.c
@@ -1,4 +1,4 @@
-/* Copyright 2009-2015,2018-2019
+/* Copyright 2009-2015,2018-2019,2022-2023
   Free Software Foundation, Inc.
  

Ditto.


 This file is part of Guile.
@@ -325,6 +325,73 @@ scm_c_take_typed_bytevector (signed char *contents, size_t 
len,
return ret;
  }
  
+SCM_DEFINE (scm_bytevector_slice, "bytevector-slice", 2, 1, 0,

+(SCM bv, SCM offset, SCM size),
+"Return the slice of @var{bv} starting at @var{offset} and 
counting\n"
+"@var{size} bytes.  When @var{size} is omitted, the slice covers 
all\n"
+"of @var{bv} starting from @var{offset}.  The returned slice 
shares\n"
+"storage with @var{bv}: changes to the slice are visible in 
@var{bv}\n"
+"and vice-versa.\n"
+"\n"
+"When @var{bv} is actually a SRFI-4 uniform vector, its element\n"
+"type is preserved unless @var{offset} and @var{size} are not 
aligned\n"
+"on its element type size.\n")
+#define FUNC_NAME s_scm_bytevector_slice
+{
+  SCM ret;
+  size_t c_offset, c_size;
+  scm_t_array_element_type element_type;
+
+  SCM_VALIDATE_BYTEVECTOR (1, bv);
+
+  /* FIXME: Until 3.0.8 included, the assembler would not set the
+ SCM_F_BYTEVECTOR_CONTIGUOUS flag on literals.  Thus, ignore it and
+ assume BV is contiguous (how could it not be anyway?).  */
+#if 0
+  if (!SCM_BYTEVECTOR_CONTIGUOUS_P (bv))
+scm_wrong_type_arg_msg (FUNC_NAME, 0, bv, "contiguous bytevector");
+#endif



I don't know what's up with that, I'm leaving this fragment of code for 
other to review.



+
+  c_offset = scm_to_size_t (offset);
+
+  if (SCM_UNBNDP (size))
+{
+  if (c_offset < SCM_BYTEVECTOR_LENGTH (bv))
+c_size = SCM_BYTEVECTOR_LENGTH (bv) - c_offset;
+  else
+c_size = 0; > +}
+  else
+c_size = scm_to_size_t (size);
+
+  if (c_offset + c_size > SCM_BYTEVECTOR_LENGTH (bv))
+scm_out_of_range (FUNC_NAME, offset);



If offset=SIZE_MAX-1 and size=1, this will overflow to 0 and hence not 
trigger the error reporting.  This bounds check needs to be rewritten, 
with corresponding additional tests.



+
+  /* Preserve the element type of BV, unless we're not slicing on type
+ boundaries.  */
+  element_type = SCM_BYTEVECTOR_ELEMENT_TYPE (bv);
+  if ((c_offset % SCM_BYTEVECTOR_TYPE_SIZE (bv) != 0)
+  || (c_size % SCM_BYTEVECTOR_TYPE_SIZE (bv) != 0))
+element_type = SCM_ARRAY_ELEMENT_TYPE_VU8;
+  else
+c_size /= (scm_i_array_element_type_sizes[element_type] / 8);


I was worried about the alignment 

Re: [PATCH] Avoid 'frame-local-ref' errors when printing backtrace.

[Ludovic Courtès]

Hi,

Andrew Whatson  skribis:

> commit 164bdce6acf53796cb96ef1930a89c6caf84bc39
> Author: Andrew Whatson 
> Date:   Wed Jan 11 14:04:32 2023 +1000
>
> Test for 'frame-local-ref' errors when printing backtrace.
> 
> This test reproduces the error from , and
> passes with the workaround which was merged in commit
> c7fa78fc751eb336bcfafbb5ac59c460ee2c5d7a.
> 
> * test-suite/tests/eval.test ("avoid frame-local-ref out of range"): New
> test.

Applied, thanks!

Ludo’.

PS: Please use ‘git format-patch’ so the patch can be directly consumed
by ‘git am’.

Re: [EXT] [PATCH] Add 'bytevector-slice'.

[lloda]

> On 11 Jan 2023, at 18:37, Thompson, David  wrote:
> 
> On Wed, Jan 11, 2023 at 12:34 PM Ludovic Courtès  wrote:
>> 
>> What could be convenient though is ‘bytevector-copy’ (no bang), which
>> would combine ‘make-bytevector’ + ‘bytevector-copy!’.
> 
> 'bytevector-copy' already exists, or do you mean some different
> implementation of it?
> 
> - Dave

The current bytevector-copy takes a single argument. I would extend it in the 
same way the srfi4 copy functions were extended recently in the following commit

http://git.savannah.gnu.org/gitweb/?p=guile.git;a=commitdiff;h=6be51f9bbf47692ee5747b2cac6b372df65de970
 


Regards

  Daniel

Re: The mysterious ‘SCM_F_BYTEVECTOR_CONTIGUOUS’

[lloda]

> On 11 Jan 2023, at 19:51, lloda  wrote:
> 
> Contiguity is a trivial check.

I mean if bytevectors even had a stride field.

Re: The mysterious ‘SCM_F_BYTEVECTOR_CONTIGUOUS’

[lloda]

I think this flag has no use and you should just remove all references to it 
and deprecate it. Contiguity is a trivial check.

We removed a similar flag in the array type not long ago.

Regards

  Daniel

The mysterious ‘SCM_F_BYTEVECTOR_CONTIGUOUS’

[Ludovic Courtès]

Ludovic Courtès  skribis:

> This is an updated version of the ‘bytevector-slice’ primitive I used in
> the linker/assembler patch series¹ that I think is ready to go.

While working on this, I noticed I might have to pay attention to
‘SCM_F_BYTEVECTOR_CONTIGUOUS’, as noted in the patch.

But it turns out that flag isn’t really used.  I found two places that
should add it and do not:

diff --git a/libguile/bytevectors.c b/libguile/bytevectors.c
index 6b920c88a..fd7fdad0b 100644
--- a/libguile/bytevectors.c
+++ b/libguile/bytevectors.c
@@ -274,7 +274,8 @@ make_bytevector_from_buffer (size_t len, void *contents,
 
   c_len = len * (scm_i_array_element_type_sizes[element_type] / 8);
 
-  SCM_SET_BYTEVECTOR_FLAGS (ret, element_type);
+  SCM_SET_BYTEVECTOR_FLAGS (ret,
+element_type | SCM_F_BYTEVECTOR_CONTIGUOUS);
   SCM_BYTEVECTOR_SET_LENGTH (ret, c_len);
   SCM_BYTEVECTOR_SET_CONTENTS (ret, contents);
   SCM_BYTEVECTOR_SET_PARENT (ret, SCM_BOOL_F);
diff --git a/module/system/repl/debug.scm b/module/system/repl/debug.scm
diff --git a/module/system/vm/assembler.scm b/module/system/vm/assembler.scm
index 165976363..61e0460ff 100644
--- a/module/system/vm/assembler.scm
+++ b/module/system/vm/assembler.scm
@@ -1857,8 +1857,9 @@ should be .data or .rodata), and return the resulting linker object.
   (define tc7-program #x45)
 
   (define tc7-bytevector #x4d)
-  ;; This flag is intended to be left-shifted by 7 bits.
+  ;; These flags are intended to be left-shifted by 7 bits.
   (define bytevector-immutable-flag #x200)
+  (define bytevector-contiguous-flag #x100)
 
   (define tc7-array #x5d)
 
@@ -2026,6 +2027,7 @@ should be .data or .rodata), and return the resulting linker object.
;; Bytevector immutable flag also shifted
;; left.
(ash (logior bytevector-immutable-flag
+bytevector-contiguous-flag
 (array-type-code obj))
 7)
   (case word-size

There are probably more.

Fundamentally, I’m not sure what this flag is supposed to mean.  AFAICS,
there’s no way to create a non-contiguous bytevector (or SRFI-4 vector).

This flag was added in 7ed54fd36d2e381aa46ef8a7d2fc13a6776b573a.  My
guess is that it was part of plan that wasn’t carried out in the end.

Andy, thoughts?  :-)

Thanks,
Ludo’.

Re: [EXT] Re: [PATCH] Add 'bytevector-slice'.

[Thompson, David]

On Wed, Jan 11, 2023 at 12:34 PM Ludovic Courtès  wrote:
>
> What could be convenient though is ‘bytevector-copy’ (no bang), which
> would combine ‘make-bytevector’ + ‘bytevector-copy!’.

'bytevector-copy' already exists, or do you mean some different
implementation of it?

- Dave

Re: [PATCH] Add 'bytevector-slice'.

[Ludovic Courtès]

Hi,

Jean Abou Samra  skribis:

> What do you think about making it more similar to substrings?
> There the 'substring' procedure makes a copy-on-write substring,
> and you have substring/shared if you really want shared mutation.
> Would something like this be meaningful / feasible / useful
> for bytevectors?

I’m not convinced there’s a need for copy-on-write bytevectors, and it
would be hard to implement, and to implement
efficiently—bytevector-mutating instructions would have to check whether
they’re accessing a CoW bytevector and DTRT.

What could be convenient though is ‘bytevector-copy’ (no bang), which
would combine ‘make-bytevector’ + ‘bytevector-copy!’.

Ludo’.

Re: [PATCH] Add 'bytevector-slice'.

[Jean Abou Samra]

Hi Ludovic,

Le 11/01/2023 à 16:00, Ludovic Courtès a écrit :

+@node Bytevector Slices
+@subsubsection Bytevector Slices
+
+@cindex subset, of a bytevector
+@cindex slice, of a bytevector
+@cindex slice, of a uniform vector
+As an extension to the R6RS specification, the @code{(rnrs bytevectors
+gnu)} module provides the @code{bytevector-slice} procedure, which
+returns a bytevector aliasing part of an existing bytevector.
+
+@deffn {Scheme Procedure} bytevector-slice @var{bv} @var{offset} [@var{size}]
+@deffnx {C Function} scm_bytevector_slice (@var{bv}, @var{offset}, @var{size})
+Return the slice of @var{bv} starting at @var{offset} and counting
+@var{size} bytes.  When @var{size} is omitted, the slice covers all
+of @var{bv} starting from @var{offset}.  The returned slice shares
+storage with @var{bv}: changes to the slice are visible in @var{bv}
+and vice-versa.




What do you think about making it more similar to substrings?
There the 'substring' procedure makes a copy-on-write substring,
and you have substring/shared if you really want shared mutation.
Would something like this be meaningful / feasible / useful
for bytevectors?

Best,
Jean




OpenPGP_signature
Description: OpenPGP digital signature

Re: [PATCH] Add 'bytevector-slice'.

[Thompson, David]

Hi Ludovic,

On Wed, Jan 11, 2023 at 10:00 AM Ludovic Courtès  wrote:
>
> +@node Bytevector Slices
> +@subsubsection Bytevector Slices
> +
> +@cindex subset, of a bytevector
> +@cindex slice, of a bytevector
> +@cindex slice, of a uniform vector
> +As an extension to the R6RS specification, the @code{(rnrs bytevectors
> +gnu)} module provides the @code{bytevector-slice} procedure, which
> +returns a bytevector aliasing part of an existing bytevector.
> +
> +@deffn {Scheme Procedure} bytevector-slice @var{bv} @var{offset} [@var{size}]
> +@deffnx {C Function} scm_bytevector_slice (@var{bv}, @var{offset}, 
> @var{size})
> +Return the slice of @var{bv} starting at @var{offset} and counting
> +@var{size} bytes.  When @var{size} is omitted, the slice covers all
> +of @var{bv} starting from @var{offset}.  The returned slice shares
> +storage with @var{bv}: changes to the slice are visible in @var{bv}
> +and vice-versa.

Just wanted to chime in to say that this is a really great new
feature! Bytevector slices will be very useful for performance
sensitive code that could benefit from using a pre-allocated memory
arena.

Really really great stuff! Thank you!

- Dave

[PATCH] Add 'bytevector-slice'.

[Ludovic Courtès]

* module/rnrs/bytevectors/gnu.scm: New file.
* am/bootstrap.am (SOURCES): Add it.
* libguile/bytevectors.c (scm_bytevector_slice): New function.
* libguile/bytevectors.h (scm_bytevector_slice): New declaration.
* test-suite/tests/bytevectors.test ("bytevector-slice"): New tests.
* doc/ref/api-data.texi (Bytevector Slices): New node.
---
 am/bootstrap.am   |  1 +
 doc/ref/api-data.texi | 46 -
 doc/ref/guile.texi|  2 +-
 libguile/bytevectors.c| 69 ++-
 libguile/bytevectors.h|  3 +-
 module/rnrs/bytevectors/gnu.scm   | 24 +++
 test-suite/tests/bytevectors.test | 53 +++-
 7 files changed, 193 insertions(+), 5 deletions(-)
 create mode 100644 module/rnrs/bytevectors/gnu.scm

Hello!

This is an updated version of the ‘bytevector-slice’ primitive I used in
the linker/assembler patch series¹ that I think is ready to go.

I went to some length to do something sensible wrt. element type of the
input, when the input is a SRFI-4 uniform vector.  The other option would
be to make the output a pure bytevector unconditionally, but I thought
it would be more consistent and useful to preserve the input element type
when possible (see tests with an f32vector).

If there are no objections, I’ll push it to ‘main’ in the coming days.

Thanks,
Ludo’.

¹ https://lists.gnu.org/archive/html/guile-devel/2023-01/msg00013.html

diff --git a/am/bootstrap.am b/am/bootstrap.am
index 0257d53dc..53ee68315 100644
--- a/am/bootstrap.am
+++ b/am/bootstrap.am
@@ -249,6 +249,7 @@ SOURCES =   \
   rnrs/arithmetic/fixnums.scm  \
   rnrs/arithmetic/flonums.scm  \
   rnrs/bytevectors.scm \
+  rnrs/bytevectors/gnu.scm \
   rnrs/io/simple.scm   \
   rnrs/io/ports.scm\
   rnrs/records/inspection.scm  \
diff --git a/doc/ref/api-data.texi b/doc/ref/api-data.texi
index 8658b9785..fe2d2af50 100644
--- a/doc/ref/api-data.texi
+++ b/doc/ref/api-data.texi
@@ -1,6 +1,6 @@
 @c -*-texinfo-*-
 @c This is part of the GNU Guile Reference Manual.
-@c Copyright (C)  1996, 1997, 2000-2004, 2006-2017, 2019-2020, 2022
+@c Copyright (C)  1996, 1997, 2000-2004, 2006-2017, 2019-2020, 2022-2023
 @c   Free Software Foundation, Inc.
 @c See the file guile.texi for copying conditions.
 
@@ -6673,6 +6673,7 @@ Bytevectors can be used with the binary input/output 
primitives
 * Bytevectors as Strings::  Interpreting bytes as Unicode strings.
 * Bytevectors as Arrays::   Guile extension to the bytevector API.
 * Bytevectors as Uniform Vectors::  Bytevectors and SRFI-4.
+* Bytevector Slices::   Aliases for parts of a bytevector.
 @end menu
 
 @node Bytevector Endianness
@@ -7108,6 +7109,49 @@ Bytevectors may also be accessed with the SRFI-4 API. 
@xref{SRFI-4 and
 Bytevectors}, for more information.
 
 
+@node Bytevector Slices
+@subsubsection Bytevector Slices
+
+@cindex subset, of a bytevector
+@cindex slice, of a bytevector
+@cindex slice, of a uniform vector
+As an extension to the R6RS specification, the @code{(rnrs bytevectors
+gnu)} module provides the @code{bytevector-slice} procedure, which
+returns a bytevector aliasing part of an existing bytevector.
+
+@deffn {Scheme Procedure} bytevector-slice @var{bv} @var{offset} [@var{size}]
+@deffnx {C Function} scm_bytevector_slice (@var{bv}, @var{offset}, @var{size})
+Return the slice of @var{bv} starting at @var{offset} and counting
+@var{size} bytes.  When @var{size} is omitted, the slice covers all
+of @var{bv} starting from @var{offset}.  The returned slice shares
+storage with @var{bv}: changes to the slice are visible in @var{bv}
+and vice-versa.
+
+When @var{bv} is actually a SRFI-4 uniform vector, its element
+type is preserved unless @var{offset} and @var{size} are not aligned
+on its element type size.
+@end deffn
+
+Here is an example showing how to use it:
+
+@lisp
+(use-modules (rnrs bytevectors)
+ (rnrs bytevectors gnu))
+
+(define bv (u8-list->bytevector (iota 10)))
+(define slice (bytevector-slice bv 2 3))
+
+slice
+@result{} #vu8(2 3 4)
+
+(bytevector-u8-set! slice 0 77)
+slice
+@result{} #vu8(77 3 4)
+
+bv
+@result{} #vu8(0 1 77 3 4 5 6 7 8 9)
+@end lisp
+
 @node Arrays
 @subsection Arrays
 @tpindex Arrays
diff --git a/doc/ref/guile.texi b/doc/ref/guile.texi
index 6a81a0893..8414c3e2d 100644
--- a/doc/ref/guile.texi
+++ b/doc/ref/guile.texi
@@ -13,7 +13,7 @@
 @copying
 This manual documents Guile version @value{VERSION}.
 
-Copyright (C) 1996-1997, 2000-2005, 2009-2021 Free Software Foundation,
+Copyright (C) 1996-1997, 2000-2005, 2009-2023 Free Software Foundation,
 Inc. @*
 Copyright (C) 2021 Maxime Devos
 
diff --git a/libguile/bytevectors.c b/libguile/bytevectors.c
index bbc23f449..6b920c88a 100644
--- a/libguile/bytevectors.c
+++ b/libguile/bytevectors.c
@@ -1,4 

problem compiling stable version of Guile

[Damien Mattei]

Hello,
i can build the *git* version of Guile on a system Ubuntu:
uname -a
Linux moita 5.15.0-56-generic #62-Ubuntu SMP Tue Nov 22 19:54:14 UTC 2022
x86_64 x86_64 x86_64 GNU/Linux
but i can not build the *stable* version:
configure without options is OK but when after make i got this error:

SNARF  regex-posix.doc
  GEN  guile-procedures.texi
Pre-boot error; key: misc-error, args: ("primitive-load-path" "Unable to
find file ~S in load path" ("ice-9/boot-9") #f)/bin/bash: line 1: 124776
Broken pipe cat alist.doc array-handle.doc array-map.doc
arrays.doc async.doc atomic.doc backtrace.doc boolean.doc bitvectors.doc
bytevectors.doc chars.doc control.doc continuations.doc debug.doc
deprecated.doc deprecation.doc dynl.doc dynwind.doc eq.doc error.doc
eval.doc evalext.doc exceptions.doc expand.doc extensions.doc
fdes-finalizers.doc feature.doc filesys.doc fluids.doc foreign.doc
fports.doc gc-malloc.doc gc.doc gettext.doc generalized-vectors.doc
goops.doc gsubr.doc guardians.doc hash.doc hashtab.doc hooks.doc i18n.doc
init.doc ioext.doc keywords.doc list.doc load.doc macros.doc mallocs.doc
memoize.doc modules.doc numbers.doc objprop.doc options.doc pairs.doc
ports.doc print.doc procprop.doc procs.doc promises.doc r6rs-ports.doc
random.doc rdelim.doc read.doc rw.doc scmsigs.doc script.doc simpos.doc
smob.doc sort.doc srcprop.doc srfi-1.doc srfi-4.doc srfi-13.doc srfi-14.doc
srfi-60.doc stackchk.doc stacks.doc stime.doc strings.doc strorder.doc
strports.doc struct.doc symbols.doc syntax.doc threads.doc throw.doc
unicode.doc uniform.doc values.doc variable.doc vectors.doc version.doc
vports.doc weak-set.doc weak-table.doc weak-vector.doc dynl.doc posix.doc
net_db.doc socket.doc regex-posix.doc
 124777 Aborted (core dumped) | GUILE_AUTO_COMPILE=0
../meta/build-env guild snarf-check-and-output-texi > guile-procedures.texi
make[3]: *** [Makefile:4560: guile-procedures.texi] Error 1
make[3]: Leaving directory
'/home/mattei/Téléchargements/guile-3.0.8/libguile'
make[2]: *** [Makefile:2632: all] Error 2
make[2]: Leaving directory
'/home/mattei/Téléchargements/guile-3.0.8/libguile'
make[1]: *** [Makefile:2061: all-recursive] Error 1
make[1]: Leaving directory '/home/mattei/Téléchargements/guile-3.0.8'
make: *** [Makefile:1947 : all] Erreur 2

seems the git version is more stable than the official stable one ;-)
Damien