Re: SRFI-143 Fixnums (was Generalized-Arrays egg v1.0.1)

[Jeremy Steward]

On 4/22/24 22:20, Diego wrote:


On Monday, April 22nd, 2024 at 11:56 PM, Jeremy Steward  
wrote:


Sorry to reply to myself, but I've provided some better patches now, see
attached.


Whoops, I got this in the middle of pushing the previous change. I'll try and 
apply the new patches. Thanks for this.

Diego


Thanks for applying those patches and pushing up v1.0.0.

I tested the re-exports locally and they seem to no longer cause 
performance problems on transducers or generalized-arrays, and new 
versions of those should be up shortly.


Cheers,
--
Jeremy Steward

Re: SRFI-143 Fixnums (was Generalized-Arrays egg v1.0.1)

[Diego]

On Monday, April 22nd, 2024 at 11:56 PM, Jeremy Steward  
wrote:

> Sorry to reply to myself, but I've provided some better patches now, see
> attached.

Whoops, I got this in the middle of pushing the previous change. I'll try and 
apply the new patches. Thanks for this.

Diego

Re: SRFI-143 Fixnums (was Generalized-Arrays egg v1.0.1)

[Diego]

Huh, looking through the log I guess I implemented the variadic operators, but
I don't remember why?

> Anyways, happy to continue to get this out to folks. I don't know to
> what extent users are currently using SRFI-143 so I'd be wary to push
> this out, but I went to the liberty of bumping the version to 1.0.0
> since this would definitely be a major breakage of the API.

I grepped through all the .egg files in eggs-5-latest and it seems only
SRFI-217 and integer-map depend on SRFI-143; they do not appear to make use of
the variadic operators. This is confirmed by passing tests even when building
these eggs with a patched SRFI-143.

> I don't have a SourceHut account, but from what I know of SourceHut I
> believe either approach can be made to work (I may need to set up
> git-send-email). For what it's worth, I've done the hard work of
> converting everything to an R7RS egg as well as fixing the tests / arity
> of every procedure to match that of (chicken fixnum) and what I presume
> to be John's original intent with the SRFI.

Thanks. I have applied your patch, tagged the commit, and pushed it.

I haven't actually been using sourcehut for that long relatively, so I'm not
sure what's required to give you access to the repo. At a glance it seems easy
to add other users by account name, but if there's another way I could look
into it (or if we'd simply want to move the egg somewhere else, fine by me).

I'll try to be at least a little more active on email (IRC, if I ever get
around to setting it up again) going forward to keep track of this and my other
eggs, should they come up.

Diego

Re: SRFI-143 Fixnums (was Generalized-Arrays egg v1.0.1)

[Jeremy Steward]

Sorry to reply to myself, but I've provided some better patches now, see 
attached.


I went ahead and fixed the Makefile and SVNWiki pages as well. I can't 
run test-new-egg but salmonella passes just fine.


Cheers,
--
Jeremy StewardFrom 0fe5cff9482bc6a11594d907128ddbcad562976c Mon Sep 17 00:00:00 2001
From: Jeremy Steward 
Date: Mon, 22 Apr 2024 21:54:21 -0600
Subject: [PATCH 3/3] Update svnwiki page with appropriate docs

---
 srfi-143.svnwiki | 43 +--
 1 file changed, 21 insertions(+), 22 deletions(-)

diff --git a/srfi-143.svnwiki b/srfi-143.svnwiki
index 7f326e9..fd4be26 100644
--- a/srfi-143.svnwiki
+++ b/srfi-143.svnwiki
@@ -52,11 +52,9 @@ in the same way as {{(chicken fixnum}}).
 
 === Deviations from the specification
 
-In the CHICKEN implementation, {{fx+}}, {{fx-}}, and {{fx*}} are variadic,
-implemented with {{foldr}}, {{foldl}}, and {{foldr}}, respectively. For
-completeness, a variadic {{fx/}} is provided as well, implemented with
-{{foldl}}. It's not clear that this is much more useful than the SRFI's
-{{fxquotient}}.
+All procedures in this library are 2-arity wherever possible. The original
+SRFI-143 document lists these procedures as variadic, but for compatibility
+with {{(chicken fixnum)}} these are kept to 2-arity and re-exported.
 
 === Specification
 
@@ -105,23 +103,23 @@ Bound to the value {{-2^w-1}}, the smallest representable fixnum. (R6RS
 Returns {{#t}} if obj is an exact integer within the fixnum range, and {{#f}}
 otherwise.
 
-(fx=? i ...)
+(fx=? i j)
 
 Semantically equivalent to {{=}}.
 
-(fx
+(fx
 
 Semantically equivalent to {{<}}.
 
-(fx>? i ...)
+(fx>? i j)
 
 Semantically equivalent to {{>}}.
 
-(fx<=? i ...)
+(fx<=? i j)
 
 Semantically equivalent to {{<=}}.
 
-(fx>=? i ...)
+(fx>=? i j)
 
 Semantically equivalent to {{>=}}.
 
@@ -145,21 +143,21 @@ Semantically equivalent to {{odd?}}.
 
 Semantically equivalent to {{even?}}.
 
-(fxmax i j ...)
+(fxmax i j)
 
 Semantically equivalent to {{max}}.
 
-(fxmin i j ...)
+(fxmin i j)
 
 Semantically equivalent to {{min}}.
 
  Basic arithmetic
 
-(fx+ i ...)
+(fx+ i j)
 
 Semantically equivalent to {{+}}.
 
-(fx- i ...)
+(fx- i j)
 
 Semantically equivalent to {{-}}.
 
@@ -167,11 +165,11 @@ Semantically equivalent to {{-}}.
 
 Semantically equivalent to {{-}}, but accepts exactly one argument.
 
-(fx* i ...)
+(fx* i j)
 
 Semantically equivalent to {{*}}.
 
-(fx/ i ...)
+(fx/ i j)
 
 Semantically equivalent to {{(foldr quotient i rest-args)}}.
 
@@ -243,15 +241,15 @@ are dropped for brevity and compatibility.
 
 Semantically equivalent to {{bitwise-not}}.
 
-(fxand i ...)
+(fxand i j)
 
 Semantically equivalent to {{bitwise-and}}.
 
-(fxior i ...)
+(fxior i j)
 
 Semantically equivalent to {{bitwise-ior}}.
 
-(fxxor i ...)
+(fxxor i j)
 
 Semantically equivalent to {{bitwise-xor}}.
 
@@ -331,17 +329,17 @@ Jaffer, Taylor Campbell, and the R6RS editors.
 === Copyright
 
  Copyright (C) John Cowan (2016). All Rights Reserved.
- 
+
  Permission is hereby granted, free of charge, to any person obtaining a copy of
  this software and associated documentation files (the "Software"), to deal in
  the Software without restriction, including without limitation the rights to
  use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies
  of the Software, and to permit persons to whom the Software is furnished to do
  so, subject to the following conditions:
- 
+
  The above copyright notice and this permission notice shall be included in all
  copies or substantial portions of the Software.
- 
+
  THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
  IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
  FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
@@ -352,6 +350,7 @@ Jaffer, Taylor Campbell, and the R6RS editors.
 
 === Version history
 
+* 1.0.0 - Convert egg to R7RS library & convert variadic procedures to 2-arity procedures.
 * 0.4.1 - Fix bug due to calling fixnum procedures with wrong number of arguments
 * 0.4 - Implement variadic arithmetic operators, use fixnum operations internally
 * 0.3 - Reorganize and reduce files
-- 
2.43.0

From c9e35def19b4f7c0753180ac395eb0f009b0ac60 Mon Sep 17 00:00:00 2001
From: Jeremy Steward 
Date: Mon, 22 Apr 2024 21:29:41 -0600
Subject: [PATCH 2/3] Add chicken-lint to Makefile / clean up makefile

Some of the commands were a bit ??? so I did a bit of cleanup on the
Makefile.
---
 Makefile | 36 
 1 file changed, 24 insertions(+), 12 deletions(-)

diff --git a/Makefile b/Makefile
index d613d59..7deea33 100644
--- a/Makefile
+++ b/Makefile
@@ -1,25 +1,37 @@
-CHICKEN_CLEAN		:= chicken-clean
-CHICKEN_INSTALL		:= chicken-install
-SALMONELLA		:= salmonella
-SALMONELLA_LOG		:= salmonella.log
-SALMONELLA_LOG_VIEWER	:= salmonella-log-viewer
-SRFI   			:= srfi-143.scm
-TEST_NEW_EGG		:= test-new-egg
+CHICKEN_CLEAN		  := chicken-clean

Re: SRFI-143 Fixnums (was Generalized-Arrays egg v1.0.1)

[Jeremy Steward]

On 4/22/24 14:55, Diego wrote:

Hi all,

Sorry to be very late/unresponsive to this discussion. As I understand it from 
reading the email thread, there are several optimizations and changes to 
SRFI-143 that would make it more performant or inline with the intent of the 
original SRFI.


Heyo! Better late than never!

I think there's two things being discussed:

1. SRFI-143 as it exists is rather slow compared to (chicken fixnum)
2. There seems to be an errata / post finalization note in SRFI-143 (the 
text) that needs to be brought up regarding 2-arity vs. variadic 
procedures in the spec. John mentioned he would pursue this, although 
looking at the archives I haven't seen it yet.



I have unfortunately not had much time for scheme in the past couple years, and 
though I'd love to get back to it eventually, I'd hate to simply sit on the 
sources for this (or any other eggs I maintain) without allowing for it to 
evolve.



I understand this feeling. Look at generalized-arrays (thread topic), I 
sat on it for years!



I'd be happy to give Jeremy and anyone else interested in this effort 
read/write access to the sources at https://git.sr.ht/~dieggsy/srfi-143, or 
transfer the egg to a different maintainer/repo entirely if that approach is 
preferred.

Diego



I don't have a SourceHut account, but from what I know of SourceHut I 
believe either approach can be made to work (I may need to set up 
git-send-email). For what it's worth, I've done the hard work of 
converting everything to an R7RS egg as well as fixing the tests / arity 
of every procedure to match that of (chicken fixnum) and what I presume 
to be John's original intent with the SRFI.


See the attached file for the full patch. One thing I did not do was 
update the svnwiki page in the repo. I can definitely do this but I only 
spent about an hour or so on this and some plugin in my editor is 
messing with the formatting. (classic excuse right?)


Anyways, happy to continue to get this out to folks. I don't know to 
what extent users are currently using SRFI-143 so I'd be wary to push 
this out, but I went to the liberty of bumping the version to 1.0.0 
since this would definitely be a major breakage of the API.


Cheers,
--
Jeremy StewardFrom d9d3a770afc54faf12558f59e37a33db7a016587 Mon Sep 17 00:00:00 2001
From: Jeremy Steward 
Date: Mon, 22 Apr 2024 20:56:03 -0600
Subject: [PATCH] Convert egg to R7RS egg & variadic procedures to 2-arity

---
 carries.scm   |  63 ---
 srfi-143-impl.scm |  98 -
 srfi-143.egg  |  12 ++-
 srfi-143.release-info |   1 +
 srfi-143.scm  | 246 +++---
 srfi-143.sld  |  83 ++
 tests/r6rs-test.scm   | 176 --
 tests/run.scm |  34 +++---
 8 files changed, 270 insertions(+), 443 deletions(-)
 delete mode 100644 carries.scm
 delete mode 100644 srfi-143-impl.scm
 create mode 100644 srfi-143.sld
 delete mode 100644 tests/r6rs-test.scm

diff --git a/carries.scm b/carries.scm
deleted file mode 100644
index 8c2253c..000
--- a/carries.scm
+++ /dev/null
@@ -1,63 +0,0 @@
- Generic implementation of carry functions from the R6RS standard.
-
-;;; These implementations of fx+/carry, fx-/carry, and fx*/carry
-;;; are very inefficient, and should be replaced by proper
-;;; assembly language operations if at all possible.
-;;; Furthermore, there are no tests for them,
-;;; because of their dependency on fx-width.
-
-(define exp-width (expt 2 fx-width))
-
-(define (fx+/carry i j k)
-  (let*-values (((s) (+ i j k))
-((q r) (balanced/ s exp-width)))
-(values r q)))
-
-(define (fx-/carry i j k)
-  (let*-values (((d) (- i j k))
-((q r) (balanced/ d exp-width)))
-(values r q)))
-
-(define (fx*/carry i j k)
-  (let*-values (((s) (+ (* i j) k))
-((q r) (balanced/ s exp-width)))
-(values r q)))
-
-;;; Helper functions from SRFI 151
-
-(define (floor-/+ n d)
-  (let ((n (- 0 n)))
-(let ((q (quotient n d)) (r (remainder n d)))
-  (if (zero? r)
-  (values (- 0 q) r)
-  (values (- (- 0 q) 1) (- d r))
-
-(define (ceiling-/- n d)
-  (let ((n (- 0 n)) (d (- 0 d)))
-(let ((q (quotient n d)) (r (remainder n d)))
-  (if (zero? r)
-  (values q r)
-  (values (+ q 1) (- d r))
-
-(define (euclidean/ n d)
-  (if (and (exact-integer? n) (exact-integer? d))
-  (cond ((and (negative? n) (negative? d)) (ceiling-/- n d))
-((negative? n) (floor-/+ n d))
-((negative? d)
- (let ((d (- 0 d)))
-   (values (- 0 (quotient n d)) (remainder n d
-(else (values (quotient n d) (remainder n d
-  (let ((q (if (negative? d) (ceiling (/ n d)) (floor (/ n d)
-(values q (- n (* d q))
-
-(define (balanced/ x y)
-  (call-with-values
-  (lambda () (euclidean/ x y))
-(lambda (q r)
-  (cond ((< r (abs (/ y 2)))
- 

Re: Generalized-Arrays egg v1.0.1

[Diego]

Hi all,

Sorry to be very late/unresponsive to this discussion. As I understand it from 
reading the email thread, there are several optimizations and changes to 
SRFI-143 that would make it more performant or inline with the intent of the 
original SRFI. 

I have unfortunately not had much time for scheme in the past couple years, and 
though I'd love to get back to it eventually, I'd hate to simply sit on the 
sources for this (or any other eggs I maintain) without allowing for it to 
evolve.

I'd be happy to give Jeremy and anyone else interested in this effort 
read/write access to the sources at https://git.sr.ht/~dieggsy/srfi-143, or 
transfer the egg to a different maintainer/repo entirely if that approach is 
preferred.

Diego

On Tuesday, February 6th, 2024 at 11:04 AM, felix.winkelm...@bevuta.com 
 wrote:

> > Aha, so as long as something is wrapped we don't see the same kind of
> > inlining.
> 
> 
> Correct.
> 
> > So what would this look like? I can see fx+ above, which uses
> > `chicken:fx+` (prefixed, 2-arity) and foldr to accomplish its multi-arg
> > case. Is adding `(inline-file)` to the srfi-143.egg enough for this? I
> > suspect partly that the foldr and `(fx+ . args)` form in general is
> > going to be hard to optimize out, but perhaps a case-lambda is faster?
> 
> 
> If we make the operations variadic, the compiler can always
> optimize (fx+ ...) into nested calls to dyadic lower-level primitives,
> but this applies only to optimizations that the compiler knows about.
> For user-defined inlinings, this is currently not possible.
> 
> > Thanks for the really informed reply Felix, this was very helpful in
> > understanding what's happening with SRFI-143. I can hopefully start
> > crafting a patch for this now.
> 
> 
> No problem, you are very welcome.
> 
> 
> cheers,
> felix

Re: Generalized-Arrays egg v1.0.1

[felix . winkelmann]

> Aha, so as long as something is wrapped we don't see the same kind of
> inlining.

Correct.


> So what would this look like? I can see fx+ above, which uses
> `chicken:fx+` (prefixed, 2-arity) and foldr to accomplish its multi-arg
> case. Is adding `(inline-file)` to the srfi-143.egg enough for this? I
> suspect partly that the foldr and `(fx+ . args)` form in general is
> going to be hard to optimize out, but perhaps a case-lambda is faster?

If we make the operations variadic, the compiler can always
optimize (fx+ ...) into nested calls to dyadic lower-level primitives,
but this applies only to optimizations that the compiler knows about.
For user-defined inlinings, this is currently not possible.

> Thanks for the really informed reply Felix, this was very helpful in
> understanding what's happening with SRFI-143. I can hopefully start
> crafting a patch for this now.

No problem, you are very welcome.


cheers,
felix

Re: Generalized-Arrays egg v1.0.1

[John Cowan]

I now understand that chicken's srfi-143 egg has a  different spec from
actual SRFI-143; in particular, the egg's procedures are variadic, whereas
the SRFI's procedures were intended not  to be.  Unfortunately, the SRFI
specifies =, <, etc. to be variadic instead of dyadic as a result of cut
and paste disease, although the SRFI sample implementations conform to my
original intention.  I'm going to get the SRFI fixed if possible, or if not
to issue a post-finalization recommendation (i.e. a recommendation by the
SRFI author (me) that only two arguments be supported).

On Mon, Feb 5, 2024 at 8:43 PM Jeremy Steward  wrote:

> On 2/4/24 05:11, felix.winkelm...@bevuta.com wrote:
> >
> >> 1. The egg itself is not compiled with -O3, whereas if I link to
> >> (chicken fixnum) I believe that these procedures will be inlined by the
> >> CHICKEN compiler when the arrays egg is compiled with -O3 or higher.
> >
> > That's correct. As -O3 implies "unsafe", the compiler can just ignore
> > type checks and always inline numeric primitives.
> >
>
> Awesome, glad my intuition about this wasn't unfounded. This is probably
> the biggest factor.
>
> >>
> >> 2. There may be some rewriting rules that the compiler uses for
> >> procedures in the (chicken ...) namespace that optimizes these directly
> >> into their equivalent optimized C procedures. I'm not sure the compiler
> >> has the same visibility if you re-export these from behind a module, and
> >> especially not if you link dynamically (any hope of inlining those is
> >> thus gone forever).
> >
> > As a general rule all renamings and rexports preserve the run-time
> > semantics - as long as a procedure definition is not wrapped inside
> > another procedure, the original procedure gets compiled and any
> > optimizations and inlinings the compiler performs for "known" primitives
> will be
> > applied. Syntax- and module-specific expansions perform renaming
> > only and never introduce run-time code that wasn't explicitly given by
> > the user.
> >
>
> Aha, so as long as something is wrapped we don't see the same kind of
> inlining.
>
> >>
> >> In fact on the latter point above, this raises an interesting question I
> >> had for the mailing list: if I re-export a CHICKEN-specific procedure,
> >> does the way the compiler handles translation units prevent certain
> >> optimizations from applying? I've somewhat noticed that to be the case
> >> but I haven't seen any writing about it nor have I understood the extent
> >> to which that might affect certain SRFIs or eggs.
> >
> > See above - unless something is wrapped, you get the inlining. But as
> > I see in the SRFI-143 spec, their operations take any number of
> > arguments, where the ones from (chicken fixnum) have arity 2,
> > so I assume the srfi code wraps the primitives into multi-argument
> > procedures and thus preserves imports from the srfi-143 module to be
> inlined.
> >
>
> And yup, that's probably it. From what I can tell from the source
> locally they are wrapped e.g.:
>
>  (define (fx+ . args)
>(foldr chicken:fx+ 0 args))
>
> >>
> >>> If you have time/energy, it would be useful to make Chicken's
> >>> implementation do what it was originally intended.  See the
> >>> "Implementation" section in the SRFI, or feel free to ask me for
> >>> explanations of details.
> >
> > One approach would be to extend those exports from (chicken fixnum)
> > that match the multi-argument operations from srfi-143 and make the
> > compiler aware of these, internally expanding the multi-arg cases into
> > nested applications of inline calls to the 2-argument C runtime primitive
> > ops.
> >
>
> So what would this look like? I can see fx+ above, which uses
> `chicken:fx+` (prefixed, 2-arity) and foldr to accomplish its multi-arg
> case. Is adding `(inline-file)` to the srfi-143.egg enough for this? I
> suspect partly that the foldr and `(fx+ . args)` form in general is
> going to be hard to optimize out, but perhaps a case-lambda is faster?
>
> I suppose in pretty much all cases since I'm using the 2-arity versions
> in the generalized-arrays library there's very little pushing me to use
> the SRFI-143 versions, but any improvement to SRFI-143 is probably
> worthwhile considering the kinds of performance differences I was
> measuring with chicken-profile.
>
> >
> >> It might also be a good idea to enforce SRFI-143's behaviour around
> >> overflow / underflow as part of CHICKEN's default behaviour in CHICKEN
> >> 6, but I don't really know how I would get started having that
> discussion.
> >
> > The SRFI-page doesn't seem to specify any behaviour. The runtime system
> > is compiled with "-fwrapv" in the hope to preserve wrapping semantics for
> > integer arithmetic, but I'm not sure how reliable that is.
> >
>
> Interesting! I believed the SRFI-143 was specifying twos-complement
> behaviour, but I now see that I was wrong. I would think in general that
> if -fwrapv is passed, we should assume that twos-complement is used.

Re: Generalized-Arrays egg v1.0.1

[Jeremy Steward]

On 2/4/24 05:11, felix.winkelm...@bevuta.com wrote:



1. The egg itself is not compiled with -O3, whereas if I link to
(chicken fixnum) I believe that these procedures will be inlined by the
CHICKEN compiler when the arrays egg is compiled with -O3 or higher.


That's correct. As -O3 implies "unsafe", the compiler can just ignore
type checks and always inline numeric primitives.



Awesome, glad my intuition about this wasn't unfounded. This is probably 
the biggest factor.




2. There may be some rewriting rules that the compiler uses for
procedures in the (chicken ...) namespace that optimizes these directly
into their equivalent optimized C procedures. I'm not sure the compiler
has the same visibility if you re-export these from behind a module, and
especially not if you link dynamically (any hope of inlining those is
thus gone forever).


As a general rule all renamings and rexports preserve the run-time
semantics - as long as a procedure definition is not wrapped inside
another procedure, the original procedure gets compiled and any
optimizations and inlinings the compiler performs for "known" primitives will be
applied. Syntax- and module-specific expansions perform renaming
only and never introduce run-time code that wasn't explicitly given by
the user.



Aha, so as long as something is wrapped we don't see the same kind of 
inlining.




In fact on the latter point above, this raises an interesting question I
had for the mailing list: if I re-export a CHICKEN-specific procedure,
does the way the compiler handles translation units prevent certain
optimizations from applying? I've somewhat noticed that to be the case
but I haven't seen any writing about it nor have I understood the extent
to which that might affect certain SRFIs or eggs.


See above - unless something is wrapped, you get the inlining. But as
I see in the SRFI-143 spec, their operations take any number of
arguments, where the ones from (chicken fixnum) have arity 2,
so I assume the srfi code wraps the primitives into multi-argument
procedures and thus preserves imports from the srfi-143 module to be inlined.



And yup, that's probably it. From what I can tell from the source 
locally they are wrapped e.g.:


(define (fx+ . args)
  (foldr chicken:fx+ 0 args))




If you have time/energy, it would be useful to make Chicken's
implementation do what it was originally intended.  See the
"Implementation" section in the SRFI, or feel free to ask me for
explanations of details.


One approach would be to extend those exports from (chicken fixnum)
that match the multi-argument operations from srfi-143 and make the
compiler aware of these, internally expanding the multi-arg cases into
nested applications of inline calls to the 2-argument C runtime primitive
ops.



So what would this look like? I can see fx+ above, which uses 
`chicken:fx+` (prefixed, 2-arity) and foldr to accomplish its multi-arg 
case. Is adding `(inline-file)` to the srfi-143.egg enough for this? I 
suspect partly that the foldr and `(fx+ . args)` form in general is 
going to be hard to optimize out, but perhaps a case-lambda is faster?


I suppose in pretty much all cases since I'm using the 2-arity versions 
in the generalized-arrays library there's very little pushing me to use 
the SRFI-143 versions, but any improvement to SRFI-143 is probably 
worthwhile considering the kinds of performance differences I was 
measuring with chicken-profile.





It might also be a good idea to enforce SRFI-143's behaviour around
overflow / underflow as part of CHICKEN's default behaviour in CHICKEN
6, but I don't really know how I would get started having that discussion.


The SRFI-page doesn't seem to specify any behaviour. The runtime system
is compiled with "-fwrapv" in the hope to preserve wrapping semantics for
integer arithmetic, but I'm not sure how reliable that is.



Interesting! I believed the SRFI-143 was specifying twos-complement 
behaviour, but I now see that I was wrong. I would think in general that 
if -fwrapv is passed, we should assume that twos-complement is used. 
Recent C standards have codified this explicitly, but pretty much every 
major compiler I've used over the last 10 years has all-but-standardized 
twos-complement as well.


I guess as long as the runtime system is always using -fwrapv it isn't a 
concern. It would be undefined behaviour without that flag though, which 
I suppose someone could maybe force, with great effort?


Either way, probably not worth worrying about.

Thanks for the really informed reply Felix, this was very helpful in 
understanding what's happening with SRFI-143. I can hopefully start 
crafting a patch for this now.


Cheers,
--
Jeremy Steward

Re: Generalized-Arrays egg v1.0.1

[felix . winkelmann]

> 1. The egg itself is not compiled with -O3, whereas if I link to
> (chicken fixnum) I believe that these procedures will be inlined by the
> CHICKEN compiler when the arrays egg is compiled with -O3 or higher.

That's correct. As -O3 implies "unsafe", the compiler can just ignore
type checks and always inline numeric primitives.

>
> 2. There may be some rewriting rules that the compiler uses for
> procedures in the (chicken ...) namespace that optimizes these directly
> into their equivalent optimized C procedures. I'm not sure the compiler
> has the same visibility if you re-export these from behind a module, and
> especially not if you link dynamically (any hope of inlining those is
> thus gone forever).

As a general rule all renamings and rexports preserve the run-time
semantics - as long as a procedure definition is not wrapped inside
another procedure, the original procedure gets compiled and any
optimizations and inlinings the compiler performs for "known" primitives will be
applied. Syntax- and module-specific expansions perform renaming
only and never introduce run-time code that wasn't explicitly given by
the user.

>
> In fact on the latter point above, this raises an interesting question I
> had for the mailing list: if I re-export a CHICKEN-specific procedure,
> does the way the compiler handles translation units prevent certain
> optimizations from applying? I've somewhat noticed that to be the case
> but I haven't seen any writing about it nor have I understood the extent
> to which that might affect certain SRFIs or eggs.

See above - unless something is wrapped, you get the inlining. But as
I see in the SRFI-143 spec, their operations take any number of
arguments, where the ones from (chicken fixnum) have arity 2,
so I assume the srfi code wraps the primitives into multi-argument
procedures and thus preserves imports from the srfi-143 module to be inlined.

>
> > If you have time/energy, it would be useful to make Chicken's
> > implementation do what it was originally intended.  See the
> > "Implementation" section in the SRFI, or feel free to ask me for
> > explanations of details.

One approach would be to extend those exports from (chicken fixnum)
that match the multi-argument operations from srfi-143 and make the
compiler aware of these, internally expanding the multi-arg cases into
nested applications of inline calls to the 2-argument C runtime primitive
ops.


> It might also be a good idea to enforce SRFI-143's behaviour around
> overflow / underflow as part of CHICKEN's default behaviour in CHICKEN
> 6, but I don't really know how I would get started having that discussion.

The SRFI-page doesn't seem to specify any behaviour. The runtime system
is compiled with "-fwrapv" in the hope to preserve wrapping semantics for
integer arithmetic, but I'm not sure how reliable that is.


cheers,
felix

Re: Generalized-Arrays egg v1.0.1

[Jeremy Steward]

On 2/3/24 16:03, John Cowan wrote:



On Sat, Feb 3, 2024 at 5:39 PM Jeremy Steward > wrote:


  Likewise, I opt to use (chicken fixnum) but this is
because I discovered that SRFI-143 is so egregiously slower than
(chicken fixnum) that it was causing serious performance regressions in
many of the array APIs.


That's very odd. The Chicken-specific library wrapper for SRFI 143 at 
GitHub (srfi-143.scm, which needs to be updated to Chicken 5) makes use 
of Chicken's native fx* procedures.  I don't know how the Chicken (srfi 
143) egg was built, though; it may be using rubber-chicken.scm and 
fxcore.scm, which are slow emulations meant for complete portability.


From what I understand on the SRFI page it is in fact re-exporting 
these procedures and not handling the overflow / underflow of some of 
the operations.


There's this note on overflow on the docs page:



Differences from the intended implementation details aside, I think the 
core of what was a performance hazard (at least as far as I looked into 
it) could have been one of two things (maybe both?):


1. The egg itself is not compiled with -O3, whereas if I link to 
(chicken fixnum) I believe that these procedures will be inlined by the 
CHICKEN compiler when the arrays egg is compiled with -O3 or higher.


2. There may be some rewriting rules that the compiler uses for 
procedures in the (chicken ...) namespace that optimizes these directly 
into their equivalent optimized C procedures. I'm not sure the compiler 
has the same visibility if you re-export these from behind a module, and 
especially not if you link dynamically (any hope of inlining those is 
thus gone forever).


In fact on the latter point above, this raises an interesting question I 
had for the mailing list: if I re-export a CHICKEN-specific procedure, 
does the way the compiler handles translation units prevent certain 
optimizations from applying? I've somewhat noticed that to be the case 
but I haven't seen any writing about it nor have I understood the extent 
to which that might affect certain SRFIs or eggs.


If you have time/energy, it would be useful to make Chicken's 
implementation do what it was originally intended.  See the 
"Implementation" section in the SRFI, or feel free to ask me for 
explanations of details.


I might have the time/energy for this, so I'll let you know if I stumble 
into anything soon. I think a CHICKEN-specific implementation probably 
wants to use the FFI since fixnums are guaranteed in their representation:




It might also be a good idea to enforce SRFI-143's behaviour around 
overflow / underflow as part of CHICKEN's default behaviour in CHICKEN 
6, but I don't really know how I would get started having that discussion.


In any case, I've cc'd Diego (SRFI-143 maintainer), maybe together we 
can find a way to improve the situation :)


Regards,
--
Jeremy Steward

Re: Generalized-Arrays egg v1.0.1

[John Cowan]

On Sat, Feb 3, 2024 at 5:39 PM Jeremy Steward  wrote:

 Likewise, I opt to use (chicken fixnum) but this is
> because I discovered that SRFI-143 is so egregiously slower than
> (chicken fixnum) that it was causing serious performance regressions in
> many of the array APIs.
>

That's very odd. The Chicken-specific library wrapper for SRFI 143 at
GitHub (srfi-143.scm, which needs to be updated to Chicken 5) makes use of
Chicken's native fx* procedures.  I don't know how the Chicken (srfi 143)
egg was built, though; it may be using rubber-chicken.scm and fxcore.scm,
which are slow emulations meant for complete portability.

If you have time/energy, it would be useful to make Chicken's
implementation do what it was originally intended.  See the
"Implementation" section in the SRFI, or feel free to ask me for
explanations of details.

Generalized-Arrays egg v1.0.1

[Jeremy Steward]

Hi all!

After about ~6 months of work, I've finally rewritten the 
generalized-arrays egg and am today releasing v1.0.1. You can find the 
repo at:




I learned a lot from this rewrite, but here's a few highlights:

1. The egg is MUCH faster now, thanks to generous use of chicken-profile 
and the statistical profiler. I've left -d2 on the egg so that others 
can profile as well.


2. The egg now heavily relies on my transducers egg. This is 
intentional, and is part of why the egg was able to be made so much 
faster than it previously was. While one could use generalized-arrays 
without transducers, it is certainly easier to leverage transducers as 
much as possible.


3. The egg now has tests and even runs these tests in GitLab CI. 
Previously, this was not true, and many APIs were just broken or 
incoherent in ways.


4. Every API is documented and has examples to demonstrate how to use it.

5. The library is now an R7RS library. The main dependencies that aren't 
R7RS (that are used) are read-syntax extensions, as well as the 
check-errors egg. Likewise, I opt to use (chicken fixnum) but this is 
because I discovered that SRFI-143 is so egregiously slower than 
(chicken fixnum) that it was causing serious performance regressions in 
many of the array APIs.


There were a ton of smaller lessons about CHICKEN, profiling, and 
thinking about array APIs in general in there as well; however, I might 
save those for a future blog post.


In any case, if you've used this egg at any point in the past and been 
disappointed, I hope you'll give it a second chance. I've even added 
some cool examples of how to use this with BLAS in the docs.


~~~

A note to Mario: since this was a rewrite, I reran test-new-egg and 
noticed that my URI in the release-info file was wrong. This was 
probably some artefact of GitLab changing their URI scheme, but it 
should be working now (I have run this with test-new-egg).


Beyond that, the egg is now CHICKEN-5 only. If it's possible, can we cut 
CHICKEN-4 support from 1.0.0 onwards? If not, let's cut out CHICKEN-4 
support entirely, since pre-1.0 versions of this egg should probably not 
be used, ever.


~~~

Thanks, and happy hacking! I hope this provides a better arrays API than 
I had previously shipped. I am definitely more proud of my egg now.

--
Jeremy Steward