[Chicken-users] Need help to figure out where this strange performance impact is coming from

[Jörg F . Wittenberger]

Hi Chickeneers,

yesterday I found that simply having a (use mailbox) in some code had a
huge impact (more than a factor of 3) at the performance of the
resulting executable.  Without using the mailbox stuff at all.

Meanwhile I figured out that this has nothing at all to do with the
mailbox egg.  But _all_ with the use of srfi-1.

But how is this possible?

Attached some test code.  As I ran into it from mailbox I prepared a
stripped down version of it to play with.  Towards the end of the file
there is a (use srfi-1) which makes all the difference.

So far I found (compiling the code with -O5 but similar things happen
with less aggressive optimization):

a) Using chicken 4.9.1 there is absolutely no difference.  Using srfi-1
or not I get roughly 100 messages passed per ms on my machine.  (But I
have to include the forced gc; see comment in the code.)

b) A slightly different version which avoids allocations in the queue
runs without the forced gc and yields about 160 ms^-1 on chicken 4.9.1
Again no difference whether or not I (use srfi-1) anywhere.

Now the interesting bits:

c) On master (built almost two weeks ago) I get - when (use srfi-1) is
included - about 180 ms^-1.  Those ~5% faster sound about right to me.

d) Comment out the (use srfi-1) at line 163 and it goes down to about
_50_ per millisecond!

e) The same happens for the alternative, allocation free version (not
attached), which uses vectors instead of pairs.


Speculating: The code I wrote has nothing to do with the difference.

But I'm confused.  Neither scheduler.scm nor srfi-18 seem to have any
dependency on srfi-1.  Also srfi-1 seems not to overwrite any global
bindings.

Should we simply always (use srfi-1) if we also (use srfi-18).  Looks
like a workaround, but not like the right thing to do.

How could I boil this down to the real reason?

Best

/Jörg
(declare
 ;;(unit mailbox)
 ;; requirements
 (disable-interrupts)
 ;; promises
 (strict-types)
 (usual-integrations)
 (no-procedure-checks-for-usual-bindings)
 (inline)
 (local)

 (no-bound-checks)
 (no-procedure-checks-for-usual-bindings)
 (bound-to-procedure
  ##sys#schedule
  ##sys#current-exception-handler)

  (always-bound
##sys#current-thread)
 )

(module
 mailbox3
 (
  make-mailbox
  mailbox?
  mailbox-empty?
  mailbox-send!
  mailbox-receive!
  )

(import scheme extras srfi-18)
(import (except chicken add1 sub1))

(: mailbox? (* --> boolean : (struct )))
(define-record-type 
  (internal-make-mailbox condition head tail pred)
  mailbox?
  (condition %mailbox-condition)
  (head %mailbox-head %mailbox-head-set!)
  (tail %mailbox-tail %mailbox-tail-set!)
  (pred mailbox-predicate))

(cond-expand
 (never
  (define-inline (mailbox-condition mb) (%mailbox-condition mb))
  (define-inline (mailbox-head mb) (%mailbox-head mb))
  (define-inline (mailbox-head-set! mb v) (%mailbox-head-set! mb v))
  (define-inline (mailbox-tail mb) (%mailbox-tail mb))
  (define-inline (mailbox-tail-set! mb v) (%mailbox-tail-set! mb v)))
 (else
  (define-inline (mailbox-condition mb) (##sys#slot mb 1))
  (define-inline (mailbox-head mb) (##sys#slot mb 2))
  (define-inline (mailbox-head-set! mb v) (##sys#setslot mb 2 v))
  (define-inline (mailbox-tail mb) (##sys#slot mb 3))
  (define-inline (mailbox-tail-set! mb v) (##sys#setslot mb 3 v))
  ))

(: make-mailbox (* --> (struct )))
(define (make-mailbox name)
  (let ((x (cons #f '(
(internal-make-mailbox
 (make-condition-variable name) x x #f)))

(: mailbox-empty? ((struct ) --> boolean))
(define (mailbox-empty? mb)
  (null? (cdr (mailbox-head mb

(: mailbox-number-of-items ((struct ) -> fixnum))
(define (mailbox-number-of-items mb)
  (length (cdr (mailbox-head mb

(define-inline (%dequeue-message mb)
  (mailbox-head-set! mb (##sys#slot #;cdr (mailbox-head mb) 1))
  (##sys#slot #;car (mailbox-head mb) 0))

(: mailbox-send! ((struct ) * -> undefined))
(define (mailbox-send! mailbox obj)
  (let ((p (cons obj '(
(##sys#setslot #;set-cdr! (mailbox-tail mailbox) 1 p)
(mailbox-tail-set! mailbox p))
  #;(condition-variable-signal! (mailbox-condition mailbox))
  (let ((x (mailbox-condition mailbox))) (if (thread? x) (thread-resume! x)))
  )

(: mailbox-receive! ((struct ) -> *))
(define (mailbox-receive! mailbox)
  ;;(dbg "receive-message! ~a ~a ~a" (current-thread) (mailbox-name mailbox) (##sys#slot (mailbox-condition mailbox) 2))
  (let loop ()
(if (null? (##sys#slot (mailbox-head mailbox) 1))
	(begin
	  (##sys#setslot mailbox 1 ##sys#current-thread)
	  (thread-suspend! ##sys#current-thread)
	  
	  (##sys#setslot mailbox 1 #f)
	  (loop))
	(let ((obj (%dequeue-message mailbox)))
	  obj

) ;; module mailbox


(module
 test
 (test-run)
 (import scheme chicken srfi-18 extras)
 (import mailbox3)
 
;;(include "bench.scm")
;;--

(define mb (make-mailbox 'm0))

(define turns 100)

(define tw
  (make-thread
   (lambda ()
 (do ((i 0 (add1 i)))
	 ((= i turns))
   (mailbox-send! mb i)
   ;; Must be active for my 

Re: [Chicken-users] Need help to figure out where this strange performance impact is coming from

[Christian Kellermann]

* Christian Kellermann  [160113 21:44]:
> * Jörg F. Wittenberger  [160113 12:38]:
> > yesterday I found that simply having a (use mailbox) in some code had a
> > huge impact (more than a factor of 3) at the performance of the
> > resulting executable.  Without using the mailbox stuff at all.
> >
> > Meanwhile I figured out that this has nothing at all to do with the
> > mailbox egg.  But _all_ with the use of srfi-1.
>
> Hm which OS and architecture are you running this on? On my OpenBSD
> amd64 system the two versions do fluctuate but are indistinguishable
> when run a couple hundred times.
>
> I have used CHICKEN Version 4.10.1 (rev f36c19c) and compiled with
> default options.

Also -O5 does not make any difference.

--
May you be peaceful, may you live in safety, may you be free from
suffering, and may you live with ease.

___
Chicken-users mailing list
Chicken-users@nongnu.org
https://lists.nongnu.org/mailman/listinfo/chicken-users

Re: [Chicken-users] Need help to figure out where this strange performance impact is coming from

[Christian Kellermann]

* Jörg F. Wittenberger  [160113 12:38]:
> yesterday I found that simply having a (use mailbox) in some code had a
> huge impact (more than a factor of 3) at the performance of the
> resulting executable.  Without using the mailbox stuff at all.
>
> Meanwhile I figured out that this has nothing at all to do with the
> mailbox egg.  But _all_ with the use of srfi-1.

Hm which OS and architecture are you running this on? On my OpenBSD
amd64 system the two versions do fluctuate but are indistinguishable
when run a couple hundred times.

I have used CHICKEN Version 4.10.1 (rev f36c19c) and compiled with
default options.

Cheers,

Christian

--
May you be peaceful, may you live in safety, may you be free from
suffering, and may you live with ease.

___
Chicken-users mailing list
Chicken-users@nongnu.org
https://lists.nongnu.org/mailman/listinfo/chicken-users

Re: [Chicken-users] Need help to figure out where this strange performance impact is coming from

[Dan Leslie]

IIRC, there's been ongoing efforts to remove SRFI-1 from core; which may 
explain your observations regarding Master.

Perhaps you should consider asking Chicken Hackers?

-Dan

Sent from my BlackBerry 10 smartphone.
  Original Message  
From: Jörg F. Wittenberger
Sent: Wednesday, January 13, 2016 3:38 AM
To: chicken-users
Subject: [Chicken-users] Need help to figure out where this strange performance 
impact is coming from

Hi Chickeneers,

yesterday I found that simply having a (use mailbox) in some code had a
huge impact (more than a factor of 3) at the performance of the
resulting executable. Without using the mailbox stuff at all.

Meanwhile I figured out that this has nothing at all to do with the
mailbox egg. But _all_ with the use of srfi-1.

But how is this possible?

Attached some test code. As I ran into it from mailbox I prepared a
stripped down version of it to play with. Towards the end of the file
there is a (use srfi-1) which makes all the difference.

So far I found (compiling the code with -O5 but similar things happen
with less aggressive optimization):

a) Using chicken 4.9.1 there is absolutely no difference. Using srfi-1
or not I get roughly 100 messages passed per ms on my machine. (But I
have to include the forced gc; see comment in the code.)

b) A slightly different version which avoids allocations in the queue
runs without the forced gc and yields about 160 ms^-1 on chicken 4.9.1
Again no difference whether or not I (use srfi-1) anywhere.

Now the interesting bits:

c) On master (built almost two weeks ago) I get - when (use srfi-1) is
included - about 180 ms^-1. Those ~5% faster sound about right to me.

d) Comment out the (use srfi-1) at line 163 and it goes down to about
_50_ per millisecond!

e) The same happens for the alternative, allocation free version (not
attached), which uses vectors instead of pairs.


Speculating: The code I wrote has nothing to do with the difference.

But I'm confused. Neither scheduler.scm nor srfi-18 seem to have any
dependency on srfi-1. Also srfi-1 seems not to overwrite any global
bindings.

Should we simply always (use srfi-1) if we also (use srfi-18). Looks
like a workaround, but not like the right thing to do.

How could I boil this down to the real reason?

Best

/Jörg

___
Chicken-users mailing list
Chicken-users@nongnu.org
https://lists.nongnu.org/mailman/listinfo/chicken-users

Re: [Chicken-users] Need help to figure out where this strange performance impact is coming from

[Jörg F . Wittenberger]

Hi Dan & chicken-hackers,

thanks for your reply.  I'm really curious how to explain these findings.

Complying with your suggestion, I'm sending this time to chicken-hackers
this time too.  I've been hoping enough of them would read chicken-users
too.

For reference to those who missed the initial posting, the message (with
attachment) is here:
http://lists.nongnu.org/archive/html/chicken-users/2016-01/msg00033.html

Am 13.01.2016 um 18:32 schrieb Dan Leslie:
> IIRC, there's been ongoing efforts to remove SRFI-1 from core; which may 
> explain your observations regarding Master.

I'm aware of that and checked the relevant git history before I posted.
 After all that's been my first guess too.  But my digging did not yield
any hints.  Looks like this is going on in it's own branch.

Another wild guess of mine was "maybe specializations?".  That's why the
(use srfi-1) is close to the end of the file.  (Even though I'm not
completely sure that's enough.)

> Perhaps you should consider asking Chicken Hackers?
> 
> -Dan
> 
>   Original Message  
> From: Jörg F. Wittenberger
> Sent: Wednesday, January 13, 2016 3:38 AM
> To: chicken-users
> Subject: [Chicken-users] Need help to figure out where this strange 
> performance impact is coming from
> 
> Hi Chickeneers,
> 
> yesterday I found that simply having a (use mailbox) in some code had a
> huge impact (more than a factor of 3) at the performance of the
> resulting executable. Without using the mailbox stuff at all.
> 
> Meanwhile I figured out that this has nothing at all to do with the
> mailbox egg. But _all_ with the use of srfi-1.
> 
> But how is this possible?
> 
> Attached some test code. As I ran into it from mailbox I prepared a
> stripped down version of it to play with. Towards the end of the file
> there is a (use srfi-1) which makes all the difference.
> 
> So far I found (compiling the code with -O5 but similar things happen
> with less aggressive optimization):
> 
> a) Using chicken 4.9.1 there is absolutely no difference. Using srfi-1
> or not I get roughly 100 messages passed per ms on my machine. (But I
> have to include the forced gc; see comment in the code.)
> 
> b) A slightly different version which avoids allocations in the queue
> runs without the forced gc and yields about 160 ms^-1 on chicken 4.9.1
> Again no difference whether or not I (use srfi-1) anywhere.
> 
> Now the interesting bits:
> 
> c) On master (built almost two weeks ago) I get - when (use srfi-1) is
> included - about 180 ms^-1. Those ~5% faster sound about right to me.
> 
> d) Comment out the (use srfi-1) at line 163 and it goes down to about
> _50_ per millisecond!
> 
> e) The same happens for the alternative, allocation free version (not
> attached), which uses vectors instead of pairs.
> 
> 
> Speculating: The code I wrote has nothing to do with the difference.
> 
> But I'm confused. Neither scheduler.scm nor srfi-18 seem to have any
> dependency on srfi-1. Also srfi-1 seems not to overwrite any global
> bindings.
> 
> Should we simply always (use srfi-1) if we also (use srfi-18). Looks
> like a workaround, but not like the right thing to do.
> 
> How could I boil this down to the real reason?
> 
> Best
> 
> /Jörg
> 


___
Chicken-users mailing list
Chicken-users@nongnu.org
https://lists.nongnu.org/mailman/listinfo/chicken-users