Re: the end of the [racket-users] mailing list and the migration to Discourse as a forum for Racket

[Dominik Pantůček]

Hello Racketeers,

I wouldn't take it as passive-aggressive per-se. It's just super 
confusing. Especially if you watch the discussion about future of Racket 
at RacketCon where Matthias has stated it pretty clearly and other 
confirmed that - the core team is doing all infrastructure maintenance 
more-or-less in their free time because their primary focus is 
development (of various parts of Racket ecosystem).


I am trying to use Discourse like ML and although it's got some really 
rough edges, it kinda works. If the new topic creation via mail gets 
resolved, it will be basically OK for me. My impression is that some of 
its features could be called "NNTP-like" and yes, I would really like to 
see NNTP interface (and then importing all those years of MLs into the 
archives would make much more sense then).


But as others have pointed out, maintaining infrastructure such as this 
requires non-negligible amount of work! So please take this as my 
personal opinion and not as an - even the slightest - attempt of 
demanding anything! I run some infrastructure and I know it needs time 
(all the time).



And thanks to John and Stephen for taking care of the Discourse instance!


Cheers,
Dominik

On 23. 11. 21 9:12, Dexter Lagan wrote:
   This is by far the strangest message I've ever read about Racket. It 
comes a close second to that all-caps spam we get constantly on the 
mailing list! I'd pay good money to know what's written between the 
lines. It sounds like really, really passive-aggressive.


Dex

On Monday, November 22, 2021 at 3:55:08 PM UTC+1 johnbclements wrote:

I’m … super confused by this message. Did I miss something? I feel
like this message has a subtext that I’m completely missing.

John

 > On Nov 22, 2021, at 08:06, Etan Wexler  wrote:
 >
 > The stewards of Racket have decided that it’s time to give up on
the mailing list (that is, racket-users, to which this message is a
contribution). The stewards of Racket have designated another forum
for Racket as the successor to racket-users. This other forum for
discussing Racket has as its basis the software named “Discourse”,
which is open‐source. Enlisting is prerequisite to contributing to
the Discursive forum for Racket. Enlisting is prerequisite to
receiving as Internet mail the contributions to the Discursive forum
for Racket. The stewards of Racket have published a facility for
enlisting as a participant in the Discursive forum for Racket.
 >

--
You received this message because you are subscribed to the Google 
Groups "Racket Users" group.
To unsubscribe from this group and stop receiving emails from it, send 
an email to racket-users+unsubscr...@googlegroups.com 
.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/racket-users/5c08b767-e837-4af4-89e0-7f859d4fc454n%40googlegroups.com 
.


--
You received this message because you are subscribed to the Google Groups "Racket 
Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to racket-users+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/racket-users/2169bc1e-7871-46bf-cbae-24adbd683c30%40trustica.cz.

[racket-users] srfi/19 implementation in Racket's srfi-lib

[Dominik Pantůček]

Hello Racketeers,

the ~S formatter [1] in srfi/19's date->string expects struct with 
nanoseconds, which means it is impossible to use it with plain date 
structs (only date*).


Is this required by how SRFI 19 should work or can I submit a patch to 
make it optional? Format strings work up to minute precision with struct 
date and for second precision date* or srfi/19 "lax-date" / srfi:date is 
required.



Cheers,
Dominik

[1] 
https://github.com/racket/srfi/blob/master/srfi-lite-lib/srfi/19/time.rkt#L1195


--
You received this message because you are subscribed to the Google Groups "Racket 
Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to racket-users+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/racket-users/29a31cf7-01dc-f023-1ad7-c8055dce5030%40trustica.cz.

Re: [racket-users] future question

[Dominik Pantůček]

Hi,

you are looking for make-fsempahore[1] and related:

===
#lang racket

(define sema (make-fsemaphore 1))

(define (make-th i)

(λ ()
  (let loop ((a 0))
(when (zero? (modulo a #e1e6))
(fsemaphore-wait sema)
"do what you have to do without disturbing other futures"
(fsemaphore-post sema))
   (loop (add1 a)
(for ((i (in-range (processor-count (future (make-th i)))
(sleep 10)
(exit)
===

The above updated source spins all your cores just fine.


Cheers,
Dominik
[1] 
https://docs.racket-lang.org/reference/futures.html#%28part._.Future_.Semaphores%29



On 12. 11. 21 18:44, Jacob Jozef wrote:

#lang racket

(define sema (make-semaphore 1))

(define (make-th i)

(λ ()

   (let loop ((a 0))

#;(when (zero? (modulo a #e1e6))

     (semaphore-wait sema)

     "do what you have to do without disturbing other futures"

     (semaphore-post sema))

    (loop (add1 a)

(for ((i (in-range (processor-count (future (make-th i)))

(sleep 10)

(exit)



--
You received this message because you are subscribed to the Google Groups "Racket 
Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to racket-users+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/racket-users/cc8c5754-9354-6367-7c62-b8c12cb60cd4%40trustica.cz.

Re: [racket-users] Racket meeting tomorrow?

[Dominik Pantůček]

I truly hope that the standard time (10pm CEST, 8pm UTC) and place 
(gather.town) tomorrow. Although I missed a few as I was busy with other 
projects over the summer, I'd like to see what others are up to again ;-)



See ya there!
Dominik

On 01. 10. 21 18:49, Hendrik Boom wrote:

Is there going to be a Racket online meeting at the gather site tomorrow
on October 2?  There have been on previous
first-Saturdays-in-the-month.If so, at what time, and what's the URL?

-- hendrik



--
You received this message because you are subscribed to the Google Groups "Racket 
Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to racket-users+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/racket-users/15f87298-9576-cd67-d087-6188f66a5d01%40trustica.cz.

Re: [racket-users] Registered on irc.libera.chat

[Dominik Pantůček]

Hello,

I am always advocating for being cautious - rather than rushing into
quick decisions (which may prove wrong later on). However, given known
facts, the situation with Freenode won't probably change for any better
in the foreseeable future.

I'd like to hear more opinions though before I (or anyone else) change
the topic on Freenode to include information about "official" (whatever
that means) channel being moved to libera.chat.

I am personally in favor of that.


Cheers,
Dominik

On 22. 05. 21 15:13, Tony Garnock-Jones wrote:
> Hi all,
> 
> I just registered the Racket project on irc.libera.chat with ownership
> of #racket and #racket-*.
> 
> Would those who have served as chanops on freenode who are willing to
> also do the same for libera.chat please email me privately to let me
> know your nick etc.?
> 
> Thanks!
> 
> Tony
> 
> -- 
> You received this message because you are subscribed to the Google
> Groups "Racket Users" group.
> To unsubscribe from this group and stop receiving emails from it, send
> an email to racket-users+unsubscr...@googlegroups.com
> .
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/racket-users/a3b34f89-10c9-4f14-915f-9622ca690e41n%40googlegroups.com
> .

-- 
You received this message because you are subscribed to the Google Groups 
"Racket Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to racket-users+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/racket-users/2a0f4cf0-00e7-2569-a0d4-2f3cf4c5fbca%40trustica.cz.

Re: [racket-users] Typed Racket: type relations

[Dominik Pantůček]

>> Of course Ben's hint at unsafe-reprovide is something I'd like to
>> investigate.
> 
> My hint turned into a PR and merged a few years ago.
> https://github.com/racket/typed-racket/pull/657
> 

I read that one yesterday - it does not address the issue fully. I meant
the hint at really implementing full (unsafe-reprovide ...) form.

> Here's the output I see from `typed-performance.rkt` . I'm on a commit
> from Jan 20. Does this look more like you'd expect?
> 
> ```
> ---
> typed: cpu time: 2990 real time: 3023 gc time: 65
> unsafe: cpu time: 405 real time: 408 gc time: 5
> typed, unsafe provided: cpu time: 692 real time: 697 gc time: 10
> plain: cpu time: 748 real time: 752 gc time: 3
> ---
> typed: cpu time: 2817 real time: 2825 gc time: 11
> unsafe: cpu time: 414 real time: 417 gc time: 3
> typed, unsafe provided: cpu time: 671 real time: 675 gc time: 4
> plain: cpu time: 750 real time: 754 gc time: 5
> ===
> ---
> typed: cpu time: 1678 real time: 1684 gc time: 0
> unsafe: cpu time: 161 real time: 165 gc time: 0
> typed, unsafe provided: cpu time: 175 real time: 178 gc time: 0
> plain: cpu time: 180 real time: 184 gc time: 0
> ---
> typed: cpu time: 1744 real time: 1755 gc time: 0
> unsafe: cpu time: 165 real time: 168 gc time: 0
> typed, unsafe provided: cpu time: 164 real time: 167 gc time: 0
> plain: cpu time: 195 real time: 198 gc time: 0
> ```
> 

No, this is still "wrong".

I experimented a bit with re-providing renamed / copied bindings as
discussed in those two issues and the results are definitely much
better. Almost perfect:


```
===rgb-average
---
typed: cpu time: 62 real time: 62 gc time: 10
typed-orig: cpu time: 1275 real time: 1275 gc time: 9
unsafe: cpu time: 50 real time: 50 gc time: 0
typed, unsafe provided: cpu time: 271 real time: 271 gc time: 0
plain: cpu time: 416 real time: 416 gc time: 0
---
typed: cpu time: 54 real time: 54 gc time: 1
typed-orig: cpu time: 1273 real time: 1273 gc time: 2
unsafe: cpu time: 49 real time: 49 gc time: 0
typed, unsafe provided: cpu time: 271 real time: 271 gc time: 0
plain: cpu time: 414 real time: 414 gc time: 0
===rgb-distance^2
---
typed: cpu time: 672 real time: 672 gc time: 1
typed-orig: cpu time: 664 real time: 664 gc time: 1
unsafe: cpu time: 12 real time: 12 gc time: 0
typed, unsafe provided: cpu time: 13 real time: 13 gc time: 0
plain: cpu time: 13 real time: 13 gc time: 0
---
typed: cpu time: 676 real time: 676 gc time: 1
typed-orig: cpu time: 664 real time: 664 gc time: 1
unsafe: cpu time: 13 real time: 13 gc time: 0
typed, unsafe provided: cpu time: 13 real time: 13 gc time: 0
plain: cpu time: 13 real time: 13 gc time: 0
```

Now the "typed" version of "rgb-average" is almost on par with usafe
one. I re-provided only rgb-average, I left rgb-distance intact. Also I
optimized it a bit (basically restructured the way the bindings are
introduced in the loop) and that helped a lot. But now...

Why is the re-provided through rename of copied binding (typed) 5 times
faster than the one which is unsafe-provide'd (typed, unsafe provided)
directly? I must be overlooking something.


Cheers,
Dominik

-- 
You received this message because you are subscribed to the Google Groups 
"Racket Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to racket-users+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/racket-users/427cc569-1929-d70c-3d0d-894de2b89176%40trustica.cz.

Re: [racket-users] Typed Racket: type relations

[Dominik Pantůček]

>>>
>>> However, I would suggest that the right fix here is to use refinement
>>> types, and specify exactly what you want. Unfortunately, the
>>> refinement types feature (good intro here:
>>> https://blog.racket-lang.org/2017/11/adding-refinement-types.html)
>>> isn't quite able to prove everything you want, but that's the
>>> direction we should go. For example, it can already prove that rd in
>>> the rgb-distance^2 function has this type: (define-type PMByte (Refine
>>> [v : Fixnum] (and (< v 256) (< -256 v
>>
>> This is exactly what I was looking at (and for), but I am unable to use
>> it correctly:
> 
> I think you need to add #:with-refinements after the #lang line.

Awesome! Now TR can properly reason about the differences.

I guess it is not possible to make TR deduce the proper type of (* rd
rd) like:

(define-type Byte^2 (Refine [v : Fixnum] (and (>= v 0) (< v (* 255 255)

Are there any plans of adding such reasoning?

Honestly, when I went through the code for binary fx... operations I
realized that making them variadic is possible but not straightforward
at all. Without digging deeper it seems to me that it is actually a lot
of work. I will look into it again later on.

Funny thing is that even trying to verify my color-blending code with TR
helped me improve that code - quite to my surprise. The original idea
was to start with lighting code (intensity, falloff, colored light) and
then proceed on to more complex parts (something like what I tried with
verifying the unsafe-structs usage). And it turned out that even that
was a bit too much so I had to go down to the color handling code.

I am afraid that at the moment TR cannot replace contracts in my design
process. That is a pity - because although I mostly do "design by
contract", it is NOT verification. So sometimes I have to run hoops thru
loops to make the program fail on particular contract to see what is
wrong. With TR my aim is to shift this effort from runtime to compile
(or more precisely design) time.

Still it is awesome tool for both code profiling and proving correctness
of individual modules.

> 
>>> 3. I don't see any boundaries where you describe -- can you say more?
>>
>> Run typed-performance.rkt and see:
> 
> This might be related to https://github.com/racket/typed-racket/issues/289.
> 

So basically my unsafe-reprovide module should probably use some syntax
trickery to generate new bindings for all typed provides it requires and
re-provide them unsafely renaming them back to original name? I think I
can do something like that using relatively simple syntax macro.

Do I understand it correctly?

Of course Ben's hint at unsafe-reprovide is something I'd like to
investigate.


Cheers,
Dominik

-- 
You received this message because you are subscribed to the Google Groups 
"Racket Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to racket-users+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/racket-users/af71d1d3-6246-bcae-31ba-be2b064baba7%40trustica.cz.

Re: [racket-users] Typed Racket: type relations

[Dominik Pantůček]

0. Thank you very much for looking into this.

On 18. 04. 21 4:57, Sam Tobin-Hochstadt wrote:
> Ok, three parts:
> 
> 1. Is it possible to make `average` on `Byte` provably produce a
> `Byte`? This is not going to be possible with plain Typed Racket, even
> with refinements to the numeric tower. The problem is that maintaining
> the invariant that a <= (* n 255) is not something that we can express
> just with the sets of values Typed Racket reasons about.

That is what I was afraid of. Mathematically speaking, the proof is
almost trivial. But expressing that turned out to be a tough nut to crack.

I looked into base-env-numeric.rkt and I see that there is quite some
type reasoning already implemented. And it works great for simple cases
like (fxand 255 anything) - : Byte.

Maybe adding an explicitly ranged Integer type and after reasoning about
the result match the final range against the set of coarse-grained types
could be a solution? Albeit not a trivial one in terms of implementing it.

> 
> 2. The rgb-distance^2 issue is really just that there's no negative
> counterpart to `Byte`, so `Byte` minus `Byte` is `Fixnum`. We could
> add that, at the cost of extra complexity in the numeric tower
> generally.
> 
> However, I would suggest that the right fix here is to use refinement
> types, and specify exactly what you want. Unfortunately, the
> refinement types feature (good intro here:
> https://blog.racket-lang.org/2017/11/adding-refinement-types.html)
> isn't quite able to prove everything you want, but that's the
> direction we should go. For example, it can already prove that rd in
> the rgb-distance^2 function has this type: (define-type PMByte (Refine
> [v : Fixnum] (and (< v 256) (< -256 v

This is exactly what I was looking at (and for), but I am unable to use
it correctly:

(define-type PMByte (Refine [v : Fixnum] (and (< v 256) (< -256 v

  (: rgb-distance^2 (-> Color Color Nonnegative-Fixnum))
  (define (rgb-distance^2 c1 c2)
(define-values (r1 g1 b1) (split-rgb c1))
(define-values (r2 g2 b2) (split-rgb c2))
(define rd : PMByte (fx- r2 r1))
(define gd : PMByte (fx- g2 g1))
(define bd : PMByte (fx- b2 b1))
(unsafe-fx+
 (fx* rd rd)
 (fx* gd gd)
 (fx* bd bd)))

typed-color.rkt:61:24: Type Checker: type mismatch
  expected: PMByte
  given: Fixnum
  in: (fx- g2 g1)
  context...:

/usr/share/racket/pkgs/typed-racket-lib/typed-racket/utils/tc-utils.rkt:123:0:
report-all-errors

/usr/share/racket/pkgs/typed-racket-lib/typed-racket/typecheck/tc-toplevel.rkt:376:0:
type-check

/usr/share/racket/pkgs/typed-racket-lib/typed-racket/typecheck/tc-toplevel.rkt:619:0:
tc-module

/usr/share/racket/pkgs/typed-racket-lib/typed-racket/tc-setup.rkt:96:12:
temp34

/usr/share/racket/pkgs/typed-racket-lib/typed-racket/typed-racket.rkt:23:4

I would really like to be able to reason about the Integer bounds (and
therefore signs) dynamically. So that when you have something like:

(: square (-> Fixnum Nonnegative-Fixnum))
(define (square x)
  (fx* x x))

You will get something better than:

multiplication.rkt:7:2: Type Checker: type mismatch
  expected: Nonnegative-Fixnum
  given: Fixnum
  in: (fx* x x)

I will look into the refinements implementation to see what are the
current limits and what the future possibilities might be.

> 
> 3. I don't see any boundaries where you describe -- can you say more?

Run typed-performance.rkt and see:

typed: cpu time: 1347 real time: 1347 gc time: 3
typed-orig: cpu time: 1367 real time: 1367 gc time: 3
unsafe: cpu time: 54 real time: 54 gc time: 1
typed, unsafe provided: cpu time: 292 real time: 292 gc time: 0
plain: cpu time: 436 real time: 436 gc time: 1

The "typed" and "typed-orig" are the same. But the "typed" and "typed,
unsafe provided" should be the same and only "typed-orig" should enforce
contracts.

The typed-unsafe-color.rkt just does:

(unsafe-provide (all-from-out "typed-color.rkt"))

Which was my attempt at re-providing typed procedures without contracts.


Cheers,
Dominik

-- 
You received this message because you are subscribed to the Google Groups 
"Racket Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to racket-users+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/racket-users/fe6b4c71-398f-efe0-ce93-59048faa4fe7%40trustica.cz.

Re: [racket-users] Typed Racket: type relations

[Dominik Pantůček]

I wanted to polish things a bit before starting a longer discussion, but
here we go ;-)

The code in question[1] is part of my work into exchanging unsafe
modules which can be used either contracted or uncontracted for TR
modules. The goal is to replace racket/unsafe/ops with TR to provide
compile-time type consistency and to have modules which are internally
consistent and if they are used with other TR code the consistency
remains. More on that later.

On 16. 04. 21 15:51, Sam Tobin-Hochstadt wrote:
> To improve this, we'd have to extend the type of `fxquotient`, which
> is reasonable, but I'm not sure what the addition would be. In
> particular, your addition is not sound:
> 
> (fxquotient 1024 2) produces 512 which is not a Byte.

Please, take a quick look at the typed-color.rkt in the repository.

The "Color" type is just a Nonnegative-Fixnum. It is a generic RGB value
with blue in lowest 8 bits, green shifted above blue and red on top. 24
bits in total. The split-rgb splits it into 3 R,G,B values - where each
of those values is a byte.

Then the rgb-average function takes an arbitrary number of Color values,
computes sums of their distinct R,G,B components and divides all of them
by the number of values before merging them together into one RGB Color
value.

Average value of (listof Byte) is definitely a Byte again. I am sorry I
didn't send the whole code in question. I was focused on other parts and
this is just a side issue in my current work.

Is there a way to express this in TR while still generating a code that
is on-par with the unsafe version? In theory, TR should excel in this.

A similar problem arises with the rgb-distance^2 function where the
component difference can be either positive or negative. But definitely
a square of whatever Integer is positive. And definitely the result
cannot be bigger than 3*255^2=195075, which is definitely
Nonnegative-Fixnum on any platform.

Again - is there a way to express this in TR and generate a code that
has no runtime checks?

The motivation here is that the performance and type soundness should go
hand in hand. If I prove the code will never get a value of a wrong
type, there is no need for runtime checks. Actually the more strict
rules, the better code can be (in theory) generated.

Then the typed/untyped boundaries introduce contracts - or they don't. I
am really confused, why there is a typed/untyped boundary between
typed-unsafe-color.rkt and typed-color.rkt. I assume this comes from my
poor understanding of TR - it will probably get better in the
forthcoming months as TR has proven to be an invaluable tool for
improving the quality of performance-oriented modules so far. More on
that later too...


Cheers,
Dominik

[1] https://gitlab.com/racketeer/typed-racket-performance

-- 
You received this message because you are subscribed to the Google Groups 
"Racket Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to racket-users+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/racket-users/d95b9140-fd78-0230-439b-aba654505fd0%40trustica.cz.

[racket-users] Typed Racket: type relations

[Dominik Pantůček]

Hello Racketeers,

working on gradually typing some of my modules I encountered an
interesting problem:

(define-type Color Fixnum)

  (: argb-average (-> Color * Color))
  (define (argb-average . argbs)
(let loop ((as : Fixnum 0)
   (rs : Fixnum 0)
   (gs : Fixnum 0)
   (bs : Fixnum 0)
   (n : Fixnum 0)
   (argbs : (Listof Color) argbs))
  (if (null? argbs)
  (make-argb (fxquotient as n)
 (fxquotient rs n)
 (fxquotient gs n)
 (fxquotient bs n))
  (let-values (((a r g b) (argb-split (car argbs
(loop (fx+ as a)
  (fx+ rs r)
  (fx+ gs g)
  (fx+ bs b)
  (fx+ n 1)
  (cdr argbs))

Type Checker: type mismatch
  expected: Byte
  given: Fixnum
  in: (fxquotient bs n)

The only way of fixing this issue was using unsafe-fxquotient which is
unsafe-require/typed accordingly:

(unsafe-require/typed
 racket/unsafe/ops
 (unsafe-fxquotient (-> Fixnum Fixnum Byte)))

Is there a better way?

The relation between Byte and (Nonnegative-)Fixnum is mathematically
sound here, but making TR understand it is apparently pretty hard...


Cheers,
Dominik

-- 
You received this message because you are subscribed to the Google Groups 
"Racket Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to racket-users+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/racket-users/608fdb93-b2ce-37e0-750c-037b47fed102%40trustica.cz.

Re: [racket-users] Variadic fixnum operations (was: Unsafe structs)

[Dominik Pantůček]

Hello Racketeers,

>> * Math/Fixnums/Flonums: All fx+/-/*/... accept two arguments only. No
>> unary fl-, no variadic-argument fl+ or fxior (this one hurt the most).
> 
> These definitely became variadic after the type definitions were
> written, but that's of course not an excuse for not updating them.
> 

are these planned for update anytime soon? Tested with latest git and
the functions are still two-argument only.

If there are no plans, I can probably have a look into that and send PR
if I am successful.


Cheers,
Dominik

-- 
You received this message because you are subscribed to the Google Groups 
"Racket Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to racket-users+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/racket-users/f6ca3a18-1cce-09cb-1599-2132f0cfa30f%40trustica.cz.

Re: [racket-users] Polished 3D package for some simulations

[Dominik Pantůček]

One clicks on your Twitch and the first thing coming from the speakers
is "Do you remember Jack? Jack Firth?"

Everyone does, of course ;-)

On 12. 04. 21 17:31, Stephen Foster wrote:
> In the CodeSpells project, we are using Racket + Unreal Engine.  This
> combo would be my recommendation for doing "3D stuff" and also anything
> with physics.  As you can see from the first few seconds of this video,
> we are spawning in Twitch users as 3D physical objects:
> 
> https://www.youtube.com/watch?v=wq8aau-fHbs
> 
> On Monday, April 12, 2021 at 4:46:56 AM UTC-7 hen...@topoi.pooq.com wrote:
> 
> On Sun, Apr 11, 2021 at 09:25:48PM -0500, Nathaniel W Griswold wrote:
> > Thanks all, i’m just going to see if Pict3D works out. I didn’t
> know about it before. I don’t really need textures for this project
> as i am starting with just particles moving around and wave
> superposition.
> 
> It might work for you then. It didn't for me. And I spent a while
> trying to
> figure out how to attach textures, but the internals of Pict3d were too
> much for me. Not enough internal documentation. It was easier to use
> sgl
> directly, but I may someday have to go to serious solid modelling.
> 
> -- hendrik
> 
> >
> > Nate
> >
> > > On Apr 8, 2021, at 2:59 PM, Hendrik Boom 
> wrote:
> > >
> > > On Thu, Apr 08, 2021 at 11:22:10AM -0700, kamist...@gmail.com
> wrote:
> > >> Have you tried out Pict3D [1]?
> > >
> > > Two significant flaws with Pict3D:
> > >
> > > * It does not have a mechanism for attaching a texture to an
> object.
> > >
> > > * It does not allow one to cut one #D object out of another.
> > > (such as using a cylinder to drill a round hole in a block of wood)
> > >
> > > Otherwise it's an impressive peve of work.
> > >
> > >> There is also a library from Matthew Flatt that can create 3D
> Text /
> > >> extruded pathes: [2]
> > >>
> > >> Personally I currently use opengl [3], these bindings support
> higher opengl
> > >> versions than the [4]
> > >> (which only support opengl 1.5 according to its documentation)
> > >>
> > >> I can't say exactly up to which opengl version [3] has working
> bindings,
> > >> these bindings were automatically generated, but that stopped
> working at
> > >> some point.
> > >> (Apparantly the input that was used for the generation isn't
> provided for
> > >> recent opengl versions anymore, so the generator has to be
> fixed or
> > >> replaced to work with a different input/description)
> > >
> > > Exactly. I've been struggling with this last year, and I sort of
> gave up.
> > > I'm thinking of looking at it again.
> > >
> > > What happened to the spec is that the entire thing has been
> recoded from a
> > > completely ad-hoc formalism to a slightly less ad-hoc XML-based
> formalism.
> > > Everything is different. And the new one is somewhat
> inconsistent about
> > > the placement of asterisks in the XML phrases that form function
> prototypes.
> > >
> > > I've been using sgl for my own work. I suspect it to be largely
> > > hand-coded, but it seems to work.
> > >
> > > There's also a binding for Vulkan, which I don't think has been
> used vary
> > > much, and so may still pose surprises.
> > >
> > > https://docs.racket-lang.org/vulkan/index.html
> 
> > >
> > > I don't know how well any of this works on a Mac, which I gather
> is going
> > > its own way for graphics, perhaps in the cause of incompatibility.
> > >
> > > -- hendrik
> > >
> > >>
> > >> [1] https://docs.racket-lang.org/pict3d/index.html
> 
> > >> [2] https://docs.racket-lang.org/pict3d-die-cut/index.html
> 
> > >> [3] https://docs.racket-lang.org/opengl/index.html
> 
> > >> [4]
> > >>
> 
> https://docs.racket-lang.org/sgl/index.html?q=opengl#%28idx._%28gentag._0._%28lib._sgl%2Fscribblings%2Fsgl..scrbl%29%29%29
> 
> 
> 
> > >>
> > >> --
> > >> You received this message because you are subscribed to the
> Google Groups "Racket Users" group.
> > >> To unsubscribe from this group and stop receiving emails from
> it, send an email to racket-users...@googlegroups.com.
> > >> To view this discussion on the web visit
> 
> https://groups.google.com/d/msgid/racket-users/67072c79-aac7-40a1-8774-2d684c7476d1n%40googlegroups.com
> 
> 

[racket-users] Strange problem with racket/gui

[Dominik Pantůček]

Hello Racketeers,

with GUI widgets tree like:

vertical-panel%
  vertical-pane%
vertical-pane%
  horizontal-pane%
text-field%
  horizontal-pane%
  horizontal-pane%
message%
vertical-pane%
  slider%

After pressing  when the text-field% has focus, the text-field%
loses and slider% gets focus. The following exception is raised though:

send: no such method
  method name: on-tab-in
  class name: wx-make-horizontal/vertical-panel%
  context...:
   /usr/share/racket/collects/racket/private/class-internal.rkt:4663:0:
obj-error
   /usr/share/racket/pkgs/gui-lib/mred/private/wxtop.rkt:546:61
   /usr/share/racket/collects/racket/private/more-scheme.rkt:148:2:
call-with-break-parameterization
   /usr/share/racket/pkgs/gui-lib/mred/private/wxtop.rkt:453:8:
handle-traverse-key method in make-top-container%
   /usr/share/racket/collects/ffi/unsafe/atomic.rkt:73:13
   /usr/share/racket/collects/racket/private/more-scheme.rkt:148:2:
call-with-break-parameterization
   /usr/share/racket/collects/ffi/unsafe/atomic.rkt:73:13
   /usr/share/racket/pkgs/gui-lib/mred/private/wx/gtk/window.rkt:836:4:
call-pre-on-char method in window%
   [repeats 3 more times]
   /usr/share/racket/pkgs/gui-lib/mred/private/wx/gtk/window.rkt:816:4:
dispatch-on-char method in window%
   /usr/share/racket/pkgs/gui-lib/mred/private/wx/common/queue.rkt:435:6
   /usr/share/racket/pkgs/gui-lib/mred/private/wx/common/queue.rkt:486:32
   /usr/share/racket/pkgs/gui-lib/mred/private/wx/common/queue.rkt:634:3


Does it ring a bell to anyone?

Welcome to Racket v8.0 [cs].
Linux telperion 5.4.0-70-generic #78-Ubuntu SMP Fri Mar 19 13:29:52 UTC
2021 x86_64 x86_64 x86_64 GNU/Linux


It is 20.04 Ubuntu LTS.


Without further investigation, it looks like some rather obscure bug
deep in pane/panel impelmentation.


Cheers,
Dominik

-- 
You received this message because you are subscribed to the Google Groups 
"Racket Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to racket-users+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/racket-users/11ce8b5e-2ead-413f-6d4a-a3b38ddf6fa0%40trustica.cz.

Re: [racket-users] Werid contract violation blame erros

[Dominik Pantůček]

It is worth noting that it is relatively easy to implement a
define/provide/contract syntax without any problems. In my two most
active projects I use it extensively. In one case even with
scribble/srcdoc to keep the contracts, scribblings and implementation in
one place.

The simplest version:

(define-syntax (define/provide/contract stx)
  (syntax-case stx ()
((_ (name args ...)
contract
body ...)
 #'(begin
 (provide (contract-out (name contract)))
 (define (name args ...)
   body ...)

Of course, this is not the best approach for all situations (and I do
not use it everywhere), but it covers many common use-cases well.


Dominik

On 07. 04. 21 22:34, Robby Findler wrote:
> The short answer: you probably should use (provide (contract-out))
> instead of define/contract.
> 
> The slightly longer answer: when you write a contract, you are not just
> describing what the legal inputs and outputs are, you are also
> establishing a *boundary* between two regions of code. In the case of
> define/contract, you are establishing a boundary between the function
> (here: f) and the module that it contains (here the file "a.rkt"). In
> the case of (provide (contract-out  ...)) the boundary is between a.rkt
> and b.rkt.
> 
> The slightly uncomfortable answer: it is my (not completely solid yet)
> opinion that the boundary that is drawn for define/contract is perhaps
> not the right / best one. In a theoretical/mathematical sense it is a
> completely fine and completely defensible one. But it trips people up
> sometimes. (There are examples others have posted in the past that are
> perhaps even stranger than yours but boil down to the same specific
> design choice for define/contract.)
> 
> Robby
> 
> 
> 
> On Wed, Apr 7, 2021 at 2:10 PM epi via Racket Users
> mailto:racket-users@googlegroups.com>>
> wrote:
> 
> Hello Racket users,
> 
> I am trying to understand a contract violation message that I am
> getting.
> Here is the file a.rkt:
> 
>     #lang racket
>     (provide f)
>     (define/contract (f a)
>       (-> boolean? any/c)
>       '())
> 
> and this is b.rkt:
> 
>     #lang racket
>     (require "a.rkt")
>     (f 3)
> 
> 
> I would expect that the caller is blamed for the contract violation,
> but the error message that I get is as follows:
> 
> 
> f: contract violation
>   expected: boolean?
>   given: 3
>   in: the 1st argument of
>       (-> boolean? any/c)
>   contract from: (function f)
>   blaming: /home/epi/snippets/a.rkt
>    (assuming the contract is correct)
>   at: /home/epi/snippets/a.rkt:3.18
>   context...:
>  
>  /usr/share/racket/collects/racket/contract/private/blame.rkt:347:0:
> raise-blame-error
>  
>  
> /usr/share/racket/collects/racket/contract/private/arrow-higher-order.rkt:379:33
>    body of "/home/dan/snippets/blameme.rkt"
> 
> So, I am a bit surprised that the error message blames the file a.rkt.
> What am I missing here?
> 
> -- 
> You received this message because you are subscribed to the Google
> Groups "Racket Users" group.
> To unsubscribe from this group and stop receiving emails from it,
> send an email to racket-users+unsubscr...@googlegroups.com
> .
> To view this discussion on the web visit
> 
> https://groups.google.com/d/msgid/racket-users/149cfc632cd666ff1a92177dce90296b%40disroot.org
> 
> .
> 
> -- 
> You received this message because you are subscribed to the Google
> Groups "Racket Users" group.
> To unsubscribe from this group and stop receiving emails from it, send
> an email to racket-users+unsubscr...@googlegroups.com
> .
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/racket-users/CAL3TdONLMx%3Dy_cgfDsY_k9L9yaX_touO52phiK9scziW_jGrOA%40mail.gmail.com
> .

-- 
You received this message because you are subscribed to the Google Groups 
"Racket Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to racket-users+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/racket-users/8208bf07-8a32-f0fa-a933-c6b3c4f15fef%40trustica.cz.

[racket-users] Next Racket Meetup: Saturday, May 1st

[Dominik Pantůček]

Hello fellow Racketeers,

although the meetup yesterday wasn't actually crowded, an interesting
discussion about fortifying macros, lambda calculus, lazy racket, or
inter-language interoperability started. We'd definitely like to see
more Racketeers with interesting papers, projects or generally anything
Racket-related!

The next meetup is planned (as you could easily infer from the subject
line) for the first Saturday of May, which happens to be 1st of May.

The place and time stays the same:

https://gather.town/app/wH1EDG3McffLjrs0/racket-users

Sat, 1 May 2021 UTC at 20:00:

Prague: 22:00 CEST
London: 21:00 BST
New York: 16:00 EDT
San Francisco: 13:00 PDT

I'll take care of promoting it at #Racket IRC channel (Freenode). Sam
mentioned he'll do the same on Slack (I guess). And as usual, if Paulo
writes about us in the next issue of Racket-News, everyone will
appreciate it as well.

Honestly, Racket-News has become the canonical location for checking
when and where the Racket meetup will be held ;-)


Cheers,
Dominik

-- 
You received this message because you are subscribed to the Google Groups 
"Racket Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to racket-users+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/racket-users/a6a57766-959a-9a7d-a8cd-71be058c7eeb%40trustica.cz.

Re: [racket-users] Word Count program/benchmark performance

[Dominik Pantůček]

Another attack of [1]. But yeah, why not do some [2].

Trees to the rescue [3].

$ racket --version
Welcome to Racket v8.0 [cs].

$ racket countwords-bogdan2.rkt https://xkcd.com/386/
[2] http://phdcomics.com/comics/archive.php?comicid=1735
[3] https://gist.github.com/dzoep/0e081d0544afac539a4829179c601e0e

On 19. 03. 21 11:18, Bogdan Popa wrote:
> I updated the gist with some cleanups and additional improvements that
> get the runtime down to a little over 1s (vs ~350ms for the optimized C
> and Rust code) on my maxed-out 2019 MBP and ~600ms on my M1 Mac Mini.
> 
> Pawel Mosakowski writes:
> 
>> Hi Bogdan,
>>
>> This is a brilliant solution and also completely over my head. It finishes
>> in ~3.75s on my PC and is faster than the Python version which basically
>> delegates all the work to C. I will need to spend some time on
>> understanding it but I am looking forward to learning something new.
>>
>> Many thanks,
>> Pawel
>>
>> On Thursday, March 18, 2021 at 7:22:10 PM UTC bogdan wrote:
>>
>>> I managed to get it about as fast as Python by making it really
>>> imperative and rolling my own hash:
>>>
>>> https://gist.github.com/Bogdanp/fb39d202037cdaadd55dae3d45737571
>>>
>>> Sam Tobin-Hochstadt writes:
>>>
 Here are several variants of the code:
 https://gist.github.com/d6fbe3757c462d5b4d1d9393b72f9ab9

 The enabled version is about the fastest I can get without using
 `unsafe` (which the rules said not to do). It's possible to optimize a
 tiny bit more by avoiding sorting, but only a few milliseconds -- it
 would be more significant if there were more different words.

 Switching to bytes works correctly for the given task, but wouldn't
 always work in the case of general UTF8 input. But those versions
 appeared not be faster for me. Also, writing my own string-downcase
 didn't help. And using a big buffer and doing my own newline splitting
 didn't help either.

 The version using just a regexp matching on a port (suggested by
 Robby) turned out not to be faster either, so my suspicion is that the
 original slowness is just using regexps for splitting words.

 Sam

 On Thu, Mar 18, 2021 at 11:28 AM Sam Tobin-Hochstadt
  wrote:
>
> Here's a somewhat-optimized version of the code:
>
> #lang racket/base
> (require racket/string racket/vector racket/port)
>
> (define h (make-hash))
>
> (time
> (for* ([l (in-lines)]
> [w (in-list (string-split l))]
> [w* (in-value (string-downcase w))])
> (hash-update! h w* add1 0)))
>
> (define v
> (time
> (for/vector #:length (hash-count h)
> ([(k v) (in-hash h)])
> (cons k v
> (time (vector-sort! v > #:key cdr))
> (define p (current-output-port) #;(open-output-nowhere))
> (time
> (for ([pair (in-vector v)])
> (write-string (car pair) p)
> (write-string (number->string (cdr pair)) p)
> (newline p)))
>
> It's much more imperative, but also pretty nice and compact. The
> `printf` optimization is significant for that portion of the program,
> but that isn't much of the running time. The overall running time for
> 10 copies of the KJV is about 9 seconds on my laptop.
>
> I think the remaining difference between Racket and other languages is
> likely the `string-split` and `string-downcase` functions, plus the
> relatively-inefficient string representation that Racket uses.
>
> Sam
>
>
> On Thu, Mar 18, 2021 at 10:28 AM Pawel Mosakowski 
>>> wrote:
>>
>> Hi David,
>>
>> Yes, the 21 seconds includes the interpreter startup time. I have
>>> done a simple test to see how long it takes:
>>
>> $ time racket -e '(displayln "Hello, world")'
>> Hello, world
>>
>> real 0m0.479s
>> user 0m0.449s
>> sys 0m0.030s
>>
>> I have also put my code inside a main function and profiled it:
>>
>> Profiling results
>> -
>> Total cpu time observed: 20910ms (out of 20970ms)
>> Number of samples taken: 382 (once every 55ms)
>> (Hiding functions with self<1.0% and local<2.0%: 1 of 12 hidden)
>>
>> ==
>> Caller
>> Idx Total Self Name+src Local%
>> ms(pct) ms(pct) Callee
>> ==
>> [1] 20910(100.0%) 0(0.0%) [running body]
>>> ...word-occurences-profile.rkt":##f
>> profile-thunk [2] 100.0%
>> --
>> [running body] [1] 100.0%
>> [2] 20910(100.0%) 0(0.0%) profile-thunk
>>> ...ket/pkgs/profile-lib/main.rkt:9:0
>> run [3] 100.0%
>> --
>> profile-thunk [2] 100.0%
>> [3] 20910(100.0%) 0(0.0%) run
>>> ...share/racket/pkgs/profile-lib/main.rkt:39:2
>> main [4] 100.0%
>> 

Re: [racket-users] Racket Meetup! 6 March

[Dominik Pantůček]

Hello Racketeers,

just a friendly reminder here on the ML - this Saturday brings the next
monthly Racket meetup(s) in Gather Town!


>From the latest Racket News[1]:

6 March 2021 TWO sessions

11 am Pacific Time
4:30 pm UK time

See you there!

Dominik

[1] https://racket-news.com/2021/03/racket-news-issue-47.html

On 06. 02. 21 21:05, Stephen De Gabrielle wrote:
> 
> Thank you to Eli Barzilay for joining us on today's meetup to discuss
> the paper 'A Foreign Function Interface'. 
> 
> And a big thank you to Sam Philips for organising this event and
> making sure it happens. (all I did was the racket news announcement so
> thank you Sam)
>  
> I had to leave a little early but I wanted to communicate the date and
> time for the next meetup:
> 
> Racket users video meetup
> 6 March 2021 at 8pm CET, via Gather Town.
> https://gather.town/app/wH1EDG3McffLjrs0/racket-users
> 
> Paper for discussion (optional pre-reading) is:  To be decided! 
> 
> (Note - we do take suggestions if you have something that interests or
> excites you - and other media would also be considered; blog post,
> video, package, app - as long as it is Racket related)
> 
> Kind regards
> 
> Stephen De Gabrielle
> 
> -- 
> You received this message because you are subscribed to the Google
> Groups "Racket Users" group.
> To unsubscribe from this group and stop receiving emails from it, send
> an email to racket-users+unsubscr...@googlegroups.com
> .
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/racket-users/CAGHj7-JQnJF18DjsN7392TSOa6JxTPFEy1OEwQQFm980zrUM3Q%40mail.gmail.com
> .

-- 
You received this message because you are subscribed to the Google Groups 
"Racket Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to racket-users+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/racket-users/32a5fc42-81bd--ced7-438c97ff5ac0%40trustica.cz.

Re: [racket-users] Bit Scan Reverse in Racket

[Dominik Pantůček]

On 25. 02. 21 10:01, Jens Axel Søgaard wrote:
> Try integer-length.
> 
> https://docs.racket-lang.org/reference/generic-numbers.html?q=integer-length#%28def._%28%28quote._~23~25kernel%29._integer-length%29%29

Oh, I completely missed this one. Thank you!!!

> 
> I don't know how it is implemented.

And now I do. On CS it is implemented in terms of fxlength, which is a
SW implementation, but still an order of magnitude faster than pure
Racket implementation.

So for my project, integer-length is an improvement I wanted. But the
general question remains. Instead of almost 100 instructions, it could
be just one or two. I'll look into it later on if no-one else finds it
interesting enough to fiddle with cpnanopass.ss and x86_64.ss ;-)


Cheers,
Dominik

-- 
You received this message because you are subscribed to the Google Groups 
"Racket Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to racket-users+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/racket-users/ff8da3ed-3fee-5c4c-11f2-eecb086fd60e%40trustica.cz.

[racket-users] Bit Scan Reverse in Racket

[Dominik Pantůček]

Hello Racketeers,

I'm slightly stuck with speeding up some calculations and the reason is
that I need to compute the index of highest set bit in a number. So for
1 it is 0, for 2 it is 1, for 8 3, 1023 9 and 1024 10 ...

The fastest Racket code I can come up with is as follows:

(begin-encourage-inline
  (define (highest-bit num)
(let loop ((num num)
   (bit 0))
  (if (unsafe-fx> num 0)
  (loop (unsafe-fxrshift num 1)
(unsafe-fx+ bit 1))
  bit

(Yes, it returns incorrect result for 0, but that must be checked
elsewhere as the result cannot be defined for 0 anyway).

However this is a single instruction operation on x86 (bsr) or two
instruction operation (rbit and clz if I am not mistaken) on ARM. Dunno
about PPC yet.

I looked at CS internals a bit and although there is a "name collision"
as the bsr mnemonics is used for ret (branch subroutine return?), it
should be fairly easy to add something like fxbsr (-> fixnum? fixnum?)
to Racket core.

I also experimented with x64asm package but the results were even worse
than the aforementioned tight loop (there is a lot of overhead in
define-λ! generated functions).

As the routine is typically called 40k-60k times per frame in my code
(real-time rendering) it could really make a difference.

Should I try to add it? How should it be named? What about BC?

And more generic question - aren't there more similar operations that
can be implemented efficiently on all supported CPUs that might be
useful in general? Yes, I am aiming at SIMD as many of you know :)
Especially with the expanded number of available FP registers to 8 on
64-bit x86 CS there surely is quite some potential to it.


Cheers,
Dominik

-- 
You received this message because you are subscribed to the Google Groups 
"Racket Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to racket-users+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/racket-users/a2a4fe83-877d-d7ed-4812-bd628c128659%40trustica.cz.

Re: [racket-users] Ubuntu PPA also updated to v8.0

[Dominik Pantůček]

On 22. 02. 21 19:56, Asumu Takikawa wrote:
> On 2021-02-18 21:44:30 -0800, Asumu Takikawa wrote:
>> Due to changes in how the new CS compiler works, some files have been
>> moved between the three .deb packages that are in the distribution.
> 
> It looks like this change broke package upgrades. I should be able to
> fix the packages later by adjusting the dependency flags.
> 
> In the meantime, if you get an error on upgrade you should be able to
> work around it by just uninstalling the racket packages and
> re-installing the latest ones.
> 

Actually `apt-get -f install` fixed that without having to manually
uninstall/install anything.


D.

-- 
You received this message because you are subscribed to the Google Groups 
"Racket Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to racket-users+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/racket-users/e00bca5f-faf4-14aa-7bb8-208badbb8b53%40trustica.cz.

Re: [racket-users] file/gzip trouble reading

[Dominik Pantůček]

On 16. 02. 21 22:27, Matthew Flatt wrote:
> At Tue, 16 Feb 2021 16:03:29 -0500, Ben Greenman wrote:
>> Sadly, I've already compressed a few files using
>> `call-with-output-string` ... is there an easy way to decompress those
>> / undo the UTF-8 encoding?
> 
> Unfortunately, the underlying `get-output-string` conversion is lossy,
> because bytes that don't form a UTF-8 encoding are converted to U+FFFD.
> 
> (I see that the docs say #\? instead of #\uFFFD, and I'll fix the docs.)
> 

#\uFFFD is #\� (bytes EF BF BD in UTF-8)

For those who do not see it (I suspect encoding issues) it is a white
question mark on black vertically elongated hexagon.

And actually Racket REPL in my terminal displays it like this (7.9 BC,
8.0 CS, both on Ubuntu 20.04 in GNOME terminal).


Dominik

-- 
You received this message because you are subscribed to the Google Groups 
"Racket Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to racket-users+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/racket-users/6a90a9b9-7853-1318-5a1a-425a535f7214%40trustica.cz.

Re: [racket-users] signup to racket users without a google account?

[Dominik Pantůček]

It works - this is how I subscribed to this list (as I am not using
Google services).


Dominik

On 08. 02. 21 20:18, Paulo Matos wrote:
> 
> Stephen De Gabrielle writes:
> 
>> Hi,
>>
>> Someone over on reddit /r/racket wants to joining the mailing list without
>> creating a goodle account - I just tried (in 'private window') and the join
>> link* redirected me to the google groups page
>> https://groups.google.com/g/racket-users
>>
>> Is there another way to sign up to racket users without a google account?
>>
> 
> Send an email to racket-users+subscr...@googlegroups.com
> 
> Should work... I hope! :)
> 
>> Kind regards
>>
>> Stephen
>>
>> * join link on
>> Join the users mailing list
>> 
>> https://groups.google.com/forum/#!forum/racket-users/join
> 
> 

-- 
You received this message because you are subscribed to the Google Groups 
"Racket Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to racket-users+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/racket-users/13dd449b-c058-9f5e-d87d-cf469936ec14%40trustica.cz.

Re: [racket-users] Unsafe structs

[Dominik Pantůček]

Hi Sam,

I went through all my notes and prepared minimal (sometimes) working
examples for most of the issues I mentioned. Let's go through it one by
one. I assume that some of the complications I encountered were because
my lack of experience with Typed Racket. I hope some of these examples
will be useful anyway.

All the examples are in a git repository on Gitlab [1].

>> * Higher-order procedures and polymorphic functions in all imaginable
>> combinations. That was a total disaster. Yes, the documentation clearly
>> states that - but typing any code using these is like trying to break a
>> badly designed cipher. Irregularities here and there. Sometimes simple
>> `inst' was enough. Sometimes casting both the function and the arguments
>> was necessary. The biggest trouble is that the error messages are very
>> cryptic and sometimes even do not point to the offending construct but
>> only the module file. I will provide MWE to show this if someone wants
>> to look into it.
> 
> An example would be great here. In general you should only need one
> use of `inst` in any of these cases, and you shouldn't ever need a
> `cast`.

Directory "polymorph". In my use-case, there is a lot of structured
S-expression data in the format:

(define tagged-data-example
  '(data-2D
((B B B B)
 (B B B

9 steps of typing the procedure to calculate the maximum length of the
2D matrix are in this directory. All the errors are documented there.
Yes, many of the intermediate constructs are wrong. The error in
polymorph4.rkt shows the biggest problem I've seen so far. As a separate
example, it is pretty easy to fix - but finding the source of the
problem in a bigger module was not easy at all. Basically I ended up
bi-partitioning the whole source code with procedure granularity and
once I found the offending procedure a linear trial-and-error approach
was the only option that allowed me to find the offending expression.

Yes, the final polymorph9.rkt contains the approach that I actually used
for resolving the issue.

> 
>> * unsafe/ops: unsafe-vector-* resisted a lot - until I just gave up and
>> require/typed it for my particular types. Maybe casting would help, but
>> the error messages were different to the classical problems of the
>> polymorphic functions in TR.
> 
> Can you say more about what happened here?

I completely mixed things up. It was not unsafe/ops, actually there was
just a minor misunderstanding on my side and after I realized what's
wrong, it worked like a charm.

The actual problem I was referring to here is nicely shown in the
"union" directory and it is about typing hash-union procedure.

I do not know what solution should be considered correct if there were
multiple differently-typed hash-tables in the same module. The explicit
type information in require/typed is definitely not a good idea - but I
found no better way.

> 
>> * Classes: My notes say "". Which roughly says it
>> all. Although I managed to back down to only using bitmap% class,
>> properly typing all procedures using it was a nightmare. By properly I
>> mean "it compiles and runs".
> 
> More detail here would be helpful as well.

The task was so simple and the solution so ugly. See the "classes"
directory with 4 steps of typing a simple program to load a bitmap.

Yes, normally I use (make-object bitmap% filename 'unknown/alpha) and it
just works (racket/draw hides all the problems from the programmer). But
I failed to require/typed it correctly. With explicit specification of
object interface that is used, it worked. The bitmap4.rkt contains the
code that I actually used.

> 
>> * with-input-from-file does not accept Path or String, only Path-String
>> and the conversion rules are either missing or strange at best.
>> Basically I ended up with just converting to String and casting to
>> Path-String to make everything work.
>>
>> * with-input-from-file also revealed that procedure signatures as types
>> can be very tricky - just passing `read' was not possible, because it
>> accepts some optional arguments. Wrapping it in thunk helped though.
> 
> I don't understand what the problem was here; for example this works for me:
> 
> #lang typed/racket
> (with-input-from-file "/tmp/x.rkt" read)

Yes, this works. The example in "input" directory shows the problem I
had. Again, the trouble was I was working with a list where the first
element is different from the rest of elements and I wanted to ensure
type consistency for the rest of the code.

> 
> and `Path-String` is just the union of Path and String.

Turns out I must have some other error there as well. I cannot reproduce
it on its own and I wasn't tagging the whole tree when I encountered new
problems. Next time I am tagging the repository so that I can get back
to the exact bigger example of the problem and eventually see what
caused it.

> 
>> * order of definitions matters. Not that this is unexpected, it is just
>> strange when working with larger 

Re: [racket-users] GUI button% string label keyboard mnemonics

[Dominik Pantůček]

> Maybe a stupid question ...
> 
> Is  "close-icon" the name of the bitmap or a function that creates a
> bitmap?

I am using images/icons/misc:

https://docs.racket-lang.org/images/Icons.html?q=close-icon#%28def._%28%28lib._images%2Ficons%2Fmisc..rkt%29._close-icon%29%29

So it generates a bitmap% in the end.

> Have you tried:
> 
>    (new button% (label close-icon "" 'left) ...)
> 

Just out of curiosity right now and no, the button% does not accept
procedure to be a part of that list.



Dominik

-- 
You received this message because you are subscribed to the Google Groups 
"Racket Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to racket-users+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/racket-users/971cfa8f-739c-96af-29f2-e243bfc35bdb%40trustica.cz.

[racket-users] GUI button% string label keyboard mnemonics

[Dominik Pantůček]

Hello Racketeers,

it's been a while since I was doing some serious GUI work but for one of
my current work projects the time has come and surprise surprise -
troubles on the way :)

When a button is created like:

(new button% (label "") ...)

everything works as expected. The #\q key becomes a keyboard shortcut
that triggers the callback correctly.

But when the string label is given in the list of icon and label like:

(new button% (label (close-icon) "" 'left) ...)

The "&" is left intact and no action is triggered by pressing the #\q
character on the keyboard.

The documentation clearly states both alternatives as supported: "If &
occurs in label (when label includes a string), it is specially parsed;
...". Am I doing something wrong or is it a bug?

If I look at the right source for my platform -
gui-lib/mred/private/wx/gtk/button (Linux/X11/Gtk), in both cases
gtk_new_with_mnemonic is used. Seems OK to me.

Any ideas what I might have overlooked?


Cheers,
Dominik

-- 
You received this message because you are subscribed to the Google Groups 
"Racket Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to racket-users+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/racket-users/64893a3c-e8c2-b007-9879-35138bbd4512%40trustica.cz.

Re: [racket-users] Unsafe structs

[Dominik Pantůček]

Hello Racketeers (and Robby especially)!

On 22. 12. 20 1:30, Robby Findler wrote:
> Is Typed Racket able to prove that your use of unsafe accessors is
> actually safe? 

Short answer: YES.

One question for a start: And what now?

Disclaimer: The following text is by no means intended as critique but
rather it should be viewed as a summary of my experience with typing
middle-sized project in two weeks.

Long rant follows.

When Ben sent the Typed Racket Survey[1] here, my experience with TR was
only minimal. I worked with the math/matrix and basically focused on
performance gains it might yield. So I honestly filled-in whatever I
knew. Now I know much more. Hope it will help.

Let's first sum up where TR helped enormously and actually surprised me
VERY pleasantly:

* Finding cases of (when ...) and similar # returning constructs
where values are to be returned and probably error should be raised
instead of returning nothing. Although the actual erroneous return never
happened in the code in question, those constructs were meant to ensure
correct value is returned... Probably some leftovers from the early days
of this project. TR spotted them immediately.

* Finding improper usage of #f for missing information. When the program
expects some value, #f is not the right choice of fallback value. It is
the right one if you want to use it as a condition. Again, these were
leftovers from various experiments - mostly when loading data. TR to the
rescue again.

* Unintended use of default procedure arguments - nice and unexpected!

And now for the worse part. TR rough edges:

* Higher-order procedures and polymorphic functions in all imaginable
combinations. That was a total disaster. Yes, the documentation clearly
states that - but typing any code using these is like trying to break a
badly designed cipher. Irregularities here and there. Sometimes simple
`inst' was enough. Sometimes casting both the function and the arguments
was necessary. The biggest trouble is that the error messages are very
cryptic and sometimes even do not point to the offending construct but
only the module file. I will provide MWE to show this if someone wants
to look into it.

* Struct: Missing struct generics - when porting code that relies on
them, there's not much that can be done.

* Math: there really is just a natural logarithm and no logarithm with
arbitrary base? Yes, one-line to implement, but why?

* Math/Fixnums/Flonums: All fx+/-/*/... accept two arguments only. No
unary fl-, no variadic-argument fl+ or fxior (this one hurt the most).

* unsafe/ops: unsafe-vector-* resisted a lot - until I just gave up and
require/typed it for my particular types. Maybe casting would help, but
the error messages were different to the classical problems of the
polymorphic functions in TR.

* Classes: My notes say "". Which roughly says it
all. Although I managed to back down to only using bitmap% class,
properly typing all procedures using it was a nightmare. By properly I
mean "it compiles and runs".

* with-input-from-file does not accept Path or String, only Path-String
and the conversion rules are either missing or strange at best.
Basically I ended up with just converting to String and casting to
Path-String to make everything work.

* with-input-from-file also revealed that procedure signatures as types
can be very tricky - just passing `read' was not possible, because it
accepts some optional arguments. Wrapping it in thunk helped though.

* order of definitions matters. Not that this is unexpected, it is just
strange when working with larger code-base where it didn't matter.
Actually the error messages were helpful here.

* Type annotations of procedures with variadic arguments. The only place
where I had to put annotations outside of the procedure definition. It
is nothing super-problematic, but it feels inconsistent with the rest.

* More modules need to be required everywhere. If module A provides a
procedure that accepts a type from module B, all modules using that
procedure must also require the module B to know the type. In normal
Racket it does not matter as long as you just pass the opaque data along.

* Syntax macros are extra hard. As I use some syntax trickery to convert
semi-regular code to "futurized" one, I basically gave up and just ran
everything single-threaded. The main issue is passing type information
from the module that uses the macro to the macro and matching it on both
sides. Also the macro must split the type annotation from argument names
in the syntax pattern when it defines new procedures - I did some ugly
hacks to get through it but in the end I just refrained from using them
when possible (except for the unsafe-struct macro, of course).

If anyone actively working on TR reads this, I'd be more than happy to
discuss my experience and share the code (although it is really ugly as
I really only intended to prove the unsafe structs are used safely).
Next Saturday would be a good time to 

Re: [racket-users] Can someone please override the Reply-To so that normal replies go to the list?

[Dominik Pantůček]

> 
> Proper mail systems also provide a command "reply to list".  The proper 
> fix is to make sure your mail system supports reply to list.
> 

Which seems to work only if the list is in the "To:" header - not "Cc:".
At least that is how my Thunderbird apparently works...


Dominik

-- 
You received this message because you are subscribed to the Google Groups 
"Racket Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to racket-users+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/racket-users/07891d54-d5e6-a9de-9165-7a82488a1dc8%40trustica.cz.

Re: [racket-users] Re: Scribble markdown renderer: tables

[Dominik Pantůček]

My impression is that the table generated by markdown render mixin is
not a markdown table.

This is what I get:

ONE  two  3
TWO  three111
THREEone  222

From:

(tabular
 (list
  (list "ONE" "two" "3")
  (list "TWO" "three" "111")
  (list "THREE" "one" "222")))

Also if you look at the actual line strings:

"ONE  two  3"
"TWO  three111  "
"THREEone  222  "

I know it might sound silly, but to me it looks like the table
delimiters are just omitted in the code[1] and judging from the file
commit history, it looks like the tables are rendered on purpose this
way - but they are supposed to be wrapped inside ```/```racket blocks.
Which they aren't in this case.

The problem might be, that the markdown specification for tables
requires them to have the separator row and assumes first line to be the
heading line. While scribble tables are more LaTeX-like and do not make
any assumptions.

I hacked together a VERY ugly awk post-processing pass to solve the one
(and only one) table that I need to render properly. However it seems to
me that having better markdown table support might be beneficial for
others as well.

My proposal is:

Add the tabular delimiters ("| ", " | ", and " |") to the output and
generate the separator row immediately after the first row of data.

Is it too radical or should I patch the markdown renderer and send a PR?



[1]
https://github.com/racket/scribble/blob/master/scribble-lib/scribble/markdown-render.rkt#L94

On 30. 12. 20 1:14, Alex Harsanyi wrote:
> 
> I am not entirely sure what it is that is not working, so I am just
> guessing here:
> 
> * If you are generating Markdown files that contain tables, you need to
> be aware that tables need to be in column 0 (i.e. NOT indented).  If
> they are indented, they will be rendered as pre-formatted text.
> * If you are using the `markdown` package to parse Markdown file, it
> does not support tables.  You either need to merge  Add support for
> table by antoineB · Pull Request #56 · greghendershott/markdown
> (github.com)  , or
> use the `markdown-ng` package.
> 
> Hope this helps,
> Alex. 
> 
> On Tuesday, December 29, 2020 at 10:17:21 PM UTC+8
> dominik@trustica.cz wrote:
> 
> Hello Racketeers,
> 
> back from my procrastination projects to work :)
> 
> In the biggest project our company is working on right now, we use
> Racket HTTP/REST backend and React.js frontend. The whole backend is
> fully contracted - even all the actual servlets that are exposed via
> HTTP are contracted. And in addition to it, they are all fully
> documented using scribble/srcdoc.
> 
> I am using some syntax trickery to generate a table of API calls, so
> that the frontend developers can easily find the right
> documentation. It
> is just a transformation of dispatch-case to scribble tabular
> procedure.
> 
> With PDF output, it works like a charm. With markdown, the tables get
> rendered as fixed-width columns - but it is still a plain text. No
> markers, nothing.
> 
> Just to complete the picture, we are using CI/CD to automatically
> update
> project's Gitlab wiki.
> 
> Am I missing something, or the markdown renderer produces incorrect
> output and I should fix it?
> 
> Any hints will be VERY appreciated as the frontend is developed mostly
> by non-programmers, so it would really help to have everything they
> need
> in one interface (CI pipeline results, issues tracking, backend
> documentation).
> 
> 
> Cheers,
> Dominik
> 
> -- 
> You received this message because you are subscribed to the Google
> Groups "Racket Users" group.
> To unsubscribe from this group and stop receiving emails from it, send
> an email to racket-users+unsubscr...@googlegroups.com
> .
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/racket-users/7d0d585f-0191-47ea-8979-ccbf71cf7a18n%40googlegroups.com
> .

-- 
You received this message because you are subscribed to the Google Groups 
"Racket Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to racket-users+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/racket-users/1784e20e-2e6b-ca88-cb37-cb226c2725b5%40trustica.cz.

[racket-users] Scribble markdown renderer: tables

[Dominik Pantůček]

Hello Racketeers,

back from my procrastination projects to work :)

In the biggest project our company is working on right now, we use
Racket HTTP/REST backend and React.js frontend. The whole backend is
fully contracted - even all the actual servlets that are exposed via
HTTP are contracted. And in addition to it, they are all fully
documented using scribble/srcdoc.

I am using some syntax trickery to generate a table of API calls, so
that the frontend developers can easily find the right documentation. It
is just a transformation of dispatch-case to scribble tabular procedure.

With PDF output, it works like a charm. With markdown, the tables get
rendered as fixed-width columns - but it is still a plain text. No
markers, nothing.

Just to complete the picture, we are using CI/CD to automatically update
project's Gitlab wiki.

Am I missing something, or the markdown renderer produces incorrect
output and I should fix it?

Any hints will be VERY appreciated as the frontend is developed mostly
by non-programmers, so it would really help to have everything they need
in one interface (CI pipeline results, issues tracking, backend
documentation).


Cheers,
Dominik

-- 
You received this message because you are subscribed to the Google Groups 
"Racket Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to racket-users+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/racket-users/1d189b9f-1bb8-34a5-83e9-8b39f468f0cc%40trustica.cz.

Re: [racket-users] Unsafe structs

[Dominik Pantůček]

On 22. 12. 20 4:40, Philip McGrath wrote:
> On Mon, Dec 21, 2020 at 7:30 PM Robby Findler  > wrote:
> 
> Is Typed Racket able to prove that your use of unsafe accessors is
> actually safe?
> 

I can try adding type annotations everywhere, but as I started with
contracts, the results should be the same. It might be interesting to
see how to prove it. There are some cross-procedure type assumptions in
my case co it means fully annotating a few modules. Will look into it.

> 
> On a similar note, my understanding is that, even without types, in a
> program like this:
> #lang racket
> (struct cell (val))
> (λ (x)
>   (if (cell? x)
>       (cell-val x)
>       ...))
> the compiler can optimize the use of `cell-val` guarded by `cell?` to
> omit the redundant check. I think the optimized version should be as
> good as `unsafe-struct-ref`.

I hope that in the simple example you provided it is as good as the
unsafe variant. Trouble is that the guards are usually not in the same
scope as the accessors. The major performance improvement I see is when
passing the results of geometry checks from one module to surface
properties handler in another. So definitely the compiler does not see
the guards.

I think Robby's question aims in the right direction. Not only it should
be possible to prove that I _can_ use unsafe accessors/mutators in given
scope, but also the compiler should be able to recognize it and do not
need them in the first place. I'll look into it more.

Although - after my experience with math/matrix - I expect the Typed
Racket (+ #:authentic, yes) to yield better results, but not as good as
the unsafe accessors.

> 
> If that's the case, the only extra invariant for `unsafe-struct*-ref` is
> that it does not work on impersonators (including chaperones, IIUC). If
> you need the extra performance, I think you can get it safely by
> declaring your structs as `#:authentic` (or equivalently adding
> `prop:authentic`), which will cause the struct types involved not to
> support impersonators: `impersonate-struct` and similar functions will
> raise exceptions. On the other hand, if you need to support
> impersonators, then it really is unsafe to use `unsafe-struct*-ref`.

#:authentic improves it only VERY slightly (didn't look at the generated
code, just did a few empirical measurements).

Dominik

-- 
You received this message because you are subscribed to the Google Groups 
"Racket Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to racket-users+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/racket-users/e7fcfe64-608f-a7ef-50ef-cb3fbec36323%40trustica.cz.

Re: [racket-users] Unsafe structs

[Dominik Pantůček]

On 21. 12. 20 18:07, David Storrs wrote:
> 
> The struct-plus-plus module also provides reflection, so you might take
> a look to see if there are any ideas in there that would be useful for
> your own module.  Accessors are included, as are constructors, rules,
> wrappers, default values, and predicates.  spp has two primary
> limitations:  You cannot use a base type and you cannot mark individual
> fields mutable, only the entire struct.
Nice one! The per-field #:mutable keyword was one of the things that
made me look into it more :)

I will look into the sources and some of the ideas there will help me
implement it in a cleaner way. That said, I am mostly interested in
providing the unsafe accessors/mutators transparently.


Cheers,
Dominik

-- 
You received this message because you are subscribed to the Google Groups 
"Racket Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to racket-users+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/racket-users/19db4267-edd6-6eea-778d-8b15643789a1%40trustica.cz.

Re: [racket-users] Unsafe structs

[Dominik Pantůček]

On 21. 12. 20 18:14, George Neuner wrote:
>
> On 12/20/2020 3:34 PM, Dominik Pantůček wrote:
>> Hello Racketeers,
>>
>> there were some discussions about structs' introspection on the IRC
>> lately and one of the questions that arose was how to get field index
>> for arbitrary struct's arbitrary field.
>>
>> Not that it is possible... But the general discussion made me think
>> about why there are no unsafe variants for structs' accessors and
>> mutators.
>
> Have you looked at 'struct-field-info-list' and 'struct-field-index'?
>
> 'struct-field-index' works if you know the field name(s), and
> 'struct-field-info-list' returns fields defined by the struct type - so
> if your structs types are flat, you can make do with these.
>
> However, 'struct-field-info-list' returns only fields defined by the
> actual type and does not include fields that were inherited.  What I
> don't see is any simple way to get at the struct's inheritance hierarchy
> - it seems that you have to iterate 'struct-type-info' to enumerate the
> supertypes.
>
> So it is "possible" (for some definition).
Yes, this approach would be useful to create the unsafe
accessors/mutators after the structs have been defined. Trouble is, one
would have to traverse the structs hierarchy and filter-out those
accessors/mutators, that are already defined by parent struct.

What I wanted to achieve was providing the unsafe variants
"transparently". Honestly, given enough time, the best solution for me
would be adding #:unsafe keyword to the struct form that would define
the unsafe variants.

My usage requires only struct-wide unsafe accessors. I implemented the
rest as an exercise to see what can be done (and to play with more
complex syntax-rules too).

It may be the case that using extract-struct-info and traversing the
structs' hierarchy during expansion phase might be more elegant solution
 for implementing the whole unsafe-struct syntax. I will definitely look
into it more in the (relatively near) future.

Thank you,
Dominik

-- 
You received this message because you are subscribed to the Google Groups 
"Racket Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to racket-users+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/racket-users/8f73dfb6-62a6-24e4-9eb0-acc5151abadc%40trustica.cz.

[racket-users] Unsafe structs

[Dominik Pantůček]

Hello Racketeers,

there were some discussions about structs' introspection on the IRC
lately and one of the questions that arose was how to get field index
for arbitrary struct's arbitrary field.

Not that it is possible... But the general discussion made me think
about why there are no unsafe variants for structs' accessors and mutators.

And given my work on some performance-demanding projects I started
testing unsafe-struct*-ref and unsafe-struct*-set! in one of my
simulations (yes, that raycasting/raytracing project). It bumped the
speed from roughly 110-120 fps to about 140-150 fps on my laptop. So
definitely worth the try.

But unsafe-struct*-ref/set! degrade structs basically to strange
vectors. And also - in my code I use struct hierarchy to decide what to
do with different kinds of data.

It didn't take long and my unsafe-struct[1] proof-of-concept was created.

The code is still a bit messy. However it keeps all the semantics of
structs and enables usage of unsafe accessors and mutators without any
hassle. Basically the unsafe-struct and unsafe-struct-out are drop-in
replacements or struct and struct-out.

A typical usage would be:

(unsafe-struct my-struct (a b (c #:mutable)) #:transparent)
(define my-val (my-struct 1 2 3))
(unsafe-set-my-struct-a! my-val 4)
(displayln (unsafe-my-struct-b my-val))

Of course, the procedures are inherently unsafe ...

In a typical scenario I am using these immediately after checking the
type of given value using the struct predicate (like my-struct? in this
case) and getting the performance boost without any real un-safety.

And now some questions (I bet you knew I will have some):

1) Is there anyone else reading this that could actually leverage these
unsafe-structs for something useful?

2) If yes, what do you miss there before I create a package from the
code? (Yes, scribblings, yes, slightly cleaning the code up - but what
else?)

3) In the syntax pattern I was unable to create a sub-pattern to get the
field-mutable for all fields (including those without any field options)
- that's the reason for most of the long let* later on. Is there a
better way? (I will think about this one, but more eyes ...)


Cheers,
Dominik

P.S.: I didn't abandon the flalgebra module, I am actually working on
another approach that should move the resulting "DSL" closer to the
flonums unboxing and register allocation code. More on that later.


[1] https://gitlab.com/racketeer/unsafe-struct

-- 
You received this message because you are subscribed to the Google Groups 
"Racket Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to racket-users+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/racket-users/90b3e86a-db1d-c658-111b-ae4010c5bea2%40trustica.cz.

[racket-users] Linear algebra performance (was: calculations with arrays)

[Dominik Pantůček]

Hello,

> 
> If you are after numerical calculations for mxn matrices, then look at
> `flomat`
> which uses the same underlying C/Fortran library that NumPy uses.
> 
> https://docs.racket-lang.org/manual-flomat/index.html
> 
> If you need rrays with more dimensions than 2 use `math/array`.
> 
> If you need simple computation of 1-dimensional data, you can use
> standard Racket vectors (or flomats or arrays).

I didn't know about `flomat'. Thanks for (yet another) good hint.

However, for my project I needed really fast matrix multiplication (and
other basic linear algebra functions). It turned out that most of the
available options are sub-optimal at best. To my surprise `flomat` is
one of those.

Mostly I am concerned with 3D and 4D matrices and vectors.

During the past few months I hacked together a fairly optimized
module[1] for performing these operations that defines algebras for
given dimension during macro expansion stage and all the procedures are
constructed in a way that helps Racket (mainly CS) perform all
operations unboxed.

In the repository, there is a benchmark script, which yields the
following (self-explanatory) results:

 Welcome to Racket v7.9.0.5 [cs].

# 100 iterations

## flalgebra

mat3x3*mat3x3! cpu time: 13 real time: 13 gc time: 0
mat3x3*mat3x3 cpu time: 23 real time: 23 gc time: 4

## math/matrix

matrix*matrix cpu time: 55892 real time: 55911 gc time: 463

## math/typed/matrix

matrix*matrix cpu time: 6861 real time: 6862 gc time: 1045

## flomat

matrix*matrix! cpu time: 887 real time: 887 gc time: 4
matrix*matrix cpu time: 1825 real time: 1825 gc time: 7

 Welcome to Racket v7.9 [bc].

# 100 iterations

## flalgebra

mat3x3*mat3x3! cpu time: 145 real time: 145 gc time: 7
mat3x3*mat3x3 cpu time: 163 real time: 163 gc time: 2

## math/matrix

matrix*matrix cpu time: 53817 real time: 53788 gc time: 733

## math/typed/matrix

matrix*matrix cpu time: 3852 real time: 3851 gc time: 730

## flomat

matrix*matrix! cpu time: 745 real time: 745 gc time: 1
matrix*matrix cpu time: 1621 real time: 1620 gc time: 1


What puzzles me the most: when I read `flomat' documentation, I thought
it must beat my implementation by far margin - it's a native code for
really basic number crunching. When using the `times!' variant of matrix
multiplication, I would expect it to outperform anything implemented in
in pure Racket.

Is there something I miss or is it really the case, that carefully
crafted Racket implementation is the fastest option right now?

I actually wanted to write a similar email some time ago - in the spring
- about the typed math/matrix variant. But I was almost certain I am
doing something wrong and should figure it out. Now I am more and more
convinced that the performance of all number crunching libraries - as
they can be used in Racket - might be the real issue here.


Cheers,
Dominik

[1] https://gitlab.com/racketeer/flalgebra

-- 
You received this message because you are subscribed to the Google Groups 
"Racket Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to racket-users+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/racket-users/c94bf70d-1563-62d5-7632-bbaade7eb97f%40trustica.cz.

[racket-users] net-lib/ftp enhancement

[Dominik Pantůček]

Hello Racketeers,

we were working on a module today that uses the net/ftp module for batch
communication with some remote systems and realized we REALLY could use
variants of ftp-download-file and ftp-upload-file that would accept
ports instead of just folders/paths.

So I looked at the source and found out the code is really easy to
extend this way and added these variants.

You can see the patch in [1].

Do you think that more people would benefit from this? If no, we'll just
use a patched version of net/ftp in our project. If yes, I will update
the scribblings as well before submitting a PR.

Any thoughts?


Cheers,
Dominik


[1]
https://github.com/dzoep/racket/commit/277f857566b48037ed57b329443b591024acf43b

-- 
You received this message because you are subscribed to the Google Groups 
"Racket Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to racket-users+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/racket-users/a358e1ee-f78f-0148-99ac-4364f7dc6e42%40trustica.cz.

Re: [racket-users] Namespaces and modules

[Dominik Pantůček]

> 
> Using ''sandbox as an argument to `namespace-require` to refer to a
> local module is something like passing 'x to `eval` to try to refer to
> a local variable `x`. The `namespace-require` procedure doesn't know
> where it's being called from, so it doesn't work.
> 
> Try `quote-module-path`, which expands at compile time to effectively
> embed its context:
> 
>  #lang racket/base
>  (require syntax/location)
> 
>  (module sandbox racket/base
>(provide #%app #%datum test)
>(define (test) (displayln 'test)))
> 
>  (parameterize ((current-namespace (make-base-empty-namespace)))
>(namespace-require (quote-module-path sandbox))
>(eval '(test)))
> 

This is exactly what I was looking for! Thank you. It's a pity I didn't
ask in the past :)

However, now I see a problem with creating executables with raco exe.

MWE follows.

 test-submod.rkt
#lang racket

(require syntax/location)

(module my-dsl racket/base

  (provide #%app my-proc)

  (define (my-proc)
(displayln "my-dsl/my-proc")))

(define (parse sexp)
  (parameterize ((current-namespace (make-base-empty-namespace)))
(namespace-require (quote-module-path my-dsl))
(eval sexp)))


(parse '(my-proc))


And then:

$ ../racket-lang/racket/racket/bin/racket --version
Welcome to Racket v7.9.0.4 [cs].
$ ../racket-lang/racket/racket/bin/racket test-submod.rkt
my-dsl/my-proc
$ ../racket-lang/racket/racket/bin/raco exe test-submod.rkt
$ ./test-submod
require: unknown module
  module name: (submod '#%mzc:test-submod my-dsl)
  context...:
   .../TD4/test-submod.rkt:12:0: parse
   body of '#%mzc:test-submod
$

When parsed, compiled and run using racket binary, it works as expected.
When parsed and compiled using raco exe, it succeeds without any error
and the resulting binary give aforementioned error.

Any idea where did I get it wrong?


Dominik

-- 
You received this message because you are subscribed to the Google Groups 
"Racket Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to racket-users+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/racket-users/d357eb72-0647-d542-5148-852b9e62c2a2%40trustica.cz.

[racket-users] Namespaces and modules

[Dominik Pantůček]

Hello Racketeers,

it seemed to me that writing a data format for my project by simply
introducing a new syntax forms and then just eval'uating the file (maybe
eval-syntax) was a great idea. I've done things like that many times but
my initial bindings were always in a separate file. Like:

 sandbox.rkt
#lang racket/base
(provide #%app #%datum test)
(define (test) (displayln 'test))


Then a simple example looks like:

(parameterize ((current-namespace (make-base-empty-namespace)))
  (namespace-require "sandbox.rkt")
  (eval '(test)))

Easy and works without a flaw.

If I however try to achieve the same goal using module form within my
module, it always fails:

(module sandbox racket/base
  (provide #%app #%datum test)
  (define (test) (displayln 'test)))
(parameterize ((current-namespace (make-base-empty-namespace)))
  (namespace-require ''sandbox)
  (eval '(test)))

It fails with:

require: unknown module
  module name: 'm

Of course, without double-quoting, it yields:

standard-module-name-resolver: collection not found
  for module path: m
  collection: "m"
...

Which is expected. And require'ing the module doesn't change a thing (of
course, (require 'm) require's it in the surrounding module).

After going through the modules and namespaces documentation back and
forth I feel like I am overlooking something.

Roughly the goal is to define a bunch of structs in the encapsulating
module, provide syntax to the evaluation namespace, evaluate the file
with data and get the result.

Any hint would be very appreciated.


Cheers,
Dominik

-- 
You received this message because you are subscribed to the Google Groups 
"Racket Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to racket-users+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/racket-users/100b00a9-33bd-1b82-a3ec-afef7d3d8fe1%40trustica.cz.

Re: [racket-users] Racket BC versus CS unsafe flonums

[Dominik Pantůček]

Wow, you are faster than I :)

On 07. 11. 20 15:36, Matthew Flatt wrote:
> At Fri, 6 Nov 2020 12:45:46 -0700, Matthew Flatt wrote:
>> I will investigate faster option. A primitive without conversion could
>> make the safe `flvector-set!` slightly faster, too, by avoiding a
>> redundant check.
> 
> Long story short, I added flvectors to the Chez Scheme level as of
> v7.9.0.4. With that change, `flvector-set!` is faster, and
> `flvector-ref` and `flvector-set!` cooperate better with flonum
> unboxing.

Ok, I originally wanted to reply to your original remark about
unsafe/safe flonum operations, but this actually extends my questions.

My current understanding is that the best performance you get from
unsafe operations while using safe operations as hints for the flonum
unboxing algorithm, right?

So with flvectors it is now the same? When I convey some safe hint to
the unboxing code, is it the best (in terms of performance) to use
unsafe flvector procedures then? To be honest, when I try to write down
"definitive" rules how to structure the code (mainly tight loops) and
use safe/unsafe operations properly, I get quickly lost.

I'll do some empirical benchmarks and see whether I can get some
generally valid answer.

> 
> For example, this microbenchmark now avoids allocation and runs about 8
> times as fast:
> 
>  (let ([v (make-flvector 100)])
>(time
> (for ([j (in-range 10)])
>   (for ([i (in-range (flvector-length v))])
> (flvector-set! v i (fl+ 1.0 (flvector-ref v i)))

I was just about to ask. This is how most of my code looks right now
(albeit more complex and with very short flvectors - 3 or 9 elements
mostly). I'll test the new implementation and see if there is a difference.

> 
> Also, your program now crashes as you intended.
> 

Aweseome! (May sound weird, but I really like consistent behavior).

> 
> (To make room in Chez Scheme's type encoding for flvectors, I removed
> immutable fxvectors. Immutable fxvectors do not seem useful, and
> there's no such thing at the Racket level.)

There definitely are scenarios, where immutable fxvectors may be a good
idea. However in those scenarios, allocations ruin the performance most
of the time. And although CS performs way better with extensive box
allocations than BC, it can still quickly become a noticeable bottleneck.


Dominik

-- 
You received this message because you are subscribed to the Google Groups 
"Racket Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to racket-users+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/racket-users/e6f12dab-7996-9479-2cc1-85422d9b6587%40trustica.cz.

[racket-users] Racket BC versus CS unsafe flonums

[Dominik Pantůček]

Hello Racketeers,

going through some more possible optimizations of my side projects, I
encountered a different behavior of unsafe-flvector-set! between BC and
CS implementations:

 CUT HERE 
#lang racket

(require racket/unsafe/ops
 racket/flonum)

(define flv (make-flvector 10))
(for ((i (in-range 10)))
  (unsafe-flvector-set! flv i i))
(for ((i (in-range 10)))
  (display (~a #:width 4 (unsafe-flvector-ref flv i
(newline)
 CUT HERE 

$ racket test-flvector.rkt
SIGSEGV MAPERR si_code 1 fault on addr 0x9
Aborted (core dumped)
$ racketcs test-flvector.rkt
0.0 1.0 2.0 3.0 4.0 5.0 6.0 7.0 8.0 9.0
$

The SIGSEGV is expected behavior (although might not get triggered every
time). The latter looks like implicit fx->fl operation.

Changing to safe flvector-set! results in expected contract violation
with both variants:

$ racketcs test-flvector.rkt
0.0 1.0 2.0 3.0 4.0 5.0 6.0 7.0 8.0 9.0
$ racket test-flvector.rkt
flvector-set!: contract violation
  expected: flonum?
  given: 0
  argument position: 3rd
  other arguments...:
   (flvector 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0)
   0
  context...:
   "/home/joe/Projects/Programming/TD4/test-flvector.rkt": [running body]
   temp35_0
   for-loop
   run-module-instance!
   perform-require!
$ racketcs test-flvector.rkt
flvector-set!: contract violation
  expected: flonum?
  given: 0
  context...:

/home/joe/Projects/Programming/racket-lang/racket/racket/collects/racket/private/for.rkt:1534:9
   body of "/home/joe/Projects/Programming/TD4/test-flvector.rkt"
$

I assume that CS' unsafe-flvector-set! is actually pretty safe when it
comes to flonum-convertible numbers. It might be a bit faster because it
lacks the contract, but definitely it is not a "low level" variant.

Also given the fact that unsafe-fl+ and others give no performance
advantage (the opposite is true for well-written algorithms), maybe the
performance guide and unsafe ops documentation should get an update
before CS becomes the default implementation.

Any thoughts?


Cheers,
Dominik

-- 
You received this message because you are subscribed to the Google Groups 
"Racket Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to racket-users+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/racket-users/c39293d0-51fa-41d8-3e0e-1fd99bca59ae%40trustica.cz.

[racket-users] Racket CPU Graphics Engine - Time to Share

[Dominik Pantůček]

Hello fellow Racketeers,

during the spring and summer I worked on a side project to test what can
be done in pure Racket when it comes to raster graphics rendering. You
can imagine it like demo or game programming before the advent of GPUs.

It turned out that quite a lot can be achieved and it also turned out
that Racket performs really well when it comes to some hard number
crunching. And what else is computer graphics than just a number crunching?

As I consider this experiment basically completed, I release the source
code[1] to the public under the usual dual Apache/MIT license. A short
(25s) video in action can be seen on the project "website"[2].

For those wondering about the tut25 naming - the project started as an
attempt to rewrite all Denthor's VGA Trainers published between 1994 and
1996. There were 21 tutorials, I added four. I am publishing only the
final result - the 25th "tutorial". If I have some spare time in the
future, I'll write a series of articles about the journey from drawing
the first pixel in the window to full 3D triangle rasterizer with alpha
blending, perspective-correct texturing, z-buffering and Goraud shading.

And for those wondering "why?" - there are actually more answers:

1) Why not?
2) Projects like this stress-test Racket implementation and therefore
help its development (more on that below).
3) I want more people to start using futures for improving performance
without sacrificing code readability.
3b) Also I don't want to hear from the people in the core team "Oh, you
are probably the only one seriously using futures" ;-)

To inspire more people to start seemingly crazy projects like this, here
is a list of things that are now better in current Racket version as a
side-effect of this project (and yes, Matthew wrote all the patches):

* futures versus GC in BC - no more crashes[3]
* futures and the write barrier in BC - no more freezes[4]
* fsemaphores - no longer can go negative (actually the bug was
elsewhere)[5]
* safe and unsafe fl->fx unification[6]
* internal stack overflow fixed[7]
* box CAS operation troubles with futures[8]

The code comes however with a big red warning: some parts are as ugly as
they can get. On the other hand, there are some nice language features
that I consider worth investigating further. Mainly:

* using syntax macros to create procedure templates and then with
another layer of syntax macros instantiate those for specific task. See
for example the define/provide-draw-proc-variants and template-clear in
rcge/drawing.rkt. Yes, they are unhygienic - but just very slightly...

* context nesting using parameters (although I personally dislike the
need to wrap everything in so many parameterize layers) - see any of
rcge/*-run.rkt files for example.

* using syntax macros to provide hints to the compiler. The best example
of this is the matrix multiplication in rcge/algebra.rkt -
define-mat4*mat4!/let. Yes, definitely not the best practice in general,
but combined with unboxed flonums on CS, it improved the performance by
an order of magnitude compared to simple loop.

* define-futurized (rcge/futures.rkt) - in an ideal world, a generalized
version of a macro like this should be able to parallelize
computation-heavy algorithms with little to no effort for the programmer.

Things NOT to do: blit-to-cairo-data*-u16vector-hack - rcge/blit.rkt is
not for the weak. But I like the fact, that it is possible to interact
between different layers of the program, compiler and JIT.


Thank you for reading this far and if you have any questions, I'll be
more than happy to answer those.


Cheers,
Dominik

[1] https://joe.cz/rcge/
[2] https://gitlab.com/racketeer/rcge-tut25
[3] https://github.com/racket/racket/issues/3145
[4]
https://github.com/racket/racket/commit/fae20b4b8904bfcfba9808f56744950b049b6afd
[5]
https://github.com/racket/racket/commit/1117392cb5524a2ce150384d405fb2a81ef47daa
[6]
https://github.com/racket/racket/commit/42cb80bc70e1fb97dabd1e0a2dff5d93257b1f40
[7]
https://github.com/racket/racket/commit/265c9eaa57661a045feed85e1db45029aa9de7e2
[8]
https://github.com/racket/racket/commit/0af11de62b2eaa1f4f0977794a4dc9dcacabd945

-- 
You received this message because you are subscribed to the Google Groups 
"Racket Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to racket-users+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/racket-users/a46338ba-51e1-e3f4-dba2-99d319b0b503%40trustica.cz.

Re: [racket-users] Re: Help implementing an early return macro

[Dominik Pantůček]

Hi racketeers,

I would second this suggestion. Although it might look slightly
un-rackety at first sight, for certain types of code flow it does the
job really well.

And most importantly - I am using escape continuations in much wilder
setup (yes, futures) and it imposes no noticeable performance impact. It
is still on my TODO list to measure that empirically though. On CS that
is - on BC I didn't have time to test it yet.


Dominik

P.S.: Thanks go out to Matthew for pointing ECs out at RacketCon
informal discussions. Futures with generic CC come with absurdly huge
performance penalty even if they stay within the future thread ...

On 28. 10. 20 12:23, Alex Harsanyi wrote:
> Are you looking for `let/ec`?
> 
> (let/ec return
>   (define x (random 10))
>   (unless (even? x)
>     (log-info "x wasn't even, x = ~a" x)
>     (return -1))
>   (define y (random 10))
>   (unless (even? y)
>     (log-info "y wasn't even, y = ~a" y)
>     (return -1))
>   (+ x y))
> 
> Alex.
> 
> On Wednesday, October 28, 2020 at 6:54:44 PM UTC+8 jackh...@gmail.com wrote:
> 
> So I'm a little tired of writing code like this:
> 
> (define x ...)
> (cond
>   [(take-shortcut? x) (shortcut x)]
>   [else
>(define y (compute-y x))
>(cond
> [(take-other-shortcut? x y) (other-shortcut x y)]
> [else
>  (define z ...)
>  (cond ...)])])
> 
> That is, I have some logic and that logic occasionally checks for
> conditions that make the rest of the logic irrelevant, such as an
> empty or false input or something else that should trigger an early
> exit. Each check like this requires me to write a |cond| whose
> |else| clause wraps the remainder of the body, leading to an awkward
> nesting of |cond| forms. I don't have this issue when the early
> exits involve raising exceptions: in those cases I can just use
> |when| and |unless| like so:
> 
> (define x ...)
> (unless (passes-check? x) (raise ...))
> (define y ...)
> (unless (passes-other-check? x y) (raise ...))
> (define z ...)
> ...
> 
> I'm aware of a few macros in the racket ecosystem that try to solve
> this problem. For example, Jay wrote a blog post
>  that
> creates a |condd| form that's like |cond| but allows embedded
> definitions using a |#:do| keyword. I've also seen various
> approaches that use escape continuations to implement the early
> exit. There's drawbacks I'm not happy about however:
> 
>   *
> 
> For |cond|-like macros that allow embedded definitions, it looks
> too different from regular straight-line Racket code. I like my
> function bodies to be a sequence of definitions and expressions,
> with minimal nesting, just like the |when| and |unless| version
> above. I don't have to use a keyword or extra parentheses to
> signal whether a form is a definition or a |when| / |unless|
> check in error-raising code, why should I have to do that in
> code that uses early returns?
> 
>   *
> 
> Continuation-based solutions impose a nontrivial performance
> penalty and have complex semantics. I don't like that the
> generated code behaves differently from the |cond| tree I would
> normally write. What happens if I stick an early exit inside a
> lambda? Or a thread? What if I set up a continuation barrier?
> Does that matter? I don't know and I don't want to think about
> that just to write what would be a simple |if (condition) {
> return ... }| block in other languages.
> 
> So I wrote a basic macro for this and I have some questions about
> how to make it more robust. The macro is called |guarded-block| and
> it looks like this:
> 
> (guarded-block
>   (define x (random 10))
>   (guard (even? x) else
> (log-info "x wasn't even, x = ~a" x)
> -1)
>   (define y (random 10))
>   (guard (even? y) else
> (log-info "y wasn't even, y = ~a" y)
> -1)
>   (+ x y))
> 
> Each |guard| clause contains a condition that must be true for
> evaluation to proceed, and if it isn't true the block takes the else
> branch and finishes. So the above would expand into this:
> 
> (block
>   (define x (random 10))
>   (cond
> [(not (even? x))
>  (log-info "x wasn't even, x = ~a" x)
>  -1]
> [else
>  (define y (random 10))
>  (cond
>[(not (even? y))
> (log-info "y wasn't even, y = ~a" y)
> -1]
>[else (+ x y)])]))
> 
> This part I got working pretty easily. Where I hit problems, and
> where I'd like some help, is trying to extend this to support two
> important features:
> 
>   *
> 
> I should be able to define macros that /expand/ into |guard|
> 

Re: [racket-users] racket user video meetup 28 November

[Dominik Pantůček]

Hi Stephen,

that Gather thing was pretty cool for RacketCon. That is not an option
here? Although I do not know about the costs of the setup...

Of course, big thanks to everyone making RacketCon happen and everyone
who attended. It was a refreshing experience to have a chat with all the
rock stars there and watch the inspiring talks!

And also big thanks to Jay who hosted it like a seasoned pro.


Cheers,
Dominik

On 19. 10. 20 0:39, Stephen De Gabrielle wrote:
> Platform to be decided; zoom  / jitsi / ? - suggestions
> --
> 30 Minutes
> 1pm PT
> 9pm UK time
> 28 November
> 
> -- If this is problematic let me know - thin intended to become a
> regular thing - every 6-12 weeks
> 
> Kind regards
> 
> Stephen & Sam Phillips
> 
> -- 
> You received this message because you are subscribed to the Google
> Groups "Racket Users" group.
> To unsubscribe from this group and stop receiving emails from it, send
> an email to racket-users+unsubscr...@googlegroups.com
> .
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/racket-users/CAGHj7-JhtdweJpKtUi2yFh-P_tp6XmJGZaf7LQri24ihCV42Nw%40mail.gmail.com
> .

-- 
You received this message because you are subscribed to the Google Groups 
"Racket Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to racket-users+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/racket-users/12a4b56a-ecd3-cc41-7455-89561ef19f99%40trustica.cz.

[racket-users] New lightweight termios package: rktermios

[Dominik Pantůček]

Hello Racketeers,

as I learned that it is not possible to use the ansi package under CS,
because the C extensions do not work under CS, I decided to put together
a minimalistic termios wrapper using only Racket code.

It provides the bare minimum to set the terminal to "raw" mode and
should work under any POSIX system (tested on Linux and NetBSD). Usage
is VERY simple[1].

The ansi package can be patched immediately to use it and therefore work
under CS and charterm can be probably be patched as well - meaning it
will no longer rely on `stty' for this functionality.

The termios package uses similar approach but tries to implement all
termios capabilities which means it has to compile at least the
constants from termios headers. The same applies for serial package. It
might be possible to generate a .rkt file with those constants from the
header files during package build - but I didn't investigate that yet.

Any feedback is, of course, welcome. And all the ideas mentioned are
only suggestions.


Cheers,
Dominik

[1] https://docs.racket-lang.org/rktermios/index.html

-- 
You received this message because you are subscribed to the Google Groups 
"Racket Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to racket-users+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/racket-users/756b73f6-919e-fef2-b81c-7ad19fa1378c%40trustica.cz.

Re: [racket-users] Package index

[Dominik Pantůček]

Hi,

I confirm the issue. I am already a registered user but adding a new
package fails with:

Proxy Error

The proxy server received an invalid response from an upstream server.
The proxy server could not handle the request GET /pkgn/package/rktermios.

Reason: Error reading from remote server

Apache/2.4.7 (Ubuntu) Server at pkgd.racket-lang.org Port 443


Can anyone, please, look into it?


Cheers,
Dominik

On 19. 09. 20 18:01, Griffin Byatt wrote:
> Hi all,
> 
> I’ve been unable to register on the package index for the last two days. 
> Entering my email address for a registration code results in an error and 
> stack trace.
> 
> Can anyone confirm that this also happens for them? 
> 
> Thanks,
> Griffin
> 

-- 
You received this message because you are subscribed to the Google Groups 
"Racket Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to racket-users+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/racket-users/2e36ecd1-a5cd-f018-40df-4dc5be46ba7c%40trustica.cz.

[racket-users] Scribbling hexadecimal values

[Dominik Pantůček]

Hello Racketeers,

I am struggling to make scribble typeset default values in
proc-doc/names in hexadecimal. An example would be:

(proc-doc/names
  name
  (->* () (integer?) void?)
  (()
   ((argument #x1f)))
  @{ some description }) ; yes, at-exp reader

The same applies to values in nested contracts of ->* - like (integer-in
0 #x1f).

Of course #,(~a "~x" #x1f) will produce the string with appropriate
contents - but enclosed in parentheses which does not help much. Also it
is not just a matter of typesetting because the provide form really
contracts the procedure being provided and the actual values should
actually be present.

I would love to see some documentation-stage parameter where I could
just (parameterize ((numbers-as-hexadecimal #t)) (integer-in ...) ...)
and it would keep the values as they are for contract purposes and
render them hexadecimal. Of course, this is quite specific - more
generic solution is probably more appropriate, this is just to explain
the problem I am trying to solve.


Cheers,
Dominik

-- 
You received this message because you are subscribed to the Google Groups 
"Racket Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to racket-users+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/racket-users/aca5b2ab-36b6-98c6-0747-9d5447ae9766%40trustica.cz.

Re: [racket-users] new racket-lang.org website

[Dominik Pantůček]

On 26. 08. 20 0:09, Robby Findler wrote:
> I've put a version that uses 550 pixels as a cutoff here, in case that
> the 1080 really was
> 540: https://users.cs.northwestern.edu/~robby/tmp/Web/www/

This is displayed correctly using any browser on the S61 now. So
probably it is the case of those 540 "pixels". Great job! (Not just the
fix, the whole new website, actually).


Dominik

> 
> Robby
> 
> 
> On Tue, Aug 25, 2020 at 5:06 PM Robby Findler  <mailto:ro...@cs.northwestern.edu>> wrote:
> 
> Thanks Dominik. Currently the site uses the predicate "less than 460
> pixels wide?" to determine if it should switch to single-column /
> phone mode. That seemed to work for the 3-4 phones (narrow) and
> tablets (wide) that I tried but I'm certainly not well versed in
> these kinds of things. The specific expression it uses is here:
> 
>   
> https://github.com/racket/racket-lang-org/blob/master/www/css/styles.css.pp#L543
> 
> (and then a bit further down is the negated version of that same
> predicate). 
> 
> If you (or anyone!) has advice on how to improve this, that'd be
> welcome.
> 
> It may be that the right approach is to adjust the font sizes, as
> 1080x1920 sounds really wide. Is it possible that's a retina-style
> display and the logical number is actually 540 wide? Maybe that's a
> better cutoff for switching?
> 
> Robby
> 
> 
> On Tue, Aug 25, 2020 at 3:45 PM Dominik Pantůček
> mailto:dominik.pantu...@trustica.cz>>
> wrote:
> 
> On 25. 08. 20 22:35, Laurent wrote:
> 
> > Or maybe Dominik checked something like "Always display the
> desktop
> 
> > version for this site"? (FWIW it displays nicely on a
> 2,280x1,080 pixels
> 
> > phone screen)
> 
> 
> 
> I wish that was the case. The screenshot comes from Lightning
> browser,
> 
> however I get the same result from Chrome (default on my Cat S61
> Phone)
> 
> and mobile Firefox too.
> 
> 
> 
> It's 1080x1920 with default Android 9 as shipped by the
> manufacturer.
> 
> Actually I am using this device for testing our mobile apps (yes
> ...).
> 
> 
> 
> Of course, changing the orientation to landscape makes it
> display the
> 
> desktop version correctly.
> 
> 
> 
> Sadly, I can only help by reporting as much information as possible.
> 
> 
> 
> 
> 
> Dominik
> 
> 
> 
> >
> 
> > On Tue, Aug 25, 2020 at 9:13 PM Robby Findler
> mailto:ro...@cs.northwestern.edu>
> 
> > <mailto:ro...@cs.northwestern.edu
> <mailto:ro...@cs.northwestern.edu>>> wrote:
> 
> >
> 
> >     It looks like your phone has enough pixels width wise that
> the site
> 
> >     thinks it is a normal machine (albeit uncomfortably
> narrow). If
> 
> >     things are slightly narrower you'll see the site changes. I am
> 
> >     attaching two screenshots from my phone. 
> 
> >
> 
> >     Robby 
> 
> >
> 
> >
> 
> >
> 
> >
> 
> >
> 
> >     On Tue, Aug 25, 2020 at 2:59 PM Dominik Pantůček
> 
> >      <mailto:dominik.pantu...@trustica.cz>
> <mailto:dominik.pantu...@trustica.cz
> <mailto:dominik.pantu...@trustica.cz>>>
> 
> >     wrote:
> 
> >
> 
> >         Hi,
> 
> >
> 
> >         apparently the new design is not very mobile-friendly
> as can be
> 
> >         seen on the attached screenshot. If somebody has the
> skills and
> 
> >         time, this should probably be fixed soon.
> 
> >
> 
> >         D.
> 
> >
> 
> >
> 
> >         On August 25, 2020 7:51:40 PM GMT+02:00, Robby Findler
> 
> >          <mailto:ro...@cs.northwestern.edu>
> <mailto:ro...@cs.northwestern.edu
> <mailto:ro...@cs.northwestern.edu>>>
> 
> >         wrote:
> 
> >
> 
> >
> 
> >
> 
> >             Hi all: as you may know if you follo

Re: [racket-users] new racket-lang.org website

[Dominik Pantůček]

On 25. 08. 20 22:35, Laurent wrote:
> Or maybe Dominik checked something like "Always display the desktop
> version for this site"? (FWIW it displays nicely on a 2,280x1,080 pixels
> phone screen)

I wish that was the case. The screenshot comes from Lightning browser,
however I get the same result from Chrome (default on my Cat S61 Phone)
and mobile Firefox too.

It's 1080x1920 with default Android 9 as shipped by the manufacturer.
Actually I am using this device for testing our mobile apps (yes ...).

Of course, changing the orientation to landscape makes it display the
desktop version correctly.

Sadly, I can only help by reporting as much information as possible.


Dominik

> 
> On Tue, Aug 25, 2020 at 9:13 PM Robby Findler  <mailto:ro...@cs.northwestern.edu>> wrote:
> 
> It looks like your phone has enough pixels width wise that the site
> thinks it is a normal machine (albeit uncomfortably narrow). If
> things are slightly narrower you'll see the site changes. I am
> attaching two screenshots from my phone. 
> 
> Robby 
> 
> 
> 
> 
> 
> On Tue, Aug 25, 2020 at 2:59 PM Dominik Pantůček
> mailto:dominik.pantu...@trustica.cz>>
> wrote:
> 
> Hi,
> 
> apparently the new design is not very mobile-friendly as can be
> seen on the attached screenshot. If somebody has the skills and
> time, this should probably be fixed soon.
> 
> D.
> 
> 
> On August 25, 2020 7:51:40 PM GMT+02:00, Robby Findler
> mailto:ro...@cs.northwestern.edu>>
> wrote:
> 
> 
> 
> Hi all: as you may know if you follow dev@, we've been
> revising the website. The new version went live last night;
> please have a look: https://www.racket-lang.org/
> 
> A big Thank You to Matthew Butterick for the previous design
> which, as you can tell, inspired the current visual design.
> 
> Matthew, Robby, Sam, Jay, John, and Matthias
> 
> 
> 
> 
> 
> 
> -- 
> Sent from my Android device with K-9 Mail. Please excuse my brevity.
> 
> 
> 
> 
> 
> 
> 
> 
> -- 
> 
> 
> You received this message because you are subscribed to the
> Google Groups "Racket Users" group.
> 
> 
> To unsubscribe from this group and stop receiving emails from
> it, send an email to racket-users+unsubscr...@googlegroups.com
> <mailto:racket-users+unsubscr...@googlegroups.com>.
> 
> 
> To view this discussion on the web visit
> 
> https://groups.google.com/d/msgid/racket-users/1FFB85C8-D1E3-4E72-8A1D-8DD38FE5FF05%40trustica.cz
> 
> <https://groups.google.com/d/msgid/racket-users/1FFB85C8-D1E3-4E72-8A1D-8DD38FE5FF05%40trustica.cz?utm_medium=email_source=footer>.
> 
> 
> 
> 
> -- 
> You received this message because you are subscribed to the Google
> Groups "Racket Users" group.
> To unsubscribe from this group and stop receiving emails from it,
> send an email to racket-users+unsubscr...@googlegroups.com
> <mailto:racket-users+unsubscr...@googlegroups.com>.
> To view this discussion on the web visit
> 
> https://groups.google.com/d/msgid/racket-users/CAL3TdONGCQyDnCRt42t%2B-KnOBq8XZmU71%2BTUxwkC%3Dw9TJsMb3w%40mail.gmail.com
> 
> <https://groups.google.com/d/msgid/racket-users/CAL3TdONGCQyDnCRt42t%2B-KnOBq8XZmU71%2BTUxwkC%3Dw9TJsMb3w%40mail.gmail.com?utm_medium=email_source=footer>.
> 
> -- 
> You received this message because you are subscribed to the Google
> Groups "Racket Users" group.
> To unsubscribe from this group and stop receiving emails from it, send
> an email to racket-users+unsubscr...@googlegroups.com
> <mailto:racket-users+unsubscr...@googlegroups.com>.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/racket-users/CABNTSaEfeHBnQkb_%3Dq2E2t6%3DD3_PEcMrN7cbSXCZza%3DXjFVfvA%40mail.gmail.com
> <https://groups.google.com/d/msgid/racket-users/CABNTSaEfeHBnQkb_%3Dq2E2t6%3DD3_PEcMrN7cbSXCZza%3DXjFVfvA%40mail.gmail.com?utm_medium=email_source=footer>.

-- 
You received this message because you are subscribed to the Google Groups 
"Racket Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to racket-users+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/racket-users/07d6669c-8b59-766d-c527-19a8e862246b%40trustica.cz.

Re: [racket-users] trying to use futures for some calculations

[Dominik Pantůček]

Hi Alex,

I finally got to investigate the issue in depth. And there are two major
problems blocking your implementation from running the futures in parallel.

1) Allocations of boxed flonums. I tried to get rid of those by
allocating "scratchpad" flvectors and mapping everything onto them. The
future scheduling started to look different, however there are many
allocations because of your loops in functions cp3-futures and
evaluate-cost.

I didn't finish the work though, because I noticed another strange
thing. The Typed Racket code in the lambda function returned by spline
(your mmax-fn argument) blocks almost entirely the parallel execution on
... type checks.

2) All those =, < and friends just block it.

So how to fix this? Well, it is relatively easy albeit quite a lot of
manual work.

(I am looking only at the cp3-futures function when talking about
possible improvements).

* Change the all the for loops for let loop forms and use preallocated
flvectors to hold all the values.
* Switch to racket/unsafe/ops for everything inside futures (this is not
necessary, but takes away a few possible surprises)
* Restructuralize the way you split the work into 30 futures and just
use a binary tree of futures as suggested earlier.
* Use racket/unsafe/ops from regular racket to implement the spline
interpolation. I would also move the coefficients into one huge flvector
and forgot about lists at all. This is very specific workload.

And do not worry about GCs. Once you get rid of all allocations inside
futures, the GCs will disappear.

Also bear in mind that my suggestions are rather low level. By following
them you will get the algorithm to scale over multiple cores if you
really need it. Just remember it will be at a huge cost to readability.
You can slightly mitigate this by some refactoring and custom syntax,
but that is even more work and I would really consider whether you need
the parallelism for a computation that takes a few seconds anyway.

Of course, if you plan to use your algorithm for much bigger data set,
this is the way to go (including the custom syntax).


Cheers,
Dominik

On 17. 06. 20 10:50, Alex Harsanyi wrote:
> 
> I am trying to speed up an algorithm using futures, but I am getting
> some unexpected results (and no real speed improvements), and I was
> wondering if someone more experienced could have a look a the code and
> tell me what am I doing wrong.
> 
> I put up the code in this repository:
> https://github.com/alex-hhh/cp3-exeriments, unfortunately it is the
> simplest meaningful example that I can come up with.  Most of the
> functions, however are just support functions and there are six
> implementation of the same algorithm.
> 
> Basically, the problem I am trying to solve is to fit a model to
> existing data and this is done by evaluating 2.5 million combinations of
> parameters.  My best, non-futures based, algorithm can do this in about
> 3 seconds (8 seconds in DrRacket).
> 
> Given that each of this 2.5 million combination is completely
> independent from the others, they could all be done in parallel.  Given
> this, I "sliced" the combinations into 30 groups and tried to perform
> each "slice" in its own future and select the best among the 30 results
> produced by these futures.
> 
> Unfortunately, while the futures versions of the algorithm produce the
> correct result, the algorithm runs at the same speed as the non-futures
> version.  My `processor-count` returns 8, so I would expect at least
> some level of parallelism.
> 
> As a first step, I tried using `would-be-future`, to see if it reported
> any operations which might block, but nothing was printed out.
> 
> I also tried using the futures visualized, and I found the following:
> 
> * the code appears to be blocking on primitive operations, such as +, -,
> < etc.
> * I suspect these operations are inside the code generated by the `for`
> loops, so I am not sure how to remove them without making the code even
> more difficult to read.
> * there seems to be a lot more time spent in the garbage collector when
> running the futures visualizer than without it (DrRacket runs with
> unlimited memory)
> 
> So I am wondering if someone who is more familiar with futures can look
> at the code and provide some hints about what can be done to make this
> code run in parallel (or if it cannot, I would like to understand why).
> 
> This is already a long message, so I will not add further details here,
> but the repository at https://github.com/alex-hhh/cp3-exeriments has an
> explanation of what every function does, and I am happy to provide
> further clarifications if needed.
> 
> Thanks,
> Alex.
> 
> -- 
> You received this message because you are subscribed to the Google
> Groups "Racket Users" group.
> To unsubscribe from this group and stop receiving emails from it, send
> an email to racket-users+unsubscr...@googlegroups.com
> .
> To view this discussion on the web visit

Re: [racket-users] Re: trying to use futures for some calculations

[Dominik Pantůček]

I've looked at it only briefly (it's the end of the semester and grading
is due soon).

> 
> 
> I would *love* to be proven wrong on this, but I think it's rare to
> be able to get decent parallelization in practice using futures. You
> may have better results using places, but it will depend on how the
> amount of processing for a unit compares to the overhead of
> communicating with the places i.e. you may get better results with 2
> places than with 8 due to place communication overhead. In your
> case, if it's easy for the places to input their own sets of
> parameters, then the place overhead may be small since I think each
> place would simply need to communicate its best value.
> 

This is not even remotely true, I am using futures to get 100%
utilization on all cores available. The current situation is that it
takes quite some effort to leverage futures to get there.

A few generic remarks first. Arbitrary partitioning does not work well
with futures. I always partition the work based on the processor-count
with something like:

(define futures-depth (make-parameter (inexact->exact (ceiling (log
(processor-count) 2)

(define-syntax (define-futurized stx)
  (syntax-case stx ()
((_ (proc start end) body ...)
 #'(begin
 (define max-depth (futures-depth))
 (define (proc start end (depth 0))
   (cond ((fx< depth max-depth)
  (define mid (fx+ start (fxrshift (fx- end start) 1)))
  (let ((f (future
(λ ()
  (proc start mid (fx+ depth 1))
(proc mid end (fx+ depth 1))
(touch f)))
 (else
  body ...)))

Of course all those fx+, fx- and fx< must be unsafe versions from
racket/unsafe/ops.

Second problem is the allocation of flonums. The inner part of the loop
looks like even with flonums inlining it triggers the allocator more
than often. With CS just forget about this before the inlined flonums
are merged. In the meantime, you can drop the for/fold and use flvector
to store and accumulate whatever you need. Using futures-vizualizer is a
good start.

I'll look into it later this week. But generally you need to stick to
unsafe ops and avoid the allocator.

> 
> 
> While this may be true, it is also the case that the design of futures
> is such that incremental work on the primitives turns into incremental
> ability to parallelize programs. So while it is likely to be more work
> today, it may also be the case that people putting effort in to help
> their own programs will help us turn the corner here. Perhaps this is a
> place where an interested contributor can help us out a lot!

It is on the list :)


Dominik

-- 
You received this message because you are subscribed to the Google Groups 
"Racket Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to racket-users+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/racket-users/16fc95cf-5e46-50b6-2d9d-0e336ff1be37%40trustica.cz.

[racket-users] Possible bug in vector-cas!

[Dominik Pantůček]

Hello Racketeers,

it didn't take long before I hit another strange behavior when
extensively using futures. The setup is relatively simple: I am
processing a large fxvector/flvector and I am processing it on a
line-by-line basis (the vectors themselves are width*height in nature).
To avoid races I setup a (make-vector height #f) to serve as a vector of
locks. Locking is simple:

(let lloop ()
  (when (not (vector-cas! locks y #f #t))
(lloop)))

.. do something ...

(vector-set! locks y #f)

Of course, the code in question is in 8 futures on my CPU and they are
scheduled in parallel pretty consistently.

Most of the time it works fine, but sometimes it just hangs with
multiple CPU cores being utilized at 100% (I assume they are in the
busy-wait "lloop").

The problem is I cannot isolate a minimal working example from that. The
code in question is about 3000 LoC and the issue expresses itself only
in about 1 in 30 times just after the program has started. The GUI
thread is running and there are lots of flonum operations and lots of
thrown away flonum results (meaning GC gets triggered quite often).

Also I cannot trigger the same issue with unsafe-vector*-cas! - I'll
look into the differences in the source.

I see this with latest Racket 3m.

Any suggestions how to approach finding the source of the problem are -
as always - very welcome.


Cheers,
Dominik

-- 
You received this message because you are subscribed to the Google Groups 
"Racket Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to racket-users+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/racket-users/0e6de88c-1d39-38fa-ea69-2043c4f4071b%40trustica.cz.

Re: [racket-users] Hunting a possible fsemaphore-post/wait bug

[Dominik Pantůček]

On 24. 05. 20 3:38, Matthew Flatt wrote:
> At Sat, 23 May 2020 18:51:23 +0200, Dominik Pantůček wrote:
>> But that is just where the issue is showing up. The real question is how
>> the counter gets decremented twice (given that fsemaphores should be
>> futures-safe).
> 
> I found a configuration that triggered the bug often enough on my
> machine. Yes, it was a dumb mistake in the implementation of
> fsemaphores. In your example, it wasn't so much that the counter
> fsemaphore was decremented twice, but that the lock fsemaphore could go
> negative --- so it didn't actually lock.
> 
> I've pushed a repair.

Thank you for the repair (and good idea with the test case).

> 
> 
> FWIW, after looking at this example more, I see why it's still not
> enough in principle to make a thread that waits on the result of
> `do-start-workers` or to run `do-start-workers` after the enqueue
> loops. The `start-workers` function can run a dequeue loop after
> starting a future to do the same, and before touching that future. So,
> the dependencies aren't as linear as I thought before.
> 
> If your real code looks anything like this, consider using a CAS
> operation, like `box-cas!`, instead of an fsemaphore as lock for the
> queue. The queue's use of an fsemaphore for counting and signaling
> seems fine, though.
> 

Yes, the real code is a binary tree of futures. However each future
performs a lot of fixnum/flonum operations.

At first I was surprised that you are basically suggesting using
spinlocks (busy-wait loops) instead of futex-backed (at least on Linux)
fsemaphores. That is a waste of CPU time... But yes, CAS-based locking
outperforms fsemaphores locking and distributes the work way more
evenly. Now when I see it in action and look at the source it kind of
makes sense as the spinlock does not need to wait long and does not
cause any re-scheduling (which is the reason why I get so uneven counts
with fsemaphore-based locking).

> In any case, the example worked well to expose the fsemaphore bug.
> 

Probably expect to see more in the future :)



Dominik

-- 
You received this message because you are subscribed to the Google Groups 
"Racket Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to racket-users+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/racket-users/9fdcb07a-a596-fc84-5529-4517f6b8b1fd%40trustica.cz.

Re: [racket-users] Hunting a possible fsemaphore-post/wait bug

[Dominik Pantůček]

On 23. 05. 20 19:24, Matthew Flatt wrote:
> I'm not sure this is the problem that you're seeing, but I see a
> problem with the example. It boils down to the fact that futures do not
> provide concurrency.
> 
> That may sound like a surprising claim, because the whole point of
> futures is to run multiple things at a time. But futures merely offer
> best-effort parallelism; they do not provide any guarantee of
> concurrency.

It might be surprising as well - but that is exactly the reason I am
using futures for this task. Up until now I always built a data set and
then have setup a futures tree to process that data set - possibly in
parallel. But with no assumptions about whether it will happen in parallel.

What I am trying to achieve now is to allow the futures scheduling start
speculatively working on the data set while it is being created. And
yes, I don't want to make any assumptions about it actually starting or
even being run in parallel. I just want to provide the futures runtime
with the best setup, so it can also do the best.

Using os-threads with explicit scheduling is something I want to
investigate as well - but generally I do not see why this shouldn't work
as expected (with the great advantage that it can fallback on
program-level single-thread and not OS-run threads interleaved on the
same core transparent to the program).

> 
> As a consequence, trying to treat an fsemaphore as a lock can go wrong.
> If a future manages to take an fsemaphore lock, but the future is not
> demanded by the main thread --- or in a chain of future demands that
> are demanded by the main thread --- then nothing obliges the future to
> continue running; it can hold the lock forever.

Duly noted. I'll look into that possibility. But honestly, I've never
encountered such behavior.

> 
> (I put the blame on femspahores. Adding fsemaphores to the future
> system was something like adding mutation to a purely functional
> language. The addition makes certain things possible, but it also
> breaks local reasoning that the original design was supposed to
> enable.)

I understand and - another surprise - I agree. The example is very
"imperative", but that is really bare-bones example of the problem.

> 
> In your example program, I see
> 
>  (define workers (do-start-workers))
>  (displayln "started")
>  (for ((i 1))
>(mfqueue-enqueue! mfq 1))
> 
> where `do-start-workers` creates a chain of futures, but there's no
> `touch` on the root future while the loop calls `mfqueue-enqueue!`.
> Therefore, the loop can block on an fsemaphore because some future has
> taken the lock but stopped running for whatever reason.
> 
> In this case, adding `(thread (lambda () (touch workers)))` before the
> loop after "started" might fix the example. In other words, you can use
> the `thread` concurrency construct in combination with the `future`
> parallelism construct to ensure progress. I think this will work
> because all futures in the program end up in a linear dependency chain.
> If there were a tree of dependencies, then I think you'd need a
> `thread` for each `future` to make sure that every future has an active
> demand.

The same thing happens if I (do-start-workers) after filling the queue
using mfqueue-enqueue!.

> 
> If you're seeing a deadlock at the `(touch workers)`, though, my
> explanation doesn't cover what you're seeing. I haven't managed to
> trigger the deadlock myself.

Should I open an issue at github and provide the gdb backtraces?

Btw, none of this behaviour is seen with Racket CS (so far) and the
observed futures scheduling is speculative very aggresively with CS
variant (which is definitely what I want).

I am running 3m v7.7.0.6 compiled from sources on Ubuntu 20.04 if that
helps.



And (as usual) - thank you very much!

Dominik

-- 
You received this message because you are subscribed to the Google Groups 
"Racket Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to racket-users+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/racket-users/393d7b45-38bb-8384-a34c-7bacbd517424%40trustica.cz.

[racket-users] Hunting a possible fsemaphore-post/wait bug

[Dominik Pantůček]

Hello again with futures!

I started working on futures-based workers and got quickly stuck with a
dead-lock I think does not originate in my code (although it is two
semaphores, 8 futures, so I'll refrain from strong opinions here).

I implemented a very simple futures-friendly queue using mutable pairs
and created a minimal-deadlocking-example[1]. I am running racket 3m
7.7.0.4 which includes fixes for the futures-related bugs I discovered
recently.

Sometimes the code just runs fine and shows the numbers of worker
iterations performed in different futures (as traced by the 'fid'
argument). But sometimes it locks in a state where there is one last
number in the queue (0 - zero) and yet the fsemaphore-count for the
count fsemaphore returns 0. Which means the semaphore was decremented
twice somewhere. The code is really VERY simple and I do not see a
race-condition within the code, that would allow any code path to
decrement the fsema-count fsemaphore twice once the worker future
receives 0.

I am able to reproduce the behavior with racket3m running under gdb and
get the stack traces for all the threads pretty consistently. The
deadlock is apparently at:

  2Thread 0x77fca700 (LWP 46368) "mfqueue.rkt"
futex_wait_cancelable (private=, expected=0,
futex_word=0x559d8e78) at ../sysdeps/nptl/futex-internal.h:183

But that is just where the issue is showing up. The real question is how
the counter gets decremented twice (given that fsemaphores should be
futures-safe).

Any hints would be VERY appreciated!


Cheers,
Dominik

[1] http://pasterack.org/pastes/28883

-- 
You received this message because you are subscribed to the Google Groups 
"Racket Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to racket-users+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/racket-users/5dcf1260-e8bf-d719-adab-5a0fd9378075%40trustica.cz.

Re: [racket-users] Cross-building Racket applications for 64bit Windows on Linux

[Dominik Pantůček]

I'll try with 3m. I am a bit skeptical here - as the resulting
program.exe is about 150M when properly build and apparently has to
include a lots of native DLLs. But reading the documentation (again)
makes me think that generally it should work.

Technically the byte-code should be platform-independent for 3m and then
it is just a matter of pre-prending the right binary and packing-in the
libraries. I will definitely look into that.


Thank you!

Dominik

On 23. 05. 20 18:38, Matthew Flatt wrote:
> Thinking about this a little more, I don't think all the pieces are in
> place for CS. At least, I don't remember setting up all the pieces...
> 
> The extra piece needed for Racket CS is a Chez Scheme cross-compiler
> that runs on Linux but produces machine code for Windows. That can be
> done --- and it is done, because our distributions for Windows are
> built on Linux (without wine). But I don't think the cross compilers
> are stashed anywhere right now, and we'll need to offer NxM cross
> compilers for N host systems and M target systems.
> 
> So, if building on a Linux target for Windows works for BC (which is a
> different question than trying to run BC on wine), then we can start
> thinking about how to fill in the last NxM pieces for CS.
> 
> At Sat, 23 May 2020 10:25:31 -0600, Matthew Flatt wrote:
>> Have you already tried using `raco exe` on Linux (i.e., using Racket
>> for Linux) but generating Windows executables?
>>
>>   https://docs.racket-lang.org/raco/cross-system.html
>>
>>
>> Note that the "tarball" distributions at places like
>>
>>https://download.racket-lang.org/releases/7.7/
>>
>> can be handy for setting up a .
>>
>>
>> At Sat, 23 May 2020 17:39:58 +0200, Dominik Pantůček wrote:
>>> Hello Racketeers,
>>>
>>> although I am developing Racket applications on Linux, our customers are
>>> usually running Windows. The good thing about Racket (and racket/gui
>>> especially) is that it requires virtually no OS-specific code for many -
>>> even non-trivial - tasks. However it is not that straightforward to
>>> produce binaries for different target platform.
>>>
>>> Since last summer, we have been successfully using wine and Racket
>>> Windows builds to create 32bit binaries. However with Racket BC it was
>>> impossible to make it work with 64bit Windows builds. This is still the
>>> case even with latest snapshots and latest wine bundled with Ubuntu 20.04.
>>>
>>> It turns out, that Racket CS works flawlessly under wine - including
>>> raco.exe exe. And with the latest patches[1] (thanks Matthew and
>>> DaLynX), it is possible to produce working 64bit Windows binaries from
>>> Racket programs on Linux using wine.
>>>
>>> As my work is heavily CPU- and memory-bound, the 32bit address space
>>> limit was quite a limit.
>>>
>>> However, once we tried to embed the build process into our
>>> (Gitlab-based) CI/CD, we failed miserably. Combining wine and Racket
>>> windows builds (7.7.0.6 snapshots from University of Utah) turned out to
>>> be a terrible nightmare under Docker.
>>>
>>> On my workstation, it is absolutely straightforward:
>>>
>>> rm -fr ~/.wine # Do not try this without backing up, if you need it
>>> wine racket-7.7.0.6-x86_64-win32-cs.exe
>>>
>>> Now just click next next next with the installer and you are ready.
>>> Building a 64bit Windows binary is just a matter of single:
>>>
>>> wine ~/.wine/drive_c/Program\ Files/Racket-7.7.0.6/raco.exe exe --gui
>>> --embed-dlls program.rkt
>>>
>>> Of course, any packages needed have to be installed using raco.exe pkg
>>> in the same wine prefix.
>>>
>>> But when you take the ~/.wine prefix and try to run it under docker,
>>> nothing works at all and tracing what is happening is extremely
>>> difficult with very cryptic results.
>>>
>>> We ruled out X11 dependency - raco.exe works just fine after I `unset
>>> DISPLAY` on my workstation. And the results in Docker are no different
>>> with Xvfb installed and running. We got to a point where the raco.exe
>>> actually starts as a process under wine in Docker and it silently fails
>>> upon startup.
>>>
>>> Any ideas where to look and what to test? I strongly assume Racket is
>>> not the root cause of the problem here, but rather the combination of
>>> wine under Docker. However, Racket sometimes uses an OS-specific
>>> trickery so I am sort of hoping that the described behav

[racket-users] Cross-building Racket applications for 64bit Windows on Linux

[Dominik Pantůček]

Hello Racketeers,

although I am developing Racket applications on Linux, our customers are
usually running Windows. The good thing about Racket (and racket/gui
especially) is that it requires virtually no OS-specific code for many -
even non-trivial - tasks. However it is not that straightforward to
produce binaries for different target platform.

Since last summer, we have been successfully using wine and Racket
Windows builds to create 32bit binaries. However with Racket BC it was
impossible to make it work with 64bit Windows builds. This is still the
case even with latest snapshots and latest wine bundled with Ubuntu 20.04.

It turns out, that Racket CS works flawlessly under wine - including
raco.exe exe. And with the latest patches[1] (thanks Matthew and
DaLynX), it is possible to produce working 64bit Windows binaries from
Racket programs on Linux using wine.

As my work is heavily CPU- and memory-bound, the 32bit address space
limit was quite a limit.

However, once we tried to embed the build process into our
(Gitlab-based) CI/CD, we failed miserably. Combining wine and Racket
windows builds (7.7.0.6 snapshots from University of Utah) turned out to
be a terrible nightmare under Docker.

On my workstation, it is absolutely straightforward:

rm -fr ~/.wine # Do not try this without backing up, if you need it
wine racket-7.7.0.6-x86_64-win32-cs.exe

Now just click next next next with the installer and you are ready.
Building a 64bit Windows binary is just a matter of single:

wine ~/.wine/drive_c/Program\ Files/Racket-7.7.0.6/raco.exe exe --gui
--embed-dlls program.rkt

Of course, any packages needed have to be installed using raco.exe pkg
in the same wine prefix.

But when you take the ~/.wine prefix and try to run it under docker,
nothing works at all and tracing what is happening is extremely
difficult with very cryptic results.

We ruled out X11 dependency - raco.exe works just fine after I `unset
DISPLAY` on my workstation. And the results in Docker are no different
with Xvfb installed and running. We got to a point where the raco.exe
actually starts as a process under wine in Docker and it silently fails
upon startup.

Any ideas where to look and what to test? I strongly assume Racket is
not the root cause of the problem here, but rather the combination of
wine under Docker. However, Racket sometimes uses an OS-specific
trickery so I am sort of hoping that the described behavior rings a bell
for someone :)


Cheers,
Dominik

[1]
https://github.com/racket/racket/commit/61cefe693a047e22ca44752eafb9eb9e2e65409f

-- 
You received this message because you are subscribed to the Google Groups 
"Racket Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to racket-users+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/racket-users/6cd41db0-a1f1-916a-edaf-20d9d0597f68%40trustica.cz.

[racket-users] JIT and futures on aarch64

[Dominik Pantůček]

Hello fellow racketeers,

after pushing futures on x86 and x86_64 to their limits (and helping
fixing two bugs), I turned my focus on ARM. Apparently everything should
work with 32bit arm without any hurdles (I am going to test that later
today), however on aarch64 (64bit arm), there is no JIT and therefore no
futures.

A quick look at the implementation hints that being stuck with forked
GNU Lightning 1.2 is the major problem here. Apparently latest versions
of Lightning support aarch64.

I am willing to invest some time into porting that, but the question is
how much work is needed. Also - is it relevant for CS variant? (I would
guess not). And of course, how to do the actual porting - it seems to me
the Racket's port is more than just a few minor changes. Maybe porting
only the aarch64 part might be a reasonable way to go?

Any suggestions would be really appreciated.


Cheers,
Dominik

-- 
You received this message because you are subscribed to the Google Groups 
"Racket Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to racket-users+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/racket-users/8e43f493-df5d-e6bb-e7f5-c8bc67dd891c%40trustica.cz.

Re: [racket-users] Another futures-related bug hunt

[Dominik Pantůček]

Hello,



I'll add a lock at lines 1092-1096 of "newgc.c", and we'll see if that
helps.


should I open the issue or will you do it? (Speaking of race conditions...).

I'll re-run the tests with the lock once it is in the repo - sometimes 
it takes hours for this bug to exhibit and with 8 HTs the process in 
question consumes slightly more than 500% of CPU time - which means the 
computer sounds it's going to take off and fly. I'll keep it up and 
running overnight again.



And thank you for the explanation, digging in Racket internals has a 
very varying degree of difficulty :)



Cheers,
Dominik

--
You received this message because you are subscribed to the Google Groups "Racket 
Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to racket-users+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/racket-users/606315e6-edd3-1ab5-48b2-770b6fd79893%40trustica.cz.

Re: [racket-users] Another futures-related bug hunt

[Dominik Pantůček]

Hello,



The most useful information here is likely to be a stack trace from
each OS-level thread at the point where the application is stuck.



would this be enough to open an issue for that?

(gdb) info threads
  Id   Target IdFrame
* 1Thread 0x77c1b300 (LWP 19075) "tut22.rkt" 
mark_backpointers (gc=gc@entry=0x559d10c0) at 
../../../racket/gc2/newgc.c:4078
  2Thread 0x77fcb700 (LWP 19076) "tut22.rkt" 
futex_wait_cancelable (private=, expected=0, 
futex_word=0x559d7d78)

at ../sysdeps/unix/sysv/linux/futex-internal.h:80
  3Thread 0x7fffe65a6700 (LWP 19077) "gmain" 
0x77d34c2f in __GI___poll (fds=0x55b82520, nfds=2, 
timeout=-1) at ../sysdeps/unix/sysv/linux/poll.c:29
  4Thread 0x7fffe5da5700 (LWP 19078) "gdbus" 
0x77d34c2f in __GI___poll (fds=0x55b94ce0, nfds=3, 
timeout=-1) at ../sysdeps/unix/sysv/linux/poll.c:29
  7Thread 0x7fffd77fe700 (LWP 19082) "dconf worker" 
0x77d34c2f in __GI___poll (fds=0x55e9e5e0, nfds=1, 
timeout=-1) at ../sysdeps/unix/sysv/linux/poll.c:29
  8Thread 0x7fffe40d4800 (LWP 19083) "tut22.rkt" 
futex_wait_cancelable (private=, expected=0, 
futex_word=0x559c499c)

at ../sysdeps/unix/sysv/linux/futex-internal.h:80
  9Thread 0x7fffd4602800 (LWP 19084) "tut22.rkt" 
futex_wait_cancelable (private=, expected=0, 
futex_word=0x559c499c)

at ../sysdeps/unix/sysv/linux/futex-internal.h:80
  10   Thread 0x7fffd4586800 (LWP 19085) "tut22.rkt" 
futex_wait_cancelable (private=, expected=0, 
futex_word=0x559c499c)

at ../sysdeps/unix/sysv/linux/futex-internal.h:80
  11   Thread 0x7fffd450a800 (LWP 19086) "tut22.rkt" 
futex_wait_cancelable (private=, expected=0, 
futex_word=0x559c499c)

at ../sysdeps/unix/sysv/linux/futex-internal.h:80
  12   Thread 0x7fffd448e800 (LWP 19087) "tut22.rkt" 
futex_wait_cancelable (private=, expected=0, 
futex_word=0x559c499c)

at ../sysdeps/unix/sysv/linux/futex-internal.h:80
  13   Thread 0x7fffd4412800 (LWP 19088) "tut22.rkt" 
futex_wait_cancelable (private=, expected=0, 
futex_word=0x559c499c)

at ../sysdeps/unix/sysv/linux/futex-internal.h:80
  14   Thread 0x7fffd4396800 (LWP 19089) "tut22.rkt" 
futex_wait_cancelable (private=, expected=0, 
futex_word=0x559c499c)

at ../sysdeps/unix/sysv/linux/futex-internal.h:80
  15   Thread 0x7fffd431a800 (LWP 19090) "tut22.rkt" 
futex_wait_cancelable (private=, expected=0, 
futex_word=0x559c499c)

at ../sysdeps/unix/sysv/linux/futex-internal.h:80
  16   Thread 0x765b2800 (LWP 21691) "tut22.rkt" 
futex_wait_cancelable (private=, expected=0, 
futex_word=0x559c4998)

at ../sysdeps/unix/sysv/linux/futex-internal.h:80
(gdb) bt
#0  0x557f5064 in mark_backpointers (gc=gc@entry=0x559d10c0) 
at ../../../racket/gc2/newgc.c:4078
#1  0x557edb2b in garbage_collect (gc=gc@entry=0x559d10c0, 
force_full=force_full@entry=0, no_full=no_full@entry=0, 
switching_master=switching_master@entry=0, lmi=lmi@entry=0x0)

at ../../../racket/gc2/newgc.c:5646
#2  0x557f0ff2 in collect_now (nomajor=, 
major=, gc=) at 
../../../racket/gc2/newgc.c:875
#3  0x557f0ff2 in collect_now (gc=0x559d10c0, major=0, 
nomajor=0) at ../../../racket/gc2/newgc.c:855
#4  0x557f9124 in allocate_slowpath (newptr=, 
allocate_size=, gc=) at 
../../../racket/gc2/newgc.c:1607
#5  0x557f9124 in allocate (type=1, request_size=out>) at ../../../racket/gc2/newgc.c:1671
#6  0x557f9124 in allocate (type=, 
request_size=) at ../../../racket/gc2/newgc.c:1636
#7  0x557f9124 in GC_malloc_atomic (s=) at 
../../../racket/gc2/newgc.c:1792
#8  0x557f9124 in GC_malloc_atomic (s=) at 
../../../racket/gc2/newgc.c:1792
#9  0x55605406 in prepare_retry_alloc (p2=, 
p=) at ../../../racket/gc2/../src/jitalloc.c:47
#10 0x55605406 in ts_prepare_retry_alloc (p=, 
p2=) at ../../../racket/gc2/../src/jitalloc.c:73

#11 0x77fbe62b in  ()
#12 0x in  ()
(gdb) thread 2
[Switching to thread 2 (Thread 0x77fcb700 (LWP 19076))]
#0  futex_wait_cancelable (private=, expected=0, 
futex_word=0x559d7d78) at ../sysdeps/unix/sysv/linux/futex-internal.h:80

80  ../sysdeps/unix/sysv/linux/futex-internal.h: No such file or directory.
(gdb) bt
#0  0x77e202c6 in futex_wait_cancelable (private=out>, expected=0, futex_word=0x559d7d78) at 
../sysdeps/unix/sysv/linux/futex-internal.h:80
#1  0x77e202c6 in __pthread_cond_wait_common (abstime=0x0, 
clockid=0, mutex=0x559d7d28, cond=0x559d7d50) at 
pthread_cond_wait.c:508
#2  0x77e202c6 in __pthread_cond_wait 
(cond=cond@entry=0x559d7d50, mutex=mutex@entry=0x559d7d28) at 
pthread_cond_wait.c:638
#3  0x557209fe in green_thread_timer 
(data=data@entry=0x559d7d10) at ../../../racket/gc2/../src/port.c:6659
#4  0x556bb8be in mzrt_thread_stub (data=0x559d7dc0) at 

Re: [racket-users] Another futures-related bug hunt

[Dominik Pantůček]

Hello,

On 08. 05. 20 14:27, Matthew Flatt wrote:

At Fri, 8 May 2020 09:34:32 +0200, Dominik Pantůček wrote:

Apart from obvious strace (after freeze) and gdb (before/after freeze)
debugging to find possible sources of this bug, is there even a remote
possibility of getting any clue how can this happen based on the
information gathered so far? My thought go along the lines:

* flonums are boxed - but for some operations they may be immediate
* apparently it is a busy-wait loop in RTT, otherwise 100% CPU usage is
impossible with this workload
* unsafe ops are always suspicious, but again, the problem shows up even
when I switch to the safe versions - it just takes longer time
* which means, the most probable cause is a race condition


The most useful information here is likely to be a stack trace from
each OS-level thread at the point where the application is stuck.

That could potentially tell us, for example, that it's a problem with
synchronization for a GC (where one of the OS threads that run futures
doesn't cooperate for some reason) or a problem with a the main thread
performing some specific work on a future thread's behalf.



I am using the build from master branch with the patch for #3145 and 
cannot make it run under gdb:


$ gdb ../racket-lang/racket/racket/bin/racket
GNU gdb (Ubuntu 8.3-0ubuntu1) 8.3
Copyright (C) 2019 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later 
<http://gnu.org/licenses/gpl.html>

This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.
Type "show copying" and "show warranty" for details.
This GDB was configured as "x86_64-linux-gnu".
Type "show configuration" for configuration details.
For bug reporting instructions, please see:
<http://www.gnu.org/software/gdb/bugs/>.
Find the GDB manual and other documentation resources online at:
<http://www.gnu.org/software/gdb/documentation/>.

For help, type "help".
Type "apropos word" to search for commands related to "word"...
Reading symbols from ../racket-lang/racket/racket/bin/racket...
(No debugging symbols found in ../racket-lang/racket/racket/bin/racket)
(gdb) run
Starting program: 
/home/joe/Projects/Programming/racket-lang/racket/racket/bin/racket

[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".
Welcome to Racket v7.7.0.4.
[New Thread 0x77fcb700 (LWP 6410)]

Thread 1 "racket" received signal SIGSEGV, Segmentation fault.
0x555e14fe in scheme_gmp_tls_unload ()
(gdb)

The same happens for the binary with debug symbols:

 gdb ../racket-lang/racket/racket/src/build/racket/racket3m
GNU gdb (Ubuntu 8.3-0ubuntu1) 8.3
Copyright (C) 2019 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later 
<http://gnu.org/licenses/gpl.html>

This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.
Type "show copying" and "show warranty" for details.
This GDB was configured as "x86_64-linux-gnu".
Type "show configuration" for configuration details.
For bug reporting instructions, please see:
<http://www.gnu.org/software/gdb/bugs/>.
Find the GDB manual and other documentation resources online at:
<http://www.gnu.org/software/gdb/documentation/>.

For help, type "help".
Type "apropos word" to search for commands related to "word"...
Reading symbols from 
../racket-lang/racket/racket/src/build/racket/racket3m...

(gdb) run
Starting program: 
/home/joe/Projects/Programming/racket-lang/racket/racket/src/build/racket/racket3m 


[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".
Welcome to Racket v7.7.0.4.
[New Thread 0x77fcb700 (LWP 6422)]

Thread 1 "racket3m" received signal SIGSEGV, Segmentation fault.
scheme_gmp_tls_unload (s=0x76114480, data=0x0) at 
../../../racket/gc2/../src/gmp/gmp.c:5822

5822  s[0] = 0;
(gdb)

I am running Ubuntu 19.10's default gdb:

$ gdb --version
GNU gdb (Ubuntu 8.3-0ubuntu1) 8.3

I assume gmp is used for bignum implementation (didn't check yet), so it 
might be relevant as well:


ii  libgmp-dev:amd64   2:6.1.2+dfsg-4 
  amd64Multiprecision arithmetic library 
developers tools
ii  libgmp10:amd64 2:6.1.2+dfsg-4 
  amd64Multiprecision arithmetic library
ii  libgmp10:i386  2:6.1.2+dfsg-4 
  i386 Multiprecision arithmetic library
ii  libgmpxx4ldbl:amd642:6.1.2+dfsg-4 
  amd64Multiprecision arithmetic library (C++ 
bindings)
ii  python-gmpy:amd64 

[racket-users] Another futures-related bug hunt

[Dominik Pantůček]

Hello fellow Racketeers,

my spare-time out-of-curiosity venture into using HPR (High-Performance 
Racket) for creating a software 3D rendering pipeline seems to be 
pushing the futures into rough edges.


The scenario is sort of "usual":

* 7 futures + 1 in RTT that form a binary tree
* GUI thread running

But this time, the futures perform not only data-heavy fixnums 
operations, but flonums operations as well.


Something along the lines of 2560x1440 fixnums and the same number of 
flonums is being handled in 8 threads effectively (give or take some 
optimizations that slightly lower the 1440 height usually).


The code in question is relatively short - say 60 lines of code - 
however it does not make much sense without the remaining 2k lines :)


If the operation runs without futures in RTT, nothing happens. But under 
a heavy load and VERY varying amount of time (seconds to hours), it 
completely freezes with:


* 1 CPU being used at 100% (top/htop shows)
* Does not handle socket operations (X11 WM message for closing the window)
* Does not respond to keyboard (or via kill) SIGINT
* Can only be forcibly stopped by SIGKILL (or similar) or forcefully 
closing the window from WM which sort of gets handled probably in the 
lower-level parts of GDK completely without Racket runtime intervention 
(just prints Killed and the exit code is 137)


Based on these observations I can only conclude that it is the RTT that 
gets stuck - but that is only the native thread perspective. From Racket 
thread perspective, it can be either the "main" application thread that 
is in (thread-wait) for the thread that performs the futures stuff and 
it can also be the GUI thread which is created with parameterizing the 
eventspace (that is just some trickery to allow me to send breaks when I 
receive window close event).


Apart from obvious strace (after freeze) and gdb (before/after freeze) 
debugging to find possible sources of this bug, is there even a remote 
possibility of getting any clue how can this happen based on the 
information gathered so far? My thought go along the lines:


* flonums are boxed - but for some operations they may be immediate
* apparently it is a busy-wait loop in RTT, otherwise 100% CPU usage is 
impossible with this workload
* unsafe ops are always suspicious, but again, the problem shows up even 
when I switch to the safe versions - it just takes longer time

* which means, the most probable cause is a race condition

And that is basically all I can tell right now.

Of course, any suggestions would be really welcome.

Cheers,
Dominik

P.S.: I am really curious, what will I find when I finally put 
fsemaphores into the mix...





--
You received this message because you are subscribed to the Google Groups "Racket 
Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to racket-users+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/racket-users/ca40f468-53c7-6fd2-4e7f-0d963e931a60%40trustica.cz.

Re: [racket-users] Futures + threads SIGSEGV

[Dominik Pantůček]

Hi Sam,

On 02. 05. 20 14:26, Sam Tobin-Hochstadt wrote:
> I successfully reproduced this on the first try, which is good. Here's
> my debugging advice (I'm also looking at it):
> 
> 1. To use a binary with debugging symbols, use
> `racket/src/build/racket/racket3m` from the checkout of the Racket
> repository that you built.
> 2. When running racket in GDB, there are lots of segfaults because of
> the GC; you'll want to use `handle SIGSEGV nostop noprint`
> 3. It may not work for this situation because of parallelism, but if
> you can reproduce the bug using `rr` [1] it will be almost infinitely
> easier to find and fix.

thanks for the hints and also thanks for opening the Github issue for
that. I'll try to post my results (if any) there.

> 
> I'm also curious about your experience with Racket CS and futures.
> It's unlikely to have the _same_ bugs, but it would be good to find
> the ones there are. :)

This is going to be a really hard one. With all the tricks I learned
during past weeks, I get almost 400 frames per second with my experiment
using 3m and unsafe operations. Without unsafe operations it goes down
to 300 and without unsafe operations and with the de-optimized flip
function as shown in the example + set-argb-pixels, I am at about 50 fps
(that is presumably a completely "safe" version without relying on my
bounds and type checking).

With CS, I am unable to get quickly working anything else than the
de-optimized version with set-argb-pixels and I am at about 5 fps. Also,
the thread scheduling is "interesting" at best. I am postponing the work
on that - I sort of assume, that it can take another few weeks to
understand how to properly use all the fixnum/flonum related stuff with CS.


Thanks again!
Dominik


> 
> [1] https://rr-project.org
> 
> On Sat, May 2, 2020 at 7:56 AM Dominik Pantůček
>  wrote:
>>
>> Hello fellow Racketeers,
>>
>> during my research into how Racket can be used as generic software
>> rendering platform, I've hit some limits of Racket's (native) thread
>> handling. Once I started getting SIGSEGVs, I strongly suspected I am
>> doing too much unsafe operations - and to be honest, that was true.
>> There was one off-by-one memory access :).
>>
>> But that was easy to resolve - I just switched to safe/contracted
>> versions of everything and found and fixed the bug. But I still got
>> occasional SIGSEGV. So I dug even deeper (during last two months I've
>> read most of the JIT inlining code) than before and noticed that the
>> crashes disappear when I refrain from calling bytes-set! in parallel
>> using futures.
>>
>> So I started creating a minimal-crashing-example. At first, I failed
>> miserably. Just filling a byte array over and over again, I was unable
>> to reproduce the crash. But then I realized, that in my application,
>> threads come to play and that might be the case. And suddenly, creating
>> MCE was really easy:
>>
>> Create new eventspace using parameterize/make-eventspace, put the actual
>> code in application thread (thread ...) and make the main thread wait
>> for this application thread using thread-wait. Before starting the
>> application thread, I create a simple window, bitmap and a canvas, that
>> I keep redrawing using refresh-now after each iteration. Funny thing is,
>> now it keeps crashing even without actually modifying the bitmap in
>> question. All I need to do is to mess with some byte array in 8 threads.
>> Sometimes it takes a minute on my computer before it crashes, sometimes
>> it needs more, but it eventually crashes pretty consistently.
>>
>> And it is just 60 lines of code:
>>
>> #lang racket/gui
>>
>> (require racket/future racket/fixnum racket/cmdline)
>>
>> (define width 800)
>> (define height 600)
>>
>> (define framebuffer (make-fxvector (* width height)))
>> (define pixels (make-bytes (* width height 4)))
>>
>> (define max-depth 0)
>>
>> (command-line
>>  #:once-each
>>  (("-d" "--depth") d "Futures binary partitioning depth" (set! max-depth
>> (string->number d
>>
>> (file-stream-buffer-mode (current-output-port) 'none)
>>
>> (parameterize ((current-eventspace (make-eventspace)))
>>   (define win (new frame%
>>(label "test")
>>(width width)
>>(height height)))
>>   (define bmp (make-bitmap width height))
>>   (define canvas (new canvas%
>>   (parent win)
>>   (paint-callback
>>(λ (c dc)
>>  

Re: [racket-users] Futures + threads SIGSEGV

[Dominik Pantůček]

Hi Dex,

On 02. 05. 20 14:10, Dexter Lagan wrote:
> Hello,
> 
>   I’ve been getting inconsistent results as well. A while ago I made a
> benchmark based on a parallel spectral norm computation. The benchmark
> works fine on Windows on most systems and uses all cores, but crashes
> randomly on other systems. I haven’t been able to figure out why. On
> Linux it doesn’t seem to use more than one core. I’d be interested to
> know if this is related. Here’s the benchmark code :
> 
> https://github.com/DexterLagan/benchmark

Beware that (processor-count) returns the number of HT-cores, so your
v1.3 is actually requesting twice the number of threads as there are
HTs. At least on Linux this is the case (checked right now).

Interesting idea... 16 threads:

$ time racket crash.rkt -d 4
SIGSEGV MAPERR si_code 1 fault on addr (nil)
Aborted (core dumped)

real6m37,579s
user32m55,192s
sys 0m35,124s

So that is consistent to what I see.

Have you tried using future-visualizer[1] for checking why it uses only
single CPU thread? Last summer I spent quite some time with it to help
me find the right futures usage patterns that actually enable the
speculative computation in parallel. Usually if your code is too deep
and keeps allocating "something" each frame, it goes back to the runtime
thread for each allocation.


Cheers,
Dominik

[1] https://docs.racket-lang.org/future-visualizer/index.html

> 
> Dex
> 
>> On May 2, 2020, at 1:56 PM, Dominik Pantůček
>>  wrote:
>>
>> Hello fellow Racketeers,
>>
>> during my research into how Racket can be used as generic software
>> rendering platform, I've hit some limits of Racket's (native) thread
>> handling. Once I started getting SIGSEGVs, I strongly suspected I am
>> doing too much unsafe operations - and to be honest, that was true.
>> There was one off-by-one memory access :).
>>
>> But that was easy to resolve - I just switched to safe/contracted
>> versions of everything and found and fixed the bug. But I still got
>> occasional SIGSEGV. So I dug even deeper (during last two months I've
>> read most of the JIT inlining code) than before and noticed that the
>> crashes disappear when I refrain from calling bytes-set! in parallel
>> using futures.
>>
>> So I started creating a minimal-crashing-example. At first, I failed
>> miserably. Just filling a byte array over and over again, I was unable
>> to reproduce the crash. But then I realized, that in my application,
>> threads come to play and that might be the case. And suddenly, creating
>> MCE was really easy:
>>
>> Create new eventspace using parameterize/make-eventspace, put the actual
>> code in application thread (thread ...) and make the main thread wait
>> for this application thread using thread-wait. Before starting the
>> application thread, I create a simple window, bitmap and a canvas, that
>> I keep redrawing using refresh-now after each iteration. Funny thing is,
>> now it keeps crashing even without actually modifying the bitmap in
>> question. All I need to do is to mess with some byte array in 8 threads.
>> Sometimes it takes a minute on my computer before it crashes, sometimes
>> it needs more, but it eventually crashes pretty consistently.
>>
>> And it is just 60 lines of code:
>>
>> #lang racket/gui
>>
>> (require racket/future racket/fixnum racket/cmdline)
>>
>> (define width 800)
>> (define height 600)
>>
>> (define framebuffer (make-fxvector (* width height)))
>> (define pixels (make-bytes (* width height 4)))
>>
>> (define max-depth 0)
>>
>> (command-line
>> #:once-each
>> (("-d" "--depth") d "Futures binary partitioning depth" (set! max-depth
>> (string->number d
>>
>> (file-stream-buffer-mode (current-output-port) 'none)
>>
>> (parameterize ((current-eventspace (make-eventspace)))
>>  (define win (new frame%
>>   (label "test")
>>   (width width)
>>   (height height)))
>>  (define bmp (make-bitmap width height))
>>  (define canvas (new canvas%
>>  (parent win)
>>  (paint-callback
>>   (λ (c dc)
>> (send dc draw-bitmap bmp 0 0)))
>>  ))
>>
>>  (define (single-run)
>>    (define (do-bflip start end (depth 0))
>>  (cond ((fx< depth max-depth)
>> (define cnt (fx- end start))
>> (define cnt2 (fxrshift cnt 1))
>> (define mid (fx+ start cnt2))
>>   

[racket-users] Futures + threads SIGSEGV

[Dominik Pantůček]

Hello fellow Racketeers,

during my research into how Racket can be used as generic software
rendering platform, I've hit some limits of Racket's (native) thread
handling. Once I started getting SIGSEGVs, I strongly suspected I am
doing too much unsafe operations - and to be honest, that was true.
There was one off-by-one memory access :).

But that was easy to resolve - I just switched to safe/contracted
versions of everything and found and fixed the bug. But I still got
occasional SIGSEGV. So I dug even deeper (during last two months I've
read most of the JIT inlining code) than before and noticed that the
crashes disappear when I refrain from calling bytes-set! in parallel
using futures.

So I started creating a minimal-crashing-example. At first, I failed
miserably. Just filling a byte array over and over again, I was unable
to reproduce the crash. But then I realized, that in my application,
threads come to play and that might be the case. And suddenly, creating
MCE was really easy:

Create new eventspace using parameterize/make-eventspace, put the actual
code in application thread (thread ...) and make the main thread wait
for this application thread using thread-wait. Before starting the
application thread, I create a simple window, bitmap and a canvas, that
I keep redrawing using refresh-now after each iteration. Funny thing is,
now it keeps crashing even without actually modifying the bitmap in
question. All I need to do is to mess with some byte array in 8 threads.
Sometimes it takes a minute on my computer before it crashes, sometimes
it needs more, but it eventually crashes pretty consistently.

And it is just 60 lines of code:

#lang racket/gui

(require racket/future racket/fixnum racket/cmdline)

(define width 800)
(define height 600)

(define framebuffer (make-fxvector (* width height)))
(define pixels (make-bytes (* width height 4)))

(define max-depth 0)

(command-line
 #:once-each
 (("-d" "--depth") d "Futures binary partitioning depth" (set! max-depth
(string->number d

(file-stream-buffer-mode (current-output-port) 'none)

(parameterize ((current-eventspace (make-eventspace)))
  (define win (new frame%
   (label "test")
   (width width)
   (height height)))
  (define bmp (make-bitmap width height))
  (define canvas (new canvas%
  (parent win)
  (paint-callback
   (λ (c dc)
 (send dc draw-bitmap bmp 0 0)))
  ))

  (define (single-run)
(define (do-bflip start end (depth 0))
  (cond ((fx< depth max-depth)
 (define cnt (fx- end start))
 (define cnt2 (fxrshift cnt 1))
 (define mid (fx+ start cnt2))
 (let ((f (future
   (λ ()
 (do-bflip start mid (fx+ depth 1))
   (do-bflip mid end (fx+ depth 1))
   (touch f)))
(else
 (for ((i (in-range start end)))
   (define c (fxvector-ref framebuffer i))
   (bytes-set! pixels (+ (* i 4) 0) #xff)
   (bytes-set! pixels (+ (* i 4) 1) (fxand (fxrshift c 16)
#xff))
   (bytes-set! pixels (+ (* i 4) 2) (fxand (fxrshift c 8) #xff))
   (bytes-set! pixels (+ (* i 4) 3) (fxand c #xff))
(do-bflip 0 (* width height))
(send canvas refresh-now))
(send win show #t)

  (define appthread
(thread
 (λ ()
   (let loop ()
 (single-run)
 (loop)
  (thread-wait appthread))

Note: the code is deliberately de-optimized to highlight the problem.
Not even mentioning CPU cache coherence here

Running this from command-line, I can adjust the number of threads.
Running with 8 threads:

$ time racket crash.rkt -d 3
SIGSEGV MAPERR si_code 1 fault on addr (nil)
Aborted (core dumped)

real1m18,162s
user7m11,936s
sys 0m3,832s
$ time racket crash.rkt -d 3
SIGSEGV MAPERR si_code 1 fault on addr (nil)
Aborted (core dumped)

real3m44,005s
user20m10,920s
sys 0m11,702s
$ time racket crash.rkt -d 3
SIGSEGV MAPERR si_code 1 fault on addr (nil)
Aborted (core dumped)

real2m1,650s
user10m58,392s
sys 0m6,445s
$ time racket crash.rkt -d 3
SIGSEGV MAPERR si_code 1 fault on addr (nil)
Aborted (core dumped)

real8m8,666s
user45m52,359s
sys 0m25,184s
$

With 4 threads it didn't crash even after quite some time:

$ time racket crash.rkt -d 2
^Cuser break
  context...:
   "crash.rkt": [running body]
   temp35_0
   for-loop
   run-module-instance!
   perform-require!

real20m18,706s
user61m38,546s
sys 0m22,719s
$


I'll re-run the 4-thread test overnight.

What would be the best approach to debugging this issue? I assume I'll
load the racket binary in gdb and see the stack traces at the moment of
the crash, but that won't reveal the source of the problem (judging
based on my previous experience of debugging heavily multi-threaded

Re: [racket-users] Rhombus project plan

[Dominik Pantůček]

Hi,

I can't leave this without reaction...

> 
>   To the point: what would make Racket2 the ultimate tool (for me):
> 
>   * Performance. Faster startup times, shorter execution times in
> general. Optionally, a ‘lite’ version of Racket that compiles
> directly to no-deps binaries, bypassing the JIT altogether, would be
> a game-changer. As far as high levels languages with functional
> concepts and metaprogramming facilities that compiles to tiny, fast
> bins, Nim comes dangerously close, but it’s not a Lisp, and it’s not
> Racket.

They say that Racket is slow. I would like to know who are "they".

Racket can be surprising. For example - our GUI application on RPi has a
start-up time of 24s... But when we compile it using `raco exe`, it goes
down under 2s including some hardware setup the program does. Usually
the slow startup times are caused by not using compiled byte-code.

As I have mentioned a few times on this list, I am working on a side
project right now which pushes Racket to its limits and let's say that
the results might look impossible to many people. Full 3D rendering
pipeline in pure Racket with no external libraries (no OpenGL, no SDL,
just pure Racket) which can run at 60fps FullHD kind of beats the
argument "Racket is slow". Racket can be REALLY fast. But you need to
know how to achieve that (think contract boundaries, unsafe ops guarded
by other mechanisms, ultimate parallelism support - yes, once you grasp
futures to their full extent, you will see what performance gains you
get virtually for free... see my sorting package[1])

>   * Production-quality, modern and fast GUI facilities. I’ll take
> properly documented, complete Qt bindings. Racket/GUI is great for
> internal tools, but it quickly becomes slow, tedious and limited for
> more complex client-facing UIs.

My experience is quite contrary to that. In the last three years we were
forced to develop a few GUI tools used by our customers and there are no
complains only against the tools using racket/gui. Yes, it is far from
perfect - but it is the only thing that really behaves consistently
across all three platforms we need to support (Linux, Mac, and ...
Windows). Python+Qt and go+Qt are absolutely insupportable without a
long list of per-platform/per-widget hacks. It might be that we are just
hitting corner cases (actually that's pretty probably), yet with Racket
there were no such hurdles.

>   * One complete, commercial-grade Web framework, inspired from Symphony
> or Laravel. Security and ease of use first, continuations later.

And this is the part that made me actually hit the "Reply List" button.
The good thing about mentioned PHP frameworks is only that anyone can
start producing something with them without any prior experience. So
right now we are encountering vulnerable and unmaintainable code
anywhere we bump into some 3rd party "web application" we are hired to
audit. With Racket - the entry barrier is really high, I agree. I
actually spent many days getting through it - but once you pass the
barrier, it really helps you write applications where you can focus on
the actual functionality and let the libraries do the rest for you. And
securing such application is almost trivial task. Yes, I have hit some
corner cases here as well, but if you follow for example Bogdan's
work[2][3], you'll see that there are solutions readily available. It is
just necessary to read the docs and understand how the whole thing works.

>   * Better documentation: Racket docs are aesthetically very pleasing,
> complete and detailed. However some parts are still very obscure and
> lacking simple examples (if only the part about Continuations
> included just one basic example, such as a ‘return’ implementation,
> on the very first page. If only the Macros documentation started
> with define-simple-macro and a few very basic, practical examples.
> Instead we’re greeted with pattern-based macros, which although very
> powerful, are very hard to comprehend for newcomers).

I am not sure whether it is the best thing to start with
define-simple-macro (if anything, I'd start with define-syntax-rule),
but yes, more examples are always useful. Just write them and send a
pull request to appropriate repository - I was pleasantly surprised when
I first did this. All the Racketeers are super helpful when it comes to
incorporating any improvements. (Thanks Ben!)

> 
> 
>   I am well aware that what I’m wishing for isn’t necessarily compatible
> with Racket’s intended public’s needs (comp-sci teachers and students?
> That’s the impression I’m getting). But Racket is the most advanced
> general purpose programming tool I’ve ever seen. Wouldn’t it be a shame
> if it was limited to academic use?

As far as I can tell, it is definitely limited to academic use. Although
I prepare support materials for my students with Racket, I actually
teach Clojure - so for me, Racket is tool mostly for my 

Re: [racket-users] Question about generating urls with dispatch-rules from imported modules

[Dominik Pantůček]

Hello,

just a quick and dirty hack I use to get the best of default
dispatch-... for my use-case when dispatching API requests:

(define-for-syntax (dispatch-api-case-helper stx)
  (syntax-parse stx
(((method:id (path+args:expr ...) proc:id) rest ...)
   #`((("api" path+args ...) #:method #,(symbol->string
(syntax->datum #'method)) proc)
  #,@(dispatch-api-case-helper #'(rest ...
(((else proc:id))
 #'((else proc)))
(()
 #'(

(define-syntax (dispatch-api-case stx)
  (syntax-parse stx
((_ rest ...)
 #`(dispatch-case
#,@(dispatch-api-case-helper #'(rest ...))

(define dispatch-api-request-real
  (dispatch-api-case
   (get ("number" (number-arg)) api-number-test)
   (get ("ping") api-ping)
   (post ("auth") auth-login)
   (delete ("auth") auth-logout)
   (put ("auth") auth-passwd)
   (get ("auth" "info") auth-user-info)
   (get ("partners") partners-list)
   (get ("partners" (string-arg)) partner-detail)
   (get ("partners" (string-arg) "payments") partner-payments)
   (get ("partners" (string-arg) "debt") partner-debt)
   (get ("debts") partners-debts)
   (get ("partners" (string-arg) "history") partner-debt-history)
   (get ("invoices" "issued") invoices-issued-list)
   (get ("invoices" "issued" (string-arg)) invoice-info)
   (post ("notes") notes-add)
   (else api-404)))

Not very general, but maybe you'll find that useful (especially if the
(string-arg) and similar can make your life easier like it was the case
for me.

When I needed a url-generation function in earlier project, I basically
did the same with more syntax mess around, that used either set! or
parameter from different module to store the dispatch-url -generated
procedure.

Basically it is about customized version of dispatch-rules that does the
dispatch-case and dispatch-url independently and provides the results by
different means.


Cheers,
Dominik

On 26. 04. 20 17:26, Yury Bulka wrote:
> Meanwhile here's my attempt at doing a koyo-like url generation with a
> wrapper around web-server/dispatch's url-generating function:
> 
> (define-values (app-dispatch app-url)
>   (dispatch-rules  ; the standard one
>[...]))
> 
> (define (app-url/names route-name . args)
>   ;; define a mapping between symbolic names and handlers
>   (define route-names
> (hash 'item-list item-list-view
>   'item-detail item-detail-view))
>   (apply app-url (hash-ref route-names route-name) args))
> 
> And then I can store the function in a parameter that is used by the
> templates.
> 
> It is of course not a general solution (for instance, the rules in
> dispatch-rules and the route-names can go out of sync if one forgets to
> update either of them) and they are not dynamic.
> 
> But for an ultra-minimalistic application this might do when you don't
> feel like adding a new external dependency.
> 
> --
> Yury Bulka
> https://mamot.fr/@setthemfree
> #NotOnFacebook
> 
> 
> 
> Yury Bulka  writes:
> 
>> Thank you for sharing this!
>>
>> I'm wondering, if this is a common use case, maybe it is worth adding to
>> web-server/dispatch in some form?
>>
>> Or, if not, maybe we can extend the documentation to include some hints
>> on how to approach this otherwise?
>>
>> It is great to have powerful libraries outside the main distribution,
>> but the main distribution is the place where newcomers like me are going
>> to look first.
> 

-- 
You received this message because you are subscribed to the Google Groups 
"Racket Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to racket-users+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/racket-users/304869ce-9ab8-0255-50ca-de7b7db0788d%40trustica.cz.

Re: [racket-users] Retro 3D maze with Pict?

[Dominik Pantůček]

Hi,

On 19. 04. 20 11:42, Aidan Gauland wrote:
> Would it be feasible to use Pict to create a very simplistic rendering
> of a first-person perspective of a 2D maze, in the style of Scarab of Ra
>  for the old
> B Macs?
> 

I've created more than a few experiments like this using dc<%> backed by
canvas% or bitmap%. For such simple game like you mention, it will work
like a charm. However it is not the fastest approach - I remember
rendering a Rubik's cube last year for the students[1] and on Linux it
ran fluently, on Mac it ran at 2 frames per second for no apparent
reason. Probably the Cairo back-end is just really slow there.

I would assume Pict is on-par with dc<%>-based approach (in the end that
is what renders the rasterized image).

Funnily enough, I am working on a graphics side project right now and I
have a working 3D rendering pipeline completely in Racket which works
surprisingly fast even on 2560x1440. I plan to publish a series of
articles about that. No OpenGL, no compiled C code, only Racket trickery.

Or you can use ASCII (as seen on RacketFest 2020)...


Cheers,
Dominik

[1]
https://trustica.cz/en/2019/05/16/teaching-cryptography-rubiks-cube-diffusion/

-- 
You received this message because you are subscribed to the Google Groups 
"Racket Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to racket-users+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/racket-users/45b3a36c-f95a-69ac-92e1-5d1b2951dec4%40trustica.cz.

Re: [racket-users] Unfinished S-Expressions in scribbled code blocks

[Dominik Pantůček]

Nope, even with eval:alts and eval:result(s) I cannot produce the
results I want.

Btw, I am subscribed to the list.


Cheers,
Dominik

On 12. 04. 20 17:22, Sage Gerard wrote:
> Does scribble/examples offer what you need?
> 
> 
> 
>  Original Message 
> On Apr 12, 2020, 4:23 AM, Dominik Pantůček <
> dominik.pantu...@trustica.cz> wrote:
> 
> 
> Hello fellow Racketeers,
> 
> I've started a small side project and as a part of that I want to write
> some articles interspersed with code. Scribble was a natural choice for
> that task.
> 
> So I started with #lang scribble/manual as usual and after writing some
> text, I tried to do something like the following:
> 
> 
> #lang scribble/manual
> 
> ... and here are the requires:
> 
> @racketblock{
> (require
> }
> 
> .. with the first one being this and we need that for 
> 
> @racketblock{
> (only-in ffi/unsafe ptr-set! _uint32)
> }
> 
> ...
> 
> 
> Of course it renders the racketblocks as string, silly me. So I go for
> @racketblock[] but that - in @-syntax translates to S-expressions with
> "(require" (without the quotes) definitely not being a valid
> S-expression. And therefore scribble cannot handle it.
> 
> Looking at [1] leaves me with an impression that it is not possible to
> typeset parts of racket code in scribble. Only valid S-expressions (and
> the #lang line, of course).
> 
> Using scribble/lp2 I can get closer to my wanted result, but really it
> just works around the issue by forming valid S-expressions and expanding
> chunks inside those.
> 
> What is the proper way of typesetting (in scribble) parts of racket code
> which do not form complete S-expression?
> 
> And yes, I know this is rather strange requirement, but in this
> particular case, I am pretty sure, I want to work with parts of
> S-expressions without balanced parentheses. (Although the minimal
> example definitely does not answer "why").
> 
> I assume I must have overlooked something, of course.
> 
> Cheers,
> Dominik
> 
> [1] https://docs.racket-lang.org/scribble/scribble_manual_code.html
> 
> --
> You received this message because you are subscribed to the Google
> Groups "Racket Users" group.
> To unsubscribe from this group and stop receiving emails from it,
> send an email to racket-users+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> 
> https://groups.google.com/d/msgid/racket-users/d4473bd1-5b9c-b658-2989-df0a143d7ae9%40trustica.cz.
> 
> -- 
> You received this message because you are subscribed to the Google
> Groups "Racket Users" group.
> To unsubscribe from this group and stop receiving emails from it,
> send an email to racket-users+unsubscr...@googlegroups.com
> <mailto:racket-users+unsubscr...@googlegroups.com>.
> To view this discussion on the web visit
> 
> https://groups.google.com/d/msgid/racket-users/kB7sQQDkefoEq86xajF1QifjjY-cWMqDfbxhD08dN2NqNBNnRn-8u6HX7lOA_JFhClSvgMhJdHxpI9MdEaB3oOpOQHEKNRFn1KCSexM86bg%3D%40sagegerard.com
> 
> <https://groups.google.com/d/msgid/racket-users/kB7sQQDkefoEq86xajF1QifjjY-cWMqDfbxhD08dN2NqNBNnRn-8u6HX7lOA_JFhClSvgMhJdHxpI9MdEaB3oOpOQHEKNRFn1KCSexM86bg%3D%40sagegerard.com?utm_medium=email_source=footer>.
> 

-- 
You received this message because you are subscribed to the Google Groups 
"Racket Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to racket-users+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/racket-users/977981fa-97c1-15ac-fcae-84cbb5330e4e%40trustica.cz.

[racket-users] Unfinished S-Expressions in scribbled code blocks

[Dominik Pantůček]

Hello fellow Racketeers,

I've started a small side project and as a part of that I want to write
some articles interspersed with code. Scribble was a natural choice for
that task.

So I started with #lang scribble/manual as usual and after writing some
text, I tried to do something like the following:


#lang scribble/manual

... and here are the requires:

@racketblock{
(require
}

.. with the first one being this and we need that for 

@racketblock{
 (only-in ffi/unsafe ptr-set! _uint32)
}

...


Of course it renders the racketblocks as string, silly me. So I go for
@racketblock[] but that - in @-syntax translates to S-expressions with
"(require" (without the quotes) definitely not being a valid
S-expression. And therefore scribble cannot handle it.

Looking at [1] leaves me with an impression that it is not possible to
typeset parts of racket code in scribble. Only valid S-expressions (and
the #lang line, of course).

Using scribble/lp2 I can get closer to my wanted result, but really it
just works around the issue by forming valid S-expressions and expanding
chunks inside those.

What is the proper way of typesetting (in scribble) parts of racket code
which do not form complete S-expression?

And yes, I know this is rather strange requirement, but in this
particular case, I am pretty sure, I want to work with parts of
S-expressions without balanced parentheses. (Although the minimal
example definitely does not answer "why").

I assume I must have overlooked something, of course.


Cheers,
Dominik


[1] https://docs.racket-lang.org/scribble/scribble_manual_code.html

-- 
You received this message because you are subscribed to the Google Groups 
"Racket Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to racket-users+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/racket-users/d4473bd1-5b9c-b658-2989-df0a143d7ae9%40trustica.cz.

Re: [racket-users] Proxying websockets via web-server

[Dominik Pantůček]

Sigh... I forgot to reply to the list... Here we go:

I want a single HTTP server like serve/servlet (or similar) that handles
all requests by the standard dispatcher/c logic - ideally using
dispatch/servlet with dispatch-case to specify the mapping. And for one
specific URL path "/sockjs-node", I need it to be able to do the
websocket HTTP 101 protocol switch.

The React.js application in the browser just communicates using HTTP and
if you run its minimal node.js server with "npm start" its HTTP
implementation contains the selective websocket URL path and it works. I
just need a way to proxy this, as my Racket backend is the HTTP server.
For static data I produce response/full with that data, for API calls,
it is response/jsexpr and for this websocket endpoint, I would like to
run copy-port and be done with it - after the protocol switch, that is.

>From what I read about rfc6455 package, it cannot be used with the
web-server package. I'll dive into the source later today.

Actually I can live without that, yet I was curious as it provides a
convenient way to reload the frontend after source changes in its code
(that is the Javascript part). For production builds, a plain
serve/servlet does all the job required.


Dominik

On 11. 01. 20 3:18, Jay McCarthy wrote:
> I don't completely understand what you want to do. Is there a reason
> you can't use the WebSocket implementation ---
> https://docs.racket-lang.org/rfc6455/index.html --- and then use
> normal inter-Racket communication like channels and stuff to work with
> the rest of your Web application?
> 
> Jay
> 
> --
> Jay McCarthy
> Associate Professor @ CS @ UMass Lowell
> http://jeapostrophe.github.io
> Vincit qui se vincit.
> 
> On Thu, Jan 9, 2020 at 5:23 PM Dominik Pantůček
>  wrote:
>>
>> Hello everyone,
>>
>> I am working on an application that uses React.js[1] for front-end and
>> Racket as HTTP back-end server.
>>
>> For production builds, the Javascript part is compiled using "npm build"
>> which generates a directory full of HTML and Javascript files which are
>> then included in the Racket application during syntax stage as an
>> immutable hash. For development builds, the Racket application runs "npm
>> start" in the front-end source directory and proxies all non-backend
>> requests to the managed node.js server.
>>
>> When run in the development setup (that is with the node.js secondary
>> HTTP server), the proxying using serve/servlet and simple dispatch-case
>> works like a charm - each servlet uses http-sendrecv to get the data
>> from the secondary HTTP server and returns it as appropriate response body.
>>
>> But the reason for this setup is that React.js can automatically reload
>> the webpage, if any of the source files change. To do this trick, it
>> uses an url "/sockjs-node". The browser sends GET request for this
>> resource and the node.js server responds with "HTTP/1.1 101 Switching
>> Protocols" like in [2].
>>
>> Apparently, to make this work, I need to establish a bi-directional
>> connection after the 101 response code. This is impossible with
>> serve/servlet. I have done some experiments with plain (serve #:dispatch
>> ...) and just cannot make it work all at once. With connection-i-port
>> and connection-o-port it should be (relatively) easy to implement. But
>> the documentation is virtually nonexistent and browsing the
>> web-server/private/ sources is a bit tricky if I do not know what I am
>> looking for.
>>
>> If anyone can give me a hint how a proper setup for websocket
>> implementation should look like, I would really appreciate it. Of
>> course, I also want to use the dispatch-case with plain requests as it
>> automates most of the real work I need to perform there.
>>
>>
>>
>> Cheers,
>> Dominik
>>
>> [1] https://reactjs.org/
>> [2] https://en.wikipedia.org/wiki/WebSocket#Protocol_handshake
>>
>> --
>> You received this message because you are subscribed to the Google Groups 
>> "Racket Users" group.
>> To unsubscribe from this group and stop receiving emails from it, send an 
>> email to racket-users+unsubscr...@googlegroups.com.
>> To view this discussion on the web visit 
>> https://groups.google.com/d/msgid/racket-users/a3834699-4ed6-f337-1409-9f175636c087%40trustica.cz.
> 

-- 
You received this message because you are subscribed to the Google Groups 
"Racket Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to racket-users+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/racket-users/8222caa5-0104-a0b4-2e08-53ec731ac2bb%40trustica.cz.

[racket-users] Proxying websockets via web-server

[Dominik Pantůček]

Hello everyone,

I am working on an application that uses React.js[1] for front-end and
Racket as HTTP back-end server.

For production builds, the Javascript part is compiled using "npm build"
which generates a directory full of HTML and Javascript files which are
then included in the Racket application during syntax stage as an
immutable hash. For development builds, the Racket application runs "npm
start" in the front-end source directory and proxies all non-backend
requests to the managed node.js server.

When run in the development setup (that is with the node.js secondary
HTTP server), the proxying using serve/servlet and simple dispatch-case
works like a charm - each servlet uses http-sendrecv to get the data
from the secondary HTTP server and returns it as appropriate response body.

But the reason for this setup is that React.js can automatically reload
the webpage, if any of the source files change. To do this trick, it
uses an url "/sockjs-node". The browser sends GET request for this
resource and the node.js server responds with "HTTP/1.1 101 Switching
Protocols" like in [2].

Apparently, to make this work, I need to establish a bi-directional
connection after the 101 response code. This is impossible with
serve/servlet. I have done some experiments with plain (serve #:dispatch
...) and just cannot make it work all at once. With connection-i-port
and connection-o-port it should be (relatively) easy to implement. But
the documentation is virtually nonexistent and browsing the
web-server/private/ sources is a bit tricky if I do not know what I am
looking for.

If anyone can give me a hint how a proper setup for websocket
implementation should look like, I would really appreciate it. Of
course, I also want to use the dispatch-case with plain requests as it
automates most of the real work I need to perform there.



Cheers,
Dominik

[1] https://reactjs.org/
[2] https://en.wikipedia.org/wiki/WebSocket#Protocol_handshake

-- 
You received this message because you are subscribed to the Google Groups 
"Racket Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to racket-users+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/racket-users/a3834699-4ed6-f337-1409-9f175636c087%40trustica.cz.

Re: [racket-users] Re: Inadvertedly requiring racket/unsafe/ops

[Dominik Pantůček]

Hello all,

a short followup on the issue.

I opened a PR[1] and Sam raised an interesting question:

(require (filtered-in
   (λ (name)
  (and (regexp-match #rx"^unsafe-fx" name)
   (regexp-replace #rx"unsafe-" name "")))
   racket/unsafe/ops))

ensures that only unsafe-fx... bindings get required as fx... bindings
and no other bindings from racket/unsafe/ops are. However, the following:

(require (filtered-in
   (λ (name) (regexp-replace #rx"unsafe-fx" name ""))
   racket/unsafe/ops))

does the same trick of not shadowing for example vector-ref with
unsafe-vector-ref - which also solves the problem I encountered and
therefore it might be better suited (as it is a smaller change).

To be honest - both solutions are OK. My personal preference for not
polluting the current namespace with all those unsafe-... bindings is
mostly irrelevant here. Can anyone using racket/unsafe/ops share their
thoughts here?

Also please note that the same applies to flonums and extflonums - which
do not mention the unsafe-... option in the documentation at all. I am
inclined to think that this might be good to fix as well.


Cheers,
Dominik

[1] https://github.com/racket/racket/pull/2975

On 15. 12. 19 11:13, Dominik Pantůček wrote:
> Hi,
> 
> On 15. 12. 19 2:57, Jack Firth wrote:
>> I think that documentation fix is a good idea.
> 
> I'll submit a PR to appropriate repository later on.
> 
>> More broadly, it seems awkward that all of the unsafe ops for 
>> different data types are combined together into a single module. I 
>> would instead expect there to be modules like racket/fixnum/unsafe, 
>> racket/struct/unsafe, racket/vector/unsafe, etc. But I've never used 
>> the unsafe ops before, maybe those who have can chime in?
> 
> it's not that easy as it is really about the "ops" part. For example if
> you need (make-fxvector ...), you still need to require it using
> (only-in racket/fixnum make-fxvector) as there is no racket/unsafe/ops
> equivalent (which is really NOT surprising).
> 
> And also splitting the racket/unsafe/ops into more modules might break a
> thing or two as it has been this way for quite some time. Also it is
> nice - if you are in need of heavy optimizations - to see the list of
> unsafe ops in one place.
> 
> 
> Cheers,
> Dominik
> 
>>
>> On Saturday, December 14, 2019 at 1:03:14 PM UTC-8, Dominik Pantůček 
>> wrote:
>>
>> Hello,
>>
>> the documentation at 
>> https://docs.racket-lang.org/reference/fixnums.html 
>> <https://docs.racket-lang.org/reference/fixnums.html> is misleading 
>> at best. If you - as I did - use the suggested approach of requiring 
>> optimized (and unsafe) fx... operations from racket/unsafe/ops with:
>>
>> (require (filtered-in (λ (name) (regexp-replace #rx"unsafe-" name 
>> "")) racket/unsafe/ops))
>>
>> You end up using _all_ symbols from racket/unsafe/ops. All of them 
>> are - of course - uncontracted. Which means that if you issue for 
>> example vector-ref on something that is not a vector, it can crash 
>> the runtime without any apparent reason - instead of just throwing
>> an exception as one would expect.
>>
>> A simple documentation fix should probably go along the lines:
>>
>> (require racket/require (filtered-in (λ (name) (and (regexp-match 
>> #rx"^unsafe-fx" name) (regexp-replace #rx"unsafe-" name ""))) 
>> racket/unsafe/ops))
>>
>> Or is it just me running into corner cases with optimized Racket 
>> code?
>>
>>
>> Cheers, Dominik
>>
>> -- You received this message because you are subscribed to the Google
>> Groups "Racket Users" group. To unsubscribe from this group and stop
>> receiving emails from it, send an email to 
>> racket-users+unsubscr...@googlegroups.com 
>> <mailto:racket-users+unsubscr...@googlegroups.com>. To view this 
>> discussion on the web visit 
>> https://groups.google.com/d/msgid/racket-users/c118c47e-fccd-4fab-b252-7a24afe6eef1%40googlegroups.com
>>
>>
>>
>>
>>
> <https://groups.google.com/d/msgid/racket-users/c118c47e-fccd-4fab-b252-7a24afe6eef1%40googlegroups.com?utm_medium=email_source=footer>.
> 

-- 
You received this message because you are subscribed to the Google Groups 
"Racket Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to racket-users+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/racket-users/5769cec9-3bb5-4e2f-ba64-23910ab91da9%40trustica.cz.

Re: [racket-users] ffi-lib and scribble/srcdoc

[Dominik Pantůček]

Hi,

On 23. 12. 19 15:29, Sam Tobin-Hochstadt wrote:
> What you're seeing is that `scribble/srcdoc` looks for the _binding_
> of `->` from `racket/contract` which is being shadowed by the version
> of `->` from `ffi/unsafe`. I would do something like this:
> 
> #lang racket
> (require (rename-in ffi/unsafe [-> -->]) scribble/srcdoc)
> (provide
>   (proc-doc/names
> a-struct?
> (-> any/c boolean?)
> (a)
> ("Returns true if a is a-struct")))
> (struct a-struct (b))
> 
> and then use `-->` in ffi declarations.

that is exactly what I was looking for. All the "unsafe" parts can get
really "unsafe" during any stage though ...


Thank you!
Dominik

-- 
You received this message because you are subscribed to the Google Groups 
"Racket Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to racket-users+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/racket-users/390de732-d2a7-343a-e36f-d693a1da8cc4%40trustica.cz.

[racket-users] ffi-lib and scribble/srcdoc

[Dominik Pantůček]

Hello,

apart from parallel sorting and working with unsafe ops, I stumbled upon
a very strange behavior using ffi-lib:

-->8--
#lang racket
(require scribble/srcdoc)
(provide
  (proc-doc/names
a-struct?
(-> any/c boolean?)
(a)
("Returns true if a is a-struct")))
(struct a-struct (b))
->8-

Running this works without a glitch:

$ racket test1.rkt
$

But if we require ffi/unsafe, something bad happens:

-->8--
#lang racket
(require ffi/unsafe scribble/srcdoc)
(provide
  (proc-doc/names
a-struct?
(-> any/c boolean?)
(a)
("Returns true if a is a-struct")))
(struct a-struct (b))
->8-

$ racket test2.rkt
test2.rkt:8:4: proc-doc/names: unsupported procedure contract form (no
argument names)
  at: (-> any/c boolean?)
  in: (proc-doc/names a-struct? (-> any/c boolean?) (a) ("Returns true
if a is a-struct"))
  location...:
   test2.rkt:8:4
  context...:
   do-raise-syntax-error
   /usr/share/racket/pkgs/scribble-lib/scribble/srcdoc.rkt:342:2:
proc-doc/names-transformer
   /usr/share/racket/pkgs/scribble-lib/scribble/srcdoc.rkt:127:24
   /usr/share/racket/pkgs/scribble-lib/scribble/srcdoc.rkt:124:0:
do-provide/doc
   /usr/share/racket/collects/racket/provide-transform.rkt:63:2:
pre-expand-export
   /usr/share/racket/collects/racket/private/reqprov.rkt:708:2: provide
   apply-transformer-in-context
   apply-transformer52
   dispatch-transformer41
   do-local-expand50
   /usr/share/racket/collects/syntax/wrap-modbeg.rkt:46:4:
do-wrapping-module-begin
   apply-transformer-in-context
   apply-transformer52
   dispatch-transformer41
   loop
   finish
   ...
$

I assume that ffi/unsafe somehow re-defines -> and it does it in a very
srcdoc-incompatible way. Did anyone spot the same behavior? Can it be
fixed? That is without rewriting the whole ffi/unsafe library.

I am using a simple workaround to scribble my libraries with ffi/unsafe
requires:

->8-
#lang racket
(require
 ffi/unsafe
 scribble/srcdoc)
(provide
  (proc-doc/names
a-struct?
(->* (any/c) () boolean?)
((a) ())
("Returns true if a is a-struct")))
(struct a-struct (b))
->8-

But it is kind of strange to use (-> any/c boolean?) contract for
predicate in one module and (->* (any/c) () boolean?) in another one.

It sort of feels like the problem with requiring unsafe-vector-ref
without knowing. Although this is also "without knowing", it at least
exhibits itself at the syntax stage (and not at run-time).



Cheers,
Dominik

-- 
You received this message because you are subscribed to the Google Groups 
"Racket Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to racket-users+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/racket-users/2ac9a533-4325-53da-6855-01bd07da81a3%40trustica.cz.

Re: [racket-users] Re: Inadvertedly requiring racket/unsafe/ops

[Dominik Pantůček]

Hi,

On 15. 12. 19 2:57, Jack Firth wrote:
> I think that documentation fix is a good idea.

I'll submit a PR to appropriate repository later on.

> More broadly, it seems awkward that all of the unsafe ops for 
> different data types are combined together into a single module. I 
> would instead expect there to be modules like racket/fixnum/unsafe, 
> racket/struct/unsafe, racket/vector/unsafe, etc. But I've never used 
> the unsafe ops before, maybe those who have can chime in?

it's not that easy as it is really about the "ops" part. For example if
you need (make-fxvector ...), you still need to require it using
(only-in racket/fixnum make-fxvector) as there is no racket/unsafe/ops
equivalent (which is really NOT surprising).

And also splitting the racket/unsafe/ops into more modules might break a
thing or two as it has been this way for quite some time. Also it is
nice - if you are in need of heavy optimizations - to see the list of
unsafe ops in one place.


Cheers,
Dominik

> 
> On Saturday, December 14, 2019 at 1:03:14 PM UTC-8, Dominik Pantůček 
> wrote:
> 
> Hello,
> 
> the documentation at 
> https://docs.racket-lang.org/reference/fixnums.html 
> <https://docs.racket-lang.org/reference/fixnums.html> is misleading 
> at best. If you - as I did - use the suggested approach of requiring 
> optimized (and unsafe) fx... operations from racket/unsafe/ops with:
> 
> (require (filtered-in (λ (name) (regexp-replace #rx"unsafe-" name 
> "")) racket/unsafe/ops))
> 
> You end up using _all_ symbols from racket/unsafe/ops. All of them 
> are - of course - uncontracted. Which means that if you issue for 
> example vector-ref on something that is not a vector, it can crash 
> the runtime without any apparent reason - instead of just throwing
> an exception as one would expect.
> 
> A simple documentation fix should probably go along the lines:
> 
> (require racket/require (filtered-in (λ (name) (and (regexp-match 
> #rx"^unsafe-fx" name) (regexp-replace #rx"unsafe-" name ""))) 
> racket/unsafe/ops))
> 
> Or is it just me running into corner cases with optimized Racket 
> code?
> 
> 
> Cheers, Dominik
> 
> -- You received this message because you are subscribed to the Google
> Groups "Racket Users" group. To unsubscribe from this group and stop
> receiving emails from it, send an email to 
> racket-users+unsubscr...@googlegroups.com 
> <mailto:racket-users+unsubscr...@googlegroups.com>. To view this 
> discussion on the web visit 
> https://groups.google.com/d/msgid/racket-users/c118c47e-fccd-4fab-b252-7a24afe6eef1%40googlegroups.com
>
>
>
>
> 
<https://groups.google.com/d/msgid/racket-users/c118c47e-fccd-4fab-b252-7a24afe6eef1%40googlegroups.com?utm_medium=email_source=footer>.

-- 
You received this message because you are subscribed to the Google Groups 
"Racket Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to racket-users+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/racket-users/4185c1c1-7e2b-ea02-a9c5-03231cb112c2%40trustica.cz.

[racket-users] Inadvertedly requiring racket/unsafe/ops

[Dominik Pantůček]

Hello,

the documentation at https://docs.racket-lang.org/reference/fixnums.html
is misleading at best. If you - as I did - use the suggested approach of
requiring optimized (and unsafe) fx... operations from racket/unsafe/ops
with:

(require (filtered-in
  (λ (name) (regexp-replace #rx"unsafe-" name ""))
  racket/unsafe/ops))

You end up using _all_ symbols from racket/unsafe/ops. All of them are -
of course - uncontracted. Which means that if you issue for example
vector-ref on something that is not a vector, it can crash the runtime
without any apparent reason - instead of just throwing an exception as
one would expect.

A simple documentation fix should probably go along the lines:

(require racket/require
 (filtered-in
  (λ (name)
(and (regexp-match #rx"^unsafe-fx" name)
 (regexp-replace #rx"unsafe-" name "")))
   racket/unsafe/ops))

Or is it just me running into corner cases with optimized Racket code?


Cheers,
Dominik

-- 
You received this message because you are subscribed to the Google Groups 
"Racket Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to racket-users+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/racket-users/1262b01a-5e9c-180e-d02a-b2e331c1f3cc%40trustica.cz.

[racket-users] futures-sort update

[Dominik Pantůček]

Hello Racketeers,

I am continuing my work on cleaning up our private source code and
publishing it as a part of my futures-sort[1] package. Current state of
the package is:

- configurable parallelism depth
- in-place parallel merge-sort of both vector and fxvector
- parallel merge-sort with results going to a freshly allocated vector
- support for calling a custom progress reporting function for all variants

The only missing piece to put this on par with vector-sort is probably
the support for start and end indices and the #:key argument (which we
are internally using). I am not entirely sure if key-caching is feasible
though.

Apart from that and some documentation improvements (mainly for the
progress reporting feature), it is basically complete. In the future I'd
like to investigate possible speedups by providing optimized code paths
for "small" ranges starting with 3 element ranges.

I would appreciate any suggestions and/or feature requests.

Btw, this is production code we are using to speed up tome tasks both on
Linux and Windows.

If you want to benchmark it, beware: it is necessary to first sort some
large dummy vector to ensure that JIT kicks in (1000 elements is enough
with 7.5 on my i7 running Linux).

Also I am curious about what is necessary to get the package to a higher
ring. There will be no backwards-incompatible changes from now.

Any suggestions are - of course - welcome.


Cheers,
Dominik


[1] https://pkgs.racket-lang.org/package/futures-sort

-- 
You received this message because you are subscribed to the Google Groups 
"Racket Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to racket-users+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/racket-users/eeb76294-4057-3226-9cab-a4e9c62afa03%40trustica.cz.

Re: [racket-users] Vector length and indices contracts

[Dominik Pantůček]

On 04. 12. 19 20:21, Matthew Flatt wrote:
> I think it makes sense to refine the contract to guarantee a fixnum
> result for `vector-length`.
> 
> This fact is currently documented in `unsafe-vetcor-length`, because
> that's the layer where it has seemed sensible to talk about fixnums in
> the past, but that's not where anyone would think to look. Meanwhile,
> the contract for unsafe operation doesn't specify a nonnegative fixnum
> as it should.
> 
> At Wed, 4 Dec 2019 14:07:47 -0500, George Neuner wrote:
>>
>>
>> It would be more correct to use  (and/c fixnum? (or/c zero? positive?)) 
>> to explicitly limit the value.

My impression is - so far - that (and/c fixnum? (or/c zero? positive?))
or (and/c fixnum? exact-nonnegative-integer?) should be the right way to
go with vector-length.

What about all the vector-ref, -set! and basically all indices
contracts? That should probably be the same.

Also updating the contracts for unsafe-vector-* seems reasonable to me.


Dominik

-- 
You received this message because you are subscribed to the Google Groups 
"Racket Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to racket-users+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/racket-users/f54df855-6ffd-4979-0f58-2126c80f577d%40trustica.cz.

[racket-users] Vector lenght and indices contracts

[Dominik Pantůček]

Hello,

looking at vector-length[1] documentation, it returns
(exact-nonnegative-integer?). However, as far as I can tell, it returns
(fixnum?). Also for subsequent contracts in the vector's documentation
all indices are assumed to be (exact-nonnegative-integer?) but usually
it is impossible on given platform to use anything larger than (fixnum?)
in reality.

For example on 64bit platforms the (fixnum?) could store numbers from (-
(expt 2 62)) to (sub1 (expt 2 62)). Vector slots cannot be smaller than
64bits (machine-dependent pointers) which means 8 bytes = 3 bits of
address. Given 64bit (VERY theoretical) size of the address space, this
leaves the possibility of storing a single vector with (expt 2 61)
elements into the memory.

If I did not overlook anything, the contracts could be safely changed to
(fixnum?).

And yes, I found this issue while cleaning up my futures-sort[2] package
discussed a few months ago here. If I assume
(exact-nonnegative-integer?), I need to manually check for (fixnum?).
Even though it - given the information above - does not really make much
sense.


Should I open a PR?


Cheers,
Dominik


[1]
https://docs.racket-lang.org/reference/vectors.html#(def._((quote._~23~25kernel)._vector-length))
[2] https://docs.racket-lang.org/futures-sort/index.html

-- 
You received this message because you are subscribed to the Google Groups 
"Racket Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to racket-users+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/racket-users/e9ea30ad-5563-fe1a-688c-73fe47b51e68%40trustica.cz.

Re: [racket-users] question re downloading a PDF file

[Dominik Pantůček]

Hi,

On 24. 11. 19 13:01, Tim Hanson wrote:
> hi,
> 
> I need to fetch a number of PDF documents and wrote a bit of racket to make 
> it easier. The part that "downloads" is this:
> 
> (let ([content 
>((compose port->string get-pure-port string->url) (url-for-doc 
> year doc-name))])
>   (let ([out (open-output-file full-path-of-file-to-write #:mode 'text)])
> (write-string content out)
> (close-output-port out)

as far as I can tell the PDF is not a string - rather it is bytes. I
would try port->bytes and write-bytes in your case.


Cheers,
Dominik

-- 
You received this message because you are subscribed to the Google Groups 
"Racket Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to racket-users+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/racket-users/cb06a85a-1840-f8c0-111f-9b5af21910f7%40trustica.cz.

Re: [racket-users] Re: Parallel merge-sort leveraging futures

[Dominik Pantůček]

Hello,

thanks everyone for helping me putting this into a reasonable shape.

My short-term plans with futures-sort are currently to check how various
construct could be improved (or'ing of touched futures is really just a
quick hack for example). Also I'd like to investigate why it didn't work
for me when I used the futures example in documentation[1] - that is to
perform one half of the computation in current context and then touch
future with the other half.

Feature-wise I'd like to implement a version that returns a fresh
vector. However I want to avoid any unnecessary copying so I want ot use
similar approach as with swapping/keeping for cnt=1 or 2. With this
ready it should be a faster drop-in replacement for all (fx)vector
sorting functions. I am not sure if similar approach is viable for lists.

And yes, it needs real tests. However I am not sure what kind of
reproducible tests to add there. Especially that the parallelism may not
be available on given platform. Maybe creating reasonably small vector
of random numbers with static seed and force two "parallel" futures no
matter what is available?

Also today I published on our company blog[2] (no need to follow this
link) some benchmarking results obtained on ThinkPad x280[3] (4 cores, 8
HT) and on a VM running on  Intel Xeon with 16 cores / 32 HT with 16
VCPUs assigned to the VM[4]. I measured 10 runs for each N where N was
n*2^18 for n in 1 to 255 which covers the range from 0 to 2^24 or
roughly 16M.

All times are measured using time-apply and the resulting data set was
processed with Gnuplot and empirical data were fitted using
f(x)=a*n*log2(n).

A few interesting remarks - once we reach the HTs (and not just cores),
the memory starts to be a real bottleneck. Hence the small difference
between depth=2 and depth=3 on i7.

Also for depth=3 and depth=4 sometimes Racket just hung and I had to
restart the tests. I will try to create a minimal working example where
futures can block indefinitely. Does anyone have similar experience for
example with pmapf?


Cheers,
Dominik


[1]
https://docs.racket-lang.org/guide/parallelism.html#%28part._effective-futures%29
[2]
https://trustica.cz/en/2019/10/10/parallel-merge-sort-leveraging-futures-in-racket/
[3]
https://trustica.cz/wp-content/uploads/2019/10/futures-sort-telperion.png
[4] https://trustica.cz/wp-content/uploads/2019/10/futures-sort-builder.png

-- 
You received this message because you are subscribed to the Google Groups 
"Racket Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to racket-users+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/racket-users/f9bdae48-d027-9ed4-0cc3-e97f230a7b21%40trustica.cz.

Re: [racket-users] Re: Parallel merge-sort leveraging futures

[Dominik Pantůček]

Hi,

On 08. 10. 19 6:58, Sam Tobin-Hochstadt wrote:
> I've submitted a pull request fixing those errors and supporting
> running in serial mode.

thank you for that, it's already fixed. I am actually pulling this from
our internal git repository and I imported older info.rkt into the
Github one.

For the contracts - now it is void? instead of suggested any/c. The
functions are modeled like vector-sort! - which returns #.

Btw, is it necessary to bump the version on pkgd.racket-lang.org in
order to get it updated? When I `raco install' the package from its
directory, it works fine now. When I use the github repository using
plain `raco pkg install futures-sort' it still uses the old version.
Just removing and re-installing does not change anything.

> 
> One thing I notice is that it's substantially faster than
> `vector-sort!` even when run in serial mode on traditional Racket (but
> not on Racket CS), so perhaps this should be integrated (or there are
> improvements that we can adopt).

We mostly discussed that with Jens Axel on #racket - main advantage at
the lowest level is the usage of fixnums for everything. This however
means that the vector's size to be sorted must be `fixnum?'. Which I
think is always the case on all supported architectures (addressable
memory and such).

> 
> Alex, just to clarify a couple things about futures: they block when
> performing operations that are either sensitive to the current
> continuation or need single-threaded access to the runtime. Vector
> operations, as you see in Dominik's code, are not in this category.
> Mutable hash table operations are blocking. Here's some numbers with
> performance on sorting a vector with random fixnums using this package
> on a 4 core machine with hyperthreading, "classic" is `vector-sort!`.
> Note that the futures-sort library creates 2^k futures.
> 
> k: 0
> cpu time: 3402 real time: 3399 gc time: 45
> k: 1
> cpu time: 3487 real time: 1834 gc time: 46
> k: 2
> cpu time: 3581 real time: 1097 gc time: 69
> k: 4
> cpu time: 5745 real time: 1014 gc time: 69
> k: 8
> cpu time: 5742 real time: 992 gc time: 45
> k: 16
> cpu time: 8111 real time: 2189 gc time: 279
> 'classic
> cpu time: 4390 real time: 4388 gc time: 98
> 
> Here are similar numbers for Racket CS:
> 
> k: 0
> cpu time: 2960 real time: 2960 gc time: 33
> k: 1
> cpu time: 3021 real time: 1594 gc time: 33
> k: 2
> cpu time: 3462 real time: 1154 gc time: 36
> k: 4
> cpu time: 4381 real time: 929 gc time: 36
> k: 8
> cpu time: 4406 real time: 889 gc time: 34
> k: 16
> cpu time: 7124 real time: 1655 gc time: 440
> 'classic
> cpu time: 2749 real time: 2749 gc time: 51

Is there a package for these benchmarks? I created some benchmarks based
on sorting vectors of varying size from 1 to 2^27 elements. It would be
really nice to test this on a number of systems to get some empirical
data here. The GC can sometimes surprise (me, at least). My benchmarks
used (current-inexact-milliseconds) before and after each run which is
not something very exact. I'll post some graphs on our company blog on
Thursday probably.

Thank you Sam and Alex!


Cheers,
Dominik

-- 
You received this message because you are subscribed to the Google Groups 
"Racket Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to racket-users+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/racket-users/7409b25d-8638-36ff-4c2e-7253da5ab672%40trustica.cz.

[racket-users] Parallel merge-sort leveraging futures

[Dominik Pantůček]

Hello,

over the course of past few months I have been tasked with solving a
(real-world) problem of sorting the sums of all possible combinations of
numbers. Some boring accounting stuff revolving around invoices. As the
total number of combinations is 2^N where N is the number of source
numbers, this task got "interesting" once there were more than 24
numbers - that is on my laptop with:

model name  : Intel(R) Core(TM) i7-8550U CPU @ 1.80GHz

4 cores, 8 hyper-threads and 16GB RAM.

Long story short, I had to implement a custom merge-sort for fxvectors
and managed to leverage futures for fully parallel execution. Now in
some spare time, I slightly polished the code and released it as
"futures-sort" package.

The source code is available on GitHub[1] and it is already in Racket
package index[2].

The package provides four sorting functions - two for vector? and two
for fxvector?. There are two variants of each, one with progress
reporting and one without. For the original task I needed to show
progress using racket/gui gauge% and it might be helpful for others
(it's just a callback that gets current and total number of merges
performed regularly).

I would really appreciate any feedback regarding the naming conventions,
code style and generally anything else.

Yes, the tests are not there (yet).

The same algorithm without futures runs on par with Racket's default
sorting routines (tested for sets upto 2^30 - only 16G of memory here)
and for fixnums it is even (very slightly but consistently) faster than
vector-sort alone. With parallelism using futures it runs roughly N
times faster where N is the number reported by (processor-count).

I have already some benchmark results available and once I get more data
I am more than happy to share it as well.

The parallelism is configurable as parameter.

And yes, all of this can be seen in the documentation which is included
but I still have to look into how to build it - I was sort of assuming
this works automatically based on what I see for example in
scribble-math[3][4]. Any help with getting this right is also more than
welcome.


Lastly - thanks go out to Jens Axel Søgaard for pointing out that
fixnums and fxvectors might help (and why) and also to other helpful
folks on #racket at Freenode.


Cheers,
Dominik


[1] https://github.com/dzoep/futures-sort
[2] https://pkgd.racket-lang.org/pkgn/package/futures-sort
[3] https://docs.racket-lang.org/scribble-math/index.html
[4] https://pkgd.racket-lang.org/pkgn/package/scribble-math

-- 
You received this message because you are subscribed to the Google Groups 
"Racket Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to racket-users+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/racket-users/bdddfd14-7e06-28e7-91c0-67a2403f5629%40trustica.cz.

Re: [racket-users] scribble/srcdoc raised exception fix and class-doc syntax form

[Dominik Pantůček]

Hi,

On 25. 09. 19 19:43, Dominik Pantůček wrote:
> 
> On 25. 09. 19 19:32, Ben Greenman wrote:
>>> Should I include a brief documentation in
>>> scribble-doc/scribblings/scribble/srcdoc.scrbl within the same PR as well?
>>
>> Yes!
> 
> Btw, looking at the struct-doc and struct*-doc descriptions maybe I
> should actually provide class-doc and class*-doc forms. The former for
> documenting classes without interfaces and the latter with interfaces
> list. The underlying defclass omits interfaces in the generated
> documentation if the interfaces list is empty (which is something the
> transformer can do for class-doc then). This way it would strictly
> follow the difference between class and class* forms.
> 

it's there as PR#213.

However, I have no clue what @history[#:added ...] should be attached to
the scribblings. The git tags are the same as for Racket releases and
the version from scribble-lib/info.rkt is 1.29. I can guess 1.30 might
be fine, but I'd rather get this confirmed.


Cheers,
Dominik

-- 
You received this message because you are subscribed to the Google Groups 
"Racket Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to racket-users+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/racket-users/8a3e1925-8c63-ef00-ce6e-c27da19eea8b%40trustica.cz.

Re: [racket-users] scribble/srcdoc raised exception fix and class-doc syntax form

[Dominik Pantůček]

On 25. 09. 19 19:32, Ben Greenman wrote:
>> Should I include a brief documentation in
>> scribble-doc/scribblings/scribble/srcdoc.scrbl within the same PR as well?
> 
> Yes!

Btw, looking at the struct-doc and struct*-doc descriptions maybe I
should actually provide class-doc and class*-doc forms. The former for
documenting classes without interfaces and the latter with interfaces
list. The underlying defclass omits interfaces in the generated
documentation if the interfaces list is empty (which is something the
transformer can do for class-doc then). This way it would strictly
follow the difference between class and class* forms.

> 
> And if there are tests for scribble/srcdoc, it'd be good to add some
> for `class-doc` before merging
> 

There are tests only for proc-doc-transformer and
proc-doc/names-transformer. All other forms are without any tests (which
sort of makes sense).


Dominik
P.S.: Btw, I am subscribed to the list ...

-- 
You received this message because you are subscribed to the Google Groups 
"Racket Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to racket-users+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/racket-users/e4f4f4db-2484-472f-78f2-89690771057f%40trustica.cz.

Re: [racket-users] scribble/srcdoc raised exception fix and class-doc syntax form

[Dominik Pantůček]

Hi,

thank you for the quick response.

On 25. 09. 19 17:11, Ben Greenman wrote:
> These changes look great. Can you open a pull request for the
> racket/scribble repo?
> 

opened PR#212.


> It looks like:
> - `class?` would be a better contract than `any/c`

Definitely makes sense, as the result is contract-out. I will change that.

> - maybe `defmethod`, `this-obj', and others should be required, in
> case the `pre-flow` needs them

Adding at least defconstructor and defmethod is definitely a good idea
(those are used heavily within defclass anyway). Although I've rarely
used this-obj so far, it looks like a very good idea indeed, so I will
add these three symbols into require value.

> 
>> 
>> It does not handle the maybe-link keyword argument as I am not using it
>> and I didn't dive into where it is used in order to test it.
> 
> That seems fine because `thing-doc` doesn't provide a maybe-link to
> `defthing` either.
> 

I see. So let's just forget about it right now then.

I will clean it up and create another PR for it.

Should I include a brief documentation in
scribble-doc/scribblings/scribble/srcdoc.scrbl within the same PR as well?


Cheers,
Dominik

-- 
You received this message because you are subscribed to the Google Groups 
"Racket Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to racket-users+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/racket-users/faf46d5a-6f88-e82c-3600-b55b03a9f95d%40trustica.cz.

[racket-users] scribble/srcdoc raised exception fix and class-doc syntax form

[Dominik Pantůček]

Hello,

as a heavy user of scribble/srcdoc I've always missed a defclass provide
form equivalent. Therefore I wanted to implement my own. I decided to
use thing-doc as a starting point and noticed that it raises misleading
message if id is not an identifier. Minimal working example follows.

 MWE: test.rkt 
#lang racket
(require scribble/srcdoc)
(provide (thing-doc 5 number? "Just a number."))
 MWE: test.scrbl 
#lang scribble/manual
@(require scribble/extract)
@include-extracted["test.rkt"]


Running scribble --pdf test.scrbl produces:

test.rkt:4:12: parameter/doc: expected an identifier
  at: 5
  in: (thing-doc 5 number? "Just a number.")
  location...:
   test.rkt:4:12
  context...:

Looking at the sources file scribble-lib/srcdoc.rkt, the line 580 lists
the wrong exception identifier. A patch against current master follows:


diff --git a/scribble-lib/scribble/srcdoc.rkt
b/scribble-lib/scribble/srcdoc.rkt
index ee977a16..e9dd3311 100644
--- a/scribble-lib/scribble/srcdoc.rkt
+++ b/scribble-lib/scribble/srcdoc.rkt
@@ -577,7 +577,7 @@
   [(_ id contract desc)
(begin
  (unless (identifier? #'id)
-   (raise-syntax-error 'parameter/doc
+   (raise-syntax-error 'thing-doc
"expected an identifier"
stx
#'id))


Also I implemented a simple defclass wrapper as a provide form named
class-doc:

 class-doc syntax form 
(define-provide/doc-transformer class-doc
  (lambda (stx)
(syntax-case stx ()
  [(_ id super (intf-id ...) pre-flow)
   (begin
 (unless (identifier? #'id)
   (raise-syntax-error 'class-doc
   "expected an identifier"
   stx
   #'id))
 (unless (identifier? #'super)
   (raise-syntax-error 'class-doc
   "expected super class identifier"
   stx
   #'id))
 (values
  #'[id any/c] ; contract not used?
  #'(defclass id super (intf-id ...) . pre-flow)
  #'((only-in scribble/manual defclass))
  #'id))])))


It does not handle the maybe-link keyword argument as I am not using it
and I didn't dive into where it is used in order to test it.

 maybe-link =   
|   #:link-target? link-target?-expr

I would appreciate any feedback and/or improvement suggestions.

Is there a chance this new provide form can be merged into
scribble-lib/srcdoc.rkt in the future or is it better to create a
separate package for it?


Cheers,
Dominik

-- 
You received this message because you are subscribed to the Google Groups 
"Racket Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to racket-users+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/racket-users/99cc1ba0-19f3-717b-b089-6cd6aa948144%40trustica.cz.