Re: [racket-dev] binding clause transformers (was: generic binding forms)

[Stephen Chang]

Hi Jens,

Your binding clause transformers looks very useful. I've definitely hacked
together a subset of your functionality in the past so I will be using your
library in the future.

And I can see the similar goals. We should compare notes. One question,
have you tried to make your def form work with function definitions? It
seems like it should be able to, by wrapping the function body with your
let-syntax. Would that not work?



On Mon, Aug 26, 2013 at 2:14 AM, Jens Axel Søgaard wrote:

> Hi Stephen,
>
> > As an educational side project, I've been toying around with a
> > different way of organizing all the binding forms. What I wanted to do
> > is remove the need to manually combine current (and future) binding
> > forms by moving the binding "logic" to the binding site itself.
>
> Your goals seem similar to the ones I had for "bind".
> The different binding forms all have binding clauses.
> I introduced binding clause transformers that decide what
> to do with a given binding clause.
>
> The binding clause transformer :delay can for example
> be used like this (where bind is my name for a let that understands
> binding clause transformers).
>
>  
> (bind
>  ([x 
> :delay
>  1]   
> (+
>  x x)))
>
> At macro expansion time this is expanded into
>
> (let
>  ([x 
> (delay
>  1)])   
> (let-syntax
>  ([clause 
> ...
> ]) 
> (+
>  x x)))
>
> where clause ... introduce a syntax binding such that x in the body
> expands to a use of x.
>
> This is just an example though. The key concept is the binding clause
> transformer.
> Here is the documentation for bind and def (let and define):
>
> http://htmlpreview.github.io/?https://github.com/soegaard/bind/blob/master/scribblings/bind.html
>
> Here is the Github repository:  https://github.com/soegaard/bind
>
> Another small example:
>
> Example:   > (bind ([v 
> :vector (for/vector ([i 5]) (random 10))])
> (displayln (~a "The vector " v " contains " v))
> (displayln (~a "The first element is: " (v 0)))
> (v! 0 42)
> (displayln (~a "The first element is now: " (v 0)))
> (displayln (~a "The middle three elements are: " (v 1 4
>
>
> This expands into uses of vector-ref and vector-set! so there is no
> runtime penalty
> for using this shorthand.
>
> /Jens Axel
>
>
> 2013/8/26 Stephen Chang 
>
>> Hi dev,
>>
>> I've noticed that Racket has a lot of convenient binding forms but
>> they don't fit together unless someone does it manually (for example
>> there's match-let and match-let-values, but no match-for).
>>
>> As an educational side project, I've been toying around with a
>> different way of organizing all the binding forms. What I wanted to do
>> is remove the need to manually combine current (and future) binding
>> forms by moving the binding "logic" to the binding site itself.
>>
>> Inspired by the in-vogue generics movement in the Racket world, I've
>> hacked together a sort of "generic interface" for bindings (in
>> reality, it's just a bunch of syntax properties right now), and
>> implemented alternate versions of some of the main binding forms that
>> suppor

Re: [racket-dev] generic binding forms

[Stephen Chang]

Thanks for all the feedback everyone.

There seems to be some interest so I've uploaded my generic binding
forms to planet2: https://pkg.racket-lang.org/info/generic-bind

I realize it's the pre-semester crunch, but if anyone wants to try it
out, I would be grateful for any feedback.

Here are some docs: http://stchang.github.io/generic-bind/generic-bind.html

I've implemented new versions of the following forms to support my
"generic bindings": define, lambda, let, let*, letrec, and all the
for/ and for*/ forms (my versions are prefixed with "~").

These forms *should be* drop-in replacements for their Racket
counterparts. For example, my ~for/?? and ~for*/?? forms pass all the
(non-buggy) for/ tests in the racket test suite.

This library is intended to be a proof-of-concept, so I have not
measured things like performance at this time. I'm pretty sure my for/
forms are much slower than the existing implementation and using my
binding forms will likely contribute a noticeable increase in compile
time.

Regarding Ian's comments, I decided that there should not be any
interaction between let binding clauses, other than the standard
behavior of let, let*, and letrec. This means that if there are
duplicate identifiers in different match patterns in different
clauses, it will behave like duplicate identifiers in let, let*, or
letrec, and *not* like match-let. The same is true in my for/ forms.

Finally, I think my macros are pretty amateur hour and I often feel
that there must be better ways to do certain things that I just am not
aware of, so if any gurus are interested in lending some time for a
code walk, I would be very appreciative.



On Mon, Aug 26, 2013 at 6:58 PM, Matthias Felleisen
 wrote:
>
> What are the performance implications of this design? Specifically how do 
> overlapping uses fare in the two systems? Microbenchmarks okay for now. -- 
> Matthias
>
>
>
> On Aug 26, 2013, at 12:54 AM, Stephen Chang wrote:
>
>> Hi dev,
>>
>> I've noticed that Racket has a lot of convenient binding forms but
>> they don't fit together unless someone does it manually (for example
>> there's match-let and match-let-values, but no match-for).
>>
>> As an educational side project, I've been toying around with a
>> different way of organizing all the binding forms. What I wanted to do
>> is remove the need to manually combine current (and future) binding
>> forms by moving the binding "logic" to the binding site itself.
>>
>> Inspired by the in-vogue generics movement in the Racket world, I've
>> hacked together a sort of "generic interface" for bindings (in
>> reality, it's just a bunch of syntax properties right now), and
>> implemented alternate versions of some of the main binding forms that
>> support "instances" of these generic bindings.
>>
>> To illustrate, here are some test cases for a generic define I
>> implemented (~define). I also implemented binding "instances" for
>> match and values (which I arbitrarily named $ and ~v below) and I can
>> use these forms in (mostly) any binding position that supports generic
>> bindings.
>>
>> ;; functions
>>> (~define (f1 x y) (+ x y))
>>> (f1 10 20)
>> 30
>>> (~define (f2 ($ (list x y))) (+ x y))
>>> (f2 (list 10 20))
>> 30
>>
>> ;; non-functions
>>> (~define a1 100)
>>> a1
>> 100
>>> (~define (~v a2 a3) (values 134 456))
>>> a2
>> 134
>>> a3
>> 456
>>
>> You can nest bind instances too:
>>
>>> (~define (~v ($ (list b1 b2)) b3) (values (list 22 33) 44))
>>> b1
>> 22
>>> b2
>> 33
>>> b3
>> 44
>>
>> Does anyone think this is useful? Or is it just a lot of work to save
>> a little bit of typing? Has anyone tried something like this before?
>>
>> It's still very much a work in progress but I figure I would ask for
>> some feedback earlier rather than too later, in case there is
>> something that makes this infeasible.
>>
>> Brave souls can look at the hackery here:
>> https://github.com/stchang/generic-bind/blob/master/generic-bind.rkt
>> (Warning: I'm still trying to figure out all the toys in the Racket
>> macro toolbox. For the most part, everything still looks like a
>> syntax-rule/case/parse/->datum nail to my hammer.)
>>
>> Technical question: I couldn't figure out a nice way to implement
>> ~let. Essentially, I want a let form where some clauses are let-values
>> and some are match-let, but I need to bind them all at the same time,
>> like let. I can't define a ~lambda that works with values because
>> functions in racket can't receive values. Anyone have any ideas?
>>
>> Side observation: Trying to get things to work with multiple return
>> values was a pain because they don't compose (as in, functions can
>> produce them but can't receive them). Not sure if anything can be done
>> about this though.
>> _
>>  Racket Developers list:
>>  http://lists.racket-lang.org/dev
>
_
  Racket Developers list:
  http://lists.racket-lang.org/dev

[racket-dev] John Carmack talks about functional programming

[Jay McCarthy]

He mentions Racket and the DrRacket OS paper (by name!)

http://www.youtube.com/watch?v=1PhArSujR_A [about 9 minutes in is the
Racket mention]

Jay

-- 
Jay McCarthy 
Assistant Professor / Brigham Young University
http://faculty.cs.byu.edu/~jay

"The glory of God is Intelligence" - D&C 93
_
  Racket Developers list:
  http://lists.racket-lang.org/dev

Re: [racket-dev] net/http-client

[Greg Hendershott]

This looks great!!

A couple suggestions:

1. Support for "Expect: 100-continue" request headers would be
helpful, and I think not too messy to add.

The big use case I'm aware of is Amazon S3. If you make a PUT or POST
request, it might need to redirect you to another URI (outage,
balancing, whatever reason). Expecting and handling 100-continue lets
you avoid transmitting potentially large amount of data that would be
discarded, and you have to send it all over again in the request to
the redirect URI. For (say) a 1 GB upload to S3, this matters.
Although I don't know for sure if other upload-ish web APIs offer
same, I'd guess some do as this is the use case for 100-continue
generally.

How I implemented this in my HTTP package was to have a
`start-request` function that sends the request line and headers,
peeks the response status line, then returns a `boolean?` whether the
PUT/POST/PATCH/whatever data should be transmitted. [1]

I think your `http-conn-send!` could do similar?

2. Support for "Content-Encoding" response headers would also be helpful.

Using the same make-pipe approach as you're doing with chunked
transfer encoding. [2]  Maybe this is mission creep: For HTTP 1.1. you
_must_ support Transfer-Encoding: chunked, whereas Content-Encoding is
just optional. However it's a good option; using compression can
really help out on time as well as bandwidth charges.


IIRC those were the two main things that motivated me to make my HTTP
package at all, to support e.g. my AWS package. If http-client added
them, I might not need my package anymore. (OK, it might take me
awhile to phase it out until I'm ready to de-support older versions of
Racket, but, I and others wouldn't need it for new projects.)


[1]: 
https://github.com/greghendershott/http/blob/master/http/request.rkt#L142-L189

[2]: By the way, do you want to pass some `limit` optional arg in the
various uses of `make-pipe`? Otherwise IIUC this will suck everything
into RAM, which might not be so great with very large request or
response entities.


On Fri, Aug 23, 2013 at 2:48 PM, Jay McCarthy  wrote:
> Based on a request back in early July to remove the restrictions that
> net/url puts on HTTP communication (vis a vis URL encoding), I have
> just pushed a new HTTP client as net/http-client.
>
> The push also changes net/url to use net/http-client so that we just
> have 1 HTTP request producer, rather than 3.
>
> It passes all of the net/* tests, but these don't use features like
> proxying, HTTP/1.1, etc. I'm slightly nervous that it doesn't do those
> correct, but not super nervous, because I just cut-and-pasted the
> code.
>
> The main approach of the library is best explained by this contract:
>
> [http-sendrecv
>(->* ((or/c bytes? string?) (or/c bytes? string?))
> (#:ssl? (or/c boolean? ssl-client-context? symbol?)
> #:port (between/c 1 65535)
> #:method (or/c bytes? string? symbol?)
> #:headers (listof (or/c bytes? string?))
> #:data (or/c false/c bytes? string?))
> (values bytes? (listof bytes?) input-port?))]
>
> Compared to net/url,
> - It supports bytes and strings everywhere
> - It supports data on every method and not just POST
> - It always returns the status line, headers, and content (as a port)
>
> I feel that the only thing it could do better is support two more
> options for #:data:
> - A input-port? to read from and copy to the HTTP connection
> - A (-> output-port? void) function to call with the HTTP connection's
> output port to stream the data
>
> But I'd like a second opinion before adding them.
>
> Jay
>
> --
> Jay McCarthy 
> Assistant Professor / Brigham Young University
> http://faculty.cs.byu.edu/~jay
>
> "The glory of God is Intelligence" - D&C 93
> _
>   Racket Developers list:
>   http://lists.racket-lang.org/dev
_
  Racket Developers list:
  http://lists.racket-lang.org/dev

Re: [racket-dev] Output value of program changes depending on whether I pretty-print an opaque struct value

[J. Ian Johnson]

I determined this morning that the cause was a bad hash code for my generic 
hashes/sets. I use an injection from the value space into the natural numbers 
by assigning atoms prime numbers (thanks math/number-theory for next-prime!), 
and I multiply them together to produce the encoding of sets (I check 
divisibility before multiplying, so I always have a <=1 multiplicity). I use 
Cantor's injection for producing the number of pairs, so I can make a set of 
pairs encode finite functions (hashes). The result should be that equal numbers 
imply equal values, so I hashed on the numbers for hash-proc and hash2-proc 
instead of the values. I still compare equality only using the numbers since 
it's 3 orders of magnitude faster. I changed the hash-proc to hash on the 
values instead and got consistent results for the analysis. I'm perplexed by 
this, so I'm trying to write a smaller example that demonstrates the problem.

I only use hasheq for seen checks when traversing through a hash that has 
circular references, and I never change the hash during these traversals, and 
the result is a set. The order doesn't matter. I also changed all hasheqs in my 
code to hashes to see if that was a problem. It wasn't.

-Ian
- Original Message -
From: "Carl Eastlund" 
To: "J. Ian Johnson" 
Cc: "Matthew Flatt" , "dev" 
Sent: Wednesday, August 28, 2013 12:34:08 PM GMT -05:00 US/Canada Eastern
Subject: Re: [racket-dev] Output value of program changes depending on whether 
I pretty-print an opaque struct value


Is it possible your analysis is depending on the order of graph traversal? That 
is, do you ever use in-hash on the hasheq, and if so is it possible the results 
of your analysis would change if in-hash produced hash table entries in a 
different order? Same for in-hash-keys or in-hash-values, obviously. 



Carl Eastlund 


On Tue, Aug 27, 2013 at 6:58 PM, J. Ian Johnson < i...@ccs.neu.edu > wrote: 


Weird you can't repro. 
I only use hasheq when I know the keys are symbols, or the table is local and 
only used for a graph traversal (where the graph is not changing during 
traversal). 
-Ian 


- Original Message - 
From: "Matthew Flatt" < mfl...@cs.utah.edu > 
To: "J. Ian Johnson" < i...@ccs.neu.edu > 
Cc: "dev" < dev@racket-lang.org > 
Sent: Tuesday, August 27, 2013 6:33:02 PM GMT -05:00 US/Canada Eastern 
Subject: Re: [racket-dev] Output value of program changes depending on whether 
I pretty-print an opaque struct value 

I haven't been able to get a different result by changing printing. 

One thing that printing might do, however, is assign `eq?`-based hash 
codes to objects that did not already have them. That assignment, in 
turn, could affect the order in which objects appear later in a hash 
table. 

I hacked Racket to make the internal hash-code generator count down 
instead of up, and I was able to get a different result that way. 

Are you using `eq?`-based hashing at all? Could there be some 
dependency on the order of values within a hash table (or hash set)? 

At Tue, 27 Aug 2013 14:02:50 -0400 (EDT), "J. Ian Johnson" wrote: 
> This is mostly an mflatt-only problem. 
> 
> My analysis framework is now using a generic dictionary for its abstract 
> heap, 
> and depending on whether or not I pretty-print this heap before continuing on 
> analyzing, the result of the analysis changes (sound versus unsound). I found 
> the problem doing some printf debugging, and when fixed, I tried removing the 
> print statements. The answer changed. There must be some problem in the C 
> code 
> somewhere, because this is just bonkers. 
> 
> (Note: I've had previous bugs where rewriting a (begin (void) e) => e caused 
> a 
> similar value-changing bug. Updating the compiler to head fixed it at that 
> point. I'm running yesterday's HEAD now, though.) 
> 
> https://github.com/dvanhorn/oaam/tree/thocon 
> 
> See the ;; mflatt: comment in code/kcfa.rkt 
> 
> To see the problem, 
> racket code/run-benchmark.rkt --lcg benchmarks/temp-c/sort2.sch 
> 
> Thanks, 
> -Ian 
> _ 
> Racket Developers list: 
> http://lists.racket-lang.org/dev 
_ 
Racket Developers list: 
http://lists.racket-lang.org/dev 

_
  Racket Developers list:
  http://lists.racket-lang.org/dev

Re: [racket-dev] Output value of program changes depending on whether I pretty-print an opaque struct value

[Carl Eastlund]

Is it possible your analysis is depending on the order of graph traversal?
That is, do you ever use in-hash on the hasheq, and if so is it possible
the results of your analysis would change if in-hash produced hash table
entries in a different order?  Same for in-hash-keys or in-hash-values,
obviously.

Carl Eastlund


On Tue, Aug 27, 2013 at 6:58 PM, J. Ian Johnson  wrote:

> Weird you can't repro.
> I only use hasheq when I know the keys are symbols, or the table is local
> and only used for a graph traversal (where the graph is not changing during
> traversal).
> -Ian
> - Original Message -
> From: "Matthew Flatt" 
> To: "J. Ian Johnson" 
> Cc: "dev" 
> Sent: Tuesday, August 27, 2013 6:33:02 PM GMT -05:00 US/Canada Eastern
> Subject: Re: [racket-dev] Output value of program changes depending on
> whether I pretty-print an opaque struct value
>
> I haven't been able to get a different result by changing printing.
>
> One thing that printing might do, however, is assign `eq?`-based hash
> codes to objects that did not already have them. That assignment, in
> turn, could affect the order in which objects appear later in a hash
> table.
>
> I hacked Racket to make the internal hash-code generator count down
> instead of up, and I was able to get a different result that way.
>
> Are you using `eq?`-based hashing at all? Could there be some
> dependency on the order of values within a hash table (or hash set)?
>
> At Tue, 27 Aug 2013 14:02:50 -0400 (EDT), "J. Ian Johnson" wrote:
> > This is mostly an mflatt-only problem.
> >
> > My analysis framework is now using a generic dictionary for its abstract
> heap,
> > and depending on whether or not I pretty-print this heap before
> continuing on
> > analyzing, the result of the analysis changes (sound versus unsound). I
> found
> > the problem doing some printf debugging, and when fixed, I tried
> removing the
> > print statements. The answer changed. There must be some problem in the
> C code
> > somewhere, because this is just bonkers.
> >
> > (Note: I've had previous bugs where rewriting a (begin (void) e) => e
> caused a
> > similar value-changing bug. Updating the compiler to head fixed it at
> that
> > point. I'm running yesterday's HEAD now, though.)
> >
> > https://github.com/dvanhorn/oaam/tree/thocon
> >
> > See the ;; mflatt: comment in code/kcfa.rkt
> >
> > To see the problem,
> > racket code/run-benchmark.rkt --lcg benchmarks/temp-c/sort2.sch
> >
> > Thanks,
> > -Ian
> > _
> >   Racket Developers list:
> >   http://lists.racket-lang.org/dev
> _
>   Racket Developers list:
>   http://lists.racket-lang.org/dev
>
_
  Racket Developers list:
  http://lists.racket-lang.org/dev