[racket-users] [ANN] A Scribble library for typesetting grammar differences

[William J. Bowman]

I just released a library for typesetting `racketgrammar`s with annotations to
highlight what has been added or removed compared to another grammar.
You can install it via `raco pkg install scribble-bettergrammar`

Documentation is available here:
  
https://docs.racket-lang.org/bettergrammar/index.html#%28form._%28%28lib._scribble%2Fbettergrammar..rkt%29._typeset-grammar-diff%29%29

Source here:
  https://github.com/wilbowma/scribble-bettergrammar

Hope you find it useful!

-- 
William J. Bowman

-- 
You received this message because you are subscribed to the Google Groups 
"Racket Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to racket-users+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/racket-users/20200718053119.GS11314%40williamjbowman.com.

Re: [racket-users] reading s-expressions from a file

[gfb]

Shouldn't the for be (for ([expr (in-producer read eof)] ⋯). One could also 
specify the input port there, as shown in reference for in-producer 

 
, possibly inside call-with-input-file (although that vs 
with-input-from-file is probably a matter of taste).

-- 
You received this message because you are subscribed to the Google Groups 
"Racket Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to racket-users+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/racket-users/d1583583-f98b-47fe-a6b8-5dd295bceef8n%40googlegroups.com.

Re: [racket-users] reading s-expressions from a file

[George Neuner]

On 7/17/2020 8:19 PM, Hendrik Boom wrote:

Yes,  I know the functino for reading s-expressions seems to be (read [in]).

I want a loop that reads S-expressions and does something to each one, until 
there are no more to be found in a file.

Now of course that's absurdly easy to do with a tail-recursice loop.

But I's like it to look like a loop, with (for ...) or (while ...) or
(loop ...) or something like that.

But I fail to fine any iterators that process a file, such as (in-file
...)

There's a long list of iterators in
https://docs.racket-lang.org/reference/for.html
and in
https://docs.racket-lang.org/guide/for.html

An I just looking in the wrong place, or are there really no iterators
for reading a stream of s-expressions from a file.

-- hendrik


This should work if you're reading the sexprs for value:

(with-input-from-file /filename/
  (lambda ()
    (for [(expr (read))]
  :
    )))

If your intent is to read/parse the sexprs as text, there isn't a simple 
way to do that.  Regex is problematic for nested expressions ... you 
could try "read-syntax" if you are familiar with macros, or "match" if 
you know what patterns to look for.


There's no "in-match" sequence, so if you want a loop that *appears* 
simple [match in the /"for-clause"/], you'll have to write some glue: a 
helper that returns one match result at a time so as to plug nicely into 
a loop.


George

--
You received this message because you are subscribed to the Google Groups "Racket 
Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to racket-users+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/racket-users/7184da64-a388-91bf-e027-a213a3d66c73%40comcast.net.

Re: [racket-users] reading s-expressions from a file

[Sorawee Porncharoenwase]

(with-input-from-file "abc" (thunk (sequence->list (in-port

Note that you need to “read” in the dynamic extent of with-input-from-file.
Outside it, the port is closed.

On Fri, Jul 17, 2020 at 5:19 PM Hendrik Boom  wrote:

> Yes,  I know the functino for reading s-expressions seems to be (read
> [in]).
>
> I want a loop that reads S-expressions and does something to each one,
> until there are no more to be found in a file.
>
> Now of course that's absurdly easy to do with a tail-recursice loop.
>
> But I's like it to look like a loop, with (for ...) or (while ...) or
> (loop ...) or something like that.
>
> But I fail to fine any iterators that process a file, such as (in-file
> ...)
>
> There's a long list of iterators in
>https://docs.racket-lang.org/reference/for.html
> and in
>https://docs.racket-lang.org/guide/for.html
>
> An I just looking in the wrong place, or are there really no iterators
> for reading a stream of s-expressions from a file.
>
> -- hendrik
>
> --
> You received this message because you are subscribed to the Google Groups
> "Racket Users" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to racket-users+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/racket-users/20200718001905.elzyuu42gje45j7t%40topoi.pooq.com
> .
>

-- 
You received this message because you are subscribed to the Google Groups 
"Racket Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to racket-users+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/racket-users/CADcuegvbps%3DtFJYg2UZ48s7uw1tVY8P58WnQYoZUMyShahCzhg%40mail.gmail.com.

[racket-users] reading s-expressions from a file

[Hendrik Boom]

Yes,  I know the functino for reading s-expressions seems to be (read [in]).

I want a loop that reads S-expressions and does something to each one, until 
there are no more to be found in a file.

Now of course that's absurdly easy to do with a tail-recursice loop.

But I's like it to look like a loop, with (for ...) or (while ...) or 
(loop ...) or something like that.

But I fail to fine any iterators that process a file, such as (in-file 
...)

There's a long list of iterators in 
   https://docs.racket-lang.org/reference/for.html
and in 
   https://docs.racket-lang.org/guide/for.html

An I just looking in the wrong place, or are there really no iterators 
for reading a stream of s-expressions from a file.

-- hendrik

-- 
You received this message because you are subscribed to the Google Groups 
"Racket Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to racket-users+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/racket-users/20200718001905.elzyuu42gje45j7t%40topoi.pooq.com.

Re: [racket-users] Best data structure for ordered data set with insertion and reordering?

[David Storrs]

Thanks George.  Much appreciated.

On Thu, Jul 16, 2020 at 11:21 PM George Neuner  wrote:

>
> Hi David,
>
> On 7/16/2020 11:44 AM, David Storrs wrote:
>
> On Thu, Jul 16, 2020 at 10:09 AM George Neuner 
> wrote:
>
>>
>> The problem seems under-specified.  Can you say more about the real
>> purpose?
>>
>
> Basic version:  It's a peer-to-peer encrypted swarmed file sharing system
> that presents like Dropbox on the front end (i.e. "make a change to the
> filesystem on peer A and peers B-Z will replicate that change") and works
> something like Bittorrent on the back end in that files are sent in chunks
> but it offers functionality that Bittorrent does not, such as encrypted
> transfer, WoT authentication, etc.
>
>
> Interesting.  So I'm guessing your problem is to (compactly) represent the
> state of the shared space.
>
> Do you plan on having index servers, or are you aiming for a fully
> distributed solution?  And, if distributed, do you want each node to
> maintain its own state picture of the shared space, or were you thinking
> that nodes could just snoop admin broadcasts looking for mention of data
> they don't currently have?  [Your question about how to pair / collapse
> messages suggests you might be considering a snoopy solution.]
>
> Asking because keeping a state picture has scalability issues, a snoopy
> solution has complexity issues, and (depending on latency) both have issues
> with performing unnecessary work.  In any event, I have some suggestions.
>
>
>
> Snoopy is the more interesting case.  You start with a queue of file
> operations to be done as gleaned from the admin messages - mkdir, rmdir,
> fetch a file, delete a file, etc. - in whatever order the messages were
> received.
>
> Separately, you maintain a (hash table) mapping from pathnames to a list
> of queue nodes that operate on that object.  The map should use weak
> references so that nodes can safely be removed from the queue and discarded
> without also needing to update the map.  If queue processing gets to some
> operation first, any map reference to it will dissolve (be replaced by
> #f).
>
> When a message is received, you lookup the pathname in the map, and if a
> complementary operation is found in the queue, you remove and discard it.
> [You can also remove references in the map or just let them dissolve
> depending on your handling.]   Then simply discard the message.
>
> Otherwise you queue whatever operation the message indicates and add a
> reference to the queue node under the object's pathname in the map.
>
> Extra complexity comes in having to notice that map entries (pathnames)
> have no operations left in the queue.  Weak references don't just disappear
> - they are changed to #f when the referenced object is no longer reachable
> - however AFAICT there is no hashtable variant that permits weak reference
> values, so you have to use weak-boxes and those continue to exist even if
> the objects they reference are gone.   Useless map entries will need to be
> identified and removed somehow.
>
>
> Modeling the filesystem can be done rather simply with a trie in which
> folders are represented by mutable hash tables and files by structures.
> You can combine this with the operation queue above, but in this case
> lookups can be done in the trie and queue references kept in the trie
> nodes.  And the trie provides a snapshot of the current state which may be
> useful for other purposes.
>
>
> The trick in either case is processing latency: you don't want to wait too
> long, but if you really want to avoid unnecessary work you need to delay
> performing file operations long enough that complementary messages are
> likely to be received.
>
>
>
> What if messages are lost permanently, e.g., due to hardware crash?
>>
>
>> What it you receive a create but a corresponding delete or update is
>> lost - then your information / picture of the file system state is wrong.
>>
>> What if you receive a file delete without a corresponding create? In the
>> absence of other information, can you even assume there *was* a create?
>> If these messages are sent in response to user actions, can they ever be
>> sent mistakenly?
>>
>>
> The ultimate answer to these questions is "If things get out of sync in a
> way that the system cannot resolve, it will be flagged for a human to
> resolve." There are things we do that mitigate them -- for example, a
> write-ahead log for messages received from peers -- but we acknowledge that
> we cannot resolve 100% of situations automatically.  Neither can any other
> file replication service.  (Dropbox, Box.com, etc)
>
> Also relevantly, differences are reconciled across multiple peers.  If
> there's 5 peers in your replication set and the other 4 agree that there
> should be a file at path P but you don't have one then it's safe to assume
> that you missed a File-Create message.  And yes, that comes with issues of
> its own (Q: What if it was deleted on your machine and none of the others
> got you

Re: [racket-users] file->bytes with large files

[Greg Rosenblatt]

Thanks.

Depending on how the increment compares to the file size, file->bytes might 
be up to 1.5x faster (on my machine at least) than the workaround.  But the 
workaround is still fast enough.

(define (file->bytes2 file-name)
  (define size (file-size file-name))
  (define bs (make-bytes size))
  (call-with-input-file
file-name
(lambda (in)
  (let loop ((i 0))
(cond ((= i size) bs)
  (else (define end (+ i (min 1073741824 (- size i  ;; 1GB 
increment
(read-bytes! bs in i end)
(loop end)))

On Wednesday, July 15, 2020 at 6:32:36 PM UTC-4, Matthew Flatt wrote:
>
> The `file->bytes` function uses the file size with `read-bytes`, and it 
> appears that the Mac OS `read` system call errors on requests of 2GB or 
> more. The right fix is for the `read` call within Racket (at the rktio 
> layer) to limit the size that it passes, and I'll make that change. 
>
> Meanwhile, you could work around the problem by limiting the size of an 
> individual request: Allocate a byte string and then use a sequence of 
> `read-bytes!` calls to read the file in increments. Each time, use the 
> number of read bytes to increment a starting position into the 
> destination byte string (which is the third argument to `read-bytes!`). 
>
> Matthew 
>
> At Wed, 15 Jul 2020 15:05:22 -0700 (PDT), Greg Rosenblatt wrote: 
> > Hi, I'm getting an error while using file->bytes to load a moderately 
> large 
> > file: 
> > 
> > > (time (void (file->bytes "my-7.6GB-file"))) 
> > ; error reading from stream port 
> > ;   port: # 
> > ;   system error: Invalid argument; errno=22 
> > ;   context...: 
> > ;/Applications/Racket v7.7/collects/racket/file.rkt:768:6: temp218 
> > ;/Applications/Racket 
> > v7.7/collects/racket/private/more-scheme.rkt:336:52 
> > ;eval-one-top 
> > ;/Applications/Racket 
> v7.7/share/pkgs/xrepl-lib/xrepl/xrepl.rkt:1478:0 
> > ;/Applications/Racket v7.7/collects/racket/repl.rkt:11:26 
> > 
> > Is there a limit to the size of files that can be used with file->bytes? 
> > 
> > I was preferring file->bytes because it seems much faster than manually 
> > reading from a port.  If file->bytes is not appropriate here, can 
> somebody 
> > recommend another fast approach? 
> > 
> > -- 
> > You received this message because you are subscribed to the Google 
> Groups 
> > "Racket Users" group. 
> > To unsubscribe from this group and stop receiving emails from it, send 
> an 
> > email to racket...@googlegroups.com . 
> > To view this discussion on the web visit 
> > 
> https://groups.google.com/d/msgid/racket-users/e99edda0-06ed-4164-b7bd-46f8a458
>  
> > c6c8o%40googlegroups.com. 
>

-- 
You received this message because you are subscribed to the Google Groups 
"Racket Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to racket-users+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/racket-users/b6d1ff87-c775-4117-8f88-7816f6434212o%40googlegroups.com.